Clearly you need more physical support. I hope that you are not also shouldering the financial side of your children alone. I hope you are holding him financially accountable for his children. Hire a weekend nanny -- perhaps one who can work 10-12 hours a day and let you rest, have individual time with each kid, etc. Make sure he is sharing in the costs of having these kids.
You mentioned 50/50 custody above -- there is no status quo of him being a caregiver for your children. If you don't want 50/50 custody, and frankly he very well might not himself, you don't have to allow him to have it. He has done nothing that shows he is a particularly committed parent, remember until recently he considered extending his contract in the other state. Look at how he talks, the kids clearly aren't his priority. He is his priority.
