I also see you trying to anticipate his thoughts and actions.
Listen, he is a big boy. If he felt/feels you monopolize time with the kids, then he needs to commit to a visitation schedule. He needs to step up as a parent and say "this is what I want."
If he can't bring himself to do either of those things but would rather sit back and blame you, that needs to be his problem.
Be clear. Be consice. Accommodate when you feel it is the right thing to do, but not just because you worry that he will become resentful. The day is going to come when he has to learn that he is responsible for his own feelings.
"Acceptance doesn't mean resignation. It means understanding that something is what it is and there's got to be a way through it."--Michael J. Fox
