I don't understand cosmic, yoda-like selfless love and acceptance as an answer to your H's behavior. You don't have to accept him. For what purpose? You can work out a visitation schedule without accepting him, without trusting someone who has proven that he cannot be trusted.
He has abandoned your children, and that's IT for me. You don't need to worry about making yourself a doormat... By going along with HIS plan that doesn't consider you or your children, you are spreading your kids out for him to walk on as well.
FOR WHAT?
Get tough!
Even if you are hung up on him coming back (oy vay,) he won't come back this way. He'll need to see you moving forward to a better life, taking care of yourself, and protecting the boys from further damage. I would bet that you're boring to him right now.