Hopefully by him getting these things off his chest it will lighten his mood. That, and the time apart might be some great relief for your sitch. When you guys get together without the kids I'd be willing to bet things will be a lot better.
I'm not sure what to tell you as far as this tendency he has to hold resentments over things you can't control. Is there a way to get him to air out his grievances more often and get them off his chest..but still not be seen as nagging? Maybe when things are on more smooth ground between you two it's something you can discuss. If it's brought out to him in a friendly environment when things are going good maybe it will make him realize how unfair it is for him to hold these things against you.
Anyways, I'm glad to see you're continuing the patience even through the most trying of times. This is definitely a long journey, even when we've made it to the piecing stage. Hang tough.
A dream it's true
But I'd see it through
If I could be
Wasting my time with you
-Band:Phish Song:Waste