Your ex is manipulating isn't he? I'd make sure to have a talk (if you haven't already) to him about what exactly your schedule is and stick to it. Talk to the kids, too. So, EVERYONE is on the same page and he can't do what he is doing. Then, years down the road, when he is more sane, you can be more flexible.
