I agree with Ahoy. And push yourself to take emotion out of it as much as possible. You can only control you. So, offer a proposal, maybe for a month in advance (even if it's general, like "H will have D one evening a week, blah blah blah, to be determined when H's schedule comes out on Sundays" (or whatever), ask him his thoughts, and say, ok let's try it for 3 months.
Then, stick to it as much as possible. And make plans for the times he is scheduled to be with D. And problem solve a back up plan together (besides just you as the default).
I'm being very prescriptive, but trying to get you to think outside the box in a problem solving way. You can't just wish him to be different, but you can be the best you given the situation.
