I think I've figured it out. I can't control what he does. If it doesn't bother him to lose his limited time with our D, that's not my issue. I don't have to take his tactic and just make plans whenever I want and lose my time with my D, but I *can* work on GAL so that I have plans during my own personal time, and not always be available as the default caregiver whenever he needs me. If I have plans, I can confidently say, "Sorry I'm not available" without feeling like I'm just being spiteful. That would be much more attractive to him than always being available.
And, at some point in the future, I can always address the equity issue if I want to change our schedule...