I do this with H if I've got a family outing to go to. I ask him and if he says no or I don't know then I just go with my son anyway as if it's no big deal. I also never remind him about it. He gets one chance to go and that's it. I know he's got a bad memory but if he really wanted to go then he'd remember.
Well done, that's exactly how MWD describes it in DR. Make plans to go whether they go or not. Invite them along, but make it clear you're going regardless. Something like "S and I are going to the zoo Saturday, you're welcome to join us if you wish." And if they refuse, then you go anyway. This may not apply to a birthday like Hurt was asking about (you can't really say 'W, I'm going to your bday party, you're welcome to come along if you want', LOL!), but it's relevant to most other stuff.