Jo, I'm sorry that you are hurting. You have every right to be concerned about X's treatment of you, as well as keeping yourself away from the OW.
You're communicating boundaries to him, which is good. I'm sure you're venting a bit more on the BB than you might to him, but I'd caution against venting anger directly at him. He just seems incapable of responding well to that. He's too weak to hold/accept your emotion right now.
I liked your suggestion of a compromise, and I wouldn't be surprised if he comes thru w/ an offer in the near future. Just as he 'learned' from DD1's Bday (not inviting you) to this one (inviting you), he may learn to consider your request for a separate celebration. I think this will be more likely the safer he feels with you, which might be driven by how friendly you 2 are. It may not feel like it right now, but you are the stronger of the two, and his control merely highlights his weakness.
Hang in there, Jo. Keep thinkign about those positives in your life.