Right now he hasn't bought a house or filed for D.
I hear what you're saying about him cake-eating. But it sounds like you not inviting him is you trying to "show him" something. "Show him" = hope to control his feelings/decisions. It won't work.
Don't let your mind mislead you here. I think I know exactly what you're thinking: well, if he thinks things are going to be this enjoyable with us not being a couple, then he will most certainly want the D because things seem better now than they were when we were a married couple.
That couldn't be further from the truth. Why? Mainly because you two have history and children together. Each time you're having fun and not stressing and not nagging and not trying to control him, he sees you in a positive light. And that's a helluva lot more attractive than you going into your room with the baby and shutting the door or you not inviting him places.
And here's the $21 million question: What do YOU want to do? Does it help you more than it hurts to have him around? Or does it hurt you more than help?
Do whatever helps YOU. Forget what's going to help or hurt or impact H even one small iota!
Girl, I know this is hard to see and understand when you're in the throes of everything, but you're getting opportunities handed to you: opportunities to be light and breezy (and even fun and flirty) ... opportunities to be the OW to this "OW" ... opportunities to let down your hair AND drop that rope you're needlessly holding.
I see you are attempting to play reverse psychology with yourself: when H acts nicer, instead of you looking at that as a baby step and realizing you've maybe found what's working, you say: "Well, he's being nicer; that must mean he's really done."
Reverse psychology doesn't work - even when we use it on ourselves. I'd maybe stop being so pessimistic and start paying attention to the cues that H is giving you. Frankly, if you were my spouse and I could feel you distancing yourself, or putting a little wall up, every time I took a step toward you to make things feel less tense or awkward, I'd stop taking steps toward you.
M: 40 H: 44 Married 14 years S11 & D6; D20 & D19 from previous M 2BDs/PAs, 8 years apart Piecing: April 2014