Well, first, what do YOU want?
Are you dying to spend some time with his relatives alone, where you can talk and share about H? Or is that a conversation you wouldn't really be having in dd's presence anyway? Would seeing these relatives be a good influence on H, or are they so mad at him that they would be uncomfortable and have trouble being civil to him?

If you think things would be comfortable between H and his relatives, then why not give him the option to come? Maybe having other adults there could help "break the ice" between you two. And maybe H will decide sleeping on the floor is so uncomfortable that he wants to climb in bed with you instead

Seems to me there's a bit of a disconnect (gee, big surprise) between what H SAYS and what he DOES. Although he's SAYING he's your EXH, this is the third weekend now in a short time that he's asked to come and stay. I can't help but think he's testing the waters. You admit yourself you weren't at your best last time. How about letting him come and see you through other people's eyes, as the charming and entertaining hostess?

Ellie