LNL -
or you could tell him:
"H, I really don't think that any man in my life would appreciate my ex-husband spending the night in my home, just as I'm sure your girlfriend(s) wouldn't appreciate you staying here either. It just seems really inappropriate at this point."

Ellie