Really? I mean, I understand but the luggage opportunity disappears.

Perhaps just confront the affair without the "you have to leave" bit.