You tell him its for your convenience and his that he has his belongings with him... And that its hurtful to you to have them hanging around the house.
You know him though, so you can always modify this according to his demeanor.
Preferably the intermediary is a strong willed person who is marriage friendly, fully informed on your WH's affair and who won't take any of his crap.
Stick to email only.
And they have to be ready to threaten calling the police in a snap.
