Tell him you can't afford to move, won't disrupt the children's lives, but can't stand your M so you are taking a lover.

Best,
Oldtimer