The anger and insults (spewage) come from guilt. And, ironically, fear that he is losing you. (Your H has always been one of the most conflicted; honestly, almost any guy who really wanted to be with OW would be living with her by now. He stays put because he doesn't want to let go of the safety net of your R.)
Have you tried this? let H know he needs to be home X night with D because you will be going out. Then don;t come home - all night - or come home at 3 am. Let him wonder where YOU are for a change.