I am really sorry you are going through this right now.
I am going to give you my opinion. He's manipulating you. He wants to have affairs with younger woman and still have the comforts of the family and home. It is good you stopped having sex with him. Sounds like your'e ready to cave which will put you back at ground zero.
Lets look at actions and consequences. He has affairs and the consequences are he sleeps in other room and you have not had sex with him in two weeks. I have married friends who go way longer then that.
This is very troublesome to read because it sounds to me that it is like a game to him.
