If you can record some of your exchanges with him and listen to them after the fact it helps.
Like when he mentions money waste hurts everyone and you counter with affairs hurt everyone...
You could say
WE have brought this marriage where it is now. WE are doing what WE can do rebuild it and strengthen it. BOTH of us made the mess and both of us can clean it up. YOU have a chance to participate in this family to strengthen it or you know where the door is. You want to walk out the door and leave this family to clean up your share of this mess I am happy to help you pack - this household does NOT need any DEAD WEIGHT. Either you are part of the solution or you and your problems can find the door.