Blues - I am so sorry to read the latest events. First and foremost, please know that you WILL recover from this.
Your H and the OW are acting like children. I think that it is time that you set some boundaries to protect yourself and your children. It does not have to be an ultimatum, because I honestly do not believe that ultimatums work.
Take some time to really think about how to proceed. At least 48 hours to process what happened and how you want to handle the situation. Once you decide how to proceed, approach your H and state what changes you want to make (if any) and explain that you need these changes for you (not as a punishment to your H).
I know that you want your H to get it together, stop the affair and stop this hurtful behavior. But as your H admitted, his life is out of control and that is likely impossible at the moment. So if you eliminate that option, what do you want? What would help you move forward with your life and stop some of the hurt that your H is causing?