LO, my H was doing the same thing. When provoked, he said he would leave me because I do not appreciate what he has done. In your case, he thinks about leaving becuase he feels you won't be able to sustain your change. He is still confused. Good part, he is confused. He still does not know who he wants, which means he has not decided. Bad part, he still wants both. He is not ready to let her go (for whatever reason), yet he is not willing to let you go either (hence being super nice to you).
At some point, you will have to decide when you have tried everything. When you are ready, then you will be ready to give him the ultimatum. Hopefully that's the point when he thinks it's a "better" option to stay with you. If not, it may be better to let him go. If you do let him go, it does not mean he won't come back. He may, after leaving, to really see the real situation. Of course, you will have to be prepared that he is gone for good.
It is all your choice now. He will be happy to live in the current situaion for a long time. It is your choice whether to wait it out or not. I think about how my life actaully will be better without H, in terms of having more time for myself, don't have to spend time and energy on M but only working for myself and my kids. OTOH, I am commmitted to M and will do everything to save this M. But at some point, I will decide I cannot give anymore and wait any longer. Then it will be the ultimatum.
Give yourself sometime to calm down and see what it is that you want to do. You know your H best. Do not concentrate on his actions/talk when he is mad. Look at the whole picture and assess where you two (and OW) are at. If you give him LRT, and he comes back, what will your conditions be? If he does not, what are your plans moving forward? Will you be OK either way?
Hugs to you. It will be a tough decision. Only you know yourself and H enough to determine which path to take. Good luck. OC