Hi - I am very very sorry for your situaion and know exactly how you feel. My A also began as an emotional affair and it built to more. When I was caught I hated everything about my marriage. Only had eyes, heart and soul for OM. BUT my H was sooooooooooo hurt that even in the midst of anger and wamting someone else I could not abandon him. We did not live in same house etc we went to MC but I still just went through the motions.
Then H seemed to change - he started telling me he was working on himself and he joined a few things to help his self esteem etc. All of a sudden he some how became more attractive to me. I had stopped all contact with OM out of guilt to H. This was key.
What i am trying to say is that i firstly needed to see his pain. It was open, raw and heart wrenching o watch. Might of been guilt that first made me take notice but who cares. If he played on my guilty feelings then it worked. If he had been okay with marriage ending I would be with OM now probably.
Trouble for me is he got stronger and I got weaker.
Another thing which at the time i hated him for was that he (like you ) had the power to expose my full affair. He talks to this day of confronting the OM and exposing everything but I love him for the fact that he as not. I dont believe he will. He acknowledges the pain it will cause me and even though by not exposing it , he looks like the crazy person who left his wife for no reason as such.
Do not say anything your not prepared to do. I dont think an ultimatum will help if he is involved with OW. I would of welcomed the ultimatum at that stage because I would used it to think "oh well - they forced it " It would of been great to not to of witnesed the hurt and pain. Let him see it. I dont mean wailing and puling your hair out. Just be honest. Your sad .
Creating more ugliness is not the answer. The truth has a funny way of coming out eventually. Act with dignity and integrity.