Am I doing everything wrong?

[Queen_of_Swords]

Confused, hard as it is you're going to have to face your fears in regard to your H.
Your fear is trapping you now.
What's the worst that can happen? What's the best? What do you have direct control over? What do you have absolutely no control over?
Control what is directly in your hands. (That would be you.) Understand you have absolutely NO control over your H. or his thoughts, feelings or actions. NONE.

So he gravitates to OW more...people here will tell you most affairs are short lived and burn out fast. So, if he does see her more it may speed things up, because the OW isn't perfect either and is on her best behaviour...for now. She's Miss right now, but not Miss right forever. Bet on it.

Take this time to work on you. Figure out what you want out of life, figure out who you really are, who you want to become and how you wish to behave.

Let him go, you know he's gone already in so many ways. The more tightly you hang onto him right now, the more you'll make him run.

This isn't easy, not by any means, I still have trouble with it.
It will get easier in time. Use it well. Decide how you will live with or without him. Make it good for you in either event.