BQT - Sigh... the more things change, the more things stay the same. Here's my advice, for what it's worth. Go pitch black. Arrange visitation through a family member so you don't have contact. Tell H that you will resume contact with him once he has had several months of anger management (from my perspective this is a million times more essential than his OW garbage) and several months with ZERO contact with OW. 2 days of either is not even close to sufficient. I see him as abusive - physically and verbally, and I think that he uses the OW as a further means of torture for you. This is not a normal WAS scenario, I'm sorry to say. It's time that he loses every form of access to you, and time for you to start being that fabulous person that we all see here.
I know this sounds harsh, but you know I'm not a rash person. It's time for a real 180, or you will find yourself doing this insane dance for years to come.