Abbey my heart goes out to you! I have lived through two A's that my W has had in the past so I know the pain you are suffering. Currently we are living together but she says we are seperated and I can tell you that if I were to find out now if she were having another A that that would be the end the M for me. Where there is smoke there is fire and if your gut is telling you he's having an A then chances are he is. I'm sorry to be blunt about it.
You have to ask yourself if this is what you really want. My opinion is is that you deserve better than this and you need to stand up to your H. Now I'm not suggesting that you do something "crazy" but from my experiences your H is in control of your M right now and he knows he can do as he pleases because you want so desperately to make things work. I know this because it was one of the things my W told me after her last A. She knew she could do whatever she wanted and she knew she had all the power in our M/R because she knew I wanted her back in the worst way possible.
It wasn't until I finally woke up one day and had had enough of everything and moved back to my parents that things began to change. Now I'm not proud of what happened when I moved back to my parents nor am I advocating this but what really woke up my W at that time was the fact that I started seeing someone. I never had any intentions of doing something like that but at that time the pain of rejection and the feeling of loneliness consumed me. That act could have very well ended my M but it didn't it was a wake up call to my W and she came to realize that at that time she did not want to get a D. Like I said we were fortunate at that time to get through all of that but I do regret how things came about.
I guess my point to you is is that I think it's time for you to take care of YOU. I don't think your H's behavior will stop until you put a stop to it. As long as you allow him to do what he is doing he won't stop. I'm sorry to say this but I'm saying this to help you but he is treating you like a doormat, just as my W did to me. It's up to you to stop this madness but you need to do it the right way not the way I did it. I hope you don't think I'm out of line here but I'm just giving you some food for thought. Your H is playing you for a fool and I think it's in YOUR best interest to put an end to it.
M 44 W 43 S 23 S 15 INILWY 9/11 Divorce Mediation started 3/13 June 30 the day W is moving out