So sorry for what you are going through. I do agree that you should trust your gut feeling. They are usually true. As far as your H is concerned, that's your call, but here again, follow your gut.
I try to remember that nothing that is worth it is easy. You don't want to put yourself in the position of his cake-eating.
Now, easier said than done, and I hope I have the courage to follow through my own advice, should the time ever come. ((HUGS))