That is just another reason to take Oldtimer's advice. Have the conversation with H and go to Retrovaille as soon as possible.
There are problems within the marriage that need to be dealt with, and then you and he will be happier together. You would not believe the difference it made in my marriage. One thing I've learned is things do not get better by themselves. They build into a lot of baggage that you just get tired of hauling around and eventually kill the marriage.