I'm afraid I'm going to hate him for it thought, there is that primal feeling that he doesn't deserve it so why should he go?
Based on his comments about feeling smothered, etc...he's getting this vibe from you too. As much as you deserve to feel this way (IMO), will it help you move forward in the direction you want to go? There's a fine balance here--you need to work through these feelings and he needs to realize there are consequences, but you both need to feel some sense of comfort/hope in each other's presence.
I hope the C sitch works out for you both. And I hope your H has the courage to really look at himself and do the hard work.