I agree - it is MUCH easier. I was very fortunate - when my now H and I got back together, we moved very quickly elsewhere (he was already planning to). It helped a LOT to have a whole new place, and he threw everything out that might make me uncomfortable. I wish you'd be able to do that too - just start fresh, a new place - maybe further than we went (nearby town) and away from OW and bad memories.
But the next best thing - definitely, get away for a bit. I don't know what to say elsewise, except to keep doing things for yourself. Don't worry about weight per se', think about health. When you become more healthy (eat better, exercise more), the weight should come off. Do some core exercises - strength training, etc. All of this will help you deal with anything to come as well, giving you a better sense of well-being. Don't put yourself down in front of your H - that is VERY important. Honestly, you are just perpetuating what he thinks about you when he's down. Talk yourself up - accentuate the good points!!!! Please!