So many similarities in our sitch. Like your H, mine is living an hour commute each way away and he's taking the bus. He hates public transport and feeling closed in by lots of people in a bus and train yet this was entirely his decision to move that far away to one of the furthest suburbs.
The routine I'm in now works very well for me too. I guess what I've realized is H must have been very either very miserable or very desperate to conduct his A with the OW without anyone around to see. I know for certain he was very uncomfortable when he lived here and yet also very self-absorbed. He kept mentioning how much weight he's lost and how stressful this was. I just held my tongue when he talked like that when he was here--yep, I'm good at holding in my anger too--and tried not to do anything passive agressive while he was around. Just pounded a few pillows and had conversations in my head like he never even noticed I had to stop wearing my wedding ring because of all the weight I've lost.
So please don't struggle with guilt. Use your sadness productively. Use it to be sad for the man he was who is dealing with something very profound. One day you may be able to laugh together about this crinkle in your relationship and maybe one day you'll look back on all this and realize that painful as it was, it led to you having the awesome life you're having at the moment.
