I see soo many positives in your post. H is with you. At many levels he has shown remorse and wants it to work. There is no baby with OW. Stop focusing on OW. You have an opportunity to rebuild. Your child will have a full time father. It is normal to be hurt but in spite of his behavior and silences, try to create the atmosphere to move forward.
None of your positive behaviors imply in any way that what he did was proper. Manage your emotions, act "as if", control the angry outbursts and try to put all of this behind you.