Sometimes , the way you describe him, it sounds like he is acting out of guilt ... As though he's afraid to say what is driving him. One thing about my wife being so mean to me and so filled with hate is that I don't hope I could keep any kind of relationships with her so her actions sting, and then are forgotten . Your h wants you happy - married or not.

Did you ever write or tell him sincerely (if true) that you forgive him and wouldn't be grudging him for what occurs or occurred - married ir not... I am trying hard to not hold my wifes nasty actions against her so I can coparentco parent and as I was told once, forgiveness helps the forgiver, and doesn't require forgetting.