Don 't leave the house and go to your girlfriends. Like I have said before to you, I think you should exclude them from your dealings with your husband. You are simply feeding his insecurities.
You also need to listen to what he has to say. Again, advice for you to put him in his place is not good advice.
He is on the cusp of a decision, and he is very seriously considering leaving you. I am surprised that he hasn't engaged in an all out affair yet.
He is obviously still hurt about the past, and is questioning his decision to marry you. That isn't some wayward fogged out mind at work, his pain is very real to him.
Let me ask you a question. It would really help if you would be brutally honest. Before you were married, how much time passed between you announcing you were broken up and you having sex with someone else? Was the break clean?
Whether you realize it or not, he is feeling trapped into the marriage, by his own nature that caused him to fight to win you back, and then marry you.
What are you doing to demonstrate to him, your level of commitment to him?
Hint; discussing your issues with others bothers him, he has said as much. That contributes to his lack of trust. Going to your girlfriends to avoid a fight with him, shows you are not committed to him.
-NOPkins-
I will ferret out an affair at any opportunity.
-An affair is the embodiment of entitlement, fueled by resentment and lack of respect. -An infidel will remain unreachable so long as their sense of entitlement exceeds their ability to reason.