Just reading what you said, I would guess that that night he went out with his "friends", he saw the OW. WHen you got back together, what did you do to start repairing? Did you go to counseling, books, etc., anything?
If you go back to the way it was, nothing will change. He feels like he is doomed in this bad relationship. You need to show him that you don't want him to come back to a bad relationship. You want him to come back to a happy one, and that you BOTH will need to make that happen.
Just going day to day will not make that happen. Even just listening and "taking it" will not make that happen.
