My honest opinion is that your husband is in deeper to this affair than you think. I'll bet it is a PA and if not that an EA that he never stopped. Something happened with the A the day he spilled his heart out to you. He either tried to end it himself or was dumped. Him seeming torn up after that was probably due to withdrawing from the affair. My h was the exact same way. When he changed his mind again it probably meant he wanted to go back to her. Sorry to say this, but that is what it really seems like to me.