I think it is cruel for him to let you belive that the marriage failed because of you if he really left because he wanted to be with the OW. He shouldn't leave you there to twist in your emotions, blaming yourself, when it really just him lusting after something new.
I would bet money that he had at least an EA before he left, though it was probably physical too.
I'm a man . . . But I can change . . . If I have to . . . I guess . . .