Thats how it goes - in the end they see you how they should have, especially if you have been the rock, while they where playing around.
Do you think you could give him a chance if he would prove to you that he wants to make a full committment to coming home to you, and putting all of this mess past you?
Divorce is messy and I wouldn't really suggest it unless there is a case of systematic abuse, and neglect and its your only alternative.
