That is only because he doesn't have to deal with the guilt when he is with her. These ow have a way of making our H feel guilt free for what they are doing. But eventually I would hope that they can longer fight the guilt and it gets the best of them.
