Didnt say there wouldnt be a downside. Suggested that the brutal reality is that if Pleasure of Time With Kids > Pain Of Unavailable Wife, and you value TWK more that Time With Spouse, then the logical decision is: Stay for the kids.
Which is a sucky choice to face. Yet is one that confronts many a Walkaway and Almost-Walkaway. Which is, you know, ironic.