"Hey, I think what you're doing is disgusting, cowardly and -- frankly -- unattractive. But you're a grown man, and I don't wish to control you. You do what you want, but I'm not willing to live in an open marriage, and I realize now that while this isn't what I wanted, and it will be difficult, that I'm going to be just fine."
That should be the attitude that oozes from your every pore.
