Here's how you could phrase it:

"Look, I'm sorry I raised my voice last nite. I should learn to control my anger better. But I will NOT apologize for not being willing to live in a marriage where my wife carries on an inappropriate relationship with another man."

Puppy