Or....say, "Yes, I can imagine that must feel very uncomfortable to be put on the spot like that."
If you did cause the intervention.
And he tries to guilt-trip you, say, "I'm sorry you FEEL that way, I'm sure others must have FELT that way when put on the spot, but you know what they've FOUND? They discovered that it's healthy to speak about these things in an open forum."
Or..."I don't care?"
Olive, what's he going to do?
Move out? Ahem...HE's TOO SPINELESS to do that.
Divorce you? Errr...HE's TOO SPINELES to do that, too.
Have an affair? Err....he's doing that.
Leave you? Err...you asked him to that, so you might get what you want.