Holy toledo. I had a whole other thought process about this, but I think Dom is right on. Its really not hard on your side, since all you will have to do is say something like he suggested and let it lie.
I can totally see where the anger comes from, but maybe Dom is right, to put it to the side and try to care for your hurting husband. Even though he has hurt you beyond belief.
I am thinking about you. When do you see him next? Do you know for sure he is at his mom's tonight without OW? I guess you could call him and say "I noticed you seemed a bit down tonight. Let me know if you want to talk about anything, I'm here".