As your wife's husband, you are responsible for PROTECTING YOUR WIFE.
The risks right now are : Pregnancy, STD, violence from the OM (he's lying to his wife so his integrity is a huge question mark right now).
If your wife was drinking heavilly, or gambling away all your savings and racking up debt, or driving at high speeds and putting her life in danger.. would you DO something THEN?
Well, the risks are just as high now, if not higher.
Affairs can result in violence (what if the wife finds out... she may have a registered firearm), financial mess, stress, anxiety, depression, unwanted pregnancy..
Are you going to sit and let that happen because some ignorant IC said "its none of your business"? which it IS since its YOUR wife who is involved here?
Protect your wife, protect other women in your community and DO SOMETHING...
Pup, i think the rationalizing here is motivated by fear.. I really don't think argumentation will help anyhow, but there you have it...