Worst case scenario ... your wife divorces you and marries the OM. Then your kids will be living in the same house with him part of the time.
I hope that doesn't happen either, but it COULD, and there's not a damn thing you can do about it, beyond making yourself crazy AND look crazy.
By which I mean .... let it go, Indiana. You can use this to gauge her sincerity and make your choices accordingly; that's about it. Assuming you're even sure of your facts; was there any face-to-face interaction at all? If you don't know, *definitely* let it go. I can understand why even the thought of your wife and your kids and OM in the same venue would piss you off .... it would me too .... but is this the hill you want to die on???
"Show me a completely smooth operation and I'll show you someone who's covering mistakes. Real boats rock." -- Frank Herbert