If you don't feel you can handle it, by all means, don't snoop. I wonder what it is that he's telling everyone, that makes them immediately believe him? Having proof that THAT is untrue would almost be more powerful than proof of the affair itself, as it would show that he's trashing your reputation unfairly. But you're right -- "blood is thicker than spaghetti sauce."
