He does work out at the same gym, and I have seen him avoiding me...
His biggest fear is that I will come up with proof and expose him to his wife. I'll just let that sit there as an open threat.
You SHOULD expose him to his wife! Why -- out of the 4 people involved -- should she be the only one NOT to know? Why shouldn't she be able to make the same decisions that she needs to make for HER family, as you are getting to thoughtfully contemplate for YOURS?
An affair has legal, financial, emotional -- even medical (just check out today's development from MarriedCrazy) -- implications on a family. A betrayed spouse has an absolute moral right to know, and I believe that you have a moral obligation to tell her.