you talking to him or not talking to him will not impact the time he spends with the OW. If he wants to spend time with the OW..he is going to. It is so easy to think out of sight..out of mind. But think about how you feel..he is out of sight- has he left your mind? no...
Keep busy busy busy. Control the situation..don't let the situation control you.