Before you ask him to leave, you need to know whether you are in a financial position to keep everything going without his assistance. If you are, then I recommend telling him that he should leave. He might not actually want to leave. The OW might not actually want him to move in with her. He might decide to leave and they might blow up quickly. Or he might leave and jump into a relationship with her. Even if its the latter, you'll pick yourself up and recover with time.

-Spiral