His sleeping with a co-worker is on him. Rationalizations aside.
What I'm going to say may sound harsh, so read it with a soft voice. He's told you he wants a divorce and he doesn't live in the house with you. Do you think this may be because regardless of legal action he's divorced you emotionally?
I know that sux to hear. Believe me. And truth be told, you're the one that has to answer this. It took me alot longer than I would have liked to see this for myself. Did it make a difference for me? Yes. It might also take his "unfaithfulness" off the table. In your mind, you're still married. In his, well, he isn't.
If you truly in your heart believe you would never be able to trust him again, what is it you're hopeful for?