OK, here is a 2x4 coming to reinforce what it looks like you are starting to realize. You need to step back and realize that allowing some other man to insert himself into your home and family like you have done is absolutely insane, and for any self-respecting person it is inconceivable that you would allow this to happen. So what if he apologized to placate you, he is not going to stop manipulating to try to take what is yours, and your wife is on his side.
Women cannot love a man the don't respect, and no one can respect a man who will accept this level of disrespect in his own home.
To your wife, he is not "almost OM", he is the man. He is actively taking over what was yours and you are standing by watching, and being polite about it. To her you are the "almost OM" she is trying to extricate from her life and her children's lives. Trust me - you will be the last person who will ever know that they are sleeping together, and laughing at you, and turning your kids against you.
So what if you are getting a divorce. This is still your house. You are still the man. It is your job to protect it from predators and "Almost OM" is a predator.