MLCers can imagine that there are people who are interested in him. For example, my xh thought that there were 4-5 women that were interested in him and at the end of the day, there was only one. If a person smiles at them or listens to what they say, many of them then think that the person is interested in them....not always the case. I would venture to say that there is only one person who is interested and he's telling her everything about him and the situation, which we all know that he should be talking to you about. She's stroking his ego, listening and validating and all of the time believing what he says...little does she know or care that some of what he's saying may not be true at all.
It's time to start thinking about your needs and the needs of your children. Why? Because no matter what you do or don't do, it will never be enough. You have to find a way to stop pretzeling yourself for him. It's time to take back control over your household and do what is best for you and the children. If he doesn't like it, too bad...but you and your children need to be comfortable in your own home. I'm going to suggest that you read HaWho's threads. She's got a man/child and she's been doing a great job in handling him while she had two children in the home.
When he starts to say hurtful things to you, stop him and say "I'm sorry you feel that way" and walk away. You are under no obligation to listen to that BS. If these people are not comfortable w/getting involved w/him while you are still in the picture, then they should cease all communication w/him. Sounds like he's trying to justify what he's doing, hoping that you'll get angry enough to pull the plug on the marriage...don't. If he wants out, then he should be the one to do the heavy lifting.
Keep the focus on you and your children.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.