I'm so sorry, sweetie. I know that is hard to hear.
So, first things first. Do not allow you wanting to stand for your marriage to interfere with you taking care of what you need to financially. Two separate things. Do what you need to do.
I know you are early into this. The thing is that when you tell them that you are standing or that they are in a MLC, they get angry. He feels as if you havent heard him and is is frustrating him.
He is saying he doesnt want to be married right now. You certainly dont have to like it or agree with it, but you do need to hear it.
Think about it this way. Supposing it was the opposite. Suppose that you are telling him that you dont love him or want to be with him. And he is saying but thats not true. You do love me and want me. You would get angry that he wasnt hearing you.
You would be saying to him, it doesnt matter what you want, it only matters what I want.
So, stop saying those things, ok.
Now you know the truth...hurtful as it is. If you keep asking him about the ow, you are giving her importance she doesnt deserve. You are giving her power. She is a bandaid.
As for the children, this is very important. Your job as a parent is to cause no harm to their relationship with their father. You want him in their lives. This is between the two of you.
It is up to him what kind of relationship he forges with them. But you should not say anything unkind about him to them. You want them to feel loved by both parents.