This is emotional chaos where nothing makes sense. Keeping yourself open to the marriage makes you feel that you have to keep everything else available, too. Even when it makes you feel icky.
First things first.
Do things that are healthy for YOU, mentally, physically, emotionally. This is about YOU, not him. Entering or continuing a relationship when it's all about the other and you're not whole is a recipe for disaster.
What difficulties do you have? Based on what I've read his dropping over unannounced, singularly deciding what's the best way to deal with your son's sadness, etc. That's where your BOUNDARIES come in, like mishka pointed out.
He left the house, the home, the marriage. He has his own place he's playing Lego's with. You are in YOUR home, YOUR house. I set a boundary with my former spouse during the divorce that he had to ring the doorbell and wait to be let in. That he was not allowed in the house unless I was there. It was hard.. BUT.. it made my house a sanctuary, MY safe place. An unannounced appearance by your spouse should not mean that invited guests have to leave. If he's there, let him take the kids out for ice cream. Your sanctuary is no longer his home. He has to feel the consequences of his actions. You need to feel that you have choices, that you can be safe.
Right now your history together is muddied. Trying to understand and be validated is just beating a dead horse. In a perfect world, he would have be able to communicate what was amiss as would you. We are all flawed. We all get to this place together. What we do when the fabric is ripped defines how we move forward through life.
This is an amazing opportunity for growth. Not by dissecting the past, because it's over. Focus on who you are in the here and now. What seems right, wrong. Act on it. You have a voice. You are wonderful.
Here's how I make decisions:
If it seems great, the answer is yes. If it seems wrong, the answer is no. If I waffle or can't make a decision, the answer is no.
Standing in the light of who you are is a wonderful thing. You're great. You're loving. Focus on what's healthy for YOU.. and the rest will fall in place.
