Ok. Here is where you need to STFU. I hope you understand that acronym. If you don't....it's Shut the Bleep Up.
This is why I say this: he's said the magically words of let's work on this, I'm sorry.
Do NOT give him the opportunity to change his mind or to say that he's changed his mind. You MUST keep it all comfortable and calm. Even if that means you leave....then by all means, LEAVE. There can not be any fights for at long as possible. He pisses you off, remember this: He may be TRYING to. He may be testing you, he maybe hoping for a fight so he can have an excuse to leave.
Don't give it to him.
You are doing great. Keep doing great.
Relax. Let him figure it out and do the work. You just keep doing and give him space. Good job.