If he can let it go and you can let it go, start from there. As long as he continues to be the victim, nothing will improve.
I get that he's been hurt and doesn't trust you to not hurt him again but he seems unwilling to look at his part in the M problems. Without self-awareness people don't change.
Create a co-parenting R from where you are today.
Me 57/H 58 M36 S 2.5yrs R 12/13
Let me give up the need to know why things happen as they do. I will never know and constant wondering is constant suffering. Caroline Myss