You sound like a great Mom and a partner. When I encourage you to be a great person only a fool would leave, I don't mean conform to your H's warped views. I mean be great in the way you want to be.
It sounds to me like your H had a difficult, if not emotionally traumatic, upbringing; what with the mother being obsessed with having a sterile house. That kind of extreme obsession is likely to translate into a lack of emotional warmth for the kids in the house, and volatility.
Me: 50, MLC/WW 45 Young kids Nov 2015: BD1 Apr 2016: BD2 Jan 2017: W filed Feb 2017: D final