You've every right to be angry at him (and yourself) and its perfectly natural reaction. Genuinely though if he is not one who will change, you deserve better and new you will have it.
That's something I feel really confident about.
Both mid 30s, 2 young kids BD 7sep14 XW moved on long ago, now living with OM1 D paperwork in progress