I don't think your H is angry at you or believes your marriage was a mistake...he's saying these things to make himself feel better at what he's doing/how he's behaving. At least subconsciously.
From what you've described of him in the past...this isn't like him, so maybe he's coming from a place of addiction, guilt etc. Who knows? But don't be concentrating on him, or worrying about what he'll feel in the future...your little family unit is what's important now.
I know it hurts, it's fresh and raw...but letting go is seriously the best medicine you can give yourself right now. It's working for me.
Huge hugs Nicole xx
Me 50 H 48 S 23 S 21 D 19 Together 31 years Married 25 years Separated April 2009 Reconciled 2010 Separated September 2017