Your H's letter shows he feel ashamed and is protecting himself in case the kids don't contact him ("I know...blah blah.) He's definitely not taking responsibility for his actions. Sorry, NLW.
I agree that you need a legal document so these arguments don't happen anymore. Have you retained a lawyer?