I know it is hard, but try not to let the letters do this to you! When it comes down to it, they don't really change anything, though they might speed things up a bit, if that's what you choose. I think you've been lucky so far, your H at least appears to not be going crazy with the money, but at some point you need to get yourself and your kids protected, maybe these letters are telling you that now is the time. As far as whether you are "done" or not, don't rush yourself. Let go, yes, but you don't need to quit.
I think avoiding any unnecessary contact makes a lot of sense. It sure isn't going to help you to see him right now.
