This is your boundary. You said your H didn't like it before but he respected it. Why do you feel the need to defend yourself?
It appears that your H is spiraling downward, and projecting all that he's feeling and done onto you. Nothing you do or say to him at this time is going to make a difference in what he thinks.
I'm not an expert, but I am wondering if saying nothing to him right now will help him to figure things out for himself. These are the consequences for his choices. I could be wrong, but I say let him stew.
