IMO, I think he went to your house and was upset he couldn't get in so therefore he is trying to retaliate.
You are not invading his privacy either. Believe nothing of what your hear.
They say and do such hurtful things while in MLC, don't take it personally. I know it hard, it took me a while to understand this.
And don't worry about messing up. We all backslide. We are not perfect. There is only so many times your H can push your buttons.
I really do think he was just shocked he was locked out of your house so therefore he is acting like a little kids and has the attitude of "well then you are not welcome at mine". So childish.
You are one strong person to put up with all that you have been putting up with. I admire your strength. One day your H is going to realize what a jerk he really was.