However if I try to touch him even on the arm he will pull away. That's pretty scary and something I am having a hard time coming to terms with.
I can understand this. If you associate the person with pain, then you don't want them touching you. If your dog constantly bites you, you might still like to pet them. But if they turn toward you, then there's fear of them biting you again.
In my case, I choose to not touch my H either (definitely don't want him to touch me) because I don't want to encourage any sort of closeness at this point. He's just too hurtful to me.
