Oh, Nevanna, I can relate! I still struggle some days - because being totally there with my H, letting down all my walls, just feels so freaking vulnerable, that sometimes they start going back up without me even noticing. Intimacy with someone who has hurt you so badly is scary, scary, scary - so we hold back a little bit, hold onto some of the pain, try to "protect" ourselves - even though this kind of "protection" actually makes it MORE likely we will end up hurt, not less.
It just takes time, honey. Your pain is legitimate - so is the pain and guilt and confusion he feels when he sees your hurt.