I think that you are doing really well. This really is a looonnnggg process so you have to try to not read into every word, look or response or you will drive yourself crazy.
And almost every time you or he move closer (ie: holding hands, sharing thoughts, etc.), he will pull back.
Hard as it is, dont take the phyical withdrawal personally. It is all part of the MLC.
Try to give him as much space as you can - that is really important.
