Water! Hi! Ok, so now I want to smack him for you! Sorry...
Yes, he has NO self-esteem. I hope he gets help too. You have good goals. You gotta take care of you, which I know you will.
In a way it is considerate of him to want to stay away from you so he cannot hurt you, but it's also the easy way out. (as someone once told me!)
Focus on having a "casual" R with him. Not necessarily sexual unless that's what you decide to do, but act as if there are no strings and you don't care. Seems like he can't handle the pressure he is putting on himself of being "committed," so maybe if he feels less committed, he'll feel more comfortable.