You are afraid to push. What I'm suggesting isn't pushing either.....it's clarifying. You need to ask questions of him when things are going fairly well....like now. Now, IMPO, is the time to either just walk up and kiss the man and take the chance at rejection. Either he will allow you to kiss him, or he will reject the kiss....and that will open up the opportunity to ask him about it.
I understand your wanting him to do things at his pace....BUT there are two of you in the R. In some ways I believe you are shooting yourself in the foot by not #1 sleeping in your own bed, and #2 addressing some of the behaviors that make you feel pushed away too. You make it so very easy for him not to deal with you.....and that makes it far too likely things will continue as they are now. Yes, you did something wrong....but you've been paying for that for a long time too.
Others may not agree with me (wouldn't be the 1st time)...but that's how I see it.