Here is my thought: be his friend. If he says he is not ready to work on the relationship, then he probably isn't.
But what has to happen to have a loving/lasting relationship? Friendship.
W and I had gone down that path - we were talking, spending time as a family, she was in her own apt, but we were over almost every single day. When she would drop off kids, she would come in, get stuff out of the fridge, talk to me, even stay for a couple of hours.
I kept pushing and pushing and pushing for more - I kept insisting that she be "open" to something more. I realize in my 20/20 hindsight, that if I had shut up, she would probably have allowed more eventually, but wasn't ready right then.
Then she found OM - someone who lived an hour plus away, was married and had three kids, and was extremely meek/subdued/quiet. A perfect person for her - she didn't have to commit, didn't have to be around him a lot, and he has the OM script - telling her how wonderful she is, and how horrible I am. Things have obviously gone straight down hill from there.
I have gone completely dark on her - won't tolerate an affair, no matter how "benign" it seems.
Take it as a lesson learned from me.
If your alternative to being friends is going to be pushing him into something he doesn't want, what choice do you have?