Good question, how? That's something you need to think about. How? Have you told him you need displays of affection ? Did you say, H I like it when you touch me in passing, or wink or whatever? He might just need to hear that from you.
When you can't make a decision because you are torn between your heart and your head, listen to the half with the brain.