Let me put my posting another way...are afraid that he's going to say no to your invitations? Are you afraid that his choices will not always be what you expect, i.e., that he'll not want to do things w/you?
No wisdom today in my posting...just something for you to think about when you are rested up. We need to delve into this "fear" you have a bit more.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.