G - There have been nights were I would have preferred to stay home vs going to spend time with the Dr. I think I have only said "no" one time in 4 months. We make plans to see each other and don't really do things spur of moment just because of kids and schedules. For example, tonight I would much rather stay home and watch the last episode of GOT however she invited me over so I will watch it when I get home. I am choosing to go spend time with her.
It very well could be a lot of things or it could nothing and that is the way he is. He says he loves you, you have met his kid and vice versa, you also do things together with the kids. Take stock of the entire R and determine if it is fitting your needs.
FWIW........unless I had to work late, sick or had something else that I couldn't get out of if the Dr. wanted me to come over for dinner I wouldn't say "no".
I think this pace stuff is a very common issue and one I might be experiencing myself. My only advice would be to communicate your needs, desires, and what you want. See what he says and go from there. Ultimately if this is who he is and he can't give you anything more right now then you have a decision to make. If he is unaware of how this is impacting you and changes/steps up or whatever you want to call it then issues resolved.
Maybe you are feeling insecure about the entire R as you have mentioned more than once about him sheltering his son then when you add this on top of it adds more fuel to the fire.