Maybe you should reply something like this “I will see if I can put it on my roster.” I seems like he wanted the confirmation that you are still there and easily agreed to dinner with him. If you put a bit resistance, maybe he will pursue you more, and you will have your steak dinner for sure. Just the thought.
If i did this, he would view it as sarcastic on my part and pull the plug, I'm sure.
I understand what you mean about being 'too easy', but with such a big step (I mean, asking ME to dinner with the kids), I'm going to reinforce rather than play hard to get.