I would personally stick to your own plans, but take a rain check... make sure you emphasize that you absolutely appreciate his offer, but unfortunately, you were not aware that there were plans confirmed with him today. Be sure to stress when and time you ARE available and ask him if he would like to reschedule... if he doesn't, it's his loss.
