Does he have scheduled time tomorrow morning? Next time, just say "ok." They can't handle politeness, (banter) etc., while you are establishing boundaries - makes their head whip back and forth too much. Just stick to the baby stuff - there will be time in the future for a more normal way of communicating, once he realizes that you are serious.
