If it's not urgent, I'd tell him you're busy today and maybe tonight or later in the week would be better.
Sounds like you did really well with your phone conversation! You ended it first, kept it light. The more you talk to him, the stronger your voice will get. Good job!
Clear out your history/cookies before he comes over to work on the computer too!