Yeah, is there really any reason you have to return this particular call? Why not just not answer - let him have to make the effort to contact you a second time. (YOU would, if you contacted him and didn't hear back - you'd contact him again if you really wanted to talk to him).
Let him experience a little uncertainty. Let him wonder where you are and what you're up to.
