Please understand -- I do NOT advocate you actually go there, or even that you actually talk to him on the phone. Just want HIM to get a message like "Some guy called for you. Wouldn't leave his name, but he sounded pretty upset. Said he was probably just going to come by and ask for you."
