I agree with Mishka--he knows you are only partially detached from him, and that he's still "got" you, and in some moods that's not very attractive to him. You're giving him too much power.
I found it best, whenever I got those phonecalls that had me wondering what he REALLY wanted, or that had uncomfortable bits in them, or that became not-how-friends-interact, to cheerfully and quickly end them. That gives you back a sense of control, and lets you reward him for good telephone episodes.
