I know you want affection from him, but other than not giving it to you what exactly is he doing that's mean?
I'm asking this b/c I got into a place where b/c H didn't talk to me much and was so icy cold, I took that as being mean. In truth if it was someone else I was close to and they treated me this way (and weren't talking about it), I'd back off and give them space. I'd tell them when they were ready to talk to call me, then I'd let it go. I think in my case my perception of "mean" was just making me more miserable. It's not like he was really doing anything (except leaving). Changing my thinking really helped.