How he's treating you after you ML, doesn't seem to be working for you. I know that you know that you're worth much more than being his "booty call". I'm not trying to be harsh, but it seems like that's what it is to him, considering how he treats you afterward.
Are you afraid to not ML, that he'll go -- or has already gone --elsewhere?
