Maybe it's the reality of having to leave, facing everything he's done each time he leaves, that makes him turn into just an @$$ each time he leaves. It seems to be a pattern, right?
So, in a way, it might be a good sign. He may know that he has really messed up and be displacing anger each time he faces the hard truth as he walks out of your door again and again.
What to do? I might bring it up to him. Say something like you get so angry and depressed each time you have to leave. Is there something we could do to help you with those feelings? You have a nice time here and I feel closer to you and then, when it's time for you to leave, I feel you getting angry and dismissive towards me. I love you and I want to help. Is there anything I can do?
Just a thought. You know him best, so if you think this would help...try it...if not...don't! HUGS...