I agree with TX and I think it's reasonable to 'call him' on this. For example - you could calmly say something like - Hey, what happened yesterday - you coming into the house & falling asleep on the bed - that doesn't work for me. This isn't your home anymore and I'll ask you to please respect that in future.
To me that would be enough to put down a firm and pleasant marker with him.
Hope this helps....and it sounds as though you are generally doing well in your interactions with him - well done
T 13 M 7 Me 48 H 46 SS 15 BD 7.14 PA D final 5.16 (H filed)
We receive & we lose, and must try to achieve gratitude & embrace with whole hearts whatever of life that remains after the losses - Dubus