I am chiming in....Your approach was fine, the reason behind emotion was that you were embarrassed and PO'd he caught you in bathrobe lmao!! I would have been as well and you handled it better than me!!

It is perfectly reasonable to ask for a head's up like anyone else before dropping by