If and when he moves out, you do get to set boundaries as to his coming there to visit, keep his stuff etc. I think it's different for everybody. For example, I don't have a problem (for now) with H coming here to visit D's on Sunday's. I use it as my day to go out and recharge. Should this change for me (or him) it is to be communicated. I know I will, not to sure he would say sh!t if he had a mouthful. It is something I get to decide as far as him coming and visiting here (he moved 60 miles away, his choice). For the moment it's ok, don't know that it will continue to be. This is our (mine and D's) home now. I get to set the rules. It is not unreaonable for you to want and expect privacy in your home.
While I know how crazy making all of this is, you have time to decide what you need/want if he leaves.