I think that's a great idea. Pack all of X's stuff up and store it somewhere. Does he really have that much left after all this time? Also put away any pictures of the two of you. Then see how you feel about the house.
Think of it this way, you need to make room for your new roommate and her stuff anyway, right?
If you love somebody, set them free. http://tinyurl.com/2empx2g