I'd say that if you don't want him to see the place tomorrow and it's not a good time for you you're prefectly within your rights to suggest an alternative.
Why not just call/message him and say something's come up and you need to reschedule? IMHO there's no need to wait for him to call or to risk him popping round unannounced- you can take control of the circumstances under which you see him...
