I would act how you act when you work. If you were working off site in an office that was not in your home you wouldn't be able to chit chat with visitors, right?
Acting "as if" and being polite but distant is not deceitful or manipulative. It is simply a way to avoid being sucked into anything with a WAH who is behaving VERY badly!
If he wants to chit chat he should arrange to come when you are not working.
