I would insist that XW not come in your house when you are not there. It is a big intrusion on your privacy.

Having joint shared dog custody seems strange to me. I mean, if you write down the arrangement/schedule, is that called a "mastering plan"?