I's tell him exactly what you wrote. I'd be as nice as possible, but firm at the same time. If that doesn't work, change the locks and tell him you're doing it before hand and why.