If you are there when he arrives you are telling him that he can just do what he wants and you will tolerate it... he's testing you.
You told him to contact and he's testing you.
Show him you won't accept his games or manipulation. He's beeing manipulating you with lies for months... Now he's trying to do it by being obscure.. don't allow it or you are showing weakness and he will use that against you.
