I am so sorry that this has happened to you. If I happen to be the one he contacts, I'll set him straight.
Yes, you've got a lot of red flags going on and I would definitely not have anything more to do w/him. It bothered me about the time he set, as well as coming to your home.
Be careful when dealing w/him.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.