He def. is trying to push your buttons, however they are your's to control not his, maintain your even keel. Much easier to make a good decision when calm. I would have to laugh at him if he were to call and try to spew venom my way...
If is up to me, then I'll kindly take the payments immediately. Is there anything he can sign to you as collateral for the payments?
You can not change your past, but you can ruin a perfectly good present by worrying about the future.