He just sent me an email. Very kind and respectful which oddly just makes it worse.
Anybody who reads this will agree with him. And he's right.
It's just easier for me to deal with everything when I think he's being a jerk. But after reading this email I know he's not coming back. There's nothing to come back from.

Back to square one!