Walking back, that is a tough situation to be in. I cannot imagine to think what it would be like. I think you handled it well, especially emailing him back when he took it the wrong way. These OH must be crazy not to see what great people they are leaving behind
Keep up the good work and just take it as it comes.