well, you did really well. If he keeps saying he wants to leave, then you keep up with "It's not what I want, it's what you want, so it's your decision." Do not let him put this on you in ANY way.
Enjoy the interaction. Change the subject. Have some fun stupid or just mundane subjects on hand....you saw someone fall on their butt at the grocery store, kid does/says something funny or sweet, whatever and change the subject. Keep it all light and comfortable.
Plus, ask him to do stupid little things around the house, like light bulbs or whatever. Make him feel needed.
