Speaking from personal experience, I realized about five years ago that I had gotten to where I was in life based on a set of behaviours. I had a loving wife, kids, great job, a lot of travel under my belt. And I sensed that something had to change about my behaviours. There was something dark. If I didn't change, I would die. So instead of dealing with things, I ended up cranking my "normal" interest in porn up to a "A with Porn" level and have destroyed my marriage in the process.
After my W dropped the bomb, I remembered finding a group called the Mankind Project, which is a non-profit org that helps men discover their true masculine calling. Men helping men. I know the "void" for me was not having a father around to model strong male maturity for me. I had a dad, but he was a workaholic. I think I'm not supposed to talk about other orgs, but you can Google it.
If you've resigned yourself to accepting that he's gotta go off and do his own thing, maybe suggest this. What the heck. And then detach yourself from whatever outcome may happen after you let him know.
Stay strong and good on ya for resisting his cute remarks. They obviously have an effect on you still.
H42 W36 M9 yrs D8 D5 d-day: 21/11/07 S and moved out: 22/2/08 Still S: 22/11/10