My two cents - show up for lunch looking great. Treat him as a friend and act like you're doing well. End the time together on a good note and don't linger. If it's at your place "hint" that you must get on to other things. If it's at his, leave earlier than he'd expect. I'd keep the interactions children centered and happy. Might could give him a little wink as you go out the door telling him it was a nice meal, you have something to attend to, but maybe you'll see him again over the holiday.