Just remember that some kids just really dislike the moment of change. He might just as easily cry and say he wants to stay with you at the end of your weekend. It's a lot like kids who cry when dropped off at preschool - go back 20 minutes later, and they're happy as clams. Then they cry when you go to pick them up.
I'd try NOT to make any kind of deal at all about handoffs.
