IMO, just go, try and get ready early enough he has time to wonder, but don't remind him. If he asks politely tell him.
IMO, he isn't doing this to upset you. It got forgotten in the fog. Just leaving nicely without reminding him make him accountable for forgetting. You're not his social secretary.
BITS Me 55, ACK, when did that happen? Doesn't feel like 55 D 30 S 27
You create your own universe as you go along - Winston Churchill