I know how easy it is to hang on what they will say or do. The more you practice not doing that, the easier it is.
Why ask him if he's going? Let him know what time you're leaving and when you expect to be back. If he chooses to come, he knows how to get in the car. I don't say this flippantly or with any attitude behind it. The only time I ask if H will be coming to something he's invited to is if it will change something (like an extra hotel room etc).
This year all I said was "we want to invite you to the upcoming holidays." My D17 told me this morning that he is coming on Thanksgiving. I have to admit, I'm really surprised. Nothing to skip a beat over though. The plans I have don't change b/c of it.
Takes time to get there and I still have moments. Just not this time and not over this.