I find myself feeling really angry with your H when I read your posts - and perhaps I am just projecting some of my own situation onto them, so read whatever I have to say with a pinch of salt and only take what is useful.
But you're not really allowed to say no to him and him still be friendly towards you, are you? You're not allowed to tell him he's not allowed a key for your car, he can pay his own rent, he's not allowed to have a party in your house and leave you to clear up his empty booze bottles, he's not allowed to speak to you with disrespect, he either participates in a proper family holiday or leaves you to enjoy one with your children, etc etc. Even a tiny sniff of non-compliance on your part - like asking for time to think about a holiday he's offering to book on your behalf, but not bother attending the majority of it, releases his teenage whinging self-centered self...
Has it always been like that? Has he always been such a self-centered child, and have you always catered to him?
I feel sorry for your kids. I know you are doing the best you can for them. But at least one of them wants a family holiday - and that means it's up to you to put yourself through the wringer to make that happen (okay, your H is arranging it, but we both know if it happens you will have to put up with so much poor treatment from him...) or to say no, and tell them why.
I can't help but think a family holiday involving your husband in any way is just another form of denial and while saying 'no' might hurt in the short term, it would be a huge step forward in terms of you accepting your situation and H realising you're not one of his employees. If he wants to bond with his kids he's more than capable of sorting out that himself. People whose marriages have ended don't go on holiday together.
Quote
my priority is my kids, and I know it would be good for them to have a nice family holiday
It won't be a nice family holiday. He can't be bothered making time for your children, and only wants to come at the weekend. He'll drink too much or he'll try not to drink too much then be resentful at you for that. He'll be rude and miserable and blame you for it. Your kids will watch you being either upset at his behaviour, or disappointed that he still isn't interested in sex or spending more than two nights in the company of the children he claims to love so much. Don't make your decisions based on a fantasy.
