If you don't want H there for Thanksgiving I don't think I would ask him. Unless the kids really, really, wanted him there. I forget how old they are.
I think the three of you can have a fine Thanksgiving! I'd be tempted to change things up a bit, really focus on the things the three of you like, and maybe less on any other stuff. I'd also think about doing something different... try to start some new traditions.
