I think you have it all settled now. You said it in your post. Let your h know how you feel about the gathering. I would feel the same as you do. I would want to be there too. My S is an adult and it still made me mad that my h took him over to her house and they met. My S hated her and called her a crack addict. It still bothered me that my h would do that.
Anyway, you have to do what you and your S feel comfortable with.
