Is there any way to tell him that something like "you're obviously going to be busy with a new baby, so we'll be fine in Vegas without you." It will let him know you don't need him there as well and you can have a better time then without him there.
I don't know what else to say. I thought this would hit hard but I wasn't sure and now I can hear it in your posts.
