It's completely reasonable not to be friends with your ExW. It's not selfish. It's a healthy boundary. I think you did the right thing about talking to your kids to make sure they understand and are onboard. Maybe have a chat with them about your granddaughters birthday as well, and request ExWs new boyfriend not attend. Seems like they'd be understanding.
Me:39 Ex-W:37 M:7 T: 9 S:6 D:3 BD/IHS/Confirm EA/PA: Feb '20 OM1 affair ends: May '20 W/OM2 & moves out: June-July '20 W files for D: Jul20 OM2 confirmed: 9/2020 Divorced: May '21