Yeah, when parents are rotating weekends with the kids, it gets complicated. I'll tell you what I am doing. If she has the kids for the weekends, I go to kids activities and events that I want to go. So, if I know about it, I am more proactive about it for the kids. I plan my stuff around that.
If you didn't want to go, that's fine too. Just see if D would have wanted you there or not. You decide the level of involvement you want when she has the kids. And vice versa too.