I understand your confusion regarding time with son as family, because that was something I my self struggled with immensely.
My advice is really simple:
It comes down to one thing: You. <-- What do I mean by that. It means that, if you can go be with your ex/wife/stbxw or whatever she is and do things with your son as a "family" without having ANY expectations in regards to your situation. If you can do that because, your sole reason for doing it, is that you have an interest in being there for your son, then by all means, attend EVERY chance you have to be with him.
The fact is, that he loves you, and he loves his mom, and its precious for him, to do things with both parents. That is not possible and applicable to all situations, because the relationship between the ex spouses does not make that possible - however if it is, and you are there only for his sake, then go.
Do NOT miss out on his life, because its not cake eating if you drop the rope - then its co-parenting and parenting at its best. Its not about her, its about you and those very very important moments with your son that matters to you. Who cares what she gets out of coming along, that not for you to read into.
Detach - drop the rope - then go spend every minute you can with that little beautiful human being, because before you know it, he is two.... I would know - my son just turned two in january.
Oh I babelled a lot, when I wanted to write a short reply, hopefully you get my point
BD: Wife says "its over" 11th august 2018. EA: June 2018 PA: August 2018 - ongoing Status: Taking turns 7 days a week to be in the house w. kids WW: no regrets, seems happy with leaving.