He has cut himself off from spending any time with our youngest child to punish me.
Maybe that is true, maybe it isn't. Only he knows and what he may come to realize is IF he did that intentionally, trying to hurt you, he is only hurting his youngest.
As for deciding he didn't want you on the vacation, that is unfortunate. My W explicitly excluded me from things her and the kids were doing, shortly before BD.
It's normal in this crises.
At this time, it would be a great idea if you haven't yet, to pick up and read Divorce Remedy and otherwise begin to implement some of the methods of taking care of your self, right now. GAL and detaching.