Unless Tuesday is his time for custody (according to your parenting agreement) your H should not be taking your kids without your prior consent.
He is plainly bullying you and doing as he pleases. Do not yield to his belligerence.
I would tell him in no uncertain terms what the schedule will be for this week or any week. Spell it out just exactly how you know it is supposed to be and stick to your guns. If he doesn't like it or wants to make a counter-offer, then fine -- just so long as he's respectful of your interests and those of the children -- if not, he can take a hike.
