Hey Dedicated. Don't get discouraged. We are here.

I think you have the right to tell him that he can't just up and decide when to take the kids without talking to you first. He is the one who threw your life for a loop and has been carrying on an affair and being a jerk.

Tell him the kids deserve this to go as smoothly as possible and he needs to respect that.