I think you should accept the day pass and go enjoy your kids. If you rejected the Father's Day gift it would make you look like a jerk b/c it's suppose to be from them.
I can only guess how difficult it must be with four children split between parents. She can't go dark about them. Maybe you could suggest a weekly summer schedule that would require texting each other on a particular day each time.
It is not about what you feel should work in your M. It is about doing the work that gets the right results. Do what works!