Jim I do not know how it is where you live but around here we routinely put in Custody Orders a right of first refusal. That means if for a certain time period typically 3 hours here, if the custodial parent is away the other parent has the right to retake custody. If the custodial parent does not inform them that they will be gone greater than three hours then you can file a contempt petition. It may be worth talking to your attorney about it.
Me 41 W 33 M 2013 Suspect A 11/15 Confirm A 1/16 She moved out 2/14/16 Stepson 13 Stepson 16