How about that you have primary custody with your H getting reasonable access and visitation? Meaning he has to arrange it with you beforehand.

This doesn't look like you're trying to force visitation or deny him visitation if he tries to rebuild his R with the kids at some point.