Your dad laid everything out beautifully. What he said is exactly what it should look like.
How do you enforce these boundaries? Well, you set up a parenting schedule you both can agree on. he has his time with the boys and you have yours. And it's not at his whim. Present him with a schedule that works for you and offer him to make adjustments where needed. This will let him really know what life is going to be like. No more family time with you, the kids and your dad. Just him and the kids in his allotted time. Having to arrange work and OW to see his kids properly and care for them on his own.
It is the reality of divorce. he needs a taste of it.
Out of curiosity, when this happened the first time, why did your dad feel he was going to do it again?