If your H doesn't want to stay within the boundaries set by the judge, then the legal system can deal with him. I doubt very much that you trying to deal with him yourself is going to do any good, as it sounds like he feels he is 'above' the rules set forth.
Too bad it sounds like he's using the kids to push your buttons.
Stay calm, but never forget that you are playing by the rules, and he is not respecting the boundaries and will have to pay the consequences if it continues.
Document each and everything that he does outside the realm of what the court set forth. Date, time, witnesses, etc. Inform your L that you will be doing this and see what he/she suggests.
Women are angels. And when someone breaks our wings, we simply continue to fly...on a broomstick. We are flexible