I would have and do suggest talking with the police, the nice part there is, you don't have to pay them an hourly rate to find out what you can or can not do in the midterm.
At your next court date I highly suggest you go forth with a temp visitation arrangement. He'll of course have time to respond to it, maybe see the err of his ways.
In the meantime, as Kat says, DOCUMENT! You never answered my question of if you called to make sure they were okay afterward. I'd hate promote a negative feeling, but my X as I said did the same, just to gage my reaction. The more times I came home to no kids and threw my hands up in the air and said 'what can I do about it' the more times X and her snake L sat in court and said, hey, "he didn't even care how his kids were for the time they were away".... watch your butt, it's a nasty, dirty, low down game when it comes to custody.
Me 35/XW 33 S13 & S12 M: 10/17/98 OM & S: 07/08 D final 06/09/09 12/03/09 - 06/13/10 "Piercing" 06/13/10: Engaged to Re-marry 10/17/10 06/25/10: Expecting baby #3 2/14/11