IP - How far in advance does H get a schedule of his shifts? You have a right to ask for that schedule. You can and should put an end to his showing up whenever he likes, but still allow him and the children plenty of time with one another.
I'd definitely let him know you're working on a schedule that brings some routine back into the children's lives, but allows you both to have more equal "free time" with them. You also need to insist he take them on weekends he's available. How else can you start GAL?
Try and flip the way you're looking at the situation. Make him babysit his own kids. If he's out having a life, why can't you have an opportunity to do the same? Put the man to work!