Agreed, better to ignore him - you are not being controlling over anything more than YOUR own life! Tough sh!t for him. Let him figure out on his own that he would have to go back to court to get the times changed - and then the court would see his disruptive pattern of non-visitation. Seriously, have you thought how those 10-15 minute visits are going to start to make baby crazy? Get her all excited and worked up, then disappear! He is an A$$ with NO idea or care of what effect his actions have on others!!
Your class sounds great, btw...hope it goes quickly and you are able to find work. But do be aware of the pitfalls of working from home - make sure to schedule grown-up, mommy time!
