You may think this is just piling on here, but....
It sounds to me like you really cannot depend on your husband for anything. Anything at all.
No, it isn't fair, it isn't right, and it isn't proper. But it is. It just is. Please take care of babysitting and other details without involving him.
It's your life, sugar, and you can't allow him to control it in any way.