BRNR, We all have had incidents whereby we've "had it out" w/them. Don't beat yourself up. You were expecting him to pick them up and he decided he wasn't going to do it. You didn't think that he would up and decide not to do it. Sometimes when they throw fly balls unexpectedly, it knocks us off course. Don't beat yourself up. You had your say and you told him how you felt about the kids and the situation. Let it go...
BTW, you didn't need to defend yourself to him. You are not in the wrong here, so in the future, when he baits you, just tell him "I'm sorry you feel that way" and then walk away. Mlcers hit below the belt at times and the they want us to take their bait so that they can point fingers at us and say "see, that's why I'm not at home". They want any type of justification for what they are doing.
Muddy or not, let it go. You are only human and sometimes you have to let it all out...come here to do that. Don't give him any more justification for what he's doing. He's a two year old that is having temper tantrums and the only way to stop them is not to buy into his antics.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.