Honestly, I don't think it was fair of your mom to bring up OW if she is not 100% sure... meaning she hasn't seen anything herself yet. I don't think it's fair to you or children especially for her to bring the subject up in front of your children.
My DH had a PA 2 years ago. For 2 years, we tried to make it work. Of course, I want(ed) to blame OW, but it went so much deeper, ya know? I know that now. If she ever comes up (which is not much if any anymore), he asks if it's easier to blame someone for what has happened to us. Maybe.
I work 3 - 4 nights a week from 9 pm until 1 am. DH usually has 7yoDS at night, but DS has been staying here more and more lately. Well, one night last week, I guess, DS didn't want to go with DH (he says DH goes to bed too early... lol). I got home and I hear 7yoDS... scared the crap out of me. So I asked DH to text me whenever DS did not go with him. He's been really good about it since. Hopefully your DH won't make that mistake again.
What was your mom doing out at 3 am?
I can't believe this is my life either. There have been times when I wonder if I shouldn't just throw in the towel, too. But honestly, DH is worth fighting for. So is my family. You keep being the best mom... the best person you can be.