Of course you have to care if there is mommy time. But the problem I see is that EVERY time to you is mommy time. Daddy time is just as important because YOU are going to model to your daughters the type of man they will eventually marry.
If you let your W run all over you, dictate how your relationship will be (especially since she's with another man) AND take the children whenever she feels like it, then that's what your D's will do to their H's and the cycle continues.
Just a friendly word of advice. Unless you start putting your foot down, your W will not respect you. And as long as she doesn't respect you, you will not save your M.
If you want your kids to see your parents, then take them. Your W doesn't need to be there. And when did you start letting a woman who is in an A tell YOU what is right and what is wrong?
M-43 W-40 2D - 9 and 5
Emotion, yet peace. Ignorance, yet knowledge. Passion, yet serenity. Chaos, yet harmony. Death, yet a new life.