If you stand up for the rights of your children, I think this will happen: she will see that you ARE responsible, that you care, and that she needs to think twice about what she's doing.
If you stay away, she will see you "giving up" on the children, on the responsibilities you have as a father.
And I need to correct and say: if your name is on the mortgage/title/lease, you have ever legal right to be there.