I would not go either. You jumped at the first invitiation she offered. The children's exchange schedule can easily be discussed via the phone, a meal isnt necessary OR in your best interst at this time. And I think it sends mixed messages to your kids.
You should have declined, said you had plans and found a mutually agreeable time to have a phone meeting. And really, dont be too quick to bend to her "plan" about changing the schedule. If it doesnt work for you then let her know.
I would call and cancel and tell her that something came up last minute and you wont be able to make dinner and then set up a phone meeting.