I'm here to 2nd and 3rd that. Coming from a person who left their ex and a person who was stuck with a person who wanted to leave desperately and also wanted to be buddies....you've made you're bed you lie in it. If you are feeling particularly kind you can say "No, you'll have to sit this one out. Maybe one day we'll be in a place to go somewhere together like that, but I can't predict the future. However, right now it's just going to be confusing for the kids. Sorry."
