I know you want to take the high road here, that doesn't mean letting him take advantage of you. Choose your battles though. For example, you both will have things come up where convenience will be a factor (with your girls). If you can be flexible where it won't hurt anything, that's being gracious. I know you think keeping the girls b/c he is (in your mind anyway) preoccupied with the OW is in their best intrest. However, I would beg to differ (unless he's dangerous, which it doesn't sound like). They need to spend time with him regardless. How many of us are always focused on our kids when we're with them? Even in a happy M the pressures of day to day stuff can get in the way.
I know it's hard to get your mind around all of this. You're right it's moving fast. Stop, breathe. you'll be ok. I know it doesn't always feel like it, but you will. My new mantra for the week is...all of this is making me who I'm meant to be. Sucks, but sometimes we have to make the best of hell and redecorate.