MO3, You have to do what is right for you, but IMO you wait for him to act, and GAL. I can tell you that we moved several times(different states) when my kids were little like yours and they did adjust quickly to new schools, friends, etc.
The divorce will be the hard part IMO on the kids, no matter how well it is handled. My STBXW believes in the utopia that being friends and using the "Sandcastles" approach, etc, and the kids will be fine. You will forever be tied together because of the children somehow. In a divorce it is a "tearing/ripping" apart, so there has to be pain with all involved and we have no idea the effect it will have on the kids. I pray for you and all others on this forum, this is not the way things are supposed to happen.