Why can't you just pack up the kids and your stuff, and move back home?? You do still have the keys to your home, don't you? What is he gonna do, stop you and his kids from living in their own home? Just go, and say to him that you two just needed some space apart for a while, but now it's time for you to come home. The physical and emotional distance between you two is making it harder for either one of you to do anything positive. And is he having an affair? Or was the arguing about other stuff?
Maybe the other more experienced DBers will say I am wrong, but I think you should try to move back. And I wouldn't ask him, but tell him you are coming back.
