Call me old fashioned but I don't think it is right for the kids to be staying with their dad when he is living in A with this ow. I believe he needs to see the kids but not at her house.
I know when my ex & I were dating, (his ex was having an affair)he wanted us to move in together(they were divorced), but I didn't feel that it was right when he had his child over because we were not married at that time. I didn't think it was right for his son to see us living together without being married. So, we had separate places b/c of his son. I know the world is totally different now about things like that but like I said I'm old fashioned & we wanted to set a good example for his son, now his Mother apparently didn't b/c eventually she moved a man in with them w/o being married, but that was her & not us.
Just food for thought, you hang in there & you do what you feel in your heart is the best for your kids!!