My H has been pushing/threatening me to move near him. He did this first at bomb drop on OW1 and now is doing it again with OW2 (whom he has never acknowledged). I think he starts future faking a life with these women and then freaks out that we won't be there with him on the sidelines.
I have thought about the points you raise, but in my case it is not enough for me to chuck my life and upend my s (D will be away in NY). I wouldn't cross the street to be with the guy he is now.
