And it isn't about him being with the baby. It's the fact that he is still so tangled up w HWW. And until he untangles from that... Or if he does... I don't want anything to do with it.
Sweetie, they will always be "tangled" up because of their link as the baby's parents. Accept this as the reality. You cannot just wish this away, honey. Perhaps learn some radical acceptance of this reality which probably will solve a of your internal crazymaking chit.
I promise you that with this radical acceptance, then you'll truly and truly be detached from whatever moves XH makes such as staying at BIL's or at HWW's. What's the differnce? Just a change of location.