I am having trouble going dark when he is still partially staying at our house, and spending every other weekend here when he has his son.. should J stop doing things with him and the kids together? Should I leave on the weekends he's here? Should I avoid him the other times during the week when he shows up?