I have young kids also and one of the first things I did was put in the divorce decree no other people non related can spend the night when children are present. May be something to think about for you. It applies to both my H and myself. My H agreed on that issue. Good luck!
W-38 H-42 T-11 M-8 C-6,2,6 months BD-Oct 1 2013 DFiled-Jan 6 2014 Went Dark - April 4, 2014