I can understand being in contact everyday for the sake of the kids, but having him hang out at your house is going to need to stop for you to move forward.