The time to have done it would have been when he asked back into the marriage and the home. "OK, but I'm afraid it's not that simple anymore. These are the things I would need," and then you let them know what your boundaries and dealbreakers are.
Were there ANY consequences for his waywardness? Or did you simply take him back?
