How can he have enough time to take care of kids while he is busy at work and meeting the OW?
He can't. Any any good divorce/custody agreement is going to give you first-right-of-refusal on any days or nights that he cannot be with them ... you have first choice to have the kids yourself, before he can hire a babysitter.
He's just talking tough, because:
a) he thinks you don't know what your legal rights are (DO you?); and
b) he thinks you'll never enforce your boundaries.
