That's ok... he will have to deal with the reality of the affair all the sooner.
The affair isn't going to be FUN anymore for him... Its going to start getting very unpleasant for him and he wont like it... If you aren't around or being mean he WILL blame the OW and they will fight... It's nothing to worry about.