You can't fight for a marriage while backpeddling.. etiher you want to TRY and EXPLORE reconcilliation or not.
if you want to try, you have steps we can help you with to end the affair, once that's done, the marriage repair can be explored. You cannot repair a marriage while an affair is taking place.
I can't quite tell if you are committed to exploring reconcilliation or not...
Again Chapter 1 of Divorce Remedy is an excellent resource to help you consider that.
It does sound like he may be afraid of exposure in the short term and the long term here. This is a lot of leverage for you if you want to explore ending his affair.