I think you should strongly consider exposing the affair to both his employer, and any elders in the congregation. Your husband is in a position of authority over others, including others' moral decisions, and they have a right to know the character of the man that they're putting in that position.
The fact that your husband has reacted so strongly to the possibility of that indicates just how strong of an impact such a bold move could have.
Do NOT do this if you'd merely be doing it out of vindictiveness, or if you don't feel you can handle the blowback (which will be considerable). Do this ONLY as a means to try to exert pressure to restore your marriage.
Exposure is controversial, and reasonable people disagree on both sides. But it's a proven technique that has worked for hundreds of thousands of couples, including me and my wife.