Sometimes leaving the town you are in is the only way to start fresh somewhere new. He's not crazy.
And look...he should be thankful that someone doesn't want to cross that line with him. I know it hurts. My wife is re-thinking our marriage and having an EA with my best friend. I started sharing this with an old college friend of mine, who is a woman, and who lives out of state. She's a Christian. She prayed for us, started sending me marriage-saving books, etc. Then she stopped returning my emails and calls. I got one last email from her saying she developed feelings for me and that she would break off all contact. I didn't share those feelings, but it was clear that she did the right thing. I emailed her back thanking her for her wisdom and told her I wouldn't call her or email her back. Ouch. A 23 year friendship down the drain. And I'm the one who is trying to save my marriage. Look...shit happens. Your husband shouldn't be surprised.
On the other hand, these people in your town (and his friends) need to know something about grace. He's repented of his adultery, he's put the other woman off. He's chosen to be with you. People should be running towards him and cheering him on!!! The father loves the prodigal when he returns. He throws a party. Maybe you can talk to his old friends and say that he needs encouragement and grace, not judgment.
