If you do want your M back. If you are willing to get past your fears.

Then the next time your H puts out the feelers about "getting the family back", you may ask him what he is prepared to do about it. Ask him if he is prepared to go to counselling. Ask him if he needs more time.

And if he says he will go to counselling, ask him to find a pro-M counsellor.

But...

He WILL have to ditch the OW, eventually, if the M is to be saved. You may have to remind him of that.

And then...

keep on moving forward with your life and see if he takes this next step to the plate...