You asked for honesty, so here it goes. I agree with Nevanna!
With regards to his relationship with her, there’s really nothing you can do – except walk if you feel it is continuing in an inappropriate manner. “No contact” letters or e-mails, even though they’re very popular with the BS for obvious reasons, don’t mean squat unless the WA backs them. There’s nothing to stop the WA from calling the OP five minutes later and saying “sorry about that letter/e-mail/phone call, my wife forced me at gunpoint and I had to go along to preserve peace at home. You know I didn’t mean it.” In fact, if you read “Other person” boards, that seems to be the modus operandi for most affair couples.
In other words – the relationship will be over on his timeline. Maybe it really is just a friendship, and he’s simply too soft-hearted to say “don’t contact me at all anymore”. Maybe he’s hoping it’ll just fizzle out and die a natural death – which, by the way, also happens quite frequently.
Focus on yourself and your R with him is my advice.