I am a big fan of Chapman's the Five Languages of Love. Could the statement that she is the only friend he can talk about his relationship with his wife, be a cry to you that he needs to have some quality time and conversation with you?
She is NOT "just a friend" and you know it. Have you read NOT "Just Friends" by Shirley Glass? It's a great book for understanding infidelity.
But it does sound like your H craves quality time; he couldn't get it from you so that's what he gets from OW. Good knowledge to have once the A is busted.
I like Cutter's suggestion for a couple days out of town to create some mystery. If H needs "a friend" to confide in then let him see what it feels like when you find that friend also. I did similar things to let BF wonder just what I was up to and with whom.
If you love somebody, set them free. http://tinyurl.com/2empx2g