A good friend is a friend of the marriage. I say keep him as a friend if you can, but only as a friend of you and your H's. Insist that your H always be involved when this friend is at a gathering. And, stop confiding in him. Those thoughts and feelings should be reserved for your H or a close girlfriend.
I too have a close male friend that confided that he's in love with me a few months ago. He's engaged to my best friend. He said he's felt this way for years, etc, doesnt want to lose our friendship.. will support whatever decision I make. I finally had to cut off almost all private contact with him. I realized how much energy our relationship was taking from our significant others. And, it wasnt fair to have that with him when we've promised to remain emotionally and physically faithful to someone else.