etb, congratulations for choosing not to be the OW.
I agree with questioning the whole "just friends" idea of having a R with this guy. Even if your intentions are honourable, it would be enabling him. I do believe that not all EAs are two way streets, but certainly can lead to that and then some.
I think it is great that his interest in you has helped you feel good and desirable. I hate to say that he is probably projecting his desires on you. While that wouldn't be a bad thing or make how you feel about his interest a bad thing, it is probably also brought on by what he feels he's missing in his M.