You bow out honestly...you just tell him that while you had a nice time and enjoyed his company that it didn't feel right to you and it made you realize that continuing would only lead to someone possibly getting hurt and you don't want that burden.
You see the wonderful weekend I spent with my "friend" after he flew clear across the continent to see me was very similar. As much as I wanted to believe things were over it was painfully obvious they weren't when he tried to kiss me good bye at the airport...I couldn't do it...hug him yes, kiss him no!
He went home and I didn't hear from him for a while...when I did he wasn't happy with me...said he could tell I wasn't over my H...but worse he didn't feel like I was really attracted to him either...so it ended up hurting his feelings...we remained in contact for a while...as just friends...but now I don't hear from him anymore...it makes me sad because he really was a nice man...a good friend...and he was there when I really needed him...he even flew at his expense to spend time with me!
But it was wrong...I was married (divorce wasn't final, just filed)...even though H could have cared less about it all (he knew)...I still wasn't over H...and as things turned out...we were not DONE...
So my motto became "Until one of us remarries...it isn't over"