Happy belated birthday to your daughter. I'm sure she had a wonderful time.
As for your friend wanting to come visit...I'd be very careful about that. You are absolutely correct in saying that if you got in a situation w/him you'd be doing the exact same thing that your h is doing. You don't want to go down that road unless you are divorced and absolutely certain you are finished w/your h.
My thoughts are to remain friends, but I would not have him come to visit and hang out unless you did it in a group setting. Even though he's a friend, he's already thinking ahead and circling for a booty call. Think w/your head, not your heart when it comes to this situation.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.