I think you would find it very worthwhile to explore these issues with an individual counselor... truly, these are huge, life-altering questions that certainly can't be dealt with in any thorough way on a message board such as this one.
I think I do know what you mean about when you were dating, you really wanted to get the woman to come home with you. I have the feeling sometimes myself that the sex really isn't about sex, it's more about validation that someone would WANT to have sex with me. It's almost as if I know they WANT to, then we don't actually have to do it. I get the points, as it were, from knowing that he wants me.
I don't know if all married men think about having sex with other women. That's the popular idea, but I don't know if it's universal. When you see an attractive woman and think about having sex with her, do you think about the actual sex act, or do you think how cool it would be if you could get her attention and be so attractive or do all the "right things" so that she would want to have sex with you?