Tripod thanks for the responses. It would be easier for you to "surender" if your sense of masculinity isn't tied completely to sex. It seems to me that a lot of guys tie their sexual desire directly in their sense of their own masculinity and to "back off" can be a threat to their sense of self.
Edit re your last post have you heard of the passion paradox?