Barring the horror story of alter boys and such, priests and monks 'control' their sex drives for most of their lives.
I realize you didn't choose to be a priest or a monk, but that does not negate the fact that it can be controlled. What you are talking about is much akin to an addict or someone who is just taking on any new discipline... the moment you decide to go on a diet, you are hungry. The moment you quit smoking, all you want is a cigarette, etc., etc., etc.
And... let's face it... you can have sex whenever you want it. It's just not the kind of sex you desire... so really, what you are talking about is not sex... it's about being able to control another's desire... which is out of your realm of control. Which drives you nuts.
So really, you don't have a 'sex' problem. You have a 'control' problem.