There must be some men who just don't have as much testerostone. If that is the case here then it would seem to me that it would hurt his ego and make his sex drive even lower to think his wife wants it more than he does.
Is he happy in your marriage? Is there any possibility that he has been having an affair and getting it somewhere else?
Are the two of you close? Connected emotionally?
Has he had some sort of an illness that could have caused this?
I dont' mean to give you an interrogation here, sorry, but knowing some of these things can help people give you some feedback.