Is it possible that he is displaying defensive, or "protective" behavior? Maybe he was more high-drive earlier in your relationship, felt rejected by you, and has now placed himself into a defensive "I won't allow myself to feel that attracted to her again" mode, because he feels it hurts to much when he's rejected?
I'm guessing that the difference in the libidos earlier in the relationship might be greater than you're indicating, or at least maybe HE would say that it was.
