Cally, You have to decide whether this is a deal breaker or not. If it is, then you have to be willing to bring this topic up frequently and deal with whatever fallout there may be. If it leads to a fight, so be it. Etc. We stayed on a course of having The Talk about once per week for a year or so. Maybe more, I've lost track.
He decided the following: He wanted to stay married to me. He wanted to stay married to me so bad that he would tolerate these frequent talks. And, finally, he wanted to stay married to me so bad that he would agree to compromise on this issue and to take me seriously.
That's really it, in a nutshell.
If you are afraid to constantly bring this issue up because he might leave you, then I'd say that sex is the least of your worries, kwim?