Oh, heck... I was afraid of this. From your opening post I thought you might be a young newlywed, but I held back because if I was wrong it'd be embarassing for me, and maybe offensive for you. That being said, I take back my advice - sharing the book might not be a great thing at this stage.
Forgive me, I've been married 25 years, 3 kids (20, 18 and 12) and have been SSM for a VERY long time. It kind of colors your outlook.
For him, I think maybe he's wondering if he got a bum deal, although if the SSM problems began before marriage, or even before engagement, then he maybe had his eyes open wider than I did. For me, the sex thing seemed to change immediately after the wedding day, and it was like day... and night.
I do think you might get into a discussion with him, and let him know that his barbs do hurt, and maybe the two of you could have a non-blaming talk about a subject that is obviously very important to both of you. Maybe he doesn't know (or trust) that it's just as important to you as it is to him. Ask him if he's willing to work together with you on solutions. Let him know that you aren't asking him to live like a monk (and prove it).
All in all, you may also have to be willing to entertain the possibility that it isn't going to work out. Please DON'T have any kids until you get this sorted out. I'm praying for you!