One of the first points in DB is that people think they know the future - just because something isn't a certain way now (or vice versa) doesn't mean it will be that way in the future. Just because he's not interested now doesn't mean he won't be in the future. Furthermore, don't let his feelings dictate yours. He IS being the martyr - and he must feel like he gave up a whole lot. Just because it was all fantasy doesn't change his experience of loss. Try and find some compassion - even if you want a D, just see him as a human who has lost something dear to him.
He's maturing, and either he's going to have to accept a lot of truths about marriage, or he's going to pursue this fantasy he has throughout his life with many partners. It's his choice, one you can't impact. It seems like he's chosen the former. He may not understand or accept all of the implications yet, but he's started down that path. Give him time to get his bearings, to observe his surroundings and make his own determinations - he is exploring new territory, and that's likely why he is defensive when you talk to him. He has given up a way of viewing the world and has to find a new one - one that will likely match yours more closely - but he seems to need to do this on his own before he can talk to you about it. But I think there's something you can learn from his experience - I think there's something you can add to your view of marriage that came out of his experience. You seem very focused on the sex aspect, so maybe this is an area that you want to grow in. Maybe you can explore this side of yourself and bring more excitement into the marriage (because that's the "love" feeling in an affair - the excitement) by changing your habits. But not too soon, and not necessarily with him initially.
He wants things to improve, otherwise he wouldn't be here. What's the saying about believing none of what you hear and half of what you see?
“No problem can be solved from the same level of consciousness that created it. ” – Albert Einstein
