I think you need to see a counselor. I see this situation as similar in some ways to Nwlywed's. I can tell there is real hurt there, and the story seems almost too incredible to believe, so there is something, and I have no idea what, that is being missed by you. Now don't confuse that with "you're doing something wrong". I'm not saying that. I just think if you can share with someone experienced in marital relationships, they should be able to give you more guidance.
Maybe I missed it, but how long have you been together?
Good luck, keep sharing, it will give everyone more insight.