I'm so glad someone else used the word abusive...it's what was in my head, and I hesitated to say that. But yes, your H's behavior is abusive.
So, I kind of think you need to get clear on your boundaries. Essentially, are you willing to continue swinging to save your marriage? The answer is either yes or no. Be honest, because there IS a price to pay when we do things that don't fit what's good for us.
Second: are you willing to allow your H to continue swinging to save your marriage? Again, yes or no, and really think about all the consequences.
Fear...well, it keeps us from acting authentically all the time, but what happens when we act out of fear is that we harm ourselves.
If the answer to the two questions above is no, then I think you owe your H a clear, honest conversation:
"H, I am sorry that I led you to believe that swinging was okay with me and my personal beliefs. I did it to please you, because I was afraid that if I didn't, you'd leave. I now realize I was wrong to be dishonest and withhold my true feelings from you.
"I am not okay with swinging, and I am not okay with you continuing down that path. You say that you will end up resenting me, you say that you love me, but I wonder if you realize that I will end up hating and resenting you for expecting me to cross boundaries I am not willing to cross.
"This has nothing to do with self-esteem or social anxiety, it has to do with my expectations from a partner. I would prefer that we work on making our marriage fully satisfying, together, but I understand I cannot make you do anything. However, I choose not to live my life this way."
Clear...direct...honest.
Honey...you and I are close in age, and I finally reached the point where I was no longer willing to compromise just to save a relationship. Not when it was detrimental to our R, like my H's continued friendship with his one-sided (him) EA that impacted our M. I love my H, but I am completely willing to walk away if he contacts her again, period.
Your H is not treating you with love. He is being manipulative and pushing you to do things you aren't comfortable with. Is it worth it?
SD
Me: 40 H: 43 H had EA from 2/06-9/06 Bomb 5/06 Piecing since 9/2006 3/2008: Boundary setting 7/2009: Boundary crossing~dropped my own bomb. 8/2010: Marriage finally on track!