Well, I sympathize with you very much - it is a tough spot. From my viewpoint, it's difficult for me to disagree with your husband completely since my wife is engaged in an affair with a "friend" who she now loves very deeply and all of my fears about her were realized when she chose not to quit seeing him even before the affair started. In our case, I began to see signs that she was getting too close to him, then I started to resist that relationship when he blatantly started inviting her to concerts and "out" without inviting me. So, I can see where your husband might feel very insecure because in my case my suspicions were dead-on and my very real fears were realized.
In my wife's case, she has a history of sexual molestation from childhood (pretty severe one too, involving her sister's fiance and later, husband). Her behavior has always been tinged with sexual overtones, it seemed to me, and there were times when I got upset with her for being too flirtatious or too affectionate with male friends, especially in front of our kids. Being in the "swinging" lifestyle didn't help, obviously, as those aspects of her personality were celebrated and enhanced, and she eventually began to embrace that part of herself as "the real her" rather than seeing it as potentially a danger to our kids and our marriage.
Now, it's true that our case is a fairly unique one, pretty extreme...but didn't you say you were engaged in sex therapy? Were you also molested as a child, or in some other way sexually compromised in your life? Our marriage counselor (who we saw 6 times when the marriage was initially breaking down) told us the incidence of infidelity and divorce among molested women is very high - that a "switch is turned" in their brains, in a way, that makes sex a very different thing for them...intimacy through sex is removed and sex becomes a game, or a fun outlet for lust, but does not hold the intimate connective quality more common among women who did not experience sexual molestation. By and large, molestation victims become promiscuous and very often seek validation from many men, etc. This has always been one of my chief fears about my wife, and I have often acted to "head her off" in the past because I perceived this in her and it scared me, and also because I was afraid the kids would be exposed to dangers that would cause them the same harm she was caused. So, I acted perhaps overbearingly and out of fear to reduce the influences of anything even slightly suggestive to our kids, especially if it was behavior from her that suggested a sexual overtone of ANY kind, even toilet humor and body-function stuff, like farts. I was pretty serious about it and to hear her tell it, hard to live with.
The bottom line is I think maybe if you are similar to my wife, your husband may have fears that cannot be fully assuaged if you tend towards these sorts of behaviors, especially given the comment you made about the kids. I totally understand where you are coming from, and I know you meant no harm...but it is hard for a man to separate those things. Even my wife knows I "meant no harm" by my hardline approach to things but nevertheless, "the best of intentions" STILL led to her choosing to get out and have an affair. In your case, at least you are still having a dialogue and hopefully no permanent harm has been done. I think you have every right to maintain your friendships, but I also believe that in a marriage, anything you do should have vigorous agreement from your spouse before you do it. I'm not saying give up your friend; far from it. But you may want to follow through on asking him to tone down sexual innuendos and indicate that your marriage is important to you and that those things cause your husband discomfort. I think if Bret is REALLY your friend he will understand and will really make that change. Only the profoundly selfish among us (unfortunately my wife is currently one of those people, as is her "friend") choose to place their wants and desires above their commitments, in my opinion. There is room for a certain amount of selfishness, but if you approach your marriage that way all the time, you may as well not be married. The integrity of commitment is directly proportional to the integrity of the people making the commitment; if your commitment is jeopardized by choices that challenge the commitment then you have to evaluate your choices.
I don't think your pact with your friend was silly, but consider when in your life you made it, and under what circumstances. Did you really have the best frame of mind to be making such a pact? And if your husband takes those sorts of things seriously, he may see it as something dangerous because to him that was a sort of commitment. By calling it silly he may be trying to minimize it's importance to reassure himself that you wouldn't stick to such a commitment - but he can't be sure since you are tenacious in your will to hang on to the friendship. In my case the affair relationship is in my opinion the sole reason that makes reconciliation with my wife impossible; so, I can never trust her and no real trust or commitment can grow between us as long as she hangs on to that relationship. In our case, her involvement with him must cease on every level in order for us to be happily married as far as I am concerned. Perhaps in your case it won't be necessary to take it that far, but at the very least you might consider at least toning down the level of involvement for a while and take the kinds of actions that show your husband you are truly sensitive to his concerns.
