Bob...its an answer that in the long run will be evident. Most people who end up swapping and swinging feel just like you do right now...that it actually enhanced their love and relationship, so how can it be bad? The problem is that in the long run, it tends to deteriorate and no matter how many rules you put into place, you will always see your spouse differently (and she you) than you did before this all started happening.
I've just known several people who fooled themselves into thinking they could handle it due to the initial excitment, but who in the long run regretted it and lost their marriages over it.
Please find me one example of a married couple who have been able to continue that lifestyle, and I'll show you that that couple is a total exception, and that they truly don't have the type of bond you can have with monogamy anyway, and that they are both breaking their own stated rules and are cheating on their spouses (by breaking the rules).
Sorry, if you really do the research you'll find this is true....but if you only scratch the surface, you'll find couples who are saying they love it, it works for them, it makes them closer, etc.
I just urge you to fully research it and try to find other couples who have remained married for many many years and who are not actually cheating on their own rules....I will bet you cannot find any. And that should be the evidence you need.
I'm sorry if I sound like I'm scolding...I've just seen a lot of pain in my years from situations like yours.