Write this up and read it 100 times a day. Read it until you dream about it.
Experience is something you don't get until just after you need it.
Now go read every thread people have started that have posted in your stitch.
Then when your done with this fear take some time out to think.
Think what will happen to your marriage if you let the affair continue. Think what will happen to your family if you let the affair continue. Think what will happen to you if you let the affair continue.