I think you should tell your wife that you've gone out with the OM's wife several times, and that, although you realize that you don't love the OM's wife she is going to have your baby. Let your wife know that you'd like to raise the child in your family.
OMG! The OM's W is pregnant with your child? And your W knows nothing?
It is not about what you feel should work in your M. It is about doing the work that gets the right results. Do what works!