I don't think you should ignore the baby - I have had 5! I just think the baby might be better off with no father than with a father who realized two or three years in that he or she wasn't enough to glue two adults together.
Right now, if the baby is told, the worst case is that he'll know that you left a wife that was unfaithful (or lies about you).
If you can't stay with her, how can you stay with her and another man's baby?
