Whatever happens, keep that connection with your son. As he gets older he's going to into the past 5 range he's going to need a solid relationship with his dad. Guidance, unconditional love, teaching him stuff.
Do not let that father son relationship go. I promise you that your son will need his dad!