Since you are the one working, I would NOT move unless YOU want to. Not when the whole relationship is so dicey.
Sounds like he should be attending SLAA meetings (sex and love addicts anonymous). I have a friend who attends them, and she says there are lots of porn-addicted guys there. They are like AA meetings.
I agree with the others, no unprotected sex with him. Plus he needs to give you complete transparency - access to his cell phone records, computer passwords, etc.
Meanwhile - may I ask what your history with him sexually is about? Were you never very interested in sex, or is this a postpartum thing, or has his not-working turned you off? Does he have a history of depression, drug use, gambling or other addictions?
Also - if you are not sure you can trust him with the baby, have you considered getting one of those babycams to see what happens when you are gone?
