Yes, it is a mistake. You do not move back in with him UNLESS he has done the hard work to FIX HIS CHRONIC INFIDELITY PROBLEM and offered you TOTAL TRANSPARENCY (access to phone records etc. )
This guy has been lying and/or cheating since your pregnancy. He moves away from one problem woman and instantly finds another.
I don't care WHAT your issues are (and yes, we all benefit here from taking a good hard look at ourselves and fixing our part)but there is NOTHING you could have been doing that explains away his behavior. He's not a one-time slip-up, he's a chronic cheater. And I bet if you're honest with yourself, you'll admit that his past history before he met you was full of red flags - isn't it?