Did your husband agree to no-contact and full transparency as part of your reconciliation? Counseling, preferably with someone specifically trained in dealing with infidelity?
That is key. Without it, you're never going to feel safe, and he's at a VERY high risk to repeat the infidelity.