I am not a vet, T0, so take all of this with a grain of salt and get someone who knows what the heck their talking about before following my advice but I'll say this:
He is going to trickle truth you to death because he's afraid if you knew ALL OF IT, you wouldn't consider working on things which is what he wants right now. You need to think about what YOU want.
Do you want to work on this?
Are you willing to do it without ALL of the story or will him thinking he got away with partial truths be all you can think about?
The "I don't remember" line is complete BS. He knows every single detail and approximate time. You're a mother - you have GREAT instincts - trust them.
Do you need to see his phone? His email accounts? Is that permitted in DB?
To me, working on the marriage means owning up to your chit. You need to do that and so does he. Starting that off with lies by omission or "I don't remember" is him avoiding the issue at hand. He can't just walk back in and say "let's pretend this never happened, I'm here now". He needs to honor your pain, your wondering, your confusion, your anger, by being upfront if that is what YOU want. Your needs matter. Make them known.