I'm certain what you are going through is 100% normal. I've about a lot of false starts in recon... picture it as you are dipping your toe in the shallow end of the pool and not jumping all in in the deep end.
Try reading The Love Path.
Also you need to be building more positive interactions that create more bonding and intimacy rather than rehashing the same old same old.
How are you with date nights?
Can you plan something you've never done before??? Go to one if those bars where you throw axes, indoor rock climbing, etc.
Recon will only work if you truly forgive your partner. So for now work on building intimacy and let the affair stuff ride a bit... it will be easier for both of you to discuss the A once you've established more intimacy.
Pull backs would be normal if he is dealing with guilt and shame and those are posing as your road blocks. Perhaps he feels the pressure of you needing all these answers from him when he may just not have them right now.
Ultimately it's up to you... do you invest further because you believe in a stronger M bond and future or are you exhausted and over it???? Again, this is your timeline so take all the time you need.
