Red, hon, just hopped on and got caught up. {{hugs}}
OK, here goes: You have trust issues with him. For good reason. And they need to be dealt with. (Coming from someone slogging hip-deep through trust issues )
That said...reading through all these posts with fresh eyes, I think the advice to work on yourself is REALLY important. I'm talking about more than just GAL and trying to be the perfect Stepford Wife so he won't want to leave. I'm talking about doing the hard work of finding and learning to LOVE yourself for who you are now, today.
Hon, you've got to let go of thinking that if you're enough for him he'll be faithful. His infidelity had/has NOTHING to do with you or your past actions. It was his choice, his weakness, his mistake. Sure, relational patterns have an effect, but his straying was NOT YOUR FAULT. (Have you considered looking into the possibility of sexual-addiction? The pattern fits: the As, the current 'lack of interest', living on the surface and saying everything is 'fine'.)
I understand the fear. I also know that acting out of fear isn't going to bring you the results you want. And it's a lot easier said than done. But whether you end up together or apart, you need to be at peace with who you are.