He is very remorseful and has done everything text book to help me reconcile the M, but I won't let him in.
This may sound silly, but, only you can let him in.
Sounds like you guys need a few rules by which to work this out. "After The Affair" by Janis Springs is a book that comes to mind. It may be helpful, because it explains to the WAS what the LBS is going through, and vice versa. You both are reacting in very normal ways. But you're both in a different mind set right now, and the key is understanding the other partner.
One of the "rules" for example is based on the LBS's need to know things about the affair. The WAS doesn't wish to discuss it, because he's remorseful and it serves up guilt (which you know WASs run from), and pain, for him and for you, and he doesn't want either of you to experience more pain.
Yet, as questions come up in your mind, you feel you need answers. The WAS however, doesn't wish to be barraged with questions out of the blue all the time. So a solution is to form a weekly session that lasts maybe one hour, in which you ask questions, he answers in full. At the end of the hour, no more questions are asked until the next session. This satisfies both partner's needs to a degree. When being told answers, be prepared for strong emotions to arise, you may be haunted by the answers. You cannot argue, blame (he gets defensive, right?), bringing up past grudges, straying off topic with new hurts, get angry, et al, otherwise the WAS will start to omit and lie and refuse to answer anymore. Eventually, your questions will subside as you gain an understanding of how this happened and the attention starts to focus on how to prevent this in the future, by making those changes that were needed to repair the relationship.
It may behoove you guys to attend MC sessions together, where an impartial counselor can offer suggestions, advice, help you both get beyond any obstacles, offer new perspectives, etc. It may be better than going it alone, as you seem to be stuck right now by your own devices.
The probable reason that you can't move forward also is due to your dwelling in the past. As long as you harbor anger and resentment over the affair, it will be difficult to move forward. And as long as you dwell on the past events, you will feel its associated hurt, anger and pain. Forgiveness is key. People make mistakes, big mistakes (and don't confuse mistakes with having to be unintended accidents), and there's hurt always associated with it, every time you think about it. Yet the mistake is now in the past, and so should the hurt be. The pain is now in the past too, but by reliving it in your mind, you give yourself misery. The pain couldn't be avoided, but misery is optional. Animals experience an event that causes pain, they suffer through it 100%, and it's in the past. Humans experience an event that serves up pain, they experience it 100% and then proceed to cling on to it and live it over again and again and experience it 10,000%. It doesn't have to be like that, because we have minds beyond the animals that while permitting us to experience misery, also permit us to deflect it and move forward.
The answers to your question cannot be contained in this post or any one post. You work with your counselor to let the past go, you work on yourself. You are the instrument to your own happiness.
Your anger and frustration are what's leading you to question whether you want to work on the relationship or not. It's better to not base decisions on your emotions.
You have "now" to work with, not the past, and if H is showing by his actions that he is working very hard to rebuild with you, that's the positive ground on which to heal. Work with what is happening now.
