I think one of the biggest challenges in rebuilding a marriage is learning that the past is the past. It can't be changed. The present and the future can however. Often though, if we can't move beyond the past, we can't ever rebuild because we can't ever forgive. While we need to learn from the past, we can't live in it, or we will stifle any good feelings we could have, close ourselves off to them, and essentially, nothing can grow.
You seem to me to be really angry right now about the past. Rightfully so. However, if this man loves you like it seems he does, you may be refusing to allow yourself to ever love him again or feel that love because of that pain of the past. Are you still going to IC? If so, I would encourage you to speak to the counselor about this anger and this resentment and letting the past go and forgiving and learning to live in the here and now. It is the hardest thing I have done in any of this. I do not have my marriage anymore. However, one thing I think constantly about is in the past when I didn't forgive and now when my H won't so he won't reopen his heart, really the person we hurt most was not spouse but it was ourself, because we denied ourselves happiness, love, and joy and instead made a choice not to forgive, to be angry, and to resent.
I hope that doesn't come off to harsh. I do think it is something worth considering....
Very little is needed to make a happy life; it is all within yourself, in your way of thinking. -Marcus Aurelius