Glam, Silent Cheerleader has posted some excellent thoughts. I also think that your h can't let the past go because he needs that excuse to hold on to so that he doesn't have to recommit fully. He's using that as a scapegoat in case things do not work out. It's his lifeline to the other side, so to speak.
If he raises the issue again, just say, "I'm really sorry you feel that way and I hope that some day you can let the past go and forgive me. I have forgiven you."
Please try not to worry about this too much. It's something you have no control over and until he completely crosses over to the rational side of life, he will hold on to this one excuse for dear life. I do hope that he will leave it all behind and soon. He really has a wonderful wife and family that want him home completely and ready to recommit to the marriage.
Glam, take are of yourself and please allow God to work on him.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.