Gwyn, Okay, I am not a guy, but I thought I'd throw in my 2 cents.
Your H's life with OW was a self delusion, a trick way to feel good about himself. Kudos to him for waking up and working his way back to reality. With you, he can be his real self, and right now he is facing his mid life concerns head on. He may be feeling depresed as he adjusts to a new life transition. I bet what he is looking for now is something real and something connected and loving with you. He may sense your mixed feelings about reconciliation and be holding back a side of himself that he may want to share, now that he's not transfixed by fantasy. I think you have a chance at real intimacy here. I know you are hurt and confused as well, but I wonder what would happen if you took steps to embrace the man he is today, giving him time to work through the sexual aspects.