Me, as a HD wife, founded myself in the same situation you were with your H, but reversed. My husband and I had a fight some time ago ( before we were in the current worse situation) where he was flabbergasted that I screamed at him that nothing have changed in the sex department and that I was not complaining because I didn't want to make matters worse by being unappreciative of his efforts, that I thought they were "baby steps" going in the right direction, so I didn't want to mess that up. Unfortunately those "baby steps" were a tremendous effort for my H. He was so offended that his efforts were not being recognized that he just stopped trying for a while.
My point is, he thought that his attempts at meeting me halfway were successful, but he was not meeting me halfway at all, and THAT was the problem. An example in frequency: I told him I would be soo happy if we could have sex about 6-7 times a week, which is my ideal, but I knew he just wouldn't bring himself to have sex more than maybe once a week, so that once a week would be our goal. I, unfortunately did not stated my real middle ground, which was 2 to 3 times a week. He thought that because he "made" himself to have sex with me once a week most times (it was more like 3 times a month on average really), that that was meeting me halfway..well, not hardly. My mistake was to agree with something I was not happy with to begin with and then see him "miss" some of the weekly sex dates we had. Why did I do such thing? because I didn't want to preassure my husband to give me what he was not ready to give and keep on complaining all the time when he didn't meet me in the 2-3 times range. I just thought that the once a week was much more workable for my H and I could work from that base, later, when the regularity was set and he was ready for a little upgrade to more, like the 2 or 3 times a week once in a while. Once a week was going to be an OK minimum. Well, I was very mistaken.
Maybe your H did this too, agreed to a very minimum, as a way to be loving and not pressure you over what you felt confortable. And if you add another aspect that is very important for a HD spouse, the lack of passion on those encounters, it just kills you. I am not saying you showed lack of passion, however think about it, maybe sometimes you did. My H did and that really counted in my head as a -1 against our minimun count. Maybe unfair, but it felt like that in my heart, it felt like I was being punished, not loved.
I did focus on 2 aspect of my problem : frequency and passion to write to you, the truth is there are many more, but these are the most clear in my mind, and I hope that maybe they help you to understand what could be going on in your husband's.