Frankly, I don't "get" your H. If I had a W who let me squeeze her tush, I'd be all over that like white on rice.
As far as the issue with the "working 5 minutes to get him aroused" goes, I wouldn't dwell on it, unless it becomes a common occurrence. Some times, especially as one gets older, you just need a little persuading.