Oh there IS good evidence for temperaments being largely genetic. But the environment plays a large role in shaping how that turns out.
Like you...something shaped your inheritance from Grandma into someone who guards their down time, but is not so extreme.
I think it is really, really good to recognize these idiosyncrasies in ourselves AND others, especially our S's. This REALLY helps with not personalizing everything so much!
I agree with KAW, for a "borderline...whatever!" your H is really, really showing that he doesn't WANT to be the recluse in a cabin! He is open to R stuff, it just does not come naturally to him.
Believe it or not, although I tend towards extraversion I can get "peopled out" too.