It is difficult not to take lack of intimacy personally - chin up, you seem to be able to see the bigger picture, and in the long haul, its the results that count, right?
Have you tried having a get-together for his colleagues - entertain them on your turf so H can see you as part of his circle?