Well, now you know not to suggest doing things with the 3 of you, let him bring it up, eh?
As for listening - I think Al Turtle has some stuff on his site, if you google his name it should be the first thing that comes up.
If he broaches the idea of deciding what S should do for the summer, could you say, "that's a great idea, S would really like that, he recently said he was interested in XYZ (or leave this part out) what would you suggest?" The way you phrased your response above sounded more like a demand that he interpreted as pressure. Alternately, you could suggest that you will take S to, e.g., swimming, and is there anything H would like to take him to?