You did well. It would have been worrisome if he had wanted to stay, as your reaction suggests that you would have let him. I don't think that would have been helpful at this point. Slow and repeat light happy times is much better than jumping too fast and having someone get cold feet. You want him to come back to stay, and him waking up the next morning feeling trapped or whatever those feelings are floating around in his skull and running would be bad.
Little bites and stepping away with both wanting more is exactly what you want. You did that. You should not read his invisibility as anything other than expected. It is a pattern of behavior you are looking for as evidence and no one event (or its absence).
You also may want to plan for a GAL activity for the day after an activity w/ your H. Helps with the let-down of withdrawal. But you can also mention what you have going on, and have something to cut him a bit short and leave him wanting more if he should ask you to do something the next day. He realizes that if he wants that, he's got to recognize what he has done to your M.
Glad it went well & you all had fun.
Me: 50 W:43 S6, S3 M: 12 yrs. T: 17 M is bad & Not happy Bomb Mar '14 S 5 Feb '15 D Bomb 13 Apr '15 (but "no hurry") DB Coach May '15 Wants proceed on D Aug '15 Starting 1-on-1 negotiations Sept '15