Exactly! You guys are going to need to amp up the communication and time spent face to face. Not all at once of course but gradual shifts will get you there. As you said before you are no longer just in stage one, I think the invite on Sunday might be the push you need to get to stage 2. But just remember to not have too high of hopes as he might be busy or just not feel like going. You have to steel yourself against the rejection and not take it personally should it come your way.
If he accepts, fantastic. If not, there will be another opening down the line. So no stress either way!
