If you ask him out - remember - have no expectations.
Even if he says Yes - don't expect that it will necessarily turn out to be a nice time.
I recall early on in my sitch asking H to lunch at 'our' restaurant for a birthday.
I was surprised when he agreed, but half-way through what was otherwise a 'normal' lunch, H launched into a monologue about how i'd have to give him our family car and buy myself a new one.
