Sounds like you two had a great convo together over the meal. I wouldn't read too much into his distance and grumpiness - as you put it, alot or most of it likely had to do with his current SA struggle. I'm actually glad to see it.
His guilt shows some possible growing empathy and care for you. I'm glad you stopped yourself from analyzing too much.