He was carrying his coffee. I can just imagine. Such a familiar sight, I'm sure.
He doesn't want to see you because of his guilt. His note proves that.
In what you've written the past few days, it's clear that you both love each other deeply. Work on showing him the joy you can have despite all this and... you never know what may happen.
My unsolicited advice is to not contact him again for a while. It will be much better for you this way. (fullmoon said it best.)