Ever the eternal optimist, I think it's probably a good thing that he admitted to you his feelings, that it effected him. It seems if he was trying to conceal something from you, he wouldn't admit to that. Also, it may just be he is keeping his musings to himself to avoid upsetting you.
I know about those walls. Mine are made of Titanium.