IMO they know what they are doing is wrong - and don't care, or have to try to believe they don't. As my H put it: "to you and the rest of the world it is wrong, but it is all right in my head - and I don't care what anyone thinks."
My guess is he acknowledged your neighbor and simply stated what he believed to be true.
I think those of them that feel/express guilt almost need others to outwardly disapprove, so he may have been fishing for confirmation that neighbor felt that way.
"Do not look back in anger or forward in fear, but around in awareness." - James Thurber
