What does it mean? I don't know. My theory is that somehow he thinks that if he says nice things, it cancels out his actions.
The other possibility is that he knows he has done wrong, but his ego and pride will never allow him to change his path. Saying those things lets him beat himself up, pay his penance, without actually doing anything.
But.... It doesn't matter! Detach your emotions from his drivel!
