Do you always have to be right?

Does your H have to be literal to you, but you are allowed linguistic and contextual leeway with him?