I would hold my ground and not apologize (remember, you did nothing wrong), but do it in a way so you do not appear a dick.