My H always gets lost. When he was younger I used to help out, but after he got older he hated it. Same scenes as you. Finally I decided it was because he felt bad about himself. It embarrassed and frustrated him that he got lost. My "helping" just made him more aware of his own (perceived) failure.
I started to say nothing. I told him, if you need my help, just ask. Then I sat back and enjoyed the ride, even if it was a bit long.
