You’re right.
Sometimes I just want to travel to a volcanic island in the middle of the ocean to have a nice conversation with my old pal, npc #347. Then I get there and his hair color is brown instead of blond, AND he changed his clothes. It breaks my heart as well as my immersion :’(