It seems completely reasonable that if an organisation is going to use personal identifying information that they are responsible for correcting it when necessary. It's not good enough to just throw your hands up and claim that you can't alter it.

Maybe LLMs need to be not be fed tons of real life information if they are then going to produce non-correctable lies about real people. Something like an anonymising filter may be needed.