But is this really about AI? Almost all aspects of our lives are already controlled by the enormously complex soulless machines of bureaucracy we call firms and governments. Is there any essential difference between a computer bug and a clerical error? If anything, I would assume the former is easier to fix.
But that’s not how it works, it’s not one in place of another; it’s one that depends on another.
If your physical bureaucratic entity delegates decision making to an AI, you have no hope of redress unless due process was mandated.
If you cut the bureaucratic middleman then you have to build an AI that can question itself and correct errors.
AI will never do that. So what you get instead is negligence because a computer makes a mathematical computation and the human element treats it as infallble.
I agree. I think the real question (which is much broader and beyond the scope of OP) is how does humanity have to operate differently with every order of magnitude it grows in size? Humanity seems to be transitioning from single digit billions to crossing the double digit billions barrier in the foreseeable future. Until 1900 humanity had been on the order of a billion people, and we are currently transitioning towards 10x as many people sustaining the planet.
I do not mean this to be about finite resources and our planet. I am referring to the way that organizations require more layers of bureaucracy as they grow. A rule of thumb is for every 10x in people, you have another layer of management. So what will this extra layer of management be for people, not just from a government perspective but also a capitalistic corporation perspective? How do Google, Facebook, Apple, Stripe, Amazon, and all of those other companies handle so many customers? How much do they use automation, how many of their rules are archaic, how hard is it to challenge the system if you think there was an error?
I am not convinced AI can solve many of the fundamental problems humanity are facing - and I think overpopulation might actually be the problem. Modern capitalism has conditioned us to believe "growth is good" but growth for the sake of growth is no different from cancer.
The whole purpose of humans inside bureaucracy is to provide humanity, to subvert/override the systems when that makes sense. If they don't do that, and instead take wage as a motorized application-stanping arm, then they are failing horribly at their job.
