> Surely the AI doomers are overstating the risk of doom - surely no-one working in safety critical systems would do things any other way? "Has there ever been a case of harm form AI misalignment?" - this is the real thrust of that question. What sort of idiot would wire an unintended consequence generator directly to anything that might harm or hurt?
You don't have to look far to find people doing exactly that. AutoGPT, ChaosGPT (!), a lot of random people copying its output into a Python prompt.
"It's not safety-critical", you might say, but that's only because the AI isn't smart enough yet. A human-level AI could easily do a lot of damage this way, and I don't think we'll learn our lesson until it's already happened. Here's hoping it happens before we get superhuman AI!
...for people playing Russian Roulette, the temptation to take pulling the trigger and surviving as evidence that it is safe to pull the trigger again appears irresistible.
You don't have to look far to find people doing exactly that. AutoGPT, ChaosGPT (!), a lot of random people copying its output into a Python prompt.
"It's not safety-critical", you might say, but that's only because the AI isn't smart enough yet. A human-level AI could easily do a lot of damage this way, and I don't think we'll learn our lesson until it's already happened. Here's hoping it happens before we get superhuman AI!