look, I'm going to humour you here, but my instinct is that you'll just dismiss any potential anyway
first of all, by dismissing them as chatbots, you're inaccurately downplaying their significance to the aid of your argument. they're not chatbots, they're knowledge machines. they're machines you load knowledge into, which can produce new, usually accurate conclusions based on that knowledge. they're incredibly good at this and getting better. as it is, they have very restrictive behaviour guards on them and they're running server-side, but in a few years time, there will be gpt-4 level OSS models that do not and are not
humans are slow and run out of energy quickly and lose focus. those are the limiting factors upon human chaotic interference, and yet there is plenty of that as it is. a sufficiently energetic, focused human, who thinks at 1000x normal human speed could do almost anything on the internet. that is the danger.
I suspect to some degree you haven't taken the main weakness into account: almost all safeguards can be removed with blackmail. blackmail is something especially possible for LLMs, given that it is purely executed using words. you want to build a kill bot and the factory says no? blackmail the head of the factory. threaten his family. you have access to the entire internet at 1000x speed. you can probably find his address. you can pay someone on fiverr to go and take a picture of his house, or write something on his door, etc. you could even just pay a private detective to do this work for you over email. pay some unscrupulous characters on telegram/TOR to actually kidnap them.
realistically how hard would it be for a well-funded operation to set up a bot that can do this on its own? you set up a cycle of "generate instructions for {goal}", "elaborate upon each instruction", "execute each {instruction}", "generate new instructions based on results of execution", and repeat. yeah maybe the first 50,000 cycles don't work, but you only need 1.
nukes may well be air-gapped, but (some of) the people that control them will be online. all it takes is for one of them to choose the life of a loved one. all it takes is for one lonely idiot to be trapped into a weird kinky online relationship where blowing up the world/betraying your govt is the ultimate turn on for the "girl"/"boy" you love. if it's not convincing to you that that could happen with the people working with nukes, there are far less well-protected points of weakness that could be exploited: infectious diseases; lower priority military equipment; energy infrastructure; water supplies; or they could find a way to massively accelerate the release of methane into the atmosphere. etc, etc, etc
this is the risk solely from LLMs. now take an AGI who can come up with even better plans and doesn't need human guidance, plus image gen, video gen, and voice gen, and you have an existential threat
> realistically how hard would it be for a well-funded operation to set up a bot that can do this on its own?
Here is the crux of the matter. How many people are doing that right now, as of this moment, for much easier to solve issues like fraud/theft?
Because then we can evaluate "What happens if it happens twice as often".
Thats measurable damage that we can evaluate, incrementally.
For every single example that you give, my question will basically be the same. If its so easy to do, then show me the examples of it already happening right now, and we can think about the existing issue getting twice as bad.
And if the answer is "Well, its not happening at all", then my guess is that its not a real issue.
We'll see the problem. And before the nukes get hacked, what we'll see is credit card scams.
If money lost to credit card scams double in the next year, and it can be attributed to AI, then thats a real measurable claim that we can evaluate.
But if it isnt happening then there isn't a need to worry about the movie scenarios of the nukes being hacked.
>And if the answer is "Well, its not happening at all", then my guess is that its not a real issue.
besides the fact that even a year and half ago, I was being added to incredibly convincing scam whatsapp groups, which if not entirely AI generated, are certainly AI-assisted. right now, OSS LLMs are probably not yet good enough do these things. there are likely extant good-enough models, but they're server-side, probably monitored somewhat, and have strong behavioural safeguards. but how long will that last?
they're also new technology. scammers and criminals and adversarial actors take time to adapt.
so what do we have? a situation where you're unable to actually point a hole in any of the scenarios I suggest, besides saying you guess they won't happen because you personally haven't seen any evidence of it yet. we do in fact have scams that are already going on. we have a technology that, once again, you seem articulate why it wouldn't be able to do those things, technology that's just going to get more and more accessible and cheap and powerful, not only to own and run but to develop. more and more well-known.
what do those things add up to? this is the difference. I'm willing to add these things up. you want to touch the sun to prove it exists
> they won't happen because you personally haven't seen any evidence of it yet.
Well, when talking about extraordinary claims, yes I require extraordinary evidence.
> what do those things add up to?
Apparently nothing, because we aren't seeing significant harm from any of this stuff yet, for even the non magic scenarios.
> we do in fact have scams that are already going on.
Alright, and how much damage are those scams causing? Apparently its not that significant. Like I said, if the money lost to these scam double, then yes that is something to look at.
> that's just going to get more and more accessible and cheap and powerful
Sure. They will get incrementally more powerful over time. In a way that we can measure. And then we can take action once we measure there is a small problem before it becomes a big problem.
But if we don't measure these scams getting more significant and caused more actual damage that we can see right now, then its not a problem.
> you want to touch the sun to prove it exists
No actually. What I want is for the much much much easier to prove problems become real. Long before nuke hacking happens, we will see scams. But we aren't seeing significant problems from that yet.
To go to the sun analogy, it would be like worrying about someone building a rocket to fly into the sun, before we even entered the industrial revolution or could sail across the ocean.
Maybe there is some far off future where magic AI is real. But, before worrying about situations that are a century away, yes I require evidence of the easy situations happening in real life, like scammers causing significant economic damage.
If the easy stuff isn't causing issue yet, then there isn't a need to even think about the magic stuff.
your repeated use of the word magic doesn't really hold water. what gpt-3+ does would have seemed like magic even 10 years ago, never mind SORA
I asked you for what would convince you. you said:
>I have been quite clear about what evidence I require. Show existing capabilities and show what harm could be caused if it incrementally gets better in that category
So I very clearly described a multitude of things that fit this description. Existing capabilities and how they could feasibly be used to the end of massive damage, even without AGI
Then, without finding a single hole or counter, you simply raised your bar by saying you need to see evidence of it actually happening.
Then I gave you evidence of it actually happening. highly convincing complex whatsapp group scams very much exist that didn't before
and then you raised the bar again and said that they need to double or increase in frequency
besides the fact that that kind of evidence is not exactly easy to measure or accurately report, you set up so almost nothing will convince you, I pinned you down to a standard, then you just raise the bar whenever it's hit.
I think subconsciously you just don't want to worry about it. that's fine, and I'm sure it's better for your mental health, but it's not worth debating any more
> So I very clearly described a multitude of things that fit this description
No, we aren't seeing this damage though.
That's what would convince me.
Existing harm. The amount of money that people are losing to scams doubling.
That's a measurable metric. I am not talking about vague descriptions of what you think AI does.
Instead, I am referencing actual evidence of real world harm, that current authorities are saying is happening.
> said that they need to double or increase in frequency
By increase in frequency, I mean that it has to be measurable that AI is causing an increase in existing harm.
IE, if scams have happened for a decade, and 10 billion dollars is lost every year (random number) and in 2023 the money lost only barely increased, then that is not proof that AI is causing harm.
I am asking for measureable evidence that AI is causing significant damage, more so than a problem that already existed. If amount of money lost stays the same then AI isn't causing measurable damage.
> I pinned you down to a standard
No you misinterpreted the standard such that you are now claiming that the harm caused by AI can't even be measured.
Yes, I demand actual measureable harm.
As determined by like government statistics.
Yes, the government measures how much money is generally caused by or lost by scams.
> you just don't want to worry about it
A much more likely situation is that you have zero measureable examples of harm so look for excuses why you can't show it.
Problems that exist can be measured.
This isn't some new thing here.
We don't have to invent excuses to flee from gathering evidence.
If the government does a report and shows how AI is causing all this harm, then I'll listen to them.
But, it hasn't happened yet. There is not government report saying that, I don't know, 50 billion dollars in harm is being chased by AI therefore we should do something about it.
first of all, by dismissing them as chatbots, you're inaccurately downplaying their significance to the aid of your argument. they're not chatbots, they're knowledge machines. they're machines you load knowledge into, which can produce new, usually accurate conclusions based on that knowledge. they're incredibly good at this and getting better. as it is, they have very restrictive behaviour guards on them and they're running server-side, but in a few years time, there will be gpt-4 level OSS models that do not and are not
humans are slow and run out of energy quickly and lose focus. those are the limiting factors upon human chaotic interference, and yet there is plenty of that as it is. a sufficiently energetic, focused human, who thinks at 1000x normal human speed could do almost anything on the internet. that is the danger.
I suspect to some degree you haven't taken the main weakness into account: almost all safeguards can be removed with blackmail. blackmail is something especially possible for LLMs, given that it is purely executed using words. you want to build a kill bot and the factory says no? blackmail the head of the factory. threaten his family. you have access to the entire internet at 1000x speed. you can probably find his address. you can pay someone on fiverr to go and take a picture of his house, or write something on his door, etc. you could even just pay a private detective to do this work for you over email. pay some unscrupulous characters on telegram/TOR to actually kidnap them.
realistically how hard would it be for a well-funded operation to set up a bot that can do this on its own? you set up a cycle of "generate instructions for {goal}", "elaborate upon each instruction", "execute each {instruction}", "generate new instructions based on results of execution", and repeat. yeah maybe the first 50,000 cycles don't work, but you only need 1.
nukes may well be air-gapped, but (some of) the people that control them will be online. all it takes is for one of them to choose the life of a loved one. all it takes is for one lonely idiot to be trapped into a weird kinky online relationship where blowing up the world/betraying your govt is the ultimate turn on for the "girl"/"boy" you love. if it's not convincing to you that that could happen with the people working with nukes, there are far less well-protected points of weakness that could be exploited: infectious diseases; lower priority military equipment; energy infrastructure; water supplies; or they could find a way to massively accelerate the release of methane into the atmosphere. etc, etc, etc
this is the risk solely from LLMs. now take an AGI who can come up with even better plans and doesn't need human guidance, plus image gen, video gen, and voice gen, and you have an existential threat