I think what an LLM is best at is fooling people into thinking it’s intelligent. It is really good at saying things in a natural sounding way, and statistically often getting it right, because certain strings of tokens are encoded. But it’s clear when you start poking that it just as easily tells you that 5/2=3, or that 2+2 != 4. it doesn’t model math or any sort of knowledge at all.
Something that I don't quite understand is why the tendency of ChatGPT to be inaccurate sometimes is a fundamental flaw rather than something that can be improved on iteratively if its just a matter of improving the statistical likelihood of accuracy. The question of whether its AGI or not is, to paraphrase the famous quote, a bit like the question of whether a submarine can swim.
Because ChatGPT isn't thinking. It's not reasoning at all. It's assembling sentences that are statistically predicted from using existing writings as the template.
Accuracy isn't a part of the process except in terms of how accurate the training data is. ChatGPT is not making any sort of truth or accuracy determination, let alone doing so poorly.
We don’t have any proof of that, as we don’t have proof also about its opposite. We have no idea why neural nets work, or how our brain works in context of this. There is definitely something human-like in neural networks, but we don’t have any idea why, and what exactly. It’s a completely empirical field and not a theoretical one. We don’t have any good idea what would happen if we could built a 180 billion neuron large neural net, because there is no theory which would prove what would happen even the current ones. That’s why I’ve seen almost every single prediction about what AI would solve in the following years in the past 40 years fail. We have no clue.
My point isn't about how good or bad this is being done. Humans, at least some of the time, attempt to assess truth and accuracy of things. LLMs do not attempt to do this.
That's why I think it's incorrect to say they're bad at it. Even attempting it isn't in their behavior set.
Isn’t this the whole point of John Searl’s “the Chinese room” thought experiment? But does it matter what is actually going on inside the room, if the effect and function is indistinguishable? Edit: after conferring with ChatGPT, Searle’s point like yours is that the man in the room doesn’t understand Chinese, he is just manipulating symbols, but from the outside, the man in the room seems to speak fluent Chinese.
I think a better analogy is asking if a water bottle can swim. It floats most of the time, and can move around if pushed.
The reason “can be inaccurate sometimes” is a fundamental flaw is because my assumption is that it will never not be inaccurate. I think it will always be inaccurate sometimes and never be accurate always.
This doesn’t mean it isn’t useful for a lot of applications. But I don’t think it is a holy grail technology, it’s not AGI, and it isn’t going to replace professions.
The whole point of using a computer instead of doing something yourself is to do something quickly and accurately. If I need someone to give me maybe-correct-maybe-not information, I'll just ask one of my coworkers.
Well, that was the point of computers until now. That doesn't mean computers can't be other things, too. ChatGPT is a lot cheaper and faster than your coworker, and it's available (almost) 24 hours a day. And the accuracy may improve!
I mean only if you want accurate information, but if you're building a misinformation network of bots to cause problems in an enemy state then a human sounding bullshit machine sounds like something any number of governments would buy into.
The best feature of this LLM is that it goes from fooling people into having people make fools of themselves when they turn around and predict the end of the world/education/programming/whatever thing they don't quite understand based on what a confidently incorrect charlatan machine told them. It's like a viral marketing gag.
