I think there’s a huge assumption here that more LLM will lead to AGI.
Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI.
LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI.
LLMs don’t “reason” in any sense of the word that I understand and I think the ability to reason is table stakes for AGI.
> I think there’s a huge assumption here that more LLM will lead to AGI.
I'm not sure you realize this, but that is literally what this article was written to explore!
I feel like you just autocompleted what you believe about large language models in this thread, rather than engaging with the article.
Engagement might look like "I hold the skeptic position because of X, Y, and Z, but I see that the other position has some really good, hard-to-answer points."
Instead, we just got the first thing that came to your mind talking about AI.
I'm not sure you realize this, but that is literally what this article was written to explore!
Yeah but it's "exploration" answers all the reasonable objections by just extrapolating vague "smartness" (EDITED [1]). "LLMs seem smart, more data will make 'em smarter..."
If apparent intelligence were the only measure of where things are going, we could be certain GPT-5 or whatever would reach AGI. But I don't many people think that's the case.
The various critics of LLMs like Gary Marcus make the point that while LLMs increase in ability each iteration, they continue to be weak in particular areas.
My favorite measure is "query intelligence" versus "task accomplishment intelligence". Current "AI" (deep learning/transformers/etc) systems are great at query intelligence but don't seem to scale in their "task accomplishment intelligence" at the same rate. (Notice "baby AGI", ChatGPT+self-talk, fail to produce actual task intelligence).
[1] Edited, original "seemed remarkably unenlightening. Lots of generalities, on-the-one-hand-on-the-other descriptions". Actually, reading more closely the article does raise good objections - but still doesn't answer them well imo.
I’ve also heard it said that “apparent” intelligence is good enough to be called “real” intelligence if it’s indistinguishable from the real thing. That’s where I have a strong feeling that we’re missing the true meaning of intelligence, reasoning and consciousness.
As you said, we may very well be a couple iterations away from a chatbot that is truly indistinguishable from a human, but I still strongly assert that even a perfectly coherent chatbot is nothing more than an automaton and we humans are not automatons.
The fact that a couple replies in this thread made me feel defensive and a bit discouraged with their condescending tone is to me an internal reaction that an LLM or similar system will never have. Maybe an appropriate emotional reaction can be calculated and simulated, but I think the nature of the experience itself is truly beyond our current comprehension.
Maybe I’m grasping at the metaphysical to rationalize my fear that we’re on the cusp of understanding consciousness… and it turns out to be pretty boring and will be included with Microsoft O365 in a couple years.
I agree with you, but I think it's more of a philosophical topic (ie. Chinese Room argument) than something that technicians working on raw LLM capabilities usually care to engage in. For them, the Turing Test and utility in applications are the most important thing.
Personally, I don't think we can construct an equivalent intelligence to a human out of silicon. That's not say AGI is unachievable or that it can't surpass human intelligence and be superficially undistinguishable from a human, but it will always be different and alien in some way. I believe our intelligence is fundamentally closer to other earth animals descended from common genetic ancestors than it can be to an artificial intelligence. As the creators of AI, we can and will paper over these differences enough to Get The Job Done™, but the uncanny valley will always be there if you know where to look.
I'm sorry I hurt your feelings, it wasn't my intention. For what its worth, I actually think there is a good chance that you are right - that there is something missing in LLMs that still won't be present in bigger LLMs. I mostly meant that an LLM would be more organized around the source material and address specific points.
I actually asked ChatGPT 4 to do so, and it produced the sort of reasonable but unremarkable stuff I've come to expect from it.
The point I was trying to make is that I think better LLMs won’t lead to AGI. The article focused on the mechanics and technology, but I feel that’s missing the point.
The point being, AGI is not going to be a direct outcome of LLM development, regardless of the efficiency or volume of data.
I can interpret this in a couple different ways, and I want to make sure I am engaging with what you said, and not with what I thought you said.
> I think better LLMs won’t lead to AGI.
Does this mean you believe that the Transformer architecture won't be an eventual part of AGI? (possibly true, though I wouldn't bet on it)
Does this mean that you see no path for GPT-4 to become an AGI if we just leave it alone sitting on its server? I could certainly agree with that.
Does this mean that something like large language models will not be used for their ability to model the world, or plan, or even just complete patterns as does our own System one in an eventual AGI architecture? I would have a lot more trouble agreeing with that.
In general, it seems like these sequence modelers that actually work right is a big primitive we didn't have in 2016 and they certainly seem to me as an important step. Something that will carry us far past human-level, whatever that means for textual tasks.
To bring it back to the article, probably pure scale isn't quite the secret sauce, but it's a good 80-90% and the rest will come from the increased interest, the shear number of human-level intelligences now working on this problem.
Too bad we haven't scaled safety nearly as fast though!
Yes, I suppose my assertion is that LLMs may be a step toward our understanding of what is required to create AGI. But, the technology (the algorithms) will not be part of the eventual solution.
Having said that, I do agree that LLMs will be transformative technology. As important perhaps as the transistor or the wheel.
I think LLMs will accelerate our ability as a species to solve problems even more than the calculator, computer or internet has.
I think the boost in human capability provided by LLMs will help us more rapidly discover the true nature of reasoning, intelligence and consciousness.
But, like the wheel, transistor, calculator, computer and internet; I feel strongly that LLMs will prove to be just another tool and not a foundational technology for AGI.
We do have systems that reason. Prolog comes to mind. It's a niche tool, used in isolated cases by relatively few people. I think that the other candidates are similar: proof assistants, physics simulators, computational chemistry and biology workflows, CAD, etc.
When we get to the point where LLMs are able to invoke these tools for a user, even if that user has no knowledge of them, and are able to translate the results of that reasoning back into the user's context... That'll start to smell like AGI.
The other piece, I think, is going to be improved cataloging of human reasoning. If you can ask a question and get the answer that a specialist who died fifty years ago would've given you because that specialist was a heavy AI user and so their specialty was available for query... That'll also start to smell like AGI.
The foundations have been there for 30 years, LLMs are the paint job, the door handles, and the windows.
> We do have systems that reason. Prolog comes to mind. It's a niche tool, used in isolated cases by relatively few people. I think that the other candidates are similar: proof assistants, physics simulators, computational chemistry and biology workflows, CAD, etc.
I think OP meant other definition of reason, because by your definition calculator can also reason. These are tools created by humans, that help them to reason about stuff by offloading calculations for some of the tasks. They do not reason on their own and they can't extrapolate. They are expert systems.
If an expert system is not reasoning, and a statistical apparatus like an LLM is not reasoning, then I think the only definition that remains is the rather antiquated one which defines reason as that capability which makes humans unique and separates us from animals.
I don't think it's likely to be a helpful one in this case.
I think he wants "reasoning" to include coming up with rules and not just following rules. Humans can reason by trying to figure out rules for systems and then see if those rules work well, on large scale that is called the scientific method but all humans do that on a small scale, especially as kids.
For a system to be able to solve the same classes of problems human can solve it would need to be able to invent their own rules just like humans can.
I think that is what I mean by reason. I set the bar for reasoning and AGI pretty high.
Though, I will admit, a system that acts in a way that’s indistinguishable from a human will be awful hard to classify as anything but AGI.
Maybe I’m conflating AGI and consciousness, though given that we don’t understand consciousness and there’s no clear definition of AGI, maybe they ought to be inclusive of each other until we can figure out how to differentiate them.
Still, one interesting outcome, I think, should consciousness be included in the definition of AGI, is that LLMs are deterministic, which, if conscious, would (maybe) eliminate the notion of free will.
I feel like this whole exercise may end up representing a tiny, microscopic scratch on the surface of what it will actually take to build AGI. It feels like we’re extrapolating the capabilities of LLMs far too easily from capable chat bots to full on artificial beings.
We humans are great at imagining the future, but not so good at estimating how long it will take to get there.
Reasoning, in the context of artificial intelligence and cognitive sciences, can be seen as the process of drawing inferences or making decisions based on available information. This doesn't make machines like calculators or LLMs equivalent to human reasoning, but it does suggest they engage in some form of reasoning.
Expert systems, for instance, use a set of if-then rules derived from human expertise to make decisions in specific domains. This is a form of deductive reasoning, albeit limited and highly structured. They don't 'understand' in a human sense but operate within a framework of logic provided by humans.
LLMs, on the other hand, use statistical methods to generate responses based on patterns learned from vast amounts of data. This isn't reasoning in the traditional philosophical sense, but it's a kind of probabilistic reasoning. They can infer, locally generalize, and even 'extrapolate' to some extent within the bounds of their training data. However, this is not the same as human extrapolation, which often involves creativity and a deep understanding of context.
Ya i feel like this issue is people think an LLM will someday "wake up" no, LLM's will just be multimodal and developed to use tools, and a software ecosystem around it will end up using the LLM to reason how to execute, basically the LLM will be the internal monologue of whatever the AGI looks like.
Agreed. I think it's more likely that we'll reach a point where their complexity is so great that no single person can usefully reason about their outputs in relation to their structure.
Not so much a them waking up as an us falling asleep.
I guess it's an "assumption", but it's an assumption that's directly challenged in the article:
> But of course we don’t actually care directly about performance on next-token prediction. The models already have humans beat on this loss function. We want to find out whether these scaling curves on next-token prediction actually correspond to true progress towards generality.
And:
> Why is it impressive that a model trained on internet text full of random facts happens to have a lot of random facts memorized? And why does that in any way indicate intelligence or creativity?
And:
> So it’s not even worth asking yet whether scaling will continue to work - we don’t even seem to have evidence that scaling has worked so far.
The conclusion that AGI will happen in 2040 is what I’m arguing against. I think 4020 is maybe a better estimate.
I don’t feel like we’re anywhere close given that we can’t even yet meaningfully define reasoning or consciousness… or as another commenter put it, what is it that differentiates us so significantly from other animals.
I really don't think there's an explanation there. All Sutskever says is the idea is to ask a LLM to be the smartest being on the planet and it magically happens.
An LLM is simply a model which given a sequence, predicts the rest of the sequence.
You can accurately describe any AGI or reasoning problem as an open domain sequence modeling problem. It is not an unreasonable hypothesis that brains evolved to solve a similar sequence modeling problem.
> It is not an unreasonable hypothesis that brains evolved to solve a similar sequence modeling problem.
The real world is random, requires making decisions on incomplete information in situations that have never happened before. The real world is not a sequence of tokens.
Consciousness requires instincts in order to prioritize the endless streams of information. One thing people dont want to accept about any AI is that humans always have to tell it WHAT to think about. Our base reptilian brains are the core driver behind all behavior. AI cannot learn that
How do our base reptilian brains reason? We don't know the specifics, but unless it's magic, then it's determined by some kind of logic. I doubt that logic is so unique that it can't eventually be reproduced in computers.
Reptiles didn't use language tokens, that's for sure. We don't have reptilian brains anyway, it's just that part of our brain architecture evolved from a common ancestor. The stuff that might function somewhat similar to an LLM is most likely in the neocortex. But that's for neuroscientists to figure out, not computer scientists. Whatever the case is, it had to have evolved. LLMs are intelligently designed by us, so we should be a little cautious in making that analogy.
"Consciousness requires instincts in order to prioritize the endless streams of information. "
What if "instinct" is also just (pretrained) model weight?
The human brain is very complex and far from understood and definitely does NOT work like a LLM. But it likely shares some core concepts. Neuronal networks were inspired by brain synapses after all.
> What if "instinct" is also just (pretrained) model weight?
Sure - then it will take the same amount of energy to train as our reptilian and higher brains took. That means trillions of real life experiences over millions of years.
Not at all, it took life hundreds of millions of years to develop brains that could work with language, and took us tens of thousands of years to develop languages and writing and universal literacy. Now computers can print it, visually read it, speech-to-text transcribe it, write/create/generate it coherently, text-to-speech output it, translate between languages, rewrite in different styles, explain other writings, and that only took - well, roughly one human lifetime since computers became a thing.
Information is a loaded word. Sure, you can say that based on our physical theories, you can think of the world that way, but information is what's meaningful to us amongst all the noise of the world. Meaningful for goals like survival and reproduction from our ancestors. Nervous systems evolved to help animals decide what's important to focus on. It's not a premade data set, the brain makes it meaningful in context of it's environment.
It depends on the goal, epicycles don't tell you about the nature of heavenly bodies - but they do let you keep an accurate calendar for a reasonable definition of accurate. I'm not sure whether I need deep understanding of intelligence to gain economic benefit from AI.
My first answer was a bit hasty, let me try again;
We are clearly a product of our past experience (in LLMs this is called our datasets). If you go back to the beginning of our experiences, there is little identity, consciousness, or ability to reason. These things are learned indirectly, (in LLMs this is called an emergent property). We don't learn indiscriminately, evolved instinct, social pressure and culture guide and bias our data consumption (in LLMs this is called our weights).
I can't think of any other way our minds could work, on some level they must function like a LLM, Language perhaps supplemented with general Data, but the principle being the same. Every new idea has been an abstraction or supposition of someones current dataset, which is why technological and general societal advancement has not been linear but closer to exponential.
Genes encode a ton of behaviors, you can't just ignore that. Tabula rasa doesn't exist among humans.
> If you go back to the beginning of our experiences, there is little identity, consciousness, or ability to reason.
That is because babies brains aren't properly developed. There is nothing preventing a fully conscious being from being born, you see that among animals etc. A newborn foal is a fully functional animal for example. Genes encode the ability to move around, identify objects, follow other beings, collision avoidance etc.
>Genes encode a ton of behaviors, you can't just ignore that.
I'm not ignoring that, I'm just saying that in LLMs we call these things weights. And i don't want to downplay the importance of weights, its probably a significant difference between us and other hominids.
But even if you considered some behaviors to be more akin to the server or interface or preprocess in LLMs it still wouldn't detract from the fact that the vast majority of the things that make us autonomous logical sentient beings come about through a process that is very similar to the core workings of LLMs. I'm also not saying that all animal brains function like LLMs, though that's an interesting thought to consider.
Look at a year old baby, there is no logic, no reasoning, no real consciousness, just basic algorithms and data input ports. It takes ten years of data sets before these emergent properties start to develop, and another ten years before anything of value can be output.
I strongly disagree. Kids, even infants, show a remarkable degree of sophistication in relation to an LLM.
I admit that humans don’t progress much behaviorally, outside of intellect, past our teen years; we’re very instinct driven.
But still, I think even very young children have a spark that’s something far beyond rote token generation.
I think it’s typical human hubris (and clever marketing) to believe that we can invent AGI in less than 100 years when it took nature millions of years to develop.
Until we understand consciousness, we won’t be able to replicate it and we’re a very long way from that leap.
Humans are not very smart, individually, and over a single lifetime. We become smart as a species in tens of millennia of gathering experience and sharing it through language.
What LLMs learn is exactly the diff between primitive humans and us. It's such a huge jump a human alone can't make it. If we were smarter we should have figured out the germ theory of disease sooner, as we were dying from infections.
So don't praise the learning abilities of little children, without language and social support they would not develop very much. We develop not just by our DNA and direct experiences but also by assimilating past experiences through language. It's a huge cache of crystallized intelligence from the past, without which we would not rule this planet.
That's also why I agree LLMs are stalling because we can't quickly scale a few more orders of magnitude the organic text inputs. So there must the a different way to learn, and that is by putting AI in contact with environments and letting it do its own actions and learn from its mistakes just like us.
I believe humans are "just" contextual language and action models. We apply language to understand, reason and direct our actions. We are GPTs with better feedback from outside, and optimized for surviving in this environment. That explains why we need so few samples to learn, the hard work has been done by many previous generations, brains are fit for their own culture.
So the path forward will imply creating synthetic data, and then somehow evaluating the good from the bad. This will be task specific. For coding, we can execute tests. For math, we can use theorem provers to validate. But for chemistry we need simulations or labs. For physics, we need the particle accelerator to get feedback. But for games - we can just use the score - that's super easy, and already led to super-human level players like AlphaZero.
Each topic has its own slowness and cost. It will be a slow grind ahead. And it can't be any other way, AI and AGI are not magic. They must use the scientific method to make progress just like us.
Humans do more than just enhance predictive capabilities. It is also a very strong assumption that we are optimised for survival in many or all aspects (even unclear what that means). Some things could be totally incidental and not optimised. I find appeals to evolutionary optimisation very tricky and often fraught.
Have you ever met a baby? They're nothing like an LLM. For starters, they learn without using language. By one year old they've taught themselves to move around the physical world. They've started to learn cause and effect. They've learned where "they" end and "the rest of the world" begins. All an LLM has "learnt" is that some words are more likely to follow others.
You know i assumed that was true until right now. But I can't think of a single example of reason and intelligence existing without any form of language. Even insects have rudimentary language, and in fact reasoning and intelligence seem to scale with the complexity of language, both by species and within species.
Do slime mold have a language? Slime mold can learn and adapt to environments, so it is intelligent and can do rudimentary reasoning, but I doubt it communicates that information to other slime molds.
It is a very different kind of life form though so many things that applies to other complex being doesn't apply to them. Being a large single cell means that they learn by changing its proteins and other internals, very hard for us humans to reason about and understand since it is so alien compared to just having nerve cells with physical connections.
Not sure i would say a slime mold has reason and intelligence .. Or if i would then so does a river. Also i think that how it changes its proteins could be considered a language, without stretching the definition of language any more than we have already stretched the definition of reason and intelligence.
Why is a slime mold a river but a human isn't? Slime mold can predict temperature changes in its environment and react before it happens, that isn't something a river could do.
So your statement just seems to be your bias thinking that a slime mold couldn't possible do any reasoning. Cells are much smarter than most thinks.
Edit: Anyway, apparently slime molds can communicate what they learn by sharing those proteins. So they do have a language, it is like a primitive version of how human bodies cells communicate. So your point still stands, reasoning seems to go hand in hand with communication. If you can reason then it is worth it to share those conclusions with your friends and family.
They also taught slime molds to cross a bridge for food, and it learned to do it. Then they got the slime mold to tell other slime molds and now those also knew how to cross the bridge. It is pretty cool that slime molds can be that smart.
Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI.
LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI.
LLMs don’t “reason” in any sense of the word that I understand and I think the ability to reason is table stakes for AGI.