> One piece of evidence that LLMs form some sort of internal reasoning mode
Isn't this a fancy way of saying "there's a scoring mechanism" like when you brute-force train a model, and certain chains/patterns of tokens score higher than others?
Like in my mind, it can learn English by you showing it a bunch of known-good sentences, telling it they score highly, and bad sentences score poorly. Noun verb structure and all that. Am I greatly oversimplifying it? I feel like I am.
In chess positions, a similar sequence of moves starting from a slightly different position would result in a completely different position. For example, imagine playing 10 moves identically in two games, but in one of them, 20 moves ago, a bishop moved into some corner that causes a piece to be pinned. This means that the right move would be completely different between these two scenarios despite the last 10 moves being the same.
To play well, the model would need to keep track of where the pieces are and be able to "reason" about the situation to some extent.
I've also tested it with some rare and unorthodox openings. For example, after 1. f3, there are only a handful of games on chessgames.com (which has over a million games), and after several more moves it's easy to get a position where even large online sites like chess.com and lichess.org don't have any games at all. Even in such cases, parrotchess was able to navigate it fairly decently despite never having ever seen that position before.
I believe you would be correct to call it a scoring mechanism, but just because it can score a sentence that doesn't mean it understands what the sentence actually said. So it isn't "learning English" as much as it is knowing which tokens would be the least wrong. "Happy" could be followed with "Birthday", "Anniversary", or "New Year" but is far less likely to be followed by "Funeral" or "Wake". Except it doesn't understand that the token 66 is the word "Happy", just that it is usually followed by tokens 789, 264, or 699 (contrived example, not real token IDs).
Humans can then interpret what that actually means in English and deem it correct or incorrect and adjust the weights if it's too incorrect. But at no point does the LLM learn any language other than token math.
> But at no point does the LLM learn any language other than token math.
Then can we make the argument that in the next 3 years, we aren't really due for any major AI breakthroughs and the current LLMs are about as good as they're going to get / they are kind of already at the limits of what they can do?
What would more GPU power, more time to training models, more tokens, more resources for high context size do if at the end of the day it's a random-nonsense generator with a really good scoring mechanism? How much more robust/advanced can you make it?
Take the context of replacing people with AI, if you can't trust the AI to be reliable + find a way to make it as accurate as humans, is there really an upcoming "AI revolution" to come?
LLMs are a pretty big breakthrough which can be useful for some applications, but not others. There is always room for futher breakthroughs, and as more time is devoted to research the LLMs of the future could be highly optimized for any number of metrics from power usage to token count.
So with that in mind, while it isn't a silver bullet which is guaranteed to lead to AGI, it can certainly fill a role in many other places and is bound to lead to cool tech layered on top of better LLMs. It's nowhere near robust enough to replace every human, but might be enough to at least displace some.
What would one of those breakthroughs hypothetically look like, if it's not an LLM (which is a robust trained model with a complex scoring system)
> So with that in mind, while it isn't a silver bullet which is guaranteed to lead to AGI
Could you give me an example on what AGI looks like/means to you specifically? People say like "oh, once AGI is here, it'll automate some tasks humans do today away and free them up to do other things"
Can you think of a task off the top of your head that is potentially realistically ripe for automation through AGI? I can't.
If I were to speculate about breakthroughs in LLMs, in another comment I have been discussing the addition of some kind of "conscience LLM" which acts as an internal dialog so an LLM can have its initial output to a question kind of "thought about" in a back and forth manner (similar to a human debating if they want soup or salad in their head). That inner LLM could be added for safety (to prevent encouraging suicide, or similar) or for accuracy (where a very purpose-trained smaller LLM could ensure output aligns with any requirement) or even as a way to quickly change the performance of an LLM without retraining -- just swap the "conscience" LLM to something different. I'd be surprised if this sort of "middle man" LLM isn't in use in some project already, but even if LLMs themselves don't have a major breakthrough they are still useful tools for certain applications.
>Could you give me an example on what AGI looks like/means to you specifically?
What I consider AGI would be a system which actually understands the input and output it is working with. Not just "knows how to work with it" but rather if I am discussing a topic with an AGI it has a theory of mind regarding that topic. A system that can think around a topic, draw parallels to potentially unrelated topics, but synthesize how they actually do relate to help generate novel hypotheses. For me, it's a pretty high bar that I don't forsee LLMs reaching alone. Such a system could actually respond if you ask it "How do you know that?" and it can explain each step without losing context after too many extra questions. LLMs could be a part of that system, in the same way it takes multiple systems for humans to be able to speak.
>Can you think of a task off the top of your head that is potentially realistically ripe for automation through AGI?
Automation isn't the only possible usage for AGI. Of course a crontab entry won't be able to think, but I have seen current industry uses for "AI" that help with tasks humans find tedious such as SIEM ticket monitoring and interpreting syslogs in realtime to keep outages as minimal as they can. Such a system would not meet my requirements to be an AGI, but would still be very useful even if it does not have any true intelligence.
> A system that can think around a topic, draw parallels to potentially unrelated topics, but synthesize how they actually do relate to help generate novel hypotheses
But it doesn't understand what it is interacting with. It only knows token math. Token math is a shortcut, not a true knowing of a subject. So if I ask what thought process it used to reach that conclusion, it can't enumerate why it chose that way other than "math said to".
Despite it being statistically improbable for it to have seen every chess position before, it nonetheless plays very strongly.