> edit (addition): What % of people can hold 25,000 words worth of information in their heads, while effectively reasoning with and manipulating it? I'm guessing maybe 10% at most, probably fewer. And they're probably the best in their fields. Now a computer has that ability. And anyone that has $20 for the OpenAI api can access it. This could get wild.
It's true that most humans cannot do this, but loading words and contexts into your working memory is not the same as intelligence. LLMs excel at this kind of task, but an expert in a field such as medicine, isn't loading an entire medical report into their working memory and then making decisions or creating new ideas using that information. There are other unsolved aspects to our intelligence that are not captured by LLMs, that are still required to be an expert in some field, like medicine.
Still an incredible leap forward in AI technology, but I disagree with the implication that the best experts in a field are simply loading words from some text and reasoning with and manipulating it.
The comparison between the context length and what humans can hold in their heads just seems faulty.
I'm not sure I can agree that humans cannot hold 25,000 words worth of information in their heads. For the average person, if they read 25,000 words, which can be done in a single sitting, they're not going to remember all of it, for sure, but they would get a lot out of it that they could effectively reason with and manipulate.
Not to mention that humans don't need to hold the entire report in their head because they can hold it in their hand and look at it.
And if anything, I think it's more significant to have a bigger working memory for GPT's own outputs than it is for the inputs. Humans often take time to reflect on issues, and we like to jot down our thoughts, particularly if it involves complex reasoning. Giving something long, careful thought allow us to reason much better.
It's true that most humans cannot do this, but loading words and contexts into your working memory is not the same as intelligence. LLMs excel at this kind of task, but an expert in a field such as medicine, isn't loading an entire medical report into their working memory and then making decisions or creating new ideas using that information. There are other unsolved aspects to our intelligence that are not captured by LLMs, that are still required to be an expert in some field, like medicine.
Still an incredible leap forward in AI technology, but I disagree with the implication that the best experts in a field are simply loading words from some text and reasoning with and manipulating it.