I think fundamentally it is that. The ability to retain information.
Like given a specific task it can do a thing amazingly well, but can it recall a thing. Its memory seems like a giant filing cabinet and it has to go scan like 20 million tokens worth of memory to recover things previously talked about.
Human memory is more graph like, we don’t recall things exactly, but one thing links to another, we create a pattern of a thing, we mark what is important, and overtime what was important degrades or becomes less so.
I feel like what makes it lack intelligence is it never seems to learn. Like it kind of does, but then doesn’t persist once too many other things are learned.
I’m sure they’re probably working on this, but I feel like that is what I want far more than even better models, is a better memory system to recall and forget things that the models work on.
Like given a specific task it can do a thing amazingly well, but can it recall a thing. Its memory seems like a giant filing cabinet and it has to go scan like 20 million tokens worth of memory to recover things previously talked about.
Human memory is more graph like, we don’t recall things exactly, but one thing links to another, we create a pattern of a thing, we mark what is important, and overtime what was important degrades or becomes less so.
I feel like what makes it lack intelligence is it never seems to learn. Like it kind of does, but then doesn’t persist once too many other things are learned.
I’m sure they’re probably working on this, but I feel like that is what I want far more than even better models, is a better memory system to recall and forget things that the models work on.