The more and faster a “mind” can infer, the less it needs to store.
Think how much fewer facts a symbolic system that can perform calculus needs to store, vs. an algebraic, or just arithmetic system, to cover the same numerical problem solving space. Many orders of magnitude less.
The same goes for higher orders of reasoning. General or specific subject related.
And higher order reasoning vastly increases capabilities extending into new novel problem spaces.
I think model sizes may temporarily drop significantly, after every major architecture or training advance.
In the long run, “A circa 2025 maxed M3 Ultra Mac Studio is all you need!” (/h? /s? Time will tell.)
I don't know who else took notes by diffing their own assumptions with lectures / talks. There was a notion of what's really new compared to previous conceptual state, what adds new information.
The more and faster a “mind” can infer, the less it needs to store.
Think how much fewer facts a symbolic system that can perform calculus needs to store, vs. an algebraic, or just arithmetic system, to cover the same numerical problem solving space. Many orders of magnitude less.
The same goes for higher orders of reasoning. General or specific subject related.
And higher order reasoning vastly increases capabilities extending into new novel problem spaces.
I think model sizes may temporarily drop significantly, after every major architecture or training advance.
In the long run, “A circa 2025 maxed M3 Ultra Mac Studio is all you need!” (/h? /s? Time will tell.)