I guess it's an "assumption", but it's an assumption that's directly challenged in the article:
> But of course we don’t actually care directly about performance on next-token prediction. The models already have humans beat on this loss function. We want to find out whether these scaling curves on next-token prediction actually correspond to true progress towards generality.
And:
> Why is it impressive that a model trained on internet text full of random facts happens to have a lot of random facts memorized? And why does that in any way indicate intelligence or creativity?
And:
> So it’s not even worth asking yet whether scaling will continue to work - we don’t even seem to have evidence that scaling has worked so far.
The conclusion that AGI will happen in 2040 is what I’m arguing against. I think 4020 is maybe a better estimate.
I don’t feel like we’re anywhere close given that we can’t even yet meaningfully define reasoning or consciousness… or as another commenter put it, what is it that differentiates us so significantly from other animals.
> But of course we don’t actually care directly about performance on next-token prediction. The models already have humans beat on this loss function. We want to find out whether these scaling curves on next-token prediction actually correspond to true progress towards generality.
And:
> Why is it impressive that a model trained on internet text full of random facts happens to have a lot of random facts memorized? And why does that in any way indicate intelligence or creativity?
And:
> So it’s not even worth asking yet whether scaling will continue to work - we don’t even seem to have evidence that scaling has worked so far.