100% this. Exponential progress is only a thing if potential progress is infinite. If the potential progress is finite (hint: it is), eventually the rate of progress hit progressively diminishing returns.
The underlying rule here, in my opinion, is the law of diminishing returns. (log shaped curve)
AlphaZero is already capable of optimizing itself in the limited problem space of Chess.
Infinitely increasing the computing power of this system won't give it properties it does not already have, there is no singularity point to be found ahead.
And I am not sure that there are any singularities lying ahead in any other domains with the current approach of ML/AI.
And building on that, the real bottleneck in most domains isn't going to be computer power, it's going to be human understanding of how to curate data or tweak parameters.
We've already seen in games with simple rules and win conditions that giving computers data on what we think are good human games can make them perform worse than not giving them data. Most problems aren't possible for humans to ebncapsulate perfectly in a set of rules and win conditions to just leave the processing power to fill in the details, and whilst curating data and calibrating learning processes is an area we've improved hugely on to get where we are with ML, it's not something where human knowledge seems more likely to reach an inflection point than hit diminishing returns.
