Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I don't understand your objection at all.

In the example, the 'world' is the grid state. Obviously that's much simpler than the real world but the point is to show that even when the model is not directly trained to input/output this world state it is still learned as a side effect of prediction the next token.



There is no world. The grid state is not a world, there is no causal relationship between the grid state and the board. No one in this debate denies that NNs approximate functions. Since a game is just a discrete function, no one denies an NN can approximate it. Showing this is entirely irrelevant and shows a profound misunderstanding of what's at issue.

The whole debate is about whether surface patterns in measurement data can be reversed by NNs to describe their generating process, ie., the world. If the "data" isnt actual measurements of the world, no one arguing about it.

If there is no gap between the generating algorithm and the samples, eg., between a "circle" and "the points on a circle" -- then there is no "world model" to learn. The world is the data. To learn "the points on a cirlce" is to learn the cirlce.

By taking cases where "the world" and "the data" are the same object (in the limit of all samples), you're just showing that NNs model data. That's already obvious, no ones arguing about it.

That a NN can approximate a discrete function does not mean it can do science.

The whole issue is that the cause of pixel distributions is not in those distributions. A model of pixel patterns is just a model of pixel patterns, not of the objects which cause those patterns. A TV is not made out of pixels.

The "debate" insofar as there is one, is just some researchers being profoundly confused about what measurement data is: measurements are not their targets, and so no model of data is a model of the target. A model of data is just "surface statistics" in the sense that these statistics describe measurements, not what caused them.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: