>Is is the cluster that is conscious? Or the individual machines, or the chips, or the algorithm, or something else?
The bound informational dynamic that supervenes on the activity of the individual units in the cluster. What people typically miss is that the algorithm when engaged in a computing substrate is not just inert symbols, but an active, potent causal/dynamical structure. Information flows as modulated signals to and from each component and these signals are integrated such that the characteristic property of the aggregate signal is maintained. This binding of signals by the active interplay of component signals from the distributed components realizes the singular identity. If there is consciousness here, it is in this construct.
Even if I take the more expansive possible interpretation of “body” typically applied to vehicles, the propeller on the back of it isn’t part of the “body” and the “body” of a submarine is rigid and immobile.
Is this an intellectual exercise for you or have you ever in your life heard someone say something like “the submarine swam through the water”? It’s so ridiculous I would be shocked to see it outside of a story intended for children or an obvious nonnative speaker of English.
>the propeller on the back of it isn’t part of the “body” and the “body” of a submarine is rigid and immobile.
That's a choice to limit the meaning of the term to the rigid/immobile parts of the external boundary of an object. It's not obviously the correct choice. Presumably you don't take issue with people saying planes fly. The issue of submarines swimming seems analogous.
>Is this an intellectual exercise for you or have you ever in your life heard someone say something like “the submarine swam through the water”?
I don't think I've ever had a discussion about submarines with anyone, outside of the OceanGate disaster. But this whole approach to the issue seems misguided. With terms like this we should ask what the purpose behind the term is, i.e. it's intension (the concept), not the incidental extension of the term (the collection of things it applies to at some point in time). When we refer to something swimming, we mean that it is moving through water under its own power. The reference to "body" is incidental.
It's not really a "choice" to use words how they are commonly understood but a choice to do the opposite. The point of Dijkstra's example is you can slap some term on a fundamentally different phenomenon to liken it to something more familiar but it confuses rather than clarifies anything.
The point that "swim" is not very consistent with "fly" is true enough but not really helpful. It doesn't change the commonly understood meaning of "swim" to include spinning a propeller just because "fly" doesn't imply anything about the particular means used to achieve flight.
>It's not really a "choice" to use words how they are commonly understood but a choice to do the opposite.
I meant a collective choice. Words evolve because someone decides to expand their scope and others find it useful. The question here shouldn't be what do other people mean by a term but whether the expanded scope is clarifying or confusing.
The question of whether submarines swim is a trivial verbal dispute, nothing of substance turns on its resolution. But we shouldn't dismiss the question of whether computers think by reference to the triviality of submarines swimming. The question we need to ask is what work does the concept of thinking do and whether that work is or can be applied to computers. This is extremely relevant in the present day.
When we say someone thinks, we are attributing some space of behavioral capacities to that person. That is, a certain competence and robustness with managing complexity to achieve a goal. Such attributions may warrant a level of responsibility and autonomy that would not be warranted without it. A system that thinks can be trusted in a much wider range of circumstances than one that doesn't. That this level of competence has historically been exclusive to humans should not preclude this consideration. When some future AI does reach this level of competence, we should use terms like thinking and understanding as indicating this competence.
This sub thread started on the claim that regular, deterministic code is “thought.” I would submit that the difference between deterministic code and human thought are so big and obvious that it is doing nothing but confusing the issue to start insisting on this.
I'm not exactly sure what you mean by deterministic code but I do think there is an obvious distinction between typical code people write and what human minds do. The guy upthread is definitely wrong in thinking that, e.g. any search or minimax algorithm is thinking. But its important to understand what this distinction is so we can spot when it might no longer apply.
To make a long story short, the distinction is that typical programs don't operate on the semantic features of program state, just on the syntactical features. We assign a correspondence with the syntactical program features and their transformations to the real-world semantic features and logical transformations on them. The execution of the program then tells us the outcomes of the logical transformations applied to the relevant semantic features. We get meaning out of programs because of this analogical correspondence.
LLMs are a different computing paradigm because they now operate on semantic features of program state. Embedding vectors assign semantic features to syntactical structures of the vector space. Operations on these syntactical structures allow the program to engage with semantic features of program state directly. LLMs engage with the meaning of program state and alter its execution accordingly. It's still deterministic, but its a fundamentally more rich programming paradigm, one that bridges the gap between program state as syntactical structures and the meaning they represent. This is why I am optimistic that current or future LLMs should be considered properly thinking machines.
LLMs are not deterministic at all. The same input leads to different outputs at random. But I think there’s still the question if this process is more similar to thought or a Markov chain.
They are deterministic in the sense that the inference process scores every word in the vocabulary in a deterministic manner. This score map is then sampled from according to the temperature setting. Non-determinism is artificially injected for ergonomic reasons.
>But I think there’s still the question if this process is more similar to thought or a Markov chain.
It's definitely far from a Markov chain. Markov chains treat the past context as a single unit, an N-tuple that has no internal structure. The next state is indexed by this tuple. LLMs leverage the internal structure of the context which allows a large class of generalization that Markov chains necessarily miss.
This is a bad take. We didn't write the model, we wrote an algorithm that searches the space of models that conform to some high level constraints as specified by the stacked transformer architecture. But stacked transformers are a very general computational paradigm. The training aspect converges the parameters to a specific model that well reproduces the training data. But the computational circuits the model picks out are discovered, not programmed. The emergent structures realize new computational dynamics that we are mostly blind to. We are not the programmers of these models, rather we are their incubators.
As far as sentience is concerned, we can't say they aren't sentient because we don't know the computational structures these models realize, nor do we know the computational structures required for sentience.
However there is another big problem, this would require a blob of data in a file to be labelled as "alive" even if it's on a disk in a garbage dump with no cpu or gpu anywhere near it.
The inference software that would normally read from that file is also not alive, as it's literally very concise code that we wrote to traverse through that file.
So if the disk isn't alive, the file on it isn't alive, the inference software is not alive - then what are you saying is alive and thinking?
This is an overly reductive view of a fully trained LLM. You have identified the pieces, but you miss the whole. The inference code is like a circuit builder, it represents the high level matmuls and the potential paths for dataflow. The data blob as the fully converged model configures this circuit builder in the sense of specifying the exact pathways information flows through the system. But this isn't some inert formalism, this is an active, potent causal structure realized by the base computational substrate that is influencing and being influenced by the world. If anything is conscious here, it would be this structure. If the computational theory of mind is true, then there are some specific information dynamics that realize consciousness. Whether or not LLM training finds these structures is an open question.
> So if the disk isn't alive, the file on it isn't alive, the inference software is not alive - then what are you saying is alive and thinking?
“So if the severed head isn’t alive, the disembodied heart isn’t alive, the jar of blood we drained out isn’t alive - then what are you saying is alive and thinking?”
- Some silicon alien life forms somewhere debating whether the human life form they just disassembled could ever be alive and thinking
Just because you saw a "HA - He used an argument that I can compare to a dead human" does not make your argument strong - there are many differences from a file on a computer vs a murdered human that will never come back and think again.
What's silly about it? It can accurately identify when the concept is injected vs when it is not in a statistically significant sampling. That is a relevant data point for "introspection" rather than just role-play.
Modern brain imaging techniques also weigh in on this issue. Mental imagery corresponds to voluntary activation of the visual cortex[1]. The quality of the self-reported imagery corresponds to the degree of activity in the visual cortex[2] while imagining some visual scene. People with aphantasia have little to no visual cortex activity.
I've been experimenting over the past year or so, and keep trying to visualise things. Part of this spun out from the fact that I can dream (I rarely seem to, or at least remember them), but when I do, I remember that it was a vivid real thing.
I actually feel like I'm closer than ever to getting towards visualisation. I've gone from a rock solid "zero" to "solid feeling, occasional split-second flash of something"
For most of the time with this exercise I was aiming for something simple. A red triangle in a blue square, but I'm not convinced that was an effective approach, I seem to be getting closer to the mark trying to picture something real.
When I want to close my eyes and distract myself, I've been visualizing the banana from the cover of that Velvet Underground album. (Not sure why I settled on that!) I can rotate it. I can peel it. With practice it has gotten larger and I can shift it away from the centre of the visual field. But I can't make it seem yellow.
Why should LLM failures trump successes when determining if it thinks/understands? Yes, they have a lot of inhuman failure modes. But so what, they aren't human. Their training regimes are very dissimilar to ours and so we should expect alien failure modes owing to this. This doesn't strike me as good reason to think they don't understand anything in the face of examples that presumably demonstrate understanding.
Because there's no difference between a success and failure as far as an LLM is concerned. Nothing went wrong when the LLM produced a false statement. Nothing went right when the LLM produced a true statement.
It produced a statement. The lexical structure of the statement is highly congruent with its training data and the previous statements.
This argument is vacuous. Truth is always external to the system. Nothing goes wrong inside the human when he makes an unintentionally false claim. He is simply reporting on what he believes to be true. There are failures leading up to the human making a false claim. But the same can be said for the LLM in terms of insufficient training data.
>The lexical structure of the statement is highly congruent with its training data and the previous statements.
This doesn't accurately capture how LLMs work. LLMs have an ability to generalize that undermines the claim of their responses being "highly congruent with training data".
Ironically, one of the ways that humans are worse than AI, is that any given human learns from an even smaller fraction of that collective experience than AI already does.
That's because it's not trying to do so. The observation is that humans are broadly unable to prepare for the failure modes of other humans, even when those failure modes have been studied and the results of those studies widely published. This means that while the failure modes of humans are indeed different from the failure modes of LLMs (and AI more broadly), these differences are not what I anticipate to be the most important next step in AI research.
AI are better than most humans at dealing with human suckage, for example because unlike humans the LLMs have read all that literature about human suckage, but that's not relevant to what I was saying.
My point is: other failure of AI are more pressing. IMO the inefficiency with regard to examples, e.g. even cancelled/sold off self-driving car projects (Uber's ATG) have more miles of experience than a human professional driver can get in their entire career, and look how bad that model was.
Making a self driving car fail like a human means getting it distracted by something on the phone. Plus a bunch of other failure modes we should ignore like "drunk" and "tired".
Even if you don't fully solve the example inefficiency, merely improving it will make a big difference to performance.
>for example because unlike humans the LLMs have read all that literature about human suckage
No they haven't. If you read the cliff notes of a book, you haven't read that book. An LLM is a generalization over their entire training set, that's not what the word "reading" has ever meant.
The LLM does not "know" anything about human suckage or how to get around it, and will not use those "learnings" in it's "thinking", it will only come up if the right nodes in it's model trigger, and then it just generates tokens that match the "shape" of writing that was written with that knowledge.
A bloom filter can be used to test for presence of something in your DB, with configurable probability even (something that LLMs massively lack), but a bloom filter does not Know what is in your DB
When you fit a linear regression to a plot of free falling speed over time, you will have an equation for acceleration of gravity, but you don't "Know" gravity, and that equation will not allow you to recover actual generalizable models of gravity. That limited model will still get you most of the way to the moon though.
Generally the next claim is "same as human brains" but no, that has not been proven and is not a given. "Neural Networks" are named that way as marketing. They've never been an accurate simulation of actual animal neurons and a single animal neuron has far more robust capabilities than even many "Neurons" interconnected. Consider how every animal neuron in an animal brain intrinsically swims in a bath of hormone gradients that can provide positional 3d information, and how the structure of those real neurons is at least partially structured based on a thousand generations of evolution, and involves highly conserved sub-structures. Brains do not learn like neural nets do.
> AI are better than most humans at dealing with human suckage
That is a valid opinion, but subjective. If I say that they're not better, we're going to be exchanging anecdotes and getting nowhere.
Hence, the need for a less subjective way of evaluating AI's abilities.
> Making a self driving car fail like a human... "drunk" and "tired"
You don't understand.
It's not about making them present the same failure rate or personality defects as a human. Of course we want self-driving cars to make less errors and be better than us.
However, when they fail, we want them to fail like a good sane human would instead of hallucinating jibberish that could catch other humans off guard.
Simplifying, It's better to have something that works 95% of the time, and hallucinates in predictable ways 5% of the time than having something that works 99% of the time but hallucinates catastrophically in that 1%.
Stick to the more objective side of the discussion, not this anecdotal subjective talk that leads nowhere.
Big scientific revolutions tend to happen before we understand the relevant mechanisms. It is only after the fact that we develop a theory to understand how it works. AGI will very likely follow the same trend. Enough people are throwing enough things at the wall that eventually something will stick.
LLMs aren't just modeling word co-occurrences. They are recovering the underlying structure that generates word sequences. In other words, they are modeling the world. This model is quite low fidelity, but it should be very clear that they go beyond language modeling. We all know of the pelican riding a bicycle test [1]. Here's another example of how various language models view the world [2]. At this point it's just bad faith to claim LLMs aren't modeling the world.
The "pelican on a bicycle" test has been around for six months and has been discussed a ton on the internet; that second example is fascinating but Wikipedia has infoboxes containing coordinates like 48°51′24″N 2°21′8″E (Paris, notoriously on land). How much would you bet that there isn't a CSV somewhere in the training set exactly containing this data for use in some GIS system?
I think that "modeling the world" is a red herring, and that fundamentally an LLM can only model its input modalities.
Yes, you could say this about human beings, but I think a more useful definition of "model the world" is that a model needs to realize any facts that would be obvious to a person.
The fact that frontier models can easily be made to contradict themselves is proof enough to me that they cannot have any kind of sophisticated world model.
> Wikipedia has infoboxes containing coordinates like 48°51′24″N 2°21′8″E
I imagine simply making a semitransparent green land-splat in any such Wikipedia coordinate reference would get you pretty close to a world map, given how so much of the ocean won't get any coordinates at all... Unless perhaps the training includes a compendium of deep-sea ridges and other features.
> The fact that frontier models can easily be made to contradict themselves is proof enough to me that they cannot have any kind of sophisticated world model.
A lot of humans contradict themselves all the time… therefore they cannot have any kind of sophisticated world model?
A human generally does not contradict themselves in a single conversation, and if they do they generally can provide a satisfying explanation for how to resolve the contradiction.
>How much would you bet that there isn't a CSV somewhere in the training set exactly containing this data for use in some GIS system?
Maybe, but then I would expect more equal performance across model sizes. Besides, ingesting the data and being able to reproduce it accurately in a different modality is still an example of modeling. It's one thing to ingest a set of coordinates in a CSV indicating geographic boundaries and accurately reproduce that CSV. It's another thing to accurately indicate arbitrary points as being within the boundary or without in an entirely different context. This suggests a latent representation independent of the input tokens.
>I think that "modeling the world" is a red herring, and that fundamentally an LLM can only model its input modalities.
There are good reasons to think this isn't the case. To effectively reproduce text that is about some structure, you need a model of that structure. A strong learning algorithm should in principle learn the underlying structure represented with the input modality independent of the structure of the modality itself. There are examples of this in humans and animals, e.g. [1][2][3]
>I think a more useful definition of "model the world" is that a model needs to realize any facts that would be obvious to a person.
Seems reasonable enough, but it is at risk of being too human-centric. So much of our cognitive machinery is suited for helping us navigate and actively engage the world. But intelligence need not be dependent on the ability to engage the world. Features of the world that are obvious to us need not be obvious to an AGI that never had surviving predators or locating food in its evolutionary past. This is why I find the ARC-AGI tasks off target. They're interesting, and it will say something important about these systems when they can solve them easily. But these tasks do not represent intelligence in the sense that we care about.
>The fact that frontier models can easily be made to contradict themselves is proof enough to me that they cannot have any kind of sophisticated world model.
This proves that an LLM does not operate with a single world model. But this shouldn't be surprising. LLMs are unusual beasts in the sense that the capabilities you get largely depend on how you prompt it. There is no single entity or persona operating within the LLM. It's more of a persona-builder. What model that persona engages with is largely down to how it segmented the training data for the purposes of maximizing its ability to accurately model the various personas represented in human text. The lack of consistency is inherent to its design.
Neither Sapir nor Whorf presented Linguistic Relativism as their own hypothesis and they never published together. The concept, if it exists at all, is a very weak effect, considering it doesn't reliably replicate.
Because parasocial relationships with ewhores isn't healthy, particularly at a stage in their life when they should be forming real relationships with their peers.
Scrolling through attractive women (generally the thirst-traps are women) doesn't imply forming a parasocial relationship. I agree that parasocial relationships are bad, but this is independent of them being thirst-traps. Internet thirst-traps are just the modern equivalent of sneaking a look at a playboy mag or a lingerie catalogue. Nothing inherently damaging about it. The scale of modern social media can make otherwise innocuous stimuli damaging, but this is also independent of it being content of sexy women.
You are the one claiming there's a problem, and you are the one (presumably) demanding legal and other action to deal with that "problem". That means that any burden of proof is 1000 percent on you.
... and before you haul out crap from BYU or whatever, be aware that some of us have actually read that work and know how poor it is.
Parasocial relationships are a different topic than pornography.
Are you saying that the intersection is uniquely bad? In either case limits to content made in an effort to minimize parasocial relationships cut across very different lines than if the goal is minimizing access to porn.
These people come out of the woodwork, when it comes to defending porn. It’s their whole identity. And unfortunately the tech scene is infested with these types.
It's goalpost shifting. If the concern is parasocail relationships to content creators formed with pornography as the hook, then pornographic content where the actors aren't cultivating or interacting with a social media followerbase should be better, right?
Then support them. Too often you show up to scream "think of the children" without actually citing any research or empirical damage. If you refuse to argue in good faith and don't want to be told you're wrong, voting is the only thing you're capable of doing. Don't tell us about it, vote.
Everyone knows those laws do nothing, though; go look at the countries that pass them. Kids share pornography P2P, they burn porno to CDs and VHS tapes and bring in pornographic magazines to school. They AirDrop pornographic contents to their friends and visit websites with pornography on them too. Worst-case scenario, they create a secondary market for illegal pornography because they'll be punished regardless - which quickly becomes a vehicle for creating CSAM and other truly reprehensible materials.
They don't do it because they're misogynistic, mentally vulnerable or lack perspective - they do it because they're horny. Every kid who aspires to be an adult inherently exists on a collision course with sexual autonomy, most people realize it during puberty. If you frustrate their process of interacting with adulthood, you get socially stunted creeps who can't respond to adult concepts.
The bound informational dynamic that supervenes on the activity of the individual units in the cluster. What people typically miss is that the algorithm when engaged in a computing substrate is not just inert symbols, but an active, potent causal/dynamical structure. Information flows as modulated signals to and from each component and these signals are integrated such that the characteristic property of the aggregate signal is maintained. This binding of signals by the active interplay of component signals from the distributed components realizes the singular identity. If there is consciousness here, it is in this construct.
reply