It would be interesting to analyze this dataset in terms of colors, layout, features, fonts, photos, etc. to be able to statistically measure the uniqueness or creativity of a given web design.
1. Natural language appears to be to be the starting point of any endeavor.
2.
> It may be illuminating to try to imagine what would have happened if, right from the start our native tongue would have been the only vehicle for the input into and the output from our information processing equipment. My considered guess is that history would, in a sense, have repeated itself, and that computer science would consist mainly of the indeed black art how to bootstrap from there to a sufficiently well-defined formal system. We would need all the intellect in the world to get the interface narrow enough to be usable, and, in view of the history of mankind, it may not be overly pessimistic to guess that to do the job well enough would require again a few thousand years.
LLMs are trying to replicate all of the intellect in the world.
I’m curious if the author would consider that these lofty caveats may be more plausible today than they were when the text was written.
One thing I’d add to all of the other comments is just to reflect on experience. Maybe I’ve mostly worked with people who are incompetent with natural language. But assuming that I’ve mostly worked with average people it’s astonishing how common miscommunication is amongst experts when discussing changes to software. I’ve always found the best way to avoid that is to drop into a more structured language. You see this with most communication tools. They add structure to avoid miscommunication.
> I’m curious if the author would consider that these lofty caveats may be more plausible today than they were when the text was written.
What is missed by many and highlighted in the article is the following: that there is no way to be "precise" with natural languages. The "operational definition" of precision involves formalism. For example, I could describe to you in english how an algorithm works, and maybe you understand it. But for you to precisely run that algorithm requires some formal definition of a machine model and steps involved to program it.
The machine model for english is undefined ! and this could be considered a feature and not a bug. ie, It allows a rich world of human meaning to be communicated. Whereas, formalism limits what can be done and communicated in that framework.
I forgot where I read it, but the reason that natural languages works so well for communication is because the terms are labels for categories instead of identifiers. You can concatenate enough to refer to a singleton, but for the person in front, it can be many items or an empty set. Some labels may even be nonexistent in their context
So when we want deterministic process, we invent a set of labels where each is a singleton. Alongside them is a set of rules that specify how to describe their transformation. Then we invented machines that can interpret those instructions. The main advantage was that we know the possible outputs (assuming a good reliability) before we even have to act.
LLMs don't work so well in that regard, as while they have a perfect embedding of textual grammar rules, they don't have a good representation for what those labels refers to. All they have are relations between labels and how likely are they used together. But not what are the sets that those labels refer to and how the items in those sets interact.
> All they have are relations between labels and how likely are they used together. But not what are the sets that those labels refer to and how the items in those sets interact.
Why would "membership in a set" not show up as a relationship between the items and the set?
In fact, it's not obvious to me that there's any semantic meaning not contained in the relationship between labels.
Because the training data does not have it. So we have the label "rock" which intersect with some other label like "hard" and "earth". But the item itself have more attributes that we don't bother assigning label to them. Instead, we just experience them. So the label get linked to some qualia. We can assume that there's a collective intersection of the qualia that the label "rock" refers to.
LLMs don't have access to these hidden attributes (think how to describe "blue" to someone born blind). They may understand that color is a property of object, or that "black" is the color you wear for funerals in some locations. But ask them how to describe the color of a specific object and the output is almost guaranteed to be wrong. Unless they are in a funeral in the above location, so he can predict that most people wear black. But it's a guess, not an informed answer.
This is a nice explanation of language, and the world-model that the language is intended to depict. If I understand you correctly, formalism is a kind of language where the world-model (ie, items and actions in the world depicted by the language) leaves no room for doubt.
Pretty much and if we take programming languages, where inputs and RNG are not specified by the formal grammar of that language, the programmer needs to split them between good values and bad values. And ideally, halt the program when it detects the latter as the result would be nonsense.
So a program is a more restrictive version of the programming languages, which itself is a more restrictive version of a computer. But the tools to specify those restrictions are not perfect as speed and intuitiveness would suffer greatly (haskell vs python).
But for most human endeavors, "operational precision" is a useful implementation detail, not a fundamental requirement.
We want software to be operationally precise because it allows us to build up towers of abstractions without needing to worry about leaks (even the leakiest software abstraction is far more watertight than any physical "abstraction").
But, at the level of the team or organization that's _building_ the software, there's no such operational precision. Individuals communicating with each other drop down to such precision when useful, but at any endeavor larger than 2-3 people, the _vast_ majority of communication occurs in purely natural language. And yet, this still generates useful software.
The phase change of LLMs is that they're computers that finally are "smart" enough to engage at this level. This is fundamentally different from the world Dijkstra was living in.
IMO, it depends. I’ve worked with hundreds of startups over the last 20 years. The most significant trend I’ve seen is cultural cohesion. Some startups have a young, energetic culture. Some have an experienced, veteran culture. Some have a quirky, trendy culture.
But overall many companies seem to value and attract more of the same culture they already have in place. So to a degree, I’d say yes ageism exists in certain companies that don’t value experience as highly as other qualities.
> I’ve worked with hundreds of startups over the last 20 years.
That's averaging 5 startups every year. How can you get good experience, or even truly understand a business, if you're averaging just around two months at a company or multiple companies in the same period?
He said "worked with" not "worked at." Likely as a contractor. And if so it's not uncommon for some projects to be very short while others might take 10-20% of your time for 2-3 years. Plenty of time to gather domain knowledge especially if clients are clustered in the same industry.
Occam's Razor should not be used for self-harm. Two of the guys at the head of the biggest open conspiracies in the world are likely to be conspiring if their actions result in favorable conditions.
This social media spat is happening immediately after "Welcomefest," a bizarre Republican-light centrist conference, and positions Elon Musk to capture the centrist Democratic apparatus. This follows a pattern of fecklessness and capitulation from the party's power centers, including major Democratic figures advocating for realignment with MAGA on core issues.
There's another pattern too: we have seen a concerted effort from the MAGA movement to capture societal institutions and forcibly reform them into alignment with their radical agenda to replace the US Constitution. Aligning the DNC into a technocratic fascist organization that supports their monarchic visions of domination will be an incredible humiliation (though far from the last). And quite simple.
I'm going to assume conspiracy from the guys arrogant enough to get caught boasting that the opposition is too credulous to accept an open conspiracy.
I’m not debating the potential accuracy of anyone’s interpretation, just commenting on the degree of simplicity of those interpretations.
It’s a Razor, not a law, so it isn’t going to be 100% accuracy or predictive. But to suggest that the Razor can be used in conjunction with a long, complex explanation is a contradiction.
> Two of the guys at the head of the biggest open conspiracies in the world are likely to be conspiring if their actions result in favorable conditions
This is not long or complex. It can be reduced to "liars lie" if that helps.
Many years ago there were guides on how to write user stories: “As a [role], I want to be able to do [task] so I can achieve [objective]”, because it was useful to teach high-level thinkers how to communicate requirements with less ambiguity.
It may seem simple, but in my experience even brilliant developers can miss or misinterpret unstructured requirements, through no fault of their own.
Most subagent examples are vague or simplistic.