Best comment I’ve seen today.

Wake me when they bother to define consciousness in a non trivial way.

>If a machine has to learn to understand humans to complete text, then that is what it has to do.

A language model completes text based on the overlapping patterns of the training data.

There absolutely was thinking involved… in the training data. Same as when you read a book, you engage with the thinking behind the text. The book isn’t thinking, and the author may be dead and gone, but there’s absolutely the traces of thinking in the text.

Language models produce mashups of texts they were trained on, and there’s absolutely the traces of thoughts behind those mashups.

Wow, such a great text only page with news reporting, I thought these were extinct!

AI investment and spending is frequently cited as one of the few bright spots in the economy, I wonder if the continued over-optimism is mostly about keeping the bubble inflated. If you are a tech CEO, would it be a disservice to your shareholders to express skepticism about AI?

I think this cuts to the point about democracy but if all the people somehow want something negative for themselves short term or long term, if you are the leader should you do what the public is saying or not?

I think there's more nuance to it but replace people with shareholder and leader with CEO.

I think that for a company to exist and thrive long term, it might need a culture which doesn't jump on every trend but it still evaluates them from time for time for a certain time and treat them as such (like tools) and if the tool is ineffective, then to not use the tool.

Unfortunately, I feel like this requires a deeper discourse and CTO's might be better suited for it or the fact that I feel like perhaps some shareholders might not be interested in the technical details so much.

I don't know but If I were a leader I would hopefully wish to make a pragmatic solution/suggestion while taking finances, current reality in mind and currently IMO AI aren't the end all, be all, that some people (with shrewd/double incentives) intend on suggesting.

A big reason there are so few bright spots is AI sucking all the air out of anything else. Both funding and attention-wise.

Not quite lines of code.

The person who takes a business problem and proposes an expansive solution that requires a big team that they lead to victory, that person climbs the ranks.

The person that takes the same business problem and carefully simplifies both the problem and the solution, and delivers it themselves, is rewarded but not at all like the team leader.

And there was a definite shift from sharing toward evangelism.

For example C was shared, C++ was evangelized. The difference is the effort put into convincing people to adopt your stuff.

Java for instance was mega evangelized, Sun thought it might reverse their fortunes.

Linux was initially “here you go, hope it works for you” but then it attracted many people who decided to create an ecosystem around it.

They have something in common, which is an endless need to chase hype cycles to lure investors.

Search on “diy radio telescope”, that gave me lots of projects and videos

Oh, you mean somewhere it is tracking the statistical likelihood of the output. Yeah I buy that, although I think it just tends towards the most likely output given the context that it is dragging along. I mean it wouldn’t deliberately choose something really statistically unlikely, that’s like a non sequitur.

Well, it's not tracking. As it predicts each token it is sampling from a probability distribution -- that's what the matrix multiplies are for. It gets a distribution over all tokens and then picks randomly according to that distribution. How flat or how spiky that distribution is tells you how confident it is in its answer.

But it then throws that distribution away / consumes it in the next token calculation. So it's not really tracking it per se.

From its point of view what does it mean "to know".

Is it the token (or set of tokens) that are strictly > 50% probable or is it just the highest probability in a set of probabilities?

While generating bullshit is not ideal for a lot of use cases you don't want your premier chat bot to say "I don't know" to the general public half the time. The investment in these things requires wide adoption so they are always going to favour the "guesses".