> The amazing thing about LLMs is that we still don’t know how (or why) they work!

You may want to take a course in machine learning and read a few papers.

Parent is right. We know mechanically how LLMs are trained and used but why they work as well as they do is very much not known.

Sorry, but that's reductionism. We don't know how human brain works, and that you won't get there by studying quantum electrodynamics.

LLMs are insanely complex systems and their emergent behavior is not explained by the algorithm alone.

Suspect you and the parent poster are thinking on different levels.

That was sarcasm by the poster, in case you failed to notice.

> the corpus of human knowledge.

Goodness this is a dim view on the breadth of human knowledge.

what do you object to about it? I don't see an issue with referring to "the corpus of human knowledge". "Corpus" pretty much just means the "collection of".

Human knowledge != Reddit/Twitter/Wikipedia

Who said it was? I’m pretty sure they’re trained on a lot more than just those.

Conversely, what do you posit is part of human knowledge but isn't scrapable from the internet?

I mean, as far as a corpus goes, I suppose all text on the internet gets pretty close if most books are included, but even then you’re mostly looking at English language books that have been OCR’d.

But I look down my nose at conceptions that human knowledge is packagable as plain text, our lives, experience, and intelligence is so much more than the cognitive strings we assemble in our heads in order to reason. It’s like in that movie Contact when Jodie Foster muses that they should have sent a poet. Our empathy and curiosity and desires are not encoded in UTF8. You might say these are realms other than knowledge, but woe to the engineer who thinks they’re building anything superhuman while leaving these dimensions out, they’re left with a cold super-rationalist with no impulse to create of its own.