In any case, see the section on Jakob Uszkoreit, for example, or Noam Shazeer. And then…
> In the higher echelons of Google, however, the work was seen as just another interesting AI project. I asked several of the transformers folks whether their bosses ever summoned them for updates on the project. Not so much. But “we understood that this was potentially quite a big deal,” says Uszkoreit.
Worth noting the value of “bosses” who leave people alone to try nutty things in a place where research has patronage. Places like universities, Xerox, or Apple and Google deserve credit for providing the petri dish.
You can understand how transformers work from just reading the Attention is All You Need paper, which is 15 pages of pretty accessible DL. That's not the part that is impressive about LLMs.
How valuable? I'll give you a hint: It's so valuable that, for some companies, it's the ONLY thing they sell... and they are worth billions of dollars. You may have heard of one or two of them. :-/
A diet that you can follow forever is called a lifestyle. For many diets and many people, it's really hard to convert into a lifestyle, because the of the restrictions and predictable cravings.
Side note, there's a thing called the Potato Hack [0] that proposes 3-5 day potato-only diets for short term fixes. A "potato fast" if you will.
I just read thousands of words about a potato-based diet on that website. Didn't think my Tuesday was gonna start that way, but here we are. HN is amazing.
Does anyone know how the headings were done in smallcaps like that? I remember my sites had the same in the mid-90s, so it must have been some simple feature of HTML, but I can't find it in the spec. I am sure this is pre-CSS.
Google invented the transformer architecture, the backbone of modern LLMs.
reply