At least my attitude is the legal limits are low enough that 'just one drink' is going to be over the legal limit (this is almost certainly not accurate, but it's my attitude; additionally when I drink, it's usually a mixed drink which are often more than '1 drink' for BAC estimation purposes). If you get a DUI ticket, even without a collision, it's a huge impact on your life: much higher insurance costs, loss of driving privileges, potentially an ignition interlock device which are expensive and inconvenient, maybe some jail time, difficulty at international borders, etc.
I have less than a handful of alcoholic drinks per decade, so it doesn't impact me, but when I would drink slightly more often, I would always be sure to have a transportation plan. Either public transit, a designated driver or a significant cooling off period between drinking and driving.
That if your drink a single drink within 8 hours of driving you are a filthy criminal that will have blood drawn and thrown in the drunk tank until monday unless you post bail and will have to go to court and prove your innocence with a lawyer.
Defensive alcoholics are a great accompaniment to yesterday's reflexive hatred of self-driving cars. Human drivers are really showing their whole selves this week.
Obviously you haven't pointlessly sat in a jail cell for 12 hours because you had an empty folded box in your back seat after sleeping 8 hours from one of the two times you drink in a year. How dare I have a 0.005 bac! Thats the equivalent of using alcohol based mouthwash before driving!
That's how it is here in Japan by law - zero tolerance, even one drink is too many.
One solution is "daikou" taxis - they drive you home in your own car, with a chase car to bring the driver back. Where I live they're subsidized (volunteers I think) and can actually be cheaper than a taxi https://ja.wikipedia.org/wiki/運転代行 (use your browsers translate tool)
"the limits are bullshit unless you're borderline unsafe when sober but don't risk it because it's a revenue thing for the state and they'll try hard to get you."
That said, my friends work on boats, cut grass, weld, service generators, etc. for a living so probably the polar opposite of HN let alone the significantly less diverse weekend crowd.
It's a limit on input tokens. So that's 3 50k requests per minute. At Cerebras speeds, that's about 5 seconds of usage per minute.
I was very excited last year for their coding plan but seeing a burst of requests pulse and then sitting there watching the cooldown reset is really not a great time.
Even though each individual request was fast, the sessions were only maybe 10% faster on wall clock time since there was so much waiting time.
They made the coding plan a bit better toward the end, but it was pretty tough to use throughout.
Seems like an Amdahl’s law of inference economics? there’s so much compute relative to SRAM on the chip and shoreline bandwidth onto the chip that caching buys ~nothing? The contended resource is SRAM and a given token of context needs just as much as another.
That’s not what caching is for. Caching lets you resume with a pre computed KV cache saving you from having to ingest everything in the chat history as input on every single round trip. You still need caching regardless of SRAM or not as it saves a huge amount (and ever growing) of compute ingesting the preceding history every time you want a completion.
I don’t know why they don’t give a price discount. Maybe their hardware is incapable for some reason of saving/restoring the state? Or maybe they just haven’t built the infrastructure to do it?
This is a misunderstanding of either the bitter lesson or what was being claimed, on multiple accounts. Firstly, the bitter lesson is merely about human expertise-tuned algorithms vs. throwing raw compute at a domain. But, notably, it is still domain-specific. No matter how much compute you throw at training an LLM, it is never going to beat a Chess engine at Chess. If you give a Chess engine 1,000,000 compute units and a general-purpose LLM 1,000,000 compute units, the Chess engine is obviously superior at Chess; ergo, there is value in throwing compute units into training models for specific tasks. This is true for within several orders of magnitude of compute, in fact. It's also true that if you give the Chess engine 1000 compute units it'll still beat the all-purpose model with 1,000,000 units, so actually there's a lot of value in training for specific tasks.
Secondly, the bitter lesson is predicated on compute being cheap. There was a period where a hand-tuned algorithm informed by human expertise would outperform a raw alpha-beta search at Chess. Then compute got cheaper, and DeepBlue ascended to the top. Compute is now expensive again relative to the tasks being performed. We are absolutely still in a period where human expertise in training LLMs will outperform a naive approach with more raw compute.
I don't know much about chess engines; do they still use hand-tuned algorithms, or are they more like AlphaZero, where they learn through self-play to beat any/all possible human contenders? I don't believe DeepBlue was automated to that extent, but it may have been.
In the latter case, the chess example would tend to support the Bitter Lesson, rather than refute it.
I would also be VERY slow to claim that general-purpose models will never be competitive at chess. It wasn't so long ago that transformers couldn't add two-digit numbers reliably without resorting to tool use. They are now as good at "mental arithmetic" as any human savant. It wouldn't surprise me at all to see someone come up with a model that just happens to be really, really good at leveraging the portions of its general training data having to do with chess.
In fact you could argue that AGI demands such a model, if we are to assume that LLMs are a guidepost in that direction.
DeepBlue beat Kasparov with essentially raw compute thrown at alpha-beta search. That does support the premise of the bitter lesson in general. But that does not mean the bitter lesson is correctly being applied here. The point is that even if throwing raw compute at a task is better than careful human-crafted algorithms, it's still task-dependent. The current trend with the people blowing hundreds of billions of dollars is developing an all-purpose model that is everything to everyone, but you don't need hundreds of billions of dollars to create a task-specific model that outperforms their model at a given task.
> I would also be VERY slow to claim that general-purpose models will never be competitive at chess.
This is not the claim. The claim is that for the same amount of compute, a general-purpose language model will never beat a Chess model. I'm dubious, but allow for the possibility that a language model could eventually compete at a top level against humans with enough compute. However, it will never compete with a dedicated Chess model with similar resources. Training a model for a specific task with the same amount of compute will outperform training a general-purpose model with the same amount of compute. This should be common sense, right? The bitter lesson was only about compute over human algorithms, not at throwing compute at a generalised domain over throwing compute at a specific domain.
You made arguments against two claims that I did not make (that I was trying to refute the bitter lesson or that I claimed that LLMs could never be competitive against humans at Chess), so I'd like to ask you read my statements a little more carefully this time.
> Training a model for a specific task with the same amount of compute will outperform training a general-purpose model with the same amount of compute. This should be common sense, right?
I see arguments like this all of the time in ML. There is a large branch of statistical theory that can cast some doubt on this.
Stein's paradox: when three or more parameters are estimated simultaneously, there exist combined estimators more accurate on average (that is, having lower expected mean squared error) than any method that handles the parameters separately.
This goes deeply against "common sense", but is mathematically proven, empirically proven, etc.
If you are just taking a chess model vs. an LLM, you are right.
But if you want a chess model, a medical diagnosis model, and customer service chat model, at a high level the intuition of Stein's paradox is that one combined model will get you better performance than three task-specific models.
Most practitioners don't get this because what amounts to graduate-level stats theory is often brushed aside, and we keep relearning it the hard way.
AFAIK Stockfish still remains a traditional (some-variant-of-) minimax-search engine, but now uses a microscopic neural network to evaluate the positions at the maximum search depth rather than heuristics. I expect any pure ML engine can learn to exploit any fixed heuristics, but now Stockfish is trying to balance both strategies, which still heavily ends up on the side of raw search depth.
Bitter lesson #1: don't waste time optimizing code when a faster processor is around the corner.
What countered it: Moore's law stopped working.
Bitter lesson #2 similarly relies on scaling laws that might have diminishing returns wrt model runtime vs intelligence. Runtime matters for turnaround on the problem you're solving.
You are technically correct. The best kind of correct.
However, what most people think of as Moore's Law--CPU speed doubles every 18 months--broke somewhere between 90nm and 22nm.
And even the actual Moore's Law--2x the transistors every 18 months--doesn't hold for all types of chips anymore. Memory only gained 2x density over 10 years.
I think the gp found a viable counter to the ggp's version of the bitter lesson, which seems so extreme as to certainly exceptions. IE, they seem to say nothing but the latest frontier model is ultimately viable as an AI business.
The actual argument of the Bitter Lesson essay is pretty limited but people's interpretation of it has gradually drifted until it's seen as prediction that current LLM will reach AGI at a large enough scale.
The issue is that GP is misusing the bitter lesson. Yes, search + learn tends to be more effective than human rules based strategies, but that's not what's being considered here. The original claim is effectively that AGI isn't needed for most tasks and more value can be created by using search + learn to solve specific problems instead of applying general models to every problem. Then GP commented a non sequitur
Article isn't about agents. It's about the inference engine itself being exploited by a malicious LLM output before it is ever sent to your machine or harness.
"malicious LLM" is just a bunch of weights. It runs on, say, Llama.cpp as regular user in separate account, often on its own hardware.
"malicious LLM output" is just a Markdown formatted Unicode-encoded text, produced by Llama.cpp and printed on the screen by my python API script.
I control the input.
Let's assume that "rogue LLM weights" from HF produce "rm -Rf" instruction. it never gets to shell.
And how that "malicious LLM" will disrupt and hack me? With swear words and em—dashes? :-)
(Adding this philosophical point: Black.Mirror.S07E04.Plaything is probably the closest scenario to what you are describing?)
Single turn set-ups may work like this. You control the thing you input, the LLM outputs something and then nothing happens further for that specific context. (Simple question/answer style interactions..)
(Multi-turn) tool calling set-ups however, you need to store the LLM output, the results of the tool calls and feed it back into the inference engine and get the output for the next tool call and/or turn. So yes, print the LLM output on screen and verify it, but maybe the LLM is able to figure out how to hide payloads from your specific set-up. E.g. perhaps it can inject raw ANSI escape codes into your terminal, with which it would be trivial.
Now you have a situation where the true chat completion payload and your view of it have significantly diverged. The LLM could in theory then try (one-shot) to hide further exploits in the hidden payload. E.g. a json parser escape specifically for the inference engine, giving it a means of RCE (although, one can debate whether this is really remote ;) ). Then from the RCE gain a shell, from the shell get access to some privileged device on the current network, and then...
This is great point, actually - "you hit the nail on the head" :-) and I would like to reply.
I am running a very long sessions with LLMs via custom python scripts. Technically, one may call them "harness" but that would be just laughable ... It's literally python script using direct API calls (Vertex in my case) and maintaining the "living session" with all turns etc and also doing the explicit caching. I'm not using LLMs for coding. That hopefully answers another comment regarding why I brought up MD – this is how LLMs output responses to my prompts.
And this is the thing: I fully control input and output and I just know it can't use any other tool. It also, as I said, runs on separate hardware if it is "obliterated" model or runs in GCP for me.
In my setup it is impossible for LLM to get anything hidden with one-shot or gain a shell, as you mentioned.
Did I understand you correctly or I missed something?
Thanks for your points.
Why are you talking about markdown? LLMs output bytes and llama.cpp has to process those bytes correctly. What if the LLM outputs a null byte and causes a buffer overflow somewhere and starts writing into executable memory?
Again, we are not talking about agents or your Python API script but instead talking about exploitable flaws within the inference engine itself. It wouldn't output `rm -rf`. It would output literal CPU instructions that llama.cpp would start executing directly. The payload would never get back to your Python script.
I would love to pay for Fable at full API pricing but unfortunately it is blocked from working on any of my projects. Looking forward to the end of the year when the truly comparable open models will drop.
Still doubt, that they are unaware of Elisp. Bare bones vanilla Emacs is maybe a bit much to get used to these days, so anyone wanting to configure their Emacs will have opened their init.el or so file and will have put some Elisp snippets there.
I've helped such people with their configs. They just know it as "commands" they put in the init file - they don't think of it as a programming language. It's rare for simple init.el files to have things like functions, loops, etc.
And, to be frank, most of them don't customize the init.el file. They just want a text editor - not an IDE. The defaults are ugly, but they're perfectly functional.
None of these people customized Notepad or any other editor either. They're simply solving the problem of "I've SSH'ed into a remote machine. How do I edit this file?"
Why are they then even setting up vanilla Emacs in particular? Not impossible, of course, but it seems like a strange choice to make when not wanting to engage with its configuration. Or maybe another question would be who set them up with vanilla, unconfigured Emacs, if they are not so technical users? Even discovering tramp is not trivial or automatic and for working on a remote machine, isn't Vim much more common?
Emacs is just there in their environment. They didn't set anything up.
> Even discovering tramp is not trivial or automatic and for working on a remote machine, isn't Vim much more common?
Full confession: I, as a power Emacs user don't use tramp. It's ok not to.
I know people disagree with me, but for the most basic editing Emacs has virtually no learning curve compared to Vim. You just open the file and start typing like you would any other editor.
What kind of environments are these? I can't say I have ever encountered emacs in the wild. The only people I have ever met who run emacs did it as an explicit decision because they wanted the malleability.
Even having emacs just there in their environment, most people would avoid it because they don't know what it is, and will open Notepad or TextEdit or whatever familiar program is at hand.
A lot of engineering companies have Linux accounts you log in to. The employees' laptops are Windows but a good amount of engineering work is in Linux. Hence they SSH in and do their work. These accounts are managed by IT. They pre install all these tools.
is it that llms write "better" typescript than let's say elixir because it has seen more of it..? or is it that you're relying on something like effect-ts to keep llms from tripping over even small things?
coincidentally, "good code" in popular lang is rarely directly attributed to only that part; and it's also about the underlying principles it tries to follow in the code... another example; is it typescript that's good, or are "types" inherently making things/feedback loops easier to reason about in llms? (only using ts here for all example because it's probably one of the most "trained on" pl)
If it's samey regardless of language, isn't that in contradiction to your original theory? " .. the most important feature for agent performance is the popularity of the language .. "
No, not really. It's a similar flavor of output, but there's less iterations to get a correct result. The training is mostly about reducing error rates on generation.
I don't get how. I'm not interacting with the lisp, and the agents don't really get frustrated with slow compilation times or anything, and are perfectly adept at debugging.
The best that people have said about lisp is that evidence LLMs perform worse with it is weak.
reply