If flying ever becomes efficient energy-wise, this may happen. However, right now, flying is very energy inefficient, so anything that doesn't need to be flown, is transported overland to save costs. A change of fuel won't change it, unless the underlying energy usage changes fundamentally.
Better batteries do not impact energy usage, only the means of energy delivery.
The data centre runs on a dedicated power line. My laptop runs on battery. Using coding agents currently drains battery quite fast, which is surprising, given that the vast majority of the work does not take place on my laptop.
Making the client side coding agent more efficient isn't about saving the climate. It is about extending the workday (which might actually make the climate worse)
While i agree, the current JS security model rally doesn't allow for distinguishing origin for JS code. Should that ever change, advertisers will just require that you compile their library into the first party js code, negating any benefit from such a security model.
> advertisers will just require that you compile their library into the first party js code, negating any benefit from such a security model.
It will become harder for advertisers to deny responsibility for ads that violate their stated policies if they have to submit the ads ahead of time. Also site operators will need a certain level of technical competence to do this.
More likely, advertisers will need you to insert a "bootloader" that fetches their code and passes it to eval().
Alternatively, they might require you to set up a subdomain with a cname alias pointing to them (or a common CDN), negating any security benefits of such a practice.
> More likely, advertisers will need you to insert a “bootloader” that fetches their code and passes it to eval().
Sounds like legal precedent waiting to be set. “Run our code so that it looks like your code, acts like your code, and has all the same access as your code” seems like it should be a slam dunk if said code ends up doing a Very Bad Thing to your visitors.
But of course that’s assuming common sense, and the law’s relationship with that isn’t always particularly apparent.
There is already plenty of precedent for real-time-served ads which are annoying, or malicious, or install malware; or outright exploit vulnerabilities in the browser.
The advantage would be that I know beforehand, and have the opportunity to test and, possibly, reject, what the advertiser want me to send to someone’s browser.
Their own (presumably cherry picked) benchmarks put their models near the 'middle of the market' models (llama3 3b, qwen3 1.7b), not competing with claude, chatgtp, or gemini. These are not models you'd want to directly interact with. but these models can be very useful for things like classification or simple summarization or translation tasks.
These models quite impressive for their size: even an older raspberry pi would be able to handle these.
There's still a lots of use for this kind of model
Hi all, Ishaan from LiteLLM here (LiteLLM maintainer)
The compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8. Those packages have now been removed from PyPI.
We have confirmed that the compromise originated from the Trivy dependency used in our CI/CD security scanning workflow.
All maintainer accounts have been rotated. The new maintainer accounts are @krrish-berri-2 and @ishaan-berri.
Customers running the official LiteLLM Proxy Docker image were not impacted. That deployment path pins dependencies in requirements.txt and does not rely on the compromised PyPI packages.
We are pausing new LiteLLM releases until we complete a broader supply-chain review and confirm the release path is safe.
From a customer exposure standpoint, the key distinction is deployment path. Customers running the standard LiteLLM Proxy Docker deployment path were not impacted by the compromised PyPI packages.
The primary risk is to any environment that installed the LiteLLM Python package directly from PyPI during the affected window, particularly versions 1.82.7 or 1.82.8. Any customer with an internal workflow that performs a direct or unpinned pip install litellm should review that path immediately.
We are actively investigating full scope and blast radius. Our immediate next steps include:
reviewing all BerriAI repositories for impact,
scanning CircleCI builds to understand blast radius and mitigate it,
hardening release and publishing controls, including maintainership and credential governance,
and strengthening our incident communication process for enterprise customers.
We have also engaged Google’s Mandiant security team and are actively working with them on the investigation and remediation.
Maybe the news has distorted a bit after crossing the Atlantic, but waren't there substantial outrages after the bits that couldn't be touched had in fact been touched?
It would be nice to aggregate all that and put it under a "profile". Kind of like facebook, but your entire profile feed is just the long list of court records, assholery and screw overs for other people. I actually saw a version that someone did for Jack (Twitter's ex founder) a few years ago and it was hilarious but cleverly informative. That's honestly where I got this idea from.
Two things are holding back current LLM-style AI of being of value here:
* Latency. LLM responses are measured in order of 1000s of milliseconds, where this project targets 10s of milliseconds, that's off by almost two orders of magnitute.
* Determinism. LLMs are inherently non-deterministic. Even with temperature=0, slight variations of the input lead to major changes in output. You really don't want your DB to be non-deterministic, ever.
From what I understand, in practice it often is true[1]:
Matrix multiplication should be “independent” along every element in the batch — neither the other elements in the batch nor how large the batch is should affect the computation results of a specific element in the batch. However, as we can observe empirically, this isn’t true.
In other words, the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies! This nondeterminism is not unique to GPUs — LLM inference endpoints served from CPUs or TPUs will also have this source of nondeterminism.
"But why aren’t LLM inference engines deterministic? One common hypothesis is that some combination of floating-point non-associativity and concurrent execution leads to nondeterminism based on which concurrent core finishes first."
How do you propose we measure signal? Lines of code is renowned for being a very bad measure of anything, and I really can't come up with anything better.
The OP said that they kept what they liked and discarded the rest. I think that's a reasonable definition for signal; so, the signal-to-token ratio would be a simple ratio of (tokens committed)/(tokens purchased). You could argue that any tokens spent exploring options or refining things could be signal and I would agree, but that's harder to measure after the fact. We could give them a flat 10x multiplier to capture this part if you want.
I personally discard code for the tiniest of reasons. If something feels off moments after I open the PR, it gets deleted. The reason we still have 1.2K open PRs is because we can't review all of them in time.
The most likely solution is to delete all of them after a month or two. By that time the open PRs on this project alone will be at least 10-20 more.
Doesn't seem like too efficient process, no? Seems to me like investment in better quality of the output is exactly what is needed here, wouldn't you agree?
I feel they sit of on the opposite end of the OP here. One wants to write out specs to control the agent implementation to achieve a one shot execution. Other side says: let’s won’t waste time of humans writing anything.
I’m personally torn. A lot of the spec talk and now here in combination with TDD etc feels like the pipe dreams of the mid 2000. There was this idea of the Architect role who writes UML and specs. And a normal engineer just fills in the gaps. Then there was TDD. Nothing against it personally. But trying to write code in test first approach when you don’t really have a clue how a specific platform/system/library works had tons of overhead. Also the side effect of code written in the most convenient way to be tested and not to be executed. All in all to throw this ideas together for AI now…
But throwing tokens out of the window and hoping for the token lottery to generate the best PR is also not the right direction in my book. But somebody needs to investigate in both extremes I say.
Actually, nobody said the spec needs to be written by humans.
My personal opinion: with today's LLMs, the spec should be steered by a human because its quality is proportional to result quality. Human interaction is much cheaper at that stage — it's all natural language that makes sense. Later, reasoning about the code itself will be harder.
In general, any non-trivial, valuable output must be based on some verification loop. A spec is just one way to express verification (natural language — a bit fuzzy, but still counts). Others are typecheckers, tests, and linters (especially when linter rules relate to correctness, not just cosmetics).
Personally, on non-trivial tasks, I see very good results with iterative, interactive, verifiable loops:
- Start with a task
- Write spec in e.g. SPEC.md → "ask question" until answer is "ok"/proceed
- Write implementation PLAN.md — topologically sorted list of steps, possibly with substeps → ask question
- For each step: implement, write tests, verify (step isn't done until tests pass, typecheck passes, etc.); update SPEC/PLAN as needed → ask question
- When done, convert SPEC.md and PLAN.md into PR description (summary) and discard
("Ask question" means an interactive prompt that appears for the user. Each step is gated by this prompt — it holds off further progress, giving you a chance to review and modify the result in small bits you can actually reason about.)
The workflow: you accept all changes before confirming the next step. This way you get code deltas that make sense. You can review and understand them, and if something's wrong you can modify by hand (especially renames, which editors like VS Code handle nicely) or prompt for a change. The LLM is instructed to proceed only when the re-asked answer is "ok".
This works with systems like VSCode Copilot, not so much with CC cli.
I'm looking forward to an automated setup where the "human" is replaced by an "LLM judge" — I think you could already design a fairly efficient system like this, but for my work LLMs aren't quite there yet.
That said, there's an aspect that shouldn't be forgotten: this interactive approach keeps you in the driving seat and you know what's happening with the codebase, especially if you're running many of these loops per day. Fully automated solutions leave you outside the picture. You'll quickly get disconnected from what's going on — it'll feel more like a project run by another team where you kind of know what it does on the surface but have no idea how. IMO this is dangerous for long-term, sustainable development.
Better batteries do not impact energy usage, only the means of energy delivery.
reply