Another comment mentioned the cost associated with the model. Setting that aside, wouldn't we also need to include all of the systems around the inference? I can imagine significant infrastructure and engineering needs around all of these various services, along with the work needed to keep these systems up and running.
Or are these costs just insignificant compared to inference?
All incremental costs should be included. If adding each 100,000 new customers requires 1 extra engineer you would include that. We don’t know those exact numbers though and the ratio is probably much higher than my example numbers. Inference costs likely dominate.
Alas, the mote around LLM integration is practically non-existent so I'd think that productization around this would be next to impossible.
Anki is already extremely extendable so I would think that with a not too much work deep LLM integration could be implemented in Anki. Like, instead of showing static content for a card, have Anki call an LLM to create the daily iteration of a given prompt.
My ideal flash card for vocabulary (foreign language) focuses on one word per card but always uses that word in a sentence since words never have one single meaning. Like a single sentence or two with the word in bold that you must define.
I think it'd be cool to then use that "{word} {part of speech} {example sentence}" info to generate more example sentences with an LLM.
That way you can grind your problem words with real sentences and cement it quickly, otherwise this process happens too slowly through regular reading.
I'm excited to see the next crop of real LLM powered language apps.
I could see it coming from 2 directions (since I think most people agree language learning and SRS go together):
- anki on steroids - a dynamic way for me to do SRS that feels more natural
- a way to do natural language interaction with the LLM (chat, voice, etc.), with SRS as an added feature or integrated more subtly
As an Emacs users who often tries to do as many things as possible in Emacs, I would say that the more stuff you can do in Emacs, the more the various features in Emacs compound with each other, giving you more utility.
For example, I use the Verb package for making HTTP requests. So with Emacs as my HTTP client, I can do bulk HTTP request calls with keyboard macros. The HTTP requests can be stored in org-mode. I can write custom Elisp for special authentication scenarios. I can create new commands if I need them.
For this example, I can imagine (haven't used this myself) scenarios like creating a keyboard macro to shave off the first X seconds of a video usable with dired.
Some non-text-editing things in Emacs that are actually extremely useful:
- Git via Magit
- Managing files with Dired
- Media player with Emms
- RSS feeds with elfeed
and the list goes on and on...
Using a well thought-out Emacs interface for anything is one of the biggest sources of joy in my technical life.
Using well thought out interfaces is a joy wherever we find them.
Something in your comment made me remember a DOS based file "explorer". Screen split down the middle with a folder-tree and file list on both sides. I remember hardly ever turning on the computer without starting that for one task or another. That was some serious UI pleasure, at least for the time. Ha, found it:
I'd say this ain't an opinion - it's a fact. Where else can you rename a bunch of items in your directory tree, recursively, using all the features of your editor - multiple cursors, keyboard macros, spellchecking, etc.?
There just doesn't exist another piece of software (go ahead, prove me wrong) where you can edit your filesystem like a wiki page.
Getting deep enough into it, it basically becomes your shell, complete with the ability shells have to orchestrate multiple programs into a meta-program.
I 100% agree with everything in this article, though I'm confused what AI has to do with any of this. People have been doing this sort of thing long before LLMs arrived. Weekend projects doing cool things where definitely a thing long before LLMs. I'd say that cloud services (e.g. Twilio) were the real enablers to these sorts of projects so it seems wrong to be crediting LLMs with this type of work.
Cloud services get us from completely impossible to doable with a small amount of work. LLMs maybe save us the time of reading a tutorial or documentation.
The LLMs come in as an enabler to get over "white page" paralysis, and/or overwhelming amounts of libraries to learn to use.
Earlier today, I was chatting with the folks in my computer club, discussing how I wrote a little program just to explore the nature of small neural networks. Then I decided to show them how I use Visual Studio Code with ChatGPT5 (from my GitHub subscription)
The next thing you know, I had a bare bones computerized bulletin board system accessible via telnet, up and running in Python, just as an example of what's possible.
Next was a small database to scan and catalog all the videos on a disk drive, with an SQLite backend. I added a web interface to it in a few minutes, thanks to the LLM.
All of this while I'm a barely passable Python programmer... my preferred language is Delph/Free Pascal.
Things that were previously overwhelming are now almost trivial. Sure, it's effectively instant legacy code... but I can live with less than 1000 lines of legacy code for myself, and nobody else. I might even study it, and learn some things. ;-)
Thank you for putting in clear info about the language, task and results! I’m going to add that you have likely been programming forever.
I am beginning to suspect that the ability to get value out of GenAI is almost entirely dependent on the ability to recognize bad patterns at a glance.
In essence, it’s a tool for power users, not for neophytes.
Good lord, its going to be the excel problem on steroids.
My fear is that this will make the gap between newcomers and veterans so much bigger that the junior market will suffer more than ever before.
However, according to a few job openings in my area, "junior AI powered engineer" is actually a thing that some companies ask. Is it a good idea? I'd say no it's not. Do the managers who do the hiring while reading all the hype care? Definitely not the ones who ask for AI powered juniors.
Excel is the worlds most popular declarative programming language.
2 things I'd fix about it, remove default assuming certain things are dates, and adding a way to mark an area of a spreadsheet as a database table, so you can't sort only some of the columns.
There’s no accountability in software, and in the tech industry people move jobs so fast they can simply blame the previous person and say they need to refactor the code to remove tech debt for a few years. Basically the three envelope situation.
I call this ‘mass produced software’. I walk into a software shop, ask for software and get more or less what I need at barely acceptable quality just like at Walmart.
I totally get the point your trying to make. I guess what I'm trying to say is that I think it's unfair/misleading that anything with a veneer of LLMs has all the credit driven to the LLM and not to the thing that provides the bulk of the value.
Like for example, clearly you are a very experienced developer with a vast amount of experience. To say that the extent and reach to which you are able to apply technologies is because LLMs seems wrong; it's your rich technical background which allow you to use LLMs in an effective manner.
The amount that can be prototyped is astronomically higher with LLM's, which lowers the barrier to do these things and troubleshoot libraries/architectures. Naturally the amount of hobby projects has exploded multiple OOM's beyond what was done before, regardless of any gatekeeping you wish you could do :P
I wonder if it would be feasible for an entity to eject certain nonsense into the internet to such an extend that, at least for certain cases degrades the performance or injects certain vulnerabilities during pre-training.
Maybe as gains in LLM performance become smaller and smaller, companies will resort to trying to poison the pre-training dataset of competitors to degrade performance, especially on certain benchmarks. This would be a pretty fascinating arms race to observe.
I've came to a very similar conclusion. Productivity SaaS apps feel exciting to get started but eventually I've abandoned them all. I feel that many others have similar experiences but I'm not exactly sure why. Like the author, I too ended up with a plain text format (org-mode) and I've happily been on it for 7 years. Some questions that came to mind:
- Is it the artificial hype and promises around certain productivity apps (e.g. youtube notion promoters) that ultimately leave one disappointed?
- Does the productization of these apps make the companies feel compelled to change too much, thus alienating users? Is this why Apple notes has such a following, since it's not a monetized product of apple?
- Is the allure of plain text the fact that it doesn't change, analogous to something written on paper?
I too have used org-mode for a while and here are some additional features which may pique your interest:
- agenda views let me create custom pages of tasks with certain states or tags
- a robust time tracking system. I use this for my freelancing work
- very nice text tables that are programmable
- a very customizable capturing system
- a huge ecosystem of plugins
- a programmable API: I'm currently working on an importer for the DayOne app as well as a fitness tracking package
- PDF export with LaTeX. I can use this for printing out my weekly plan for example
- in addition to deadlines, a scheduled property for when you intend to start a task
- extensive linking system (https://orgmode.org/guide/Hyperlinks.html#External-Links-1) I often have todos linking to places in code
I think that org-mode could use better learning resources. There's pretty much the manual and blog posts by experienced users, neither are especially aimed towards new users.
The beauty of Emacs though is that it puts the user in full control. There is nothing in the world stopping anyone from modifying anything in Emacs (at least on the Elisp layer), hence packages like this.
VS Code on the other hand is designed to fracture [1]. MS can and has given proprietary API access to their blessed tools, forcing others to go through their much less capable extension API, hence the plethora of vscode forks. Even if you had the most motivated, excited group of people wanting to work on the latest and greatest LLM interactions with VS code, they would most likely be forced to fork. On the other hand, it just takes one motivated Elisp dev to implement whatever they want and make any external package they want integrate with it.
Also, I think the derision from the Emacs community may be a bit overblown. I'm constantly seeing AI/LLM related plugins appearing for Emacs and they tend to get decent traction (e.g. https://github.com/karthink/gptel).
I've felt similar to the author, a sort of despair that the only point of writing software now is to prop up the valuation of AI companies, that quality no longer matters, etc.
Then I realized that nothings stopping me from writing software how I want and feel is best. I've stopped using LLMs completely and couldn't be happier. I'm not even struggling at work or feeling like I'm behind. I work on a number of personal projects too, all without LLMs, and I couldn't feel better.
MIT isn’t “weak” because it allows LLM training; it’s weak because it puts zero obligations on the recipient.
Blocking “LLM training” in a license feels satisfying, but I’ve run into three practical issues while benchmarking models for clients:
1. Auditability — You can grep for GPL strings; you can’t grep a trillion-token corpus to prove your repo wasn’t in it. Enforcement ends up resting on whistle-blowers, not license text.
2. Community hard-forks — “No-AI” clauses split the ecosystem. Half the modern Python stack depends on MIT/BSD; if even 5 % flips to an LLM-ban variant, reproducible builds become a nightmare.
3. Misaligned incentives — Training is no longer the expensive part. At today’s prices a single 70 B checkpoint costs about \$60 k to fine-tune, but running inference at scale can exceed that each day. A license that focuses on training ignores the bigger capture point.
A model company that actually wants to give back can do so via attribution, upstream fixes, and funding small maintainers (things AGPL/SSPL rarely compel). Until we can fingerprint data provenance, social pressure—or carrot contracts like RAIL terms—may move the needle more than another GPL fork.
Happy to be proven wrong; I’d love to see a case where a “no-LLM” clause was enforced and led to meaningful contributions rather than a silent ignore.
This is also a good opportunity to remember that MIT is not a strong enough open source license, and if you want to prevent corporations making money off your work, make it AGPL or even SSPL, plus a statement that AI training creates a derivative work (the latter may or may not have any legal effect).
MIT is a donation of your labour to corporations. With a stronger license, at least they're more likely to contribute back or to pay you for a looser license.
As far as I was able to tell, every single coding LLM out there still violates the terms of the MIT license, because the license requires attribution - and LLMs rarely (if ever?) provide any.
I've not used AI to program and have very little interest in using AI to program, but I fail to see how laundering code through massive probabilistic lossy compression (silicon) should be treated any differently than laundering code through massive probabilistic lossy compression (biological). Should humans have to keep track of which software codebases they learn each pattern from, too?
Calling humans massive probabilistic lossy compressors is an insult to curiosity, creativity, compassion, and any number of other traits that push us to advance technology. We've invented everything from Babbage&Ada's, vacuum tubes, punch cards, to GPUs.
Code regurgitators can't even design a coherent API.
My understanding is this was part of the reasoning for a certain US court to rule that AI art is (at least in a default sense) fair use. You're right, both humans and AI "create" things by using things we have seen before... some say art itself can only ever be the sum of our past influences.
The point is that people who think they want permissive licenses usually don't, and eventually regret choosing them when a corporation treats their work as donated labour (because it is), assuming their software is important enough to be picked up by them (if not then license choice doesn't matter anyway).
I always found this stance puzzling. If the point of open source is to give your code to the public, why do people get upset when corporations do exactly what you told them they could do?
If you didn't want to give it to everyone, you shouldn't have chosen that license.
And if you choose a non-commercial license, people get upset that it's "not technically open source because the OSI says so" as if they are somehow the arbiter of this (or even should be). It's not like anyone owns the trademark to the term "open source" for software either.
Ironically, I've seen a lot of people in the last several years quit open source entirely and/or switch to closed source.
Yes I understand... but they already knew that the license explicitly allows this, and they already knew companies regularly take advantage of FOSS without giving back, so I'm not sure why they were expecting to get lucky or something.
To me this is just like getting upset when someone forks your open source project. Which ironically I've seen happen a LOT. Sometimes the original developer/team even quits when that happens.
It's like... they don't actually want it to be open source, they want to be the ONLY source.
Because they don't think about it deeply - that's why reminders are necessary. They think they're only donating to people with similar attitudes to themselves. xGPL licenses (SSPL included) are the license family most similar to that...
... but MIT is what corporations told them they want. There has been a low-level but persistent campaign against xGPL in the past several years and the complaints always trace back to "the corporation I work for doesn't like xGPL." No individual free software developer has a problem with xGPL (SSPL not included).
> No individual free software developer has a problem with xGPL
I do... I consider it the opposite of freedom. I think it places severe restrictions on your project that make it hard/impossible for some people (like companies) to use, especially if your project contains lots of code from other people that make it really hard/impossible to try to re-license if one day you decide you like/need money (assuming you have no CLA, I don't like those either).
But I also realize there's different kinds of freedom... freedom TO vs freedom FROM.
Some want the freedom TO do whatever they want... and others want freedom FROM the crazy people doing whatever they want.
I wish there was a happy medium but centrism doesn't seem to be very popular these days.
Tangentially, I wonder if logins and click-throughs can help address this on the legal front.
If you set up a login flow with a click through that explicitly sets the terms of access, specifying no cost for access by a person, and some large cost for access by AI.
Stepping past this prompt into the content would require an AI to either lie, committing both fraud and unauthorized access of content.. or behave truthfully, opting in the proprietor of the API to the associated costs.
In either case, the site operator can then go after the company doing the scraping to collect the fees as specified in the copyright contract (and perhaps some additional delta of punitive fines if content was accessed fraudulently).
Given Meta's history of torrenting every book it could get its hands on for training, I'm not convinced that the majority of AI companies would respect that license. Maybe if we also had a better way to prove that such code was part of the training set and see a couple of solid legal victories with compensation awarded.
I'm pretty astounded that "The Stack" at least did and effort, and continue to do so by weeding out GPL or similar strong copyleft source code from their trove, and even implemented an opt-out mechanism [0].
They look like saints when compared to today's companies.
They're also getting sued for it, and the judge ruled they had no right to torrent those books so now it's just a matter of calculating how many trillions Meta has to pay, then extracting it from them.
Because Meta got caught. I'm not convinced that every random OSS lib will have the resources to audit every model out there for a hypothetical GPL+no training violation.
*Unless you're a member of the capital class, in terms of being a corporation or a wealthy individual, who can then make our two-tiered justice system work for you. As Disney is seemingly looking to do. Then it will absolutely work for you.
This is why I and people like me so often say "there is no war but the class war." Arguing about copyright misses the entire point: The law serves the large stakeholders in the system, not the people. The only thing that's changed is there is now a large stakeholder of whom a core pillar of their ongoing business is the theft of data at industrial scale which happens to include data of other large stakeholders which is why we're now seeing the slap fight.
By all means enjoy it, it's very entertaining watching these people twist themselves into knots to explain why it's okay for Nintendo to sue people into the ground for distributing copies of games they no longer sell in any capacity but simultaneously it's okay for OpenAI to steal absolutely goddamned everything on the grounds that nothing has been "really" taken due to being infinitely replicable, or because it's a public research org, or whatever flimsy excuse is being employed at the time.
As it has been from the beginning, my position is: whatever the rule we decide on, it should apply to everyone. A very simple statement on very basic ethics that seems to make a lot of people very angry for some reason.
"The law, in its majestic equality, forbids rich and poor alike to sleep under bridges, to beg in the streets, and to steal their bread." Anatole France
the AGPL says that you can ignore any restrictions the author tried to impose on you, that's why you frame it as LLM training is already violating the AGPL by making a derivative work.
> MIT is a donation of your labour to corporations.
Unless you are willing to spend yourself into financial ruin pursuing legal action against some faceless megacorp - it literally doesn't matter what license you use.
I've lived enough to know there is "what should be" and then there is what actually happens in reality. We don't live in a reality where everyone just does things out of the goodness of their heart...
Adding some text to your project, hosted on a public website for all to see means some people will take your code regardless of the license or your intent - and, realistically, what are you going to do about it? Nothing...
So... please, let's get off this GPL high-horse. It's not some end-all-be-all holy text that solves all of the world's problems.
I've been at the same company both before and after the AI revolution. I've felt something similar. People seem to be more detached, more aloof in their work. I feel like we're discussing our code less and are less able to have coherent big-picture plans concerning the code at-large.
The ultimate purpose of AI is indeed to remove cognitive autonomy. Little pieces here and there may seem empowering, but added all up, it pretty much takes control away from people.
Or are these costs just insignificant compared to inference?