Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
This is probably the wrong denominator. There are more than a billion white-collar workers. Making them all just 10% more effective would possibly be worth more than $650B/year (~$650/worker).
> If AI makes software easier to create, that will drive the price down.
Supposedly AI drives down the cost of producing software,not the "price".
> How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software?
Currently, the cost of AI is between $20/month and around $200/month per developer.
I think the huge billions you're seeing in the news are the investment cost on AI companies, who are burning through cash to invest in compute infrastructure to allow both training and serving users.
> This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.
You can run huge local models slowly with the weights stored on SSDs.
Nowadays there are many computers that can have e.g. 2 PCIe 5.0 SSDs, which allow a reading throughput of 20 to 30 gigabyte per second, depending on the SSDs (or 1 PCIe 5.0 + 1 PCIe 4.0, for a throughput in the range 15-20 GB/s).
There are still a lot of improvements that can be done to inference back-ends like llama.cpp to reach the inference speed limit determined by the SSD throughput.
It seems that it is possible to reach inference speed in the range from a few seconds per token to a few tokens per second.
That may be too slow for a chat, but it should be good enough for an AI coding assistant, especially if many tasks are batched, so that they can progress simultaneously during a single read pass over the SSD data.
You can do that, but you're going to have rather low throughput unless you have lots of PCIe lanes to attach storage to. That's going to require either a HEDT or some kind of compute cluster.
Batching inferences doesn't necessarily help that much since as models get sparser the individual inferences are going to share fewer experts. It does always help wrt. shared routing layers, of course.
> Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.
Depends how big the models are, how fast you want them to run and how much context you need for your usage. If you're okay with running only smaller models (which are still very capable in general, their main limitation is world knowledge) making very simple inferences at low overall throughput, you can just repurpose the RAM, CPUs/iGPUs and storage in the average setup.
I got a 128 GB MBP, and the current models are fit enough to manage the calendar or do research on web (very slowly), not to be useful companions for coding as I hoped.
> This demand for RAM is built on a foundation of sand
Not exactly.
LLMs are already quite useful today if you use them as a tool, so they are there to stay. The remaining problem is scalability, a.k.a. how to make LLMs cheap to use.
But scalability is not really a requirement when you look the bigger picture. If smaller software company/projects can't afford to use AI, the bigger ones might just. Eventually they will discover variable use cases for such tech, even if it only serves big firms i.e. defense, resource extraction, war, finance etc.
To the other end, if scalability is achieved, the use of LLM products will be cheaper too, so smaller project can also use them. But of course, if LLM usage is too cheap, then many were-to-be-consumers will just create software projects by themselves at their homes.
The advanced workflows such as programming are not the only use cases for LLMs. There are much simpler work such as answering simple questions or arrange simple tasks. Both will improve productivity. And more advanced model should improve productivity even more.
The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.
> The usage of LLMs is continuing to increase ~exponentially
I would like a source for that statement. Additionally, I want to know by who? Because it certainly isn't end users. Inflating token usage doesn't make it any more economically viable if your user base, b2b or not, hasn't increased with it. On the contrary, that is a worse scenario for providers.
1. As a consultant pretty much every company I have worked with in the last 2 years are doing some kind of in-house "AI Revolution", I'm talking making "AI Taskforce" teams, having weekly internal "AI meetings" and pushing AI everywhere and to everyone. Small companies, SMEs and huge companies. From my observation it is mainly due to C-level being obsessed by the idea that AI will replace/uplift people and revenue will grow by either replacing people or launching features 10x quicker.
2. Did you see software job-boards recently? 9/10 (real) job listings are to do with AI. Either it is fully AI company (99% thin wrapper over Anthropic/OpenAI APIs) or some other SME that needs some AI implementations done. It is truly a breath of fresh air to work for companies that have nothing to do with AI.
The biggest laugh/cry for me are those thin wrappers that go down overnight - think all the "create your website" companies that are now completely useless since Ahtropic cut the middleman and created their own version of exactly that.
Yeah, my only hope is that this is unsustainable, admittedly for selfish reasons.
I know plenty of engineers being forced to use these tools whether they want to or not. A lot of which are okay with using AI liberally, but don't particularly like generative AI and see it as pretty irresponsible (which feels more true by the week and it is clear from first hand experience). I don't know, there is a huge gradient of users, but I would argue that in previous revolutionary technologies, we didn't have to force people to use a good tool. I didn't have to be forced to use Google search or Google Maps, tech that is now ubiquitous with western society. It seems really suspect that suits have to enforce the use of something that is supposed to change the way we work and be a force multiplier.
From my limited experience in multiple companies, as stated before I see one very common pattern - The process from feature idea to development is just bad. PMs do not know what exactly they want. C-level interjects in the middle and changes requirements. QAs are unsure what to test because acceptance criteria is vague.
C-level strongly believes that AI will fix all these issues. They believe that AI will fix their broken processes.
I see strong resemblance with "Agile Development" ~15 years ago. Extremely hyped, noone asked if their org even is a fit for it or need it, and most importantly - the only way to fix agile is to do more agile. Same with AI right now.
Users are willingly paying for larger volumes of tokens. You are layering your own unproven interpretation onto that. I would have arrived at an opposite interpretation given the available facts. Models are becoming more token efficient for the same task, such as ChatGPT 5.3 versus 5.2 which halved the token count, and capabilities show a log relationship with the number of tokens since o1 preview was revealed in September 2024.
No, you have gone off in your own tangent. The person you're responding to is talking about money and my point is that you're using a misleading metric. Even if the current user base is paying more for the "exponential token usage", it does not add up to the industry's cost of maintaining and building on this technology, especially since we are not taking into account what that token usage costs the provider. First you said Anthropic as your source, but now you're talking about OpenAI's ChatGPT, who are floundering for a product and user base, which they themselves claim will be profitable through subscriptions at numbers never seen before in a subscription business model.
If they were motivated enough by this story to delete 20 years worth of history maybe they were motivated enough to create an account and talk about it?
I don't care. The UX means I can't give it any credibility.
For all I know this could be somebody's OpenClaw spouting bullshit. The default credibility of all throwaways is zero and that was even true before 2023.
If you let it influence your opinion in any way you're a fool.
The content of the message is the credibility. It doesn't matter where it came from or who posted it. This exact topic comes up every time Google reveals its true self and lots of us have a resurgence of our latent interest to de-Google (the massive inconvenience being the major barrier).
From busterarm's profile: "Most people are stupid and/or on drugs."
The account is from 2013 but given that profile, I can't give it any credibility. After all, it could be somebody's OpenClaw having been granted control of the account.
> After all, it could be somebody's OpenClaw having been granted control of the account.
Luckily for HN, I actually have a post history. You can use my post history, textual analysis and statistics to make an informed decision about whether I'm a bot or not. Whether I'm being consistent or spouting any random bs.
The account I was responding to doesn't have anything.
> The account is from 2013 but given that profile, I can't give it any credibility.
What's in my profile is a statistical fact. It's there as a reminder, to me, not to expect everyone to see the world the same way that I do. To be comfortable with strong disagreement.
Just a hair shy of half the population is below average intelligence. Roughly 1 in 4 people has a cognitive impairment. This is of any age but trends upwards with age, reaching 2 in 3 by age 70. 1 in 4 Americans take psychiatric medication. 1 in 4 participates in illegal drug use. We haven't even touched on alcohol abuse.
My profile statement is just objective reality, whether you're comfortable with being stated openly or not.
“They can pass a law that says that all left-handed people are subject to a 50% income tax even though such a thing would clearly violate the constitution”
I think that would be constitutional, but in conflict with other laws.
How so? Left handed people aren't human. Just like how criminals aren't treated like normal humans with equal rights.
Seriously though, I don't think it technically violates anything given that we do have a set of humans (criminals) that we treat unequally. Culturally we believe theft and murder gives us the right to treat such people who do such things unequally and we've encoded that into law. It is simply another culture shift to interpret left handedness as the same thing.
I mean the example is absurd but it's a valid example. Maybe a more realistic example is pronoun usage and the forced recognition of multiple genders other than two. Taken to the extreme we would have to accept that anyones made up gender is real and we will be forced to recognize their beliefs that these things exist.
In CA you can already get this classified as harassment and get fired from your workplace.
And just to be clear I agree with the whole made up gender and pronoun thing. If you want me to refer to you with they instead of she or he that's fine, but the point is that all of this is clearly culture/opinion based and none of it is a universal right because what is "universal" is ALSO an opinion.
The left handed tax law could be passed and declared unconstitutional almost instantly under a challenge brought by any individual that referenced the equal protection clause of the 14th amendment, to any federal court. if the government appealed it at all, the appeals court would agree with the lower court, if the government appealed it again, the Supreme Court would also agree with the appeals court and lower court, and it would be a phenomenal waste of time.
That's the only validity of the example.
Nothing occurring in the court system matches the angst of people that view recent appointments and decisions to be invalid. Anything overturned only affected the day to day life because there was never an actual federal law passed at all. The courts are operating much closer to how people imagine them, than in prior times, despite people believing the opposite is occurring. Media.
Blows me away that energy policy is so political, and that somehow self-styled libertarians who don’t say a peep about oil subsidies are deeply offended by renewable ones. It you consider yourself libertarian can you at least be forward-thinking enough to see that shifting to renewables is also a step towards decentralization?
> How is it decentralization
Producers of electricity being everywhere is more distributed than relatively centralized power generation stations. Regardless of who paid (part of) it
The technologies for renewable energy are inherently more decentralized than those for fossil fuels. My point was clearly this and nothing else: given that there are subsidies for both, a libertarian should be less upset about renewable subsidies since it is an enabling force for individual liberties when it comes to energy production. In practice, they are very outspoken about renewable subsidies and fairly quiet on oil subsidies.
I think you are talking about a different sort of decentralization than libertarians talk about, maybe?
Libertarians want decentralized political/coercive power. When the government is paying for power generation in smaller amounts at larger numbers of locations that's not decentralizing the governments political/coercive power.
And again, I can't imagine a libertarian who when questioned would be ok with oil subsidies. Point them my way and I'll give them a stern talking to.
It is indeed harder if you topography writes off certain kind of electricity production.
But of course, good for them that they used their mountains / rivers ! All all power to them to electrify their economies if they have surplus ! And even better if they can use some reservoirs and pump to store, because it means they can diversify their productions by adding wind / solar and store a lot, until batteries of seasonal scale become more widespread.
reply