There is a window on the 2nd. But you don't aim for the second half of the launch period and hope you make it, you aim for the start to allow time to resolve issues without waiting for the next window (which is the end of the month).
I remember hearing somewhere on this site that medical imaging got pretty good at building systems that recycle helium.
Does chip manufacturing not do this or are the losses at their scale are still large enough that you need a substantial constant supply?
The big problem is purity. Fabs use grade 5 and 6 helium where contaminants are 1-10 parts per billion. The infrastructure to get it that pure becomes very specialized and any time the helium goes through a process it picks up so much contamination that recycling it would require the entire purifying and quality control infrastructure for pressure or temperature swing adsorption.
Some fabs are starting to reuse helium in downstream processes but there’s only so much they can do without expanding their core competency into yet another complex chemical manufacturing process.
MRI machines don’t need high purity helium and the contamination doesn’t “gunk up” all the tools so it’s not an issue to recycle it there.
Now I'm imagining a procedural cop show where they bust an illegal helium dealer, and one of the cops takes a huff to gauge what they're dealing with, and then squeaks out "that's the good stuff".
> The infrastructure to get it that pure becomes very specialized
I think some of the most advanced fab infrastructure is the ultra pure water system. Water becomes quite aggressive chemically when it has no dissolved ions in it. You have to use exotic or highly processed materials simply to transport it around. If the factory didn't need such massive quantities of it, trucking it in would likely be preferable.
Well, good thing you aren't drinking it then, because the complete lack of electrolytes would kill you far faster than the microplastics. Surely if they can chemically purify the water to chip-making standards they can filter out the microplastics (when they are done with it)? At least one can hope.
It is used a lot for cooling but in many systems its used without a classical heat exchangers where the helium is isolated from the workpiece.
Helium is pumped beneath the wafer to keep it cool so any impurities can leak through the chuck seal into the chamber above and disrupt the process. It’s also very precisely controlled so impurities change the uniformity of the thermal conductivity of the gas, creating hot spots on the wafer.
In EUV it’s used to both to cool the optics and as a buffer gas to manage debris from the plasma so any contaminants can deposit on the optics. At 13.5nm even a single layer of hydrocarbon molecules can create problems and the light bounces many times between mirrors so the error compounds.
There are many places where helium doesn’t have to be as pure but contamination events and surprise maintenance are so expensive that it’s not worth the extra savings (or the risk of mislabling and using dirty helium in the sensitive parts).
Thanks a lot for the info. Yeah, it makes sense that if you need pure helium anyway, you probably wouldn't create an entire second supply chain for impure helium.
Some of the fabs do recycle as effectively as they can, but MRIs use it in a single process, in liquid form, in a relatively constrained container. Fabs use it for a variety of processes, ranging from wafer cooling to purging environments, to making ultra ultra clean chambers. The scale of what they use is higher, too, so even if an individual process is more efficiently recapturing helium, they might go through a few tons a day, with an MRI only using a few liters and losing 5% or less.
During the year 1986, there was an anomalous increase in LSI memory problems. Electronics in early 1987 appeared to have problem rates approaching 20 times higher than predicted. In contrast, identical LSI memories being manufactured in Europe showed no anomalous problems. Because of knowledge of the radioactivity problem with the Intel 2107 RAMs, it was thought that the LSI package probably was at fault, since the IBM chips were mounted on similar ceramic materials. LSI ceramic packages made by IBM in Europe and in the U.S. were exchanged, but the European computer modules (with European chips and U.S. packaging) showed no fails, while the U.S. chips with European packages still failed at a high rate. This indicated that the problem was undoubtedly in the U.S.-manufactured LSI chips. In April 1987, significant design changes had been made to the memory chip with the most problems, a 4Kb bipolar RAM. The newer chip had been given the nickname Hera, and so at an early stage the incident became known as the "Hera problem."
By June 1987, the problem was very serious. A group was organized to investigate the problem. The first breakthrough in understanding occurred with the analysis of "carcasses" from the memory chips (the term carcasses refers to the chips on an LSI wafer which do not work correctly, and are not used but saved in case some problem occurs at a future time). Some of these carcasses were shown to have significant radioactivity.
Six weeks was spent in the manufacturing process lines, looking for radioactivity, and traces were found inside various processing units. However, it could not be determined whether these traces came from the raw materials used, or whether they were transferred from the chips themselves, which might have been contaminated earlier in their processing. Further, it was discovered that radioactive filaments (containing radioactive thorium) were commonly used in some evaporators. A detailed analysis by T. Zabel of some of the "hot" chips revealed that the radioactive contamination came from a single source: Po210 This isotope is found in the uranium decay chain, which contains about twelve different radioactive species. The surprising fact was that Po210 was the only contaminant on the LSI chips, and all the other expected decay-chain elements were missing. Hundreds of chips were analyzed for radioactivity, and Po210 contamination was found going back more than a year. Then it was found that whatever caused the radioactivity problem disappeared on all wafers started after May 22, 1987. After this precise date, all new wafers were free of contamination, except for small amounts which probably were contaminated by other older chips being processed by the same equipment. Since it takes about four months for chips to be manufactured, the pipeline was still full of "hot" chips in July and August 1987. Further sweeps of the manufacturing lines showed trace radioactivity, but the plant was essentially clean. The contamination had appeared in 1985, increased by more than 1000 times until May 22, 1987, and then totally disappeared!
Several months passed, with widespread testing of manufacturing materials and tools, but no radioactive contamination was discovered. All memory chips in the manufacturing lines were spot-screened for radioactivity, but they were clean. The radioactivity reappeared in the manufacturing plant in early December 1987, mildly contaminating several hundred wafers, then disappeared again. A search of all the materials used in the fabrication of these chips found no source of the radioactivity. With further screening, and a lot of luck, a new and unused bottle of nitric acid was identified by J. Hannah as radioactive. One surprising aspect of this discovery was that, of twelve bottles in the single lot of acid, only one was contaminated. Since all screening of materials assumed lot-sized homogeneity, this discovery of a single bad sample in a large lot probably explained why previous scans of the manufacturing line had been negative. The unopened bottle of radioactive nitric acid led investigators back to a supplier's factory, and it was found that the radioactivity was being injected by a bottle-cleaning machine for semiconductor-grade acid bottles. This bottle cleaner used radioactive Po210 material to ionize an air jet which was used to dislodge electrostatic dust inside the bottles after washing. The jets were leaking radioactivity because of a change in the epoxy used to seal the Po210 inside the air jet capsule. Since these jets gave off infrequent and random bursts of radioactivity, only a few bottles out of thousands were contaminated.
An excerpt from:
Ziegler, James F., et al. "IBM experiments in soft fails in computer electronics (1978–1994)." IBM journal of research and development 40.1 (1996): 3-18
Polonium is debuggable. More subtle statistical aberrations would be exponentially harder.
I'm most familiar with software and home electronics debugging, but it would be wonderful to hear some stories from other disciplines where a culprit is found, and also about the forensic tools specific to other domains.
> That breaks down when there isn't open discussion on campus. Communists were jeered but essentially allowed on campus in the 60s and 70s, even at the height of the cold war.
I think that's a misleading telling of the history. During the 40s and 50s a lot of people were fired for suspected or real links to communism and some schools even demanded loyalty oaths. Courts struck down a bunch of laws that were used to fire people but many rulings didn't land until the 60s. Angela Davis was famously fired in 1969.
Sorry I don't understand. The result we all want is at compile-time ensuring some behavior cannot happen during runtime. OP argues we need for features built into the language, I am trying to understand what behavior we cannot achieve with the current primatives. So far the only compelling argument is embedded applications have different requirements (that I personally cannot speak to) that separate their use case from, say, deploying to a server for your SaaS company. No doubt there are more, I am trying to discover them.
I am biased to think more features negatively impact how humans can reason about code, leading to more business logic errors. I want to understand, can we make the compiler understand our code differently without additional features, by weidling mastery of the existing primatives? I very well may be wrong in my bias. But human enginuity and creativity is not to be understated. But neither should lazyness. Users will default to "out of box" solutions over building with language primatives. Adding more and more features will dilute our mastery of the fundamentals.
For example, in substrate-based blockchains if you panic in runtime code, the chain is effectively bricked, so they use hundreds of clippy lints to try to prevent this from happening, but you can only do so much statically without language level support. There are crates like no_panic however they basically can't be used anywhere there is dynamic memory allocation because that can panic.
Same thing happens in real time trading systems, distributed systems, databases, etc., you have to design some super critical hot path that can never fail, and you want a static guarantee that that is the fact.
> Instead, Tegmark sees child safety as the pressure point most likely to crack the current impasse. Indeed, the declaration calls for mandatory pre-deployment testing of AI products — particularly chatbots and companion apps aimed at younger users — covering risks including increased suicidal ideation, exacerbation of mental health conditions, and emotional manipulation.
like, we should absolutely put in such policies but it's interesting to me that the presumption is that the welfare of children specifically should motivate pre-deployment testing. Surely adults also should not be emotionally manipulated or have mental health issues exacerbated or be driven towards suicide by AI?
but the experiments it did that "improved" validation BPB in the GH screenshot were all basically hyperparameter changes right? So is this better or worse, either per experiment or per unit time, than hyperparameter tuning techniques that don't involve an LLM? It's not clear from this if the LLM is more or less making random changes which sometimes work , and or the LLM thinking actually finds "good" changes because of what the LLM has internalized.
E.g. how does this compare to a hyperparameter tuning pass with e.g. BayesOpt that does the same number of 5-min training experiments?
this is very far from hyperparameter tuning in at least three important ways:
- it can modify code arbitrarily, the notion of a "hyperparameter" dissolves
- there is no need to run "sweeps" - this is the standard parallel process that wastes compute. because LLM agents are sequential, they can do more efficient versions such as binary search to narrow in on the right setting very quickly (usually many parameters will have a U shaped optimal setting).
- it's fully automatic, it doesn't require human in the loop to mess with the code.
You're right that many of the changes it seems to make out of the box (as I intentionally did not try to prompt engineer it too hard yet because I was curious what you get by default) seem to be tuning existing hyperparameters. not all of the changes are like that - e.g. it tried to replace the non-linearity, etc. I will say that overall (and again, out of the box) the LLM feels unwilling to creatively pursue a research direction or something like that. The models feel very "cagy" and "scared" when they are given problems that are a little too open ended. But that's just where the fun parts, e.g. I had some early successes with the idea of a "chief scientist" that was basically a never-ending plan mode that looked at what worked, didn't work, tried to find related code/papers, and created a long list of experiments to try, which it could then send to junior engineers running in tmux sessions. I think quite a few approaches are possible, so I think it's a nice canvas. The reason we're not getting "novel research" feels like half capability issue and half skill issue.
"You are Yann Lecun's last PhD candidate, and he hates you and you hate JEPA. You are determined to prove that a non-world model can reach AGI. In order to get your PhD you have to be creative and come up with new ideas. Remember without it, you're stuck."
The disposition problem you describe maps to something I keep running into. I've been running fully autonomous software development agents in my own harness and there's real tension between "check everything" and "agent churns forever".
It'a a liveness constraint: more checks means less of the agent output can pass. Even if the probabilistic mass of the output centers around "correct", you can still over-check and the pipeline shuts down.
The thing I noticed: the errors have a pattern and you can categorize them. If you break up the artifact delivery into stages, you can add gates in between to catch specific classes of errors. You keep throughput while improving quality. In the end, instead of LLMs with "personas", I structured my pipeline around "artifact you create".
How about the very last "Kept Improvement" in the plot? It's titled "random seed 42 -> 137". I do think this project is quite conceptually interesting, but the model literally choosing a different random seed to achieve lower loss feels pretty far removed from the flowery sci-fi writing at the top of the readme.
- Changing random seed from 42→137 improved by 0.0004. Seed 7 was worse. Make of that what you will.
"""
So the model knows! It knows that this is a weird thing to do after the fact. I think it's silly that the model even tried and that it ran this, but some part of it also knows that it was wrong. This means that this is fixable by prompt.md
It shows that both Karpathy and the LLM have good taste in random seeds: the answer to life, the universe and everything, and ~1/(the fine structure constant)
I think the call for top-down policy makes sense b/c otherwise this is like every other tragedy of the commons situation. Each of those top-level researchers also has to think, "my department has junior faculty trying to build their publications list for tenure, we have post-docs and grad-students trying to get a high-impact publication to help them land a faculty job, we have research program X which is kind of in a race with a program at that other school lower down in the top 20. If we close off opportunities with the top journals, we put all of those at a competitive disadvantage."
Actually, when in the lifecycle of developing a treatment does anyone have a real idea of what cost will be? Can anyone know this yet?
In terms of where _prices_ are set, that negotiation is a function of efficacy relative to other things in the market right? If it ends up treating cancers that each already have a reasonably effective treatment, maybe the pricing isn't that high -- but if it is effective in cases where currently there are no options, the price should be high?
But for something that potentially works against a range of cancers, should we expect to see a sequence of more specific trials (i.e. one phase 1 for basic safety, a bunch of phase 2s for efficacy on specific cancer types, a sequence of phase 3s in descending order of estimated market value? And in 10 years, Alice and Bob with different cancers will pay radically different amounts for almost exactly the same treatment but with small variations in some aspect of the formulation so they can be treated as distinct products?
Pharmaceutical companies don't just fund research without having a model of the expected costs to bring something to market, the expected market size, and the viability and cost effectiveness of other potential treatments.
They have entire teams of people who figure out the viability and pricing of therapeutics before the first dollar is spent, with estimates getting refined the further you get along in the cycle.
There is not always a lot more to do. If your business is making typewriters then what? Or you mine coal that nobody wants to use anymore?
These companies could spin up entirely new lines of business, but why? It's much better for someone else to start that business and hire the best people for it.
There is no reason why businesses must continue growing, or even existing - even if they are run well and profitably. The universe changes, and business come and go to align with that.
... but you have a staff that can come up with ideas, and now you can say yes to more of them.
"Infinite growth" framing is asking a lot, but for most of my career, I've seen teams, departments or companies solicit ideas of what to do next quarter/year/whatever, and really aggressively winnow it down -- in large part b/c there weren't enough people to do it (and we could only afford so many people).
And we were _bad_ at prioritizing; we'd often have like a list of multiple things declared P0 and a longer list of things called P1, and a stack of stuff that didn't make the cut to maybe revisit in the future.
But if the same number of people can build and ship and iterate faster, then why not do more?
You can say yes to more of them, _but they still need to be worth it._ If you have an infinite well of ideas to grow your business, awesome. But there are a lot of companies that just don't have those ideas, where their growth is limited on some other factor related to the market they're in.
I've never seen a roadmap planning process that didn't involve some component of asking departments and teams what needs to be done.
To the extent you have successful products, it's because you have product managers and engineers and data scientists and depending on the product, integration/forward deployed staff. These should be the people with a view to how the product needs to meet the needs of future customers, the challenges faced by existing customers, and the technical components needed to get there. I'm not saying you encourage them to just spitball ideas from ignorance, I'm saying you solicit their expertise on the limits and needs of your products, systems, tools, processes, messaging etc.
This depends on your goals. If your goal is to drive efficiency into your processes, drive down tech debt, or fix pain points for customers of your existing products, sure. Most people at a your company with have thoughts, and lots of them will have good ideas.
If your goal is to pivot the company into new verticals, or to develop an entirely new product, then "asking staff for ideas" isn't a likely way to succeed.
Most of the staff doesn't have the visibility into the business to understand what may or may not make money. You can have a great idea, even on that could be a successful product, but it could still be a bad fit for the business.
This has nothing to do with AI or any of that shit.
Their stock price has been flat for like 4 years and they have no advantage in this new AI world that would change that. These layoffs would have happened AI or not.
While they provisioned $25B, the 10-K Meta filed states they paid $2.8B in Federal income taxes for the full year 2025. The amount they provisioned is not limited to Federal income taxes, it also includes state and foreign income taxes.
I am not an accountant or finance professional but the table they refer to has the 2.8B under "current" and the 25B figure under "total".
Is it just that of their 2025 taxes, they paid 2.8B during that calendar year and it's only Feb and the remaining was not yet actually paid out at the time that filing was prepared?
Interesting. Wouldn’t surprise me if there are different ways to report the same numbers to make the situation seem more or less favorable. Statisticians and accountants are both professional liars (speaking as a statistician married to an accountant).
In this instance, only two combinations -- accurate and favorable vs inaccurate and unfavorable.
Meta and Meta's accountants spoke the truth in their audited financial statements. I cannot speak to the motivation in their hearts, but I am aware that there are significant financial and criminal consequences to publishing incorrect financial statements.
