I have a PhD in material science and worked with superconductors, here are two things that is often misunderstood:
1) it takes a undergrad to make and measure a superconductor but a Nobel laureate to explain it.
2) in theory if you understand the mechanism of something you can improve it. In material science however almost always all you need is a trial and error approach to end up with a good result.
Conclusion: the effect is probably true, the explanation not
> 1) it takes a undergrad to make and measure a superconductor by a Nobel laureate to explain it.
Another example of Taleb’s idea that experimentation comes first and theories second. Universities pretend that it’s the other way around but apart from an Einstein it’s usually not.
That's what I am thinking when commenters compare LLMs with humans. Humans got the experimental loop going through the real world, pure LLMs are brains in vats. They need experimental confirmation to become truly creative.
But on the other hand are the commenters who think AGI to ASI is going to take hours. When is there time to experiment anything? Even ASI needs real world confirmation to do research.
AGI to ASI might take hours if all that's needed is more compute. Such an intelligence might exponentially spread across the internet. Though I don't think it's the most likely scenario.
not really?, Dirac, Von Neumann, etc. taking a look at the history of the nobel prize, its typically the case that something is predicted by theory, then once an experiment is able to verify, then prizes are given out.
Good science has often proceeded from 'weird' observations, leading to experiments trying to isolate the weirdness. The weird is necessarily outside the zone of existing theory, and requires experimentation to recreate the effect reliably and quantifiably. Once that's done, you can iterate on conjectures and experiments to try to get to the bottom of what's going on.
In reality, I suspect that the divide between theorists and experimentalists is really only a fundamental physics thing. And physics has been more-or-less at an impasse for the last thirty years, so that there are relatively few experiments worth running, and the bulk of the theorists are just (making stuff/rebranding as mathematicians) up because they don't know what else to do.
In other areas, there's so much weird still untouched that you don't get the same division of labor. Take a look at CRISPR - there was a lot of bench work and curiosity-driven exploration involved, simply because there was no theory describing what they were discovering... Or machine learning - the theory is still quite tenuous and mainly follows the experimental results.
If we are sufficiently expansive in our definition of what theory is, then sure, there is no experiment without theory, and experimentalists only discover things by being secretly theorists who occasionally stop thinking long enough to get their hands dirty. But I don't think this is typically what is meant by 'theory.'
CRISPR is again a good example - the initial 'weirdness' was an observation of lots of long-ish palindromic subsequences in bacterial DNA. To my knowledge, there was no pre-existing theory on the preponderance of palindromic subsequences in DNA.
Now, we could say that these researchers were proceeding from a theory that there's no easily discernable combinatorial macroscopic structure in DNA sequences, but I think that this would stretch the idea of 'theory' beyond common usage or even usefulness: There's no theorem or axioms of DNA sequencing being violated here.
In fact, I would expect that weirdness is a good sign of missing theory - a repeatable observation which is unsupported by existing theory. For perturbations in the orbit of Mercury, we see an explicit violation of Newtonian mechanics, but in many other cases (like CRISPR and palindromic subsequences) we have observations of structure in areas where theory simply does not yet exist.
(Apologies for the harsh first sentence; FWIW, your comment seemed belittling of the work of a large fraction of important scientists.)
how exactly has physics been at an impasse? this trope is so common now. did you hear it from Weinstein? we've had a metric ton of mindblowing theory emerge in the past 30 years. do you even study physics?
I was careful to say fundamental physics - there's great stuff happening in materials science.
We are in a world where exponentially growing amounts of energy are needed to confirm/deny an increasingly small slice of the standard model. We still don't know what dark matter or energy is (almost literal holes in theory), we haven't figured out how to scale up quantum computers, and we don't have scalable fusion reactors. The cost of progress is growing and the rewards are diminishing; I call that an impasse.
(Arguably we could call the ongoing advances in materials science cases of applied quantum mechanics, with some blurry line with fundamental physics.)
You are right that I'm not a physicist - I'm trained as a mathematician, and these days work in the intersection of machine learning, acoustics, and ecology. Having looked around a lot with impact-colored glasses, I don't see the argument for fundamental physics, but am happy to be wrong.
none of the things you listed have anything to do with fundamental or theoretical physics. It also seems that you don't know about some of the recent advances. we don't need to deny the standard model. Nature has already shown us proof that it's not sufficient. but the standard model is not actually fundamental physics. It's basically just a mash up of a bunch of aspects of particle systems. There's, again, a ton of work on alternatives. and I would have to disagree, it would seem that we do know how to scale, quantum computers, but we're still doing the material science for it. Physics is probably far more advanced than almost anyone really realizes... and that is the bigger issue. We are still working out what advances to pay attention to. Yet what we do know about , if the engineering work were done, is enough to get us "free" energy, for example.
Call me when someone finally schedules a test to possibly disprove string theory. It's been stuck in "The math works but we haven't solidified it" for like decades. It makes testable predictions right?
We discovered first "crystal diodes" in 1870s. There was a whole idustry doing "magic needle+crystal diodes" without understanding why they work. They just knew you had to touch a special spot on the crystal with a needle. Some spots worked better than others. It was art not science.
We had the first theory on how they operate in 1920s. Eventually we got transistors from that theory in 1950s.
I used to think this way but it’s significantly more liberating and true to see theory as plausible explanations. Theories are very often afterthoughts, where the utility is prediction of new events, or narrowing the search space of experiments. A theory explaining past events only is not really a theory at all. But you can narrow search space in many ways without elaborate theories. For instance, you may assume that since birds can fly, if we make a machine that looks similar, we might be able to fly as well. Terribly simplistic theory, but nevertheless a good starting point.
Calling theories useful if they predict future events is called Instrumentalism and it’s a dead end if your goal is to understand reality.
“The magician will pull a rabbit from the hat” might reliably predict what happens, and be testable, but it is not a good scientific explanation because it doesn’t help anyone understand how the trick works.
While correct you're being downvoted because you're equivocating on the term "theory." People are assuming you're talking about a fully baked, documented theory couched in precise language and equations. What you're actually talking about is tacit assumptions / models that are built in observations.
"Theory comes first" is particularly unfortunate wording because it seems to imply the naive point of view that people first come up with a thorough mathematical model of some phenomenon before testing and validating it with experiments. That's obviously not how science has worked throughout history.
Saying "all observations/measurements imply an underlying theory" would have been maybe more accurate.
You are assuming that we have thought of all theories that an experiment can confirm, before conducting the experiment. Considering the large search space, that would be quite surprising.
Can you tell me why a scientist would decide to do a certain experiment without a theory about what to test and why? Why that experiment and not any of infinite others?
We do this much of the time in genetics, genomics, and neuroscience.
New technology;
Large set of samples;
Quantify x<1> … x<n> mRNAs, proteins, lipids, metabolites;
Estimate how units stick together (statistically or literally);
Develop a “theory” of what units and groups of units interact to account for and predict higher order phenotypes (risk of neurodegeneration; lifespan).
Data take precedence. Mini-theories of molecular and cellular causality are assembled with some basic brain power and yes—-a dollop of theory and priors—-on the back of a massive pool of well structured data.
Exploratory biology of this type is/was insulted using the terms “fishing” or “mere description” but with current high throughput and high content technologies should be considered research “trawling” and factory-level science; not a cottage industry of small labs. This new style of science can be highly effective in biology and in astronomy as we are learning from Webb.
But it bugs the hell out of some classically trained reductionists who demand that clear hypotheses should drive science forward.
Much of the progress in modern biology falls into this alternative almost hypothesis-free style. I would say “story-free” style of science. Too damn many story-tellers.
I wouldn't disparage that method of discovery at all. But some of the theories required to do it are:
- Many brute-force combinations of these specific organic molecules will yield interesting results (this is the main hypothesis of the experiment)
- We don't need to include the molecules that we haven't included
- We have good criteria for determining what results are interesting
- Our instruments/methods of phenotype prediction are well-understood and working as expected
Once the experiment is run, the observations are meaningless unless interpreted in the context of whatever prevailing theories the scientists have in mind.
Which properties to check? Why that sample and not countless others? Why not spend the rest of his life checking different properties of the one sample?
Why would someone do an experiment without a theory to test? How would they know that a particular result was interesting without a theory about the expected result?
For the wetter sciences, the answer is somewhere in between. We usually have somewhat of an understanding of the prior theory. We then throw a lot of stuff at the wall…. Oh, this sticks? Eh, why? And then usually some more clever person will come up with reasons for why X works beyond what we already knew. A few years later, the field reaches some sort of consensus around one of the hypothesis.
This is only true from the perspective of developing new theories for why are things happen. Of course new theories arise from other theories.
It's not true for Discovery in general. New phenomenon can be created and observed without any theory for why they occur, either before or after observation
You can't even observe something without theories. For example:
- I need to observe here and not anywhere else
- I can reliably interpret my senses/the instrumentation is working correctly
- Objects of this type normally behave in X way, because of Y
- etc
Think of it a different way. You can come up with a theory without any observation whatsoever. Black holes, for example, were conjectured well before they were observed.
Well sure, that's the point that you're trying to make. I thought anything most people felt you were claiming you need a theory related to the novel Discovery or convention. That is to say how it might work and what the expected outcomes are. I can't go to the lab without a theory that my car can get me there. That doesn't mean I have a hypothesis for what will happen when I mix two substances in the lab.
Thinking that something will either happen or not happen if I mix those two substances is not a theory.
Nothing, just words that don't have any predictive value or convey any understanding of the world.
Maybe an example would help clear things up. go into the lab with a 1 lb weight and a 2 lb weight and weigh them together.
Saying that the total weight could either equal 3 lb or any value other than 3 lb does not constitute a predictive theory for how the physics of summing Mass works.
It might be a theory that a scale display a value when I put things on top of it, but that is a different topic, and not what I'm testing.
This Theory doesn't tell me how the world works and if the expected value is 0 lb, 3 lb, or 1 million pounds.
I could go into the lab with no operating Theory or hypothesis on what the value of two masses should be when added together and collect data.
I can collect data with no expectation of correlation, and after measuring the combination of many weights, deduce that there is a relation between the combined weights and the total mass, and in fact it is a simple sum.
> Maybe an example would help clear things up. go into the lab with a 1 lb weight and a 2 lb weight and weigh them together.
> Saying that the total weight could either equal 3 lb or any value other than 3 lb does not constitute a predictive theory for how the physics of summing Mass works.
The relevant theoretical background here is hidden in the "weigh them together" step: that there is such a thing as weight, it's described by a single real number, you can measure it in such and such a way, and so on.
You don't notice these considerations when it comes to weight and speed and size because they're hardwired into our brains by evolution. We're not so lucky when it comes to, for instance, the quark mixing angles - we can't even conceive of them without a background theory, let alone start measuring them.
Im not making the claim that theory Never informs experimentation. I'm making the claim that it possible to make discovery without theory about what you are exploring.
If you want to count distant theories like "I exist" or "The world exists", then sure, every action starts with theory. But like I said, that is every different than a specific theory about what outcome an experiment has, and the underlying physics that make it so.
If you think nobody can discover anything without a theory for what could be discovered, you are flat out wrong.
Consider the story of an artificial sweetener being discovered by a chemist who didn't wash their hands properly and the bread they ate that night was sweet.
Are you suggesting this was only possible because they had a "theory" that bread is not sweet?
Only if it's significantly more than pyrolytic graphite, which appears to be the record-holder at the moment. It would have to be sufficiently strongly diamagnetic to levitate over ordinary magnets, which could have interesting applications. Think low-friction and low-vibration bearings, etc...
Also PhD Materials Chemist. Agree completely. I follow superconductor publications and this is the only RT claim that has made me tell my friends "this could be the one". Using an electron paramagnetic resonance instrument like a beach combing detectorist is too hilarious to not be true.
I read that the development of the alloy used in the oxi pre-preburner side of Spacex’s Raptor engine (the main blocker to full flow combustion) was a trial/error grind vs an “a-ha” moment. Material Science sounds rough.
> material science however almost always all you need is a trial and error
Is it science if you essentially fiddle something until you get a result you are after and then try to fit a theory that would explain the result of the fiddle?
It is if you do it in a way that's reproducible, and if the methods used are based on already-known science. It wouldn't be scientific if there's only one machine in the entire world that can get the desired effect and nobody can figure out why, or if one of the steps involves praying to Cthulhu.
1) it takes a undergrad to make and measure a superconductor but a Nobel laureate to explain it.
2) in theory if you understand the mechanism of something you can improve it. In material science however almost always all you need is a trial and error approach to end up with a good result.
Conclusion: the effect is probably true, the explanation not
edit: typo