Because experts with domain-specific knowledge tend to have better heuristics for this sort of thing than the average interested outsider. (They have to, or else they'd get nothing done except debunking flawed claim after flawed claim. Which might be worthwhile, but doesn't move the field forward.)
I've seen plenty of "theories of particle physics" that evidently look fairly plausible to non-physicists (and maybe even to non-specialists) that I can recognize immediately as crackpottery by my own heuristics (but that would take at least a weekend's work to convincingly demonstrate as such, if I wanted to invest that kind of time: I've done that, too). I'm not saying that anyone should take anyone's heuristic guesses in such cases as gospel (once or twice a century we might actually get a delightful surprise), but they can serve as a useful restraining influence if you're tempted to get excited about a headline.
An expert would be able to skim it, recognize the path of reasoning as something he's seen a before, and narrow down on the error quickly. Reasoning about something by how many "waves in the community" it has made is a deference of personal judgement to the social network an individual is embedded in, which is a useful shortcut when you are not an expert relative to your peers, but dangerous set of mind to operate in for any lengthy amount of time, as it can be the basis of cults and the like.
> An expert would be able to skim it, recognize the path of reasoning as something he's seen a before, and narrow down on the error quickly.
This is almost exactly equivalent to saying that an expert programmer should be able to determine whether an unfamiliar, complicated code base contains a bug by skimming the source. The hard bugs are going to be subtle.
Not at all equivalent, because code isn't meant to be human readable, but computer interpretable. Because of these different goals, computer code is actually quite a bit harder to read than a well structured academic paper. I think programmers have a lot to learn about logical structure from writing meant to be read by humans [1].
Note how many people on this thread seem to be understanding and discussion its contents, whereas if I were to post 13 pages of "unfamiliar, complicated code" that claimed to do something, I can't imagine anyone would help me debug it.
So then the answer is certainly "yes" with the same level of certainty that algorias is claiming. Large, complicated solutions are inherently likely to have subtle flaws.
I agree, but not all bugs are subtle. This seems to be the equivalent to someone having trouble with "hello world" because they forgot to include iostream.
I think debunking crackpottery is helpful for society, at least. I’d even argue that it helps the specific scientific field in question to move forward if the claimer is an active researcher, and is losing their time by not realizing it’s wrong. To sceptically scrutinize claims is what moves us forward. Of course, one has to choose their battle, but if those, who can, debunk crackpottery from time to time, then, I think, the general picture of science gets better. I think public interaction is important for science — the constructive lectures of what we know aswell as the destruction of false claims.
Because even the superficial analysis provides more than zero useful information. (Nothing super-obviously wrong. Nothing that looks clever enough to overcome the guess that, none the less, something's probably wrong.)
