> “The end result is positive for the scientific community,” Mur said. But because the editors seemed to address the issue he raised only after he and his colleagues submitted a comment article, and “either did not read or did not understand the response to our comment,” he said the experience “made me doubt the integrity of the editorial process at the journal.”
I think one issue we have is that authors of a paper shouldn’t be forced to provide “tech support” for life under the sword of retraction. I try to reimplement analysis and algorithms in the literature all the time, and it’s hard to get it right. I don’t want to write to the authors for help without trying my best first. And I don’t get to write to the editors complaining about no response unless I’m thoroughly convinced of an error.
That’s why I think the comment article triggering a real investigation is the right process. It showed that Mur put enough effort into his replication effort by thoroughly describing the process.
The final exchange with the authors was confusing to me, I don’t get what happened. But I do think Mur was right to be frustrated at that moment.
I think another issue we have is that Mur had to put in roughly the amount of work needed to get a publication, without any public acknowledgment. He's a postdoc, who should be spending time on getting sufficient quantity/quality of publications to warrant tenure or tenure track somewhere. This entire process was of benefit to science in general and the journal specifically, yet they refused to give him even the (academically lesser) credits of a comment published in their journal.
To add some context: a postdoc position typically lasts 1 or 2 years. The investigation plus discussion with the journal easily took 4 months - at least a sixth of the good dr.'s time. And he's got nothing to show for it - he isn't even named in the retraction notice. In this case, it would have been trivial for the journal to do better... but they opted to do nothing, even after being notified of problems, until they received a thorough description. Which they then rejected.
TLDR: the journal's editors, likely all tenured academics, refused to take action when notified and refused to give credit once a postdoc spent months of their time.
After a fast search about the subject, what I understood is that the retracted paper had method problems. Some other place says that a meta-analysis (supposedly of papers with correct method/analysis) confirms similar findings, i.e. hearing loss management benefits for dementia.
> the editors seemed to address the issue he raised only after he and his colleagues submitted a comment article
Yikes. And the Lancet is generally regarded as a very reputable source. After reading the linked post I think I understand their initial defense, a little anyway.
As a social scientist, I'd question that. I have seen so much bad social science done in the Lancet that I groan when I hear the name. YMMV, because doctors are in general pretty dumb about social science; maybe it is a great journal in other fields.
Lancet was the journal to publish the vaccine and Autism link and the editor famously gave an interview during Covid about how they chose articles to advance certain issues.
But Jennifer Ann McCarthy and Oprah Winfrey were a lot more pivotal in starting this problem with real and avoidable children deaths more so than a fraudulent RCT from a now unlicensed charlatan.
I believe it is important to remember people's contribution to problems and where they sat on which side of history.
Related this very recent article from JAMA Otolaryngology (Denmark study):
Cantuaria ML, Pedersen ER, Waldorff FB, et al. Hearing Loss, Hearing Aid Use, and Risk of Dementia in Older Adults. JAMA Otolaryngol Head Neck Surg. Published online January 04, 2024. doi:10.1001/jamaoto.2023.3509
Conclusions and Relevance: The results of this cohort study suggest that hearing loss was associated with increased dementia risk, especially among people not using hearing aids, suggesting that hearing aids might prevent or delay the onset and progression of dementia. The risk estimates were lower than in previous studies, highlighting the need for more high-quality longitudinal studies.
Some time ago I was offered to start using hearing aid, and I must say when I tried it I felt like something opened in my brain. So now I read this article where it says wearing hearing aid might rise the risk of ending up with dementia... and I'm confused, I thought hearing aid activates brain regions not used previously. I thought it's reasonable to assume wearing hearing aid is good for your brain, it's making me sad it might be the opposite...
From the article, describing the now-retracted paper:
“In people with hearing loss,” the authors wrote, “hearing aid use is associated with a risk of dementia of a similar level to that of people without hearing loss.” They proposed “up to 8% of dementia cases could be prevented with proper hearing loss management.
From the article, describing the replication work which led to its retraction:
Mur and his colleagues intended to build on the article with a related analysis on the same UK Biobank data. But when he couldn’t replicate the main findings, Mur scrutinized the paper more deeply. [...] Most notably, he found that hearing aid use did not correspond to a lower rate of dementia for people with hearing loss, as the authors reported. He found the opposite: among people with hearing loss, the dementia rate was higher for those using hearing aids.
Isn't there an obvious confounding variable here where probably the hearing-aid using population has bad hearing, and if their hearing aid was not updated then it's more likely they would develop dementia? So the results from the paper could technically be true for the subset of hearing aid users that don't go for checkups.
People who publish papers labeled as science studies need to take more care and responsibility as this can lead to life-changing decisions by public...
This is a common misunderstanding of how science works.
For life changing decitions people should wait until the study has been replicated a few times, and preferabely destiled to a general advice by the FDA or whoever is in charge.
Isolated papers are better than blogpost in allcaps, but they are not "setled" science.
The problem is that the press release of the universities and the report by science "journalism" present these single paper result as a definitive result.
So is hearing loss not associated with dementia or is it that hearing loss is associated with dementia and hearing aids might increase the chances for dementia?
I don't see this one retraction proves/disproves causality one way or the other; without looking at all other studies on the same thing.
It does however prove that The Lancet isn't very scrupulous about forcing authors to retract even when a paper is discredited, or properly crediting third-parties with discovering serious errors (rather than fogging things to suggest the authors voluntarily initiated the retraction, which they didn't).
First, as sibling comment stated: unclear. A thought-to-exist correlation was shown to be spurious.
Second: in my complete and utter layman's view, it is far more likely that the causation is the other way around. That is: dementia could come across as hearing loss for the affected individual (or their nearest). As such, people with dementia would be more likely to get hearing aids - and therefore people with hearing aids would be more likely to suffer from dementia than people without.
This is complete and utter speculation, but this line of reasoning makes more sense to me than the reverse (using hearing aids would aggravate dementia).
I am almost completely deaf on my right side, having lost hearing in two separate episodes. The first only took the upper frequencies, so I was able to wear an in-ear hearing aid to compensate. I absolutely hated wearing it and eventually stopped and just made do. When I lost the rest of the hearing, then I was a candidate for a bone anchored hearing aid and that is a completely different beast. I can wear it all day with zero discomfort since the processor sits behind and above my ear and there’s nothing in my ear. If you still have bone conduction function, for sure ask your audiologist about it!
If by any chance this was an Over The Counter brand bone-anchored hearing aid, can you specify which it is? I have a family member who is quite annoyed by his in-ear aid.
Sorry, it was not. There’s an internal component that you get implanted that takes the sound from the processor and sends it through the skull to your ear. The brand is a Cochlear Osia 2
Retractions can be healthy but in this case it seems to have taken a good deal of effort from another group, which isn't the best sign about the process of the original authors or the journal.
It's true there will probably be an increase in this kind of bad article retraction before we can see a decrease, and that wouldn't necessarily indicate an increase in bad science -- similar to certain crime statistics when reporting starts to be taken seriously. But I still think the article raises quite a few ongoing concerns with the current academic system, which does have capacity pressure/incentive problems to a much greater degree than previous generations.
I do agree retractions in general should be more normalized, as there are also plenty of cases where honest non-reckless mistakes were made, and sometimes even quality work that turns out to have issues in retrospect e.g. new information invalidates a previously common assumption. But I'm not sure any of that applies to this particular paper, and regardless the original authors apparently avoided addressing any of it.
The reasons for resistance are reasonably understandable - most science withstands the scrutiny. Think of the alternative, where a barrage of complaints can get almost anything retracted and require the original authors to jump through hoops again and again. In a world where there's huge political value in criticising science generally, and dismantling particular areas specifically, is this what we really want?
If "most" means "most published papers" and "scrutiny" means "can replicate" then no, it doesn't withstand scrutiny: most published research is wrong.
https://youtu.be/42QuXLucH3Q
(2016)
(the exact percentages of non-replicatable articles may be different in different fields)
This may be true in specific domains but does not stand across the board. Otherwise, we would not see any progression in the application of new discoveries in industry. It seems to me that the evidence of the last couple of centuries indicate that in many fields we are seeing 'progress' in the application of new knowledge.
EDITED to add:
What I mean by progress, is measurable changes in society e.g. the improvement of medical science leading to measurably better survival, longevity etc outcomes, or the improvement in chemistry, materials and solid-state science effecting significant change in electronics, communications, batteries etc.
We are at the all-time low when it comes to people trust in science and science related institutions. And the scientists have mostly themselves to blame for this sad state of affairs.
> And the scientists have mostly themselves to blame for this sad state of affairs.
Ehh, that's debatable.
There's been a decades-long campaign to malign science and science-related institutions by big tobacco, big oil, various religious sects, and political groups because science regularly gets in the way of those groups' profits/beliefs/legal maneuvers.
While scientists do regularly pick up a footgun, it's important to acknowledge the various powerful entities that want them distrusted and pay big money to further that goal.
I think one issue we have is that authors of a paper shouldn’t be forced to provide “tech support” for life under the sword of retraction. I try to reimplement analysis and algorithms in the literature all the time, and it’s hard to get it right. I don’t want to write to the authors for help without trying my best first. And I don’t get to write to the editors complaining about no response unless I’m thoroughly convinced of an error.
That’s why I think the comment article triggering a real investigation is the right process. It showed that Mur put enough effort into his replication effort by thoroughly describing the process.
The final exchange with the authors was confusing to me, I don’t get what happened. But I do think Mur was right to be frustrated at that moment.