Retracting publications doesn't stop them from influencing science
"Zombie papers" keep on getting cited, with huge ripple effects
Annie Spratt via Unsplash
What do studies on omega-3 fatty acids, hydroxyethyl starch, and COVID-19 have in common? The resulting scientific papers were all retracted.
Nevertheless, they have all been cited countless times since then. As an eraser of mistakes in scientific publishing, retraction discredits the validity of a study’s claims due to flawed methodology, biased interpretation, or even fabrication of data. Unfortunately, it seems it is not so easy to erase a published study from the collective scientific consciousness. A retracted study should be a relic belonging to the virtual museum of past science – but like zombies of the scientific publishing world, they are kept alive by continued citations.
A study in Science analyzed the post-retraction citations of two COVID-19 papers originally published in The Lancet and in The New England Journal of Medicine. As both studies were linked to the same dubious database as the source of their raw data, their retractions were followed by a widespread media scandal.
Despite all the attention these studies received, researchers at Science found that they have been cited more than 100 times as support for scientific findings without mentioning their retracted status. These incorrect citations showed up in even the most prominent scientific journals, such as PLOS ONE, the Proceedings of the National Academy of Sciences, and Nature Communications. Retraction does not seem to achieve its desired effect – in fact, it seems that in the long term, retraction does not have any effect on the number of citations a paper receives.
Science is influential. Since it's used to drive medical, policy, and scientific decisions, the consequences of citing retracted studies as credible science can be dire. It is essential that both authors and publishers take care to stop citing them.
Incorrect claims, stemming from the citation of a "zombie" paper, can ripple misinformation into many more new studies. Another recent study looked at the citation patterns of a paper on the use of omega-3 supplementation for COPD patients which was retracted 11 years ago due to fraud. The authors found that 96% of papers citing the study did not mention its retracted status and 41% did so while even describing the study in detail. At this point, the spread of misinformation seems inevitable, as even the authors who take care not to cite retracted studies will trust the information and continue to cite these non-retracted papers that do cite a fraudulent study.
Keeping flawed science alive can even be life-threatening – as in the case of the fraudulent studies of the infamous Joachim Boldt with 103 retractions tied to his name to date. He supported an intravenous therapy using hydroxyethyl starch to stabilize patients during and after surgery. In independent studies on the compound, its use was linked to higher patient mortality. Unfortunately, many of his studies are still being cited today.
As well as spreading misinformation and harming patients, flawed studies can have negative societal and policy consequences too. A recent article published in Nature Communications suggested that female mentorship in academia had detrimental effects on the careers of female mentees and attacked current diversity policies in academia. Following outrage from both scientists and the public, the study was shortly retracted. As well as erroneously equating co-authorship with mentorship, it also disregarded gender discrimination in science. However, if retraction cannot fully discredit the study as it should, the efforts to reach greater gender equality in academia will be damaged.
The root cause of the issue is often simple – many scientists would likely admit to having at least once copied relevant citations from another paper without reading them, just out of sheer convenience. Moreover, while the retraction notice is present on the publisher’s site, it often fails to appear on other platforms, such as PubMed, making the author believe that the study is valid.
This is a major glitch in the system. We are scientists, dedicating our whole lives to painstakingly pipetting or analyzing massive databases. We should not lack the diligence to pay attention to such an important and glaring detail like whether a paper was retracted. In each and every publication, author guidelines should include that the author is needed to check all citations for possible retractions. Today numerous citation software are available to do this with ease; such as Zotero, scite.ai, and RedacTek alert users for any retracted papers in the reference list. As well as more care from authors, preventing post-retraction citations is a responsibility of publishers too. Along with double-checking the reference list of papers to be published, they should also make sure that retraction notices appear on all platforms where the study is available.
Peer Commentary
Feedback and follow-up from other members of our community
Simon Spichak
Neuroscience
This is a fascinating piece! The retraction notice on the web page is relatively small. What do you think a reasonable solution would be for citing papers? The more papers we cite within our scientific paper, the greater the chance we make a mistake. It’s also really hard for a few people to carefully read through 200 different studies. Also, what to do with a published paper based on a previous study that’s only recently been retracted?
You can compare the retraction notices as they appear in the Lancet and in NEJM. The former is quite good, but I would easily miss it in the latter. As for your question, I think the software that I mention at the end of the article provide an easy and modern solution for checking all the references (and as an author, I think you should AT LEAST open up all the articles that you cite and have a look at the abstract and a possible retraction notice).
As for published papers that cite retracted studies, I think disclaimers should appear on them, noting that a certain reference has been retracted. It is a bit like weeding, and if you fail to do it there, it easily spreads. In the Science story that I mention, you can see how different editors think about this issue, the reactions are mixed…
Rebecca Lea Morris
Mathematics
Getting the word out about how poor citation practices spread misinformation and can negatively affect real people will hopefully help scientists to be more aware of the problem and motivate them to take extra care to avoid citing retracted papers. I did not know that Zotero could identify retracted papers and had never heard of tools like scite.ai or RedacTek that can do this too, so pointing out that these tools exist provides a useful, concrete, place to start checking citations. I think it’s also something that will be very helpful for non-scientists to be aware of, as retracted articles may end up available in other places on the web as discussed in this article (which itself was corrected) where non-scientists might read them, for example support groups for people with certain medical conditions.
Marnie Willman
Virology
University of Manitoba Bannatyne, National Microbiology Laboratory
This is a fascinating topic that is rarely discussed among scientists and laypeople alike, but needs serious attention. It has become a Mariana’s Trench of the field, hiding misinformation and keeping incorrect ideas and harmful inaccuracies circulating as if they were fact. I love the term zombie paper because this is exactly what they feel like - zombies that never do seem to go down when they should.
It has been my belief for quite some time, as you suggested in the article, that there should be a banner or large print declaration at the top of a retracted paper. This would clearly indicate to both the casual reader and researcher looking for accurate citations that the paper is no longer considered accurate, correct, or factual. It should be a rule that reference lists and appendices of future publications should not include retracted papers.
I think what you did here is a fundamental requirement in journalism and writing. You established an area that is clearly lacking the regulation and rigidity that we scientists are so proud of, and suggested ways in which it could be improved. When a retraction occurs, the paper should be gone. Rather like Andrew Wakefield’s retracted MMR Vaccine paper which continues to form the basis of anti-vaccine arguments globally, papers should not be publication zombies, forever sharing their ideas that have since been shown to be either inaccurate or just plain wrong. Whether the answer is using the databases you listed in the article or clear labels on retracted papers, there is one thing that we can all agree on … The current system is not working and must change in order for science to maintain the reliability and integrity that it is known for.
Sarah Heidmann
Fish Ecology
University of the Virgin Islands
For such a pervasive issue, retracted papers are not discussed enough and you are right to call attention to them.
The error of citing a retracted paper could be caught at so many points along the path to publication; the fact that they persist demonstrates a complete breakdown of the system.
I wonder if paywall-passing systems like Sci-Hub are contributing to the problem?
That’s a great question!
While personally I like what Sci-Hub stands for, it is possible that the retraction notices do not appear there. However, I don’t think that it is a major issue, because retracted studies are open access already, and I think most people first check if they can access a paper and only resort to Sci-Hub when they are blocked.
On the other hand, the fact that valid science is behind a paywall, but retracted articles become accessible could actually be a major contributing factor. It is easy to imagine why this would cause them to be cited more, seems like a backwards system to me.