Editor’s Note: This piece is part of an ongoing series of articles by Professor Bruce Gilley. To read the other articles in the series, click here.

In 1980, two doctors from the Boston University Medical Center published a five-sentence letter in the New England Journal of Medicine noting that only four of their 11,882 patients prescribed opioids developed an addiction. They concluded that “the development of addiction is rare” when opioids are prescribed. They provided no evidence and did not caution that their letter referred only to in-hospital applications rather than personal prescriptions.

From then until 2017, the letter was cited 439 times in scientific literature to support the claim that addiction was rare in patients given opioids, according to a 2017 analysis by University of Toronto scholar Pamela Leung and colleagues. This one junk citation, they believe, was a contributing factor to the opioid epidemic in the United States. “The crisis arose in part because physicians were told that the risk of addiction was low when opioids were prescribed for chronic pain,” they wrote. The letter “was widely invoked in support of this claim.”

While not usually so deadly, junk citations are widespread in academic research. The term refers to a citation that is wrong, irrelevant, misleading, corrupt, uninformative, useless, or purely rhetorical. At root, junk citations have become a way to signal the scientific basis of a claim without actually explaining what that basis is, a lazy shortcut that has turned millions of academic minds to mush. In the words of Ole Bjørn Rekdal, a Norwegian anthropologist who has written extensively on junk citations, “At times, I get the feeling that references have been placed in quantities and with a degree of precision reminiscent of last minute oregano flakes being sprinkled over a pizza on the way to the oven.”

Such straightforward citational errors are the core of the problem, which I discussed last year in a talk that can be viewed online. The problem existed long before the Internet and citation software, but these tools made it exponentially worse. Today, much credentialled research is built upon a cannonade of junk citations, which forces the reader to submit in the face of an overwhelming bombardment. As the scientific foundations of published research have been weakened by junk citations, the undisciplined ideologies and agendas of researchers have been given freer rein.

In the first part of this series, I discussed a citation in a grant application that the applicant had not read, had not understood, or had wanted to pass off as scientific despite its obvious flaws. Such a failure to tell the reader anything about a cited work is the cardinal sin on which many junk citations rely. It violates the fundamental rule, laid down by Wayne Booth and colleagues in their widely-used The Craft of Research, that anytime we cite the work of others, we must tell the reader what research they did, what their findings were, and what, if anything, limits these findings’ relevance to the current argument. “Don’t accept a claim just because an authority asserts it,” they warned (on page 87 of the third edition published in 2008).

[Related: “How Junk Citations Have Discredited the Academy: Part I”]

There is nothing wrong with providing the reader with a set of “further readings” or “related research.” But if citations are deployed as part of an argument, including an argument about what other research has been done on a topic, then they cannot be on a “Just trust me” basis. Reader beware.

Dozens of studies have shown that many citations contained in academic research are, to varying degrees, flawed. This could be as simple as getting the page number wrong. But more often, it involves misrepresenting or misunderstanding the source’s claims. In a 2017 exposé of bad citations in 472 articles in three peer-reviewed library science journals, Wilfrid Laurier University librarians Peter Genzinger and Deborah Wills found that 30% of the citations misrepresented the cited work, either wholly or in part.

Junk citations weaken not only evidence but also objectivity. They provide a bully pulpit from which scholars can assert that their claims are “well-documented” or “the overwhelming consensus” of the field. Critics of my 2017 article “The Case for Colonialism,” such as Ohio University’s Brandon Kendhammer, used fusillades of junk citations that, I showed elsewhere, were almost all incorrect. Kendhammer’s junk citation–based article is now cited by other ideologues. In their 2022 article demanding reparations from the West, Elise Klein and Elizaveta Fouksman cite it as “a good overview” of the “extensive” evidence of “the brutal legacy of colonialism.” Junk food leads to obesity, and junk citations lead to academic propaganda.

The truth is, we are all junkies. But there is hope. I recently published my first academic article that does not contain a single junk citation. The experience was liberating. I did not feel the need to take a shower afterward because my co-author and I knew that we had read and explained every source we cited and had used them in a way consistent with the findings and duly respectful of the authors. Rather than treating the work of others as doormats that we trampled upon while rushing to assert our own “contribution,” we carefully read and described it, putting our own contribution in a more modest light.

In future installments, I will show how junk citations have taken on darker purposes.

Image: Adobe Stock