Editor’s Note: The following is a short excerpt from an article originally published on the author’s Substack Purpose and Desire on October 16, 2024. With edits to fit MTC’s style, it is crossposted here with permission.
I have a recent publication in Minding the Sciences concerning the “irreproducibility crisis”—also variously named the “replication crisis” or “reproducibility crisis.” The crisis, however, it’s named, was precipitated by a landmark paper by UCLA’s John Ioannidis, who examined several thousand papers in the biomedical sciences for how well they measured up to the supposed gold standard for good science: whether the results can be replicated. Very few papers lived up to the standards, and Ioannidis concluded that, in his words, “most published research findings are false.” As I wrote in Minding the Sciences, that “put the cat amongst the pigeons.” To quote from the article:
“Nature magazine devoted a special issue to the problem, which included several mea culpas: yes, we have sinned and probably we could do better. Ecologists have chimed in to admit that ’82-89% of ecological research … has limited or no use to the end user.’ OK, good to know, taxpayers might say. Not to worry, taxpayers, henceforth we should embrace preregistration to ‘reduce research waste.’ Not so fast, say others. What’s the best use of our time and resources: replicating garbage results or tending to our work? Still, others have asserted that the irreproducibility crisis is no big deal, and attention to it can be a diversion from performing ‘robust and efficient science.’ Psychology and behavioral sciences have come down solidly—perhaps defensively—in favor of replication and pre-registration, and—reproducible?—data back them up. Replication is better, for example, than the common alternative of ‘meta-analysis,’ which takes many studies and applies statistical magic dust to tease out something sensible. ”
The Minding the Sciences article was prompted by a high-profile retraction of a paper in Nature Human Behavior that supposedly demonstrated that replication works, and it is good. Unfortunately, for that message, NHB retracted the paper because it failed to follow the protocols that are supposed to make replication work. Oh well.
There’s a deeper context to this story. I wasn’t able to flesh it out in the Minding the Sciences article, so I’ll do so here. This is a bureaucratic solution to a cultural problem. As such it will either be ineffective or do active harm to the thing it is intended to protect.
The cultural problem is that science is presently governed by a landscape of perverse incentives that both encourages and subsidizes disintegrity. If there is disintegrity in science—and I don’t doubt that there is—it will not be solved by looking for examples of it and punishing the perpetrators. Finding examples is virtually guaranteed because disintegrity is built into modern science. It will only be solved by addressing the perverse incentives that encourage it.
Stephen Turner and Darryl Chubin have outlined the cultural problem in their provocative essay The Changing Temptations of Science. Where science once was governed by an ethic of discovery, it is now governed by an ethic of production. With the transformation has come a shift in the ethical norms that govern scientists’ careers. In science governed by an ethic of discovery, the ethical norms are the so-called Mertonian norms, encapsulated in the acronym CUDOS (Figure 1).
In a science governed by an ethic of production, in contrast, the ethical norms are encapsulated in the acronym PLACE. PLACE is where science presently sits. Science is a competition, in which the currency of career success is governed by numbers: of grants brought in, of new scientists produced, and of papers published. It is no wonder that the voluminous scientific literature—460,000 published annually—is filled with unreadable, unread, superfluous, and yes, dishonest, publications. If we are to restore integrity to the sciences, returning science to an ethic of discovery is the obvious course of action to follow.
Read the remainder of the article here.
Image by Kiattisak — Adobe Stock — Asset ID#: 928007195
John Ionnaidis? At least get right that he is at Stanford, not UCLA. But Ioannidis of the “irreproducibility crisis” published some sorta shoddy stuff on covid, in my humble opinion, and more to the point, that of people in the field who qualify as epxerts.
But most to the point here, J. Scott Turner knocks a lot of work as “dreck” and its originators as “grifters.” But he does not himself appear to me to be any kind of giant of science.