Science has a trustworthiness problem. Public trust in science, scientists, and in the worthiness of scientific research for society, has been on a steady decline since 2019, according to Pew Research Center.
To be frank, “science” is lucky its trustworthiness problem is not worse, because the public has long been unaware just how deep the rot goes. Take the mordantly named scientific “literature.” In 2022, U.S.scientists published roughly 460,000 scientific papers—Chinese scientists published roughly 900,000. Yet, an alarming proportion of those papers are never read, do not produce reliable results, and have no measurable effect on our scientific understanding of the world. If the public —who funds the bulk of scientific research—knew the full extent of this, trust in science would decline roughly to the level of those master grifters, members of Congress.
The foremost question for anyone who values science should be how to make it trustworthy again. There is no shortage of ideas. Early in its term, for example, the Biden administration set up new procedures for reporting and adjudicating scientific misconduct. How are they working out? Like any bureaucratic program, government oversight of scientific misconduct is proving to be rife with incompetence, cluelessness, inefficiency, and waste, coming nowhere close to tackling the likely scope of the problem. Like all mindless bureaucracies, government oversight of science is also prone to brute-force solutions like criminalizing scientific misconduct.
On their part, scientists have been making efforts at self-policing.
There has arisen a cottage industry, for example, of independent researchers taking the role of sniffing out scientific misconduct upon themselves. These have been notable mainly for destroying careers over seemingly small shortcomings in published work, as in the defenestration of Stanford president Marc Tessier-Lavigne.
More positively, many scientists post “preprints” on sites like bioRXive prior to submission to a journal, adding a layer of peer scrutiny of papers before they get into print in the hope that problems can be unearthed and corrected quickly. More formal efforts like The Open Science Society promote reforms such as preregistration of hypotheses prior to an experiment to guard against post-hoc adjustment of hypotheses to fit the data.
Looming large in all these discussions is the “irreproducibility crisis”, which can be summarized thus: if reproducibility of experiments is the gold standard of scientific inquiry, why is there so little of it?
Nearly twenty years ago, John Ioannidis argued that common flaws of experimental design, sampling protocols, and poor analysis make most published research impossible to replicate, and this is why, in his words, “most published research findings are false.”
Ioannidis’s assertion certainly put the cat amongst the pigeons. Nature magazine devoted a special issue to the problem, which included several mea culpas: yes, we have sinned and probably we could do better. Ecologists have chimed in to admit that “82-89% of ecological research … has limited or no use to the end user.” OK, good to know, taxpayers might say. Not to worry, taxpayers, henceforth we should embrace preregistration to “reduce research waste.” Not so fast, say others. What’s the best use of our time and resources: replicating garbage results or tending to our work? Still, others have asserted that the irreproducibility crisis is no big deal, and attention to it can be a diversion from performing “robust and efficient science.”
Psychology and behavioral sciences have come down solidly—perhaps defensively—in favor of replication and pre-registration, and—reproducible?—data back them up. Replication is better, for example than the common alternative of “meta-analysis,” which takes many studies and applies statistical magic dust to tease out something sensible. The data say so! A recent article in Nature Human Behaviour showed that due attention to replication and pre-registration can actually produce replicable and robust findings. Phew! “What reproducibility crisis?,” crowed a news article in Nature reporting on the article’s findings. The celebration was premature. About a year after the paper was published, NHB retracted the article, citing a “lack of transparency and misstatement of the hypotheses and predictions [the study] was designed to test,” as well as shortcomings in preregistration: all the shortcomings that plague the scientific literature at large, in other words.
What went wrong?
Stephanie Lee perceptively explored this question in The Chronicle of Higher Education, detailing how even the best of intentions – promoting transparency and reproducibility in scientific research – quickly degenerated into prejudice, misinterpretation, crossed signals, befuddlement, and fear of retaliation. Shucks, scientists are flawed human beings, in other words, just like the rest of us. Scientists can sometimes be “knuckleheads,” to use a current term of deflection. Hardly adequate to address science’s trustworthiness problem.
The uncomfortable truth here is this: the scientific literature is filled with irrelevant dreck. Most of it churns below the surface out of sight, and sometimes it rises to outright fraud and deliberate misrepresentation. This is not due to some failure of policy or procedure. Rather, the literature is filled with dreck because we extravagantly subsidize its production. Nearly $90 billion is shoveled into academic research annually, the majority of it from the government. Since 1953, public funding for science began to take off, and governments have poured roughly a trillion dollars into research. These unprecedented expenditures have not produced more science, as they were intended to do. What they have done, rather, is fundamentally transformed the culture of science. Stephen Turner and Darryl Chubin put it this way in their provocative essay, “The Changing Temptations of Science.” Whereas science had once been dominated by an ethic of discovery, it is now governed by an ethic of production, much of it driven by intense competition for government money. Scientists are now judged by numbers: numbers of papers published, numbers of students graduated, numbers of grant monies brought in. As measures of production have come to dominate scientific careers, new knowledge has become largely irrelevant to scientists’ careers. No wonder science is in crisis.
If science has an integrity—replicability, reliability, truthfulness, fill in blank—crisis, it will not be solved by ever more stringent policies and procedures. It can only be solved by addressing the perverse incentives built into the “ethic of productivity” problem. The solution will be painful: the entire science ecosystem, from universities to funding agencies and to Congress to the academic publishing industry, everyone is in on the grift. Taking away the network of perverse incentives will mean taking away the money incentivizing the production of dreck. Everything else is chipping away at the margins.
Image by MarekPhotoDesign.com — Adobe Stock — Asset ID#: 387186655
I say again, a few Federal false claim prosecutions and a lot of this will disappear, and I fail to see why the False Claims Act shouldn’t apply as most of this has Federal funds involved.
A better solution would be to eliminate our modern emphasis on research and go back to an earlier era where the emphasis was on undergraduate teaching.
We were at war* for fifty years — 1941-1991 — and the American academy transformed itself from teaching undergraduates to doing research to help defeat first the Nazis and then the Soviets. It wasn’t just the atomic bomb but a thousand other things ranging from welding to food preservation to radar to high octane gasoline that allowed airplanes to cross the Atlantic, even if they did have to stop for fuel at both Gander and Shannon after having left Maine with full tanks.
The social sciences came to play during the Cold War when we had to convince first Europe and then Africa & Asia that Communism wasn’t the answer. The 1965 Higher Ed Act didn’t help because it guaranteed undergrads without any evaluation of value added, and hence — not surprisingly — the era of “publish or perish” arrived.
But the war ended 30 years ago…
No matter how much Federal money a Harris Administration may chose to throw at Higher Ed (God help us), there is a demographic bomb that detonates in the Fall of 2026 when the babies not born in 2008 won’t be arriving as freshmen. Throw in both students and families now questioning what they are getting for their tuition dollar and I can see some rather ruthless competition for students coming in the near future.
Forget the fact that most of this academic literature is never read, forget the related fact that most of it is written in academic dialects so field-specific that it is incomprehensible to anyone outside the narrow field the author is in — and often then to those in the same philosophical school as the author**. And forget the fact that much of the academic literature is poorly written from a strict grammar, usage and style perspective.
Forget all of this — the simple fact is that Johnny is going to show up at University A rather than University B because of what he perceives to be a better student environment. Much like a century ago, institutions will be competing with each other on the basis of the quality of the undergraduate education they provide to undergraduates.
Neither he nor his parents will care how close Professor Jones is to inventing a room-temperature superconductor — it’s going to be how well whoever is teaching his class explains the concepts of electrical resistance and transmission loss.
Hence the crisis of science may fix itself — while the research overhead is nice, it’s the bodies in the classrooms that fund the universities and only those institutions that care about that will survive.
______
* The classic 1943 Norge Refrigerator ad showing how much the country had transformed to the war effort: https://i.redd.it/51pvqmaf58591.jpg Copper was needed for bullets so 1943 pennies were made out of steel (and often confused for dimes.)
** The field of education is notorious for this — I’ve seen concepts described with completely different words and phrases to the point where I had to find out what those words and phrases meant before I realized that the author was discussing something that I was quite familiar with.