Minding the Sciences—Science’s Goose is Cooked: Seven Pillars of Folly

The era of Big Science began formally in 1950, when the National Science Foundation opened its doors. Its mission was to fulfill a hopeful promise: for government to fund the very best academic science, to explore science’s “endless frontier,” in the inspiring words of Vannevar Bush, President Roosevelt’s—and subsequently President Truman’s— science czar.

There was a tradeoff, to be sure. There was a worry among scientists that government funding would compromise their independence, but the promise of better funding offset the concern. Congress was equally cautious: what would the public get for their money? Rest assured, science had served the country well during WWII, and making science a public good would sweeten the deal, even if it was not always clear what the result might be. And so, the bargain was struck.

Since then, the “NSF model” has been adopted by more than a dozen government agencies, making the federal government now the dominant funder of academic science, to the tune of roughly $50 billion annually.

From the beginning, the mainstay of the NSF model has been the “investigator-initiated” research proposal, which the NSF solicits semi-annually. Scientists, or teams of scientists, submit research proposals to explore what they think is a bright idea. Presently, less than one-in-four of the roughly 50,000 investigator-initiated proposals submitted to the NSF each year are funded. One must ask: how does the NSF separate the wheat—that “very best science” —from the chaff?

The NSF—and other government funders—make the judgment through a formal process of “merit review.” Outside scientists—typically three to five—are invited to comment knowledgeably on the science being proposed. These outside reviews are then discussed by panels of other scientists who then make recommendations to the NSF for which of the one-in-four proposals are worthy of funding.

Trouble creeps in because there is no real agreement on what is meant by “merit review.”

Early in NSF’s history, reviewers were asked to judge proposals on “scientific merit” and “competence of the investigator.” Since then, the NSF’s own guidance on merit review has shifted substantially. It’s changing still.

Since 1996, merit review has turned on two broad criteria: “Intellectual Merit”, and “Broader Impacts.” The Intellectual Merit criterion is close to the NSF’s original intent. The scientists submitting their grant proposals generally have no problem justifying it, and reviewers generally have no problem evaluating it. The Broader Impacts criterion is another story, usually leaving scientists and reviewers scratching their heads. Just what is a Broader Impact? How do we judge it or compare one proposal’s Broader Impacts against another’s? These are not easy questions to answer.

The NSF has doggedly been trying for decades to flesh out the idea, often prodded by Congressional pressure, but with little improvement in clarity.

So, Broader Impacts remains the bastard child of merit review, to no one’s satisfaction. Scientists give it little weight in their proposals. Reviewers generally return the favor. Both scientists and reviewers seem united in believing that Broader Impacts is mostly performance art and is not really to be taken seriously. The government that is picking up the tab still wants to know what it’s getting for its money, however, so the NSF soldiers on.

Since 1996, when the current merit review criteria were established, the National Science Board—the NSF’s governing body—has reviewed Broader Impacts several times in the hope that some clarity might be brought to bear. They are now reviewing it again and will issue a set of recommendations in May. However, the chair of the committee is being cagey about what their recommendations will be.[1] Inquiring minds should want to know: What’s going on?

Readers’ eyes may well have glazed over by now, but please read on, because something is troubling afoot.

The NSF’s hopeful founding 75 years ago marked the beginning of a radical experiment: that government should be the benefactor of basic research.[2] How has the experiment worked out? The experiment has been a remarkable success by the metric of spending money: government expenditures on academic research have grown exponentially since the 1950s. Federal, state, and local governments now constitute the majority partner in academic research, with the federal government being the Big Kahuna. This presents a marked contrast to the years before 1940 (Figure 1), when government expenditures on research were modest. World War II marked the beginning of a meteoric rise in federal support of science that has continued since.²

In terms of whether the experiment has accomplished its goals—funding the “very best” academic science—the record is mixed.

While American science continues to show remarkable strength—American scientists dominate in Nobel Prizes awarded, for example—it is not clear that this can be attributed to the exponentially rising government expenditures on academic research. A recent analysis of millions of scientific papers and patents indicated that the incidence of “intellectually disruptive” papers and patents has been declining exponentially since the 1950s, the inverse of the exponential increases of spending. [3] So, it’s not clear that the gusher of money being poured into academic science is fertilizing basic science as the NSF’s founders had hoped..

It remains an open question: is the NSF doing merit review right?

Figure 1. Expenditures on scientific research as a portion of Gross Domestic Product (“National Income”) from 1920 to 1945. Source: Science the Endless Frontier, Appendix C.

In its latest review, the National Science Board proposes to replace the vague Broader Impacts criterion with a new criterion of “Societal Benefit.” Also, rather than being folded into proposals, as Broader Impacts currently are, the Societal Benefit criterion would stand as an independent “pillar” of the review process, to be evaluated separately from the proposal itself and its Intellectual Merit.

It’s that seemingly innocuous word “pillar” that is concerning.

The NSF has used that word before, for example, in competition for high-budget Engineering Research Centers (ERCs). These are multi-university consortia focused on a particular research area, such as telecommunications or materials research. ERC proposals go through a different review process from the normal investigator-initiated proposals. Rather than Intellectual Merit and Broader Impacts, ERC proposals are evaluated on four “pillars” of excellence, with judgments for each pillar being made separately by different groups of NSF management. For a proposal to be successful, all pillars must be judged sturdy. Falling short on one pillar dooms the proposal, no matter how sturdy the other pillars might be.

And what are these pillars? Three of them address conventional aspects of merit review—scientific strength, competence of the partners, and effective training of young scientists. The fourth—and problematic—pillar is diversity and inclusion, which the NSF’s Office of Diversity and Inclusion is charged with evaluating. Given that weakness in one pillar dooms a proposal, this gives NSF diversicrats effective veto power over any ERC proposal that they deem unworthy. In this way, even the “very best science” is subordinated to an anti-science ideology, directly contradicting the NSF’s founding goal.

This is the concern: setting Societal Benefit off as a separate pillar will be a mechanism for imposing the same ideology on those 50,000 investigator-initiated research proposals the NSF receives each year. If adopted, this new NSF merit review model will likely spread to all the government agencies that now fund academic science. Those tens of billions of dollars, which are predicated on supporting the “very best science” will now go to funding the race, gender, and sexual-proclivity spoils system that is at the heart of “diversity, equity, and inclusion” ideology.

The founders of the NSF were well aware of the risk that government funding would suborn science to essentially political ends. Throughout the NSF’s founding manifesto, Science: The Endless Frontier, there are warnings that government support of basic research would kill the goose that had been laying all those golden science eggs. I quote from one such warning expressed by Isaiah Bowman, who headed a subcommittee charged with addressing the issue of “Science and the Public Welfare”:

The committee foresees that an increased measure of Federal support will raise new problems. We have, therefore, carefully considered the possibility of increasing Federal aid for scientific research without, at the same time, Introducing undesirable paternalism. For, in order to be fruitful, scientific research must be free—free from the influence of pressure groups, free from the necessity of producing immediate practical results, free from dictation by any central board.² [emphasis added]

The ongoing saga of merit review illustrates the irreconcilable contradiction that has sat at the heart of the NSF experiment since it began. Initially, the NSF’s structured its science funding program to keep such political ideologies and pressure groups at bay. Even so, government funding is always subject to legitimate demands for accountability, which explains the ongoing contortions over Broader Impacts.

Which brings us to the present danger. Money and power always speak with the loudest voices, and the more there is of both, the greater the incentive to strip those safeguards away. Science—like the rest of our society—is now in the grips of a political ideology that places the pursuit of power and money over all, and that has little regard for niceties like intellectual autonomy. This is why science has become overtly and alarmingly politicized.

Can science be brought back? While the Broader Impacts criterion has long been considered an annoyance by scientists and reviewers, its ongoing vagueness has actually been scientists’ friend, the last barrier holding back the “undesirable paternalism” that Isaiah Bowman warned against. That may be about to go away.

[1] Mervis, Jeffrey. 2024. National Science Foundation grant reviewers urged to think more about ‘societal benefits’, Science 23 February 2024.

[2] Bush, Vannevar. 1945. Science. The Endless Frontier. A Report to the President by Vannevar Bush, Director of the Office of Scientific Research and Development, July 1945. Washington, D.C.: US Government Printing Office. https://www.nsf.gov/about/history/EndlessFrontier_w.pdf

[3] M. Park, E. Leahey and R. J. Funk. 2023. Papers and patents are becoming less disruptive over time. Nature 2023 Vol. 613 (7942): 138-144.

Art by Joe Nalven