Shaky Studies on Women and STEM

Readers of the higher education press and literature may be forgiven for supposing that there is more research on why there are not more women in STEM fields than there is actual research in the STEM fields themselves. The latest addition to this growing pile of studies appeared a few months ago in Science, and now Science has just published a new study refuting the earlier one.

In the earlier study, “Expectations of Brilliance Underlie Gender Distributions Across Academic Disciplines,” Sarah-Jane Leslie, a philosophy professor at Princeton, and several co-authors surveyed more than 1800 academics across 30 disciplines — graduate students, postdocs, junior and senior faculty — to determine the extent of their agreement with such statements as, “Being a top scholar of [your field] requires a special aptitude that just can’t be taught” and whether “men are more often suited than women to do high-level work in [your field.]”

Fields that believe innate brilliance is essential to high success, such as physics and philosophy, have a significantly smaller proportion of women than fields that don’t, such as Psychology and Molecular Biology.

Shunning Women?

This study caused a big stir. It was deemed so important that it was accompanied in Science by an introductory article explaining its findings and significance, “Gender Inequality in Science,” by Prof. Andrew Penner, a sociologist at the University of California, Irvine. Admiring summaries and discussions can also be found at the National Science Foundation, in a long piece from the Princeton news service, in the Chronicle of Higher Education, Inside Higher Ed, Science Mag, and Scientific American, just to name a few. Reuters was typical: “fields that cherish sheer genius shun women.”

Leslie et al. conclude that “women are underrepresented in fields whose practitioners believe that raw, innate talent is the main requirement for success” and this underrepresentation occurs “because women are stereotyped as not possessing such talent.” The authors hedge their bets, however, about the actual causal mechanism at work. They note, for example, “The practitioners of disciplines that emphasize raw aptitude may … exhibit biases” against women, but on the other hand “[t]he emphasis on raw aptitude may activate the negative stereotypes in women’s own minds [leading them to] internalize the stereotypes [and] decide that these fields are not for them.”

The Role of Brilliance

The study is cagey, in short, in glossing over the border between facts and stereotypes, and its significance. On one hand, they insist that the “pervasive cultural associations” linking men more than women with raw talent are stereotypes that do not reflect reality. On the other, they ask, are women “less likely to have the natural brilliance that some fields believe is required for top-level success?” and answer: “our assessment of the literature is that the case has not been made.”

It also appears that the authors doubt that high levels of success depend on native brilliance even in those fields that claim it does. Lead author Sarah-Jane Leslie, for example, told a Princeton interviewer that “in her own field of philosophy in the 1980s, philosophers would sometimes speak of ‘the beam’ — the idea that some lucky individuals were born with a metaphorical beam of light coming from their foreheads, which they could shine on any subject matter they chose, thereby illuminating it without prior study.” But “the fact is,” she added, “that any of us who are successful in our fields only got there through incredible amounts of hard work and dedication.”

If it is a “fact” that highly successful women even in fields that claim to require brilliance “only” succeeded through hard work, then the practitioners of those fields — significantly, they find, both women and men — are somehow mistaken about what their fields require.

A Smoking Analogy

Grappling unsuccessfully with this caginess, Prof. Andrew Penner, writing in Science’s article introducing and explaining the brilliance study, concludes, “It is likely impossible to disentangle the effects of societal bias and individual preferences, because people’s understanding of gender differences shape their preferences.” That conclusion, however, is a non-sequitur. The fact that an individual preference may be affected by — even to some degree the result of — social bias does not mean it does not exist as an individual preference. (See my “The Misguided Push for STEM Diversity for an extreme example of the belief that stereotypes rob women of free will altogether.)

Leslie et al., in short, do not resolve the central questions their study raises: Do certain fields actually reward brilliance more than hard work; are there in fact fewer brilliant women available and interested in joining those fields?

Now comes a new study that throws considerable light on those questions. In a “Technical Comment” recently published in Science, Prof. Donna Ginther, an economist at the University of Kansas, and Prof. Shulamit Kahn of the Questrom School of Business, Boston University, reexamined the data and analysis of the Leslie et al. study “and found that “[f]emale representation was associated with the field’s math-intensive content — especially relative to the field’s verbal content — and, controlling for this, faculty beliefs about innate ability were irrelevant.”

Math-Intensive Fields

What Ginther and Kahn found, in short, is that it was not “expectations of brilliance” that predicted the representation of women in various fields “but mathematical ability, especially relative to verbal ability…. While field-specific ability beliefs were negatively correlated with the percentage of female Ph.D.s in a field, this correlation is likely explained by women being less likely than men to study these math-intensive fields.”

Ginther’s and Kahn’s argument was anticipated and developed even beyond theirs by psychiatrist Scott Alexander in a brilliant long entry on his widely read Slate Codex blog, “Perceptions of Required Ability Act As A Proxy For Actual Required Ability In Explaining The Gender Gap.” His criticism of Leslie et al. is even more devastating:

Imagine a study with the following methodology. You survey a bunch of people to get their perceptions of who is a smoker (“97% of his close friends agree Bob smokes.”) Then you correlate those numbers with who gets lung cancer. Your statistics program lights up like a Christmas tree with a bunch of super-strong correlations. You conclude, “Perception of being a smoker causes lung cancer,” and make up a theory about how negative stereotypes of smokers cause stress which depresses the immune system. The media reports that as “Smoking Doesn’t Cause Cancer, Stereotypes Do.”

This is the basic principle behind Leslie et al.

Like Ginther and Kahn, who did not cite his work, Alexander disaggregated the quantitative from the verbal GRE scores and found that the correlation between quantitative GRE score and percent of women in a discipline to be “among the strongest correlations I have ever seen in social science data. It is much larger than Leslie et al’s correlation with perceived innate ability. Alexander’s piece, and in fact his entire blog, should be required reading.

Change the Message?

Gintner and Kahn also make the seemingly required bow to stereotypes, concluding that “it is the stereotypical beliefs of the teachers and parents of younger children that become part of the self-fulfilling belief systems of the children themselves from a very early age” that prevent young women from choosing to pursue math, and thus they that “the message” delivered to young women needs to be changed.

Certainly Leslie et al. blow that horn with enthusiasm, and their message, sadly, need not even be accurate. “Is natural brilliance truly more important to success in some fields than others?” they ask.

The data presented here are silent on this question. However, even if a field’s beliefs about the importance of brilliance were to some extent true, they may still discourage participation among members of groups that are currently stereotyped as not having this sort of brilliance. As a result, fields that wished to increase their diversity may nonetheless need to adjust their achievement messages.

Well, of course! If brilliance really is required, that would certainly tend to discourage interest on the part of people who do not think of themselves as brilliant, “stereotyped” or not. But is that a justification to “adjust the message” to downplay the importance of brilliance in those fields where the belief in its importance is accurate?

Several years ago I sent a piece I’d written, “Wanted: More WIS (Women in Science),” to my daughter, Jessie, a talented young research physicist who was then a fifth-year graduate student at Caltech (she finished her Ph.D. work just before her 23rd birthday). In the piece I quoted Prof. Barbara Bogue of Penn State , the co-founder of the Society of Women Engineers, who “warned against ‘negative role models’ who give the impression that they are overly obsessed with their work and drive people away by making the field seem too demanding.”

Jessie replied that she particularly liked the Bogue quote — “Because of course we wouldn’t want anyone giving an honest impression of the field.”