In Minds on Fire: How Role Immersion Games Transform Colleges, Mark C. Carnes makes the case that they might. Students at Pace University have become so engrossed in a game called Henry VIII and the Reformation Parliament that class spills into the dorms: “students endlessly debated, gossiped, and strategized Tudor religion and politics.” At Dordt College, students playing Rousseau, Burke, and Revolution in France 1791, ask to begin class half an hour early so that they can complete the game. Every student attends the four early morning sessions. As one of them explains, “We read more . . .than we had at any time before in the class…It had become more than a class to us at that point. ”
How to Play
These students are participating in Reacting to the Past, Carnes’s creation. Reacting games typically unfold over three to four weeks. In the Rousseau, Burke, and Revolution game, for example, the first , “traditional,” classes introduce the historical and intellectual background against which the National Assembly drafted the Constitution of 1791. As in all Reacting games, classic texts, in this case Rousseau’s Social Contract, are central. Next, students are assigned roles—a Jacobin, Louis XVI, an “indeterminate,” possessing certain attributes but belonging to no particular faction— and objectives, which may change. During the game phase, students convene as the National Assembly. Using what they’ve already learned, documents and background provided in a “game book,” and research they conduct independently, students give speeches, produce factional newspapers, make deals, and otherwise work with and against each other to shape the Constitution and achieve their objectives.
Some outcomes are determined by a vote, while others—like what happens if one faction stirs up a popular uprising—are determined by a roll of the dice. The professor, as “game master,” answers questions, reminds players of their objectives, and sometimes throws a wrench, in the form of new documents or events, into the proceedings. In a final stage, the class returns to a more traditional format to evaluate the game, see where and why they have diverged from what the Assembly really did, and consider historiographical controversies concerning the Revolution.
A Fix for Higher-Ed
Carnes had no special interest in “fate of higher education” when he stumbled on the idea of Reacting almost 20 years ago. Instead, like many good instructors, he didn’t think that what he was doing was working very well. Our “regular classrooms,” he says, often “resemble a movie in which excellent actors struggle to breathe life into a so-so script. Carrnes says he simply “sought a more stimulating classroom experience” for himself and his students.
When Carnes reflects on Reaction in the light of contemporary higher education controversies, however, he sees it a cure for many ills. Books like Academically Adrift argue that our students aren’t working hard enough. Reacting to the Past, Carnes responds, gets students to work very hard, voluntarily. Books like Paying for the Party argue that colleges facilitate the party scene. Carnes responds that Reacting, while it does not get students to quit their fraternities to learn more about the French Revolution, is compelling enough to engage students both socially and intellectually, even on nights and weekends. Reacting, which puts students in charge of the classroom, and permits them to yell, interrupt, and even sing in class, is a form of “subversive play” that helps students learn. Books like Clayton Christensen’s The Innovative University argue that brick and mortar colleges need to do more to compete with online upstarts. Reacting, Carnes responds, takes full advantage of the residential college. Many of Carnes’s claims are backed by anecdotal and peer-reviewed evidence, of which there is a fair deal, since Reacting has been around for over a decade, includes a wide variety of games, and has been tried at hundreds of places.
Reacting is not merely an answer to various external challenges but also a way to work on the Socratic mission that animates many of our colleges and universities. Carnes tells us about a Muslim student asked to play David Ben Gurion, and a Jewish student asked to play Awni Abd al-Hadi, head of the Arab Independence Party, in The Struggle for Palestine 1936. Such exercises, in which many students become deeply invested, give students an outsider’s perspective on their cherished opinions in a way that more direct Socratic cross-examination, which puts them on their guard, might not. In addition, Reacting, though “only a game,” attempts to cultivate the judgment of participants by putting them into positions of responsibility in situations that are complex in at least two ways: first, principles are at stake and contested; second, they must be implemented in conditions of considerable uncertainty, since the actions of others, not to speak of the dice, are unpredictable.
What Does the Research Show?
Focus groups and evaluations strongly suggest that Reacting achieves what ordinary college instruction cannot. To that end, it’s hard to quarrel with Carnes’s argument that more teachers and colleges should at least give it a try. Moreover, colleges that claim and desperately try to foster residential communities should be impressed by reports of students debating history, philosophy, and strategy, late into the night. “What you have to understand,” says one student, “is that after midnight a whole Reacting world comes alive in the dorms.”
Carnes is sometimes a less effective advocate of the program than the students. At his most irritating, he insists that faculty who don’t embrace Reacting “cling to familiar practices” in the face of clear evidence that Reacting works. However, the evidence is much more mixed than Carnes implies. Let three examples suffice. Carnes refers to an unpublished study of first year seminars at one private college. The study showed that “Reacting students surpassed the other students in end-of-semester, critical-thinking essay writing.” But he does not mention that the study also showed that although students identified as “Best” and “Worst” on the basis of midterm examinations benefited from Reacting, students identified as “Middling” did better in more traditional seminars. One need not be an educational reactionary to worry about this result.
Carnes also neglects to mention here that a much wider study showed that Reacting has no advantage over more traditional seminars in improving student writing. Also, Carnes says that this same “major study has confirmed that students become better speakers in Reacting class.” He does not mention that it confirms only that students “improved marginally,” a disappointing result for a class that emphasizes speech so much. He also does not mention that students in the Reacting seminars failed to improve their confidence as speakers, either absolutely or in comparison to students in other seminars. Finally, Carnes also tends to exaggerate. He claims that Reacting classes will make students more empathetic: “[perhaps] we all will benefit from the time they spent pacing in their dorm rooms, pondering antique moral dilemmas and imagining what it was like to walk in the shoes of people whose bones have long since turned into dust.” However, the study shows that the Reacting group did not show a significant gain in empathy, as measured by a standard test, though they did differ slightly from the non-Reacting group, which, for some reason, had shown “a slight decrease in empathy” from the beginning of the semester. Moreover, the assessments Carnes points to will not support the wild suggestion, which Carnes seems to make in all seriousness, “that role-immersion games can solve almost all of the problems afflicting higher education.” Finally, the reader will justly feel condescended to, not to mention unconvinced, when Carnes suggests that “those who venture beyond the box in any field often discover a landscape that extends beyond anything they had imagined.”
To be sure, though, Carnes’s excesses should not stop faculty and administrators from exploring Reacting to the Past. To borrow a slogan from Lincoln Center’s Mostly Mozart Festival, it sounds like “serious fun.”
Thanks to Mark Carnes for his willingness to engage in a conversation about Minds on Fire. I think that at this point it would be hard for an observer to adjudicate the disputes between us without reading the book. Therefore, rather than try to respond to Carnes’s response, I will end in a way I trust Carnes will not object to: people interested in the issues raised here should read Minds on Fire.
A generous response indeed.
I appreciate Professor Marks acknowledgement of his error. Now let me respond to his other assertions. (They focus on two of the studies mentioned in the book. For simplicity here, I shall refer to them as the 4-college study (because it was based on data from four colleges and was published in the Journal of Educational Psychology) and as the 1-college study [an unpublished study the first-year students who took Reacting and traditional seminars at a single school].
1) Marks writes that I fail to mention that the 4-college study indicated that “Reacting has no advantage over more traditional seminars in improving student writing.” This sentence, however, can be restated more simply: “Reacting classes were as effective as traditional seminars in improving student writing.” Omission of such a statement hardly constitutes evasion; I could have trumpeted it as evidence that Reacting promotes public speaking without sacrificing development of writing skills.
2) Marks writes that I fail to note that the 4-college study indicated that Reacting students improved as speakers only “marginally,” which he called “a disappointing result.” I can see how Marks could arrive at such a conclusion; probably I should have included a long footnote on this point. Allow me to do so now: The researchers in this study invited first-year students, one-by-one, to come to an interview room. The students answered a long set of questionnaires and completed a 10-minute impromptu writing exercise. Then they were given background notes on a subject such as the merits of gun control and were told to prepare a three- to five-minute speech on the subject to be delivered to a tape-recorder. They had five minutes to prepare their speeches. Evaluators scored the tape-recorded speeches according to rubrics from the Competent Speakers Speech Evaluation Form. The CSSEF employs a 21-point scale which was designed for a “high range” of speaking competencies. The grading rubric explains that outstanding speeches would likely be rated above 13. The evaluators for the 4-college study, listening to students who had been admitted to highly selective colleges, assigned a median rating of 17.4 for all the speeches at the outset of the semester. The scores were nearly identical for students in the Reacting and in the traditional classes. The same process was repeated at the end of the semester. The scores for students from the regular seminars went up to 17.6 while those for the Reacting students increased to 18.2. In statistical terms, the Reacting students “improved marginally.” But this statistical bump conceals the fact that the initial high score (17.4) left little room for improvement, since 21 was the maximum score possible. Moreover, several of the CSSEF grading rubrics presume that speakers had time to organize and prepare their speeches. (One rubric, for example, assays whether the speaker “uses an organizational pattern appropriate to the topic, audience, occasion and purpose.”) With only five minutes to read the background sheets and prepare their speeches, the students in the 4-college had little time to do much organization or attend to other rubric categories, such as pacing and phrasing. That is to say, it would have been difficult for any speech to get scores much above 19. In this light, the 18.2 was not “disappointing” at all. The authors of the study summarized it best: the Reacting students’ improvement was numerically “marginal” and “modest”. It was also significant.
3) Marks writes that I fail to mention that the Reacting students in the 4-college study “failed to improve their confidence as speakers.” But the authors of the study addressed this issue. “The lack of an effect on students’ confidence in public speaking was surprising. Many students in the focus group discussions commented on having increased proficiency in oratorical skills. One possible reason that the effect did not emerge might be that students in other [first-year seminars], in which debate and oratory are less emphasized, might have inaccurate, or even misplaced, confidence in their oratorical abilities. An effect would not emerge if students from the Reacting to the Past courses were accurately confident about their speaking skills, whereas students from the other seminars were inaccurately but equally confident about their abilities” (p.611). I didn’t suppress the conclusion; I just didn’t know what the numbers signified. Nor did the authors of the study;
4) Marks writes, as proof that I exaggerate: “Carnes claims that Reacting classes will make students more empathetic.” I do cite students who describe the intensity of their identification with the historical figures they played. Insofar as empathy is another word for putting oneself in the shoes of someone else, it did not seem impossible that role-immersion might have some effect on empathy. The 4-college student provided further support of this premise. I wrote that by the end of the semester, the empathy scores of the students in the traditional seminars “had declined slightly while those of the Reacting class rose. This juxtaposition generated statistical significance. In other words, after taking Reacting, students were more likely to cry at the end of sad movies and otherwise respond positively to statements that have proven to be consistent indicators of empathy” (p.224). Marks is skeptical of the decline of the empathy scores in the other classes. Why would the empathy levels of students “strangely backslide” during college? But is this phenomenon really so strange? “Jaded seniors” is a cultural commonplace; “jaded freshmen” is an oxymoron. Nevertheless, I do not end this section, as Marks asserts, by claiming that Reacting classes will make students more empathetic. Let me cite the final paragraph: “Role-immersion games appear to create empathetic feelings and also encourage deep reflection on moral matters. But is it really plausible that a college course can ‘teach’ these psychological predispositions? (p.225)” I leave the question hanging. The chapter is entitled, “Inculcating Morality and Empathy (!)”, the exclamation point, in parenthesis, signifying the speculative character of the premise;
5) Marks did spot an error, and I thank him for bringing to my attention. I quoted a section of the 1-campus study and claimed that it showed that Reacting students had outperformed the regular students in critical writing. I took the wrong quote, as Marks notes; worse, I took it from a component of the study which the authors found to be inconclusive. I should have cited their summary judgment for the study as a whole: “This is a very positive set of findings for the Reacting curriculum–and one that seems to line up with anecdotal and intuitive reports. Based on these findings, we can conclude that the Reacting experiment was a success ” (p.14).
***
At the end of my book, after summarizing a wide range of benefits I associate with Reacting, I smile at the grandiosity of the vision: “The suggestion that role-immersion games can solve almost all of the problems afflicting higher education is so sweeping that sensible readers will likely dismiss it out of hand.” Marks perceived this whimsy as a “wild suggestion.” Let me assure earnest readers that I was not speaking “in all seriousness,” as Marks thinks. I do not believe that role-immersion games can improve dining hall food or lower the salaries of football coaches.
I think these two studies, and many others, sustain the arguments I make. Yet I agree with Marks that the strongest evidence comes from the words of the many students cited in the book, and I appreciate his suggestion that Reacting just might be “serious fun.”
Mark Carnes is, however, right, that I missed the footnote in which he concedes that middling students did better in traditional seminars than they did in reacting seminars. I apologize for that error, which, is indeed, serious. However, as long as Carnes draws our attention to that footnote, let’s observe that it includes this quotation: “As is apparent [from the data on student performance], students in the Reacting sections outperformed their Thematic peers in all groups and on all measures” [with one exception]. What Carnes *doesn’t* say is that this quote describes student performance *before* the Reacting part of the course started! For whatever reason, before any exposure to the Reacting curriculum, the students assigned to Reaction sections performed better than students who were not. So although I do regret missing this particular note, I think my point still stands: Carnes tends to oversell the data on Reaction. A more measured conclusion would have been that Reaction greatly outdid traditional seminars on every measure of student satisfaction and perceived learning (also how faculty perceived student learning and satisfaction), but results were mixed on the objective measure–improvement from midterm (only after which the Reacting approach was taken in some of the groups) to final.
Let me respond to two of Mark Carnes’s points, reserving the third for when I have his book in front of me again.
1. Carnes says I erred concerning Reacting’s effect on speaking skills by conflating Phase 1 and Phase 2 of the study. Yet Stroessner et. al say in discussing the results of Phase 2: “For rhetorical skills, there emerged significant main effects for Time, F(1, 76) = 7.28, p < .01, [eta]p2 = .09, and Institution, F(1, 76) = 17.13, p < .01, [eta]p2 = .31, and a Reacting × Time interaction, F(1, 76) = 4.26, p < .05, [eta]p2 = .05. The interaction (shown in Figure 1) shows that rhetorical skills improved marginally for students in Reacting to the Past from the beginning to the end of the semester (M = 17.43, SE = .39 vs. M = 18.23, SE = .37), t(29) = 1.76, p = .09, but they did not differ for students in control seminars (M = 17.33, SE = .33 vs. M = 17.56, SE = .27; t < 1). " So the effects may have been generally larger than in Phase 1, but–unless Stroessner has inexplicably shifted to discussing Phase 1 in the midst of his discussion of Phase 2, they remain "marginal." Stroessner et, al's conclusion is that we see "some benefit"–in a course that focuses on speaking skills to the extent that Reacting does, it would be deeply disappointing not to achieve this result,. So I do not think I have committed a serious mistake or any mistake at all here. I note that just to be certain of my interpretation, I contacted Stroessner. But he never replied.
2. Carnes also suggests that I committed a serious error concerning empathy. But all I pointed out–and it's simply a result of the study Carnes cites–is that the students in Reacting made no statistically significant gain on the empathy inventory from the beginning of the semester to the end of it. That suggests to me that Carnes is reaching a bit when he says:“[perhaps] we all will benefit from the time they spent pacing in their dorm rooms, pondering antique moral dilemmas and imagining what it was like to walk in the shoes of people whose bones have long since turned into dust.” Perhaps, but the study doesn't really even hint at that result, regardless of whether Reacting is a better intervention than some others (in which empathy, strangely backslides). Some studies have shown that students who do not attend college backslide in math skills over time. Does that mean we should think that math courses that show no statistically significant gains in mathematics ability are going to someday produce revolutionary change in mathematics?
My review of Minds on Fire takes note of the astounding anecdotal evidence concerning Reacting to the Past. I find this evidence compelling and said that focus groups and evaluations strongly suggest that "Reacting achieves what ordinary instruction cannot." I suppose that it is a sign of Carnes's love for Reacting that he considers this only "a fair summary of the argument of my book, and of Reacting." More power to him. Good programs need champions, even when they are just a little prejudiced.
Marks offers a fair summary of the argument of my book, and of Reacting. But he makes several serious mistakes. He states: “[Carnes] does not mention” that the author of a study found that Middling students at one college did better in traditional seminars. I quoted, verbatim, seven lines from that study on precisely this point (p.311). He also writes that in the multi-campus study, Reacting students’ speaking skills improved “only marginally”. That was the researcher’s conclusion after the first phase of the study; further study showed, however “generally larger” effects. Consider this, too: For the past decade, study after study indicates that students experience for small gains after taking 30-40 college courses. That a single course has been proven to improve students’ speaking skills and influence their capacity for empathy is worthy of note, it seems to me.