Molly Bishop Shadel, Sophie Trawalter & J.H. Verkerke

Gender Differences in Law School Classroom Participation: The Key Role of Social Context

Introduction

Even though women make up roughly half of the students enrolled in law school today, they do not take up roughly half of the speaking time in law school classes. “Speak Up” and similar studies1 1.While these studies carry different names (including “Speak Up” and “Speak Now”), we call them Speak Up studies throughout this paper.Show More that have been conducted at several law schools (including Harvard,2 2.Adam Neufeld, Costs of an Outdated Pedagogy? Study on Gender at Harvard Law School, 13 J. Gender, Soc. Pol’y & L. 511, 561–62 (2005) (“The study results show systemic differences between female and male students’ experiences at Harvard Law School in areas ranging from classroom participation and self-confidence to grades and employment.”).Show More Yale,3 3.Yale L. Women, Yale Law School Faculty and Students: Speak Up about Gender: Ten Years Later 13–14(2012).Show More Stanford,4 4.Daniel E. Ho & Mark G. Kelman, Does Class Size Affect the Gender Gap? A Natural Experiment in Law, 43 J. Legal Stud. 291, 293 (2014).Show More the University of Chicago,5 5.Mallika Balachandran, Roisin Duffy-Gideon & Hannah Gelbort, Speak Now: Results of a One-Year Study of Women’s Experiences at the University of Chicago Law School, 2019 U. Chi. Legal F. 647, 647–48 (2019).Show More and the University of Pennsylvania6 6.Lani Guinier, Michelle Fine & Jane Balin, Becoming Gentlemen: Women, Law School, and Institutional Change 1–2 (1997).Show More) have demonstrated that women are less likely to speak in the law school classroom than are men. Why do women speak less than men? Is that effect fixed, or does it change if circumstances change? And if so, what, if anything, should be changed in the classroom?

To answer these questions, we undertook three studies to examine when and why women at an elite law school, the University of Virginia, speak in class. We found that both social and academic contexts affect women’s participation. In Study 1, we transcribed and coded recordings from 107 class sessions of required first-year law school courses taught during a single academic year. Replicating previous Speak Up studies, we found that female students speak less than male students. That gap closes, however, when professors call on students systematically and when class size is smaller. In Study 2, we surveyed four different graduating classes at four different points during their law school experiences—at orientation, after their first semester, before their second year, and shortly before graduation. We found that women report speaking less than men, but that gap appears during the first year and disappears by the end of the third year. Interestingly, we found that women and men at orientation report similar expectations about how much they will speak in class and that women near graduation report speaking at the same levels that men report speaking. In other words, gender gaps in speaking are not fixed, but dynamic. And, as we will explain, they are a function of context, such as social concerns and dislike of particular pedagogies. Finally, in Study 3, we surveyed students from the Class of 2021 specifically about the social costs of speaking in class. We found that women, more than men, report backlash for speaking in class, and this difference affects their willingness to participate in the law school classroom.

This Article is organized as follows. Part I summarizes previous studies that have been done in this field and introduces the framework of the project. Part II describes the methodology of the study and reports the results. Part III interprets the results. We conclude that gender differences in speaking are best understood as a function of social and academic context. Achieving gender parity in speaking requires reimagining the context of the classroom, not “fixing women.”

I. Previous Studies and Project Framework

We began our collaboration because of questions raised by two law students on our team: Why did it seem that their female classmates—highly accomplished and competent women—were less likely to speak in class than male students? To address this question, we first survey the existing literature in the field.

A. Prior Literature: Women May Speak Less, Depending on Context

Social scientists have documented that women are less likely to speak up in historically male-dominated spaces. For example, at academic conferences, female scholars are less likely to speak than male scholars, and when they do speak, they speak for shorter amounts of time.7 7.See, e.g.,Julia Schroeder et al., Fewer Invited Talks by Women in Evolutionary Biology Symposia, 26 J. Evolutionary Biology 2063, 2068 (2013) (finding that female scholars were more likely to decline invitations to a major biology conference); Lynda R. Wiest et al., Researcher Study Thyself: AERA Participants’ Speaking Times and Turns by Gender, 39 Equity & Excellence in Educ. 313, 319 (2006) (finding that when women do speak, they speak for less time than their male counterparts).Show More Female Supreme Court advocates speak less than their male counterparts.8 8.Dana Patton & Joseph L. Smith, Lawyer, Interrupted: Gender Bias in Oral Arguments at the US Supreme Court, 5 J.L. & Courts 337, 352 (2017).Show More A 2017 New York State Bar Association report found that women attorneys are less likely to have speaking roles “at every level and in every type of court: upstate and downstate, federal and state, trial and appellate, criminal and civil, ex parte applications and multi-party matters.”9 9.Debra Cassens Weiss, Only 25 percent of lead counsel roles in New York courts are held by women, study finds, ABA (Aug. 7, 2017, 7:00 AM), https://www.abajournal.com/‌news/article/only_25_percent_of_lead_counsel_roles_in_new_york_courts_are_held_by_women. [https://perma.cc/9N7N-DPN2].Show More

Just as social scientists have found that women speak less in male-dominated spaces, researchers have found that women speak less in the law school classroom. One of the earliest studies to investigate how gender affects the law school classroom documented the experiences of twenty women in the Yale Law Class of 1987, based on the premise that “men and women experience law school differently.”10 10.Catherine Weiss & Louise Melling, The Legal Education of Twenty Women, 40 Stan. L. Rev. 1299, 1300 (1988).Show More Soon thereafter, Professor Lani Guinier and her colleagues at the University of Pennsylvania published a groundbreaking book called Becoming Gentlemen. They found that women were much less likely to participate in classes that use the Socratic Method.11 11.Guinier et al., supra note 6, at 13–14.Show More Women also performed less well academically than did their male counterparts, despite entering law school with similar entry-level test scores.12 12.Id. at 8.Show More

More recently, law students have undertaken their own investigations of women’s experiences in law school in a series of reports known as Speak Up studies. The first of these was published in 2005 by Adam Neufeld, then a student at Harvard Law School, finding that women were less likely than men to speak in class.13 13.Neufeld, supranote 2, at 531. These results seem to have held over time. In 2013, The Harvard Crimson reported:Among the top students in their graduating classes, men and women entering Harvard Law School earn similar undergraduate grades and LSAT scores. But as soon as students step into Wasserstein Hall, a dramatic gender disparity emerges. Indicators suggest that female students participate less and perform worse than their male counterparts over the course of their three years at the Law School.Dev A. Patel, In HLS Classes, Women Fall Behind,Harv. Crimson (May 8, 2013), https://www.thecrimson.com/article/2013/5/8/law-school-gender-classroom/. [https://perma.cc/4UUJ-9DPV].Show More Similarly, in 2012, Yale Law Women, studying the experience of female law students at Yale, found that “[p]articipation by women students continues to lag behind their men colleagues,” and are likely to experience “feelings of isolation and alienation.”14 14.Yale L. Women, supra note 3, at 3, 13–14.Show More Similar investigations at other elite law schools show similar results.15 15.See, e.g.,Ho & Kelman, supranote 4, at 293; Balachandran et al., supra note 5, at 663, 680.Show More

While popular culture suggests that women speak less than men because they lack confidence,16 16.See, e.g., Sheryl Sandberg, Lean In: Women, Work, and the Will to Lead 8 (2013); Katty Kay & Claire Shipman, The Confidence Code: The Science and Art of Self-Assurance—What Women Should Know xviii (2014).Show More we believe that gender differences are best understood as a function of social context, not inherent gender differences. Sometimes, women behave differently than men because social norms constrain their freedom to participate.

Numerous studies have shown that, in some contexts, women pay a heavy price for speaking up. For example, female CEOs who speak for disproportionately longer amounts of time than other participants in meetings are rated as significantly less competent and less suitable for leadership than male CEOs who speak for the same amount of time.17 17.Victoria L. Brescoll, Who Takes the Floor and Why: Gender, Power and Volubility in Organizations, 56 Admin. Sci. Q. 622, 636 (2011).Show More In studies focused on negotiations, assertive, self-advocating women are shown to suffer social backlash because participants see them as socially unskilled and unlikeable. In contrast, male negotiators, exhibiting identical behaviors, face no negative consequences.18 18.Emily Amanatullah & Catherine Tinsley, Punishing Female Negotiators for Asserting Too Much…Or Not Enough: Exploring Why Advocacy Moderates Backlash Against Assertive Female Negotiators, 120 Org. Behav. & Hum. Decision Processes 110, 119 (2013).Show More

When social costs are eliminated, however, female behavior can change. For example, a different study assigned female negotiators the task of negotiating on behalf of another person, positing that negotiation on behalf of others is socially acceptable for women and does not carry the same threat of backlash. As predicted, the study found that “[w]hen these social costs are eliminated . . . women exhibit the same assertive behaviors and successful outcomes as men.”19 19.Emily Amanatullah & Michael Morris, Negotiating Gender Roles: Gender Differences in Assertive Negotiating Are Mediated by Women’s Fear of Backlash and Attenuated When Negotiating on Behalf of Others, 98 J. Personality & Soc. Psych. 256, 263 (2010).Show More We hypothesize that law school creates a particular context in which some women may not want to speak up and that this behavior can change if the context changes.

B. Understanding the Specific Context of the Law School Classroom

Many students’ law school experiences are affected by the signature pedagogy of legal education—the Socratic Method. While not every law school professor in the United States uses the Socratic Method, many do, particularly in required first-year courses. During this time, students are making sense of what it means to be a law student and learning how to behave in and out of a law school classroom.

The Socratic Method is a form of discussion meant to stimulate critical thinking by teaching students to answer (and ask) questions. It typically involves the professor asking questions that in theory help students discover better answers by systematically identifying and eliminating those that lead to contradictions. The goal is to teach students to “think like a lawyer” by being able to see all sides of an argument.

The Socratic Method can feel adversarial. Often, professors challenge students’ points of view to force them to justify their answer and to show how a rule that generates a just outcome in one situation may produce a problematic result in different circumstances. Professors also operate on a spectrum of predictability when using the Socratic Method, ranging from calling on students with no notice to providing advance notice that a student will be on call. Professors also vary in tone and in whether they will permit a student to decline to answer.

The Socratic Method is also inherently public, exposing students to the judgment of their peers. For many students of any gender, the experience can provoke intense feelings of anxiety.20 20.Cf. William M. Sullivan et al., Educating Lawyers: Preparation for the Profession of Law 2 (2007).Show More Some students find themselves the topic of after-class discussion because of their answers during class. The studies described above conducted at Harvard, Yale, and the University of Pennsylvania specifically identify the Socratic Method as an important cause of gender differences in law students’ experiences.21 21.Neufeld, supra note 2, at 538, 562 (finding that the Socratic method “may amplify the effect of differences in confidence levels,” but also finding that the “combative” nature of this method did not alone cause gender differences); Yale L. Women, supra note 3, at 14; Guinier et al., supra note 6, at 15.Show More

For purposes of this paper, we define “Socratic” or “cold call” to mean any non-volunteered answer by a student to a professor’s question.22 22.We realize that students might experience a question for which they have advance notice or which is offered with a friendly tone (sometimes called a “warm call”) differently than a hostile question asked with no advance notice (a “cold call”). We intend to explore these effects in a subsequent paper. For purposes of this paper, we are focusing on the difference between voluntary and professor-controlled participation, and therefore use ‘cold call’ to include both warm and cold calls.Show More We hypothesize that gendered expectations in the classroom exacerbate the stress of the Socratic Method for women. Female law students must simultaneously exhibit the confidence and analytic competence the professor demands while also navigating the tricky gender norm of remaining likeable. We also hypothesize, however, that when a professor calls on students systematically, the practice can operate as an equalizing mechanism that reduces gender disparity in classroom participation.

C. Our Project Design

Our work builds in many ways on the Speak Up studies, which are important research efforts led by law students showing that women participate in class less than their male counterparts. As we designed our investigation, we sought to extend and deepen the work of the Speak Up studies.

In a typical Speak Up study, students observe law school classes in real time to document professor and student interactions. They attend a class to observe who is speaking and for how long. However, the presence of an observer may affect classroom participation, and coding in real time limits the amount of data that can be gathered.

The first part of our research (Study 1) also involved coding classes, but we obtained and transcribed archival audio recordings. Working from recordings and transcripts eliminated the risk that the researchers’ presence in the classroom could affect students’ and professors’ behavior. It also enabled us to code every utterance in more detail. For example, we were able to obtain information about whether a students’ answer was ‘on point’ or a digression and whether an answer projected confidence as indicated by a lack of verbal fillers. We also limited our sample to required first-year courses. For these courses, law school administrators assign students and professors to sections. Thus, we avoided any selection bias that might result from the freedom students have in elective courses to choose their professors and their classmates. Study 1 allowed us to document gender gaps and begin exploring the influence of context.

Prior studies have also relied on surveys in which students self-report their experiences. Because the surveys were typically not repeated, however, their results captured a moment in time and cannot provide insight into whether participants’ responses change over time. Speak Up student investigators graduate and move on, making it difficult for them to follow student participants throughout their entire law school experience. Also, many of the Speak Up studies frame survey questions that make their focus on gender explicit, thus potentially skewing the results.

Keeping these challenges in mind, in the second part of our project (Study 2), we conducted a longitudinal study in which students provided information about their law school experiences. This study followed four different graduating classes (including anyone in the class willing to participate), checking in with them at four significant points during their law school experiences—on the first day of orientation, after their first semester, before their second year, and shortly before graduation. We collected data from 928 law students, representing, on average, 75% of the students in each cohort. While a typical Speak Up study surveys students once, Study 2 followed four cohorts throughout their law school careers. This study design allowed us to examine whether and how gender gaps in speaking change across time. We also took care not to reveal to participants that gender was a focus of the research. Instead, we told them that the study sought their insights about the student experience at the University of Virginia School of Law.

The third part of our project (Study 3) asked third-year students in the class of 2021 more directly than we did in Study 2 about the social costs of speaking in class. These students began their law school careers with normal in-person classes. But the global COVID-19 pandemic forced them to pivot to entirely online classes midway through their second year. For their third year, they could choose between classes that took place entirely online or classes that included students participating both in person and online. Based on our observations in Study 2, we expected that we would see no gender difference among these students because they were at the end of their third years, but we wondered if the pandemic would change the context in ways that might reintroduce gender gaps. We framed this survey as an inquiry into how students’ experiences during the pandemic compared to their experiences in classes taken before the pandemic—again not signaling that gender was a focus of the project. Most importantly, this survey allowed us to examine specifically whether gender gaps in speaking are related to students’ social experiences, including potential social backlash for speaking.

We made the following predictions: Women would speak less, but this gender difference would be moderated by social context. As the context changes, women’s participation would also change. To characterize social context, we explored the Socratic Method and student attitudes toward the Socratic Method, class size, and student concerns about and experiences of social backlash. We also examined timing and considered whether gender differences in speaking are fixed or dynamic across time. All materials and data can be found at the OSF data repository.23 23.On the Importance of Social Context for Women’s Voices: Gender Differences in Law School Classroom Participation, OSF (last updated Jan. 5, 2022, 9:45 AM), https://osf.io/xat7p/?view_only=33dd7c17ade5437cbaf2a86d0f11752b. [https://perma.cc/78DM-GZ6P].Show More

II. Study Methodology and Results: Women and Men Have Different Experiences in Law School

In this Part, we describe the three studies we undertook to investigate women’s experiences in the law school classroom. Additional details about our study design, methodology, and results are reported in a more technical social science journal article reporting on our work.24 24.Sophie Trawalter, Molly Shadel & J.H. Verkerke, On the Importance of Social Context for Women’s Voices: Gender Differences in Law School Classroom Participation, 2(on file with authors at https://osf.io/937cg/?view_only=33dd7c17ade5437cbaf2a86d0f11752b). [https://perma.cc/2VXX-HDN3].Show More In Part III, we discuss the implications of our findings for law students and teachers.

Study 1: Archival Study

In Study 1, we documented students’ speaking in law school classrooms. We predicted that, as in previous studies, women would speak less than men. In addition, we explored whether women were more uncertain and nervous when speaking.

We obtained permission to transcribe and code audio recordings of 107 class sessions of required first-year courses from a single academic year. Of these, 74% were taught by male faculty and 90% were taught by white faculty. Notably, 86% of the classes included some version of the Socratic Method. Many of these classes also allowed students to volunteer comments. Classes were transcribed by at least one research assistant,25 25.Id. at 11 n.1. Hereinafter abbreviated as “RA.” Five classes were transcribed by five RAs, to train RAs on transcription, and one class was inadvertently transcribed twice. All other classes were transcribed by one RA.Show More and each utterance in each class was coded by at least two RAs, a primary coder and a secondary coder.26 26.Id. at 11 n.2. “Five classes were coded by five RAs, to train RAs on coding, and one class was inadvertently coded by only one RA.”Show More The resulting dataset included 14,403 distinct utterances (8,402 from faculty and 6,001 from students) across 107 classes.

RAs coded the transcripts on various dimensions including: (1) whether each utterance was by the professor or a student, (2) whether the utterance was made by a man or woman, (3) whether the utterance was an interruption, (4) whether the utterance was qualified, (5) whether the utterance was “on point,” (6) and the nature of the utterance. Inter-rater agreement was quite high across all categories.27 27.Id. at 12.Show More

Our analytic approach was as follows: first, we ran class-level analyses to document gaps in speaking and the role of context on speaking. On average, the cohorts included 45% women and 55% men.28 28.See id. at 44. The Law School website published this enrollment information and did not report any students identifying as gender non-binary. In Studies 2 and 3, students were able to identify as gender non-binary, though no study participants selected that option.Show More Men spoke disproportionally more often than women (62% men to 38% women).

Next, we examined speaking length. We found, again, that women spoke less often than men. On average, male students who talked in class talked for 302 seconds whereas female students who talked in class talked for only 194 seconds. Taken together, these findings replicate the results of previous work, including several Speak Up studies.

Because we have data at the utterance level, we also examined additional characteristics of student utterances. RAs coded whether students were “on point” (exhibiting a correct understanding of the material) when they spoke. RAs also coded two additional behaviors: verbal fillers and qualifiers. Fillers are utterances like “um,” “uh,” and “er.” People use fillers when they need more time to think, when they are unsure about what to say, and often, when they are nervous about speaking.29 29.See Herbert H. Clark & Jean E. Fox Tree, Using Uh and Um in Spontaneous Speaking, 84 Cognition 73, 90–91 (2002).Show More Qualifiers are expressions of uncertainty (“I think the author means…” “perhaps the case reveals”). They weaken the judgment or statement made, often without adding information (“Their position is kind of strong”). People use qualifiers to convey tentativeness and often use qualifiers when anxious.30 30.See Mansur Lalljee, & Mark Cook, Anxiety and Ritualized Speech, 66 Brit. J. Psych. 299, 300 (1975).Show More Students’ use of verbal fillers and qualifiers offers a window into whether they were uncertain and nervous about speaking. Interestingly, we found that by these metrics, female and male students sound quite similar. We found no gender differences for qualified utterances, verbal fillers, or utterances that were on point.

Next, we considered the role of context in shaping gender gaps in classroom participation. Specifically, we considered whether utterances were in response to a cold call or volunteered. For volunteered answers, we observed a large and significant gender difference with men volunteering disproportionately more than women. In Figure 1 below, the distribution of men’s participation reveals a strong central tendency for men to volunteer more often than women.

Figure 1: Proportion of Volunteered Answers from Men

For cold calls, we observed a radically different pattern. Men also answered disproportionately more cold calls, but this difference was not statistically significant. The distribution of men’s participation in Figure 2 shows why. Unlike the distribution of volunteered answers, the distribution of answers to cold calls had three peaks. In some classes, men answered all the cold calls, and in others, women answered all the cold calls. This is likely because these professors called on few students in each class session. In other classes, both men and women spoke, but cold calling disrupted the central tendency we saw in volunteered answers. This practice changed the context by creating more variance in speaking opportunities. Thus, it appears that a systematic plan for calling on students can increase gender parity in speaking turns by decreasing opportunities for men to volunteer.

Figure 2: Proportion of Cold Calls Answered by Men

Next, we examined how women (vs. men) experience cold calls (vs. volunteering). Specifically, we examined the number of fillers students used, whether they used qualifiers, and whether their utterance was “on point.” Regardless of gender, students answering a cold call (vs. volunteering) used more fillers and qualifiers, suggesting they felt less confident about their answers to a cold call. Results also revealed that students were more likely to be on point in response to a cold call (vs. volunteering), and again this effect was not qualified by gender.

Considering class size, we found gender differences in the tendency to speak in smaller and larger classes. Women speak more often in small classes relative to larger classes, and men speak more often in larger classes relative to small classes.31 31.See Trawalter, Shadel & Verkerke, supra note 24, at 15 n.4. We define small classes as those containing roughly 30 students, medium classes as those containing roughly 60 students, and large classes as those containing roughly 90 students.Show More We observe gender parity in the proportion of utterances in small classes, while men speak more than expected in medium-sized and larger classes.32 32.For a similar finding at Stanford Law School, see Ho & Kelman, supra note 4, at 310 (documenting the impact class size can have on GPA across genders).Show More (See Figure 3.)

Figure 3: Utterances by Student Gender and Class Size

Study 2: Longitudinal Study

In Study 2, instead of inferring student experiences from their classroom behavior, we asked students directly about their attitudes towards speaking in class and the Socratic Method. We predicted that women would report speaking less in class than men. We also explored how speaking is related to students’ experience with the Socratic Method and with the social context of classrooms. We hypothesized that professors’ and peers’ judgments would influence students’ decisions about speaking in class.

We collected survey data from 928 law students across four graduating classes representing, on average, 75% of the students in each cohort. Students were invited to take our survey at four time points: at orientation (Time 1), after their first semester (Time 2), before their second year (Time 3), and shortly before graduating (Time 4).33 33.See Trawalter, Shadel & Verkerke, supra note 24, at 38 for descriptive statistics at each time point and attrition checks.Show More

Participants answered closed-ended questions, including questions about speaking in class and the Socratic Method. They also answered open-ended questions about their classroom experience and law school more generally. Open-ended questions were analyzed using text analysis software and human coding.

We examined speaking in class, reasons for not speaking in class, impressions of the Socratic Method, and sense of belonging. Students’ answers to the open-ended questions were hand-coded for mentions of liking or disliking the Socratic Method, seminars, small classes, large classes, experiential/clinical/practical skills classes, lectures, and cold calls. The responses were also coded for mentions of particular issues that appeared frequently across responses, including not having a job, the single-exam grading system, the forced curve, reading load, the cost of law school, and various issues related to the culture of the school.

Results revealed that, on average, women reported speaking less than did men. This result replicates the findings of previous studies at other institutions. Importantly, however, these gender differences varied across time. Our analyses showed that, at Time 1, during orientation, women anticipated speaking just as much as men. And at Time 4, when they neared graduation, they reported speaking up as much as men. But, at Times 2 and 3 (after the first and second semesters respectively of students’ first year), women reported speaking less than did men.

Figure 4: Self-Reported Speaking in Class by Gender anxd Time

Notably, male and female students reported different reasons for not speaking. Compared with women, men were more likely to report a lack of “interest in the subject matter under discussion” as their reason for not speaking. In contrast, women were more likely to report concerns about “classmates’ responses to your statements,” concerns “about how classmates will judge you,” “classroom size,” and “personality of the professor (e.g. supportive vs. challenging).”

We also created an index to measure students’ perceptions of the Socratic Method. Compared with men, women’s perceptions were less positive at all time points. (See Figure 5.) We observed that students’ feelings toward the Socratic Method predicted speaking in class. Students who disliked the Socratic Method more spoke less in class.

Figure 5: Feelings Toward the Socratic Method by Gender and Time

For sense of belonging, we observed a similar pattern. Women and men reported comparable levels of belonging at Time 1. At Time 2, men reported marginally greater belonging, and at Time 3, significantly greater belonging. But by Time 4, that gender gap disappeared; women and men again reported comparable levels of belonging. (See Figure 6.) Moreover, students who reported greater belonging also reported more speaking in class.

Figure 6: Sense of Belonging by Gender and Time

Textual analyses of students’ responses to open-ended survey questions using Linguistic Inquiry and Word Count (“LIWC”) revealed that, on average, women’s statements revealed more anxiety than men’s responses. In addition, women were more likely to mention the Socratic Method and cold calls. They were also more likely to mention disliking the single exam system and experiencing a negative student culture. Mentions of these classroom conditions were correlated with anxious language in students’ open-ended comments. Although these are comparatively small correlations, it is notable that our open-ended questions did not ask about the Socratic Method or gender. And these patterns also mirror results from the closed-ended data analyses.

Students’ writing also revealed what kind of learning environment they prefer. Male students mentioned liking the Socratic Method more than they mentioned liking small classes or experiential classes/seminars. In contrast, female students mentioned liking seminars more than they mentioned liking the Socratic Method.

Study 3: Survey Study

As we wrote up the results of Studies 1 and 2, the global pandemic of 2020 began, dramatically altering the classroom experience for all law students. During the 2020-2021 school year, some UVA students attended class online, while others participated in-person, masked, and physically distanced. This was a time of intense isolation for many students.

In Study 3, we asked third-year students from the class of 2021 about their classroom experiences in the context of the pandemic. We also asked explicitly about the social costs of speaking in class (which we had not asked directly during Study 2). We surveyed the class of 2021 only once, during the final month of their third year of law school. We framed this final study as an inquiry about students’ experiences during the pandemic, again not signaling that gender differences were a focus of the project.

We collected survey data from 164 third-year law students from the class of 2021, which represents 51% of the class. Of these participants, 54% were female, 77% white, with an average age of 26.4. This sample, unlike our Study 2 sample, was majority female.

The timing of this study corresponds to Time 4 of Study 2—surveying third-year students near the end of their time in law school. The pandemic disproportionally affected women in many contexts,34 34.See Emily Lim, Covid-19 Brief: Impact on Women and Girls, U.S. Global Leadership Coalition, (last updated Aug. 30, 2021), https://www.usglc.org/coronavirus/women-and-girls/. [https://perma.cc/2MGQ-LLB6].Show More leading us to wonder whether it also might make it more difficult to participate in class. But our data indicate that, consistent with Study 2, men and women reported speaking in equal measures during their third year, even in the midst of a pandemic. Interestingly, men reported speaking slightly more pre‑pandemic; in other words, participation by men fell during the pandemic. (See Figure 7.)

Figure 7: Speaking by Gender and Time

We also inquired about students’ perceptions of the Socratic Method. Replicating Study 2, women reported disliking the Socratic Method significantly more than did men. Also replicating Study 2, perceptions of the Socratic Method predicted speaking pre-pandemic, such that those who reported disliking the Socratic Method more reported speaking less.

This relationship, however, did not hold during the pandemic. Attitudes towards the Socratic Method predicted gender differences in speaking pre‑pandemic but not during the pandemic. These findings suggest yet another way context appears to shape students’ experiences. Here, we find that the pandemic context—likely due to features of the online environment—changed students’ relationship with the Socratic Method and reduced its gendered impacts. We suspect that professors were less able to use traditional Socratic questioning during the pandemic. Some upper-level classes shifted to recorded lectures and an asynchronous teaching mode, and many adopted a gentler version of asking and answering questions.

Our primary goal in Study 3, however, was to investigate the potential social backlash associated with speaking in class. Women and men reported similar levels of backlash35 35.See Trawalter, Shadel & Verkerke, supra note 24, at 29–30. We did not define backlash in our survey instrument. Based on open-ended responses in both Studies 2 and 3, students conceive of backlash as a kind of social penalty, including unkind comments or loss of social status.Show More against students who talk in class. Women reported more backlash against women than men, and men reported marginally more backlash against men than women. Notably, women reported more backlash against women than men reported against men. In other words, consistent with predictions, the highest levels of backlash were reported by women and against women.

Figure 8: Self-Reported Backlash for Speaking in Class by Gender

III. General Discussion

Just as the “Speak Up” studies found, we too observe that women speak less than men in the law school classroom. However, this effect is not fixed, but dynamic. Based on our data, we conclude the following.

In general, men speak more than women in law school classes. Women spoke less than men in the classes included in Study 1. Men made up 55% of the cohort but were speaking 62% of the time, while women spoke 38% of the time. Men spoke on average for 302 seconds, while women spoke for 194 seconds. These results closely mirror previous Speak Up studies.

Gender gaps in participation are not inevitable. It is significant that in Study 2, observing students at four different points during their legal education, men and women look largely identical at orientation (Time 1). During the orientation survey, in all four cohorts, men and women report equal levels of belonging and are equally optimistic about how much they will be willing to speak in class. Similarly, just before graduation (Time 4), women and men report speaking in class to the same extent and report equal levels of belonging. The gaps in both participation and belonging appear at Times 2 and 3, during the first year of law school. While these differences are self-reported, they mirror the gaps in participation we document in Study 1.

Gender gaps in participation are more likely to appear when students volunteer or when classes are large. A systematic plan for calling on students or recognizing volunteers can increase gender parity, as can smaller class size. We found in Study 1 that men speak more than women, but this effect was driven largely by volunteered answers, not cold calls. In fact, in our data, a systematic plan for participation seems to close gender gaps by preventing volunteering students from dominating class discussion. We also find that women speak less than men in large classes, but not in smaller classes. When answering cold calls or when speaking in smaller classes, women speak just as much as men. In other words, gender gaps in speaking are dynamic, not fixed. They are not inherent to gender but responsive to context.

Social context drives women’s willingness to speak. Our data point to the social context—specifically the social costs of speaking—as the principal explanation for why women speak less under some conditions but not others. In Study 2, at Times 2 and 3 (after the first and second semesters of law school), women report less willingness to speak in class than their male counterparts. They are more likely to report concerns about backlash (“concern about how classmates will judge you” and “classmates’ responses to your statements”) as reasons for not speaking. (In contrast, men were more likely to report a lack of “interest in the subject matter under discussion” as the reason for not speaking.) When considering whether to volunteer an answer, women are weighing how what they say might be perceived, and what it means to have taken up class time with their thoughts. In Study 3, when we explicitly asked about social backlash, both women and men reported that students who speak in class experience backlash. Notably, the highest levels of backlash were reported by women and against women. Students who report backlash against women were also less likely to speak in class.

Women’s responses to open-ended questions in Study 2 also reveal more anxiety than men’s. They are more likely to mention the Socratic Method, cold calls, the single exam system, and a negative culture as reasons for anxiety around speaking. Women also mention liking seminars more than liking the Socratic Method.

The Socratic Method closes gender gaps but is disliked more by women than by men. Study 1 results suggest the systematic cold calling of the Socratic Method can promote gender parity in speaking. But Study 2 data suggest that women do not like the Socratic Method and like it significantly less than men do, across all time points. We think this may be because the Socratic Method triggers greater perceived costs for women. Our data suggest that negative feelings toward the Socratic Method are tied to social backlash and specifically to backlash against women. Students who reported disliking the Socratic Method were also more likely to report that students who speak in class are subject to backlash, and women reported more negative feelings and more backlash, on average.

Social context can be changed. We believe it is significant that, as they approach graduation (Time 4), the women in both Studies 2 and 3 are just as likely as the men to report that they speak in class. They continue to report anxiety about the social and academic context (as evidenced by how they describe the culture and their experiences of the Socratic Method). But they are just as likely as the men to report that they speak in class, despite these negative responses.

Women and men are equally capable of answering questions in law school. In Study 1, in which we analyzed audio recordings of class sessions, we observe similar performance by men and women. Both men and women in these recordings seem to respond adeptly to the Socratic Method. Their answers are equally on point. Both men and women are more likely to use verbal fillers and qualify their answers when answering a Socratic question than when they volunteer. These findings suggest that women and men perform equally well in response to Socratic questioning. They suggest that men and women experience cold calls similarly, at least in terms of their ability to perform—they are equally anxious, and equally on point. As such, our data are not consistent with the notion that women do not speak up because they are less capable, or even less confident. When asked to speak, they appear to be as confident and on point as their male peers.

Implications for Theory. The present findings contribute to the existing literature on gender and confidence, and the so-called confidence gap.36 36.See, e.g.,Kay & Shipman, supra note 16, at xviii; Sandberg, supra note 16, at 8.Show More Many believe that women do not speak up because they are less confident than men. Our data add to a growing literature suggesting otherwise.37 37.See, e.g., Amanatullah, supranote 19, at 256; Hannah Riley Bowles, Linda Babcock & Lei Lai, Social Incentives for Gender Differences in the Propensity to Initiate Negotiations: Sometimes it Does Hurt to Ask, 103 Org. Behav. & Hum. Decision Processes 84, 84 (2007); Andreas Leibbrandt & John A. List, Do Women Avoid Salary Negotiations? Evidence from a Large-Scale Natural Field Experiment, 61 Mgmt Sci. 2016, 2016 (2015).Show More Our findings suggest gender gaps in participation in the law school classroom are a function of context: the use of the Socratic Method, classroom size, and concerns about peer evaluation and social backlash.

Implications for Practice. Law schools can help to shift the context to promote gender equity in class participation.38 38.See Sullivan, supranote 20, at 2 (discussing the anxiety fostered by the Socratic method). As we discussed supra note 11 and supra note 15, these kinds of anxiety-inducing environments can have the effect of favoring men in large classroom discussions. We offer these suggestions tentatively, as we have not designed a study to measure their effects in the classroom. However, we believe they are suggested by our results. We plan to further develop these suggestions in future work.Show More For example, our data show that systematic cold calling can close participation gaps. This system need not involve the Socratic method. For example, students could be asked to lead class discussion, make presentations, and provide feedback to other students according to a preassigned schedule.

Our results also show that, left unchecked, volunteering may produce participation gaps. Faculty can help close this gap by being mindful of whom they call on when students volunteer answers. A professor might consider declaring equal participation among all students a goal of the class so that the class is not dominated by a handful of voices. A professor could limit how often students are allowed to volunteer or place a panel of students “on call” for the week, only allowing those students to volunteer during that time period. Previous work suggests that eliminating social costs changes the behavior of women.39 39.See Amanatullah, supra note 19, at 256.Show More Consequently, we hypothesize that telling students that it is “their job” to volunteer in class during a particular week might result in greater participation by those women. This practice would also shut down opportunities for those not on call to dominate discussion.

Our survey responses also show that many students, both women and men, are made anxious by the Socratic Method. Professors can alleviate this anxiety by giving students advance notice of when they will be speaking and by approaching the colloquy with kindness. They might give students the opportunity to practice their answers by discussing a topic in small groups before sharing those ideas with the class. For discussions of sensitive topics—such as rape or racial discrimination—professors may offer students the option to listen without speaking. Or they might eschew the Socratic Method altogether.

Students improve their ability to articulate their ideas aloud the more practice they have doing so. Calling on a student only once (or twice) during a semester-long class does not offer that student an opportunity to improve and bears very little resemblance to any task the student will encounter as a practicing lawyer. Very infrequent cold calls also raise the stakes and increase the distress students experience when they believe they have not performed well. Law professors who offer students the opportunity to speak more frequently give those students an opportunity to practice and ultimately to succeed. When each student speaks more often, a single “bad” answer becomes simply one of many experiences along the road to mastery. Our data also suggest that smaller classes promote gender parity in participation. Thus, law schools that offer smaller classes will enable more voices to be heard.

Finally, professors who are mindful that students are concerned about social backlash can help lessen that burden. They can alter the social context by stating that all students are practicing the art of listening, articulating their ideas aloud, and responding professionally to one another.40 40.See, e.g., Yale L. Women, supranote 3, at 21; see also Guinier, supra note 6, at 4.Show More A professor might invite students to think of each class as a collaboration, in which they are working together to master the essential skills of thinking and speaking like a lawyer. The classroom offers an important opportunity to practice the respectful professional relationships they hope to have in the future.

Conclusion

Despite the gains women have made in the field of law, for a variety of reasons they continue to be underrepresented in leadership positions in the profession.41 41.For example, in 2017, only 19% of equity partners at law firms were women. See Marc Brodherson, Laura McGee, & Mariana Pires dos Reis, Women in Law Firms, McKinsey & Co.,3 (Oct. 2017), https://www.mckinsey.com/~/media/mckinsey /featured%20‌insights/gender%20equality/women%20in%20law%20firms/women-in-law-firms-final-103017.pdf. [https://perma.cc/737K-Y8BP].Show More Gender differences persist throughout the legal world, including on the judicial bench,42 42.Only about 1/3 of federal judges are female. See U.S. Courts, Women’s History Month, (Jan. 20, 2022), https://www.uscourts.gov/about-federal-courts/educational-resources/annual-observances/womens-history-month. [https://perma.cc/UV4L-RLUS].Show More in legal academia,43 43.Roughly 25% of tenured law school faculty are female. See Am. Bar Found., After Tenure: Post-Tenure Law Professors in the United States, 15, http://www.americanbar foundation.org/uploads/cms/documents/after_tenure_report-_final-_abf_4.1.pdf.Show More and even in the amount of time that women speak in court.44 44.Patton & Smith, supra note 8, at 337.Show More We believe that speaking differences we observe in the law school classroom may contribute to later disparities. Thus, law schools are uniquely positioned to counter these inequities.

The underrepresentation of women in the legal profession is not just an issue of fairness or equity. Lawyers in America play an influential part in government, politics, business, and society. The laws that we make, the populations that we serve, the way that we think about the law, and the way that we use the law are indelibly shaped by the lawyers themselves. Making sure that all students that graduate from law school—women and men—are armed with the experiences that they need to succeed makes it more likely that our female graduates will reach for leadership roles. But our data and the Speak Up studies reveal a systemic problem—men are dominating the conversation. This difference can have consequences for these women, both for their learning and for their willingness to speak up in the future. It also has consequences for other students in the classroom, who miss out on the benefit of hearing diverse voices, and for society at large, if these female voices are unheard in the legal profession. Our work shows how women may be affected by a context that impedes their willingness to speak in class. These effects in turn may hamper their ability to practice raising their voices and collaborating. We hope that our investigation contributes to the ongoing conversation about how law schools and faculty might respond to create a more equitable classroom.

  1. * Molly Bishop Shadel is a Professor Law, General Faculty, at the University of Virginia School of Law. Sophie Trawalter is an Associate Professor of Public Policy and Psychology at the Frank Batten School of Leadership and Public Policy at the University of Virginia. J.H. Verkerke is the T. Munford Boyd Professor of Law at the University of Virginia School of Law. We are grateful to an incredible team of research assistants for their work on the archival study, including Katrina Moberg and Danit Carrier Tal, whose perceptive questions triggered this investigation; Sedi Asem, Pete Chattrabhuti, Patrick Greco, Jonathan Guest, Hyunji Lee, and Halima Nguyen, who worked tirelessly during law school on this project; and undergraduates Derek Zeigler, Clair Jones, Danika Brackett, Katharine Friel, Nikki Kaul, Anthony K. Onuoha, and Chul (Caden) Park. We also thank Jonathan Ashley, Alexander Jakubow, and the UVA Legal Datalab for helping with our data analysis, and Barbara Spellman, John Monahan, Kimberly Robinson, Kim Forde-Mazrui, Naomi Cahn, Anne Coughlin, Kimberly Krawiec, Mitu Gulati, and participants at a UVA Law School faculty workshop for helpful conversations and feedback. ↑
  2. While these studies carry different names (including “Speak Up” and “Speak Now”), we call them Speak Up studies throughout this paper. ↑
  3. Adam Neufeld, Costs of an Outdated Pedagogy? Study on Gender at Harvard Law School, 13 J. Gender, Soc. Pol’y & L. 511, 561–62 (2005) (“The study results show systemic differences between female and male students’ experiences at Harvard Law School in areas ranging from classroom participation and self-confidence to grades and employment.”). ↑
  4. Yale L. Women, Yale Law School Faculty and Students: Speak Up about Gender: Ten Years Later 13–14 (2012). ↑
  5. Daniel E. Ho & Mark G. Kelman, Does Class Size Affect the Gender Gap? A Natural Experiment in Law, 43 J. Legal Stud. 291, 293 (2014). ↑
  6. Mallika Balachandran, Roisin Duffy-Gideon & Hannah Gelbort, Speak Now: Results of a One-Year Study of Women’s Experiences at the University of Chicago Law School, 2019 U. Chi. Legal F. 647, 647–48 (2019). ↑
  7. Lani Guinier, Michelle Fine & Jane Balin, Becoming Gentlemen: Women, Law School, and Institutional Change 1–2 (1997). ↑
  8. See, e.g., Julia Schroeder et al., Fewer Invited Talks by Women in Evolutionary Biology Symposia, 26 J. Evolutionary Biology 2063, 2068 (2013) (finding that female scholars were more likely to decline invitations to a major biology conference); Lynda R. Wiest et al., Researcher Study Thyself: AERA Participants’ Speaking Times and Turns by Gender, 39 Equity & Excellence in Educ. 313, 319 (2006) (finding that when women do speak, they speak for less time than their male counterparts). ↑
  9. Dana Patton & Joseph L. Smith, Lawyer, Interrupted: Gender Bias in Oral Arguments at the US Supreme Court, 5 J.L. & Courts 337, 352 (2017). ↑
  10. Debra Cassens Weiss, Only 25 percent of lead counsel roles in New York courts are held by women, study finds, ABA (Aug. 7, 2017, 7:00 AM), https://www.abajournal.com/‌news/article/only_25_percent_of_lead_counsel_roles_in_new_york_courts_are_held_by_women. [https://perma.cc/9N7N-DPN2]. ↑
  11. Catherine Weiss & Louise Melling, The Legal Education of Twenty Women, 40 Stan. L. Rev
    .

    1299, 1300 (1988). ↑

  12. Guinier et al., supra note 6, at 13–14. ↑
  13. Id. at 8. ↑
  14. Neufeld, supra note 2, at 531. These results seem to have held over time. In 2013, The Harvard Crimson reported:Among the top students in their graduating classes, men and women entering Harvard Law School earn similar undergraduate grades and LSAT scores. But as soon as students step into Wasserstein Hall, a dramatic gender disparity emerges. Indicators suggest that female students participate less and perform worse than their male counterparts over the course of their three years at the Law School.

    Dev A. Patel, In HLS Classes, Women Fall Behind, Harv. Crimson (May 8, 2013), https://www.thecrimson.com/article/2013/5/8/law-school-gender-classroom/. [https://perma.cc/4UUJ-9DPV]. ↑

  15. Yale L. Women, supra note 3, at 3, 13–14. ↑
  16. See, e.g., Ho & Kelman, supra note 4, at 293; Balachandran et al., supra note 5, at 663, 680. ↑
  17. See, e.g., Sheryl Sandberg, Lean In: Women, Work, and the Will to Lead 8 (2013); Katty Kay & Claire Shipman, The Confidence Code: The Science and Art of Self-Assurance—What Women Should Know xviii (2014). ↑
  18. Victoria L. Brescoll, Who Takes the Floor and Why: Gender, Power and Volubility in Organizations, 56 Admin. Sci. Q. 622, 636 (2011). ↑
  19. Emily Amanatullah & Catherine Tinsley, Punishing Female Negotiators for Asserting Too Much…Or Not Enough: Exploring Why Advocacy Moderates Backlash Against Assertive Female Negotiators, 120 Org. Behav. & Hum. Decision Processes 110, 119 (2013). ↑
  20. Emily Amanatullah & Michael Morris, Negotiating Gender Roles: Gender Differences in Assertive Negotiating Are Mediated by Women’s Fear of Backlash and Attenuated When Negotiating on Behalf of Others, 98 J. Personality & Soc. Psych. 256, 263 (2010). ↑

  21. Cf. William M. Sullivan et al., Educating Lawyers: Preparation for the Profession of Law 2 (2007). ↑

  22. Neufeld, supra note 2, at 538, 562 (finding that the Socratic method “may amplify the effect of differences in confidence levels,” but also finding that the “combative” nature of this method did not alone cause gender differences); Yale L. Women, supra note 3, at 14; Guinier et al., supra note 6, at 15. ↑
  23. We realize that students might experience a question for which they have advance notice or which is offered with a friendly tone (sometimes called a “warm call”) differently than a hostile question asked with no advance notice (a “cold call”). We intend to explore these effects in a subsequent paper. For purposes of this paper, we are focusing on the difference between voluntary and professor-controlled participation, and therefore use ‘cold call’ to include both warm and cold calls. ↑
  24. On the Importance of Social Context for Women’s Voices: Gender Differences in Law School Classroom Participation, OSF (last updated Jan. 5, 2022, 9:45 AM), https://osf.io/xat7p/?view_only=33dd7c17ade5437cbaf2a86d0f11752b. [https://perma.cc/78DM-GZ6P]. ↑
  25. Sophie Trawalter, Molly Shadel & J.H. Verkerke, On the Importance of Social Context for Women’s Voices: Gender Differences in Law School Classroom Participation, 2 (on file with authors at https://osf.io/937cg/?view_only=33dd7c17ade5437cbaf2a86d0f11752b). [https://perma.cc/2VXX-HDN3]. ↑
  26. Id. at 11 n.1. Hereinafter abbreviated as “RA.” Five classes were transcribed by five RAs, to train RAs on transcription, and one class was inadvertently transcribed twice. All other classes were transcribed by one RA. ↑
  27. Id. at 11 n.2. “Five classes were coded by five RAs, to train RAs on coding, and one class was inadvertently coded by only one RA.” ↑
  28. Id. at 12. ↑
  29. See id. at 44. The Law School website published this enrollment information and did not report any students identifying as gender non-binary. In Studies 2 and 3, students were able to identify as gender non-binary, though no study participants selected that option. ↑
  30. See Herbert H. Clark & Jean E. Fox Tree, Using Uh and Um in Spontaneous Speaking, 84 Cognition 73, 90–91 (2002). ↑
  31. See Mansur Lalljee, & Mark Cook, Anxiety and Ritualized Speech, 66 Brit. J. Psych. 299, 300 (1975). ↑
  32. See Trawalter, Shadel & Verkerke, supra note 24, at 15 n.4. We define small classes as those containing roughly 30 students, medium classes as those containing roughly 60 students, and large classes as those containing roughly 90 students. ↑
  33. For a similar finding at Stanford Law School, see Ho & Kelman, supra note 4, at 310 (documenting the impact class size can have on GPA across genders). ↑
  34. See Trawalter, Shadel & Verkerke, supra note 24, at 38 for descriptive statistics at each time point and attrition checks. ↑
  35. See Emily Lim, Covid-19 Brief: Impact on Women and Girls, U.S. Global Leadership Coalition, (last updated Aug. 30, 2021), https://www.usglc.org/coronavirus/women-and-girls/. [https://perma.cc/2MGQ-LLB6]. ↑
  36. See Trawalter, Shadel & Verkerke, supra note 24, at 29–30. We did not define backlash in our survey instrument. Based on open-ended responses in both Studies 2 and 3, students conceive of backlash as a kind of social penalty, including unkind comments or loss of social status. ↑
  37. See, e.g., Kay & Shipman, supra note 16, at xviii; Sandberg, supra note 16, at 8. ↑
  38. See, e.g., Amanatullah, supra note 19, at 256; Hannah Riley Bowles, Linda Babcock & Lei Lai, Social Incentives for Gender Differences in the Propensity to Initiate Negotiations: Sometimes it Does Hurt to Ask, 103 Org. Behav. & Hum. Decision Processes 84, 84 (2007); Andreas Leibbrandt & John A. List, Do Women Avoid Salary Negotiations? Evidence from a Large-Scale Natural Field Experiment, 61 Mgmt Sci. 2016, 2016 (2015). ↑
  39. See Sullivan, supra note 20, at 2 (discussing the anxiety fostered by the Socratic method). As we discussed supra note 11 and supra note 15, these kinds of anxiety-inducing environments can have the effect of favoring men in large classroom discussions. We offer these suggestions tentatively, as we have not designed a study to measure their effects in the classroom. However, we believe they are suggested by our results. We plan to further develop these suggestions in future work. ↑
  40. See Amanatullah, supra note 19, at 256. ↑
  41. See, e.g., Yale L. Women, supra note 3, at 21; see also Guinier, supra note 6, at 4. ↑
  42. For example, in 2017, only 19% of equity partners at law firms were women. See Marc Brodherson, Laura McGee, & Mariana Pires dos Reis, Women in Law Firms, McKinsey & Co., 3 (Oct. 2017), https://www.mckinsey.com/~/media/mckinsey /featured%20‌insights/gender%20equality/women%20in%20law%20firms/women-in-law-firms-final-103017.pdf. [https://perma.cc/737K-Y8BP]. ↑
  43. Only about 1/3 of federal judges are female. See U.S. Courts, Women’s History Month, (Jan. 20, 2022), https://www.uscourts.gov/about-federal-courts/educational-resources/annual-observances/womens-history-month. [https://perma.cc/UV4L-RLUS]. ↑
  44. Roughly 25% of tenured law school faculty are female. See Am. Bar Found., After Tenure: Post-Tenure Law Professors in the United States, 15, http://www.americanbar foundation.org/uploads/cms/documents/after_tenure_report-_final-_abf_4.1.pdf. ↑

  45. Patton & Smith, supra note 8, at 337. ↑

©Virginia Law Review Association. All rights reserved.