Novel Measures Reveal Subtle Gender Bias in Academic Job Recommendations

Linguistic analysis of 2,206 lettres of recommendation compared gender bias in two disciplines differing in women's representation: experimental; particle physics (EPP, 60% female). Standard lexical measures (e.g., "communal," "agentic," and "standout") did not show bias against women in either discipline. On the contrary, in letters about women, female physicists used more positive-affect words, while male physicists used fewer negative-affect words as well as more references to hard work/effort. Neither discipline showed gender differences in the rank of letter writers, but social scientists wrote longer letters about women and wrote more often for candidates of their own gender. However, standard lexical measures assess only overt expressions of bias and may miss more subtle gendered language. We therefore developed a novel, open-ended measure of gendered usages. In striking contrast to conventional measures, our open-ended analysis uncovered troubling gender disparities. Positive references (e.g., to talent, innovation, creativity, etc.) appeared more often in letters about men in EPP and about women in social science (although female EPP candidates were more likely to be characterized as "brilliant"). Two of the largest gender disparities were for references to "physicist" in EPP and "science" in social science. These terms were used in more letters for men in both disciplines, possibly indicating unconscious gender stereotypes, even in a majority-female discipline. We conclude that future studies of linguistic bias should include open-ended measures, and that policies to correct gender imbalances in physics should raise awareness of subtleties of potential bias in letters of recommendation.