Gender Bias in Teaching Evaluations

Academics go through a fairly well-defined career ladder. We all know more or less what we should do to be able to work and progress at the higher education institute from the beginning of our careers which usually start as early our undergraduate degree.

Yet as in many industries, the sector is not immune to gender discrimination and gender biases in career progression. From the start, male students are more favored to be hired as research assistants in science labs than female students (Moss-Racusin et al. 2012). Then once they are hired, female researchers are judged more harshly for their research outputs and face more obstacles in publication process than male researchers (Sarson 2017; Lee and Ellemers 2015). But this does not end in here...

The recent evidence in two prominent economic studies (and many more studies in sociology and education literature) find that women receive lower teaching evaluations from their students than men. Both Boring (2017) and Mengel et al. (2019) analyse the data from large number of students in French and Dutch universities and find that female instructors receive lower teaching evaluation scores than male lecturers. Their studies control for large number of other possible variables that could have an effect on male and female lecturer evaluation scores. Having access to a large number of control variables is always encouraging and credible study design which strengthens their results.

I enjoyed reading the working papers and listening to the presentations of Anne Boring and Friederike Mengel back in 2016. This inspired me and two fellow researchers Irina Cojuharenco and Nigel Burnell to extend their work and experimentally investigate the reasons behind the differences in teaching evaluations for male and female lecturers. It set us on the course of running four large scale and a few small scale pre-test studies to test for several possible mechanisms behind the gender bias in teaching evaluations.

In Study 1, we hypothesized that the teaching style of male and female instructors may be different and this may influence the gender bias. We recruited 18-30 year olds from an online platform and asked them to read an astronomy lecture either prepared by a male or a female lecturer. We found that women received lower scores for the lecture that was written in colder style than men. So being cold or some even would call it "more professional" carried a penalty for female lecturers but not for male lecturers. This was indeed what could have contributed to the findings of Boring and Mengel et al.

In Study 2, we checked whether in the colder lectures, senior female academics are judged less harshly than males ones compared to the difference between the junior ones. Indeed we find that when the lecturer is a professor, there is no difference in evaluations of the same lecture content prepared either by a male or a female academic. We also found that simply being aware of the gender bias in academia did not affect the evaluations of the participants. So simply letting students know about the gender bias in academia may not be an effective tool to eliminate it.

We decided to continue this line of research and investigate whether the results are robust by running Studies 3 and 4. Can we extrapolate the results into other academic disciplines like Finance - which is another stereotypically male-dominated discipline. We conducted further two studies to test the effect of teaching style and academic credentials on student evaluations. We recruited more than 2000 student participants from an online platform to read a Finance lecture and evaluate the lecturer. Interestingly enough, in these latter two studies we did not find any gender bias in teaching evaluations.

A researcher always faces a dilemma of what to conclude from studies that provide different and sometimes inconsistent results. We asked ourselves repeatedly what could have contributed to the differences between these studies. Why did we find a clear-cut gender bias in Study 1 and 2 but no gender bias whatsoever in Study 3 and 4? In addition to changing the discipline from astronomy to finance, we also changed how we measure evaluations (with less or more questions), where our participants come from (from the UK or from all over the world) and whether the participants were students or not (all 18-30 year olds versus only student 18-30 year olds).

Yet at this point we can only speculate whether this or that study design choice contributed to uncovering the gender bias or covering it up. More studies probably are needed to answer why gender bias is present in some but not other studies. For now, we have written a detailed description of the whole project and published a discussion series paper here for those interested. We hope the paper sparks interest from fellow researchers and there will be more work in the topic.

Published: 30/04/2020