‘There is little correspondence between learner perceptions of lesson effectiveness and actual instructional value. In short, liking is not the same as learning.’ (Bjork, et al. 2013)

'Students’ judgments of their own learning are often misled by intuitive yet false ideas about how people learn. In educational settings, learning experiences that minimize effort and increase the appearance of fluency, engagement, and enthusiasm often inflate students’ estimates of their own learning, but do not always enhance their actual learning. We review the research on these “illusions of learning,” how they can mislead students’ evaluations of the effectiveness of their instructors, and how students’ evaluations of teaching effectiveness can be biased by factors unrelated to teaching. We argue that the heavy reliance on student evaluations of teaching in decisions about faculty hiring and promotion might encourage teaching practices that boost students’ subjective ratings of teaching effectiveness, but do not enhance—and may even undermine—students’ learning and their development of metacognitive skills' (Carpenter, Witherby, Tauber)

Measuring learning is also not easy. Thalheimer's Learning Transfer Evaluation (LTEM) model helps consider what kind of learning you’re actually measuring, and avoid poor proxies for measuring learning.

Bibliography:

Barre, E. (2018) 'Research on Student Ratings Continues to Evolve. We Should, Too'. in Rice CTE Blog

Bennet, R., and Kane, S. (2014) Students' interpretations of the meanings of questionnaire items in the National Student Survey. Quality in Higher Education, 20:2, pp. 129-164

Bjork, R., Dunlosky, J., and Kornell, N. (2013) 'Self-Regulated Learning: Beliefs, Techniques, and Illusions'. Annual Review of Psychology, 64, pp417–44.

Boring, A., Ottoboni, K., and Stark, P. (2016) 'Student evaluations of teaching (mostly) do not measure teaching effectiveness'. OpenScience Research.

Boring, A., Ottoboni, K., and Stark, P. (2015) 'Student evaluations of teaching are not only unreliable, they are significantly biased against female instructors'. In: LSE Impact Blog.

Braga, M., Paccagnella, M., and Pellizzari, M. (2014) 'Evaluating students’ evaluations of professors'. Economics of Education Review 41, 71–88.

Buckley, A. (2019) 'How much are your NSS results really telling you?' In: wonkhe

Carpenter, S. K., Witherby, A. E., and Tauber, S. K. (2020) 'On Students’ (Mis)judgments of Learning and Teaching Effectiveness'. Journal of Applied Research in Memory and Cognition. Vol. 9, issue 2. pp 137-270

Carrell, S. E., and West, J. E. (2010) 'Does professor quality matter? Evidence from random assignment of students to professors'. Journal of Political Economy, 118, 409–432.

Kornell, N., & Hausman, H. (2016) 'Do the best teachers get the best ratings?' Frontiers in Psychology, 7, 570:1-8.

LSE Impact Blog 'Research on gender bias receives less attention than research on other types of bias'

Handley, I. M., Brown, E. R., Moss-Racusin, C. A., L. Smith, J. (2015) 'Quality of evidence revealing subtle gender biases in science is in the eye of the beholder'. Proceedings of the National Academy of Sciences Oct 2015, 112 (43) 13201-13206; https://www.pnas.org/content/pnas/112/43/13201.full.pdf

Mengel, F., Sauermann, J., Zölitz, U. (2019) 'Gender Bias in Teaching Evaluations', Journal of the European Economic Association, Volume 17, Issue 2, Pages 535–566, https://doi.org/10.1093/jeea/jvx057

Neelan, M., and Kirschner, P. A. (2020) 'Have they learnt something?' in 3 Star Learning Experiences. Available at https://3starlearningexperiences.wordpress.com/2020/01/07/have-they-learnt-something/

Peters, D. (2019) 'Do universities put too much weight on student evaluations of teaching?' In: University Affairs

Sumeracki, M. (2020) 'Teaching Effectivesness vs Teaching Evaluations'. In The Learning Scientists.

Thalheimer, W. (2018) 'The learning-transfer evaluation model: Sending messages to enable learning effectiveness'. Available at https://WorkLearning.com/Catalog

Uttl, B., White, C. A., & Gonzalez, D. W. (2017) 'Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related'. Studies in Educational Evaluation, 54, 22-42.