3rd International Workshop on

Personalising Feedback

March 2020

Frankfurt, Germany as part of the LAK20 conference

Effective and timely feedback of and for learning has been shown to be essential in influencing students’ achievement and promoting autonomy and self-regulation. While assessment practices have received considerable attention over the past two decades, the focus on feedback to students has remained relatively scarce.

Scaled and personalised feedback (using Learning Analytics) has become a sort of holy grail for educators aspiring to improve their students’ learning and their satisfaction with the learning experience.

In most cases there are three reasons why feedback fails:

  1. students do not perceive feedback as feedback, or they don’t use it effectively (i.e. there are differences in how students and teachers understand feedback);
  2. the feedback is not appropriate or good enough (this could be because of how feedback is provided or because the focus and purposes are not shared: i.e. providing feedback is reduced to a summative, corrective and transmissive process);
  3. teachers fail to provide added value (i.e. too generic or superficial, providing a final judgement on students’ submitted assignments instead of an opportunity to reflect and improve).

Although LA have made tangible connections with critical aspects that can strongly shape learning, such as learning design and self-regulation, the provision of feedback to students has been relatively neglected. This is despite the affordances of LA to leverage the generation of theoretical and technical mechanisms for understanding and improving learning by "informing and empowering instructors and learners" (Siemens & Baker, 2012).

To allow this to happen, teachers need concrete tools and approaches to bridge the gap between LA research and classroom practice. LA systems are starting to support teachers with means to provide rich feedback beyond typical early warning messages (e.g. SRES, OnTask), but it is clear that there is a need and appetite in the LA community of research and practice to further explore data-informed student-centred pedagogies for scaling up feedback processes.

Teachers need concrete approaches and support mechanisms to bridge the gap between LA research and classroom practice. This third workshop at LAK specifically focuses on educators and practitioners, their experiences and their stories in engaging with feedback and assessment practices supported by LA tools and approaches.

With two successful workshops (LAK18, LAK19 sites) delivered at LAK which explored the issues of tools able to support the scaling of personalized feedback and how these benefits students, this third workshop shifts the attention to educators and practitioners.

Scope of the workshop

This workshop brings together scholars and practitioners to explore examples of how educators use information (data) to enhance the feedback process for increasing students’ engagement and performance, by scaffolding their learning processes with appropriate feedback on both content and strategies. The workshop has three primary goals:

  • Give a multidisciplinary theoretical foundation for practitioners and researchers in LA for the effective data-informed feedback practices in HE;
  • Showcase extant or planned approaches for scaling feedback and consider how students receive and use the feedback; special attention is on approaches that are data-driven and personalised;
  • Promote reflection on both pedagogical and technological approaches to improve feedback practices targeted at the improvement of student learning and their ability to self-regulate learning.

Who is the workshop for?

Those who wish to understand and apply principles of ‘good’ data-driven feedback for learning. Given the explicit multidisciplinary nature of the workshop we expect that it will provide an opportunity to discuss and share innovations, impact on learning, and explore future directions in the application of learning analytics (LA) to personalisation of feedback. Likely interested participants are:

  • Educators/teachers and researchers
  • Technologists and educational developers
  • Learning scientists and data scientists/analysts
  • Academic managers
  • and anyone else interested in personalisation of learning and teaching

We expect a range of presentations that will cover practical, evidence-based approaches to personalising data-driven feedback at scale with a focus on teacher and practitioner perspectives. Participants will be able to:

  • Obtain a broad perspective of different approaches to using data for personalising feedback
  • Enhance their understanding of the forms of feedback that could improve student learning
  • Gain an appreciation of the range of contexts where feedback can be valuable, and how data can inform these
  • Discuss cases, issues, and potential solutions to implementing LA-enhanced feedback practices
  • Connect with researchers and practitioners seeking to personalise high-quality feedback, yielding opportunities for collaborating on approaches and tools across attending institutions. After the workshop, given the commitment to further collaborations, contributors will be invited to consider more substantial submissions with the intention to collate the works into a special issue of a journal or an edited book on the topic.

Some References

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698-712. doi:10.1080/02602938.2012.691462
Black, P., Harrison, C., & Lee, C. (2003). Assessment for learning: Putting it into practice. McGraw-Hill Education (UK).
Brown, S., & Knight, P. (1994). Assessing learners in higher education. Psychology Press.
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219–233. https://doi.org/10.1080/03075070600572132
Carless, D. (2019). Feedback loops and the longer-term: Towards feedback spirals. Assessment & Evaluation in Higher Education, 44(5), 705–714. https://doi.org/10/gf33kc
Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395–407. https://doi.org/10/ftvjq4
Evans, C. (2013). Making Sense of Assessment Feedback in Higher Education. Review of Educational Research, 83(1), 70–120. https://doi.org/10/gf82tm
Forsythe, A., & Johnson, S. (2017). Thanks, but no-thanks for the feedback. Assessment & Evaluation in Higher Education, 42(6), 850–859. https://doi.org/10.1080/02602938.2016.1202190
Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. routledge.
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Henderson, M., Ryan, T., & Phillips, M. (2019). The challenges of feedback in higher education. Assessment & Evaluation in Higher Education, 1-16. doi:10.1080/02602938.2019.1599815
Hounsell, D. (2003). Student feedback, learning and development. Higher Education and the Lifecourse, 67–78.
Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research & Development, 27(1), 55–67. https://doi.org/10.1080/07294360701658765
Krause, K. L., Hartley, R., James, R., & McInnis, C. (2005). The first year experience in Australian universities: Findings from a decade of national studies. Centre for the Study of Higher Education, University of Melbourne Melbourne. Retrieved from http://www.cshe.unimelb.edu.au
Lackritz, J. R. (2004). Exploring burnout among university faculty: Incidence, performance, and demographic issues. Teaching and Teacher Education, 20(7), 713–729. https://doi.org/10/fgnrv8
Liu, D. Y.-T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2017). Data-Driven Personalization of Student Learning Support in Higher Education. In Learning Analytics: Fundaments, Applications, and Trends (pp. 143–169). Springer, Cham. https://doi.org/10.1007/978-3-319-52977-6_5
Lizzio, A., & Wilson, K. (2008). Feedback on assessment: students’ perceptions of quality and effectiveness. Assessment & Evaluation in Higher Education, 33(3), 263–275. https://doi.org/10.1080/02602930701292548
McDowell, L., Smailes, J., Sambell, K., Sambell, A., & Wakelin, D. (2008). Evaluating assessment strategies through collaborative evidence-based practice: can one tool fit all? Innovations in Education and Teaching International, 45(2), 143– 153. https://doi.org/10.1080/14703290801950310
Nicol, D. (2009). Assessment for learner self-regulation: enhancing achievement in the first year using learning technologies. Assessment & Evaluation in Higher Education, 34(3), 335–352. https://doi.org/10.1080/02602930802255139 Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501–517. https://doi.org/10.1080/02602931003786559
O’Donovan, B. M., den Outer, B., Price, M., & Lloyd, A. (2019). What makes good feedback good? Studies in Higher Education, 1-12. doi:10.1080/03075079.2019.1630812
Orsmond, P., Maw, S. J., Park, J. R., Gomez, S., & Crook, A. C. (2013). Moving feedback forward: theory to practice. Assessment & Evaluation in Higher Education, 38(2), 240–252. https://doi.org/10.1080/02602938.2011.625472
Pace, F., D’Urso, G., Zappulla, C., & Pace, U. (2019). The relation between workload and personal well-being among university professors. Current Psychology. https://doi.org/10/gf3kqp
Pardo, A., Bartimote-Aufflick, K., Buckingham Shum, S., Dawson, S., Gao, J., Gašević, D., . . . Vigentini, L. (2018). OnTask: Delivering Data-Informed Personalized Learning Support Actions. Journal of Learning Analytics, 5(3), 235-249. doi:10.18608/jla.2018.53.15
Pardo, A. (2017). A feedback model for data-rich learning experiences. Assessment & Evaluation in Higher Education, 0(0), 1–11. https://doi.org/10.1080/02602938.2017.1356905
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2017.). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1). https://doi.org/10.1111/bjet.12592
Pitt, E., & Norton, L. (2017). ‘Now that’s the feedback I want!’ Students’ reactions to feedback on graded work and what they do with it. Assessment & Evaluation in Higher Education, 42(4), 499–516. https://doi.org/10.1080/02602938.2016.1142500
Poulos, A., & Mahony, M. J. (2008). Effectiveness of feedback: the students’ perspective. Assessment & Evaluation in Higher Education, 33(2), 143–154.https://doi.org/10.1080/02602930601127869
Radloff, A., Coates, H., James, R., & Krause, K.-L. (2011). Report on the Development of the University Experience Survey. Higher Education Research. Retrieved from http://works.bepress.com/hamish_coates/79
Rowe, A. D., & Wood, L. N. (2009). Student Perceptions and Preferences for Feedback. Asian Social Science, 4(3), 78. https://doi.org/10.5539/ass.v4n3p78
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144. https://doi.org/10.1007/BF00117714
Sadler, D. R. (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535–550. https://doi.org/10.1080/02602930903541015
Siemens, G., & Baker, R. S. J. d. (2012). Learning Analytics and Educational Data Mining: Towards Communication and Collaboration. In Proceedings of the 2Nd International Conference on Learning Analytics and Knowledge (pp. 252–254). New York, NY, USA: ACM. https://doi.org/10.1145/2330601.2330661
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47(Supplement C), 157–167. https://doi.org/10.1016/j.chb.2014.05.038
Vigentini, L., Liu, D. Y.-T., Lim, L., & Martinez-Maldonado, R. (2018). Personalising feedback at scale: approaches and practicalities. In Companion Proceedings of the 8th International Conference on Learning Analytics & Knowledge (LAK’18). Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors’ written responses. Assessment & Evaluation in Higher Education, 31(3), 379–394. https://doi.org/10.1080/02602930500353061
Williams, J., & Kane, D. (2008). Exploring the National Student Survey Assessment and feedback issues. The Higher Education Academy, Centre for Research into Quality.