Description
View the full paper version with tables and images here!
Challenges and Opportunities of Learning Analytics Adoption in Higher Education Institutes: A European Perspective
L.W. van Meeuwen1, B. Rienties2, H. Drachsler3,4, O. Viberg5, R. Conijn1, C.A. Vonk1, J.W.A. Brijan6, M. Specht7
1 Eindhoven University of Technology, The Netherlands
2 The Open University, The United Kingdom
3 DIPF - Leibniz Institute for Research and Information in Education, Germany
4 Open Universiteit, The Netherlands
5 Royal Institute of Technology, Sweden
6 University of Groningen, The Netherlands
7 Delft University of Technology, The Netherlands
ABSTRACT: The successful application of Learning Analytics (LA) can improve student learning outcomes, student support and teaching. The key-challenges for LA adoption (i.e., Ethics, Leadership, Analytics culture, Analytics capabilities, Stakeholder involvement, and Technology) have been investigated . However, large-scale adoption remains lacking as does research into it.
This half-day workshop organized in cooperation of several European universities has the aim to provide support to researchers and practitioners for realizing large scale adoption of practicable LA within higher education and the essential research thereon. Research and insights from the varied European contexts will be presented in this workshop for comparison with researchers from other non-European and other European contexts. The idea is that this exchange will provide insights to learn from the differences and overcome global challenges for successful adoption of LA at scale.
Keywords Adoption at Scale, Contextual challenges, Contextual opportunities, Higher education
Introduction
The practical application of Learning Analytics (LA) can benefit learning (e.g., Foster & Francis, 2020) and has the potential to further improve educational quality in higher education (Drachsler 2023; Praharaj, 2021; Seufert et al., 2019; Viberg & Gronlund, 2023). It is therefore valuable to see the ambition of institutions to put LA to practical use in order to improve educational quality (e.g., The Open University, 2015; Eindhoven University of Technology, 2018). A range of studies have shown that actual adoption by educators and institutions are still in its infancy more than a decade since the first Learning Analytics Knowledge conference was held in 2011 (Tsai et al. 2018, Hernandez-de-Menendez et al., 2022, Viberg et al., 2018) except for some examples (e.g., Herodotou et al., 2019). The key challenges for LA adoption include concerns around: Ethics, Leadership, Analytics culture, Analytics capabilities, Stakeholder involvement, and Technology (cf., Alzahrani, 2023). These challenges must be overcome for implementation at scale. Insights from different contexts can support this.
European Contexts
While the introduction of General Data Protection Regulation (GDPR; European Union, 2018) across Europe and the recent AI policy on data in teaching and learning for educators (European Commission, 2022) provide guidelines about how to deal with data protection and integration, it is widely documented that each member state has their own perspective on ethics, privacy and data (Drachsler & Greller, 2016; Holmes et al., 2021; Korir et al., 2023). For example, German educational systems use a rather strict interpretation of data and ethics, while in the UK there seems to be a greater appetite for implementing learning analytics and data infrastructures to support students and educators. This might be a reflection of the underlying cultural differences, interpretations of regulations, the engagement of key stakeholders, and ways of collaboration between institutions within a country (e.g., 4TU collaboration the Netherlands).
Comparing contexts to learn
Next to the differences within Europe, there are several aspects in the European context that bring up different challenges than in other international education systems, including GDPR and a more Humboldtian vision of higher education rather than a human capital perspective. In Europe, the barrier to obtaining data and getting started with LA in educational practice is often high. Institutions need to develop capabilities in ethics and privacy within an constantly changing environment (Knobbout et al. 2023; Prinsloo et al., 2023), which can be considered a data ecology rather than a closed ecosystem. Also, stakeholder engagement develops: conditions are being created and barriers overcome within institutions that allow early adopters to start pilot implementations in conjunction with research (e.g., Knobbout et al., 2023). To meet educational standards and adhere to institutional ethical and privacy guidelines, intricate design processes in collaboration with Educational Technology vendors are often requisite, as off-the-shelf solutions may not sufficiently address these criteria (Hernandez-de-Menendez et al., 2022; Knobbout et al., 2023; Tsai & Gasevic, 2017; Drachsler & Greller, 2016). How do the two challenges listed below compare to other contexts in the European dimension, and what can stakeholders (like researchers) learn from other contexts to increase LA adoption globally?
Challenges
In summary, the two key challenges will be discussed in this workshop:
Challenge 1: Although there is an increased uptake of LA applications that go beyond pilots (Leitner et al., 2017), few higher education institutions have yet implemented Learning Analytics at scale (Knobbout et al., 2023).
Challenge 2: As a result, scientific output in this area of practical use of scalable LA in higher education is still limited (SoLAR, 2023).
Aim and scope of the workshop
More specifically, the aim of the workshop is to deal with the two challenges stated above by learning from other contexts. Therefore, the workshop will zoom in on contextual influences on two main bottlenecks for the issue of limited adoption at scale: 1. The process of obtaining data and 2. Stakeholder engagement (See Figure 1).
From the research perspective, participants get to exchange expertise in the domain of practical application of LA from a different context (e.g., European, Asian, etc.); gain insights in research on practical application of LA (e.g., case studies); and get the opportunity to share state-of-the-art research on improving data processes and stakeholder engagement in other contexts.
From a practical perspective, participants gain insight into potential bottlenecks for LA applications in their own institutions and simultaneously get tools to solve them.
Workshop Structure
The workshop design will allow for half a day meeting and comprises a combination of an Interactive Workshop and a Mini-track Symposium. The design includes discussions, group discussions, presentations, and voluntary contributions.
We ask all participants to consider a research contribution detailing (case-)studies where LA benefits students learning, specifying approaches to improving data processes, and stakeholder engagement (such as participatory design and institutional collaboration). Contributions of 10-minute presentations may be submitted in the form of an abstract of up to 300 words. Based on the abstract, the workshop organization will carefully review the submissions and select a compelling array of diverse contributions that fit the workshop structure, schedule, and enrich the workshop. Further interaction before and after the workshop will be supported by a website.
Attract participants
The organizing committee anticipates two groups of participants. First, participants who add a scientific contribution to the workshop (e.g., PhD candidates, Professors) and second, participants who want to share more practical experiences about contextual influences on LA adoption (e.g., (program)managers, research support staff) that can enable research.
To approach both target groups, the organizers’ professional networks will be used as well as the university alliance networks and the institutional members from SoLaR. In addition, (sample) posts will be made available for LinkedIn, and relevant mailing lists to recruit broadly. The workshop organization is aiming for 25 - 35 participants.
Program Design Outline
The organizer presents the scope-framework of the workshop during the opening. The workshop follows the two pillar structure (see Figure 1). Each pillar will be introduced by the organization including examples from existing literature. The pillars will be enriched by the input from the participants who send in research contributions. Each pillar ends with group discussions on contextual impact of the bottlenecks to overcome and how and what we can learn from the differences. In the final part, the organization wraps up the workshop and summarizes the conclusions obtained in the break-out rooms. This will be used to sketch the open research questions and eye-openers on contextual influences that were obtained by comparing insights in LA adoption from several contexts.
References
Alzahrani, A. S., Tsai, Y. S., Iqbal, S., Marcos, P. M. M., Scheffel, M., Drachsler, H., ... & Gasevic, D. (2023). Untangling connections between challenges in the adoption of learning analytics in higher education. Education and Information Technologies, 28(4), 4563-4595.
Drachsler, H., & Greller, W. (2016). Privacy and Analytics–it’sa DELICATE issue. A Checklist to establish trusted Learning Analytics. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, Edinburgh.
Drachsler, H. (2023). Towards Highly Informative Learning Analytics. Open Universiteit, Netherlands, Heerlen. https://doi.org/10.25656/01:26787
Eindhoven University of Technology, 2023. TU/e Vision on Education. Retrieved from https://www.tue.nl/en/our-university/about-the-university/tue-strategy-2030/talent
European Commission. (2022). Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for educators. Publications Office of the European Union. https://doi.org/doi/10.2766/153756
European Union. (2018). General Data Protection Regulation. Retrieved from https://gdpr-info.eu/
Foster, C., & Francis, P. (2020). A systematic review on the deployment and effectiveness of data analytics in higher education to improve student outcomes. Assessment & Evaluation in Higher Education, 45(6), 822-841.
Herodotou, C., Rienties, B., Verdin, B., & Boroowa, A. (2019). Predictive learning analytics ‘at scale’: Guidelines to successful implementation in Higher Education based on the case of the Open University UK. Journal of learning Analytics, 6(1), 85-95.
Hernández-de-Menéndez, M., Morales-Menendez, R., Escobar, C. A., & Ramírez Mendoza, R. A. (2022). Learning analytics: state of the art. International Journal on Interactive Design and Manufacturing (IJIDeM), 16, 1209–1230. https://doi.org/10.1007/s12008-022-00930-0
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., & Koedinger, K. R. (2021). Ethics of AI in Education: Towards a Community-Wide Framework. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-021-00239-1
Jivet, I., Wong, J., Scheffel, M., Valle Torre, M., Specht, M., & Drachsler, H. (2021, April). Quantum of Choice: How learners’ feedback monitoring decisions, goals and self-regulated learning skills are related. In LAK21: 11th international learning analytics and knowledge conference (pp. 416-427).
Korir, M., Slade, S., Holmes, W., Héliot, Y., & Rienties, B. (2023). Investigating the dimensions of students’ privacy concern in the collection, use and sharing of data for learning analytics. Computers in Human Behavior Reports, 9, 100262. https://doi.org/https://doi.org/10.1016/j.chbr.2022.100262
Knobbout, J., 2023. Learning Analytics Ontcijferd [learning analytics decomposed]. BOOM
Knobbout, J., & van der Stappen, E. (2020). A capability model for learning analytics adoption: Identifying organizational capabilities from literature on big data analytics, business analytics, and learning analytics. Journal of Learning Analytics and Artificial Intelligence for Education. http://www. i-jai. org, 2(1), 47-66.
Knobbout, J., van der Stappen, E., Versendaal, J., & van de Wetering, R. (2023). Supporting Learning Analytics Adoption: Evaluating the Learning Analytics Capability Model in a Real-World Setting. Applied Sciences, 13(5), 3236.
Leitner, P., Khalil, M., & Ebner, M. (2017). Learning analytics in higher education—a literature review. Learning analytics: Fundaments, applications, and trends: A view of the current state of the art to enhance E-learning, 1-23.
Praharaj, S., Scheffel, M., Drachsler, H., & Specht, M. (2021). Literature review on co-located collaboration modeling using multimodal learning analytics—Can we go the whole nine yards?. IEEE Transactions on Learning Technologies, 14(3), 367-385.
Prinsloo, P., Khalil, M., & Slade, S. (2023, August). A Critical Consideration of the Ethical Implications in Learning Analytics as Data Ecology. In European Conference on Technology Enhanced Learning (pp. 371-382). Cham: Springer Nature Switzerland.
Seufert, S., Meier, C., Soellner, M., & Rietsche, R. (2019). A pedagogical perspective on big data and learning analytics: A conceptual model for digital learning support. Technology, Knowledge and Learning, 24, 599-619.
SoLAR (2023). SoLAR strategic plan April 2023 - May 2025. https://www.solaresearch.org/
The Open University (2015). Policy on ethical use of student data for learning analytics. The Open University. September 2014 (2015), 1–11.
Tsai, Y. S., & Gasevic, D. (2017, March). Learning analytics in higher education---challenges and policies: a review of eight learning analytics policies. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 233-242).
Tsai, Y. S., Moreno-Marcos, P. M., Tammets, K., Kollom, K., & Gašević, D. (2018, March). SHEILA policy framework: informing institutional strategies and policy processes of learning analytics. In Proceedings of the 8th international conference on learning analytics and knowledge (pp. 320-329).
Viberg, O., & Grönlund, Å. (Eds.). (2023). Practicable Learning Analytics. Springer Nature.
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89(December 2018), 98-110. https://doi.org/10.1016/j.chb.2018.07.027