5th Workshop on Culturally Aware Learning Analytics (CARLA)

Value-sensitive Design

CARLA workshop at ECTEL 2023, Seprtember 4-8, Aveiro, Portugal

 

Workshop theme

Over the last decade, we have seen many examples of learning analytics (LA) systems being implemented and used in various countries in different ways, however, often at a limited scale (Viberg et al. 2018). The scalability problem can be associated with, but not limited to, the differences in the expectations that stakeholders such as teachers and students have towards LA (e.g., Gray et al. 2022), concerns about ethical and privacy issues (e.g., Hoel & Chen 2019), and heterogeneity in the effects of LA interventions on learners across countries (e.g., Davis et al. 2017; Kizilcec & Cohen, 2017). All these differences can make the transfer of LA systems from one country to another challenging since there are varying contextual, technical, and also cultural factors that play an important role. Whereas technical and contextual aspects of LA systems’ design and implementation have been addressed by LA scholars and practitioners, important cultural factors have hitherto received limited attention (Viberg et al. in press). There have been efforts in the community to examine contextual and cultural factors, including at conferences and new journals on the topic (e.g., Computer-based Learning in Context). Considering the role that culture – both at the individual and national level – may have on the adoption of LA systems across countries and earlier research in related domains (see e.g., Leidner & Kayworth, 2006) on the role of culture in information systems design, or Baker et al. (2019) for computer-based learning systems), there is a critical need for more research and discussion of culture in LA.

Previous research in this area has hypothesized how culture might play a role in the implementation process of LA systems (e.g., Vatrapu, 2011). Although not many studies have looked at how students’ learning patterns and learning strategies in higher education differ across cultures (Marambe et al., 2012), there is evidence that learners use the learning environment differently across countries and cultures (Liu et al. 2016; Rizvi et al. 2022). Cultural differences play a role in educational technology acceptance and use (e.g., Nistor et al. 2013) and students’ collaborative learning practices (e.g., Vatrapu & Suthers, 2007). Cultural differences also influence the effectiveness of LA interventions that encourage self-regulated learning (Davis et al. 2017; Kizilcec & Cohen, 2017). This early body of work underlines the importance of not only considering the stakeholders’ cultural values in the evaluation of LA systems, but also the importance of designing culture- and value-sensitive LA systems to increase their acceptance and use at a scale.

This workshop has two goals: 

1. discuss and identify possible cultural differences and similarities for the wider adoption of LA systems at scale

2. introduce and practice several culture- and value-sensitive design methods on selected LA tools.

The TEL community would benefit from enabling a discussion and drafting a set of suggestions on how to create more inclusive tools that put stakeholders and their cultural values at the center of the design, and obtaining practical skills of working with the selected value-sensitive design methods. 


We believe that the proposed workshop is of particular interest to the TEL community for several reasons, including a need to: 


References

Baker, R., Ogan, A., Madaio, M., & Walker, E.(2019). Culture in computer-based learning systems: challenges and opportunities. https://edarxiv.org/ad39g/ 

Davis, D., Jivet, I., Kizilcec, R., Chen, G., Hauff, C., & Houben, G.(2017). Follow the successful crowd: raising MOOC completion rates through social comparison at scale. In LAK17 (pp. 454-463).

Gray, G., Schalk, A.E., Cooke, G., Murnion, P., Rooney, P., & O’Rourke, K. (2022). Stakeholders’ insights on learning analytics: Perspectives of students and staff. Computers & Education, 187, 104550.

Hoel, T., & Chen, W. (2019). Privacy engineering for learning analytics in a global market – defining a point of reference. International Journal of Information and Learning Technology, 36(4), 288-298

Hofstede, G. (2001). Cultural consequences: comparing values, behaviors, institutions, and organisations across nations. London: Sage Publications Ltd.

Kizilcec, R., & Cohen, G. (2017). Eight-minute self-regulation intervention raises educational attainment at scale in individualist but not collectivist cultures. Proceedings of the National Academy of Sciences, 114(17), 4348-4353.

Leidner, E. & Kayworth, T. (2006). A review of culture in information systems research: Toward a theory of information technology cultural conflict. MIS Quarterly 30(2), 357-399. 

Liu, Z., Brown, R., Lynch, C. F., Barnes, T., Baker, R., Bergner, Y., & McNamara, D. (2016). MOOC Learner Behaviors by Country and Culture; an Exploratory Analysis. International Educational Data Mining Society

Marambe, K., Vermunt,  J.,& Boshuizen, H.(2012). A cross-cultural comparison of  student  learning patterns in higher education. Higher Education, 64(3), 299-316. 

Mittelmeier, J., Tempelaar, D., Rienties, B., & Nguyen, Q.(2016). Learning Analytics to Understand Cultural Impacts on Technology Enhanced Learning. International Association for Development of the Information Society. 

Nistor, N., Göğüş, A., & Lerche, T. (2013). Educational technology acceptance across national and professional cultures: a European study. Educational Technology Research and Development, 61(4), 733-749.

Rizvi, S., Rienties, B., Rogaten, J., & Kizilcec, R. F. (2022). Beyond one-size-fits-all in MOOCs: Variation in learning design and persistence of learners in different cultural and socioeconomic contexts. Computers in Human Behavior, 126, 106973.

Vatrapu, R.(2011). Cultural considerations in learning analytics. In LAK11 (pp.127-133).

Vatrapu, R., & Suthers, D. (2007). Culture and computers: A review of the concept of culture and implications for intercultural collaborative online learning. In International Workshop on Intercultural Collaboration (pp. 260-275). 

Viberg, O., Jivet, I., & Scheffel, M. (in press). Designing Culturally Aware Learning Analytics: A Value Sensitive Perspective. Accepted for publication in Viberg, O., & Grönlund, Å. (Eds.). Practicable LA. Springer Nature.

Workshop goals and structure


This workshop, building on the earlier three workshops conducted during the 12th International Conference on Learning Analytics and Knowledge (LAK22) and the European Conference on Technology Enhanced Learning (ECTEL21 and ECTEL22), aims to:


To achieve these aims, we will use design approaches explored in the CHI community to facilitate a discussion on the possible cultural specificities to be considered for the design, adoption, and use of LA. As a proxy for ‘culture’ we will use Hofstede’s model (Hofstede, 2001), defining what he claims to be national cultures, as a starting point for the analysis of chosen existing LA tools and the design of a selected LA tool. While these categories are contested as measures of national cultures, they contain elements that also in other contexts have been suggested to play a role in people’s behavior and attitudes toward education (Mittelmeier et al., 2016) and technologies used in educational settings to improve students’ learning (Baker et al., 2019). This means that even though they may not indicate the culture of entire nations, they might affect the adoption and implementation of LA worldwide. In other words, we do not rely on Hofstede’s national cultural profiles to offer design recommendations for specific countries but explore cultural dimensions that may affect: i) students’ expectations and attitudes towards LA services and related privacy concerns, and ii) the acceptance, design, and use of designed LA tools. 

If there is interest, we would also be happy to support the formation of a SIG interested in the topic


Workshop schedule

We are planning an interactive full-day workshop with a maximum of 30 participants. The participants should expect participants should expect: symposia elements, discussion groups, group-based demos, presentations. Before the workshop, we might ask participants to fill out a questionnaire on the topic of the workshop. The results will be used as input for discussion during the workshop. 

Workshop outline:


A detailed schedule will be shared closer to the event.

Previous editions

Organisers

Olga Viberg

KTH Stockholm, Sweden

Ioana Jivet

Goethe University Frankfurt and DIPF, Germany

Rene Kizilcec

Cornell University, USA

Maren Scheffel

Ruhr-University Bochum, Germany

If you wish to get in contact with the organisers, please write to oviberg@kth.se.

Acknowledgements

Part of this work was funded by the Hessian Ministry for Science and the Arts (HMWK) as part of the project HessenHub – Netzwerk digitale Hochschullehre Hessen, Trusted Learning Analytics (https://www.hessenhub.de/).