Workshop on Culture and Values in Learning Analytics:
A Human-Centered Design and Research Approach
Workshop at the LAK 2024 conference, March 18, Kyoto (Japan)
Workshop theme
The recent eruption of artificial intelligence (AI) into the public discourse -in line with this year’s conference theme “Learning Analytics in the Age of Artificial Intelligence”- has prompted many questions about its impact on education (including issues of assessment, accountability, and literacies). These new questions, along with long-standing issues like fairness and bias in machine learning and AI as well as other ethics and privacy issues are pushing the learning analytics (LA) community to call for a more humane alignment of the LA systems we design (e.g., the notion of human-centered LA, see Buckingham Shum et al., 2019). However, in LA research and practice we have not systematically and explicitly taken such goals and values into account, with only a few examples available (Campos et al., 2023; Chen & Zhu, 2019). Luckily, we can turn to other fields that have been working on frameworks and methods to both model and investigate the goals, needs and values of groups and individuals, at different levels.
From the field of cross-cultural psychology, it is now well-established that culture is a primary way in which certain values are reflected. One of the ways to examine and understand culture is through its values. Cultural values are understood as “collective tendencies to prefer a certain course of events above another, expressed by qualifications such as good and bad, dirty and clear, ugly and beautiful” (Hofstede et al., 2010). According to Viberg et al. (2023), the values emphasized in a society may be “the most central feature of culture” (Schwartz, 2006, p. 139) as these values describe a shared understanding of what society views as good, right and desirable (Williams, 1970). For example, if a society values success and ambition, this might be reflected in “a highly competitive economic system [...] and child-rearing practices that pressure children to achieve” (Schwartz, 2006, p. 139). In an educational setting, such an environment might foster competition among students as ‘being better than your peers’ defines a successful learner, encouraging the use of social comparison features in the design of LA dashboards (Jivet et al., 2017). It is also well established in psychology that values, i.e., what a person considers important in life, are also a key motivational construct at the individual level, related to well-being, planned behavior (including learning behavior) and even neural correlates (Sagiv & Schwartz, 2022).
In the field of human-computer interaction, building on this rich social sciences research background, value-sensitive design (VSD) has been proposed as “a theoretically grounded approach to the design of technology that accounts for human values in a principled and comprehensive manner” (Friedman et al., 2017). Yet, VSD is more than a philosophy, and it has developed specific methods to consider human values (e.g., privacy, trust, and autonomy) in a systematic fashion throughout system design and research processes. Thus, VSD holds great potential as a concrete way to consider cultural aspects and individual values in LA (Chen & Zhu, 2019; Viberg et al., 2023). The main expected benefits of using VSD in LA include: making LA more relevant to a wider range of stakeholders and facilitating (and understanding) the transfer of LA innovations to new contexts and across cultures.
This workshop aims to both popularize VSD in learning analytics and help participants incorporate such methods (and cultural and individual value considerations more generally) into LA design and research processes. The workshop builds upon two previously separate workshop series at the LAK and EC-TEL conferences (workshop names blinded for review), thus combining the efforts, contents, and experiences for both of those aspects to provide workshop participants with the best of both.
References
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9.
Campos, F., Nguyen, H., Ahn, J., & Jackson, K. (2023). Leveraging cultural forms in human‐centred learning analytics design. British Journal of Educational Technology, bjet.13384. https://doi.org/10.1111/bjet.13384
Chen, B., & Zhu, H. (2019). Towards Value-Sensitive Learning Analytics Design. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 343–352. https://doi.org/10.1145/3303772.3303798
Friedman, B., Hendry, D. G., Borning, A., & others. (2017). A survey of value sensitive design methods. Foundations and Trends® in Human–Computer Interaction, 11(2), 63–125.
Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations: Software of the mind: Intercultural cooperation and its importance for survival. McGraw-Hill.
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: Pitfalls of learning analytics dashboards in the educational practice. European Conference on Technology Enhanced Learning, 82–96.
Sagiv, L., & Schwartz, S. H. (2022). Personal Values Across Cultures. Annual Review of Psychology, 73(1), 517–546. https://doi.org/10.1146/annurev-psych-020821-125100
Schwartz, S. (2006). A Theory of Cultural Value Orientations: Explication and Applications. Comparative Sociology, 5(2–3), 137–182. https://doi.org/10.1163/156913306778667357
Viberg, O., Jivet, I., & Scheffel, M. (2023). Designing Culturally Aware Learning Analytics: A Value Sensitive Perspective. In O. Viberg & Å. Grönlund (Eds.), Practicable Learning Analytics (pp. 177–192). Springer International Publishing. https://doi.org/10.1007/978-3-031-27646-0_10
Williams, R. M. (1970). American society: A sociological interpretation. Knopf.
Workshop goals
Against this backdrop, the present workshop aims to:
introduce the participants to cultural considerations and VSD methods;
further explore and raise awareness of possible influences of stakeholders’ values and preferences on the acceptance of LA systems, the design of LA tools, and the evaluations of LA interventions;
practice selected VSD methods that can be used to inform more responsible and human-centered design of LA and AI in education; and
invite participants to jointly plan how to incorporate VSD methods into actual LA design or research processes, e.g., to perform transfer or multi-site/multi-cultural LA studies.
Workshop structure and schedule
The workshop will take the form of a “design challenge” where participants join in small teams to de-construct and re-design specific LA systems, or to plan cross-cultural LA studies, with non-expositional scaffolding from the organizers.
Before the workshop
Please register for the workshop during your LAK24 conference registration, or add it to your existing registration (this is important! the workshop may be cancelled if there are not enough registrations)
If applicable, please propose a "case study" (using this form) for the design challenge part of the workshop, i.e., an LA tool or research study you would like to (re-)design taking culture and values into account. If you just want to learn about culture and values in LA but don't have a clear case of application, just skip this step.
Express your interest in the "case studies" proposed by you or other participants (you will receive an email about this before the workshop event)
During the workshop
Introduction
What do we mean by “culture” and “values”?
Frameworks for cultural and individual value investigation
Value sensitive design (VSD) and its applicability in LA design and research
Guided exploration of value-sensitive design methods (group work)
Becoming knowledgeable about a VSD methods
Application of the VSD methods to a facilitator-proposed case study
☕️ Coffee Break ☕️
Design Sprint: (re-)designing an LA technology/study (group work)
Informal presentation of participant-proposed case studies
Deconstructing, redesigning and evaluating the case studies (structured collaborative activities)
Wrap-Up
Future work of the work groups on their projects
Potential outlets: Special issue on VSD at BJET
After the workshop
Continue working with your group on your design challenge tool/study.
If applicable, submit a paper to the upcoming Special Section on VSD at the British Journal of Educational Technology.
Previous editions
EC-TEL 2023, September 4-8, Aveiro, Portugal
LAK 2023, March 13-17, Arlington, Texas, USA
cancelled
EC-TEL 2022, September 12-16, Toulouse, France
LAK 2022, March 21-25, online
Invited keynote: Rene Kizilcec, Cornell University
EC-TEL 2021, September 20-24, online
Invited keynote: Rene Kizilcec, Cornell University
Organisers
Luis P. Prieto
University of Valladolid, Spain
Olga Viberg
KTH Stockholm, Sweden
Ioana Jivet
Goethe University Frankfurt and DIPF, Germany
FernUniversität in Hagen, Germany
Maren Scheffel
Ruhr-University Bochum, Germany
María Jesús Rodríguez-Triana
Tallinn University, Estonia
Bodong Chen
University of Pennsylvania, US
If you wish to get in contact with the organisers, please write to luispablo.prieto@uva.es.
Acknowledgements
The workshop is supported by grant RYC2021-032273-I, financed by MCIN/AEI/ 10.13039/501100011033 and the European Union's NextGenerationEU/PRTR. It has also received support from the Regional Government of Castile and Leon, under project grant VA176P23.