NORMALize is a half-day workshop taking place in the afternoon (13:30-17:30) on September 22, 2025, in Room D4, scheduled according to local time in Prague, Czech Republic (GMT+2). This year, we accepted two types of contributions to be presented at the workshop: early-stage research with 5 minutes of presentation time and previously published manuscripts with 7 minutes. Each will be followed by 2 minutes of questions.
13:30 - 13:45 Welcome & Introduction
13:45 - 14:45 Session 1
Civic Ground Truth in News Recommenders: A Method for Public Value Scoring
James Meese and Kyle Herbetson
Early-stage research
Are we the Monsters? Handling Values in Production News Recommender Systems
Robin Verachtert
Early-stage research
Nudges for News Recommenders: Prominent Article Positioning Increases Selection, Engagement and Recall of Environmental News, but Reducing Complexity Does Not
Nicolas Mattis, Lucien Heitz, Philipp K. Masur, Judith Moeller, and Wouter van Atteveldt
Previously published manuscript
“I must have clicked on something” - Users´ Experiences and Evaluations of News Recommender Systems
Árni Már Einarsson, Elisabetta Petrucci, Jannie Møller Hartley, Stine Lomborg, and Johannes Kruse
Previously published manuscript
14:45 - 15:30 Breakout session
15:30 - 16:00 Break
16:00 - 16:40 Session 2
A Right to Constructive Optimization: A Public Interest Approach to Recommender Systems in the Digital Services Act
Laurens Naudts, Natali Helberger, Michael Veale, and Marijn Sax
Previously published manuscript
Intended, afforded, and experienced serendipity: overcoming the paradox of artificial serendipity
Annelien Smets
Previously published manuscript
Speculative Legal Design and AI Futures Research: A Discussion
Freyja van den Boom
Early-stage research
16:40 - 17:20 Session 3
The Trouble with diverse books (Part I & Part II)
E.E. Lawrence
Previously published manuscript
Selling Books with Algorithms
Anna Muenchrath
Previously published manuscript
Towards Public Service Recommender Systems
Jerome Ramos, Oliver Elliot, Fernando Diaz, and Georgina Born
Early-stage research
17:20 - 17:30 Closing
Civic Ground Truth in News Recommenders: A Method for Public Value Scoring
James Meese and Kyle Herbetson
Research in news recommendation systems (NRS) continues to explore the best ways to integrate normative goals such as editorial objectives and public service values into existing systems. Prior efforts have incorporated expert input or audience feedback to quantify these values, laying the groundwork for more civic-minded recommender systems. This paper contributes to that trajectory, introducing a method for embedding civic values into NRS through large-scale, structured audience evaluations. The proposed civic ground truth approach aims to generate value-based labels through a nationally representative survey that are generalisable across a wider news corpus, using automated metadata enrichment.
------------------------------------------------
Are we the Monsters? Handling Values in Production News Recommender Systems
Robin Verachtert
News Recommendation Systems are a technology that most news organisations are embracing the last few years. Due to its impact on society, it is also a domain where norms and values are heavily discussed, in research, in policy making and also in news organisations themselves. In this work I present the impact norms and values have on the teams implementing recommendation systems in production at news corporations. I will discuss the motivations and constraints encountered when implementing and productionising norms and values.
------------------------------------------------
Speculative Legal Design and AI Futures Research: A Discussion
Freyja van den Boom
------------------------------------------------
Towards Public Service Recommender Systems
Jerome Ramos, Oliver Elliot, Fernando Diaz, and Georgina Born
In contrast with many commercial settings where recommendation focuses on personalization, public service media (PSM) organizations need to address the collective and cumulative social impacts of algorithmic content distribution. This involves measuring how effectively systems distribute editorially-curated content across user populations. This paper presents results from an innovative collaboration between academic researchers and the BBC to evaluate recommender systems under a public service media paradigm. Building on our previous work, we apply the commonality metric — a novel approach that operationalizes PSM principles of universality and diversity — to real-world podcast recommendations on the BBC’s streaming audio service. Through experimental testing with historical audience data, we evaluate the relationships between commonality and traditional engagement metrics. This interdisciplinary collaboration demonstrates how public values can be meaningfully integrated into algorithmic system design, offering valuable insights for both PSM organizations and the broader field of responsible algorithmic development.