You can find the CEUR Proceedings here.
9am Welcome (by Gianluca Demartini)
9.15am Keynote - Jahna Otterbacher - Revealing Social Bias in Human-Machine Information Systems (Introduced by Cristina Sarasua)
10.15am Topics for discussion in the afternoon session (Moderator: Ujwal Gadiraju)
10.30am Coffee break
11am Paper Presentations (Session Chair: Alessandro Checco)
- Agathe Balayn, Panagiotis Mavridis, Alessandro Bozzon, Benjamin Timmermans and Zoltán Szlávik. Characterising and Mitigating Aggregation-Bias in Crowdsourced Toxicity Annotations (paper) (slides)
- Pavlos Vougiouklis, Eddy Maddalena, Jonathon Hare and Elena Simperl. How Biased Is Your NLG Evaluation? (paper) (slides)
- Christoph Hube, Besnik Fetahu and Ujwal Gadiraju. LimitBias! Measuring Worker Biases in the Crowdsourced Collection of Subjective Judgments (paper) (slides)
12pm Lunch break
1pm Paper presentations (Session Chair: Cristina Sarasua)
- Rehab Qarout, Alessandro Checco and Kalina Bontcheva. Investigating Stability and Reliability of Crowdsourcing Output (paper) (slides)
- Panagiotis Mavridis, Markus De Jong, Lora Aroyo, Alessandro Bozzon, Jesse De Vos, Johan Oomen, Alec Badenoch and Antoaneta Dimitrova. A Human in the Loop Approach to Capture Bias and Support Media Scientists in Video News Analysis (paper)
- Michael Barz, Neslihan Büyükdemircioglu, Rikhu Prasad Surya, Tim Polzehl and Daniel Sonntag. Device-Type Influence in Crowd-based Natural Language Translation Tasks (paper)
- Yuko Sakurai and Satoshi Oyama. Shapley Value Regression for Bias Reduction in Answer Aggregation (paper)
2.15pm Moderated Discussion (Moderated by Ujwal Gadiraju)
3pm Coffee break
3.30pm Keynote - Kristy Milland - Bias from the Worker Perspective (Introduced by Cristina Sarasua)
4.30pm Closing remarks / next steps (Gianluca Demartini)
5pm End