AI Colloquium

The Bernoulli Institute AI Colloquium is a regular meeting where faculty and students  can hear and discuss the current research related to the institute's three research themes (Computing and Cognition; Geometry and its Applications; Systems, Data and Society) from inside and outside the University of Groningen. 

Colloquia are scheduled monthly during the teaching blocks, with the exception of December, preferentially on Thursday afternoon. Meetings at the end of the day are followed by the PoCoBo - a post-colloquium borrel.

For external colloquia, there is a limited opportunity to have dinner with the speaker in the evening. If you are interested in this, please contact Harmen or Stephen a few days before the colloquium.

MultiBLiMP: A Multilingual Benchmark of Linguistic Minimal Pairs

Dr Jaap Jumelet, University of Groningen

Wednesday 18 June 2025 16:00-17:30

Bernoulliborg room BB134

We introduce MultiBLiMP, a massively multilingual benchmark of linguistic minimal pairs, covering 101 languages, 6 linguistic phenomena and containing more than 120.000 minimal pairs. Our minimal pairs are created using a fully automated pipeline, leveraging the large-scale linguistic resources of Universal Dependencies and UniMorph. MultiBLiMP evaluates linguistic abilities of LLMs at an unprecedented multilingual scale, and highlights the shortcomings of the current state-of-the-art in modelling low-resource languages.

Uncertainty in Machine Learning Symposium

Monday 16 June 2025 13:00-17:30

Bernoulliborg room BB105

On June 16th the Bernoulliborg will be hosting the Uncertainty in Machine Learning symposium. Various researchers working on Uncertainty in ML will come together to talk about the work that they’re doing. This is a great opportunity for people with knowledge of Machine Learning to learn about current research in Uncertainty Estimation. The event is well suited for (2nd-3rd year) BSc, MSc, PhD students and staff members if they already have some knowledge of Machine Learning.  

Our special guest will be dr. Michael Kirchhof. He’s coming from Paris to tell us about the work that he’s doing at Apple on uncertainty in Diffusion models (image generation) and LLMs. Additionally, we’ll have speakers from our AI department, the UMCG, Radboud University and the UG Computational Linguistics department. 

Very briefly on what Uncertainty in ML is: Typically with Machine Learning we’re focussed on making the best predictions. However, ML models are almost never 100% correct. With uncertainty estimation we try to know where we are likely to be incorrect (and quantify how incorrect). Uncertainty in ML tries to distinguish between when an ML system makes an educated guess, and when it makes a random guess. 

If you want to attend (parts of) the symposium, please sign up here: https://forms.gle/nAzgAgfeuSPFyf32A 

Interested people outside of the Bernoulli Institute are also welcome to join! 

Schedule:

• 13:00-13:40 -- Ivo Pascal de Jong – An Introduction to Uncertainty & Disentanglement Error – To get everyone up to speed I will explain the basics of Uncertainty in ML. Then I will talk about how to evaluate different kinds of uncertainty with Disentanglement Error.

• 13:40-13:45 -- Questions

• 13:45-14:00 -- Break

• 14:00-14:15 -- dr. Matias Valdenegro-Toro – Uncertainty in Large Vision-Language Models and Computer Vision Applications – In this talk I will show research applications of neural networks with uncertainty quantification, covering Computer Vision, Large Language Models and Vision-Language Models. This includes super-resolution, frame generation, verbalized uncertainty, and robustness to corrupted inputs.

• 14:15-14:25 -- Questions & setup next speaker

• 14:25-14:40 -- Leonidas Zotos – Leveraging Uncertainty for the task of Multiple-Choice Question Difficulty Estimation – Predicting how many students will answer an exam question correctly is challenging but essential for designing high-quality assessments. In this talk, I will present a novel approach that leverages the uncertainty of test-taking large language models (LLMs) to simulate a field-testing environment.

• 14:40-14:45 -- Questions

• 14:45-15:00 -- Break

• 15:00-15:40 -- dr. Michael Kirchhof -- Keynote speaker – (De/Re)constructing uncertainties for generative vision and language – When we think about uncertainty, we historically think in terms of classification, for example aleatoric and epistemic uncertainties of images. But does this apply to generative tasks as well, in Vision and Language? In this talk, I will present Apple's recent efforts to go beyond, towards new forms of generative uncertainties both in Diffusion models and LLMs

• 15:40-15:45 -- Questions

• 15:45-16:00 -- Break

• 16:00-16:15 -- Merlijn Krale – Uncertainty in partially observable Markov decision processes (POMDP)

• 16:15-16:20 -- Questions

• 16:20-16:35 -- Steff Groefsema – Responsible Medical AI and How To Reach It 

• 16:35-16:40 -- Questions

• 16:40-16:55 -- Joëlle van Aalst – Towards uncertainty in practice: a tool for quality assurance in radiotherapy AI – Deep learning segmentation models in radiotherapy require rigorous quality assurance because segmentations directly impact the treatment plan. This talk will highlight two practical applications of uncertainty quantification: detecting when models encounter unfamiliar data, and exploring clinicians’ preferences for presenting uncertainty in ways that support trust and improve interpretation when editing deep learning segmentations.

• 16:55-17:00 -- Questions

• 17:00-17:30 -- Drinks & conversations

Upcoming speakers

• Peter Hendrix, Tilburg University

• Tsegaye Misikir Tashu, University of Groningen

• Catherine Sibert, University of Groningen

• Freek Stulp, Institute of Robotics and Mechatronics