FRANCESCO QUINZAN, Ph.D.

Hi! I am a Senior Research Associate affiliated with the Machine Learning Research Group at the University of Oxford, hosted by Stephen Roberts. In addition to my primary role, I serve as an Associate Member at the Trustworthy AI Lab within the Institute of Technology and Humanity at the University of Cambridge. Previously, I was a Research Associate and a member of the ELSA Network of Excellence at the University of Oxford, hosted by Marta Kwiatkowska. I held a Postdoc position at the Division of Decision and Control Systems at KTH, where I worked with Stefan Bauer and Cristian Rojas. I obtained my Ph.D. in Computer Science from the Hasso Plattner Institute in Germany, where I was supervised by Tobias Friedrich. I was fortunate to visit various institutes and research groups, including the Max Plank Institute for Intelligent Systems, where I was hosted by Bernhard Schölkopf, and the Learning & Adpative Systems Group of Andreas Krause at ETH. I studied mathematics at the University of Roma Tre, where I graduated with honors.

dblp  |  google scholar  |  linkedin

francesco.quinzan[ at ]eng.ox.ac.uk

I am interested in AI safety

Recent reports and expert commentary emphasize that AI safety is no longer a speculative concern but a critical, time‑sensitive issue. The First International AI Safety Report published in January 2025 and endorsed by 96 leading experts warns that current national and corporate readiness is insufficient to address risks arising from advanced AI systems, making safety a top global priority. In response to these growing concerns, my work focuses on developing foundational theory and computational tools to strengthen AI safety. I aim to rigorously characterize and mitigate risks posed by advanced AI systems, with the goal of ensuring their behavior remains robust, predictable, and aligned with human values—even under complex, real-world conditions.

Doubly Robust Alignment for Large Language Models

Erhan Xu*, Kai Ye*, Hongyi Zhou*, Luhan Zhu, Francesco Quinzan†, Chengchun Shi†

NeurIPS (2025)

*equal contribution, †joint senior contributors

arXiv e-Print: CoRR abs/2506.01183  (2025) 

A preliminary version of this work was presented at the Second ICML Workshop on Reliable and Responsible Foundation Models (2025) and the Second ICML Workshop on Human Feedback for AI Alignment (2025).

This paper studies reinforcement learning from human feedback (RLHF) for aligning large language models with human preferences. While RLHF has demonstrated promising results, many algorithms are highly sensitive to misspecifications in the underlying preference model (e.g., the Bradley-Terry model), the reference policy, or the reward function, resulting in undesirable fine-tuning. To address model misspecification, we propose a doubly robust preference optimization algorithm that remains consistent when either the preference model or the reference policy is correctly specified (without requiring both). Our proposal demonstrates superior and more robust performance than state-of-the-art algorithms, both in theory and in practice. The code is available at https://github.com/DRPO4LLM/DRPO4LLM

Double Machine Learning Based Structure Identification from Temporal Data

Emmanouil Angelis*, Francesco Quinzan*, Ashkan Soleymani, Patrick Jaillet, Stefan Bauer

Trans. Mach. Learn. Res. (2025)

*equal contribution

arXiv e-Print: CoRR abs/2311.06012  (2023) 

Learning the causes of time-series data is a fundamental task in many applications, spanning from finance to earth sciences or bio-medical applications. Common approaches for this task are based on vector auto-regression, and they do not take into account unknown confounding between potential causes. However, in settings with many potential causes and noisy data, these approaches may be substantially biased. Furthermore, potential causes may be correlated in practical applications. Moreover, existing algorithms often do not work with cyclic data. To address these challenges, we propose a new doubly robust method for Structure Identification from Temporal Data ( SITD ). We provide theoretical guarantees, showing that our method asymptotically recovers the true underlying causal structure. Our analysis extends to cases where the potential causes have cycles and they may be confounded. We further perform extensive experiments to showcase the superior performance of our method.

Pruning Cannot Hurt Robustness: Certified Trade-Offs in Reinforcement Learning

James Pedley, Benjamin Etheridge, Stephen J. Roberts, Francesco Quinzan

arXiv e-Print: CoRR abs/2510.12939 (2025) 

Reinforcement learning (RL) policies deployed in real-world environments must remain reliable under adversarial perturbations. At the same time, modern deep RL agents are heavily over-parameterized, raising costs and fragility concerns. While pruning has been shown to improve robustness in supervised learning, its role in adversarial RL remains poorly understood. We develop the first theoretical framework for certified robustness under pruning in state-adversarial Markov decision processes (SA-MDPs). For Gaussian and categorical policies with Lipschitz networks, we prove that element-wise pruning can only tighten certified robustness bounds; pruning never makes the policy less robust. Building on this, we derive a novel three-term regret decomposition that disentangles clean-task performance, pruning-induced performance loss, and robustness gains, exposing a fundamental performance--robustness frontier. Empirically, we evaluate magnitude and micro-pruning schedules on continuous-control benchmarks with strong policy-aware adversaries. Across tasks, pruning consistently uncovers reproducible ``sweet spots'' at moderate sparsity levels, where robustness improves substantially without harming - and sometimes even enhancing - clean performance. These results position pruning not merely as a compression tool but as a structural intervention for robust RL. 

Double Machine Learning for Conditional Moment Restrictions: IV regression, Proximal Causal Learning and Beyond

Daqian Shao, Ashkan Solymani, Francesco Quinzan, Marta Kwiatkowska

arXiv e-Print: CoRR abs/2506.14950  (2025) 

Solving conditional moment restrictions (CMRs) is a key problem considered in statistics, causal inference, and econometrics, where the aim is to solve for a function of interest that satisfies some conditional moment equalities. Specifically, many techniques for causal inference, such as instrumental variable (IV) regression and proximal causal learning (PCL), are CMR problems. Most CMR estimators use a two-stage approach, where the first-stage estimation is directly plugged into the second stage to estimate the function of interest. However, naively plugging in the first-stage estimator can cause heavy bias in the second stage. This is particularly the case for recently proposed CMR estimators that use deep neural network (DNN) estimators for both stages, where regularisation and overfitting bias is present. We propose DML-CMR, a two-stage CMR estimator that provides an unbiased estimate with fast convergence rate guarantees. We derive a novel learning objective to reduce bias and develop the DML-CMR algorithm following the double/debiased machine learning (DML) framework. We show that our DML-CMR estimator can achieve the minimax optimal convergence rate of O(√(1/N)) under parameterisation and mild regularity conditions, where is the sample size. We apply DML-CMR to a range of problems using DNN estimators, including IV regression and proximal causal learning on real-world datasets, demonstrating state-of-the-art performance against existing CMR estimators and algorithms tailored to those problems.

AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysis

Haroui Ma*, Francesco Quinzan*, Theresa Willem, Stefan Bauer

*equal contribution

arXiv e-Print: CoRR abs/2504.03784  (2025) 

Machine learning (ML) systems for medical imaging have demonstrated remarkable diagnostic capabilities, but their susceptibility to biases poses significant risks, since biases may negatively impact generalization performance. In this paper, we introduce a novel statistical framework to evaluate the dependency of medical imaging ML models on sensitive attributes, such as demographics. Our method leverages the concept of counterfactual invariance, measuring the extent to which a model's predictions remain unchanged under hypothetical changes to sensitive attributes. We present a practical algorithm that combines conditional latent diffusion models with statistical hypothesis testing to identify and quantify such biases without requiring direct access to counterfactual data. Through experiments on synthetic datasets and large-scale real-world medical imaging datasets, including \textsc{cheXpert} and MIMIC-CXR, we demonstrate that our approach aligns closely with counterfactual fairness principles and outperforms standard baselines. This work provides a robust tool to ensure that ML diagnostic systems generalize well, e.g., across demographic groups, offering a critical step towards AI safety in healthcare. Code: https://github.com/Neferpitou3871/AI-Alignment-Medical-Imaging. 

Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning

Kai Ye*, Hongyi Zhou*, Jin Zhu*, Francesco Quinzan†, Chengchun Shi†

*equal contribution, †joint senior contributors

arXiv e-Print: CoRR abs/2504.03784  (2025) 

A preliminary version of this work was presented at the Second ICML Workshop on Human Feedback for AI Alignment (2025).

Reinforcement learning from human feedback (RLHF) has emerged as a key technique for aligning the output of large language models (LLMs) with human preferences. To learn the reward function, most existing RLHF algorithms use the Bradley-Terry model, which relies on assumptions about human preferences that may not reflect the complexity and variability of real-world judgments. In this paper, we propose a robust algorithm to enhance the performance of existing approaches under such reward model misspecifications. Theoretically, our algorithm reduces the variance of reward and policy estimators, leading to improved regret bounds. Empirical evaluations on LLM benchmark datasets demonstrate that the proposed algorithm consistently outperforms existing methods, with 77-81% of responses being favored over baselines on the Anthropic Helpful and Harmless dataset.

When Should We Orchestrate Multiple Agents?

Umang Bhatt*, Sanyam Kapoor*, Mihir Upadhyay, Ilia Sucholutsky, Francesco Quinzan, Katherine M Collins, Adrian Weller, Andrew Gordon Wilson, Muhammad Bilal Zafar

*equal contribution

arXiv e-Print: CoRR abs/2503.13577  (2025) 

Strategies for orchestrating the interactions between multiple agents, both human and artificial, can wildly overestimate performance and underestimate the cost of orchestration. We design a framework to orchestrate agents under realistic conditions, such as inference costs or availability constraints. We show theoretically that orchestration is only effective if there are performance or cost differentials between agents. We then empirically demonstrate how orchestration between multiple agents can be helpful for selecting agents in a simulated environment, picking a learning strategy in the infamous Rogers' Paradox from social science, and outsourcing tasks to other agents during a question-answer task in a user study. 

Learning Decision Policies with Instrumental Variables through Double Machine Learning

Daqian Shao, Ashkan Soleymani, Francesco Quinzan, Marta Kwiatkwska

ICML (2024) 

arXiv e-Print: CoRR abs/2405.08498  (2024) 

A common issue in learning decision-making policies in data-rich settings is spurious correlations in the offline dataset, which can be caused by hidden confounders. Instrumental variable (IV) regression, which utilises a key unconfounded variable known as the instrument, is a standard technique for learning causal relationships between confounded action, outcome, and context variables. Most recent IV regression algorithms use a two-stage approach, where a deep neural network (DNN) estimator learnt in the first stage is directly plugged into the second stage, in which another DNN is used to estimate the causal effect. Naively plugging the estimator can cause heavy bias in the second stage, especially when regularisation bias is present in the first stage estimator. We propose DML-IV, a non-linear IV regression method that reduces the bias in two-stage IV regressions and effectively learns high-performing policies. We derive a novel learning objective to reduce bias and design the DML-IV algorithm following the double/debiased machine learning (DML) framework. The learnt DML-IV estimator has strong convergence rate and O(N^{-1/2}) suboptimality guarantees that match those when the dataset is unconfounded. DML-IV outperforms state-of-the-art IV regression methods on IV regression benchmarks and learns high-performing policies in the presence of instruments.

Learning Counterfactually Invariant Predictors

Francesco Quinzan*, Cecilia Casolo*, Krikamol Muandet, Yucen Luo†, Niki Kilbertus† 

Trans. Mach. Learn. Res. (2024)

*equal contribution; †joint senior contributors

arXiv e-Print: CoRR abs/2207.09768 (2022) 

A preliminary version of this work was presented at the First NeurIPS Workshop on Algorithmic Fairness through the Lens of Causality and Privacy (2022) and at the First ICML Workshop on Formal Verification of Machine Learning (2023).

Counterfactual invariance has proven an essential property for predictors that are fair, robust, and generalizable in the real world. We propose a general definition of counterfactual invariance and provide simple graphical criteria that yield a sufficient condition for a predictor to be counterfactually invariant in terms of (conditional independence in) the observational distribution. Any predictor that satisfies our criterion is provably counterfactually invariant. In order to learn such predictors, we propose a model-agnostic framework, called Counterfactual Invariance Prediction (CIP), based on a kernel-based conditional dependence measure called Hilbert-Schmidt Conditional Independence Criterion (HSCIC). Our experimental results demonstrate the effectiveness of CIP in enforcing counterfactual invariance across various types of data including tabular, high-dimensional, and real-world dataset.

Diffusion Based Causal Representation Learning

Amir Mohammad Karimi Mamagham, Andrea Dittadi, Stefan Bauer, Karl Henrik Johansson, Francesco Quinzan

Entropy : 26(7) 556 (2024)

arXiv e-Print: CoRR abs/2311.05421 (2022) 

Causal reasoning can be considered a cornerstone of intelligent systems. Having access to an underlying causal graph comes with the promise of cause-effect estimation and the identification of efficient and safe interventions. However, learning causal representations remains a major challenge, due to the complexity of many real-world systems. Previous works on causal representation learning have mostly focused on Variational Auto-Encoders (VAE). These methods only provide representations from a point estimate, and they are less effective at handling high dimensions. To overcome these problems, we propose Diffusion-based Causal Representation Learning (DCRL) framework which uses diffusion-based representations for causal discovery in the latent space. DCRL provides access to both single-dimensional and infinite-dimensional latent codes, which encode different levels of information. In a first proof of principle, we investigate the use of DCRL for causal representation learning in a weakly-supervised setting. We further demonstrate experimentally that this approach performs comparably well in identifying the latent causal structure and causal variables.

DRCFS: Doubly Robust Causal Feature Selection

Francesco Quinzan*, Ashkan Soleymani*, Patrick Jaillet, Cristian R. Rojas, Stefan Bauer

ICML : 28468-28491 (2023) 

*equal contribution

arXiv e-Print: CoRR abs/2306.07024  (2023) 

Knowing the features of a complex system that are highly relevant to a particular target variable is of fundamental interest in many areas of science. Existing approaches are often limited to linear settings, sometimes lack guarantees, and in most cases, do not scale to the problem at hand, in particular to images. We propose DRCFS, a doubly robust feature selection method for identifying the causal features even in nonlinear and high dimensional settings. We provide theoretical guarantees, illustrate necessary conditions for our assumptions, and perform extensive experiments across a wide range of simulated and semi-synthetic datasets. DRCFS significantly outperforms existing state-of-the-art methods, selecting robust features even in challenging highly non-linear and high-dimensional problems.

Fast Feature Selection with Fairness Constraints

Francesco Quinzan, Rajiv Khanna, Moshik Hershcovitch, Sarel Cohen, Daniel G. Waddington, Tobias Friedrich, Michael W. Mahoney 

AISTATS : 7800-7823 (2023) 

arXiv e-Print: CoRR abs/2202.13718  (2022) 

A preliminary version of this work was presented at the First ICML Workshop on Formal Verification of Machine Learning (2023).

We study the fundamental problem of selecting optimal features for model construction. This problem is computationally challenging on large datasets, even with the use of greedy algorithm variants. To address this challenge, we extend the adaptive query model, recently proposed for the greedy forward selection for submodular functions, to the faster paradigm of Orthogonal Matching Pursuit for non-submodular functions. The proposed algorithm achieves exponentially fast parallel run time in the adaptive query model, scaling much better than prior work. Furthermore, our extension allows the use of downward-closed constraints, which can be used to encode certain fairness criteria into the feature selection process. We prove strong approximation guarantees for the algorithm based on standard assumptions. These guarantees are applicable to many parametric models, including Generalized Linear Models. Finally, we demonstrate empirically that the proposed algorithm competes favorably with state-of-the-art techniques for feature selection, on real-world and synthetic datasets.

Optimal Transport for Correctional Learning

Rebecka Winqvist, Inês Lourenço, Francesco Quinzan, Cristian R. Rojas, Bo Wahlberg

CDC : 6806-6812 (2023)

arXiv e-Print: CoRR abs/2207.09768 (2023) 

The contribution of this paper is a generalized formulation of correctional learning using optimal transport, which is about how to optimally transport one mass distribution to another. Correctional learning is a framework developed to enhance the accuracy of parameter estimation processes by means of a teacher-student approach. In this framework, an expert agent, referred to as the teacher, modifies the data used by a learning agent, known as the student, to improve its estimation process. The objective of the teacher is to alter the data such that the student's estimation error is minimized, subject to a fixed intervention budget. Compared to existing formulations of correctional learning, our novel optimal transport approach provides several benefits. It allows for the estimation of more complex characteristics as well as the consideration of multiple intervention policies for the teacher. We evaluate our approach on two theoretical examples, and on a human-robot interaction application in which the teacher's role is to improve the robots performance in an inverse reinforcement learning setting.

Adaptive Sampling for Fast Constrained Maximization of Submodular Functions

Francesco Quinzan, Vanja Doskoc, Andreas Göbel, Tobias Friedrich 

AISTATS 2021: 964-972 

arXiv e-Print: CoRR abs/2102.06486  (2021) 

Several large-scale machine learning tasks, such as data summarization, can be approached by maximizing functions that satisfy submodularity. These optimization problems often involve complex side constraints, imposed by the underlying application. In this paper, we develop an algorithm with poly-logarithmic adaptivity for non-monotone submodular maximization under general side constraints. The adaptive complexity of a problem is the minimal number of sequential rounds required to achieve the objective. Our algorithm is suitable to maximize a non-monotone submodular function under a p-system side constraint, and it achieves a (p+O(p^(1/2)))-approximation for this problem, after only poly-logarithmic adaptive rounds and polynomial queries to the valuation oracle function. Furthermore, our algorithm achieves a (p+O(1))-approximation when the given side constraint is a p-extendible system. This algorithm yields an exponential speed-up, with respect to the adaptivity, over any other known constant-factor approximation algorithm for this problem. It also competes with previous known results in terms of the query complexity. We perform various experiments on various real-world applications. We find that, in comparison with commonly used heuristics, our algorithm performs better on these instances.

Evolutionary algorithms and submodular functions: benefits of heavy-tailed mutations

Francesco Quinzan, Andreas Göbel, Markus Wagner, Tobias Friedrich:

Nat. Comput. 20(3): 561-575 (2021) 

arXiv e-Print: CoRR abs/2102.06486 (2018)

A core feature of evolutionary algorithms is their mutation operator. Recently, much attention has been devoted to the study of mutation operators with dynamic and non-uniform mutation rates. Following up on this line of work, we propose a new mutation operator and analyze its performance on the (1+1) Evolutionary Algorithm (EA). Our analyses show that this mutation operator competes with pre-existing ones, when used by the (1+1) EA on classes of problems for which results on the other mutation operators are available. We show that the (1+1) EA using our mutation operator finds a (1/3)-approximation ratio on any non-negative submodular function in polynomial time. We also consider the problem of maximizing a symmetric submodular function under a single matroid constraint and show that the (1+1) EA using our operator finds a (1/3)-approximation within polynomial time. This performance matches that of combinatorial local search algorithms specifically designed to solve these problems and outperforms them with constant probability. Finally, we evaluate the performance of the (1+1)EA using our operator experimentally by considering two applications: (a) the maximum directed cut problem on real-world graphs of different origins, and with up to 6.6 million vertices and 56 million edges and (b) the symmetric mutual information problem using a four month period air pollution data set. In comparison with uniform mutation and a recently proposed dynamic scheme our operator comes out on top on these instances. 

Non-Monotone Submodular Maximization with Multiple Knapsacks in Static and Dynamic Settings

Vanja Doskoc, Tobias Friedrich, Andreas Göbel, Aneta Neumann, Frank Neumann, Francesco Quinzan*

ECAI 2020: 435-442

*leading author; the authors are listed in alphabetical order

arXiv e-Print: CoRR abs/1811.05351 (2018)

We study the problem of maximizing a non-monotone submodular function under multiple knapsack constraints. We propose a simple discrete greedy algorithm to approach this problem, and prove that it yields strong approximation guarantees for functions with bounded curvature. In contrast to other heuristics, this requires no problem relaxation to continuous domains and it maintains a constant-factor approximation guarantee in the problem size. In the case of a single knapsack, our analysis suggests that the standard greedy can be used in non-monotone settings. Additionally, we study this problem in a dynamic setting, by which knapsacks change during the optimization process. We modify our greedy algorithm to avoid a complete restart at each constraint update. This modification retains the approximation guarantees of the static case. We evaluate our results experimentally on a video summarization and sensor placement task. We show that our proposed algorithm competes with the state-of-the-art in static settings. Furthermore, we show that in dynamic settings with tight computational time budget, our modified greedy yields significant improvements over starting the greedy from scratch, in terms of the solution quality achieved.