Research & Projects

Representation drift under text overlays (blue) against the distance between different images (orange): cosine distance, 600 SEED-Bench images. The two distributions overlap substantially.

Robustness of vision-language models

Same Answer, Different Representations: Hidden Instability in VLMs

EMNLP 2026, main conference · first author

A vision-language model can keep the same answer while its internal representation moves a long way. I worked on this question as a visiting researcher at the University of Edinburgh, hosted by Dr Pasquale Minervini, who is a co-author of the paper. We compare a clean image with perturbed versions of it under the same question, and we measure three things together: whether the answer changes, how far the hidden state moves, and how the smoothness of the vision tokens changes.

What we found

For Qwen3-VL-2B on SEED-Bench, 37.6% of images have at least one answer flip across the perturbation suite. Text overlays flip 19.2% of individual inputs and rotations 12.2%.
Mean representation drift under text overlays (0.177 in cosine distance) is close to the mean distance between different images (0.228).
Larger models are not consistently more stable: across Qwen3-VL sizes, accuracy gains and answer stability do not move together.
On POPE, perturbations can suppress hallucinations and destroy correct detections at the same time.

Preprint (arXiv:2602.06652) · Code

Follow-up: Cross-Modal Attention Acts as a Frequency Filter

Why verbose prompts improve robustness in vision-language models. First author, under review.

In the follow-up we separate extra wording from extra reasoning. Every question has a verbose mirror that keeps the same options and the same gold answer. Across Qwen3-VL 2B and 8B on GQA and CLEVR, reasoning complexity destabilises the gold-answer score under image corruptions and supportive verbosity stabilises it: on the 8B model the verbose mirrors lower the drift variance by 70 to 81%. We read text-to-image attention as a spectral profile, and its overlap with the spectrum of the perturbation correlates positively with the drift (r between 0.14 and 0.46).

MNIST with 50% symmetric label noise. Cross-entropy ends up fitting the wrong labels and its test accuracy falls; with NCOD, accuracy on the noisy labels stays near zero and test accuracy remains high.

Learning with noisy labels

NCOD: Learning with Noisy Labels through Learnable Weighting and Centroid Similarity

IJCNN 2024 · first author

Deep networks learn useful patterns first and memorise wrong labels later. With NCOD (Noisy Centroids Outlier Discounting) we use this early separation between clean and noisy samples to limit memorisation. We give each sample a soft target weighted by its similarity to the centroid of its observed class, and a learnable discount that grows when the model keeps disagreeing with the observed label. The noise rate never enters the objective.

Key contributions

We propose a loss that uses the learned similarity between a sample and its class centroid as a soft label.
We add a theoretically grounded regularisation term that learns a discount for each sample.
We test on synthetic and real-world noise. On CIFAR-100 with two PreActResNet-34 networks, NCOD+ reaches 81.66%, 78.00% and 68.40% test accuracy under 20%, 50% and 80% symmetric label noise.
On mini-WebVision, which contains real annotation noise from web data, NCOD reaches 78.44% with InceptionResNetV2, against 76.6% published for SOP.
The method needs neither the noise rate of the dataset nor anchor points, which keeps it simple to apply.

Paper · Code

Average Dirichlet energy during training on the controlled PPA subset. With 20% label noise, energy rises as cross-entropy fits the noisy labels; with GCOD it stays close to the noise-free run.

Graph learning under label noise

The Energy of Memorization: Why Graph-Level Label Noise Sharpens Representations and How to Stop It

GCOD · first author · under review

After NCOD we took the same idea to graphs, and found that graph neural networks do not always memorise noisy graph labels. On the full ogbg-ppa dataset a 5-layer GIN is under-parameterised and cross-entropy is already robust; memorisation appears with fewer classes, smaller graphs or smaller training sets. When it does appear, it leaves a signature: the Dirichlet energy of the learned node representations rises as the network fits the noisy labels, and the rise concentrates in the high-frequency component.

What we did

We prove a conditional lower bound that links the separation of two graphs with close mean representations to their Dirichlet energy and spectral gap.
We redesigned NCOD for graphs as GCOD (Graph Centroid Outlier Discounting), with training-accuracy feedback that switches the discount on gradually and a KL term that separates clean from noisy graphs.
On a controlled subset of ogbg-ppa with 40% label noise, final test accuracy is 91.08% with GCOD against 51.83% with cross-entropy, for 2.9% additional training time.
GCOD improves over cross-entropy under symmetric and asymmetric noise on six of seven further graph-classification benchmarks; REDDIT is unchanged.
We also show that clean overfitting raises energy, only far less steeply, so energy is a diagnostic and not a noise detector with a universal threshold.

Preprint (arXiv:2412.08419)

More research

Corrective machine unlearning: CUTS

Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmetic. ECML PKDD 2026, Research Track.

I co-led this project. A trained model contains label-noise or backdoor effects and the original training data are no longer available. We treat the corruption as a task in weight space: we fine-tune the corrupted model on a small proxy set that reproduces the corruption, estimate the resulting update and subtract a calibrated multiple of it. Across CLIP models, CUTS closes 69.4% of the accuracy gap to the oracle under label noise on average, and on backdoors it reaches a mean poison score of 90.3% against 54.5% for Potion. Preprint

Graph out-of-distribution generalisation: PISA

PISA: Prioritized Invariant Subgraph Aggregation. Learning on Graphs Conference (LoG) 2026.

I co-supervised this project. With PISA we learn how to combine the predictions of several invariant subgraph predictors: the branches are trained first and frozen, and a small MLP then learns an input-dependent combination of their class scores. Compared with SuGAr, the average rises from 80.41 to 83.37 on six synthetic settings and from 63.66 to 65.40 on nine realistic ones. Preprint

IJCNN 2025 Competition: Learning with Noisy Graph Labels

With my colleagues I organised a public competition on graph classification under label noise: four datasets built from ogbg-ppa with symmetric and asymmetric noise at 20% and 40%, a GIN baseline, a withheld private test set and mandatory code submission. We received nine successful submissions. Paper · Competition materials

Audio deepfake detection

I co-authored Navigating the Soundscape of Deception, a survey of audio deepfake generation and detection published in Foundations and Trends in Privacy and Security (2024). My part was GAN-based voice conversion and the comparative analysis of detectors. I also co-authored HCN-TA, a hierarchical capsule network with temporal attention (ACM SAC 2025), which reaches an equal error rate of 0.42% on ASVspoof 2019 LA; there I worked on the attention and locality design, the evaluation protocol and the ablations. Survey · HCN-TA

Survey and Benchmark of Graph Learning under Label Noise

I am joint first author and corresponding author of this survey and benchmark of methods for graph learning under label noise. It is under review. Preprint (HAL) · Benchmark code

For the full publication list, please see the

Resume

Page updated

Report abuse