Research

Multimodal learning for resource-constrained, safety-critical environments

The settings in which an undetected failure matters most (endoscopic and surgical video, diagnostic imaging, perception on operational industrial sites) are frequently the settings in which annotated data is scarcest, in which data cannot be pooled across institutions for governance reasons, and in which inference must run within a constrained budget. A model in these environments must also reason over several modalities at once: image and clinical text, video and instrument signal, imaging and patient record. The programme approaches that single problem from four sides; it is not a set of separate topics.

Reliability under distribution shift. Establishing when a multimodal model is operating outside the distribution on which it was trained, such that it may decline to answer rather than answer confidently: out-of-distribution detection, neuron-level attribution, and the quantification of hallucination in medical vision-language models (T2FNorm, ReweightOOD, NERO, TTA-OOD, MedTech-Bench).

Learning under resource constraint. Training where the constraint is binding rather than incidental: modalities missing at some sites and present at others, data that cannot leave the institution, annotation that is scarce and noisy, inference budgets that exclude large models, and languages without a substantial corpus (CAR-MFL, ProMoE-FL, FedVG, Dimension Mixer, NepBERTa).

Multimodal perception in the field. Perception that must hold up on equipment, procedures and populations absent from the training data: semantic segmentation, monocular depth and camera pose in intracorporeal imaging, guidewire reconstruction from bi-planar X-ray, and latent action models for surgical world models (SurgTransfer, Guide3D, FetReg2021, CholecTriplet2022).

Evaluation resources. Claims regarding reliability cannot be substantiated in domains for which no annotated evaluation data exist. We construct these resources with clinical collaborators and release them under open licences: SAGE, MedTech-Bench, the GI endoscopy hallucination benchmark, Guide3D and the NepBERTa corpus.

Where the work is deployed. Clinical: gastrointestinal endoscopy, fetoscopy, surgical video, non-contrast CT head and cardiac imaging, with partners in the United Kingdom, Nepal and Switzerland. Industrial: as Chief AI Scientist at Fogsphere (Redev AI Ltd), I direct the research programme underlying the company's perception systems for safety-critical industrial environments, including energy, oil and gas, manufacturing, logistics, mining and construction.

Earlier and adjacent work. Before 2022, at Imperial College London and Normandie Universite, my work concerned face analysis, conditional generative models and large-scale metric learning; those papers remain in the publication list.