Generative Models for Synthesizing Medical Images

About

Medical imaging is essential for diagnosis and treatment but faces challenges like high costs, patient risks (e.g., radiation exposure), and limited datasets, which hinder the generalizability of AI models. Generative models—such as GANs, VAEs, and Diffusion Models—offer solutions by synthesizing realistic medical images to augment datasets, reduce biases, and simulate disease progression. However, medical data scarcity, privacy concerns, and the complexity of high-dimensional 3D imaging (e.g., MRIs) demand specialized approaches to ensure anatomical accuracy and ethical compliance.

This tutorial addresses these challenges through hands-on exploration of generative architectures, privacy-preserving techniques (e.g., federated learning), and multi-modal synthesis using tools like PyTorch and MONAI. It emphasizes practical implementation with real-world datasets (e.g., ADNI) and evaluation frameworks that combine technical metrics (e.g., visual realism) with clinical validation to meet regulatory standards. Focus areas include 3D MRI/CT generation and strategies to balance innovation with patient safety and data integrity.

Date, Time, and Location

Date: September 23, 2025

Time: 8am-12:30pm (see Schedule below)

Location: DCC1-1F-106

Format and Schedule

Each speaker will present for 35 minutes, followed by 10 minutes for Q&A.

The final panel discussion will be attended by available speakers and moderated by an organizer.

08:00am-08:30am - Alan Wang

Title: Introduction and overview

Slides: 

Abstract: This timeslot will serve as a general overview of the state of research, current approaches and architectures, and best code to get started with generative models for medical images. Details will be discussed including overviews of cutting-edge approaches like diffusion and autoregressive models, toolboxes like MONAI, and representative papers and open problems in the field.

08:30am-09:15am - Kilian Pohl

Title: Building Diffusion Models to Accelerate Discovery in Neuroscience

Abstract: While generative models of structural brain MRIs generally prioritize capturing coarse, visible effects of diseases (such as ventricular enlargement in Alzheimer’s disease), many neuroscience studies focus on detecting morphometric group differences between cohorts that are too subtle to identify visually in individuals. In this talk, we review several diffusion models designed to generate high-quality T1-weighted MRIs specifically tailored for neuroscience discovery. We present a systematic procedure for evaluating the ability of these synthetic MRIs to capture macrostructural properties of brain regions including subtle, disease-related effects. Finally, we highlight the successes and limitations of current technology in meeting critical criteria for incorporating synthetic MRIs into neuroscience research.

09:15am-10:00am - Qingyu Zhao

Title: Generating and Evaluating Anatomical Plausibility in Brain Morphometry and Longitudinal Change

Abstract: Accurately characterizing and predicting changes in brain morphometry is essential for understanding normal neurodevelopment and aging, as well as deviations associated with brain disorders. In this talk, I will present recent advances in generating and evaluating the realism of brain morphometry using anatomically informed AI models. These approaches include deformation-based modeling, latent diffusion, spatiotemporal modeling, and segmentation-guided supervision to produce realistic morphometric changes from limited data while preserving topology and tissue boundaries. I will also highlight methods for rigorously assessing the anatomical plausibility of synthetic brain MRIs and their longitudinal trajectories. Together, these advances offer a pathway toward both generating realistic brain morphometry over time and rigorously validating its anatomical plausibility for research and clinical applications.

10:00am-10:30am - Break

10:30am-11:15am - Ulas Bagci

Title: Eyes Tell the Truth: GazeVal highlights the shortcomings of Generative AI in Medical Imaging

Abstract: Current evaluations for synthetic medical imaging data predominantly rely on computational metrics that fail to align with human expert recognition. This leads to synthetic images that may appear realistic numerically but lack clinical authenticity, posing significant challenges in ensuring the reliability and effectiveness of AI-driven medical tools. To address this gap, we will talk about a practical framework (called GazeVal) that synergizes expert eye-tracking data with direct radiological evaluations to assess the quality of synthetic medical images. GazeVal leverages gaze patterns of radiologists as they provide a deeper understanding of how experts perceive and interact with synthetic data in different tasks (ie, diagnostic or Turing tests). We will also give some initial results on evaluation of synthetic medical scans. Experiments with sixteen radiologists revealed that 96.6% of the generated images (by the most recent state-of-the-art AI algorithm) were identified as fake, demonstrating the limitations of generative AI in producing clinically accurate images.

11:15am-12:00pm - Rajat Rasal

Title: Deep Structural Causal Models

Abstract: In this talk, we will discuss a causal generative modelling framework for the synthesis of high-fidelity image counterfactuals. We will revisit some principles of causal modelling and make a case for why causality matters in medical imaging. The capability of deep structural causal models is demonstrated on multiple applications, including the targeted synthesis of chest radiographs and digital mammograms with specific characteristics. We will also discuss some of the limitations of this approach and lay out directions for future work.

12:00pm-12:30pm - Panel discussion