AI for health webinar

Apr 2, 2025 (2-3PM CT)

Title: Enhancing Temporal Processing in EHR Foundation Models

Abstract: We present two complementary approaches for modeling longitudinal Electronic Health Records (EHRs). TIMER (Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records) improves temporal analysis through a time-aware benchmark and instruction-tuning methodology, boosting performance by 7-9% on benchmarks. Simultaneously, we address context length limitations using subquadratic architectures like Mamba, enabling models to process complete patient records exceeding 10,000 events. Our long-context model outperforms state-of-the-art on 9/14 EHRSHOT prediction tasks while showing greater robustness to EHR-specific challenges: copy-forwarded diagnoses, irregular time intervals, and progressive disease complexity. These approaches significantly enhance clinical prediction tasks by better capturing the longitudinal nature of patient care.

Bio: Sanmi Koyejo is an assistant professor in the Department of Computer Science at Stanford University and a co-founder of Virtue AI. At Stanford, Koyejo leads the Stanford Trustworthy Artificial Intelligence (STAIR) lab, which works to develop the principles and practice of trustworthy AI, focusing on applications to science and healthcare. Koyejo has been the recipient of several awards, including a Skip Ellis Early Career Award, a Presidential Early Career Award for Scientists and Engineers (PECASE), a Sloan Fellowship, a Terman faculty fellowship, an NSF CAREER award, a Kavli Fellowship, and an IJCAI early career spotlight. Koyejo serves on the Neural Information Processing Systems Foundation Board, the Association for Health Learning and Inference Board, and as president of the Black in AI Board.

March 19, 2025 (2-3PM CT)

Title:  Multimodal, Generative and Agentic AI for Pathology

Abstract: Advances in digital pathology and artificial intelligence have presented the potential to build models for objective diagnosis, prognosis and therapeutic-response and resistance prediction. In this talk we will discuss our work on: (1) Data-efficient methods for weakly-supervised whole slide classification with examples in cancer diagnosis and subtyping (Nature BME, 2021), identifying origins for cancers of unknown primary (Nature, 2021) and allograft rejection (Nature Medicine, 2022) (2) Discovering integrative histology-genomic prognostic markers via interpretable multimodal deep learning (Cancer Cell, 2022; IEEE TMI, 2020; ICCV, 2021; CVPR, 2024; ICML, 2024). (3) Building unimodal and multimodal foundation models for pathology, contrasting with language and genomics (Nature Medicine, 2024a, Nature Medicine 2024b, CVPR 2024). (4) Developing a universal multimodal generative co-pilot and chatbot for pathology (Nature, 2024). (5) 3D Computational Pathology (Cell, 2024) (6) Bias and fairness in computational pathology algorithms (Nature Medicine, 2024; Nature BME 2023).

Bio: Dr. Mahmood is an Associate Professor of Pathology at Harvard Medical School and the Division of Computational Pathology at the Brigham and Women’s Hospital. He received his Ph.D. in Biomedical Imaging from the Okinawa Institute of Science and Technology, Japan and was a postdoctoral fellow at the department of biomedical engineering at Johns Hopkins University. His research interests include pathology image analysis, morphological feature, and biomarker discovery using data fusion and multimodal analysis

Feb 26, 2025 (2-3PM CT)

Title: AI agents for biomedicine and healthcare

Abstract: This talk will explore how generative AI agents can enable scientific discoveries. First, I’ll introduce the Virtual Lab—a collaborative team of AI scientist agents conducting in silico research meetings to tackle open-ended R&D projects. The Virtual Lab designed new nanobody binders to recent Covid variants that we experimentally validated. Then I will discuss the role of AI agents in healthcare.  

Bio: James Zou is an associate professor of Biomedical Data Science, CS and EE at Stanford University. He works on advancing the foundations of ML and in-depth scientific and clinical applications. Many of his innovations are widely used in tech and biotech industries.  He has received a Sloan Fellowship, an NSF CAREER Award, two Chan-Zuckerberg Investigator Awards, a Top Ten Clinical Achievement Award, several best paper awards, and faculty awards from Google, Amazon, Adobe and Apple. His research has also been profiled in popular press including the NY Times, WSJ, and WIRED. 

March 12, 2025 (2-3PM CT)

Title:  Large language models for clinical trial design, execution, and analysis

Abstract: Large language models (LLMs) hold great promise for accelerating clinical trials, but fully realizing their potential in medicine requires targeted applications across key research tasks. This talk focuses on three essential steps in clinical trials: design, execution, and analysis. Specifically, with a highlight on literature research, clinical research, and data science research. I will first introduce LEADS and TrialMind, which build specialized LLMs for literature mining tasks and boost human-AI collaboration. Next, I will present TrialGPT, an LLM pipeline that streamlines patient recruitment in clinical trials. Finally, I will discuss how LLMs can accelerate hypothesis validation through code generation and analysis of medical and biomedical data.

Bio: Zifeng Wang is the cofounder of Keiji AI. He is dedicated to leveraging artificial intelligence (AI) to accelerate scientific discovery in medicine. His research focuses on AI-driven literature research, data science research, and clinical research, with an emphasis on improving literature search, screening, and data extraction, automating the analysis and modeling of clinical trials and real-world data, and optimizing clinical trial design, recruitment, and outcome analysis. His work has been featured in Nature, the National Institutes of Health (NIH), the NIH Director's Blog, POLITICO, and other media outlets. Before joining UIUC, he earned his bachelor's and master's degrees from Tongji University and Tsinghua University, respectively.

Feb 12, 2025 (2-3PM CT)

Title: Generative AI for Multimodal Biomedicine: applications in pathology and retinal imaging

Abstract: Biomedicine is inherently multimodal, including imaging modalities such as pathology, CT, MRI, X-ray and ultrasounds, as well as omics modality such as genomics, epigenomics and transcriptomics. General domain multimodal approaches are not applicable to biomedicine because biomedical images are very different from general domain images, thus necessitating the development of modality-specific approaches. In this talk, I will introduce three recent works towards building multimodal biomedicine foundation models. First,  I will introduce GigaPath, the first whole-slide pathology foundation model that can handle gigapixel-level pathology images. GigaPath exploits a novel vision transformer architecture and achieves the state-of-the-art results on 23 out of 26 cancer tasks, including subtyping and biomarker prediction. Next, I will introduce OCTCube, the first 3D OCT retinal imaging foundation model. OCTCube significantly outperformed 2D models on 27 out of 29 tasks, including retinal disease prediction, cross-modality analysis, cross-device generalization and systemic disease prediction. Finally, I will introduce BiomedParse, a multi-modal foundation model that integrates 9 major biomedical imaging modalities by projecting all of them into the text space, resulting in superior performance on segmentation, detection, and recognition, paving the path for large-scale image-based biomedical discovery. I will conclude this task with discussion on how multi-modal generative AI can advance future medical applications through multi-agent framework and integration with multi-omics datasets.

Bio: Dr. Sheng Wang is an assistant professor in the School of Computer Science and Engineering at the University of Washington Seattle. He obtained his B.S. degree in Computer Science from Peking University, Ph.D. degree in Computer Science from University of Illinois at Urbana Champaign, and conducted postdoc training at Stanford School of Medicine. Sheng is currently interested in developing large-scale models for biomedical applications, with a focus on digital pathology, medical imaging foundation models, chromatin structure prediction, and genomics-based drug discovery. His research has been published in top venues such as Nature, Science, Nature Biotechnology, Nature Methods, Nature Machine Intelligence and The Lancet Oncology, and used by major biomedical institutes, including Mayo Clinic, Chan Zuckerberg Biohub, UW Medicine, and Providence genomics.

Feb 20, 2025 (2-3PM CT)

Title: White Coat, Black Box - Navigating Pitfalls in Using AI to Augment Diagnostic Decision Making

Jenna Wiens, PhD - Associate Professor of Computer Science and Engineering at U-Michigan

Abstract: AI tools designed to aid clinicians in complex diagnostic decisions have the potential to enhance treatment selection and consequently improve patient outcomes. Despite their promise, simply applying existing AI approaches carries a significant risk of inadvertently perpetuating or even exacerbating biases present in clinical care. In this talk, I will describe our work in developing AI systems to diagnose common causes of acute respiratory failure (i.e., pneumonia, heart failure and/or chronic obstructive pulmonary disease), highlighting challenges and presenting potential strategies to mitigate the risk of these systems replicating harmful biases.

Bio: Jenna Wiens is an Associate Professor of Computer Science and Engineering (CSE), Associate Director of the Artificial Intelligence Lab, and Co-director of Precision Health at the University of Michigan in Ann Arbor. Her primary research interests lie at the intersection of machine learning and healthcare. Wiens received her PhD from MIT in 2014, received an NSF CAREER award in 2016, was named to the MIT Tech Review’s list of Innovators Under 35 in 2017, was awarded a Sloan Research Fellowship in Computer Science in 2020, and most recently received a Humboldt Research Award in recognition of her career achievements to date.

Jan 29, 2025  (2-3PM CT)

Title: Providing Rare Hope for Even Rarer Diseases: How AI can enable broad mining of knowledge and data to power drug repurposing for rare diseases

Abstract: 

Leveraging the power of natural language capabilities it is now possible to do immense scale extraction of codified knowledge into repositories that enable novel and extensive analysis.  In a demonstration of that, the extraction a graph of knowledge from public literature has shown the significant potential to identify hypotheses for accelerated treatment of rare diseases leveraging off-target effects of known and approved drug products.  The demonstration will briefly illustrate the process of extracting the knowledge, querying for insights, and then the challenges that remain to share and validate those insights in the real world.

Bio:

Brian Martin is Chief AI Product Owner, ACOS Senior Research Fellow at AbbVie

Feb 5, 2025  (2-3PM CT)

Title: Taking the Pulse Of Ethical ML in Health

Abstract: Machine learning in health has made impressive progress in recent years, powered by an increasing availability of health-related data and high-capacity models. While many models in health now perform at, or above, humans in a range of tasks across the human lifespan, models also learn societal biases and may replicate or expand them. In this talk, Dr. Marzyeh Ghassemi will focus on the need for machine learning researchers and model developers to create robust models that can be ethically deployed in health settings, and beyond. Dr. Ghassemi's talk will span issues in data collection, outcome definition, algorithm development, and deployment considerations.

Bio:

Dr. Marzyeh Ghassemi is an Associate Professor at MIT in Electrical Engineering and Computer Science (EECS) and Institute for Medical Engineering & Science (IMES), and a Vector Institute faculty member holding a Canadian CIFAR AI Chair and Canada Research Chair. She holds MIT affiliations with the Jameel Clinic and CSAIL.

Professor Ghassemi holds a Herman L. F. von Helmholtz Career Development Professorship, and was named a CIFAR Azrieli Global Scholar and one of MIT Tech Review’s 35 Innovators Under 35. Previously, she was a Visiting Researcher with Alphabet’s Verily and an Assistant Professor at University of Toronto. Prior to her PhD in Computer Science at MIT, she received an MSc. degree in biomedical engineering from Oxford University as a Marshall Scholar, and B.S. degrees in computer science and electrical engineering as a Goldwater Scholar at New Mexico State University.

Professor Ghassemi has previously served as a NeurIPS Workshop Co-Chair and General Chair for the ACM Conference on Health, Inference and Learning (CHIL). She also founded the non-profit Association for Health Learning and Inference. Professor Ghassemi has published across computer science and clinical venues, including NeurIPS, KDD, AAAI, MLHC, JAMIA, JMIR, JMLR, AMIA-CRI, Nature Medicine, Nature Translational Psychiatry, and Critical Care. Her work has been featured in popular press such as Fortune, MIT News, NVIDIA, and The Huffington Post.

Sep 25, 2024 (12:30-1:30PM CT special time)

Title: The Generalist Medical AI Will See You Now

Abstract: 

Accurate interpretation of medical images is crucial for disease diagnosis and treatment, and AI has the potential to minimize errors, reduce delays, and improve accessibility. The focal point of this presentation lies in a grand ambition: the development of 'Generalist Medical AI' systems that can closely resemble doctors in their ability to reason through a wide range of medical tasks, incorporate multiple data modalities, and communicate in natural language. Starting with pioneering algorithms that have already demonstrated their potential in diagnosing diseases from chest X-rays or electrocardiograms, matching the proficiency of expert radiologists and cardiologists, I will delve into the core challenges and advancements in the field. The discussion will navigate towards the topic of label-efficient AI models: with a scarcity of meticulously annotated data in healthcare, the development of AI systems capable of learning effectively from limited labels has become a key concern. In this vein, I'll delve into how the innovative use of self-supervision and pre-training methods has led to algorithmic advancements that can perform high-level diagnostic tasks using significantly less annotated data. Additionally, I will talk about initiatives in data curation, human-AI collaboration, and the creation of open benchmarks to evaluate the generalizability of medical AI algorithms. In sum, this talk aims to deliver a comprehensive picture of the state of 'Generalist Medical AI,' the advancements made, the challenges faced, and the prospects lying ahead.

Bio:

Pranav Rajpurkar is an Assistant Professor at Harvard University in the Department of Biomedical Informatics. His research focuses on developing AI systems that can interpret medical data, reason through complex problems, and communicate at an expert level, with the goal of creating AI doctors that can work independently or alongside human physicians. Rajpurkar has published over 100 academic articles, garnering more than 25,000 citations in prestigious journals. He has been recognized with numerous awards, including Forbes 30 Under 30 in science '22, MIT Tech Review's Innovator Under 35 '23, and Google Research Scholar.

Oct 9, 2024

Title: Baking Responsibility into the AI for Health Lifecycle

Abstract: AI, and particularly machine learning, is reshaping the way we think about scientific research and healthcare.  And yet, the collection and use of patient data, its subsequent conversion into foundation and predication models, and their application in biomedical research and healthcare raises many ethical questions and societal quandaries that have the potential to thwart such activities.  The goal of this presentation is to discuss how ethical reasoning and computational decision making can be embedded into the AI for health lifecycle and ultimately maximize social good.  Along the way, we will review several case studies to understand how things have gone wrong in the past, but also, what can go right!  This presentation will touch on issues of trust, algorithmic fairness, and data privacy.

Bio: Bradley Malin is the Accenture Professor of Biomedical Informatics, Biostatistics, and Computer Science at Vanderbilt University, as well as the Vice Chair for Research Affairs in the Department of Biomedical Informatics at Vanderbilt University Medical Center, where he co-founded and co-directs the AI Discovery and Vigilance to Accelerate Innovation and Clinical Excellence (ADVANCE) Center.  His research is in the development of trustworthy artificial intelligence methodology and infrastructure. He is a principal investigator of several large consortia on artificial intelligence sponsored by the National Institutes of Health, including the Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity (AIM-AHEAD) and Bridge2AI.  Among various honors, he is an elected fellow of the National Academy of Medicine (NAM), the American College of Medical Informatics (ACMI), the American Institute for Medical and Biological Engineering (AIMBE), the International Academy of Health Sciences Informatics (IAHSI), and the Institute of Electrical and Electronics Engineers (IEEE).  He received his bachelor’s in biological sciences, master’s in public policy and management, and doctorate in computer science from Carnegie Mellon University.

Sep 11, 2024

Title: “Unleashing the power of Medical Imaging Foundation Models” 

Description: This session will explore cutting-edge AI and machine learning advancements that are set to transform workflow across technologists and clinicians. Discover how the power of recent advances in AI/ML and generative AI can optimize processes and revolutionize healthcare. 

Bio: Parminder Bhatia (Chief AI Officer - GE Healthcare)

With an impressive track record of 58 FDA clearances, Parminder has been instrumental in driving the integration of AI into medical devices at GE HealthCare, with a clear strategic vision aimed at enhancing patient care and outcomes. His achievements have not gone unnoticed, as evidenced by his recognition as one of the top 10 Chief AI Officers, highlighting his significant influence and forward-thinking approach in the industry.

Parminder's expertise extends beyond his tenure at GE HealthCare, with notable contributions during his time at Amazon, where he led machine learning efforts for Generative AI technologies akin to Chat GPT. His diverse experiences have enriched his understanding of AI technologies and their applications in healthcare, leading to groundbreaking innovations that have revolutionized clinical decision-making and patient care in the field of cancer treatment as well as his contributions during Covid. 

Parminder's commitment to advancing healthcare extends globally, as demonstrated by initiatives like the Caption Guidance product, which aims to provide efficient ultrasound assessments to healthcare professionals worldwide, particularly emphasizing accessibility in low-and-middle income countries (LMIC).

Sep 18, 2024

Title: Multimodal Generative AI for Precision Health

Abstract: The dream of precision health is to develop a data-driven, continuous learning system where new health information is instantly incorporated to optimize care delivery and accelerate biomedical discovery. The confluence of technological advances and social policies has led to rapid digitization of multimodal, longitudinal patient journeys, such as electronic medical records (EMRs), imaging, and multiomics. Our overarching research agenda lies in advancing multimodal generative AI for precision health, where we harness real-world data to pretrain powerful multimodal patient embedding, which can serve as digital twins for patients. This enables us to synthesize multimodal, longitudinal information for millions of cancer patients, and apply the population-scale real-world evidence to advancing precision oncology in deep partnerships with real-world stakeholders such as large health systems and pharmaceutical companies.

Bio: Hoifung Poon is the General Manager at Health Futures in Microsoft Research and an affiliated faculty at the University of Washington Medical School. He leads biomedical AI research and incubation, with the overarching goal of structuring medical data to optimize delivery and accelerate discovery for precision health. His team and collaborators are among the first to explore large language models (LLMs) and multimodal generative AI in health applications, producing popular open-source foundation models such as PubMedBERT, BioGPT, BiomedCLIP, LLaVA-Med. His latest publication in Nature features GigaPath, the first whole-slide digital pathology foundation model pretrained on over 1 billion pathology image tiles. He has led successful research partnerships with large health providers and life science companies, creating AI systems in daily use for applications such as molecular tumor board and clinical trial matching. He has given tutorials on these topics at top AI conferences such as ACL, AAAI, and KDD, and his prior work has been recognized with Best Paper Awards from premier AI venues such as NAACL, EMNLP, and UAI. He received his PhD in Computer Science and Engineering from the University of Washington, specializing in machine learning and NLP.

May 24, 2024
Title: Getting past the academics of AI to generate financial value.  Case studies in AI for Clinical Trials.

Abstract

In the dynamic landscape of pharmaceutical research and development (R&D), integrating artificial intelligence (AI) has become increasingly crucial. However, successful implementation is not guaranteed, and failures typically arise from sources not taught in graduate school. This talk will delve into the challenges faced during AI deployment within the pharmaceutical industry, highlighting both unsuccessful endeavors and transformative successes.

Understanding User Adoption:

AI solutions can be technically robust, yet their adoption hinges on how end users interact with them. Failures often occur when developers overlook the user experience, leading to poor adoption rates.  Case studies will illustrate instances where AI predictions were accurate but failed to gain traction due to usability issues.

Navigating Resistance:

Existing processes and practices within pharmaceutical R&D can be resistant to change. AI disrupts established workflows, and overcoming this resistance is critical.  We’ll explore scenarios where AI initiatives faced pushback from stakeholders, hindering their effectiveness.

Cracking the Commercial Model:

AI’s success extends beyond technical prowess—it must align with a viable commercial model. Failing to understand this can lead to missed opportunities. Real-world examples will showcase AI implementations that thrived due to a well-defined commercial strategy.

By examining both failures and triumphs, we aim to equip AI professionals with insights to navigate the complex R&D business landscape effectively.

Bio:

Lucas Glass is the Senior Vice President of Technology at IQVIA where he oversees enterprise architecture and technology strategy.  IQVIA is the largest healthcare data provider and clinical research organization in the world.  Prior to his current role, Lucas was the Vice President of AI at IQVIA.  Lucas has launched over a dozen machine learning offerings within R&D such as site recommender systems, trial matching solutions, enrollment rate algorithms, drug target interactions, drug repurposing, and molecular optimization.  Lucas’ machine learning research which is dedicated to R&D has been accepted at AAAI, WWW, ICML, JAMIA, KDD, and many others.

May 1, 2024

Title: Foundation Models and Generative AI for Medical Imaging Segmentation in Ultra-Low Data Regimes

Abstract: Semantic segmentation of medical images is pivotal in disease diagnosis and treatment planning. While deep learning has excelled in automating this task, a major hurdle is the need for numerous annotated masks, which are resource-intensive to produce due to the required expertise and time. This scenario often leads to ultra-low data regimes where annotated images are scarce, challenging the generalization of deep learning models on test images. To address this, we introduce two complementary approaches. One involves developing foundation models. The other involves generating high-fidelity training data consisting of paired segmentation masks and medical images. In the former, our bi-level optimization based method can effectively adapt the general-domain Segment Anything Model (SAM) to the medical domain with just a few medical images. In the latter, our multi-level optimization based method can perform end-to-end generation of high-quality training data from a minimal number of real images. On eight segmentation tasks involving various diseases, organs, and imaging modalities, our methods demonstrate strong generalization performance in both in-domain and out-of-domain settings. Our methods require 8-12 times less training data than baselines to achieve comparable performance. 

Bio: Pengtao Xie is an assistant professor in the Department of Electrical and Computer Engineering at the University of California San Diego. His research interest lies in machine learning for healthcare. His PhD thesis was selected as a top-5 finalist for the Doctoral Dissertation Award of the American Medical Informatics Association (AMIA). He was recognized as Global Top-100 Chinese Young Scholars in Artificial Intelligence by Baidu, Tencent AI-Lab Faculty Award, Innovator Award by the Pittsburgh Business Times, Amazon AWS Machine Learning Research Award, among others. He serves as an associate editor for the ACM Transactions on Computing for Healthcare, senior area chair for AAAI, area chairs for ICML and NeurIPS, etc.

Apr 24, 2024

Title: "AI and the healthcare data logistics problem"

Abstract: 

Healthcare decisions depend heavily on comprehensive, accessible data. Without these, patient outcomes may suffer. However, healthcare data is often fragmented across multiple silos, limiting its true utility. In this lecture, we will discuss healthcare data fragmentation, and how overcoming it can lead to better datasets that can be used to train and fine-tune AI models. We will explore the concept of "Real World Data" (RWD) — information gathered from routine healthcare activities and daily life, and the use case applications of RWD.

We will also address how AI tools and healthcare intersect in two important ways. First, AI can improve healthcare data through standardization, curation, and more. On the flip side, healthcare data can improve and fine-tune AI models. We will discuss the importance of using complete and representative data to train AI models to avoid bias, and ensuring compliance with privacy regulations like HIPAA and GDPR.

The session will conclude with some interesting predictions on the intersection of healthcare and AI. I encourage the class to think of your own predictions as well.

My bio:

Vera Mucaj, Ph.D.,

Chief Scientific Officer, Datavant

Vera is the Chief Scientific Officer at Datavant, where she leads cross-functional teams dedicated to advancing technology that powers health data logistics. As a Product leader, Vera has successfully guided the development  and widespread adoption of technology supporting healthcare data connectivity through Privacy Preserving Record Linkages. Her subject-matter expertise includes the technical and use case considerations for linking clinical trial data to real-world data, and the creation of large-scale real-world data research databases in both the public and private sectors.

Vera is a research scientist by training, and holds a BA in Biochemistry from Harvard College, and a PhD in Cell and Molecular Biology from the University of Pennsylvania Perelman School of Medicine. She has authored multiple peer-reviewed scientific papers in cancer research. Prior to joining Datavant, Vera worked at McKinsey & Company, where she supported pharmaceutical clients on growth strategy, business development, and M&A.