Multimodal Learning
for Materials Science
MM4Mat
Embedded Files
CVPR 2025 Workshop
June 12, 2025 1:00 - 5:00 PM
Music City Center, Nashville, Tennessee
Level 1, Room 110 A
About
About
Multimodal learning stands at the cutting edge of AI research, revolutionizing fields such as medical science and text-to-speech conversion. Materials science datasets often encompass diverse data types and critical feature nuances, presenting a distinctive and exciting opportunity to extend the development and application of multimodal learning architectures to tackle scientific challenges.
The Multimodal Learning for Materials Science (MM4Mat) workshop aspires to shape and nurture interdisciplinary discussions and collaborations between machine learning experts and materials scientists. By bringing together world-leading experts in multimodal AI and data-driven materials discovery, this workshop will highlight innovative methodologies and their real-world applications in materials science.
Key topics of focus include:
Multimodal Architectures: Strategies for integrating diverse materials science data.
Encoder and Decoder Design: Tailored solutions for important materials science modalities.
Real-World Applications: Demonstrating impactful uses of multimodal learning in materials science.
Join us at our half-day, hybrid workshop as we advance the intersection of AI and materials science, fostering innovation and collaboration!
Keynote
Speakers
Keynote
Speakers
Greg Slabaugh
Queen Mary University of London
Dr. Greg Slabaugh is Professor and Director of the Digital Environment Research Institute at Queen Mary University of London and an expert in computer vision and artificial intelligence. His research includes deep learning, computational photography, and medical image computing. Prior to joining Queen Mary, he was Chief Scientist in Computer Vision (EU) for Huawei and has held other positions in industry and Siemens and Medicsight. His work on multimodal AI and medical image analysis aligns with the workshop's focus, offering valuable insights. Dr. Slabaugh frequently serves on program committees for leading computer vision and machine learning conferences such as CVPR, NeurIPS, and AAAI.
Tian Xie
Microsoft
Dr. Tian Xie is a principal research manager and project lead at Microsoft Research AI for Science. He leads a highly interdisciplinary team of researchers, engineers, and program managers to develop the foundational AI capabilities for materials discovery. Before joining Microsoft, he was a postdoc in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT from 2020 to 2022, co-advised by Tommi Jaakkola and Regina Barzilay. He got his PhD in Materials Science and Engineering at MIT in 2020, advised by Jeffrey C. Grossman. His experience also includes research internships at DeepMind and Google X. Tian’s most noticeable work before Microsoft includes the development of CDVAE and CGCNN.
Website
Sergei Kalinin
University of Tennessee, Knoxville
Dr. Sergei Kalinin is the Weston Fulton Chair Professor in the Department of Materials Science and Engineering at the University of Tennessee, Knoxville. From 2022 to 2023, he was a principal scientist on the Amazon Special Projects team. He spent the prior 20 years at the Center for Nanophase Materials Sciences at Oak Ridge National Laboratory. He received his MS degree from Moscow State University and PhD from the University of Pennsylvania, advised by Dawn Bonnell. His research focuses on the applications of machine learning and artificial intelligence in nanotechnology, atomic fabrication, and materials discovery via scanning transmission electron microscopy, as well as mesoscopic studies of electrochemical, ferroelectric, and transport phenomena via scanning probe microscopy.
Website
Panelists
Panelists
Arghya Bhowmik
Technical University of Denmark
Dr. Arghya Bhowmik is Associate Professor in the Department of Energy Conversion and Storage at the Technical University of Denmark (DTU), specializing in using artificial intelligence and machine learning to accelerate the discovery and design of materials, particularly for renewable energy applications. His research combines high-throughput computations, AI-driven approaches, and materials informatics to address challenges in materials discovery, optimization, and deployment. Dr. Bhowmik has extensive experience in developing computational frameworks to facilitate the design of energy-efficient materials, contributing significantly to advancements in renewable energy technologies.
Website
Linda Hung
Toyota Research Institute
Dr. Linda Hung is a senior manager in the Energy & Materials division at Toyota Research Institute (TRI). Dr. Hung's current work integrates machine learning with materials science simulation and high-throughput experiment datasets, with the goals of shortening the discovery timeline and improving the fundamental understanding of materials. She obtained her Ph.D. in applied and computational mathematics from Princeton University and has held research positions at the Ecole Polytechnique (France), the University of Illinois Chicago, and the National Institute of Standards and Technology.
Website
Aditi Krishnapriyan
University of California Berkeley
Dr. Aditi Krishnapriyan is Assistant Professor with joint appointments in Chemical Engineering and EECS at UC Berkeley. She is a member of Berkeley AI Research (BAIR) and focuses on AI-driven methods to tackle challenges in materials science. Her work blends advanced AI techniques with rigorous mathematical foundations, making her a strong contributor to discussions on multimodal learning in materials science.
Colin Ophus
Stanford University
Dr. Colin Ophus is Associate Professor of Materials Science and Engineering and Center Fellow at the Precourt Institute for Energy. He specializes in advanced microscopy techniques, particularly scanning transmission electron microscopy (STEM) and 4D-STEM. His work enhances imaging and analysis tools, helping bridge experimental data and theoretical models. His expertise in integrating multimodal data from microscopy and simulations is highly relevant to the workshop's themes.
Website
Organizers
Organizers
Weike Ye
Toyota Research Institute
Mathew J. Cherukara
Argonne National Laboratory
Leena Sansguiri
Toyota Research Insitute
Santosh Suram
Toyota Research Institute
Jiarui (Jerry) Zhang
University of Sourthern California
Flora Chen
Toyota Research Institute
Program
Committee
Program
Committee
The success of this workshop would not be possible without the support of our program committee. These individuals generously provided their time and expertise to review submissions and help shape a program of exceptional quality. We are deeply grateful for their contributions to the community.
Technical University of Denmark
Massachusetts Institute of Technology
University of Toronto
Toyota Research Institute
Indian Institute of Technology Delhi
Technical University of Denmark
Schedule
Schedule
1:00 – 1:10 PM Welcome Weike Ye
1:10 – 1:30 PM Keynote 1 Sergei Kalinin
Autonomous Microscopy for Materials and Physics Discovery: Rewards are All We Need
1:30 – 1:55 PM Spotlights
Utkarsh Pratiush, Austin Houston, Gerd Duscher, Sergei Kalinin
stemOrchestrator: Enabling Seamless Hardware Control and Multimodal Workflows on Electron MicroscopesYu (Richard) Liu, Utkarsh Pratiush, Sergei Kalinin
Realizing Fully Autonomous Materials Discovery via ML-Enabled SPM
1:55 – 2:15 PM Keynote 2 Greg Slabaugh
Building Foundations: Towards Multimodal AI for Healthcare and Materials Science
2:15 – 2:50 PM Spotlights
Yuta Suzuki, Tatsunori Taniai, Ryo Igarashi, Kotaro Saito, Naoya Chiba, Yoshitaka Ushiku, Kanta Ono
Bridging Text and Crystal Structures: Literature-Driven Contrastive Learning for Materials ScienceCan Polat, Hasan Kurban, Erchin Serpedin, Mustafa Kurban
Understanding the Capabilities of Molecular Graph Neural Networks in Materials Science Through Multimodal Learning and Physical Context EncodingAdib Bazgir, Rama chandra Praneeth Madugula, Yuwen Zhang
Multicrossmodal Automated Agent for Integrating Diverse Materials Science Data
2:50 – 3:15 PM Panel Discussion
Greg Slabaugh, Tian Xie, Arghya Bhowmik, Linda Hung, Aditi Krishnanpriyan, Colin Ophus
3:15 – 3:40 PM Poster Session & Coffee Break
3:40 – 4:00 PM Keynote 3 Tian Xie
Accelerating materials design with AI emulators and generators
4:00 – 4:55 PM Spotlights
Shehroz Ahmad Shoaib, Burhan SaifAddin
Bridging Atomic and Bond Modalities: Crystal-X Network for Enhanced Bandgap PredictionFang Sun, Zhihao Zhan, Hongyu Guo, Ming Zhang, Jian Tang, Yizhou Sun
Controllable Generation of Drug-like Molecular Materials with Multi-modal Variational FlowLogan Robinson, Pedro Davila, Sergio J. Contreras, Pragathi Durga Rajarajan, Jose Daniel Castillo Rodriguez, Michelle Hope Voges, Cyana Zaragosa, Patrick Henry Warren, Elizabeth S. Sooby, Amanda S. Fernandez
Evaluation and a New Benchmark for Defect Detection in Materials FabricationDaniel Mohanadhas, Logan Robinson, John Wilburn, Kosi Atupulazi, Michelle Hope Voges, Cyana Zaragosa, Patrick Henry Warren, Elizabeth S. Sooby, Amanda S. Fernandez
Multimodal Approaches for Realistic Synthetic Data Generation in Materials FabricationTrevor Bohl, Minasadat Attari, Matthew Maschmann, Filiz Bunyak
CNT-Former: Transformer GAN for Carbon Nanotube Modeling in Frequency Domain
4:55 – 5:00 PM Closing Remarks
The Venue
The Venue
Music City Center
201 Rep. John Lewis Way South
Nashville, TN 37203
615.401.1400
Level 1, Room 110 A
The Music City Center is Nashville's convention center located in the heart of downtown. The 2.1 million square foot facility opened in 2013 and was built so that Nashville could host large, city-wide conventions in the downtown area.
Sponsored by
Page updated
Google Sites
Report abuse