Abstract:

Recent advances in vision and language models have significantly improved visual understanding and generation tasks. In this talk, I will present our latest research on designing effective tokenizers for transformers and our efforts to adapt frozen large language models for diverse vision tasks. These tasks include visual classification, video-text retrieval, visual captioning, visual question answering, visual grounding, video generation, stylization, outpainting, and video-to-audio conversion. I will also present the most recent results on generating consistent video and images. 

Biography [detail]

Ming-Hsuan Yang is a Professor at the University of California, Merced, and a Research Scientist at Google DeepMind. He received numerous awards, including the Google Faculty Award (2009), NSF CAREER Award (2012), Nvidia Pioneer Research Award (2017 and 2018), and Sony Faculty Award (2025). He received the Best Paper Honorable Mention at UIST 2017, CVPR 2018, and ACCV 2018, the Longuet-Higgins Prize (Test of Time Paper) at CVPR 2023, Best Paper at ICML 2024, and Test-of-Time award from WACV 2025. Yang is an Associate Editor-in-Chief of IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) and an Associate Editor for the International Journal of Computer Vision (IJCV).  He is a Fellow of IEEE, ACM, and AAAI.

Abstract:

The rapid evolution of generative AI has brought unprecedented challenges to the integrity of social media information ecosystem. While detection methods have achieved notable success on benchmarking datasets, our experience shows that they often struggle in real-world scenarios suffering significant performance drops and mistrust from the practitioners. We analyze the root causes of these issues and overview our recent work to alleviate them.  

Biography [detail]

Siwei Lyu is a SUNY Empire Innovation Professor at the Department of Computer Science and Engineering, the Director of the UB Media Forensic Lab (UB MDFL), and the founding Co-Director of the Center for Information Integrity (CII) at the University at Buffalo, State University of New York, USA.

Before joining UB, Dr. Lyu served as an Assistant Professor (2008-2014), tenured Associate Professor (2014-2019), and Full Professor (2019-2020) at the Department of Computer Science at the University at Albany, State University of New York. He is the Founding Director of UAlbany's Computer Vision and Machine Learning Lab (CVML). From 2005 to 2008, he worked as a Post-Doctoral Research Associate at the Howard Hughes Medical Institute and the Center for Neural Science at New York University. In 2001, he was an Assistant Researcher at Microsoft Research Asia. Dr. Lyu earned his Ph.D. in Computer Science from Dartmouth College in 2005 and both his M.S. (2000) and B.S. (1997) degrees in Computer Science and Information Science, respectively, from Peking University, China.

Dr. Lyu's research interests include media forensics, computer vision, and machine learning. He has published over 190 refereed journal and conference papers. His research projects are funded by NSF, DARPA, and the US Department of Homeland Security. Dr. Lyu has received numerous awards, such as the IEEE Signal Processing Society Best Paper Award (2011), the National Science Foundation CAREER Award (2010), SUNY Albany's Presidential Award for Excellence in Research and Creative Activities (2017), SUNY Chancellor's Award for Excellence in Research and Creative Activities (2018), Google Faculty Research Award (2019), and IEEE Region 1 Technological Innovation (Academic) Award (2021).

Dr. Lyu has served on the IEEE Signal Processing Society's Information Forensics and Security Technical Committee and held editorial positions with several prestigious journals. Dr. Lyu holds prestigious memberships and distinctions, including being a Fellow of IEEE, IAPR, and AAIA. He is also a Distinguished Member of ACM, a Senior Member of the Sigma Xi Society, and a Member of the Omicron Delta Kappa society.

Abstract:

The convergence of vision, language, and generative modeling is driving the next wave of AI applications. In this talk, we explore how recent advances in efficiency and customization are enabling more practical and powerful vision-language models (VLMs). We begin by surveying new distillation techniques that significantly reduce the size and inference cost of VLMs without compromising their multimodal reasoning capabilities. We then examine emerging strategies for model personalization, which allow VLMs to be tailored for specific users, domains, or downstream tasks. Finally, we discuss how these developments are reshaping the landscape of visual reasoning, opening the door to scalable, adaptive, and user-centric multimodal systems.

Biography [detail]

Yu-Chiang Frank Wang received his B.S. degree in Electrical Engineering from National Taiwan University in 2001. He received his M.S. and Ph.D. degrees in Electrical and Computer Engineering from Carnegie Mellon University in 2004 and 2009, respectively. In 2009, Dr. Wang joined the Research Center for Information Technology Innovation (CITI) of Academia Sinica, leading the Multimedia and Machine Learning Lab. Dr. Wang joined the Department of Electrical Engineering at National Taiwan University as an Associate Professor in 2017, and was promoted to Professor in 2019. Since 2022, Dr. Wang joins NVIDIA, where he serves as the Research Director in Deep Learning & Computer Vision and leads NVIDIA Research Taiwan. With continuing research focuses on computer vision and machine learning, Dr. Wang's recent research topics include deep learning for vision & language, transfer learning, and 3D vision. Dr. Wang serves as organizing committee members and area chairs of multiple international conferences such as CVPR, ICCV, ECCV, and ACCV. Several of his papers are nominated for the best paper awards, including IEEE ICIP, ICME, AVSS and MVA. Dr. Wang is twice selected as the Outstanding Young Researcher by the Ministry of Science and Technology of Taiwan (2013-2015 and 2017-2019), as well as the Technological Research Innovation Award from the College of EECS at NTU. In 2022, Dr. Wang receives the Y. Z. Hsu Scientific Paper Award in Artificial Intelligence from the Far Eastern Y. Z. Hsu Science & Technology Memorial Foundation. In 2023, Dr. Wang is recognized as the Outstanding Young Scholar by the Foundation for the Advancement of Outstanding Scholarship.