Tutorial: Heterogeneity in Federated Learning

SDM 2024

Time: 3:15 PM - 5:15 PM (CDT), April 19, 2024

Location: Room: Hibiscus 3, Westin Houston, Houston, TX, U.S.

Slides: Link

Description:

Federated learning is a distributed machine learning paradigm, which enables multiple participants to cooperate in training machine learning models without sharing data. Heterogeneity is one of the main challenges in federated learning. To solve this challenge, in this tutorial, we will cover the state-of-the-art federated learning techniques to handle the heterogeneity issue. In particular, we focus on the following three aspects: (1) providing a comprehensive review of heterogeneity challenges in federated learning from three perspectives, including data heterogeneity, model heterogeneity, and system heterogeneity; (2) introducing cutting-edge techniques to solve the heterogeneity issue in federated learning from both algorithm and application perspectives; and (3) identifying open challenges and proposing convincing future research directions in heterogeneous federated learning. We believe this is an emerging and potentially high-impact topic in distributed machine learning, which will attract both researchers and practitioners from academia and industry.

Outline:

Part 1: Introduction (20 mins)
- Federated learning basics
- Heterogeneity challenges in federated learning
- Applications in federated learning
Part 2: Data Heterogeneity (30 mins)
- Label and feature skew
- Quantity skew
- Modality skew
Part 3: Model Heterogeneity (30 mins)
- Partial heterogeneity
- Complete heterogeneity
Part 4: System Heterogeneity (30 mins)
- Participant presence and absence
- Physical status heterogeneity
Part 5: Conclusion and Future Work (10 mins)
- Summary and discussion
- Future work

Presenters:

Jiaqi Wang is currently a Ph.D. candidate from the PSU Data Science Lab in the College of Information Sciences and Technology at the Pennsylvania State University. He received his B.E. and M.S. from Zhejiang University and the University of Georgia. His research interests are federated learning and healthcare informatics. More specifically, he is working on exploring heterogeneity in federated learning, personalized federated learning, and federated learning applications in healthcare. His research has been published in conferences and journals such as NeurIPS, SDM, ECML-PKDD, BigData, and Rare Disease and Orphan Drugs Journal. He has also served as a reviewer or committee member in major machine learning and data mining conferences such as EMNLP, KDD, ACL, and AAAI. More information can be found via the link: https://jackqqwang.github.io.

Dr. Fenglong Ma is currently an Assistant Professor in the College of Information Science and Technology at the Pennsylvania State University (PSU), leading the PSU Data Science Lab. He received his Ph.D. from the Department of Computer Science and Engineering, University at Buffalo (UB) in 2019, and subsequently joined PSU. His research interests lie in data mining and machine learning, with an emphasis on mining health-related data. His research interests also include federated learning, multimodal learning, health informatics, natural language processing, and security. He has publications in top conferences and journals such as KDD, NeurIPS, WWW, AAAI, IJCAI, ACL, EMNLP, CIKM, WSDM, ICDM, SDM, and TKDE. He was honored to be recognized as the awardee of the NSF CAREER Award, Sony Research Award, and UB CSE 2019 Best Ph.D. Dissertation Award. He was also recognized as AI 2000 Most Influential Scholar Honorable Mention in Data Mining (2022 and 2023) and 2022 Global Top 50 Chinese Rising Stars in Data Mining. More information can be found at his website: https://fenglong-ma.github.io.

Page updated

Google Sites

Report abuse