Federated Learning One World Seminar

Abstract: Federated Learning (FL) is transforming how we train machine learning models across decentralized devices while keeping data private. Yet, FL faces tough challenges, from privacy leaks to vulnerabilities like poisoning attacks that can threaten model integrity. Many current defenses rely on external datasets or prior assumptions, which limit scalability and real-world application. In this talk, we will explore these issues and present a new privacy-preserving defense framework, powered by a Conditional GAN (cGAN), that generates synthetic data at the server to verify client updates, eliminating the need for external data. Our approach is designed to be scalable and easily integrated into FL frameworks such as FEDn and others, making it practical for real-world deployments. We will walk through how it compares to popular defenses like Krum and Multi-Krum, and how it performs across challenging attack scenarios. Beyond that, we will open up exciting research opportunities, especially for students, showing where the next breakthroughs in federated learning might come from. Whether you are just starting out or already deep into FL research, this talk aims to inspire ideas that push the field forward.

Relevant Papers:

• Usama Zafar, André Teixeira, Salman Toor. Robust Federated Learning Against Poisoning Attacks: A GAN-Based Defense Framework, arXiv 2025.

• Morgan Ekmefjord, Addi Ait-Mlouk, Sadi Alawadi, Mattias Åkesson, Prashant Singh, Ola Spjuth. Scalable federated machine learning with FEDn, 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), 2022.

Abstract: Asynchronous Stochastic Gradient Descent (Asynchronous SGD) is a cornerstone method for parallelizing learning in distributed machine learning. However, its performance suffers under arbitrarily heterogeneous computation times across workers, leading to suboptimal time complexity and inefficiency as the number of workers scales. While several Asynchronous SGD variants have been proposed, recent findings by Tyurin & Richtárik (NeurIPS 2023) reveal that none achieve optimal time complexity, leaving a significant gap in the literature. In this paper, we propose Ringmaster ASGD, a novel Asynchronous SGD method designed to address these limitations and tame the inherent challenges of Asynchronous SGD. We establish, through rigorous theoretical analysis, that Ringmaster ASGD achieves optimal time complexity under arbitrarily heterogeneous and dynamically fluctuating worker computation times. This makes it the first Asynchronous SGD method to meet the theoretical lower bounds for time complexity in such scenarios.

Relevant Papers:

• Alexander Tyurin. Tight Time Complexities in Parallel Stochastic Optimization with Arbitrary Computation Dynamics, ICLR 2025.

Abstract: Asynchronous Stochastic Gradient Descent (Asynchronous SGD) is a cornerstone method for parallelizing learning in distributed machine learning. However, its performance suffers under arbitrarily heterogeneous computation times across workers, leading to suboptimal time complexity and inefficiency as the number of workers scales. While several Asynchronous SGD variants have been proposed, recent findings by Tyurin & Richtárik (NeurIPS 2023) reveal that none achieve optimal time complexity, leaving a significant gap in the literature. In this paper, we propose Ringmaster ASGD, a novel Asynchronous SGD method designed to address these limitations and tame the inherent challenges of Asynchronous SGD. We establish, through rigorous theoretical analysis, that Ringmaster ASGD achieves optimal time complexity under arbitrarily heterogeneous and dynamically fluctuating worker computation times. This makes it the first Asynchronous SGD method to meet the theoretical lower bounds for time complexity in such scenarios.

Relevant Papers:

• Artavazd Maranjyan, Alexander Tyurin, Peter Richtárik. Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity, arXiv 2025.

Abstract: In this paper, we investigate the impact of compression on stochastic gradient algorithms for machine learning, a technique widely used in distributed and federated learning. We underline differences in terms of convergence rates between several unbiased compression operators, that all satisfy the same condition on their variance, thus going beyond the classical worst-case analysis.

To do so, we focus on the case of least-squares regression (LSR) and analyze a general stochastic approximation algorithm for minimizing quadratic functions relying on a random field. We consider weak assumptions on the random field, tailored to the analysis (specifically, expected Hölder regularity), and on the noise covariance, enabling the analysis of various randomizing mechanisms, including compression. We then extend our results to the case of federated learning.

Relevant Papers:

• Compressed and distributed least-squares regression: convergence rates with applications to Federated Learning, JMLR 2024, C. Philippenko and A. Dieuleveut.