Codebase, summary of results, and datasets.
Federated Learning (FL) aims to infer a shared model from private and decentralized data stored by multiple clients. Personalized FL (PFL) enhances the model’s fit for each client by adapting the global model to the clients. A significant level of personalization is required for highly heterogeneous clients but can be challenging to achieve, especially when clients’ datasets are small.
To address this issue, we introduce the PAC-PFL framework for PFL of probabilistic models. PAC-PFL infers a shared hyper-posterior and treats each client’s posterior inference as the personalization step. Unlike previous PFL algorithms, PAC-PFL does not regularize all personalized models towards a single shared model, thereby greatly enhancing its personalization flexibility.
By establishing and minimizing a PAC-Bayesian generalization bound on the average true loss of clients, PAC-PFL effectively mitigates over-fitting even in data-poor scenarios. Additionally, PAC-PFL provides generalization bounds for new clients joining later. PAC-PFL achieves accurate and well-calibrated predictions, as supported by our experiments.
PAC-PFL is a probabilistic method that yields accurate and well-calibrated predictions, which is essential in safety-critical applications.
PAC-PFL is applicable to scenarios with highly heterogeneous or multimodal clients.
PAC-PFL minimizes a bound on the generalization error, enabling it to learn complex models from small client datasets while mitigating the risk of overfitting.
PAC-PFL allows for the progressive collection of new data over time.
Contact mahrokh.ghoddousiboroujeni@epfl.ch to get more information on the project