Federated Learning Research Track

Overview

This page is for students who want to study from deep learning fundamentals to federated learning (FL) and eventually work on research topics such as:

• Federated Learning + Domain Shift / Domain Generalization

• Federated Learning + Personalization

• Federated Learning + Incremental / Continual Learning / Concept Drift

The page is designed for self-study. Students are recommended to read the papers, run small-scale experiments, and contact me after they have built enough depth.

How to use this page

1. 1. Start from Core FL.

   2. Choose one track among Domain Shift / Personalization / Incremental.

   3. Read papers in order.

   4. Reproduce at least one baseline using the provided code.

What I am looking for

I am looking for students who can gradually become capable of writing papers on FL-related topics.

Good candidates usually:

• 1. know basic PyTorch training

  2. can read experiments carefully

  3. can compare methods instead of only summarizing them

  4. can reproduce at least one baseline

  5. can propose a small but concrete research question

Part I. What I recommend students do first

Option 1. Easiest and most practical path

1. 1. FedAvg

   2. FedProx

   3. SCAFFOLD

   4. Read the DG survey

   5. Read FedDG-GA

   6. Reproduce FedDG-GA or CCST

   7. Then move to SGPT or FedPAC

Option 2. Best path for publishable ideas

1. 1. FedAvg

   2. FedProx

   3. FedDG-GA

   4. Federated Class-Incremental Learning (CVPR 2022)

   5. TARGET (ICCV 2023)

   6. Choose one among SGPT / LANDER / LoRM

Option 3. Best path for students interested in current trends

1. 1. SGPT

   2. FedPAC

   3. pFedMoAP

   4. LANDER

   5. LoRM

Part II. Suggested first mini-projects

Students should not begin with a huge project. Start with a small project that can realistically be finished.

Project idea A — domain shift

Compare:

• 1. FedAvg

  2. FedDG-GA

  3. CCST

on a small multi-domain benchmark. Then ask whether a simple client-specific module improves unseen-domain performance.

Project idea B — personalization

Compare:

• FedAvg

• FedPAC

• SGPT

on small image datasets under non-IID partition. Then test whether a simple prompt/local-head design improves personalization.

Project idea C — continual / drift

Compare:

• Federated Class-Incremental Learning (CVPR 2022)

• TARGET (ICCV 2023)

• LANDER (CVPR 2024)

under a small class-incremental setting. Then study forgetting vs communication vs privacy constraints.

Part III. What to avoid at the beginning

Do not begin with:

• huge foundation-model fine-tuning

• very large medical datasets

• methods that are elegant but highly optimization-heavy

• papers with no public code unless you are already experienced

• projects requiring large-scale distributed systems

The goal is to build depth, not to fail because the setup is too large.

Part IV. When to contact me

Contact me after you satisfy some of the following.

• 1. I understand FedAvg / FedProx / SCAFFOLD at a conceptual level.

  2. I selected one track among personalization / domain / incremental.

  3. I read at least 3 papers in that track.

  4. I ran at least one public codebase successfully.

  5. I can explain the strengths and weaknesses of two methods.

  6. I can suggest one concrete research question.

  7. I prepared a 1–2 page memo.

Part V. Core FL

These papers are the minimum background. Read them first.

1. FedAvg (AISTATS 2017)

• Paper: Communication-Efficient Learning of Deep Networks from Decentralized Data

• PDF: https://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf

• Why read it: the starting point of modern FL

• Focus on: local update, server aggregation, communication rounds, IID vs non-IID intuition

2. FedProx (MLSys 2020)

• • Paper: Federated Optimization in Heterogeneous Networks

  • PDF: https://proceedings.mlsys.org/paper_files/paper/2020/file/1f5fe83998a09396ebe6477d9475ba0c-Paper.pdf

  • Why read it: one of the standard baselines for heterogeneity

  • Focus on: client drift, proximal term, robustness under heterogeneity

3. SCAFFOLD (ICML 2020)

• Paper: SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

• PDF: https://proceedings.mlr.press/v119/karimireddy20a/karimireddy20a.pdf

• Why read it: important for understanding variance / drift correction

• Focus on: why local updates drift away, and how control variates help

Part VI. Track A — FL under Domain Shift / Domain Generalization

This track is recommended before personalization. Students often understand FL much more naturally once they first build intuition about distribution shift in the centralized setting.

Typical question:

If different clients correspond to different domains, what kind of model should we learn, and what should generalize across domains?

Why start with this track

This track helps students build intuition for:

• what domain shift is

• why a model can fail even when training accuracy is high

• why federated learning becomes harder when each client behaves like a different domain

It is also close to realistic deployment settings:

• • hospital-to-hospital shift

  • device shift

  • environment shift

  • unseen-client generalization

Recommended order

A0. First build intuition in the centralized setting

Before jumping into FL-domain papers, start with one tutorial/survey and one simple domain-generalization paper.

• Paper: Domain Generalization: A Survey

• Why read it: broad overview of what domain generalization means

• Focus on: domain shift, train/test domain mismatch, invariant features, and evaluation protocols

A1. FedDG-GA (CVPR 2023)

• Paper: Federated Domain Generalization with Generalization Adjustment

• Code: https://github.com/MediaBrain-SJTU/FedDG-GA

• Why read it: strong CVPR paper with code and clear problem setting

• Good for students because: experimental and extensible, without being too optimization-heavy

A2. CCST (WACV 2023)

• Paper: Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer

• Code: https://github.com/JeremyCJM/CCST

• Why read it: style-transfer intuition makes the domain issue easy to visualize

• Good for students because: algorithmic idea is easier to understand than more theory-heavy papers

A3. FDG Survey (Proceedings of the IEEE 2025)

• Paper: Federated Domain Generalization: A Survey
  Why read it: use after reading 2–3 concrete papers above

• Purpose: organize the field after building intuition from simpler papers

What students should reproduce first in this track

Recommended progression:

1. Read the DG survey first to understand the basic problem setting.

2. Read FedDG-GA carefully and understand its problem setting and evaluation protocol.

3. Reproduce either FedDG-GA or CCST.

4. Use the other one as a comparison or extension reference.

Good starter datasets for this track

• Digits-style multi-domain benchmarks — small and easy to start with

• PACS — a standard DG benchmark with manageable scale if the protocol is kept small

• Office-Home — useful later, but better after the student becomes comfortable with DG/FDG settings

Avoid very large domain benchmarks at the beginning.

Part VII. Track B — Personalized Federated Learning

Once students understand that different clients may correspond to different domains or data distributions, personalization becomes much easier to motivate.

Typical question:

Why is a single global model insufficient when each client has a different data distribution?

Why this track is good

This track is suitable for students who want:

• a clear research motivation

• relatively intuitive algorithmic ideas

• manageable experiments on small/medium datasets

• a good path toward publishable extensions

Recommended order

B1. SGPT (CVPR 2024)

• Paper: Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning

• Code: https://github.com/ubc-tea/SGPT

• Why read it: reflects a recent and promising trend

• Main idea: prompt tuning to bridge global generalization and local personalization

• Good for students because: trendy and algorithmically intuitive

B2. FedPAC (ICLR 2023) — strong recommendation

• Paper: Personalized Federated Learning with Feature Alignment and Classifier Collaboration

• Code: https://github.com/JianXu95/FedPAC

• Why read it: one of the best entry papers for modern personalization

• Main idea: shared representation learning + client classifier collaboration

• Good for students because: strong experiments, intuitive algorithmic story, and not excessively mathematical

B3. pFedMoAP (ICLR 2025) — advanced trend

• Paper: Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models

• Code: https://github.com/chenxingqiang/PFLoRA-lib

• Why read it: useful for students interested in parameter-efficient or prompt-based personalization

• Note: more advanced than the papers above

What students should reproduce first in this track

Choose one:

• SGPT

• FedPAC

Then use one advanced paper as follow-up reading:

• pFedMoAP

Good starter datasets for this track

• CIFAR-10

• CIFAR-100

• MNIST / Fashion-MNIST / EMNIST

• Tiny-ImageNet (only if resources allow)

Avoid very large-scale setups in the beginning.

Part VIII. Track C — FL + Incremental / Continual Learning / Concept Drift

This track is highly promising for publishable work. It combines temporal shift with client heterogeneity.

Typical questions:

• How do we avoid forgetting in federated continual learning?

• How do we handle distributed concept drift?

• How can we learn new classes without storing old private data?

Why this track is attractive

• strong current relevance

• practically meaningful

• good room for extensions

• many problems are still open

Recommended order

C1. Federated Class-Incremental Learning (CVPR 2022)

• Paper: Federated Class-Incremental Learning

• Code: https://github.com/conditionWang/FCIL

• Why read it: one of the clearest starting points for federated class-incremental learning

• Main idea: global-local forgetting compensation

• Good for students because: the problem setting is easy to understand and the algorithmic story is clear

C2. TARGET (ICCV 2023)

• Paper: TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation

• Code: https://github.com/zj-jayzhang/Federated-Class-Continual-Learning

• Why read it: a strong ICCV paper that is easier to motivate than more optimization-heavy methods

• Main idea: exemplar-free distillation with synthetic data generation

• Good for students because: intuitive continual-learning story without requiring stored old real data

C3. LANDER (CVPR 2024)

• Paper: Text-Enhanced Data-free Approach for Federated Class-Incremental Learning

• Code: https://github.com/tmtuan1307/lander

• Why read it: a recent CVPR paper with a very concrete algorithmic idea

• Main idea: use label text embeddings as anchors for data-free knowledge transfer

• Good for students because: modern, visual, and easier to explain than more mathematical formulations

C4. LoRM (ICLR 2025)

• Paper: Closed-Form Merging of Parameter-Efficient Modules for Federated Continual Learning

• Code: https://github.com/aimagelab/fed-mammoth

• Why read it: useful if you want to explore parameter-efficient continual FL

• Note: more advanced than FedCCFA / Powder

C5. FedCCFA (NeurIPS 2024) — optional extension

• Paper: Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Drift

• Code: https://github.com/Chen-Junbao/FedCCFA

• Why read it: useful if the student wants to move from class-incremental FL to concept-drift-aware FL

• Main idea: classifier clustering + feature alignment under drift

• Good for students because: experimentally strong and still fairly intuitive

What students should reproduce first in this track

Choose one:

• Federated Class-Incremental Learning (CVPR 2022)

• TARGET (ICCV 2023)

Then move to:

• LANDER (CVPR 2024)

Use FedCCFA as follow-up reading for extension ideas.

Good starter datasets for this track

• Split CIFAR-10 / Split CIFAR-100

• class-incremental image classification protocols

• small concept-drift simulation settings

Avoid large-scale continual benchmarks at the beginning.

Part IX. Recommended code / benchmark libraries

These libraries are useful because students should not waste too much time building everything from scratch.

1. PFLlib (strongly recommended for personalization)

• Website: https://www.pfllib.com/

• GitHub: https://github.com/TsingZ0/PFLlib

• Why use it: a good starting point for personalized FL experiments and small-scale benchmarking

2. DomainBed (recommended for domain-shift intuition)

• GitHub: https://github.com/facebookresearch/DomainBed

• Why use it: a standard benchmark codebase for understanding centralized domain generalization before moving to FL-domain papers

3. FCIL official code

• GitHub: https://github.com/conditionWang/FCIL

• Why use it: a practical starting point for federated class-incremental learning

4. TARGET official code

• GitHub: https://github.com/zj-jayzhang/Federated-Class-Continual-Learning

• Why use it: a good reference implementation for exemplar-free continual FL

5. Awesome Personalized FL list

• GitHub: https://github.com/boyi-liu/Awesome-Personalized-Federated-Learning

• Why use it: paper navigation after finishing the core list

6. Awesome-FL

• GitHub: https://github.com/youngfish42/Awesome-FL

• Why use it: broad overview of tools, papers, tutorials, and datasets