Research Questions (for now)

RQ1) Learning Challenges in Distributed Heterogeneous Environments: How can we tackle the complexities associated with statistical heterogeneities in collaborative learning environments? Keywords: Federated Learning, Model Merging.

RQ2) Safety and Robustness Challenge: How do we improve the robustness of models against adversarial attacks in collaborative learning environments? Keywords: Trust, Safety, Alignment

RQ3) Efficiency and Sustainability: How can we do more (train, inference) with less (memory, compute)? Keywords: Green AI, Efficient AI

RQ4) Applications: Beyond baselines and benchmarks, how can we solve domain-specific problems with our proposed solutions?

🤓We particularly value works led by our own team members, or works originated at our lab by visiting students. 😆

*denotes our team members (my NTU students/staff)

#denotes our visiting students (work completed during visiting attachment to our team at NTU)

Project Page with Paper + Code (Will Update)

Fan Wu*, Xinyu Yan*, Jiabei Liu*, Wei Yang Bryan Lim. "FedRAM: Federated Reweighting and Aggregation for Multi-Task Learning." NeurIPS 2025.

TL;DR: We introduce a three-stage federated multi-task learning method that leverages a small proxy model to learn task and client weights, enabling efficient and accurate aggregation during fine-tuning of larger models under heterogeneity.

Addresses: RQ1, RQ3

Project Page with Paper + Code (Will Update)

Che Wang#, Ziqi Zhang, Yinggui Wang, Tiantong Wang*, Yurong Hao*, Jianbo Gao, Tao Wei, Cao Yang, Zhong Chen,
Wei Yang Bryan Lim. "AegisGuard: RL-Guided Adapter Tuning for TEE-Based Efficient & Secure On-Device Inference." NeurIPS 2025.

TL;DR: AegisGuard protects only the most model-stealing-sensitive LoRA adapters inside a TEE, compressing them to achieve near black-box stealing resistance while reducing latency, memory footprint, and accuracy loss.

Addresses: RQ2, RQ3

Project Page with Paper + Code

Fuyao Zhang*, Xinyu Yan*, Tiantong Wu*, Wenjie Li#, Tianxiang Chen#, Yang Cao, Ran Yan, Longtao Huang,
Wei Yang Bryan Lim, Qiang Yang. "Oblivionis: A Lightweight Learning and Unlearning Framework for Federated Large Language Models." Preprint 2025.

TL;DR: We introduce Oblivionis, a lightweight framework that integrates federated learning and targeted unlearning for LLMs, to enable privacy-preserving training and compliance with GDPR's right to be forgotten.

Addresses: RQ1, RQ2, RQ3

Project Page with Paper + Code

Jingyuan Zhang*, Yiyang Duan*, Shuaicheng Niu, Yang Cao, and Wei Yang Bryan Lim. "Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning." ICLR 2025.

TL;DR: We introduce a one-round federated domain adaptation approach that replaces averaging-based model aggregation with server-side fine-tuning of a global adapter on each client’s small prototype set, with differential privacy guarantees.

Addresses: RQ1, RQ2

Project Page with Paper + Code

Zhengyi Zhong#, Weidong Bao, Ji Wang, Shuai Zhang, Jingxuan Zhou, Lingjuan Lyu, Wei Yang Bryan Lim. "Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter" CVPR 2025.

TL;DR: We first identify only the most “knowledge-sensitive” layers of a model, then overwrite them via sparsely trained adapters while freezing everything else. This achieves effective forgetting (on par with full retraining) yet remains reversible, costs far fewer resources, and preserves other knowledge better than standard methods.

Addresses: RQ2, RQ3

Project Page with Paper + Code

Wenjie Li#, Kai Fan, Jingyuan Zhang*, Hui Li, Wei Yang Bryan Lim, and Qiang Yang. "Enhancing Security and Privacy in Federated Learning using Low-Dimensional Update Representation and Proximity-Based Defense." TKDE 2025.

TL;DR: We propose FLURP, which compresses each client’s updates into a low-dimensional representation, and design a privacy-preserving, proximity-based rule to filter out poisoned updates. Experiments show that this drastically reduces secure-computation overhead while maintaining high accuracy against various Byzantine attacks.

Addresses: RQ2