Localized Learning Workshop

Decentralized Model Updates via Non-Global Objectives

See https://icml.cc/virtual/2023/workshop/21484 for the content and videos of the workshop!

---

TL;DR

Topics: Any aspect of localized learning broadly defined.

Paper submission deadline: Wednesday, May 24, 2023 EXTENDED TO Monday, May 29, 2023 (11:59pm AOE)

Format: 4 page papers in ICML format or already accepted papers in original format.

Awards: Outstanding contributed papers (2-3) will receive a free workshop registration.

Submission site: OpenReview

Camera-ready: Camera-ready icml2023.sty file (up to 8 pages of content excluding references and appendices)

Overview

Despite being widely used, global end-to-end learning has several key limitations. It requires centralized computation, making it feasible only on a single device or a carefully synchronized cluster. This restricts its use on unreliable or resource-constrained devices, such as commodity hardware clusters or edge computing networks. As the model size increases, synchronized training across devices will impact all types of parallelism.

Global learning also requires a large memory footprint, which is costly and limits the learning capability of single devices. Moreover, end-to-end learning updates have high latency, which may prevent their use in real-time applications such as learning on streaming video.

Finally, global backpropagation is thought to be biologically implausible, as biological synapses update in a local and asynchronous manner. To overcome these limitations, this workshop will delve into the fundamentals of localized learning, which is broadly defined as any training method that updates model parts through non-global objectives.

Topics

Relevant topics include but are not limited to:

Forward-forward learning
Greedy training
Decoupled or early-exit training
Iterative layer-wise learning
Asynchronous model update methods
Biologically plausible methods for local learning
Localized learning on edge devices
Self-learning or data-dependent functions
New applications of localized learning

Submission Instructions

We welcome submissions of original research papers, work-in-progress reports, and position papers on any aspect of localized learning. The submissions should be in anonymous ICML format with a maximum length of 4 pages with unlimited pages for references and supplementary material. Previously accepted papers (including ones accepted to the ICML main conference) are allowed and can be submitted in their original format. Outstanding papers will be selected for short oral presentations, while all accepted papers will be presented as posters. At least one author from each accepted paper is expected to attend the workshop to present their paper in person.

Notification of acceptance: Monday, June 19, 2023