Call for papers
Submission Deadline: Feb 5, 2024, 11:59 pm AOE
Submission portal: https://openreview.net/group?id=ICLR.cc/2024/Workshop/BGPT
Submission Deadline: Feb 5, 2024, 11:59 pm AOE
Submission portal: https://openreview.net/group?id=ICLR.cc/2024/Workshop/BGPT
This workshop aims to identify and narrow the gap between learning theory and practice. For this purpose, we welcome submissions that meet one of the following criteria:
Works that highlight a discrepancy between existing theoretical analyses and practice
Works that propose new experimental observations that can help advance our understanding of the underlying mechanisms of deep learning.
Works that offer theoretical insights that match existing experimental observations.
Works that propose algorithms based on theoretical derivations which perform well in practice.
On the other hand, this workshop will not cover works that only prove tighter bounds without offering new insights for practice, works that propose new empirical tricks or use existing ones without providing theoretical insights (even if these tricks achieve SOTA performance), works that neither lead to practical applications nor provide explanations for practical phenomenons.
The detailed topics of this workshop include (but are not limited to) the following topics:
Optimization theory for deep learning. Several subareas may include: Edge of Stability (EoS) phenomenon, adaptive optimizers, non-smoothness of neural network landscape, the role of initialization, architectural design, and optimization tricks in influencing the convergence.
Generalization theory for deep learning. Several subareas may include: the implicit bias of gradient-based optimizers, effects of overparameterization, loss landscape flatness, and more generally, how neural network architectures, data distribution, optimizers, and initialization impact the generalization performance.
Theory of large language models. Several subareas may include: understanding the scaling law and emergence, theory of in-context learning, theory of chain-of-thought, the expressive power of autoregressive Transformers, and more fundamentally, what the key reasons behind the success of large language models are.
Submission Link: https://openreview.net/group?id=ICLR.cc/2024/Workshop/BGPT
Formatting Instructions: To ensure your submission is considered, please adhere to the following guidelines:
Please use the same LaTeX style files as the main conference ICLR 2024. No page limits are put.
The reviewing process will be double-blind and all submissions must be anonymized. Please do not include author names, author affiliations, acknowledgments, or any other identifying information in your submission. Unless accepted, submissions and reviews will not be made public.
Dual Submissions: This workshop is non-archival and will not have official proceedings. Workshop submissions can be submitted to other venues. We welcome ongoing and unpublished work, including papers that are under review at the time of submission. However, we do not accept submissions that have already been accepted for publication in other venues with archival proceedings (including ICLR 2024 main conference). Such submissions will be desk-rejected once noticed.
Reviews: The review process will be double-blind. All submissions must be anonymized and the leakage of any identification information is prohibited.
Paper submission opens: Jan 8, 2024, 12:00 am AOE
Deadline for paper submission: Feb 5, 2024, 11:59 pm AOE
Deadline for review submission: Feb 22, 2024, 11:59 pm AOE
Notification: Mar 3, 2024, 11:59 pm AOE
Workshop: May 11, 2024
The BGPT workshop is non-archival, and should thus generally not violate dual submission policies at other archival venues; if unsure, please check yourself with the corresponding venue.
Contact the organizers: workshopbgpt@gmail.com