Call for Papers
Important dates
Important dates
Submission Opens: 6th of May
Submission Opens: 6th of May
Submission Deadline: May 31, 2024 (Anywhere on Earth)
Submission Deadline: May 31, 2024 (Anywhere on Earth)
Acceptance Notification: June 17, 2024 (Anywhere on Earth)
Acceptance Notification: June 17, 2024 (Anywhere on Earth)
Workshop Date: July 26, 2024
Workshop Date: July 26, 2024
The workshop will be held in person at ICML 2024 in Vienna, Austria, on July 26, 2024.
The workshop will be held in person at ICML 2024 in Vienna, Austria, on July 26, 2024.
Topics
Topics
We accept submissions on a diverse range of topics, including, but not limited to
We accept submissions on a diverse range of topics, including, but not limited to
• Memory: How to effectively discover or model long-range correlations? How to deal with long context? What types of memory behavior can these models exhibit?
• Memory: How to effectively discover or model long-range correlations? How to deal with long context? What types of memory behavior can these models exhibit?
• Theory: What are the limitations of current architectures? How can we understand the emerging properties of language models?
• Theory: What are the limitations of current architectures? How can we understand the emerging properties of language models?
• Reasoning: Can we better understand and improve in-context learning and the chain of thought? Can current model reason or execute algorithms?
• Reasoning: Can we better understand and improve in-context learning and the chain of thought? Can current model reason or execute algorithms?
• Generalization: How does the sequence model generalize to different lengths and tasks? How robust are these models? What are different types of OOD generalization we should study, and how does generalization interact with memory or context?
• Generalization: How does the sequence model generalize to different lengths and tasks? How robust are these models? What are different types of OOD generalization we should study, and how does generalization interact with memory or context?
• Improving architectures: Some of the recent studies that would fall in this category are, for example, mixture of expert models such as Mixtral or hardware-aware architecture designs like FashAttention.
• Improving architectures: Some of the recent studies that would fall in this category are, for example, mixture of expert models such as Mixtral or hardware-aware architecture designs like FashAttention.
• Recurrent neural networks and state-space models: Some recent examples are Mamba, Griffin, Hawk, LRU, S4D, H3, etc.
• Recurrent neural networks and state-space models: Some recent examples are Mamba, Griffin, Hawk, LRU, S4D, H3, etc.
• Scaling studies: Can we improve our understanding of scaling properties for different foundational models?
• Scaling studies: Can we improve our understanding of scaling properties for different foundational models?
• Data-centric approaches to improve the performance of existing models such as data deduplication, diversification and curriculum.
• Data-centric approaches to improve the performance of existing models such as data deduplication, diversification and curriculum.
• Downstream applications, such as language modeling, vision, biological data, and beyond.
• Downstream applications, such as language modeling, vision, biological data, and beyond.
How to submit
How to submit
We expect submissions to present original research. Submitted manuscripts should be composed of a main body, which can be up to four pages long, followed by an unlimited number of pages for references and appendices, all in a single file. All submissions must be anonymous and should not include any information that violates the double-blind review process, including citing authors' prior work or sharing links in a way that can reveal the identities of authors to potential reviewers. Submissions that do not conform to these instructions may be desk-rejected at the Program Committee's discretion to ensure a fair review process for all potential authors. After submission and during the review period, authors can post their papers as technical reports on arXiv, or other public forums. For details concerning the format of the papers, please see the LaTeX style files.
We expect submissions to present original research. Submitted manuscripts should be composed of a main body, which can be up to four pages long, followed by an unlimited number of pages for references and appendices, all in a single file. All submissions must be anonymous and should not include any information that violates the double-blind review process, including citing authors' prior work or sharing links in a way that can reveal the identities of authors to potential reviewers. Submissions that do not conform to these instructions may be desk-rejected at the Program Committee's discretion to ensure a fair review process for all potential authors. After submission and during the review period, authors can post their papers as technical reports on arXiv, or other public forums. For details concerning the format of the papers, please see the LaTeX style files.