Call for Paper
Call for Paper
Introduction
In recent years, pre-trained language models (PLM) such as BERT, GPT-2, or T5 have shown significant improvement in various NLP tasks. Trained on massive raw text available on the internet, PLMs can effectively represent human knowledge encoded in textual format. As such, the application in the field of NLP is deemed to be increased. However, despite all these achievements, there still are significant concerns and challenges associated with training, applications, and consequences of PLMs in the NLP and AI community and more broadly in society. These concerns and challenges range from technical issues due to the ever-increasing size of PLMs and reformulating traditional NLP tasks to be solved by generative language models to ethical concerns due to the reflected human biases in pre-trained language models on text written by humans. As such, it is necessary to summon researchers from different communities to discuss what has been solved, what challenges remain, and what directions seem to be more promising for the future of this field. To address these questions, LPLM is the first workshop on advances, applications, challenges, and concerns of large-scale pre-trained language models.
Topics of Interest
LPLM invites a diverse group of researchers from artificial intelligence, natural language processing, cognitive science, linguistics, system engineering, and environmental research to present their research and discuss the remaining challenges on topics included but not limited to:
Training and evaluating domain-specific language models
Constructing challenging tasks and benchmarks for pre-trained language models
Multilingual and multi-modal PLM
Eliciting knowledge from and constructing knowledge graphs using PLMs
PLMs as few-shot or zero-shot learners
Reformulating non-generative tasks as a generative problem
Evaluation of model generated texts
Meta-learning for pre-trained language models
The computation cost of pre-training large transformer-based language models
Correlation of performance and corpus size in pre-training stage
Biases, such as gender, age, and ethnicity, encoded in PLMs
Environmental effect of the pre-training stage of PLMs
Application of PLMs for social good
Submission
Authors are invited to submit their unpublished work that represents novel research. The papers should be written in English using the NAACL 2022 templates and follow the NAACL 2022 formatting guidelines. Authors can also submit supplementary materials, including technical appendices, source codes, datasets, and multimedia appendices. All submissions, including the main paper and its supplementary materials, should be fully anonymized. For more information on formatting and anonymity guidelines, please refer to NAACL 2022 call for paper page.
All papers will be double-blind peer-reviewed. LPLM accepts papers in two tracks:
Regular Workshop Paper: Novel papers that are not published can be submitted as regular workshop papers. In this track, LPLM, accepts both short (up to 4 pages + unlimited references) and long (up to 8 pages + unlimited references) papers. Upon acceptance, the authors are provided with 1 more page to address the reviewer's comments.
Extended Abstract: Preliminary but interesting ideas or results that have not been published before may be submitted as extended abstracts. We can also accept a small number of published papers with interesting and relevant ideas as extended abstracts for the LPLM workshop. Extended abstracts can be short (up to 4 pages + unlimited references) or long (up to 8 pages + unlimited references) papers.
Three reviewers with the same technical expertise will review each paper. Authors of the accepted papers will present their work in either the Oral or Poster session. Only regular workshop papers will appear in the workshop proceedings.
At least one author of each accepted paper should register at the conference and present the work at the workshop.
Submission should be done electronically in PDF format via START. LPLM will not accept any submission from other mechanisms such as Email.
Paper submission deadline: May 27, 2022
Notification date: June 15, 2022
Camera-ready submissions due: June 20, 2022
LPLM workshop at NAACL 2022: July 15, 2022
All deadlines are “anywhere on earth” (UTC-12)