call for task proposals

IberLEF 2021: Call for Task Proposals

The goal of IberLEF is to encourage the research community to organize competitive text processing, understanding and generation tasks, with the aim of defining new research challenges and setting new state-of-the-art results for the Natural Language Processing community, involving at least one of the following Iberian languages: Spanish, Portuguese, Catalan, Basque or Galician. IberLEF is co-located with the SEPLN Conference.

Researchers and practitioners from all areas of Natural Language Processing and related communities are invited to submit proposals for evaluation tasks that involve text processing in at least one of the languages spoken in the Iberian Peninsula or the Latin-American region.

Proposals must be submitted (as a pdf file) to iberlef@googlegroups.com, and should include the following fields:

- Title of the task.

- Description of the task, highlighting:

* Relevance and novelty of the task, and the challenges involved.

* Evaluation measures, and other relevant methodological aspects.

* Expected target community, and actual or potential industrial take up.

* Related evaluation activities, if any.

* Previous editions of the task, if any. If it has been organized previously, what is the roadmap and what are the novelties for 2021?

* Linguistic resources to be gathered, created and/or reused. Please include as many details on data gathering, selection and annotation procedures as possible: sources and representativity, training/validation/test sizes, harvesting procedures, profile of annotators (experts, linguists, crowdworkers, etc.), multiple annotation policy, IPR issues, baseline, etc.

* Tentative schedule (note that camera ready versions of the proceedings must be ready by July 5, 2021).

- Organization committee: full name and affiliation of the organizers, with a succinct description of their research interests, areas of expertise and experience organizing similar events.

- Funding, if available.

- Contact person.

- Any other relevant issues.

Task organizers duties

Note that organizers of accepted tasks are expected to:

- Setup the evaluation exercise according to the submitted proposal.

- Promote the task within the target research community.

- Manage the submission and scientific evaluation of the system description papers of the corresponding systems submitted by the participants. The accepted papers will be published in the IberLEF proceedings.

- Prepare and submit an overview of the evaluation exercise.

- Attend (at least one organizer) the IberLEF 2021 workshop and present the results of the task.

Task selection procedure

Each submitted proposal will be reviewed by members of the the IberLEF steering and program committee, and decisions will be sent back to the task organizers by December 23, 2020.

Proceedings

IberLEF 2021 Proceedings including the description of the participating systems will be published at CEUR-WS.org. Task descriptions and overviews will be published in the journal Natural Language Processing (http://www.sepln.org/en/journal, in Clarivate ESCI and Elsevier SJR) in its September 2021 issue. Task Organizers are expected to send the camera ready task and system description papers for their task to IberLEF organizers by July 5, 2021.

Important dates

- Task proposals due: December 1, 2020.

- Notification of acceptance: December 23, 2020.

- Camera ready submissions due: July 5, 2021.

- IberLEF Workshop: September 2021.

IberLEF general chairs:

Julio Gonzalo, UNED (Spain)

Manuel Montes y Gómez, INAOE (Mexico)

Paolo Rosso, Universitat Politècnica de València (Spain)

Contact

E-mail: iberlef@googlegroups.com