Evaluation

Training corpus

Train data is now available

The traing data can be downloaded [here]. Both files, train.xlsx and development.xlsx, are the training data for this competition.

Evaluation rules

Test data is now available

The test data can be downloaded [here].

The test file contains a column of Id, Header, and Text. Note that some instances have an empty header intentionally

The performance of your fake news detection solution will be ranked by the F1 measure on the Fake class.

Runs will be received from 19th April 0:01 until 8th May 23:59 (-0600 UTC)

Participants are allowed to submit up to two runs for each track: one primary and one secondary. The participants must clearly flag each of the two.

CodaLab page for the task

The CodaLab page for the task is now available [here]. On this page, participants can evaluate their proposals in the different phases of the task.

Output Submission

Submissions formatted as described below and sent to the CodaLab page.

​Your software has to output for each task of the dataset a corresponding txt file. The file must contain one line per classified instance. Each line looks like this:

"TaskName"\t"IdentifierOfAnInstance"\t"Class"\n

It's important to respect the format with the " character, \t (tabulator) and \n (linux enter). The naming of the output files is up to you, we recommend to use the author and a run's identifier as filename with "txt" as extension.

For the fake news track the possible labels are:

  • TaskName: fakenews

  • IdentifierOfAnInstance: NumberOfDocument

  • Class: {True, Fake}

  • Output example:

"fakenews" "1" "True"

"fakenews" "2" "Fake"

"fakenews" "3" "Fake"

"fakenews" "4" "True"

"fakenews" "5" "True"

Paper Submission

Participants of the tasks will be given the opportunity to write a paper that describes their system, resources used, results, and analysis that will be part of the official IberLEF-2021 proceedings. System description papers should be formatted according to the Springer Conference Proceedings style. Latex and Word templates can be found there. The minimum length of a regular paper should be 5 pages. There is no maximum page limit.

Papers must be written in English.