The dataset will consist of English, Hindi, and Bengali claims collected from major news websites such as India TV, Aaj Tak, Sangbad Pratidin, Bartaman, etc. After collecting raw claims, we hired 10 annotators proficient in reading, writing, and understanding English, Hindi, Bengali, and CodeMixed. The annotators were asked to collect a concise piece of evidence of no more than 150 words from trustworthy sources. To enhance diversity and difficulty, in addition to manual evidence, we provide two LLM-curated pieces of evidence: one from GPT-4.1-mini and another from Llama-4. Next, we asked the annotators to select the best evidence from the three pieces. Followed by this, select a veracity label as supported or refuted based on the claim and the information in the best evidence. For two distinct subtasks, we followed two dataset design principles:
Subtask 1 Dataset:
Claims paired with the best evidence
A pool of all evidence (both LLM-generated and human-curated)
Veracity labels (supports/ refutes)
Subtask 2 Dataset:
Claims only.
Veracity labels (supports/ refutes)
The dataset will be made available only to the registered participants through their registered Email ID.
Subtask 1 Dataset:
Training and Development Data: ~4k instances and ~1k instances for the training and development phase.
Test Data: Will be made available later.
Subtask 2 Dataset:
Development Data: 500 instances with a claim and its corresponding veracity label (SUPPORTS/ REFUTES).
Test Data: Will be made available later.
Data will be shared via email after registration. Upon registering for this competition, every participant is understood to have agreed to use the data only for non-profit academic and research purposes.
Note: Participants may use additional training data to develop their models/frameworks. However, the additional data must be shared with the organizers after the shared task.