MULTI-Fake-DetectiVE

News!

• May 30, 2023. Results are in!

• May 12, 2023. Test Data is available!

• May 5, 2023. Baseline models are available!

• February 10, 2023. Development data is available to download!

• February 10, 2023. Guidelines are available!

• February 7, 2023. The MULTI-Fake-DetectiVE dataset will be available on Friday (February 10th). Stay tuned!

• November 10, 2022. Web site online!

• October 25, 2022. MULTI-Fake-DetectiVE has been accepted at EVALITA 2023!

Task description. MULTImodal Fake News Detection and VErification (MULTI-Fake-DetectiVE)

The task focuses on the automatic detection of fake news in a multimodal setting including texts and images. The goal of the task is twofold: 

• i.) given a piece of content (e.g., a social media post or a news article) that includes both a visual and a textual component, determining its likelihood of being a real or a fake news; 

• ii.) understanding how the visual and textual components of news can influence each other: given a text and an accompanying image, is the combination of the two aimed at misleading the interpretation of the reader about one or the other, or not?

The dataset for the task will include social media posts and news articles, containing both a textual and a visual component, concerning one or more real world events that are known to have been subject to the generation of fake news. In particular, we will focus on the Ukrainian-Russian war started in February 2022. Since the early stages of the war, a number of news and more generally media content emerged that correspond to the definitions of fake news given in Allcot and Gentzkow (2017) and Zubiaga et al. (2015). Gathered data will potentially include fake news and misleading information on the event.

Motivation and Relevance

Recent years have seen a large increase in the online proliferation of disinformation and fake news. This is especially true in the context of real-world events that are reported as breaking news. This phenomenon has further increased with the outbreak of the Russian war against Ukraine. Like in all conflicts, misinformation has become a powerful strategic weapon. This in turn has led to the creation of numerous initiatives for independent fact-checking and fake news detection, and this topic has increased its centrality in the research community.

Despite the potential of other modalities in influencing the spread of disinformation and fake news, multimodality is still rather under-explored in this context (Alam and others, 2022). Although some efforts in this direction have been made (Dimitrov and others, 2021), models combining multiple modalities for detecting fake news remain a major open challenge in the literature, as well as datasets including different modalities and different sources of fake news (Alam and others, 2022). 

The problem of disinformation and fake news detection and verification is gaining attention in the research community especially the NLP one. A number of international shared tasks has been organized both for fake news and fact-checking (Nakov and others, 2022).

A shared task aimed at addressing not only the textual modality, but a visual one as well, may be helpful in broadening the horizon of research on disinformation, as well as to produce new resources for Italian on this topic.

References

• Alam, F., Cresci, S., Chakraborty, T., Silvestri, F., Dimitrov, D., Martino, G. D. S., ... & Nakov, P. (2021). A survey on multimodal disinformation detection. arXiv preprint arXiv:2103.12541. 

• Bondielli, A., & Marcelloni, F. (2019). A survey on fake news and rumour detection techniques. Information Sciences, 497, 38-55. 

• Dimitrov, D., Ali, B. B., Shaar, S., Alam, F., Silvestri, F., Firooz, H., ... & Martino, G. D. S. (2021). SemEval-2021 task 6: detection of persuasion techniques in texts and images. arXiv preprint arXiv:2105.09284. 

• Nakov, P., Barrón-Cedeño, A., da San Martino, G., Alam, F., Struß, J. M., Mandl, T., ... & Köhler, J. (2022). Overview of the clef–2022 checkthat! lab on fighting the covid-19 infodemic and fake news detection. In International Conference of the Cross-Language Evaluation Forum for European Languages (pp. 495-520). Springer, Cham. 

• Passaro, L. C., Bondielli, A., Dell’Oglio, P., Lenci, A., & Marcelloni, F. (2022). In-context annotation of topic-oriented datasets of fake news: A case study on the notre-dame fire event. Information Sciences, 615, 657-677.