DIPROMATS 2024

Automatic Detection and Characterization of Propaganda Techniques and Narratives from Diplomats of Major Powers

Shared task at IberLEF 2024

Welcome to DIPROMATS, the shared task on propaganda and narrative detection at the 2024 Iberian Languages Evaluation Forum  (IberLEF)


This NLP challenge asks participants to develop systems able to detect and characterize propagandistic techniques and strategic narratives in tweets written by diplomats and authorities from USA, Europe, Russia and China in English and Spanish

Call for participation

Following the success of the inaugural edition in IberLEF 2023, DIPROMATS 2024 extends its challenge to achieve a more comprehensive diagnosis of propaganda. In addition to examining propaganda techniques, this edition addresses the big picture: the narratives that underlie persuasion efforts.    


Propaganda bypasses rational thought through techniques that appeal to emotions and prejudices, facilitating propagation. Propagandistic endeavors are essential components of larger campaigns to influence audiences' perceptions and political beliefs. These campaigns are built upon carefully crafted narratives that incorporate plausible suggestions, half-truths and manipulative assertions. In this context, tweets serve as brief narrative snippets, progressively piecing together coherent storylines that offer a plausible way to understand and interpret events. 


The deceptive intent of propaganda and narratives may be more subtle and devious than disinformation. Their content does not have to be false, and their effects may only become discernible through systematic observation over time. DIPROMATS 2024 has a two-fold goal. Firstly, by detecting propaganda techniques, the task aims to identify hostile, misleading, and emotionally-driven claims. Secondly, by detecting narratives, it seeks to enhance the ability to uncover disinformation and propaganda campaigns on a broader scale, capturing agents, messages, and plots that contribute to shaping perceptions. 


The provided task corpus comprises tweets in Spanish and English from diplomats representing four international actors: China, Russia, the United States, and the European Union. These authorities include government accounts, embassies, ambassadors, and other diplomatic profiles such as consuls and missions.   

DIPROMATS 2024 invites participants to categorize tweets based on two tasks. Participants have the flexibility to choose whether to engage in both of them, select a specific task, or focus on individual subtasks.

 

Task 1: propaganda identification and characterization. 


Compared to last edition, DIPROMATS 2024 proposes a more balanced distribution of techniques by simplifying the typology. 


Subtask 1a:  Propaganda identification. Systems must decide whether a given tweet contains propaganda techniques (binary classification problem)


Subtask 1b: Propaganda characterization, coarse-grained. Systems must decide, for each tweet, which of the four available categories it fits: Not propagandistic, Appeal to commonality, Discrediting the opponent, Loaded language (Multiclass, multilabel clasification task)


Subtask 1c: Propaganda characterization, fine grained. Systems must classify the message according to the type of technique it contains. There are a negative class and seven positive classes: Flag Waving, Ad Populum / Ad antiquitatem, Name Calling/Labelling, Undiplomatic Assertiveness / Whataboutism, Appeal to Fear, Doubt, and Loaded Language.  


Task 2: narrative detection.


The second task is a multiclass multilabel classification problem. Given a series of predefined narratives of each international actor, systems must determine which narrative the tweets belong to. Systems will receive the description of each narrative and a few examples of tweets in both languages (English and Spanish) that belong to each of them (few-shot). A tweet may be associated with one, several or none of the narratives. In total, for each actor there are six types of narratives, plus a negative class (the typology can be consulted here).


We encourage participation from both academic institutions and industrial organizations. To participate in the task, please fill the registration form at https://sites.google.com/view/dipromats2024/registration 


Important Dates: 


Organizers:  

Pablo Moral, Guillermo Marco,  Julio Gonzalo,  Anselmo Peñas and Jesús Fraile. Universidad Nacional de Educación a Distancia (UNED)


Contact: 

If you have any questions or need more information, please do not hesitate to contact us at dipromats@lsi.uned.es .