OffensEval 2020

Description

This is the website of the shared task on Identifying and Categorizing Offensive Language in Social Media (OffensEval) organized at SemEval 2020 (task 12). 

This is the second iteration of OffensEval. The first edition was organized at SemEval 2019OffensEval 2019 used the Offensive Language Identification dataset (OLID) a dataset containing English tweets annotated using a hierarchical three-level annotation model described in this paper.

Nearly 800 teams signed up to participate in OffensEval 2019. The competition received more than 100 submissions across three sub-tasks.The findings are described in the OffensEval 2019 reportThe response received in 2019 by far exceeded our expectations and motivated us to organize OffensEval 2020. More information will be available here soon.

Motivation

Offensive language is pervasive in social media. Individuals frequently take advantage of the perceived anonymity of computer-mediated communication, using this to engage in behavior that many of them would not consider in real life. Online communities, social media platforms, and technology companies have been investing heavily in ways to cope with offensive language to prevent abusive behavior in social media. One of the most effective strategies for tackling this problem is to use computational methods to identify offense, aggression, and hate speech in user-generated content (e.g. posts, comments, microblogs, etc.). 

This topic has attracted significant attention in recent years as evidenced in recent publications (Waseem et al. 2017; Davidson et al., 2017, Malmasi and Zampieri, 2018, Kumar et al. 2018) and workshops such as AWL and TRAC and competitions such as HatEval 2019 (Basile et al. 2019), HASOC 2019, and OffensEval 2019 (Zampieri et al. 2019).

More information on OffensEval 2020 will be available soon.

Registration

Please fill out this form to receive more information.

Data
  • Trial data: available here.
  • Training and development data: coming soon.
  • Test data: coming soon.
Important Dates

  • Trial data ready: July 31, 2019 (available here)
  • Training and development data: November 8, 2019
  • Test data ready: December 15, 2019
  • Evaluation starts: January 10, 2020
  • Evaluation ends: January 31, 2020
  • Paper submission deadline: February 23, 2020
  • Notification to authors: March 29, 2020
  • Camera-ready due: April 5, 2020
  • SemEval workshop: Summer 2020

Organizers

T.B.A.

Contact

Two mailing lists are available:

Organizers: semeval-2020-task-12-organizers@googlegroups.com
All participants: semeval-2020-task-12-all@googlegroups.com

References

Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Pardo, F.M.R., Rosso, P. and Sanguinetti, M., (2019) Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation (pp. 54-63).

Davidson, T., Warmsley, D., Macy, M. and Weber, I. (2017) Automated Hate Speech Detection and the Problem of Offensive Language. Proceedings of ICWSM.

Kumar, R., Ojha, A.K., Malmasi, S. and Zampieri, M. (2018) Benchmarking Aggression Identification in Social Media. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC). pp. 1-11.

Malmasi, S., Zampieri, M. (2018) Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence. Volume 30, Issue 2, pp. 187-202. Taylor & Francis. 

Waseem, Z., Davidson, T., Warmsley, D. and Weber, I. (2017) Understanding Abuse: A Typology of Abusive Language Detection Subtasks. Proceedings of the Abusive Language Online Workshop.

Previous OffensEval

REPORT
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N. and Kumar, R. (2019) SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of the 13th International Workshop on Semantic Evaluation. pp. 75-86.

DATASET
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N. and Kumar, R. (2019) Predicting the Type and Target of Offensive Posts in Social Media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp. 1415-1420.