2. Memotion

[This project is a part of my tenure at University of South Carolina as an affiliate Researcher]

Problem

The internet and especially social media platforms have provided users a medium to communicate their thoughts without any restrictions. The power of free uncensored speech however can cause considerable angst in the online community by demeaning other people. One popular form and crisp mechanism of producing such harmful content is the creation of memes. Memes generally consist of popular images and text associated with them that intend to spark humor, sarcasm, offense, and motivation among readers.

Problem Understanding (Cont.)

Memes are continuously evolving and add additional load on hate-classification systems because

  • They can be multi-modal in nature

  • They might not use explicit hate content/words but subtler forms of aggression like satire or sarcasm

  • They can contain code-mixed content which is harder to parse and detect.

Solution

  • Building emotion classification framework on multimodal multilingual domain

What's special about the third iteration of Memotion?

  • In Memotion 3.0, an additional layer of complexity is added by introducing flavors of codemixing and subtler forms of aggression like satire and sarcasm.

Task Details

  • Task A: Sentiment Analysis - Given an internet meme, the first task is to classify it as a positive, negative, or a neutral meme

  • Task B: Emotion Classification - Given an internet meme, the system has to identify the type of emotion expressed. The categories are humorous, sarcastic, offensive, and motivational. And it can have more than 1 category

  • Task C: Scales/Intensity of Emotion Classes - The third task is to quantify the extent to which a particular emotion is expressed

Implementaion

  1. Data Set Collection

  • Reddit scrapping from meme handles

  • Twitter

  • Facebook

  • Instagram

  1. Data Annotation

  • Data Annotation was done by 35+ annotators

  1. Classification Modeling & evaluation

  • Experimented with multilingual multimodal baseline model for above mentioned 3 tasks

Result

  • Two Paper accepted at Defactify workshop at AAAI'23

My Contribution

  • Managing and evaluating dataset collection and annotation tasks, building baseline models, and working on the manuscript