2. Memotion
[This project is a part of my tenure at University of South Carolina as an affiliate Researcher]
Problem
The internet and especially social media platforms have provided users a medium to communicate their thoughts without any restrictions. The power of free uncensored speech however can cause considerable angst in the online community by demeaning other people. One popular form and crisp mechanism of producing such harmful content is the creation of memes. Memes generally consist of popular images and text associated with them that intend to spark humor, sarcasm, offense, and motivation among readers.
Problem Understanding (Cont.)
Memes are continuously evolving and add additional load on hate-classification systems because
They can be multi-modal in nature
They might not use explicit hate content/words but subtler forms of aggression like satire or sarcasm
They can contain code-mixed content which is harder to parse and detect.
Solution
Building emotion classification framework on multimodal multilingual domain
What's special about the third iteration of Memotion?
In Memotion 3.0, an additional layer of complexity is added by introducing flavors of codemixing and subtler forms of aggression like satire and sarcasm.
Task Details
Task A: Sentiment Analysis - Given an internet meme, the first task is to classify it as a positive, negative, or a neutral meme
Task B: Emotion Classification - Given an internet meme, the system has to identify the type of emotion expressed. The categories are humorous, sarcastic, offensive, and motivational. And it can have more than 1 category
Task C: Scales/Intensity of Emotion Classes - The third task is to quantify the extent to which a particular emotion is expressed
Implementaion
Data Set Collection
Reddit scrapping from meme handles
Twitter
Facebook
Instagram
Data Annotation
Data Annotation was done by 35+ annotators
Classification Modeling & evaluation
Experimented with multilingual multimodal baseline model for above mentioned 3 tasks
Result
Two Paper accepted at Defactify workshop at AAAI'23
My Contribution
Managing and evaluating dataset collection and annotation tasks, building baseline models, and working on the manuscript