My research team has developed groundbreaking approaches to sarcasm detection and synthesis, focusing on multimodal analysis that combines acoustic features, text, and other cues to identify and reproduce sarcastic speech.
This project addresses several challenging aspects of sarcasm in human and machine communication:
Multimodal Sarcasm Detection: Developing systems that can identify sarcasm by analyzing speech, text, and other contextual cues
Sarcastic Speech Synthesis: Creating natural-sounding sarcastic speech for more expressive and nuanced AI voices
Cross-Cultural Sarcasm: Investigating how sarcasm markers differ across languages and cultures
Applications: Exploring uses in sentiment analysis, conversational AI, and assistive technology
“AMuSeD: An Attentive Deep Neural Network for Multimodal Sarcasm Detection Incorporating Bi-modal Data Augmentation” (2024)
“SarcasticSpeech: Speech Synthesis for Sarcasm in Low-Resource Scenarios” (2023)
“Deep CNN-based Inductive Transfer Learning for Sarcasm Detection in Speech” (2022)
“Building a better sarcasm detector” (2024)
Our research on sarcasm detection has gained significant international media attention:
The Guardian: “Why ’emotional AI’ is fraught with problems” (June 2024)
The Guardian: “Researchers build AI-driven sarcasm detector”(May 2024)
EuroNews: “AI can detect sarcasm now. Great…” (May 2024)
Popular Science: “Neural network trained on ‘Friends’ can recognize sarcasm” (May 2024)
Interesting Engineering: "‘Sarcasm detector’: Scientists finally create AI that can understand a joke" (May 2024)
BBC Radio 4: “The World this Weekend” (May 2024)
BBC Radio 5 Live: Interview on sarcasm detection (May 2024)
CBC Radio: “As It Happens with Nil Köksal, Chris Howden” (May 2024)
ISCA Podcast: Discussion on sarcasm detection research (June 2024)
NOS: “Computer herkent sarcasme steeds beter” (May 2024)
EW Magazine: “Een robot die sarcasme herkent, wat is dat nou weer?” (June 2024)
RTV Noord: “Onderzoekers RUG ontwikkelen, met behulp van Friends, een sarcasmedetector” (March 2024)
Leeuwarder Courant: “Waarom wetenschappers uit Leeuwarden een sarcasmedetector ontwikkelden (en hoe Friends daarbij hielp)” (March 2024)
EurekAlert: “Building a better sarcasm detector” (May 2024)
Improving multimodal fusion techniques for sarcasm detection
Developing more natural sarcastic speech synthesis methods
Creating resources for sarcasm detection in multiple languages
Exploring the ethical dimensions of emotion detection in speech technology
Investigating the role of cultural context in sarcasm interpretation
Our work uses innovative approaches such as attention mechanisms, contrastive learning, and novel data augmentation techniques to overcome the challenges of limited training data and the subtle nature of sarcastic expressions.