Yoon Jeong Cha | 차윤정

Sticker Misinterpretations in Instant Messaging

🌱From the HCI course (taught by Uichin Lee, mentor: Sangkeun Park) during MS

Yoonjeong Cha, Jongwon Kim, Sangkeun Park, Mun Yong Yi, and Uichin Lee. 2018. Complex and Ambiguous: Understanding Sticker Misinterpretations in Instant Messaging. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 30 (November 2018), 22 pages. https://doi.org/10.1145/3274299 [pdf]

CSCW'18 Paper Abstract

Stickers, though similar in appearance to emoji, have distinct characteristics because they often contain animation, diverse gestures, and multiple characters and objects. Stickers can convey richer meaning than emoji, but their complexity and placement constraint may result in miscommunication. In this paper, we aim to understand howpeople perceive emotion in stickers, aswell as howmiscommunication related to sticker occurs in actual chat contexts. Toward this goal,we conducted an online survey (n = 87) and in-depth interviews (n = 28) in South Korea.We found emotional and contextual aspects of sticker misinterpretation. In particular, emotion misinterpretation mostly happened due to stickers’ ambiguous (multiple) facial/bodily expressions and different perception of dynamism in gestures. In real chat settings, there were also contextual misinterpretations where senders and receivers differently interpret stickers’ visual representation/reference, or/and corresponding textual messages. Based on these findings, we provide several practical design implications such as context awareness support in sticker interaction.

Background & Research Question

In recent years, stickers have become widely used in instant messaging apps such as WhatsApp, WeChat, LINE, and KakaoTalk.

In this work, we focus on sticker misinterpretation in instant messaging. Unlike traditional emoticon and emoji, stickers may contain animation with multiple characters and objects. More importantly, the placement constraint of stickers is different from emoji, as stickers must be sent as singular entities and cannot be included next to text in the same way that emoji can.

Method

Study 1 Method

We used Russell’s core affect theory to measure the emotional misinterpretations associated with each sticker. Russell’s model demonstrates that human emotions can be represented through a combination of two dimensions: valence and arousal levels. The valence level denotes how pleasant (or displeasant) a user’s emotion is, and the arousal level describes how active (or inactive) a user’s emotion is. For each sticker in KakaoTalk, we asked about valence and arousal levels.

Study 1 Findings

In Study 1, we used an online survey to investigate users’ perceptions about the emotional meanings of stickers in instant messaging. Our results showed that the stickers can be interpreted at varying levels—in terms of valence level, because of a character’s ambiguous facial or bodily expressions, and in terms of arousal level, because of the intensity and dynamism of a character’s gestures.

These varying interpretations can be severe in real-world, computer-mediated communications where diverse stickers are used. This leads us to the following study in the next section.

Study 2 Method

Before the interview, we asked each pair of participants to screen-capture 10 chat instances in which they had exchanged a sticker in the past three months and then submit the captured images to the researchers via email.

After collecting these images, we conducted a survey first to see their different interpretation of sticker usage and then interviewed the pairs of participants. We asked them to explain each instance of sticker usage in detail to check whether 1) the sender’s and receiver’s understandings of the situation matched, and whether 2) the sender’s intentions and the receiver’s interpretations were in agreement.

Study 2 Findings

We uncovered two major patterns of context misconstruals related to the real chat settings: mismatch in the sticker’s visual representation/reference and message correspondence mismatch.

Visual representation and reference mismatch refers to cases where the sender and the receiver understood the visual representations and references in the stickers differently, as we observed in Study 1. Message correspondence mismatch denotes cases where the texts that a sticker corresponds to is different.

Design Implications

Page updated

Report abuse