Research

The Impact of Platform Policy Interventions: Mitigating Harmful Behaviors Online

Link to Paper

Harmful user-generated content (UGC) such as verbal aggression and harassment, misinformation, as well as illegal content are widely and rapidly circulated on online platforms. As a result, these platforms have adopted various content moderation policies. The goal of these policies is to decrease the volume and impact of harmful content circulating on these platforms. The purpose of this work is to examine the effectiveness of these two types of content moderation policies applied to multiple types of harmful content, and to evaluate any associated externalities. 


We analyze two Reddit content moderation policies that are applied on the forum level. The goal of the first policy, forum prominence reduction, is to hide a problematic forum from other users to lessen its impact. The second policy, forum banning, deletes a forum from the platform but leaves its participants on the platform to participate in other forums. We collect large datasets of user posts from Reddit and use machine learning prediction algorithms to analyze the text and classify harmful content. This work applies econometric methods to estimate the causal impacts of these interventions. 


We first assess these policies’ impact on a forum high in verbal aggression (which includes hate speech and violent threats). We find that while both hiding and banning the forum directly decrease its activity, they lead to spillovers of verbal aggression to other Reddit forums that are ideologically similar to the focal forum. Interestingly, banning produces a wider spillover: users increase verbal aggression in both ideologically similar as well as ideologically dissimilar forums. To examine the cause of this wider spillover upon banning the forum, we further analyze the behaviors of different types of users. We identify two broad categories of users – multihomers (or users that concurrently operate accounts on Reddit as well as an external platform with similar functionality) and non-multihomers (users that continue to post only on Reddit).


We find that the wider spillover to ideologically dissimilar forums comes from multihoming users alone. Having access to an external alternative leads these users to be more aggressive in ideologically dissimilar and potentially unfriendly forums on Reddit, risking further sanctions. We run supplemental analyses that provide support for this explanation. We conclude that applying group content moderation policies to verbal aggression produces negative spillovers and that the more drastic the policy, the wider the spillover. Furthermore, the impact of these policies is significantly different for multihoming users. Finally, our results indicate that these verbal aggression spillovers are highly contagious to other users. 


The findings of this study provide an interesting contrast to the findings from the impact of hiding policies to contain misinformation. In addition to studying the impact of these policies on mitigating verbal aggression we also examine the effect of hiding Reddit forums that have a high degree of misinformation within them. We empirically assess the effectiveness, as well as spillover effects, of this hiding policy, utilizing machine learning techniques to accurately identify misinformation. We find that this policy diminishes misinformation within directly impacted spaces on a platform, as well as the dispersion of misinformation. However, we observe misinformation spillovers in ideologically neutral spaces. We find that this spillover of misinformation to neutral spaces declines over time, and is not contagious to native users that were previously present.


While in both cases – misinformation as well as verbal aggression – we find spillovers to nearby forums, hiding the problematic forum works in the case of misinformation, while it is less effective in the case of verbal aggression. We find that a key difference between the two contexts is that while the spillover of verbal aggression is contagious and ignites reciprocal behaviors, the spillover of misinformation does not, and this lack of reciprocity tends to dampen the further spread of misinformation. To synthesize, these studies highlight that there is no one-size-fits-all approach to content moderation: platforms must consider the type of user, type of content, and the potential for spillovers when designing and implementing these policies.


Papers

Under preparation to be submitted to Management Science.

Under first round of revisions (revise & resubmit) at Production and Operations Management Journal (POM).


The Effect of Incentive Structure Upon User Contribution to Online Platforms

On user-generated content (UGC) platforms, it has been shown that contributors at the very top of the distribution create most of the content present. These high-ability and high-contribution users make the most valuable contributions, are greatly influential to other users, and attract solution-seekers. Platforms utilize a variety of incentives to motivate users to participate and perform at a high level. Prior research has examined how the presence of various incentives affects user behavior within a crowdsourcing task. However, it is unclear how a platform’s incentive structure affects a platform’s ability to retain high-value users, as well as their subsequent behavior. This research seeks to causally evaluate how a platform's monetary incentives impact user outcomes. 

We study the effect of winning a monetary reward upon the subsequent performance, effort, and team composition of high-contribution, high-ability users (deemed ‘superstars’) on crowdsourcing platforms. Utilizing a novel regression discontinuity design, we are able to isolate the effect of a monetary reward from other confounding factors. We find that winning a monetary reward positively influences subsequent superstar performance, on top of the presence of significant reputational and learning incentives. We find this effect is driven by future team composition rather than by differences in effort. Individuals that win monetary rewards have a greater ability to retain and recruit high-ability team members for future contests due to a signaling effect. They are thus more likely to perform well and win in future contests. These findings indicate that financial incentives give users an advantage in assembling high-quality teams for subsequent tasks, which could lead to a positive feedback loop that could be contrary to platform aims. 


Papers

Under preparation to be submitted to Management Science.


The Democratization of Machine Learning Methodologies

My research- which utilizes text mining and machine learning techniques- led to an interest in examining the growing usage of machine learning in business analytics research in recent years. Application programming interfaces and tools such as SciKitLearn, Gensim, and TensorFlow have democratized machine learning, making it possible for researchers and practitioners with relatively limited formal training to utilize advanced methodologies in their work. This work seeks to examine how easy access to these tools shapes the direction of business analytics research, highlighting any significant trends in usage, as well as assessing positive and negative externalities.

 

We have preliminary results regarding one machine learning tool: topic modeling. Our work addressing this begins with a survey of business analytics papers between 2003 and 2020.  We find that topic modeling is a widely used technique, giving researchers the ability to incorporate large datasets of unstructured text into their analyses. However, knowledge of proper evaluation methods is often not highlighted. Topic modeling output is frequently transformed to create novel measures for empirical analysis, which are then placed into econometric specifications. We also conduct an experiment, examining the stability of regressions in which topic modeling measures are covariates. We find that these measures have the potential to be unstable in regressions; the choice of topic model to produce a metric can significantly alter the sign, magnitude, and statistical significance of a regression coefficient. In ongoing work, we assess the usage of multiple other machine learning tools to gain a full understanding of this trend across academic and industry contexts.


Papers

Under preparation to be submitted to MIS Quarterly.


AI Augmented Systems in Online Platform Content Moderation

Due to the staggering volume of UGC uploaded to the internet daily, many online platforms have begun deploying augmented AI systems to evaluate content. To investigate this topic, I am currently conducting research comparing the impact of augmented bot systems and human moderators on problematic user behavior. 


The research context of this work is the online discussion forum Reddit. My dissertation research focuses on centralized content moderation policies applied by Reddit administration in extreme cases.  

However, the platform currently utilizes two methodologies of decentralized content moderation for low-level daily decisions within forums. These methodologies are manual moderation enforced by volunteer human moderators, as well as augmented bot moderation. This work compares the effect of using human versus an augmented bot system upon user behavior. 


Both decentralized methodologies primarily involve removing posts that violate forum-level rules. This can be enforced by humans, or by a pre-programmed bot. Human moderators can review bot decisions, which makes it an augmented system. Each forum typically utilizes both types of moderation. To assess the differential effects of both approaches, I am assembling a dataset of removed posts across a range of forums addressing different topics. Due to differences in bot programming across forums, rule-violating posts with markedly similar content may be removed by human or bot, depending on the forum. I compare the impact of removal by bot versus human by econometrically matching posts on text, user, post, and forum-level features, and comparing subsequent user behavior across both types of removal. The results of this study will highlight the advantages, as well as potential drawbacks associated with semi-automating platform content moderation.


Papers

Under preparation to be submitted to Information Systems Research.