Papers

Peer-reviewed publications

Josh A. Goldstein, Jason Chao, Shelby Grossman, Alex Stamos, and Michael Tomz. "How Persuasive is AI-Generated Propaganda?" pdf; appendices; pre-registration plan; replication data and code (PNAS Nexus)

Can large language models, a form of artificial intelligence (AI), generate persuasive propaganda? We conducted a preregistered survey experiment of US respondents to investigate the persuasiveness of news articles written by foreign propagandists compared to content generated by GPT-3 davinci (a large language model). We found that GPT-3 can create highly persuasive text as measured by participants’ agreement with propaganda theses. We further investigated whether a person fluent in English could improve propaganda persuasiveness. Editing the prompt fed to GPT-3 and/or curating GPT-3’s output made GPT-3 even more persuasive, and, under certain conditions, as persuasive as the original propaganda. Our findings suggest that propagandists could use AI to create convincing content with limited effort.

Josh Goldstein, Shelby Grossman, and Meredith Startz. "Belief in COVID-19 Misinformation in Nigeria."  pdf; appendix (The Journal of Politics)

Research suggests that partisanship and social media usage correlate with belief in COVID-19 misinformation, and that misinformation shapes citizens' willingness to get vaccinated. However, this evidence comes overwhelmingly from frequent internet users in rich, Western countries. We run a panel survey early in the pandemic leveraging a pre-pandemic sample of urban middle-class Nigerians, many of whom do not use the internet. Analysis registered under our pre-analysis plan shows that opposition party support and social media usage are correlated with belief in anti-government misinformation, but not other types of COVID-19 misinformation. Surprisingly, we find no relationship between overall belief in misinformation and willingness to be vaccinated several weeks later. Partisanship and ethnicity are predictive of vaccine hesitancy, while men are both more likely to believe misinformation and more willing to be vaccinated. These findings have significant implications for understanding vaccine hesitancy in Nigeria and beyond.

Renée DiResta, Shelby Grossman, and Alexandra Siegel. "In-House vs. Outsourced Trolls: How Digital Mercenaries Shape State Influence Strategies." pdf (Political Communication)

When governments run influence operations they may leverage in-house capabilities, outsource to digital mercenaries, or use a combination of these strategies. We theorize that governments outsource because it provides plausible deniability if the operation is uncovered, and offers access to cutting-edge influence tactics beyond those common to established government institutions. Using data from Facebook, Twitter, and Instagram, we test implications of this theory via two covert online influence campaign case studies, each focused on Syria, executed by Russia’s military intelligence agency (colloquially known as the GRU), and by the Internet Research Agency (IRA), a privately owned company. We find that the GRU focused on the creation of front media properties that produced longform journalistic content, an established tactic more amenable to reaching general audiences. By contrast, the IRA exploited the architecture of social media platforms to target specific audiences with memes and customized messages that were more narrowly tailored than those spread by the GRU. We also find that the tailored content produced by the IRA received higher engagement than GRU longform articles when posted to the same platforms, even if we include cascades of interactions from re-posts of GRU-authored articles that spread beyond their own Facebook page. Our findings highlight the importance of disaggregating information operations by actor type and across platforms to better understand their tactics and impact.

Shelby Grossman, Katie Jonsson, Nicholas Lyon, and Lydia Sizer. "Slanted Narratives, Social Media, and Foreign Influence in Libya." pdf (Journal of Quantitative Description: Digital Media)

The rise of social media has lowered barriers for both creators and consumers to engage with mass communication. In fragile contexts such as Libya where social media penetration is high, foreign social media outlets with political interests can use these platforms to influence the country's volatile political climate. In this study, we assess how social media content varies by the country of the information producer. To do so, we create a dataset of the universe of Facebook posts about a strongman’s recent attack on Tripoli (N=16,662) and leverage a Facebook feature that provides Page administrator locations. We find that more than half of the posts originated from outside Libya and that there is a substantively meaningful relationship between the location of content producers and a post's slant: posts from countries aligned with the Tripoli-based government are biased in that direction and posts from countries aligned with the eastern-based strongman are biased toward his forces. However, many Pages are not slanted: the correlations are instead driven by a smaller number of hyperpartisan Pages. Our findings have implications for our understanding of how social media content - especially from abroad - could shape citizen perceptions of the legitimacy of competing political actors.

Olivia Borge, Victoria Cosgrove, Elena Cryst, Shelby Grossman, Shelby Perkins, and Anna Van Meter. "How Search Engines Handle Suicide Queries." pdf (Journal of Online Trust and Safety)

The suicide contagion effect posits that exposure to suicide- related content increases the likelihood of an individual engaging in suicidal behavior. Internet suicide-related queries correlate with suicide prevalence. However, suicide-related searches also lead people to access help resources. This article systematically evaluates the results returned from both general suicide terms and terms related to specific suicide means across three popular search engines—Google, Bing, DuckDuckGo— in both English and Spanish. We find that Bing and DuckDuckGo surface harmful content more often than Google. We assess whether search engines show suicide prevention hotline information, and find that 53% of English queries have this information, compared to 13% of Spanish queries. Looking across platforms, 55% of Google queries include hotline information, compared to 35% for Bing and 10% for DuckDuckGo. Specific suicide means queries are 20% more likely to surface harmful results on Bing and DuckDuckGo compared to general suicide term queries, with no difference on Google.  

Shelby Grossman and Alisha Holland. “The Collusion Dilemma: Theory with Evidence from Informal Markets in Lagos, Nigeria.” pdf (World Development, special issue honoring Robert Bates)

Informal actors often compete with formal or regulated ones. Regulated actors therefore should be natural allies in government attempts to enforce laws and regulations. Surprisingly, they often are not. We argue that the lack of cooperation stems from a collusion dilemma. Collectively, formal actors are better off if informal actors are removed; individually, they can benefit from their presence. We demonstrate these dynamics in the context of Lagos, Nigeria, where millions of informal street vendors compete with traders in licensed markets. We draw on original survey data from 1,179 market traders across 199 associations and qualitative interviews with organization leaders, market traders, street vendors, and government officials in two markets with varying organizational capacity. We show how a negative equilibrium emerges in which limited state enforcement makes it difficult for societal partners to pursue their collective interests. The lack of societal collaboration in turn further erodes state enforcement capacity.

Nathaniel Leff, Rachael Behr, Jeffry Frieden, and Shelby Grossman. “Trust and Envy: The Political Economy of Business Groups in Developing Countries.” pdf (Studies in Comparative International Development)

Diversified business groups play a major role in the economies of many developing countries. Business group members, often from the same communal, ethnic, or tribal group, have or develop inter-personal relations that make it easier to obtain information and monitor compliance related to transactions that require a strong measure of trust. This in-group cohesion facilitates profitable and productive economic activity. However, it can create resentment among other members of society who are barred from membership in a group that is, of necessity, exclusive. This envy can fuel a self-reinforcing cycle of societal hostility and group protectiveness that can deprive society of the economic benefits the groups can provide. There are several possible reactions such as “affirmative action” programs that can slow or stop the cycle of envy and group vulnerability.

Shelby Grossman. “The Politics of Order in Informal Markets: Evidence from Lagos.” pdf; appendix; replication data; VoxDev writeup; Winner of African Politics Conference Group - African Affairs Best Graduate Student Paper 2015/16 (World Politics)

Property rights are important for economic exchange, but in much of the world they are not publicly guaranteed. Private market associations can fill this gap by providing an institutional structure to enforce agreements, but with this power comes the ability to extort from group members. Under what circumstances do private associations provide a stable environment for economic activity? Using survey data collected from 1,179 randomly sampled traders across 199 markets in Lagos, I find that markets maintain institutions to support trade not in the absence of government, but rather in response to active government interference. I argue that associations develop pro-trade institutions when threatened by politicians they perceive to be predatory, and when the organization can respond with threats of its own; the latter is easier when traders are not competing with each other. In order to maintain this balance of power, the association will not extort because it needs trader support to maintain the credibility of its threats to mobilize against predatory politicians.

Hannah Baron, Robert A. Blair, and Shelby Grossman. “Teaching Trump: Why Comparative Politics Makes Students More Optimistic About US Democracy.” pdf; appendix; Inside Higher Ed write up; APSA Comparative Politics newsletter (PS: Political Science and Politics)

How does learning about democratic erosion in other countries shape opinions about the state of democracy in the US today? We describe lessons learned from a collaborative course on democratic erosion taught at nearly two dozen universities over the 2017-18 academic year. We use survey data, student-written blog posts, exit questionnaires, and interviews with students who did and did not take the course to explore the effects of studying democratic erosion from a comparative perspective. Do comparisons foster optimism about the relative resilience of democracy in the US, or pessimism about its vulnerability to the same risk factors that have damaged other democracies around the world? Somewhat to our surprise, we find that the course increased optimism about US democracy, instilling greater confidence in the relative strength and longevity of American democratic norms and institutions. We also find, however, that the course did not increase civic engagement, and, if anything, appears to have exacerbated skepticism towards activities such as protest. Students who took the course became increasingly sensitive to the possibility that some forms of civic engagement reflect and intensify the same threats to democracy that the course emphasized—especially polarization.

Shelby Grossman, Jonathan Phillips, and Leah Rosenzweig. “Opportunistic Accountability: State-Society Bargaining Over Shared Interests” pdf; appendix; replication data; Monkey Cage/Washington Post write up (Comparative Political Studies)

Conflicting preferences between the state and society underpin most accountability mechanisms by providing a credible way for society to impose costs on the state. Adapting a classic bargaining framework, we argue that broader conditions can support state-society bargaining. Policies that both the state and society value can also enhance society’s negotiating power provided society has a lower valuation and is more patient than the state. By threatening to sabotage their own interests but hurt the impatient state even more, citizens can compel the state to deliver broader policy benefits. We illustrate this logic with the case of polio vaccination in northern Nigeria, where entire communities have resisted the vaccine as a strategy to bargain for more desired services. To resolve and preempt non-compliance, the Nigerian government has enhanced service delivery in other areas, demonstrating the opportunity for improved accountability in the presence of shared-interest policies. 

Shelby Grossman and Dan Honig. "Evidence from Lagos on Discrimination across Ethnic and Class Identities in Informal Trade." pdf; replication data (World Development)

This paper investigates the determinants of price discrimination in the rice market in one neighborhood of Lagos, Nigeria. There has been little empirical study of how ethnicity and class shape economic outcomes in informal market interactions. We conduct an audit experiment – one of the first audit experiments in Africa – seeking to address this gap. We experimentally manipulate class, with confederates presenting as different classes; this may be the first audit study to take this approach. This is also one of the first in-person audits to have multiple transactions for each buyer and seller, thus allowing for the use of buyer and seller fixed effects. We find little evidence that, all else equal, sharing an ethnicity on its own influences market treatment. Class, however, does have substantial effects, at least for non-coethnics. High class non-coethnics receive higher prices per unit than low class non-coethnics. Our findings suggest that the boundaries of group identity appear to be at least partially defined by class in the informal economy.

Working papers

Rishi Bomassani et al. "On the Opportunities and Risks of Foundation Models." pdf

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.