With Professor Abhijit Banerjee, Nobel Laureate in Economics and coauthor of the MTV Shuga studies on HIV and GBV prevention.
Embedded Files
Publications
Community Health Workers as Key Providers of Easy-to-Use Contraceptive Injectables : Experimental Evidence from Rural Burundi
Michele Andreottola, Olivier Basenya; Victor Orozco-Olvera, Arndt Reichert and Paula Spinola
This study employs a cluster randomized controlled trial and administrative health center data to investigate the effects of authorizing community health workers to deliver a new generation of contraceptive injections directly to women during routine home visits following comprehensive training. The paper observes a significant increase of approximately 70 percent in the administered quantity of these injections, which provide average protection for three months. However, the results suggest that the intervention does not produce a statistically significant change in contraceptive coverage because of significant substitution effects away from long-acting contraceptive implants and intrauterine devices that women might otherwise have adopted.
Can Facebook Ads Prevent Malaria ? Two Field Experiments in India
Dante Donati, Ana Maria Munoz-Boudet, Victor Orozco-Olvera and Nandan Rao
This study uses a cluster randomized controlled trial to evaluate the impact of a nationwide malaria prevention advertising campaign delivered through social media in India. Ads were randomly assigned at the district level, and the study relies on data from two independently recruited samples (8,257 individuals) and administrative records. Among users residing in solid (concrete) dwellings, where malaria risk is lower, the campaign led to an 11 percent increase in mosquito net usage and a 13 percent increase in timely treatment seeking. Self-reported malaria incidence decreased by 44 percent. Consistently, recorded health facility data indicate a reduction in urban monthly incidence of 6.2 cases per million people, corresponding to 30 percent of the overall monthly incidence rate of malaria. Conversely, the study finds no impact on households living in non-solid dwellings, which face higher malaria risk, nor among rural settlements where such dwellings are more prevalent. To disentangle if this lack of impact stems from ineffective content or insufficient reach, an individual-level trial was conducted (1,542 individuals), ensuring campaign exposure for both household types. The findings indicate an increase in bed net usage and timely treatment seeking for both groups, underscoring the need for improved targeting in social media campaigns to fulfill public health goals.
Behaviorally Informed Messages Increase COVID-19 Vaccination Intentions : Insights from a Global Meta-Analysis
Daniel Pinzon Hernandez, Jungkyu Rhys Lim, Michelle Dugas, Ellen Moscoe, Mohamad Chatila, Corey Cameron, Renos Vakis, Zeina Afif, and Victor Orozco-Olvera
During the COVID-19 pandemic, low- and middle-income countries struggled with lower vaccination rates compared to wealthier countries, posing challenges to reducing virus transmission, mitigating healthcare system pressures, and promoting economic recovery. Communications campaigns offer low-cost opportunities to overcome such challenges by strengthening vaccine confidence and intentions to get vaccinated, but empirical testing is needed to identify which messages will be most effective in different contexts. To support policy-making efforts to design effective communication rapidly during the pandemic, a global research program of 28 online experiments was conducted by recruiting respondents (123,270 individuals) through social media between January 2021 and June 2022 across 23 mostly low- and middle-income countries and territories. An individual participant data (IPD) meta-analysis of these data summarizes the results of this research program testing the impact of behaviorally informed messaging on vaccine intentions. Results from the meta-analysis show that among unvaccinated survey respondents, behaviorally informed messages significantly increased the odds of vaccination intention by 1.28 times overall and up to 1.93 times in individual studies (safety messages in Papua New Guinea). Significant pooled effects of specific framings ranged from increasing the odds of vaccination intention by 1.16 times (variant framing) to 1.45 times (experts and religious leaders framing). This research underscores the importance of communication tailored to address different drivers of vaccine hesitancy and offers insights for handling future health crises with behavioral communication strategies leveraging rapid insights afforded by social media.
HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter
Manuel Tonneau, Diyi Liu, Niyati Malhotra, Scott Hale, Samuel Fraiberger, Victor Orozco-Olvera, and Paul Röttger
To tackle the global challenge of online hate speech, a large body of research has developed detection models to flag hate speech in the sea of online content. Yet, due to systematic biases in evaluation datasets, detection performance in real-world settings remains unclear, let alone across geographies. To address this issue, we introduce HateDay, the first global hate speech dataset representative of social media settings, randomly sampled from all tweets posted on September 21, 2022 for eight languages and four English-speaking countries. Using HateDay, we show how the prevalence and composition of hate speech varies across languages and countries. We also find that evaluation on academic hate speech datasets overestimates real-world detection performance, which we find is very low, especially for non-European languages. We identify several factors explaining poor performance, including models' inability to distinguish between hate and offensive speech, and the misalignment between academic target focus and real-world target prevalence. We finally argue that such low performance renders hate speech moderation with public detection models unfeasible, even in a human-in-the-loop setting which we find is prohibitively costly. Overall, we emphasize the need to evaluate future detection models from academia and platforms in real-world settings to address this global challenge.
NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Data
Manuel Tonneau, Pedro Vitor Quinta de Castro, Karim Lasri, Ibrahim Farouq, Lakshminarayanan Subramanian, Victor Orozco-Olvera and Samuel Fraiberger
To address the global issue of online hate, hate speech detection (HSD) systems are typically developed on datasets from the United States, thereby failing to generalize to English dialects from the Majority World. Furthermore, HSD models are often evaluated on non-representative samples, raising concerns about overestimating model performance in real-world settings. In this work, we introduce NaijaHate, the first dataset annotated for HSD which contains a representative sample of Nigerian tweets. We demonstrate that HSD evaluated on biased datasets traditionally used in the literature consistently overestimates real-world performance by at least two-fold. We then propose NaijaXLM-T, a pretrained model tailored to the Nigerian Twitter context, and establish the key role played by domain-adaptive pretraining and finetuning in maximizing HSD performance. Finally, owing to the modest performance of HSD systems in real-world conditions, we find that content moderators would need to review about ten thousand Nigerian tweets flagged as hateful daily to moderate 60% of all hateful content, highlighting the challenges of moderating hate speech at scale as social media usage continues to grow globally. Taken together, these results pave the way towards robust HSD systems and a better protection of social media users from hateful content in low-resource settings.
Can mass media campaigns increase demand for renewable energy? Experimental evidence from Senegal
Aidan Coville, Victor Orozco-Olvera and Arndt Reichert
This paper examines public investment in mass media campaigns aimed at stimulating market-led growth within the renewable energy sector in a developing country setting. We run a randomized field experiment to estimate the impact of a large-scale information campaign on the adoption of renewable energy technology, which includes clips broadcasted on national radio and print materials distributed in rural villages in Senegal. While the radio campaign primarily influences the number of renewable energy products owned by existing users (intensive margin), the combination of radio and print significantly increases the number of new adopters (extensive margin). However, when scaled nationally, we calculate a meaningful advantage of the radio clips over the print materials in terms of the cost per additional sales of solar lamps. The study further introduces an innovative methodological approach for examining nationally broadcasted information campaigns, offering valuable insights for future research and policy evaluations.
Media and Health Outcomes
Víctor Orozco-Olvera and Niyati Malhotra
This chapter takes stock of the research that studies the relationship between mass media campaigns and behavioral outcomes that have an impact on health. It first discusses the theoretical frameworks underpinning health-focused entertainment media and social and behavior change communications (SBCCs), followed by recent natural experiments that exploit new datasets and statistical approaches for rigorously evaluating the causal effect of “real-world” media on health outcomes. This discussion covers both the intentional use of media through entertainment education, or “edutainment,” and also the varied effects of commercial media consumption on health. The chapter then discusses the literature linking traditional and new media with health by considering reviews and high-quality evaluations of SBCC media campaigns targeting sexual and reproductive health, child survival, violence against women, non-communicable diseases – in particular those associated with tobacco consumption and other substances – and healthcare-seeking and health-preserving behaviors, particularly in the context of the COVID-19 pandemic. The chapter concludes with a discussion highlighting the main findings and existing knowledge gaps of the literature and a discussion of new media for improving health outcomes at scale.
Large-Scale Demographic Inference of Social Media Users in a Low-Resource Scenario
Karim Lasri, Manuel Tonneau, Haaya Naushan, Niyati Malhotra, Ibrahim Farouq, Víctor Orozco-Olvera, and Samuel Fraiberger
Characterizing the demographics of social media users enables a diversity of applications, from better targeting of policy interventions to the derivation of representative population estimates of social phenomena. Achieving high performance with supervised learning, however, can be challenging as labeled data is often scarce. Alternatively, rule-based matching strategies provide well-grounded information but only offer partial coverage over users. It is unclear, therefore, what features and models are best suited to maximize coverage over a large set of users while maintaining high performance. In this paper, we develop a cost-effective strategy for large-scale demographic inference by relying on minimal labeling efforts. We combine a name-matching strategy with graph-based methods to map the demographics of 1.8 million Nigerian Twitter users. Specifically, we compare a purely graph-based propagation model, namely Label Propagation (LP), with Graph Convolutional Networks (GCN), a graph model that also incorporates node features based on user content. We find that both models largely outperform supervised learning approaches based purely on user content that lack graph information. Notably, we find that LP achieves comparable performance to the state-of-the-art GCN while providing greater interpretability at a lower computing cost. Moreover, performance does not significantly improve with the addition of user-specific features, such as textual representations of user tweets and user geolocation. Leveraging our data collection effort, we describe the demographic composition of Nigerian Twitter finding that it is a highly non-uniform sample of the general Nigerian population.
Improving Enrollment and Learning Through Videos and Mobiles: Experimental Evidence from Northern Nigeria
Victor Orozco-Olvera and Ericka Rascón-Ramirez
In northern Nigeria, half of primary school-age children attend school, half of girls are married before turning 15, and one in five people can read a whole sentence. Conducted in rural, low literate communities governed by traditional norms, this paper presents the results of a cluster randomized controlled trial that tested community screenings to reshape parental aspirations and attitudes toward education, and as a reinforcing arm, the distribution of mobiles with engaging apps to teach 6-9-year-old children to read. Twelve months after the screenings, children were 42 percent less likely to be out of school, but as expected, their learning levels did not improve. In the communities that were provided the mobile reinforcer, literacy and numeracy skills increased by 0.46 and 0.63 standard deviation, respectively. The impacts of the combined intervention on school attendance and learning gains were similar for boys and girls. For non-targeted older siblings, the intervention increased learning by 0.34 and 0.47 standard deviation and reduced the likelihood of teenage pregnancy and early entrance into the labor market by 13 and 14 percent, respectively. The mechanisms behind these effects include improved parental aspirations and expectations, improved attitudes and social norms, higher self-efficacy beliefs of parents, and increased time for home learning activities. Relative to other educational investments that have been evaluated in developing countries, the combined intervention is highly effective and cost-effective.
Using Social Media to Change Gender Norms: An Experimental Evaluation Within Facebook Messenger in India
Dante Donati, Victor Orozco-Olvera, Nandan Rao
This paper experimentally tests the effectiveness of two short edutainment campaigns (under 25 minutes) delivered through Facebook Messenger at reshaping gender norms and reducing social acceptability of violence against women (VAW) in India. Participants were randomly assigned to watch video-clips with implicit or explicit messaging formats (respectively a humorous fake reality TV drama or a docu-series with clear calls to action). After one week, intent-to-treat effects of the implicit format on knowledge, gender norms, and acceptability of VAW oscillated between 0.16 and 0.21 standard deviations yet impacts diminished after four months. On other hand, the explicit format was more impactful in the short-term in increasing willingness to share video-clips with friends and promoting online information-seeking behaviors; and in the medium-term, individuals exposed to the docu-series were 91% (7.5 p.p.) more likely to add a frame against VAW in their Facebook profile picture, a public display of their disapproval of this harmful practice. The general lack of heterogeneous effects across social status indicators suggest social media as a potential medium for reaching different online populations, including vulnerable ones.
To pay or not to pay: Measuring risk preferences in lab and field
Pablo Brañas, Lorenzo Estepa, Diego Jorrat, Victor Orozco-Olvera and Ericka Rascón-Ramirez
Measuring risk preferences using monetary incentives is costly. In the field, it might be also unfair and unsafe. The commonly used measure of Holt and Laury (2002) relies on a dozen lottery choices and payments, which make it time consuming and expensive. It also raises moral concerns as a result of the unequal payments generated by good and bad luck. Paying some but not all subjects may also create tensions between the researcher and subjects. In a pre-registered study in Honduras, Nigeria and Spain, we use a short version of Holt and Laury where we address all three concerns. We find in the field that not paying at all or paying with and without probabilistic rules makes no difference. Our hypothetical and short version makes our measurement of risk cheaper, fairer and safer.
SSRN working paper
Conducting Surveys and Interventions Entirely Online : A Virtual Lab Practitioner’s Manual
Dante Donati, Victor Orozco-Olvera, Nandan Rao
Online and social media campaigns reach billions of people every day. While commercial companies have built up extensive expertise in using these tools to recruit and build relationships with customers, researchers and policymakers have been slower to take full advantage of these new digital tools. Virtual Lab is an open-source set of tools developed by the World Bankʼs Development Impact Department (DIME) for online recruitment, intervention, and surveying via digital advertising and social-media platforms. It was built specifically for researchers and policymakers. An online study can only work if it performs inference on groups that can at least be found online, even if they are underrepresented, and only if such inference can be performed entirely from those individuals who are found online, even if they are hard to find. This manual provides an introduction to the survey theory underpinning Virtual Lab, lays out detailed guidelines for designing online studies within the platform, shares costs data and lessons learned from recent trials, and lays out promising areas for future research. Virtual Lab can be self-hosted on private or public cloud infrastructure. All code is open source.
The Entertaining Way to Behavioral Change: Fighting HIV with MTV
Abhijit Banerjee, Eliana La Ferrara, and Victor Orozco-Olvera
This paper tests the effectiveness of an entertainment education television series, MTV Shuga, aimed at providing information and changing attitudes and behaviors related to HIV/AIDS. Using a simple model, the paper shows that “edutainment” can work through an individual or a social channel. This study is a randomized controlled trial conducted in urban Nigeria, where young viewers were exposed to MTV Shuga or a placebo television series. Among those exposed to MTV Shuga, the trial created additional variation in the social messages they received and the people with whom they watched the show. The study finds significant improvements in knowledge and attitudes toward HIV and risky sexual behavior. Treated subjects are twice as likely to get tested for HIV eight months after the intervention. The study also finds reductions in sexually transmitted diseases among women. These effects are stronger for viewers who reported being more involved with the narrative, consistent with the psychological underpinnings of edutainment. The trial’s experimental manipulations of the social norm component did not produce significantly different results from the main treatment. The individual effect of edutainment thus seems to have prevailed in the context of this study.
Entertainment, Education, and Attitudes Toward Domestic Violence
Abhijit Banerjee, Eliana La Ferrara, and Victor Orozco-Olvera
We experimentally measure the effectiveness of the edu-tainment TV series MTV Shuga on domestic violence outcomes, a secondary theme of the studied season. We find that MTV Shuga induced an improvement in men’s attitudes toward women eight months later. We also find that attitudes toward GBV significantly improve for men and women who report occasionally thinking about the characters and who remember specific facts about them. These findings call for a deeper analysis of the links between attention, empathy and the policy impact of edu-tainment programs.
Payments for environmental services supported social capital while increasing land management
Alix-Garcia, Jennifer M., Katharine R. E. Sims, Victor H. Orozco-Olvera, Laura E. Costica, Jorge David Fernández Medina, Sofía Romo Monroy
Payments for environmental services (PES) programs incentivize landowners to protect or improve natural resources. Many conservationists fear that introducing compensation for actions previously offered voluntarily will reduce social capital (the institutions, relationships, attitudes, and values that govern human interactions), yet little rigorous research has investigated this concern. We examined the land cover management and communal social capital impacts of Mexico’s federal conservation payments program, which is a key example for other countries committed to reducing deforestation, protecting watersheds, and conserving biodiversity. We used a regression discontinuity (RD) methodology to identify causal program effects, comparing outcomes for PES participants and similar rejected applicants close to scoring cutoffs. We found that payments increased land cover management activities, such as patrolling for illegal activity, building fire breaks, controlling pests, or promoting soil conservation, by ∼50%. Importantly, increases in paid activities as a result of PES did not crowd out unpaid contributions to land management or other prosocial work. Community social capital increased by ∼8–9%, and household-level measures of trust were not affected by the program. These findings demonstrate that major environmental conditional cash transfer programs can support both land management and the attitudes and institutions underpinning prosocial behavior. Rigorous empirical research on this question can proceed only country by country because of methodological limitations, but will be an important line of inquiry as PES continues to expand worldwide.
Hard Skills or Soft Talk: Unintended consequences of a vocational training and an inspiration talk on childbearing and sexual behavior in vulnerable youth
Maria Jones, Victor Orozco-Olvera and Ericka Rascon-Ramirez
Hard and soft skills have been shown to be relevant for labour market and educational outcomes, however, little is known about their influence on fertility decision-making and sexual behaviour. This paper analyses to what extent a hard skill (vocational training) and a soft skill (inspirational talk) intervention affected childbearing decisions, HIV testing and transactional sex in young people. Using baseline and follow-up data of a randomised control trial in Malawi, we find that receiving an offer to attend a vocational training programme decreased the chances of becoming a mother and increased the chances of being HIV-tested in both females and males when comparing with the control group. The chances of being demanded for transactional sex decreased for those women receiving the offer to attend the vocational training. Comparing the effects of both interventions between adolescents (under 20) and young adults (20-24), we observe that both ‘hard and soft skills’ reduced the chances of becoming a parent and increased the chances of being HIV-tested in both groups. For transactional sex, we observe opposite effects. While vocational training reduced the chances of being demanded for transactional services, inspirational talks increased the chances of demanding such services. Adolescents who received ‘hard skills’ are less likely to offer transactional sex after the intervention than those in the control group. These results shed light on gender and age differences in the impact of vocational training interventions on non-labour outcomes and on how low-cost ‘soft skills’ interventions, such as inspirational talks, may affect long term outcomes.
Can Environmental Cash Transfers Reduce Deforestation and Improve Social Outcomes? A Regression Discontinuity Analysis of Mexico’s National Program (2011-2014)
Alix-Garcia, Jennifer M., Katharine R. E. Sims, Victor H. Orozco-Olvera, Laura E. Costica, Jorge David Fernández Medina, Sofía Romo Monroy, and Stefano Pagiola
Environmental conditional cash transfers, or “payments for ecosystem services” are a centerpiece of global efforts to protect biodiversity, safeguard watersheds, and mitigate climate change by reducing forest loss. This paper evaluates the impacts of Mexico’s national payments for ecosystem services program, which provides five years of payments to landowners in exchange for maintaining and managing natural land cover. Using a regression discontinuity design, the paper studies impacts on environmental, socioeconomic, and social capital outcomes for the 2011–14 program cohorts. The analysis finds that treated communities increased management activities to protect land cover, such as patrolling for illegal conversion or combating soil erosion (by 48 percent compared to controls). The program reduced the loss of tree cover in areas at high risk of deforestation (by 29 percent compared to controls), with effects being larger for those that have been in the program the longest (38 percent compared to controls).
The effectiveness of using entertainment education narratives to promote safer sexual behaviors of youth: A meta-analysis, 1985-2017
Victor Orozco-Olvera, Fuyuan Shen and Lucie Cluver
Risky sexual behaviors are associated with the transmission of sexually transmitted infections (STIs) and unwanted pregnancies, both major health concerns for youth worldwide. This review studies the effectiveness of narrated mass media programs in promoting safer sexual practices among youth in developed and developing countries. Electronic and manual searches were conducted to identify experimental and quasi-experimental studies with robust counterfactual designs published between 1985 and the first quarter of 2017. Effect sizes were meta-analyzed using mixed-effects models. Eight experimental and two quasi-experimental studies met our inclusion criteria. The aggregated sample size was 23,476 participants, with a median of 902 participants per study. Entertainment education narratives had small but significant effects for three sexual behaviors. It reduced the number of sexual partners [standardized mean difference, (SMD) = 0.17, 95% confidence interval (CI) = 0.02–0.33, three effect sizes], reduced unprotected sex (SMD = 0.08, 95% CI = 0.03–0.12, nine effect sizes), and increased testing and management for STIs (SMD = 0.29, 95% CI = 0.11–0.46, two effect sizes). The interventions were not effective in reducing inter-generational sex, measured through the age-gap with sexual partners (SMD = 0.06, 95% CI = -0.06–0.19, four effect sizes). Entertainment education had medium-size effects on knowledge outcomes (SMD = 0.67, 95% CI = 0.32–1.02, seven effect sizes), where a time-decay relationship is observed. No effects were found on attitudes. Although mass media entertainment had small effects in promoting safer sexual practices, its economies of scales over face-to-face interventions suggest its potential to be a cost-effective tool above an audience threshold. The use of study participants from the general youth population and the use of mostly effectiveness trials mitigate concerns regarding its scalability. The overall paucity of high-quality studies affirms the need for strengthening the evidence base of entertainment education. Future research should be undertaken to understand the moderator effects for different subgroups and intervention characteristics.
Gender Differences in the Effects of Vocational Training in Malawi: Constraints on Women and Drop-Out Behavior
Y. Cho, D. Kalomba, M. Mobarak, V. Orozco-Olvera and D. Wolfson
We provide experimental evidence on the effects of vocational and entrepreneurial training for Malawian youth in an environment where access to schooling and formal sector employment is extremely low. The training results in skills development, continued investment in human capital and improved well-being, with more positive effects for men, but no improvements in labor market outcomes in the short-run. We find that women make decisions in a more constrained environment, and their participation is affected by family obligations. We also find that participation is more expensive for women compared to their male counterparts, resulting in a worse training experience. Additionally, we track a large fraction of program dropouts – a common phenomenon in the training evaluation literature – which allows us to examine the determinants and consequences of the dropout decision and how it mediates the effects of such programs.
Moving from efficacy to effectiveness: Using behavioral economics to improve the impact of water, sanitation and hygiene interventions
Aidan Coville and Victor Orozco-Olvera
This paper explores cognitive and behavioural constraints identified in the behavioural economics literature and how interventions have successfully accounted for these constraints in their design to increase demand for services and positive habit formation.
Our edutainment partners at Impact(Ed) International and Fundi Films receiving an International Emmy Award for their locally produced African program My Better World. Our studies show that community screenings of the series in northern Nigeria significantly increased school attendance and reduced teenage marriage and teenage parenthood at both the 1-year and 3-year follow-ups.
Google Sites
Report abuse