Day 2

Estimating and Evaluating Treatment Effect Heterogeneity: A Causal Forest Approach

Abstract

In this paper, we introduce the causal forest algorithm (Athey et al., 2019) and its possible applications in social sciences to address treatment effect heterogeneity. Compared with existing methods such as the multiplicative interaction model and traditional semi-/non-parametric estimation, causal forests are more flexible for complex data generating process and more powerful in handling high dimensional data. Specifically, causal forests admit estimation and inference on heterogeneous treatment effects in the presence of many moderators. To reveal its usefulness, we revisit three studies in political science and economics which argue for a constant treatment effect, treatment effects conditional on one moderator, and negligible treatment effects, respectively. Based on causal forests estimations, we provide various evaluations of treatment effect heterogeneity. We uncover new information from this heterogeneity hidden by original estimation strategies while produce findings that are consistent with conventional methods.

Causal Inference under Temporal and Spatial Interference

Abstract

Many social events and policies generate spillover effects in both time and space. Their occurrence influences not only the outcomes of interest in the future, but also these outcomes in nearby areas. In this paper, we propose a design-based approach to estimate the direct and indirect/spillover treatment effects of any event or policy under the assumption of sequential ignorability, when both temporal and spatial interference are present. The proposed estimators are shown to be consistent and normally distributed if the degree of interference dependence does not grow too fast relative to the sample size. The conventional difference-in-differences (DID) or two-way fixed effects model, nevertheless, leads to biased estimates in this scenario. We apply the method to examine the impact of Hong Kong’s Umbrella Movement on the result of the ensuing election and how an institutional reform affects real estate assessment in New York State.

Government Data Manipulation and Public Support: Evidence from A Survey Experiment in China

Abstract

Governments enact a broad array of policies that affect the economy, influencing citizens’ welfare. They also produce and disseminate information about these issues to citizens and, therefore, can lie through the provision of falsified data. We study how the glowing manipulated official economic statistics and the discovery of data manipulation influence citizens’ view of the government by a survey experiment in China that employs the real event of growth falsification by a local government. We do not find any persuasion effect of the manipulated data on evaluations of the economy or the government. However, people who have lower prior economic confidence reduce their trust and satisfaction in government. Therefore, the backfire of data manipulation is through losing trust rather than economic confidence. Furthermore, bridging the literature on statistics learning and economic voting, we provide evidence that the lack of persuasion effect is due to not linking economic performance to the data rather than ignoring the data.

Evo Morales and Electoral Fraud in Bolivia: A Natural Experiment and Discontinuity Evidence

Abstract

This paper uses a unique data set and a natural experiment based on the shutdown in the official preliminary vote counting system to identify and estimate the size of electoral fraud in the 2019 Bolivian presidential elections. The 2016 Constitutional Referendum and the participation of other political parties serve as controls to estimate various difference-in-differences and difference-in-difference-in-differences specifications. The results show evidence of a statistically significant electoral case of fraud that increased the votes of the incumbent Movimiento al Socialismo and decreased the votes of the runner up Comunidad Ciudadana. We estimate that the extent of the fraud is 2.50% of valid votes, sufficient to change the outcome of the election. We report a break in trend and evidence of fraud beyond the shutdown. Our results are robust to polling-station-level shocks common across 2019 and 2016, as well as 2019 specific shocks. This controls for geography (e.g., rural vs. urban), unobserved voting preferences, voter’s last names, and endogeneity in the arrival of the polling stations. We document a statistically significant discontinuous jump in the gap between the incumbent and the runner up during the shutdown.

Electoral Systems and the Substantive Representation of Marginalized Groups: Evidence from Women's Issue Bills in South Korea

Abstract

How do electoral rules shape the substantive representation of traditionally under-represented groups? Using an original dataset of introduced and passed bills in the Korean National Assembly, which has both single-member districts and proportional representation, we examine the extent to which institutions condition the relationship between lawmaker gender and the substantive representation of women. While women lawmakers engage in higher levels of substantive representation of women, proportional representation allows both women and men to introduce more women’s issue bills than their counterparts elected through single-member districts. Furthermore, legislators elected through proportional representation are more effective at achieving passage of women’s issue legislation when compared to those elected in single-member districts, and this effect is especially pronounced for men. Our findings show that electoral systems matter for the representation of marginalized groups and that proportional representation systems allow both female and male politicians to increase their substantive representation of women.

Unraveling Diversionary Tactics: Diversionary Utility and the Use of Verbal Aggression

Abstract

There has been a controversy over the use of diversionary tactics in international relations. Conventional wisdom states that countries are more likely to divert from domestic unrest by seeking outside enemies and initiating wars internationally. However, several scholars have pointed out that this is not empirically verified, as there are limited or insignificant associations between domestic unrest and whether states initiate militarized disputes. This study suggests that countries seek instead to divert domestic grievances to international targets via non-violent options. We argue that countries use non-violent, verbal aggression to divert public attention from domestic instability, avoiding politically costly military options. Using a multivariate multilevel analysis, we retrieved and examined verbally and physically aggressive actions from the Global Database of Events, Language, and Tone (GDELT) taken by countries such as threats, diplomatic relations with targeted countries, and the use of conventional weapons. When there is domestic instability due to security concerns, political leaders use both physically and verbally motivated aggression to target rivals and mobilize support for the incumbent government. Democratic regimes are likely to use both types of diversions, while mixed regimes are least likely to use any type of diversionary actions.

Using Causal Mediation Analysis for Policy Evaluation: Detecting, Estimating, and Comparing Direct and Indirect Effects

Abstract

Causal mediation analysis has become a popular tool to identify how a specified factor can moderate or mediate the effect of a cause on the outcome of interest. Recent innovations in biostatistics seek to unpack a causal effect into different types of direct and indirect effects through the mediator. We apply the framework of effect decomposition to the context of policy evaluation by considering policy interventions, undertaken in the face of major political and socioeconomic events, as the mediator rather than the treatment. By comparing different direct and indirect effects, we seek to understand how the impact of these events can change through the policy and institutional response. We provide an estimation strategy that is easy to implement and demonstrate the whole process with four empirical applications from the American and comparative politics literatures. These examples examine whether the Chinese government’s patriotic education facilities can mitigate the negative impact on political trust in China.

Double regression with post-stratification (DRP) for analyzing high-dimensional survey data

Abstract

An important challenge in modern survey research is to find calibrated weights when covariates are high dimensional and especially when deep interactions are important. Traditional approaches, like raking, can perform poorly in this setting, typically balancing a small number of marginal distributions while failing to balance higher-order interactions. In this paper, we propose a class of generalized regression estimation that combines calibrated weights with a (multilevel) outcome model. We first construct an approximate calibration weighting estimator that enforces tight balance constraints for marginal balance and looser constraints for higher-order interactions; we then correct for the bias due to the relaxed constraints via an outcome model. This bias-corrected estimator is driven primarily by the weights where data are plentiful, relying instead on the outcome model when extrapolation is necessary. We also show that the approximate calibration estimator has a dual representation as a multilevel model for survey response. Thus, we view our approach as a generalization of standard Multilevel Regression with Post-Stratification (MRP). Since we also allow for a multilevel model for the weights we refer to our proposal as Double Regression with Post-Stratification (DRP). We assess the performance of this method with extensive simulation studies and apply it to a recent large-scale survey of political attitudes.

A Practical Guide to Counterfactual Estimators for Causal Inference with Time-Series Cross-Sectional Data

Abstract

This paper introduces a unified framework of counterfactual estimation for time-series cross-sectional data, which estimates the average treatment effect on the treated by directly imputing treated counterfactuals. Its special cases include several newly developed methods, such as the fixed effects counterfactual estimator, interactive fixed effects counterfactual estimator, and matrix completion estimator. These estimators provide more reliable causal estimates than conventional two-way fixed effects models when the treatment effects are heterogeneous or unobserved time-varying confounders exist. Under this framework, we propose two sets of diagnostic tests, tests for (no) pre-trend and placebo tests, accompanied by visualization tools, to help researchers gauge the validity of the no-time-varying-confounder assumption. We illustrate these methods with two political economy examples and develop an open-source package, fect, in both R and Stata to facilitate implementation.

The Effects of Land Tenure Security on Agricultural and Environmental Outcomes in Benin: New Evidence from Satellite Data

Abstract

Satellite imagery offers researchers the unprecedented ability to measure outcomes on the ground at high spatial and temporal frequency. This paper develops and validates a set of methods which take advantage of spectral, temporal, and spatial variation to measure outcomes and infer unobserved outcomes. I show how traditional time series methods as well as neural networks with recurrent and convolutional structures can accurately classify pixels and objects. I show how these measurement strategies naturally fit into cutting edge causal inference methods for very high-dimensional data. I use double machine learning methods to model treatment propensity and outcomes as functions of geophysical and human processes. Combined with the enormous amount of data encoded in the satellite record these methods allow researchers to measure causal relationships in settings where experiments are impossible. Finally I demonstrate these methods by testing whether a land-titling reform program in Benin resulted in productivity-enhancing improvements and changes to nearby ecosystems.

Cause, Not Effect: Partisan Identification and Voter Turnout

Variance of Average Marginal Effect in Conjoint Analysis: VMCE

People Write Like their Friends: Improving Neural Text Classification with Author Node Embeddings

Indonesian Settler Colonialism in West Papua

Is Islam Incompatible with Democracy? A Disaggregated Study Based on Survey Data

Image Clustering: An Unsupervised Approach to Categorize Visual Data in Social Science Research

Abstract

Automated image analysis has received increasing attention in social scientific research, yet existing scholarship has focused on the application of supervised machine learning to classify images into pre-defined categories. This study focuses on the task of unsupervised image clustering that automatically finds categories from image data. We first review the steps to perform image clustering and argue that one key challenge is to find low-dimensional representations of images. We then focus on several methods of extracting low-dimensional representations of images, including self-supervised learning from scratches and transfer learning. We compare these methods using two datasets containing images related to protests on Weibo and posts about climate change on Instagram. Results show that transfer learning significantly outperforms learning image representations from scratches as well as traditional feature extraction algorithms. Regarding transfer learning, the dataset in the pre-trained model critically determines what categories we can discover.

Factionalism and Red Guards under the Cultural Revolution: Ideal Point Estimation Using Text-as-Data Scaling Method

Abstract

This article estimates the spatial positioning of the political elites and participants in the Cultural Revolution through analyzing expressed political views in propaganda publication in China. Prior theoretical research indicates that the political elites and the Red Guards lost control of the social movement, and it evolved into sheer verbal violence and physical skirmishes across all provinces in the end (MacFarquhar and Schoenhals 2006). We show that in this political chaos, the Red Guards and the elite were dynamically fractioned, as is reflected in self-printed propaganda, such as big-character posters (Dazibao 大字报) and tabloids (Xiaobao 小报). This paper also develops a novel approach that augments TextRank algorithm to extract keywords and Chinese phrases, on top of the Poisson scaling model (Wordfish) to estimate the differences of spatial positions using the extracted textual key terms. Results are shown to be consistent with the literature.

Understanding Hong Kong Nationalism with Topic Network

Abstract

This paper seeks to understand the online discourse of Hong Kong nationalism using computational text analysis. Facebook is selected as the data source due to its popularity and key roles in Hong Kong political movements; all posts on the key nationalist organisations’ Facebook page dated between January 2015 and January 2018 are analysed. Latent Dirichlet allocation topic modelling is used to identify the recurring themes within the corpora. Instead of treating the topics with the highest quantity as the key topics. This work considers political discourse a network phenomenon and adopts a novel method that transforms the association between topics into a topic network and uses centrality measures from social network analysis to identity core topics in the topic model. The analysis shows that early Hong Kong nationalism discourse began as a branch of localism and adopted the rhetoric of nationalism in 2016. This work also reveals the dynamics of topic prominence throughout the study period.

Building Longitudinal Google Trends to Measure

Abstract

Google search indices can be useful for measuring dynamic local-level attention to public issues for which survey data are rarely available. However, there is a practical difficulty with generating longitudinal Google Trends. Google Trends provides normalized counts from zero to 100 instead of absolute counts, thereby placing its cross-sectional indices across different times on different scales. Thus, merely pooling cross-sectional data fails to create desirable longitudinal data. We develop a method for rescaling Google Trends indices to build longitudinal data at the United States media market level. We illustrate this method with applications to the issues of employment and the coronavirus. This new tool opens the door to using Google searches merged with various kinds of time-series cross-sectional data, which has not been possible.

Magnified Coercion: the Effect of Sanctions on Nuclear States’ Crisis Outcomes

Abstract

The efficacy of nuclear weapons in compellent threats has long been the center of the debate regarding coercive diplomacy. Nevertheless, the imposition of sanctions in nuclear crises has received little scholarly interest. This article argues that the imposition of sanctions increases credibility of the nuclear threat by escalating the crisis with actual costs, which addressed the nuclear superior states’ advantage in risk-taking. Using a purposefully created data set covering 52 symmetric nuclear dyad cases and 133 asymmetric nuclear dyad cases, this article conducts empirical analysis to examine whether the nuclear state’s compellent threats enjoy magnified coercive effect in conjunction with sanctions, and how that effect differs in symmetric and asymmetric nuclear dyads. The empirical findings suggest that the imposition of sanctions significantly enhances the effect of nuclear threats for nuclear superior states, improving the probability of winning by more than 20%. However, the magnifying effect does not extend to asymmetric nuclear dyads. This article further illustrates how the sanctions address the nuclear weapons’ coercive effect in the complex real political environment through the qualitative case study of the DPRK denuclearization.

What Environmental Issues do Elites Communicate and Why? Automated Text Analysis of Italian Environmental Executives’ Speeches

Abstract

Climate change and environmental concerns represent one of the biggest and longest looming issues on the societal horizon and represent salient issues. While a rich literature analyses on agenda-setting provides clues on when are environmental issues discussed in legislative speeches, relatively little is known how political executives communicate this specific policy issues to wider public. Drawing on topic modelling analyses of 2500 press releases from Italian Ministry of Environment (2008–2020), the article focuses on the contents of environmental policy communication and tests whether executives are more likely to make statements on issues important to the government coalition parties (issue ownership theory). Political executives should also emphasize issues of general public importance (riding the wave theory) and issues relating to focusing events (short-term crisis management) in the field. They might also decide to emphasize own solutions found for the environmental issues (management capacity). I find empirical support for issue ownership and management capacity expectations.