In many social science applications, researchers use the difference-in-differences (DID) estimator to establish causal relationships, exploiting cross-sectional variation in a baseline factor and temporal variation in exposure to an event that presumably may affect all units. This approach, which we term factorial DID (FDID), differs from canonical DID in that it lacks a clean control group unexposed to the event after the event occurs. In this paper, we clarify FDID as a research design in terms of its data structure, feasible estimands, and identifying assumptions that allow the DID estimator to recover these estimands. We frame FDID as a factorial design with two factors: the baseline factor, denoted by G, and the exposure level to the event, denoted by Z, and define the effect modification and causal interaction as the associative and causal effects of G on the effect of Z, respectively. We show that under the canonical no anticipation and parallel trends assumptions, the DID estimator identifies only the effect modification of G in FDID, and propose an additional factorial parallel trends assumption to identify the causal interaction. Moreover, we show that the canonical DID research design can be reframed as a special case of the FDID research design with an additional exclusion restriction assumption, thereby reconciling the two approaches. We extend this framework to allow conditionally valid parallel trends assumptions and multiple time periods, and clarify assumptions required to justify regression analysis under FDID. We illustrate these findings with empirical examples from economics and political science, and provide recommendations for improving practice and interpretation under FDID.
The rise of social media in the digital era poses unprecedented challenges to authoritarian regimes that aim to influence public attitudes and behaviors. In this paper, we argue that authoritarian regimes have adopted a decentralized approach to producing and disseminating propaganda on social media. In this model, tens of thousands of government workers and insiders are mobilized to produce and disseminate propaganda, and content flows in a multi-directional, rather than a top-down manner. We empirically demonstrate the existence of this new model in China by creating a novel dataset of over five million videos from over 18,000 regime-affiliated accounts on Douyin, the Chinese branding for TikTok. This paper supplements prevailing understandings of propaganda by showing theoretically and empirically how digital technologies are changing not only the content of propaganda, but also the way in which propaganda materials are produced and disseminated.
In 1986, Robert LaLonde published an article that compared nonexperimental estimates to experimental benchmarks (LaLonde 1986). He concluded that the nonexperimental methods at the time could not systematically replicate experimental benchmarks, casting doubt on the credibility of these methods. Following LaLonde's critical assessment, there have been significant methodological advances and practical changes, including (i) an emphasis on estimators based on unconfoundedness, (ii) a focus on the importance of overlap in covariate distributions, (iii) the introduction of propensity score-based methods leading to doubly robust estimators, (iv) a greater emphasis on validation exercises to bolster research credibility, and (v) methods for estimating and exploiting treatment effect heterogeneity. To demonstrate the practical lessons from these advances, we reexamine the LaLonde data and the Imbens-Rubin-Sacerdote lottery data. We show that modern methods, when applied in contexts with significant covariate overlap, yield robust estimates for the adjusted differences between the treatment and control groups. However, this does not mean that these estimates are valid. To assess their credibility, validation exercises (such as placebo tests) are essential, whereas goodness of fit tests alone are inadequate. Our findings highlight the importance of closely examining the assignment process, carefully inspecting overlap, and conducting validation exercises when analyzing causal effects with nonexperimental data.
Two-way fixed effects (TWFE) models are widely used for causal panel analysis in political science. However, recent methodological advancements cast doubt on the validity of the TWFE estimator in the presence of heterogeneous treatment effects (HTE) and violations of the parallel trends assumption (PTA). In response, researchers have proposed a variety of novel estimators and testing procedures. Our study aims to gauge the effects of these issues on empirical political science research and evaluate the practical performance of these new proposals. We undertake a review, replication, and reanalysis of 38 papers recently published in the American Political Science Review, American Journal of Political Science, and Journal of Politics that use observational panel data with binary treatments. Through both visual and statistical tests, we discover that the use of HTE-robust estimators, while potentially impacting precision, generally does not alter the substantive conclusions originally drawn from TWFE estimates. However, violations of the PTA and lack of sufficient statistical power persist as the primary challenges to credible inference. In light of these findings, we put forth recommendations to assist empirical researchers to improve practices.
Why do authoritarian regimes like China devote resources to gathering information about the policy preferences of the public? We argue that if the public exhibits stable, multi-dimensional preferences, authoritarian regimes have incentives to gather information about these preferences in order to divide and rule and guard against elite competition. By conducting two online, longitudinal surveys, we show that in China, policy preferences are indeed multi-dimensional, stable, and predictable over time. We use multiple items to measure preferences in six policy domains: political liberalism, market economy, nationalism, traditionalism, social equality, and ethnic accommodation. We observe only moderate correlations between political liberalism, market economy, nationalism, and traditionalism, which implies multi-dimensionality. Preferences for all issue areas except social equality exhibit relatively high levels of intertemporal stability and predictability over a one-month period comparable to what is seen in competitive electoral democracies. Those with higher levels of education and political knowledge have more stable preferences. We provide guidance on how to measure these relatively coherent and temporally stable preferences, or ideology, using surveys.
The cross-border flow of people for educational exchange in Western democracies is seen as a way to transfer democratic values to non-democratic regions of the world. What happens when students studying in the West encounter discrimination? Based on an experiment among hundreds of Chinese first-year undergraduates in the United States, we show that discrimination interferes with the transfer of democratic values. Chinese students who study in the United States are more predisposed to favor liberal democracy than their peers in China. However, anti-Chinese discrimination significantly reduces their belief that political reform is desirable for China and increases their support for authoritarian rule. These effects of discrimination are most pronounced among students who are more likely to reject Chinese nationalism. Encountering non-racist criticisms of the Chinese government does not increase support for authoritarianism. Our results are not explained by relative evaluations of US and Chinese government handling of covid-19. Media Coverage: Newsweek, Politico, Monkey Cage, South China Morning Post, The Economist
We introduce trajectory balancing, a general reweighting approach to causal inference with time-series cross-sectional (TSCS) data. We focus on settings in which one or more units is exposed to treatment at a given time, while a set of control units remain untreated throughout a time window of interest. First, we show that many commonly used TSCS methods imply an assumption that a unit's non-treatment potential outcomes in the post-treatment period are linear in that unit's pre-treatment outcomes as well as time-invariant covariates. Under this assumption, we introduce the mean balancing method that reweights the control units such that the averages of the pre-treatment outcomes and covariates are approximately equal between the treatment and (reweighted) control groups. Second, we relax the linearity assumption and propose the kernel balancing method that seeks an approximate balance on a kernel-based feature expansion of the pre-treatment outcomes and covariates. The resulting approach inherits the property of handling time-vary confounders as in synthetic control and latent factor models, but has the advantages of: (1) improving feasibility and stability with reduced user discretion compared to existing approaches; (2) accommodating both short and long pre-treatment time periods with many or few treated units; and (3) achieving balance on the high-order ``trajectory" of pre-treatment outcomes rather than their simple average at each time period. We illustrate this method with simulations and two empirical examples.
This paper takes a novel perspective on the selection of leaders in the Chinese Communist Party (CCP) by focusing on career tracks of high-level CCP officials. Career tracks are defined as clusters of similar career trajectories with respect to both vertical and horizontal movements. We measure career tracks of full and alternate CCP Central Committee members from 1982 to 2017 using machine learning techniques and investigate their role in political selection. We find that the career tracks corresponding to high starting positions and frequent rotations are associated with significantly higher probabilities of obtaining top leadership positions in the CCP despite the influence of patronage networks. Moreover, when comparing the relative importance of career tracks and personal connections over time, we find suggestive evidence that the Chinese political system has become more personalistic in the past few years.
In many social science applications, researchers use the difference-in-differences (DID) estimator to establish causal relationships, exploiting cross-sectional variation in a baseline factor and temporal variation in exposure to an event that presumably may affect all units. This approach, which we term factorial DID (FDID), differs from canonical DID in that it lacks a clean control group unexposed to the event after the event occurs. In this paper, we clarify FDID as a research design in terms of its data structure, feasible estimands, and identifying assumptions that allow the DID estimator to recover these estimands. We frame FDID as a factorial design with two factors: the baseline factor, denoted by G, and the exposure level to the event, denoted by Z, and define the effect modification and causal interaction as the associative and causal effects of G on the effect of Z, respectively. We show that under the canonical no anticipation and parallel trends assumptions, the DID estimator identifies only the effect modification of G in FDID, and propose an additional factorial parallel trends assumption to identify the causal interaction. Moreover, we show that the canonical DID research design can be reframed as a special case of the FDID research design with an additional exclusion restriction assumption, thereby reconciling the two approaches. We extend this framework to allow conditionally valid parallel trends assumptions and multiple time periods, and clarify assumptions required to justify regression analysis under FDID. We illustrate these findings with empirical examples from economics and political science, and provide recommendations for improving practice and interpretation under FDID.
In 1986, Robert LaLonde published an article that compared nonexperimental estimates to experimental benchmarks (LaLonde 1986). He concluded that the nonexperimental methods at the time could not systematically replicate experimental benchmarks, casting doubt on the credibility of these methods. Following LaLonde's critical assessment, there have been significant methodological advances and practical changes, including (i) an emphasis on estimators based on unconfoundedness, (ii) a focus on the importance of overlap in covariate distributions, (iii) the introduction of propensity score-based methods leading to doubly robust estimators, (iv) a greater emphasis on validation exercises to bolster research credibility, and (v) methods for estimating and exploiting treatment effect heterogeneity. To demonstrate the practical lessons from these advances, we reexamine the LaLonde data and the Imbens-Rubin-Sacerdote lottery data. We show that modern methods, when applied in contexts with significant covariate overlap, yield robust estimates for the adjusted differences between the treatment and control groups. However, this does not mean that these estimates are valid. To assess their credibility, validation exercises (such as placebo tests) are essential, whereas goodness of fit tests alone are inadequate. Our findings highlight the importance of closely examining the assignment process, carefully inspecting overlap, and conducting validation exercises when analyzing causal effects with nonexperimental data.
Two-way fixed effects (TWFE) models are widely used for causal panel analysis in political science. However, recent methodological advancements cast doubt on the validity of the TWFE estimator in the presence of heterogeneous treatment effects (HTE) and violations of the parallel trends assumption (PTA). In response, researchers have proposed a variety of novel estimators and testing procedures. Our study aims to gauge the effects of these issues on empirical political science research and evaluate the practical performance of these new proposals. We undertake a review, replication, and reanalysis of 38 papers recently published in the American Political Science Review, American Journal of Political Science, and Journal of Politics that use observational panel data with binary treatments. Through both visual and statistical tests, we discover that the use of HTE-robust estimators, while potentially impacting precision, generally does not alter the substantive conclusions originally drawn from TWFE estimates. However, violations of the PTA and lack of sufficient statistical power persist as the primary challenges to credible inference. In light of these findings, we put forth recommendations to assist empirical researchers to improve practices.
Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010-2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, the commonly used t-test for two-stage-least-squares (2SLS) estimates frequently underestimates uncertainties, resulting in uncontrolled Type-I errors in many studies. Third, in most replicated studies, 2SLS estimates are significantly larger than ordinary-least-squares estimates, with their ratio negatively correlated with instrument strength in studies with non-experimentally generated instruments, suggesting potential violations of unconfoundedness or the exclusion restriction. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice.
We introduce trajectory balancing, a general reweighting approach to causal inference with time-series cross-sectional (TSCS) data. We focus on settings in which one or more units is exposed to treatment at a given time, while a set of control units remain untreated throughout a time window of interest. First, we show that many commonly used TSCS methods imply an assumption that a unit's non-treatment potential outcomes in the post-treatment period are linear in that unit's pre-treatment outcomes as well as time-invariant covariates. Under this assumption, we introduce the mean balancing method that reweights the control units such that the averages of the pre-treatment outcomes and covariates are approximately equal between the treatment and (reweighted) control groups. Second, we relax the linearity assumption and propose the kernel balancing method that seeks an approximate balance on a kernel-based feature expansion of the pre-treatment outcomes and covariates. The resulting approach inherits the property of handling time-vary confounders as in synthetic control and latent factor models, but has the advantages of: (1) improving feasibility and stability with reduced user discretion compared to existing approaches; (2) accommodating both short and long pre-treatment time periods with many or few treated units; and (3) achieving balance on the high-order ``trajectory" of pre-treatment outcomes rather than their simple average at each time period. We illustrate this method with simulations and two empirical examples.
The rise of social media in the digital era poses unprecedented challenges to authoritarian regimes that aim to influence public attitudes and behaviors. In this paper, we argue that authoritarian regimes have adopted a decentralized approach to producing and disseminating propaganda on social media. In this model, tens of thousands of government workers and insiders are mobilized to produce and disseminate propaganda, and content flows in a multi-directional, rather than a top-down manner. We empirically demonstrate the existence of this new model in China by creating a novel dataset of over five million videos from over 18,000 regime-affiliated accounts on Douyin, the Chinese branding for TikTok. This paper supplements prevailing understandings of propaganda by showing theoretically and empirically how digital technologies are changing not only the content of propaganda, but also the way in which propaganda materials are produced and disseminated.
Why do authoritarian regimes like China devote resources to gathering information about the policy preferences of the public? We argue that if the public exhibits stable, multi-dimensional preferences, authoritarian regimes have incentives to gather information about these preferences in order to divide and rule and guard against elite competition. By conducting two online, longitudinal surveys, we show that in China, policy preferences are indeed multi-dimensional, stable, and predictable over time. We use multiple items to measure preferences in six policy domains: political liberalism, market economy, nationalism, traditionalism, social equality, and ethnic accommodation. We observe only moderate correlations between political liberalism, market economy, nationalism, and traditionalism, which implies multi-dimensionality. Preferences for all issue areas except social equality exhibit relatively high levels of intertemporal stability and predictability over a one-month period comparable to what is seen in competitive electoral democracies. Those with higher levels of education and political knowledge have more stable preferences. We provide guidance on how to measure these relatively coherent and temporally stable preferences, or ideology, using surveys.
The cross-border flow of people for educational exchange in Western democracies is seen as a way to transfer democratic values to non-democratic regions of the world. What happens when students studying in the West encounter discrimination? Based on an experiment among hundreds of Chinese first-year undergraduates in the United States, we show that discrimination interferes with the transfer of democratic values. Chinese students who study in the United States are more predisposed to favor liberal democracy than their peers in China. However, anti-Chinese discrimination significantly reduces their belief that political reform is desirable for China and increases their support for authoritarian rule. These effects of discrimination are most pronounced among students who are more likely to reject Chinese nationalism. Encountering non-racist criticisms of the Chinese government does not increase support for authoritarianism. Our results are not explained by relative evaluations of US and Chinese government handling of covid-19. Media Coverage: Newsweek, Politico, Monkey Cage, South China Morning Post, The Economist
This paper takes a novel perspective on the selection of leaders in the Chinese Communist Party (CCP) by focusing on career tracks of high-level CCP officials. Career tracks are defined as clusters of similar career trajectories with respect to both vertical and horizontal movements. We measure career tracks of full and alternate CCP Central Committee members from 1982 to 2017 using machine learning techniques and investigate their role in political selection. We find that the career tracks corresponding to high starting positions and frequent rotations are associated with significantly higher probabilities of obtaining top leadership positions in the CCP despite the influence of patronage networks. Moreover, when comparing the relative importance of career tracks and personal connections over time, we find suggestive evidence that the Chinese political system has become more personalistic in the past few years.
This article studies whether “pure” legality, stripped of the normative components that are conceptually necessary for “the rule of law,” can convey meaningful amounts of perceived legitimacy to governmental institutions and activity. Through a survey experiment conducted among urban Chinese residents, it examines whether such conveyance is possible under current Chinese sociopolitical conditions, in which the Party-state continues to invest heavily in “pure legality,” but without imposing meaningful legal checks on the Party leadership’s political power, and without corresponding investment in substantive civil rights or socioeconomic freedoms. Among survey respondents, government investment in legality conveys meaningful amounts of political legitimacy, even when it is applied to actions, such as online speech censorship, that are socially controversial or unattractive, and even when such investment does not clearly enhance the predictability of state behavior. However, the legitimacy-enhancing effects of legality are likely weaker than those of state investment in procedural justice.
Why do authoritarian regimes charge political opponents with non-political crimes when they can levy charges directly related to opponents' political activism? We argue that doing so disguises political repression and undermines the moral authority of opponents, minimizing backlash and mobilization. To test this argument, we conduct a survey experiment, which shows that disguised repression decreases perceptions of dissidents' morality, decreases people's willingness to engage in dissent on behalf of the dissident, and increases support for repression of the dissident. We then assess the external validity of the argument by analyzing millions of Chinese social media posts made before and after a large crackdown of vocal government critics in China in 2013. We find that individuals with larger online followings are more likely to be charged with non-political crimes, and those charged with non-political crimes are less likely to receive public sympathy and support.
Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010-2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, the commonly used t-test for two-stage-least-squares (2SLS) estimates frequently underestimates uncertainties, resulting in uncontrolled Type-I errors in many studies. Third, in most replicated studies, 2SLS estimates are significantly larger than ordinary-least-squares estimates, with their ratio negatively correlated with instrument strength in studies with non-experimentally generated instruments, suggesting potential violations of unconfoundedness or the exclusion restriction. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice.
We develop an R package panelView and a Stata package panelview for panel data visualization. They are designed to assist causal analysis with panel data and have three main functionalities: (1) they plot the treatment status and missing values in a panel dataset; (2) they visualize the temporal dynamics of the main variables of interest; and (3) they depict the bivariate relationships between a treatment variable and an outcome variable either by unit or in aggregate. These tools can help researchers better understand their panel datasets before conducting statistical analysis.
We study the effect of a victimization narrative on national identity and public opinion in China experimentally. Previous research has suggested that governments can shape public opinion by guiding citizens' collective memories of historical events, but few studies have established a clear causal link. By conducting an online survey experiment among approximately 2,000 urban Chinese citizens, we examine the causal impact of historical narratives on political attitudes. We find that, compared to control conditions, a narrative focusing on China’s humiliating past in the late Qing significantly reinforces respondents' attachment to the victim side of the Chinese national identity, raises suspicion of the intention of foreign governments in international disputes, stimulates preference for more hawkish foreign policies, and strengthens support for China's current political system. These effects are particularly strong among respondents without a college degree.
This paper introduces a unified framework of counterfactual estimation for time-series cross-sectional data, which estimates the average treatment effect on the treated by directly imputing treated counterfactuals. Examples include the fixed effects counterfactual estimator, interactive fixed effects counterfactual estimator, and matrix completion estimator. These estimators provide more reliable causal estimates than conventional twoway fixed effects models when treatment effects are heterogeneous or unobserved time-varying confounders exist. Under this framework, we propose a new dynamic treatment effects plot, as well as several diagnostic tests, to help researchers gauge the validity of the identifying assumptions. We illustrate these methods with two political economy examples and develop an open-source package, fect, in both R and Stata to facilitate implementation.
We introduce hierarchically regularized entropy balancing as an extension to entropy balancing, a reweighting method that adjusts weights for control group units to achieve covariate balance in observational studies with binary treatments. Our proposed extension expands the feature space by including higher-order terms (and interactions) of covariates and then achieves approximate balance on the expanded features. To prioritize balance on variables important for treatment assignment and prevent overfitting, the method imposes ridge penalties with a hierarchical structure on the higher-order terms. Compared with entropy balancing, this extension relaxes model dependency and improves robustness of causal estimates while avoiding optimization failure and highly concentrated weights. It prevents specification searches by minimizing user discretion in selecting features to balance on. Our method is also computationally more efficient than kernel balancing, a kernel-based covariate balancing method. We demonstrate its performance through simulations and an empirical example.
We introduce Bayesian Rule Set (BRS) as an alternative to Qualitative Comparative Analysis (QCA) when data are large and noisy. BRS is an interpretable machine learning algorithm that classifies observations using rule sets, which are conditions connected by logical operators, e.g., IF (condition A AND condition B) OR (condition C), THEN Y~=~TRUE. Like QCA, BRS is highly interpretable and capable of revealing complex nonlinear relationships in data. It also has several advantages over QCA: It is compatible with probabilistically generated data; it avoids overfitting and improves interpretability by making direct trade-offs between in-sample fitness and complexity; and it remains computationally efficient with many covariates. Our contributions are threefold: We modify the BRS algorithm to facilitate its usage in the social sciences, propose methods to quantify uncertainties of rule sets, and develop graphical tools for presenting rule sets. We illustrate these methods with two empirical examples from political science.
Research shows that government-controlled media is an effective tool for authoritarian regimes to shape public opinion. Does government-controlled media remain effective when it is required to support changes in positions that autocrats take on issues? Existing theories do not provide a clear answer to this question, but we often observe authoritarian governments using government media to frame policies in new ways when significant changes in policy positions are required. By conducting an experiment that exposes respondents to government-controlled media---in the form of TV news segments---on issues where the regime substantially changed its policy positions, we find that by framing the same issue differently, government-controlled media moves respondents to adopt policy positions closer to the ones espoused by the regime regardless of individual predisposition. This result holds for domestic and foreign policy issues, for direct and composite measures of attitudes, and persists up to 48 hours after exposure.
This paper proposes a Bayesian alternative to the synthetic control method for comparative case studies with a single or multiple treated units. We adopt a Bayesian posterior predictive approach to Rubin's causal model, which allows researchers to make inferences about both individual and average treatment effects on treated observations based on the empirical posterior distributions of their counterfactuals. The prediction model we develop is a dynamic multilevel model with a latent factor term to correct biases induced by unit-specific time trends. It also considers heterogeneous and dynamic relationships between covariates and the outcome, thus improving precision of the causal estimates. To reduce model dependency, we adopt a Bayesian shrinkage method for model searching and factor selection. Monte Carlo exercises demonstrate that our method produces more precise causal estimates than existing approaches and achieves correct frequentist coverage rates even when sample sizes are small and rich heterogeneities are present in data. We illustrate the method with two empirical examples from political economy.
When COVID-19 first emerged in China, there was speculation that the outbreak would trigger public anger and weaken the Chinese regime. By analyzing millions of social media posts from Sina Weibo made between December 2019 and February 2020, we describe the content and sentiment of public, online discussions pertaining to COVID-19 in China. We find that discussions of COVID-19 became widespread on January 20, 2020, consisting primarily of personal reflections, opinion, updates, and appeals. We find that the largest bursts of discussion coincide with the January 23 lockdown of Wuhan and the February 7 death of Dr. Li Wenliang and contain simultaneous spikes of criticism and support targeting the Chinese government. Criticisms are directed at the government for perceived lack of action, incompetence, and wrongdoing—in particular, censoring information relevant to public welfare. Support is directed at the government for aggressive action and positive outcomes. As the crisis unfolds, the same events are interpreted differently by different people, with those who criticize focusing on the government’s shortcomings and those who praise focusing on the government’s actions.
Multiplicative interaction models are widely used in social science to examine whether the relationship between an outcome and an independent variable changes with a moderating variable. Current empirical practice tends to overlook two important problems. First, these models assume a linear interaction effect that changes at a constant rate with the moderator. Second, estimates of the conditional effects of the independent variable can be misleading if there is a lack of common support of the moderator. Replicating 46 interaction effects from 22 recent publications in five top political science journals, we find that these core assumptions often fail in practice, suggesting that a large portion of findings across all political science subfields based on interaction models are modeling artifacts or are at best highly model dependent. We propose a checklist of simple diagnostics to assess the validity of these assumptions and offer flexible estimation strategies that allow for nonlinear interaction effects and safeguard against excessive extrapolation. These statistical routines are available in both R and STATA. -- Awarded the 2020 Miller Prize for "the best work appearing in Political Analysis in 2019."
Although researchers have often considered democracy and state capacity to be key predictors of cross-national variations in human welfare, few have investigated the relationship between the two variables themselves. We argue that democratization may have a positive, causal effect on state capacity. Employing a time-series cross-national dataset from 1960 to 2009, we document that democratization leads to a substantial increase in state capacity in the long run. Our results prove robust to a rich set of potential confounders and alternative coding of key variables. To further address the problem of endogeneity, we use an instrumental variable strategy that exploits exogenous variations in regional democratic diffusions. We also provide suggestive evidence that democratization enhances state capacity through increasing political contestation. -- Awarded the 2015 Malcolm Jewell Award for the best graduate student paper presented at the SPSA annual meeting.
Given widespread perceptions of risk and uncertainty in nondemocratic systems and developing democracies, why do some citizens still take action and make complaints to authorities? The resource mobilization model identifies the importance of time, money, and civic skills as resources that are necessary for participation. In this paper we build on this model and argue that political connections – close personal ties to someone working in government – can also constitute a critical resource, especially in contexts with weak democratic institutions. Using data from both urban and rural China, we find that individuals with political connections are more likely to contact authorities with complaints about government public services, despite the fact that they do not have higher levels of dissatisfaction with public service provision. We conduct various robustness checks, including a sensitivity analysis, and show that this relationship is unlikely to be driven by an incorrect model specification or unobserved confounding variables.
The study of ideology in authoritarian regimes---of how public preferences are configured and constrained---has received relatively little scholarly attention. Using data from a large-scale online survey, we study ideology in China. We find that public preferences are weakly constrained, and the configuration of preferences is multi-dimensional, but the latent traits of these dimensions are highly correlated. Those who prefer authoritarian rule are more likely to support nationalism, state intervention in the economy, and traditional social values; those who prefer democratic institutions and values are less likely to be nationalistic or support traditional social values but more likely to support market reforms. This latter set of preferences appears more in provinces with higher levels of development and among wealthier and better educated respondents. These findings suggest preferences are not simply split along a pro-regime or anti-regime cleavage, and indicate a possible link between China's economic reform and societal cleavages. -- SSRN Top Papers of 2015; Media Coverage: NYT, WSJ, FP, ChinaFile.
How much does it matter which party controls the government? On one hand, campaign positions and roll-call records suggest that contemporary American parties are very ideologically polarized. On the other hand, the existing evidence that electing Democrats into office causes the adoption of more liberal policies is surprisingly weak. We bring clarity to this debate with the aid of a new measure of the policy liberalism of each state in each year 1936-2014, using regression-discontinuity and dynamic panel analyses to estimate the policy effects of the partisan composition of state legislatures and governorships. We find that until the 1980s, partisan control of state government had negligible effects on the liberalism of state policies, but that since then partisan effects have grown markedly. Even today, however, the policy effects of partisan composition remain small relative to differences between states---less than one-tenth of the cross-sectional standard deviation of state policy liberalism. This suggests that campaign positions and roll-call records may overstate the policy effects of partisan selection relative to other factors, such as public opinion.
Why would an authoritarian regime allow citizens to voice opinions publicly if the exchange of information among citizens spurs social instability as has been often alleged? In this paper, we develop a game theoretic model and show that an authoritarian regime can strengthen its rule by allowing citizens to communicate with each other publicly. From the government’s perspective, such communication has two interrelated functions. First, if public communication reveals a shared feeling of dissatisfaction towards government policies among the citizens, the government will detect the danger and improve policies accordingly. Second, and perhaps more interestingly, public communication disorganizes the citizens if they find themselves in disagreement over the policies. We show that the government allows public communication if and only if it perceives sufficient heterogeneity in preferences among the citizens. The model also illustrates that public communication could serve as a commitment device ensuring government responsiveness when it faces high dissatisfaction, which in turn makes the government better off than with private polling.
Difference-in-differences (DID) is commonly used for causal inference in time-series cross-sectional data. It requires the assumption that the average outcomes of treated and control units would have followed parallel paths in the absence of treatment. In this paper, I propose a method that not only relaxes this often-violated assumption, but also unifies the synthetic control method (Abadie, Diamond and Hainmueller 2010) with linear fixed effect models under a simple framework, of which DID is a special case. It imputes counterfactuals for each treated unit in post-treatment periods using control group information based on a linear interactive fixed effect model that incorporates unit-specific intercepts interacted with time-varying coefficients. This method has several advantages. First, it allows the treatment to be correlated with unobserved unit and time heterogeneities under reasonable modelling assumptions. Second, it generalizes the synthetic control method to the case of multiple treated units and variable treatment periods, and improves efficiency and interpretability. Third, with a built-in cross-validation procedure, it avoids specification searches and thus is transparent and easy to implement. An empirical example of Election Day Registration and voter turnout in the United States is provided. -- Awarded the 2014 John T. Williams Dissertation Prize for the best dissertation proposal in political methodology; the 2017 Political Analysis Editors' Choice; the 2018 Miller Prize for "the best work appearing in Political Analysis in 2017."
We develop a theory of how an authoritarian regime interactively uses information manipulation, such as propaganda or censorship, and policy improvement to maintain social stability. The government can depict the status quo policy more popularly supported than it actually is, while at the same time please citizens directly by enacting a costly reform. We show that the government's ability of making policy concessions reduces its incentive to manipulate information and improves its credibility. Anticipating a higher chance of policy concessions and less information manipulation, citizens are more likely to believe the government-provided information and support the regime. Our model provides an explanation for the puzzling fact that reform coexists with selective information disclosure in authoritarian countries like China.
Scholars have established that authoritarian regimes exhibit responsiveness to citizens, but our knowledge of why autocrats respond remains limited. We theorize that responsiveness may stem from rules of the institutionalized party regime, citizen engagement, and a strategy of preferential treatment of a narrow group of supporters. We test the implications of our theory using an online experiment among 2,103 Chinese counties. At baseline, we find that approximately one third of county level governments are responsive to citizen demands expressed online. Threats of collective action and threats of tattling to upper levels of government cause county governments to be considerably more responsive. However, while threats of collective action cause local officials more publicly responsive, threats of tattling do not have this effect. We also find that identifying as loyal, long-standing members of the Communists Party does not increase responsiveness. -- Awarded AJPS Best Paper 2016; AJPS Top Cited Articles Virtual Issue, 2018.
Do informal institutions, rules and norms created and enforced by social groups, promote good local governance in environments of weak democratic or bureaucratic institutions? This question is difficult to answer because of challenges in defining and measuring informal institutions and identifying their causal effects. In the paper, we investigate the effect of lineage groups, one of the most important vehicles of informal institutions in rural China, on local public goods expenditure. Using a panel dataset of 220 Chinese villages from 1986 to 2005, we find that village leaders from the two largest family clans increased local public investment considerably. This association is stronger when the clans appeared to be more cohesive. We also find that clans helped local leaders overcome the collective action problem of financing public goods, but there is little evidence suggesting that they held local leaders accountable.
The Stata package ebalance implements entropy balancing, a multivariate reweighting method described in Hainmueller (2011) that allows users to reweight a dataset such that the covariate distributions in the reweighted data satisfy a set of speci ed moment conditions. This can be useful to create balanced samples in observational studies with a binary treatment where the control group data can be reweighted to match the covariate moments in the treatment group. Entropy balancing can also be used to reweight a survey sample to known characteristics from a target population.
Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010-2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, the commonly used t-test for two-stage-least-squares (2SLS) estimates frequently underestimates uncertainties, resulting in uncontrolled Type-I errors in many studies. Third, in most replicated studies, 2SLS estimates are significantly larger than ordinary-least-squares estimates, with their ratio negatively correlated with instrument strength in studies with non-experimentally generated instruments, suggesting potential violations of unconfoundedness or the exclusion restriction. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice.
We develop an R package panelView and a Stata package panelview for panel data visualization. They are designed to assist causal analysis with panel data and have three main functionalities: (1) they plot the treatment status and missing values in a panel dataset; (2) they visualize the temporal dynamics of the main variables of interest; and (3) they depict the bivariate relationships between a treatment variable and an outcome variable either by unit or in aggregate. These tools can help researchers better understand their panel datasets before conducting statistical analysis.
This paper introduces a unified framework of counterfactual estimation for time-series cross-sectional data, which estimates the average treatment effect on the treated by directly imputing treated counterfactuals. Examples include the fixed effects counterfactual estimator, interactive fixed effects counterfactual estimator, and matrix completion estimator. These estimators provide more reliable causal estimates than conventional twoway fixed effects models when treatment effects are heterogeneous or unobserved time-varying confounders exist. Under this framework, we propose a new dynamic treatment effects plot, as well as several diagnostic tests, to help researchers gauge the validity of the identifying assumptions. We illustrate these methods with two political economy examples and develop an open-source package, fect, in both R and Stata to facilitate implementation.
We introduce hierarchically regularized entropy balancing as an extension to entropy balancing, a reweighting method that adjusts weights for control group units to achieve covariate balance in observational studies with binary treatments. Our proposed extension expands the feature space by including higher-order terms (and interactions) of covariates and then achieves approximate balance on the expanded features. To prioritize balance on variables important for treatment assignment and prevent overfitting, the method imposes ridge penalties with a hierarchical structure on the higher-order terms. Compared with entropy balancing, this extension relaxes model dependency and improves robustness of causal estimates while avoiding optimization failure and highly concentrated weights. It prevents specification searches by minimizing user discretion in selecting features to balance on. Our method is also computationally more efficient than kernel balancing, a kernel-based covariate balancing method. We demonstrate its performance through simulations and an empirical example.
We introduce Bayesian Rule Set (BRS) as an alternative to Qualitative Comparative Analysis (QCA) when data are large and noisy. BRS is an interpretable machine learning algorithm that classifies observations using rule sets, which are conditions connected by logical operators, e.g., IF (condition A AND condition B) OR (condition C), THEN Y~=~TRUE. Like QCA, BRS is highly interpretable and capable of revealing complex nonlinear relationships in data. It also has several advantages over QCA: It is compatible with probabilistically generated data; it avoids overfitting and improves interpretability by making direct trade-offs between in-sample fitness and complexity; and it remains computationally efficient with many covariates. Our contributions are threefold: We modify the BRS algorithm to facilitate its usage in the social sciences, propose methods to quantify uncertainties of rule sets, and develop graphical tools for presenting rule sets. We illustrate these methods with two empirical examples from political science.
This paper proposes a Bayesian alternative to the synthetic control method for comparative case studies with a single or multiple treated units. We adopt a Bayesian posterior predictive approach to Rubin's causal model, which allows researchers to make inferences about both individual and average treatment effects on treated observations based on the empirical posterior distributions of their counterfactuals. The prediction model we develop is a dynamic multilevel model with a latent factor term to correct biases induced by unit-specific time trends. It also considers heterogeneous and dynamic relationships between covariates and the outcome, thus improving precision of the causal estimates. To reduce model dependency, we adopt a Bayesian shrinkage method for model searching and factor selection. Monte Carlo exercises demonstrate that our method produces more precise causal estimates than existing approaches and achieves correct frequentist coverage rates even when sample sizes are small and rich heterogeneities are present in data. We illustrate the method with two empirical examples from political economy.
Multiplicative interaction models are widely used in social science to examine whether the relationship between an outcome and an independent variable changes with a moderating variable. Current empirical practice tends to overlook two important problems. First, these models assume a linear interaction effect that changes at a constant rate with the moderator. Second, estimates of the conditional effects of the independent variable can be misleading if there is a lack of common support of the moderator. Replicating 46 interaction effects from 22 recent publications in five top political science journals, we find that these core assumptions often fail in practice, suggesting that a large portion of findings across all political science subfields based on interaction models are modeling artifacts or are at best highly model dependent. We propose a checklist of simple diagnostics to assess the validity of these assumptions and offer flexible estimation strategies that allow for nonlinear interaction effects and safeguard against excessive extrapolation. These statistical routines are available in both R and STATA. -- Awarded the 2020 Miller Prize for "the best work appearing in Political Analysis in 2019."
Difference-in-differences (DID) is commonly used for causal inference in time-series cross-sectional data. It requires the assumption that the average outcomes of treated and control units would have followed parallel paths in the absence of treatment. In this paper, I propose a method that not only relaxes this often-violated assumption, but also unifies the synthetic control method (Abadie, Diamond and Hainmueller 2010) with linear fixed effect models under a simple framework, of which DID is a special case. It imputes counterfactuals for each treated unit in post-treatment periods using control group information based on a linear interactive fixed effect model that incorporates unit-specific intercepts interacted with time-varying coefficients. This method has several advantages. First, it allows the treatment to be correlated with unobserved unit and time heterogeneities under reasonable modelling assumptions. Second, it generalizes the synthetic control method to the case of multiple treated units and variable treatment periods, and improves efficiency and interpretability. Third, with a built-in cross-validation procedure, it avoids specification searches and thus is transparent and easy to implement. An empirical example of Election Day Registration and voter turnout in the United States is provided. -- Awarded the 2014 John T. Williams Dissertation Prize for the best dissertation proposal in political methodology; the 2017 Political Analysis Editors' Choice; the 2018 Miller Prize for "the best work appearing in Political Analysis in 2017."
The Stata package ebalance implements entropy balancing, a multivariate reweighting method described in Hainmueller (2011) that allows users to reweight a dataset such that the covariate distributions in the reweighted data satisfy a set of speci ed moment conditions. This can be useful to create balanced samples in observational studies with a binary treatment where the control group data can be reweighted to match the covariate moments in the treatment group. Entropy balancing can also be used to reweight a survey sample to known characteristics from a target population.
This article studies whether “pure” legality, stripped of the normative components that are conceptually necessary for “the rule of law,” can convey meaningful amounts of perceived legitimacy to governmental institutions and activity. Through a survey experiment conducted among urban Chinese residents, it examines whether such conveyance is possible under current Chinese sociopolitical conditions, in which the Party-state continues to invest heavily in “pure legality,” but without imposing meaningful legal checks on the Party leadership’s political power, and without corresponding investment in substantive civil rights or socioeconomic freedoms. Among survey respondents, government investment in legality conveys meaningful amounts of political legitimacy, even when it is applied to actions, such as online speech censorship, that are socially controversial or unattractive, and even when such investment does not clearly enhance the predictability of state behavior. However, the legitimacy-enhancing effects of legality are likely weaker than those of state investment in procedural justice.
Why do authoritarian regimes charge political opponents with non-political crimes when they can levy charges directly related to opponents' political activism? We argue that doing so disguises political repression and undermines the moral authority of opponents, minimizing backlash and mobilization. To test this argument, we conduct a survey experiment, which shows that disguised repression decreases perceptions of dissidents' morality, decreases people's willingness to engage in dissent on behalf of the dissident, and increases support for repression of the dissident. We then assess the external validity of the argument by analyzing millions of Chinese social media posts made before and after a large crackdown of vocal government critics in China in 2013. We find that individuals with larger online followings are more likely to be charged with non-political crimes, and those charged with non-political crimes are less likely to receive public sympathy and support.
We study the effect of a victimization narrative on national identity and public opinion in China experimentally. Previous research has suggested that governments can shape public opinion by guiding citizens' collective memories of historical events, but few studies have established a clear causal link. By conducting an online survey experiment among approximately 2,000 urban Chinese citizens, we examine the causal impact of historical narratives on political attitudes. We find that, compared to control conditions, a narrative focusing on China’s humiliating past in the late Qing significantly reinforces respondents' attachment to the victim side of the Chinese national identity, raises suspicion of the intention of foreign governments in international disputes, stimulates preference for more hawkish foreign policies, and strengthens support for China's current political system. These effects are particularly strong among respondents without a college degree.
This paper examines the role of social capital, embedded in kinship-based clans, in disaster relief during China's Great Famine (1958-1961). Using a county-year panel and a difference-in-differences strategy, we find that the rise in the mortality rate during the famine years is significantly less in counties with a higher clan density. Analysis using a nationally representative household survey corroborates this finding. Investigation of potential mechanisms suggests that social capital's impact on famine may have operated through enabling collective action against excessive government procurement. These results provide evidence that societal forces can ameliorate damages caused by faulty government policies in times of crisis.
Research shows that government-controlled media is an effective tool for authoritarian regimes to shape public opinion. Does government-controlled media remain effective when it is required to support changes in positions that autocrats take on issues? Existing theories do not provide a clear answer to this question, but we often observe authoritarian governments using government media to frame policies in new ways when significant changes in policy positions are required. By conducting an experiment that exposes respondents to government-controlled media---in the form of TV news segments---on issues where the regime substantially changed its policy positions, we find that by framing the same issue differently, government-controlled media moves respondents to adopt policy positions closer to the ones espoused by the regime regardless of individual predisposition. This result holds for domestic and foreign policy issues, for direct and composite measures of attitudes, and persists up to 48 hours after exposure.
When COVID-19 first emerged in China, there was speculation that the outbreak would trigger public anger and weaken the Chinese regime. By analyzing millions of social media posts from Sina Weibo made between December 2019 and February 2020, we describe the content and sentiment of public, online discussions pertaining to COVID-19 in China. We find that discussions of COVID-19 became widespread on January 20, 2020, consisting primarily of personal reflections, opinion, updates, and appeals. We find that the largest bursts of discussion coincide with the January 23 lockdown of Wuhan and the February 7 death of Dr. Li Wenliang and contain simultaneous spikes of criticism and support targeting the Chinese government. Criticisms are directed at the government for perceived lack of action, incompetence, and wrongdoing—in particular, censoring information relevant to public welfare. Support is directed at the government for aggressive action and positive outcomes. As the crisis unfolds, the same events are interpreted differently by different people, with those who criticize focusing on the government’s shortcomings and those who praise focusing on the government’s actions.
Given widespread perceptions of risk and uncertainty in nondemocratic systems and developing democracies, why do some citizens still take action and make complaints to authorities? The resource mobilization model identifies the importance of time, money, and civic skills as resources that are necessary for participation. In this paper we build on this model and argue that political connections – close personal ties to someone working in government – can also constitute a critical resource, especially in contexts with weak democratic institutions. Using data from both urban and rural China, we find that individuals with political connections are more likely to contact authorities with complaints about government public services, despite the fact that they do not have higher levels of dissatisfaction with public service provision. We conduct various robustness checks, including a sensitivity analysis, and show that this relationship is unlikely to be driven by an incorrect model specification or unobserved confounding variables.
The study of ideology in authoritarian regimes---of how public preferences are configured and constrained---has received relatively little scholarly attention. Using data from a large-scale online survey, we study ideology in China. We find that public preferences are weakly constrained, and the configuration of preferences is multi-dimensional, but the latent traits of these dimensions are highly correlated. Those who prefer authoritarian rule are more likely to support nationalism, state intervention in the economy, and traditional social values; those who prefer democratic institutions and values are less likely to be nationalistic or support traditional social values but more likely to support market reforms. This latter set of preferences appears more in provinces with higher levels of development and among wealthier and better educated respondents. These findings suggest preferences are not simply split along a pro-regime or anti-regime cleavage, and indicate a possible link between China's economic reform and societal cleavages. -- SSRN Top Papers of 2015; Media Coverage: NYT, WSJ, FP, ChinaFile.
Why would an authoritarian regime allow citizens to voice opinions publicly if the exchange of information among citizens spurs social instability as has been often alleged? In this paper, we develop a game theoretic model and show that an authoritarian regime can strengthen its rule by allowing citizens to communicate with each other publicly. From the government’s perspective, such communication has two interrelated functions. First, if public communication reveals a shared feeling of dissatisfaction towards government policies among the citizens, the government will detect the danger and improve policies accordingly. Second, and perhaps more interestingly, public communication disorganizes the citizens if they find themselves in disagreement over the policies. We show that the government allows public communication if and only if it perceives sufficient heterogeneity in preferences among the citizens. The model also illustrates that public communication could serve as a commitment device ensuring government responsiveness when it faces high dissatisfaction, which in turn makes the government better off than with private polling.
We develop a theory of how an authoritarian regime interactively uses information manipulation, such as propaganda or censorship, and policy improvement to maintain social stability. The government can depict the status quo policy more popularly supported than it actually is, while at the same time please citizens directly by enacting a costly reform. We show that the government's ability of making policy concessions reduces its incentive to manipulate information and improves its credibility. Anticipating a higher chance of policy concessions and less information manipulation, citizens are more likely to believe the government-provided information and support the regime. Our model provides an explanation for the puzzling fact that reform coexists with selective information disclosure in authoritarian countries like China.
Scholars have established that authoritarian regimes exhibit responsiveness to citizens, but our knowledge of why autocrats respond remains limited. We theorize that responsiveness may stem from rules of the institutionalized party regime, citizen engagement, and a strategy of preferential treatment of a narrow group of supporters. We test the implications of our theory using an online experiment among 2,103 Chinese counties. At baseline, we find that approximately one third of county level governments are responsive to citizen demands expressed online. Threats of collective action and threats of tattling to upper levels of government cause county governments to be considerably more responsive. However, while threats of collective action cause local officials more publicly responsive, threats of tattling do not have this effect. We also find that identifying as loyal, long-standing members of the Communists Party does not increase responsiveness. -- Awarded AJPS Best Paper 2016; AJPS Top Cited Articles Virtual Issue, 2018.
Do informal institutions, rules and norms created and enforced by social groups, promote good local governance in environments of weak democratic or bureaucratic institutions? This question is difficult to answer because of challenges in defining and measuring informal institutions and identifying their causal effects. In the paper, we investigate the effect of lineage groups, one of the most important vehicles of informal institutions in rural China, on local public goods expenditure. Using a panel dataset of 220 Chinese villages from 1986 to 2005, we find that village leaders from the two largest family clans increased local public investment considerably. This association is stronger when the clans appeared to be more cohesive. We also find that clans helped local leaders overcome the collective action problem of financing public goods, but there is little evidence suggesting that they held local leaders accountable.
Although researchers have often considered democracy and state capacity to be key predictors of cross-national variations in human welfare, few have investigated the relationship between the two variables themselves. We argue that democratization may have a positive, causal effect on state capacity. Employing a time-series cross-national dataset from 1960 to 2009, we document that democratization leads to a substantial increase in state capacity in the long run. Our results prove robust to a rich set of potential confounders and alternative coding of key variables. To further address the problem of endogeneity, we use an instrumental variable strategy that exploits exogenous variations in regional democratic diffusions. We also provide suggestive evidence that democratization enhances state capacity through increasing political contestation. -- Awarded the 2015 Malcolm Jewell Award for the best graduate student paper presented at the SPSA annual meeting.
How much does it matter which party controls the government? On one hand, campaign positions and roll-call records suggest that contemporary American parties are very ideologically polarized. On the other hand, the existing evidence that electing Democrats into office causes the adoption of more liberal policies is surprisingly weak. We bring clarity to this debate with the aid of a new measure of the policy liberalism of each state in each year 1936-2014, using regression-discontinuity and dynamic panel analyses to estimate the policy effects of the partisan composition of state legislatures and governorships. We find that until the 1980s, partisan control of state government had negligible effects on the liberalism of state policies, but that since then partisan effects have grown markedly. Even today, however, the policy effects of partisan composition remain small relative to differences between states---less than one-tenth of the cross-sectional standard deviation of state policy liberalism. This suggests that campaign positions and roll-call records may overstate the policy effects of partisan selection relative to other factors, such as public opinion.
This chapter surveys new development in causal inference using time-series cross-sectional (TSCS) data. I start by clarifying two identification regimes for TSCS analysis: one under the strict exogeneity assumption and one under the sequential ignorability assumption. I then review three most commonly used methods by political scientists, the difference-in-differences approach, two-way fixed effects models, and the synthetic control method. For each method, I examine its assumptions, explain its pros and cons, and discuss its extensions. I then introduce several new methods under strict exogeneity or sequential ignorability, including the factor-augmented approach, panel matching, and marginal structural models. I conclude by pointing to several directions for future research.
Over the past ten years, the “causal inference revolution” has dramatically changed the landscape of political science and social sciences in general. Two of the most commonly used groups of methods for causal inference with observational data are (1) methods related to panel data or time-series-cross-section (TSCS) data and (2) regression discontinuity (RD) designs. Articles in this virtual issue represent the efforts of political methodologists either to develop more reliable and versatile approaches to panel data analysis and RD designs or to better understand the advantages and disadvantages of existing common practices in recent years.
The performance of the Simple Poverty Scorecard is compared against the performance of established regression-based estimators. All estimates are benchmarked against observed poverty status based on household expenditure (or income) data from household socioeconomic surveys that span nearly a decade and are representative of subnational populations. When the models all adopt the same "one-size-fits-all" training approach, there is no meaningful difference in performance and the Simple Poverty Scorecard is as good as any of the regression-based estimators. The findings change, however, when the regression-based estimators are "trained" on "training sets" that more closely resemble potential subpopulation test sets. In this case, regression-based models outperform the nationally calculated Simple Poverty Scorecard in terms of bias and variance. These findings highlight the fundamental trade-off between simplicity of use and accuracy.