Replicating Chen, Pan, and Xu(2016)
Jinwen Wu
2024-11-05
Chen, Pan, and Xu (2016) investigate the sources of authoritarian responsiveness to citizen demands through an online field experiment of 2,103 Chinese counties, focusing on three potential factors: pressure from below (threat of collective action), pressure from above (threat of tattling to superior), and entreaties of loyal insiders (claim of Chinese Communist Party (CCP) membership). They found that the two threats caused county-level governments to be considerably more responsive, but loyalty to the party did not yield significant improvements in government responsiveness.
This RMarkdown tutorial replicates the core analyses in “Sources of Authoritarian Responsiveness: A Field Experiment in China”. The replication, conducted by Jinwen Wu, a predoctoral fellow at Stanford University, is supervised by Professor Yiqing Xu. The replication summarizes the main data analyses from the article; please refer to the original paper for a comprehensive understanding of the ideas presented.
Click the Code button in the top right and select
Show All Code to reveal all code used in this RMarkdown.
Click Show in paragraphs to reveal the code used to
generate a finding. The original replication files can be downloaded
from here.
1 Conceptual Framework
1.1 Responsiveness
In the study, responsiveness is defined as the “the extent to which officials in the regime adhere to the demands of societal actors” (Grant and Keohane 2005; Malesky and Schuler 2010). Government responses can take on three forms: first, the regime can provide desired outcome by directly enacting new policies or conferring benefits as requested; second, it can take other actions to help generate the desired outcome; lastly, the government can inform people how to advocate for themselves to potentially achieve their goals.
1.2 Potential Causal Path
The authors examine three sources of government responsiveness using a four-arm experimental design:
Responding to Collective Action Threats: Authoritarian governments are highly concerned about rebellion threats, thus, often resorting to proactive measures (Bernstein and Lu 2003; Chen 2012; King, Pan, and Roberts 2013, 2014; Lorentzen 2013; O’Brien and Li 1999; Perry 2002; Wasserstrom and Perry 1994). In China, specifically, studies have found that performance metrics of government officials, especially those at the lower levels, are dependent upon maintaining social stability (Boix and Svolik 2013; Levitsky and Way 2010; Svolik 2012; Wintrobe 2007).
Responding to Pressure from Above: In the authoritarian cadre evaluation system, citizens feedback serves as oversight to evaluate and monitor lower-level officials. Upper supervisors often assess local government performance based on complaint frequency. As a result, lower-level officials fear being reported to higher authorities, which could make them appear incompetent or delinquent to the upper-level government officials (Nathan 2003; Pye 1980; Edin 2003; Fukuyama 2014).
Responding to Loyal Supporters: Authoritarian leaders maintain a stable ruling coalition by responding to loyal insiders (Broockman 2013; Butler and Broockman 2011; Geddes 2006; Hanson 2013; Lust-Okar 2005; Magaloni and Wallace 2008; Rueda 2005).
1.3 Hypotheses
Based on existing literature, the authors tested three hypotheses.
- H1: Requests that imply potential for collective action increase local officials’ responsiveness to citizen demands.
- H2: Requests that suggest the likelihood of evoking upper-level government oversight increase local officials’ responsiveness to citizen demands.
- H3: Requests that demonstrate CCP membership or loyalty to the Party increase local officials’ responsiveness to citizen demands.
2 Experimental Design
The study’s experimental design leverages China’s “Open Government Information Ordinance” transparency initiative. Enacted in April 2007, this policy requires local governments to establish online platforms where citizens can post questions, complaints, or requests. As of 2016, about 3,000 county-level governments nationwide have adopted some formats of feedback collection systems on their official websites (Heffer and Schubert 2023). These platforms are managed primarily by local information management offices; they determine what content should appear, how responses will be handled, and which issues need to be forwarded to higher-level authorities or specific agencies.
2.1 Set-ups
Researchers organized, tested, and recorded characteristics of 2,869 government websites that people can directly submit feedback reports on the open forums.
The team attempted to post on 2,227 county websites(77.6%), successfully submitting requests on 2,103 forums (73.3%). Then, the same type of request—asking for assistance with social welfare -Dibao- was submitted. Each post was tracked over time, with response checks at 10 and 20 days, to assess how quickly and effectively each government responded.
There are three different types of treatment appeals added to the end of the straightforward baseline(control) request.
The additional appeals:
- Collective Action Threat: implies that a high possibility of collective action and public unrest if the government fails to respond.
- Tattling Threat: suggests that failure to respond would lead to oversight or intervention by higher levels of government.
- Claims of Loyalty: is framed to emphasize the requester’s loyalty to the Chinese Communist Party (CCP) and dedication to party principles.
2.2 Explanation of the Design
First, the researchers chose Dibao, the Minimum Livelihood Guarantee, as the policy focus because it is implemented across all Chinese counties. Dibao requires local governments to follow a national standard at least and offer financial subsidies to all eligible citizens. Since 2000, the welfare program has covered all counties in China (Kakwani et al. 2019). The researchers found the subject topic as an ideal example to fairly assess government responsiveness nationwide.
Second, the requested assistance has already been frequently posted on government forums. This mitigates ethical concerns by minimizing potential disruptions to local governments.
Third, government responsiveness to Dibao is similar to how they respond to other issues like tax reform, support for private enterprises, unemployment benefits, and more (Solinger 2005). Thus, such a design enhances external validity.
Beyond treatment selection, the authors also recorded the features of government opinion forums, as reported in Figures 1 and 2. For the regression analysis, both the full sample of 2,869 government websites and a subset of 2,103 websites where requests were posted have been used. Figure 3 compares the causal effects of the three treatments, with estimates showing the proportion of government responses received.
3 Replicating the Main Findings
3.1 Installing Packages
Several R packages are required for the data analysis and visualization. The code chunk below checks for all required packages and installs the missing ones.
Packages: “haven”, “tidyr”, “dplyr”, “knitr”, “kableExtra”, “ggplot2”, “sandwich”, “estimatr”, “modelsummary”, “janitor”.
# Function to install missing packages
install_all <- function(packages) {
installed_pkgs <- rownames(installed.packages())
for (pkg in packages) {
if (!pkg %in% installed_pkgs) {
install.packages(pkg, repos = 'http://cran.us.r-project.org')
}
}
}
# List of packages to be installed
packages <- c(
"haven", "tidyr", "dplyr", "knitr",
"kableExtra", "ggplot2", "sandwich",
"estimatr", "modelsummary", "janitor"
)
# Run the function
install_all(packages)
# Load all packages without output
invisible(lapply(packages, function(pkg) suppressPackageStartupMessages(require(pkg, character.only = TRUE))))After installation, call to load the packages. Then, import the
experiment data. The data file is in the Replication folder
titled forum.dta.
library(tidyr)
library(dplyr)
library(knitr)
library(kableExtra)
library(ggplot2)
library(sandwich)
library(estimatr)
library(modelsummary)
library(janitor)
library(haven)
data <- read_dta("forum.dta")3.2 Summary Statistics
Before running the regressions, the authors checked the covariate balance. The covariate controls are grouped into two categories: sociodemographic or web feature controls. The table offers a broad overview of the characteristics of the 2,869 counties.
# Define the covariates, see codebook for further explanation.
covar <- c("logpop", "logpop00", "popgrow", "sexratio", "logpopden", "migrant", "nonagrhh", "urban", "eduyr", "illit", "minor", "unemploy", "wk_agr", "wk_ind", "wk_ser", "income", "logincome", "loggdp", "avggrowth", "logoutput_agr", "logoutput_ind", "logoutput_ser", "numlfirm", "loginvest", "logsaving", "loggov_rev", "loggov_exp")
variable_names <- c(
logpop = "Log population",
logpop00 = "Log population (2000)",
popgrow = "Population growth (2000-10%)",
sexratio = "Gender ratio (female = 1.00)",
logpopden = "Log population density (person/km2)",
migrant = "Migrant share of population (%)",
nonagrhh = "Non-agriculture household share of population (%)",
urban = "Permanent urban residents share of population (%)",
eduyr = "Average years of education",
illit = "Illiteracy rate among population age above 15 (%)",
minor = "Ethnic minority share of population (%)",
unemploy = "Unemployment rate (%)",
wk_agr = "Work force in agriculture (%)",
wk_ind = "Work force in industry (%)",
wk_ser = "Work force in services (%)",
income = "GDP per capita (1,000 CNY)",
logincome = "Log GDP per capita",
loggdp = "Log GDP",
avggrowth = "Average nominal GDP growth (2000-10)",
logoutput_agr = "Log agricultural output",
logoutput_ind = "Log industrial output",
logoutput_ser = "Log services output",
numlfirm = "Number of enterprises above designated size (annual sales above 5 million CNY)",
loginvest = "Log total investment from households, enterprises, and government",
logsaving = "Log total saving (total outstanding bank deposits of rural and urban households)",
loggov_rev = "Log total government revenue",
loggov_exp = "Log total government expenditure"
)
# Function to generate summary statistics
func_gen_summary_stats <- function(data, var, variable) {
data %>%
summarise(
variable = variable,
count = sum(!is.na(.data[[var]])),
mean = round(mean(.data[[var]], na.rm = TRUE), 2),
sd = round(sd(.data[[var]], na.rm = TRUE), 2),
min = round(min(.data[[var]], na.rm = TRUE), 2),
max = round(max(.data[[var]], na.rm = TRUE), 2),
median = round(median(.data[[var]], na.rm = TRUE), 2)
) %>%
as.data.frame()
}
# Generate summary statistics for each variable
summary_stats <- do.call(rbind, Map(function(var, variable) func_gen_summary_stats(data, var, variable), covar, variable_names))
rownames(summary_stats) <- NULL
kable(summary_stats, format = "html") %>%
kable_styling(full_width = F, bootstrap_options = c("striped", "hover", "condensed"))| variable | count | mean | sd | min | max | median |
|---|---|---|---|---|---|---|
| Log population | 2869 | 12.73 | 0.90 | 7.80 | 15.92 | 12.85 |
| Log population (2000) | 2869 | 12.68 | 0.90 | 7.74 | 15.89 | 12.80 |
| Population growth (2000-10%) | 2869 | 5.01 | 3.85 | -7.95 | 21.69 | 4.85 |
| Gender ratio (female = 1.00) | 2869 | 1.06 | 0.07 | 0.73 | 2.52 | 1.05 |
| Log population density (person/km2) | 2869 | 14.87 | 1.96 | 7.23 | 20.87 | 14.93 |
| Migrant share of population (%) | 2869 | 16.90 | 14.51 | 0.43 | 91.00 | 11.29 |
| Non-agriculture household share of population (%) | 2869 | 29.53 | 23.56 | 1.58 | 99.40 | 20.04 |
| Permanent urban residents share of population (%) | 2869 | 46.91 | 25.68 | 1.38 | 100.00 | 38.95 |
| Average years of education | 2869 | 8.71 | 1.47 | 2.00 | 13.14 | 8.62 |
| Illiteracy rate among population age above 15 (%) | 2869 | 6.34 | 7.21 | 0.09 | 66.22 | 4.20 |
| Ethnic minority share of population (%) | 2869 | 16.23 | 29.00 | 0.00 | 99.78 | 1.51 |
| Unemployment rate (%) | 2869 | 3.29 | 2.95 | 0.00 | 22.26 | 2.24 |
| Work force in agriculture (%) | 2869 | 51.79 | 27.52 | 0.00 | 97.09 | 59.32 |
| Work force in industry (%) | 2869 | 20.63 | 14.97 | 0.00 | 76.13 | 17.06 |
| Work force in services (%) | 2869 | 27.52 | 17.58 | 0.00 | 97.26 | 21.93 |
| GDP per capita (1,000 CNY) | 2821 | 24.96 | 19.59 | 1.81 | 259.60 | 19.71 |
| Log GDP per capita | 2821 | 9.89 | 0.64 | 7.50 | 12.47 | 9.87 |
| Log GDP | 2821 | 8.84 | 1.13 | 4.63 | 12.25 | 8.99 |
| Average nominal GDP growth (2000-10) | 2821 | 0.15 | 0.05 | 0.01 | 0.41 | 0.15 |
| Log agricultural output | 2821 | 7.13 | 0.99 | 2.77 | 9.09 | 7.32 |
| Log industrial output | 2821 | 7.97 | 1.39 | 2.71 | 11.81 | 8.14 |
| Log services output | 2821 | 7.69 | 1.11 | 3.78 | 11.27 | 7.77 |
| Number of enterprises above designated size (annual sales above 5 million CNY) | 2821 | 51.29 | 81.49 | 0.00 | 802.00 | 27.00 |
| Log total investment from households, enterprises, and government | 2821 | 8.43 | 1.11 | 3.66 | 11.37 | 8.58 |
| Log total saving (total outstanding bank deposits of rural and urban households) | 2821 | 6.80 | 1.32 | 0.00 | 9.86 | 7.01 |
| Log total government revenue | 2821 | 5.74 | 1.27 | 1.10 | 9.70 | 5.78 |
| Log total government expenditure | 2821 | 7.14 | 0.66 | 4.62 | 9.55 | 7.16 |
Replicating Table 1 in the article.
The plot below further examines covariate balance of sociodemographic factors using standardized mean differences. The estimates cluster closely around 0, suggesting that treatment randomization was successful. Note, the figure does not include logged covariates or proportions, as the standardized differences in means are too small to visualize clearly.
# Calculate the mean of each covariate for the 4 experimental groups
standardized_data <- data %>%
mutate(across(all_of(covar), ~ (. - mean(.)) / sd(.)))
# The following variables are dropped because they are either:
# 1) logged values, or
# 2) proportions
# small_vars <- c("logpop", "logpop00", "logpopden", "logincome", "loggdp", "logoutput_agr", "logoutput_ind", "logoutput_ser", "loginvest", "logsaving", "loggov_rev", "loggov_exp", "income", "numlfirm","avggrowth")
non_logged_vars <- c("popgrow", "sexratio", "migrant", "nonagrhh", "urban","eduyr", "illit", "minor", "unemploy", "wk_agr","wk_ind", "wk_ser")
names_non<- c(
"Population growth (2000-10%)",
"Gender ratio (female = 1.00)",
"Migrant share of population (%)",
"Non-agriculture household share of population (%)",
"Permanent urban residents share of population (%)",
"Average years of education",
"Illiteracy rate among population age above 15 (%)",
"Ethnic minority share of population (%)",
"Unemployment rate (%)",
"Workforce in agriculture (%)",
"Workforce in industry (%)",
"Workforce in services (%)"
)
# Summarize means for each treatment group
means <- data %>%
select(all_of(non_logged_vars), treat) %>%
mutate(across(all_of(non_logged_vars), ~ (. - mean(.)) / sd(.))) %>%
group_by(treat) %>%
summarize(across(all_of(non_logged_vars), mean, na.rm = TRUE))
# Reshape data for plotting, keeping only non-logged covariates
means_long <- means %>%
pivot_longer(cols = -treat, names_to = "Covariate", values_to = "Mean") %>%
mutate(Covariate = factor(Covariate, levels = non_logged_vars, labels = names_non))
# Define unique symbols and colors for the 4 groups
symbols <- c(16, 17, 22, 23)
# Create the plot with adjusted legend position, layout, and colors
ggplot(means_long, aes(x = Mean, y = Covariate, shape = as.factor(treat))) +
geom_point(size = 3) +
scale_shape_manual(values = symbols,
labels = c("Control",
"Collective Action Threat",
"Tattling Threat",
"Claims of Loyalty")) +
labs(title = "Covariate Balance",
x = "Standardized Mean Differences",
y = "") +
theme_minimal() +
theme(legend.position = "bottom",
legend.box = "horizontal",
legend.title = element_blank(),
legend.text = element_text(size = 8),
legend.key.size = unit(1, "lines"),
legend.spacing = unit(0.2, "lines"),
legend.margin = margin(0, 0, 0, 0)) +
guides(shape = guide_legend(ncol = 2)) +
scale_x_continuous(limits = c(-1, 1))
Before moving to visualization, let’s perform all the following regressions and store the estimates.
# Regression Analysis for Tables
vars_to_demean <- c("forum_response", "forum_recent_response",
"forum_imme_viewable", "post_email", "post_name",
"post_idnum", "post_tel", "post_address",
"logpop", "nonagrhh", "urban", "eduyr",
"unemploy", "minor")
# Function to demean variables
demean_vars <- function(data, vars) {
data %>%
mutate(across(all_of(vars),
list(dm = ~ . - mean(. , na.rm = TRUE)),
.names = "dm_{.col}"))
}
data <- demean_vars(data, vars_to_demean)
# Control_dem: Interaction between treat and demeaned demographic variables
control_dem <- "(dm_logpop + dm_nonagrhh + dm_urban + dm_eduyr + dm_unemploy + dm_minor)"
# Control_web: Interaction between treat and demeaned web-related variables
control_web <- "(dm_forum_response + dm_forum_recent_response + dm_forum_imme_viewable + dm_post_email + dm_post_name + dm_post_idnum + dm_post_tel + dm_post_address)"
# -----------------------------
# Table 2
# -----------------------------
# Table 2 Regressions
# -----------------------------
# 2(a). Conditional Regressions (if posted)
# -----------------------------
# a. Basic OLS regression without fixed effects
model_uncond_ols <- lm_robust(response ~ tr1 + tr2 + tr3, clusters = prov, data = data)
# b. Fixed effects regression with city fixed effects
model_uncond_fe <- lm_robust(response ~ tr1 + tr2 + tr3, clusters = prov, fixed_effects = ~city, data = data)
# c. Fixed effects with demographic controls
model_uncond_fe_dem <- lm_robust(as.formula(paste("response ~ tr1 + tr2 + tr3 +", control_dem)), fixed_effects = ~city,clusters = prov, data = data)
# -----------------------------
# 2(b). Conditional Regressions (if posted)
# -----------------------------
# Subset data where posted == 1
forum_subset <- subset(data, posted == 1)
# a. Basic OLS regression on posted data
model_cond_ols <- lm_robust(response ~ tr1 + tr2 + tr3, clusters = prov, data = forum_subset)
# b. Fixed effects regression with city fixed effects on posted data
model_cond_fe <- lm_robust(response ~ tr1 + tr2 + tr3, clusters = prov, fixed_effects = ~city, data = forum_subset)
model_cond_c1 <- lm_robust(as.formula(paste("response ~ tr1 + tr2 + tr3 +", control_dem)), fixed_effects = ~city,clusters = prov, data = forum_subset)
# c. Fixed effects with demographic and web controls on posted data
model_cond_fe_controls <- lm_robust(as.formula(paste("response ~ tr1 + tr2 + tr3 +", control_dem, "+", control_web)), clusters = prov, fixed_effects = ~city, data = forum_subset)
# -----------------------------
# Table 3
# -----------------------------
# Table 3 Regressions
# a. Basic OLS regression for response_public
model_public_uncond_ols_2 <- lm_robust(response_public ~ tr1 + tr2 + tr3, clusters = prov, data = data)
# Fixed effects regression with city fixed effects and demographic controls
model_public_uncond_fe_dem_2 <- lm_robust(as.formula(paste("response_public ~ tr1 + tr2 + tr3 +", control_dem)), clusters = prov, fixed_effects = ~city, data = data)
# b. Conditional regressions for response_public if posted
# Basic OLS on posted data
model_public_cond_ols_2 <- lm_robust(response_public ~ tr1 + tr2 + tr3, clusters = prov, data = forum_subset)
# Fixed effects with controls on posted data
model_public_cond_fe_controls_2 <- lm_robust(as.formula(paste("response_public ~ tr1 + tr2 + tr3 +", control_dem, "+", control_web)), clusters = prov, fixed_effects = ~city, data = forum_subset)
models_2 <- list(
"Unconditional (1)" = model_public_uncond_ols_2,
"Unconditional (2)" = model_public_uncond_fe_dem_2,
"Conditional (3)" = model_public_cond_ols_2,
"Conditional (4)" = model_public_cond_fe_controls_2
)
# Regression Analysis for plotting
outcome_vars <- c("attempted", "posted", "forum_response", "forum_recent_response", "forum_imme_viewable", "post_name", "post_tel", "post_email", "post_address", "post_idnum")
# -----------------------------
# Figure 1 & 2
# -----------------------------
results <- outcome_vars %>%
lapply(function(var) {
lm_model <- lm_robust(as.formula(paste(var, "~ tr1 + tr2 + tr3")), data = data)
lm_summary <- summary(lm_model)
# Extract coefficients and standard errors
intercept <- coef(lm_model)["(Intercept)"]
tr1_coef <- intercept + coef(lm_model)["tr1"]
tr2_coef <- intercept + coef(lm_model)["tr2"]
tr3_coef <- intercept + coef(lm_model)["tr3"]
# Calculate modified estimates and confidence intervals
estimates <- c(intercept, tr1_coef, tr2_coef, tr3_coef)
se <- c(
lm_summary$coefficients["(Intercept)", "Std. Error"],
lm_summary$coefficients["tr1", "Std. Error"],
lm_summary$coefficients["tr2", "Std. Error"],
lm_summary$coefficients["tr3", "Std. Error"]
)
lower <- estimates - 1.96 * se
upper <- estimates + 1.96 * se
# Create a data frame of results for this outcome variable
data.frame(
outcome = var,
treatment = c("Control", "T1", "T2", "T3"),
estimate = estimates,
se = se,
lower = lower,
upper = upper
)
}) %>%
bind_rows()
results <- results %>%
mutate(
outcome = recode(outcome,
"attempted" = "Posting attempt made",
"posted" = "Request successfully posted",
"post_address" = "Requiring address",
"post_email" = "Requiring email",
"post_idnum" = "Requiring ID number",
"post_tel" = "Requiring telephone contact",
"post_name" = "Requiring name"),
treatment = factor(treatment, levels = c("Control", "T1", "T2", "T3"),labels = c("Control", "Collective Action Threat", "Tattling Threat", "Claims of Loyalty"))
)
# -----------------------------
# Figure 3 & 5
# -----------------------------
main_vars <- c("response","response_public", "response_content", "response_time")
results_main <- main_vars %>%
lapply(function(var) {
# Fit the robust linear model
lm_model <- lm_robust(as.formula(paste(var, "~ tr1 + tr2 + tr3")), data = forum_subset)
lm_summary <- summary(lm_model)
# Extract coefficients
intercept <- coef(lm_model)["(Intercept)"]
tr1_coef <- intercept + coef(lm_model)["tr1"]
tr2_coef <- intercept + coef(lm_model)["tr2"]
tr3_coef <- intercept + coef(lm_model)["tr3"]
# Calculate the adjusted estimates
estimates <- c(intercept, tr1_coef, tr2_coef, tr3_coef)
se <- c(
lm_summary$coefficients["(Intercept)", "Std. Error"],
lm_summary$coefficients["tr1", "Std. Error"],
lm_summary$coefficients["tr2", "Std. Error"],
lm_summary$coefficients["tr3", "Std. Error"]
)
# Calculate the adjusted confidence intervals
lower <- estimates - 1.96 * se
upper <- estimates + 1.96 * se
# Create a data frame of results for this outcome variable
data.frame(
outcome = var,
treatment = c("Control", "Collective Action", "Tattling Threat", "Claims of Loyalty"),
estimate = estimates,
se = se,
lower = lower,
upper = upper
)
}) %>%
bind_rows()
results_main <- results_main %>%
mutate(
outcome = recode(outcome,
"response" = "Government response received",
"response_public" = "Government response received and publicly viewable",
"response_content" = "Government response content",
"response_time" = "Government response time"
)
)3.3 Availability of County Government Web Forums
In this experiment, the authors organized 2,869 county-level government websites and identified 2,227 opinion forums where requests for assistance in registering for Dibao could be posted.
Figure 2 illustrates the distribution of government web forums across different treatment groups: control, Collective Action Threat, Tattling Threat, and Claims of Loyalty. Specifically, 519 posts were submitted to the control group, 525 to the first, 531 to the second, and 528 to the third treatment group.
ggplot(results %>% filter(outcome %in% c("Posting attempt made", "Request successfully posted")),
aes(x = treatment, y = estimate, ymin = lower, ymax = upper, color = outcome)) +
geom_point() +
geom_errorbar(width = 0.2) +
facet_wrap(~ outcome, scales = "fixed") +
ylim(0, 1) +
labs(title = "Availability and Openness of County Government Web Forums",
x = "Treatment Group", y = "Estimate") +
theme_minimal() +
theme(legend.position = "none",
axis.text.x = element_text(angle = 45, hjust = 1))
Replicating Figure 1 in the article.
The results demonstrate that the distributions of different types of forums and the success rate of posting on them are balanced across all treatment groups.
3.4 Openness of County Government Web Forums
Figure 3 presents the openness characteristics of county government web forums focusing on five parameters.
ggplot(results %>% filter(outcome %in% c("Requiring name", "Requiring telephone contact", "Requiring email", "Requiring address", "Requiring ID number")),
aes(x = treatment, y = estimate, ymin = lower, ymax = upper, color = outcome)) +
geom_point() +
geom_errorbar(width = 0.2) +
facet_wrap(~ outcome, scales = "fixed") +
ylim(0, 1) +
labs(title = "Requirements for Posting to County Government Web Forums",
x = "Treatment Group", y = "Estimate") +
theme_minimal() +
theme(legend.position = "none",
axis.text.x = element_text(angle = 45, hjust = 1))
Replicating Figure 2 in the article.
Approximately 70% of the forums allow request posting without log in or create an account. Notably, less than 5% of forums release the submitted posts immediately; the majority of requests undergo a review process before becoming publicly accessible. These openness metrics are evenly distributed among all treatment groups, ensuring consistency in for later analysis.
3.5 Causal Effects of Treatments on Government Response
Table 2 is divided into unconditional models (including all counties) and conditional models (restricted to counties with successful post submissions). Across all models, Collective Action Threat consistently demonstrates the highest significant positive effect, while Tattling Threat also shows a significant positive impact. In contrast, Claims of Loyalty has a minimal effect, around 2-4 percentage points, and is not always statistically significant. Adding control variables and provincial dummies does not substantially alter these estimates, underscoring the robustness of the findings.
models <- list(
"All: Benchmark" = model_uncond_ols,
"All: w/ FE" = model_uncond_fe,
"All: w/ FE & Controls" = model_uncond_fe_dem,
"Conditional: Benchmark" = model_cond_ols,
"Conditional: w/ FE" = model_cond_fe,
"Conditional: w/ FE & Controls" = model_cond_fe_controls
)
coef_map <- c(
`(Intercept)` = "Constant",
`tr1` = "T1: Collective Action Threat",
`tr2` = "T2: Tattling Threat",
`tr3` = "T3: Claims of Loyalty"
)
mo1 <- modelsummary(
models,
coef_map = coef_map,
statistic = "({std.error}{stars})",
gof_omit = "IC|LL|AIC|BIC|Log",
coef_omit = "Intercept",
fmt = 3,
output = "kableExtra"
)
mo1 %>%
kable_styling(full_width = F, bootstrap_options = c("striped", "hover", "condensed"))| All: Benchmark | All: w/ FE | All: w/ FE & Controls | Conditional: Benchmark | Conditional: w/ FE | Conditional: w/ FE & Controls | |
|---|---|---|---|---|---|---|
| T1: Collective Action Threat | 0.077 | 0.065 | 0.066 | 0.101 | 0.090 | 0.102 |
| (0.020***) | (0.020**) | (0.020**) | (0.027***) | (0.026***) | (0.026***) | |
| T2: Tattling Threat | 0.068 | 0.061 | 0.062 | 0.083 | 0.094 | 0.106 |
| (0.023**) | (0.023**) | (0.023**) | (0.028**) | (0.028***) | (0.029***) | |
| T3: Claims of Loyalty | 0.033 | 0.025 | 0.025 | 0.040 | 0.030 | 0.042 |
| (0.024) | (0.024) | (0.024) | (0.030) | (0.031) | (0.031) | |
| Num.Obs. | 2869 | 2869 | 2869 | 2103 | 2103 | 2103 |
| R2 | 0.005 | 0.272 | 0.272 | 0.007 | 0.278 | 0.347 |
| R2 Adj. | 0.004 | 0.173 | 0.172 | 0.005 | 0.143 | 0.219 |
| RMSE | 0.45 | 0.38 | 0.38 | 0.48 | 0.41 | 0.39 |
| Std.Errors | by: prov | by: prov | by: prov | by: prov | by: prov | by: prov |
Replicating Table 2 in the article.
3.6 Causal Effects of Treatments on Publicly Viewable Responses
Similar to Table 2, the estimates in Table 3 are calculated from both the unconditional full sample and the conditional subsample. The results indicate that Collective Action Threat significantly increases the probability of public responses, while the other two treatments show smaller, non-significant increases. These outcomes reinforce the finding that threats of collective action are more effective in eliciting government responses compared to the other treatment conditions, as observed in Table 2.
models_2 <- list(
"All: Benchmark" = model_public_uncond_ols_2,
"All: FE + Controls" = model_public_uncond_fe_dem_2,
"Conditional: Benchmark" = model_public_cond_ols_2,
"Conditional: FE + Controls" = model_public_cond_fe_controls_2
)
mo2 <- modelsummary(
models_2,
coef_map = coef_map,
statistic = "({std.error}{stars})",
gof_omit = "IC|LL|AIC|BIC|Log",
coef_omit = "Intercept",
fmt = 3,
output = "kableExtra"
)
mo2 %>%
kable_styling(full_width = F, bootstrap_options = c("striped", "hover", "condensed"))| All: Benchmark | All: FE + Controls | Conditional: Benchmark | Conditional: FE + Controls | |
|---|---|---|---|---|
| T1: Collective Action Threat | 0.079 | 0.076 | 0.106 | 0.111 |
| (0.022***) | (0.022***) | (0.028***) | (0.028***) | |
| T2: Tattling Threat | 0.038 | 0.038 | 0.046 | 0.068 |
| (0.024) | (0.024) | (0.030) | (0.031*) | |
| T3: Claims of Loyalty | 0.032 | 0.030 | 0.040 | 0.045 |
| (0.023) | (0.022) | (0.029) | (0.030) | |
| Num.Obs. | 2869 | 2869 | 2103 | 2103 |
| R2 | 0.005 | 0.219 | 0.007 | 0.312 |
| R2 Adj. | 0.004 | 0.112 | 0.006 | 0.177 |
| RMSE | 0.39 | 0.35 | 0.44 | 0.36 |
| Std.Errors | by: prov | by: prov | by: prov | by: prov |
Replicating Table 3 in the article.
The figure below visualizes the causal effect as presented in Table 2 and Table 3 with the full dataset. The estimates are calculated from the baseline OLS model with robust standard error but without controls or fixed effects.
The left panel showcases the impact of the three treatments on county governments’ overall responsiveness, while the right panel represents the effect on public viewability of government responses.
ggplot(results_main %>% filter(outcome %in% c("Government response received", "Government response received and publicly viewable")),
aes(x = factor(treatment, levels = c("Control", "Collective Action", "Tattling Threat", "Claims of Loyalty")),
y = estimate, ymin = lower, ymax = upper, color = outcome)) +
geom_point() +
geom_errorbar(width = 0.2) +
facet_wrap(~ outcome, scales = "fixed", nrow = 1) +
ylim(0, 0.65) +
labs(title = "Causal Effects of Treatments on Responsiveness",
x = "Treatment Groups", y = "Estimate") +
theme_minimal() +
theme(legend.position = "none",
axis.text.x = element_text(angle = 45, hjust = 1))
Replicating Figure 3 and 5 in the article.
3.7 Causal Effects of Treatments on Reply Content
Based on government replies, the researchers examined various strategies applied in response to four types of requests. Government replies are categorized into three types: Deferral, Referral, and Direct Information.
- Deferral responses highlight that the initial request lacks sufficient information and encourage follow-up with additional details, with no further assistance (to submit complementary files).
- Referral responses also suggest information inadequacy, which the researchers intentionally designed to minimize disruption for local governments. However, these responses direct the requester to another agency or provide contact information for further assistance.
- Direct Information responses offer comprehensive and detailed guidelines that address the request directly, including specific steps and requirements to apply for Dibao, along with relevant contact details.
The control group experienced the highest rate of No Responses (76.9%) and the lowest rates of Deferral (4.6%) and Direct Information (12.7%).
The statistics of reply content type across each treatment group are reported in the table below.
# -----------------------------
# Table 4
# -----------------------------
# Create a cross-tab with row proportions
cross_tab <- data %>%
mutate(
treat = case_when(
treat == 0 ~ "Control",
treat == 1 ~ "T1: Collective Action Threat",
treat == 2 ~ "T2: Tattling Threat",
treat == 3 ~ "T3: Claims of Loyalty"
),
response_content = case_when(
response_content == 0 ~ "No Response",
response_content == 1 ~ "Deferral",
response_content == 2 ~ "Referral",
response_content == 3 ~ "Direct Info"
)
) %>%
tabyl(treat, response_content) %>%
adorn_percentages("row") %>%
adorn_pct_formatting(digits = 2) %>%
adorn_ns()
kable(cross_tab, format = "html") %>%
kable_styling(full_width = F, bootstrap_options = c("striped", "hover", "condensed"))| treat | Deferral | Direct Info | No Response | Referral |
|---|---|---|---|---|
| Control | 4.60% (33) | 12.69% (91) | 76.85% (551) | 5.86% (42) |
| T1: Collective Action Threat | 5.02% (36) | 18.55% (133) | 69.18% (496) | 7.25% (52) |
| T2: Tattling Threat | 6.97% (50) | 16.88% (121) | 70.01% (502) | 6.14% (44) |
| T3: Claims of Loyalty | 5.43% (39) | 12.95% (93) | 73.54% (528) | 8.08% (58) |
Replicating Table 4 in the article.
The Collective Action Threat increases the proportion of Direct Information responses, while reducing Deferral responses compared with the Claims of Loyalty group. The Tattling Threat causes a comparatively large increase in Deferral responses and modestly affects Direct Information. Claims of Loyalty also leads to an increase in Direct Information but to a lesser extent than the other two groups.
cross_tab_long <- cross_tab %>%
pivot_longer(
cols = -treat,
names_to = "response_content",
values_to = "percentage"
) %>%
separate(
percentage,
into = c("pct", "n"),
sep = "%",
convert = TRUE
)
ggplot(cross_tab_long, aes(x = response_content, y = pct / 100, color = treat)) +
geom_point(size = 3) +
geom_errorbar(aes(ymin = (pct - 5) / 100, ymax = (pct + 5) / 100), width = 0.2) +
facet_wrap(~ treat) +
ylim(0, 1) + # Set fixed y-axis limits
labs(
title = "Causal Effects of Treatments on Reply Content",
x = "Response Content",
y = "Causal Effect of Treatments on Reply Content (%)",
color = "Treatment Group"
) +
theme_minimal() +
theme(
plot.title = element_text(hjust = 0.5, size = 16, face = "bold"),
axis.text.x = element_text(angle = 45, hjust = 1),
legend.position = "bottom"
)
4 Conclusion
The study demonstrates that in authoritarian regimes such as China, government responsiveness is significantly influenced by both citizen-initiated pressures and formal institutional frameworks. Specifically, threats of collective action and reporting to higher-level supervisors effectively compel county-level officials to address citizen demands, thereby enhancing overall responsiveness. Interestingly, merely claiming loyalty to the CCP does not elicit increased responsiveness. The researchers suspect this phenomenon may be attributed to the absence of internal reporting networks within the CCP. In authoritarian regimes where coalition-building is central or where strong internal reporting and oversight mechanisms exist, such treatment effects may be heterogeneous and remain uncertain.
While formal mechanisms like the cadre evaluation system compel officials to respond to citizen demands, threats of collective action can further enhance government responsiveness in authoritarian regimes.