Order Bounds

Another approach to bounding the distribution of treatment effects is to place a restriction on the unknown distribution using information about the setting or situation. For example, in economics it is common to assume that individuals choose things like going to college for a reason. In particular, a person may choose to go to college because they believe that their income will be higher from attending college than from not attending college. One such assumption is that the observed distribution of outcomes first order stochastically dominates the unobserved distribution of potential outcomes.

Consider we are interested in the value of graduating high school. For example, we may be interested in a policy that would require all children to complete grade 12. It is difficult to run on an experiment on such a policy, although we may be able to observe different states institute different policies and infer something from the "laboratories of democracy." That said we have lots of information on income for people with different levels of education.

The chart gives the cumulative distribution of log wages for individuals (men from NLS 66), ten years after graduation. It shows that those students who finished high school tended to have higher incomes than students who didn't finish high school. In fact, the income distribution of high school students first order stochastically dominates the income distribution of those who didn't finish high school.

The problem with simply comparing these lines and making a policy recommendation is that we do not observe the distribution of potential outcomes and we do not know the treatment effect of a policy that would require students to graduate high school.

In order to get a handle on the treatment effect we could estimate the natural bounds. About 83% of the students in the data graduate high school. The problem is that the bounds will be so wide that they will not inform our policy question.

An alternative approach is to impose an assumption on the ranking of the observed distributions and the unobserved distributions. For example, we can assume that for those that do not finish high school their observed income distribution first order stochastically dominates their unobserved income distribution from attending college. This is a revealed preference assumption. There are numerous assumptions that could be made to bound the unobserved potential outcome distribution. Another is called "monotonic treatment effects". One version of this assumption states that the unobserved distribution of potential income from graduating high school for students who did not graduate high school, must first order stochastic dominate the observed income distribution for the same students from not graduating high school. The treatment of graduating high school must increase income in the sense of increasing the likelihood of higher income. This assumption is called monotone treatment response. A similar, but different assumption is monotone treatment selection. Under this assumption, those that graduate high school have higher incomes than those that do not. Yet another assumption can be called monotone treatment matching. Under this assumption, those that graduate high school of higher incomes from graduating high school than those that do not graduate high school. The students are "matched" to their best treatment.

We can show that the revealed preference ordering, monotone treatment response ordering and monotone treatment selection ordering do not provide informative bounds on the proportion of people who are better off graduating high school (see Bounds Math). However, the monotone treatment matching assumption is informative.

The chart above presents the bounds under the monotone treatment matching assumption (see Bounds Math). The assumption states that those assigned to the treatment have better outcomes for that treatment, than those not assigned to the treatment. Here it would mean that those who leave before the end of high school do better than those who graduate high school if high school graduates had not graduated high school. I'm not saying this assumption is true. We are simply making it in order to illustrate the idea.

The chart presents the upper bound on the distribution of income from not graduating high school and the lower bound on the distribution of income from graduating high school. It shows that at least 19% of the population is better off if they were forced to graduate high school, although it cannot rule out the possibility that some people are made worse off by the policy.

Hadar, J. and W. Russell, 1969, "Rules for Ordering Uncertain Preferences," in American Economic Review.

Manski, C. and J. Pepper, 2010, "Monotone Instrumental Variables: with an application to returns to schooling," Econometrica.

R Code

Data from David Card.

# load data in from proximity.zip and unzip.

# http://davidcard.berkeley.edu/data_sets.html

x <- read.delim("nls.dat",sep="",header=FALSE, stringsAsFactors = FALSE) # it is a SAS file.

# note that the working directory is set to be the location of the file, which is also where the data is located.

y <- read.csv("names.csv",stringsAsFactors = FALSE,header = FALSE) # I created this file from the log file with variable names.

colnames(x) <- as.vector(y$V1)

x$lwage76 <- as.numeric(x$lwage76)

# Plot distribution of income by education

H1 <- ecdf(x[x$ed76<12,]$lwage76) # creates a cumulative distribution function of log wages measured in 1976, conditional on having 12 or fewer years of education.

H2 <- ecdf(x[x$ed76>11,]$lwage76)

plot(H1,main="Distribution of Income by Education Level",xlab="log wages")

lines(H2,col="red")

legend(6.3,0.3,c("Grade 11 or Below", "Grade 12 or above"),c("red", "black"))

# MTS bounds

p <- mean(ifelse(x$ed76 > 11,1,0),na.rm=TRUE)

lwage_min <- min(x$lwage76, na.rm=TRUE)

lwage_max <- max(x$lwage76, na.rm=TRUE)

x$hs_wage_min <- ifelse(x$ed76>11,x$lwage76,lwage_min) # natural upper-bound on high school income.

x$hs_wage_max <- ifelse(x$ed76>11,x$lwage76,lwage_max)

x$nhs_wage_min <- ifelse(x$ed76>11,lwage_min,x$lwage76)

x$nhs_wage_max <- ifelse(x$ed76>11,lwage_max,x$lwage76)

# minimum on grade 11 income is the observed income.

lower_bound <- ks.test(x[x$ed76<12,]$lwage76, x$hs_wage_min, alternative = "less")

upper_bound <- ks.test(x[x$ed76<12,]$lwage76, x$hs_wage_min, alternative = "greater")

lower_bound

upper_bound # the minimum

# Plot

F2_max <- ecdf(x$coll_wage_min)

F1_min <- ecdf(x[x$ed76<12,]$lwage76)

plot(F2_max,main="Distribution of Income by Education Level",xlab="log wages")

lines(F1_min,col="red")

legend(6.1,0.3,c("Grade 12 or above (min)", "Grade 11 or Below (max)"),c("black","red"))