Do Charter Schools Work?

by Ray Fisman

On June 4, 1991, Minnesota Gov. Arne Carlson signed into law a bill that set in motion one of the most significant—and controversial—education reform movements in modern history. Minnesota’s charter school law allowed educators and other concerned individuals to apply to the state for permission to operate a government-funded school outside of the public education system. In order to obtain and keep their licenses, these new schools needed to show they were serving their students effectively, based on goals laid out in the school’s “charter.” City Academy, America’s first charter school, opened in St. Paul the following year. Its mission was to get high-school dropouts on track to vocational careers, and it is still operating today. One early enrollee, Demetrice Norris, told the Minneapolis Star-Tribune in 1992 that he had spent years, “being lazy – not doing nothing” before he “got a life back here in school” and “got a chance to be something.”

Whether charter schools have actually lived up to their initial promise is a hotly contested topic in the education reform debate. An entire field of education research aims to assess whether students are better off at charter schools than in the public system. The latest findings, based on six well-regarded charter schools in Boston, released Wednesday by the Boston Foundation and MIT’s School Effectiveness and Inequality Initiative, adds to the accumulating evidence that at least a subset of high-performing charters are measuring up to the movement’s early aspirations of giving disadvantaged kids a shot at a better life. The study shows that the Boston schools’ students did better on SAT and Advanced Placement tests and are vastly more likely to enroll at four year colleges—and to do so on scholarship—than otherwise identical students in the Boston public school system.

What makes a charter school different from other public schools? While they’re funded with public money, they generally operate outside of collective bargaining agreements (only about one-tenth of charter schools are unionized) and other constraints that often prevent principals in public schools from innovating for the good of their students (so the argument goes). In exchange for this freedom, they generally get less funding than public schools (though they’re free to look for private donations, and many do) and have to prove that they are making good on the promises set out in their charters, which often means showing that they improve their students’ performance on statewide standardized tests.

It’s an idea that’s resonated with a surprisingly wide swath of American society, from free-marketeers who like the idea of reducing government involvement in education to anti-poverty activists frustrated by the slow rate of social progress. (Many charter schools focus on serving minority or low-income communities.) Carlson authorized eight charter schools in Minnesota; there are now nearly 6,000 nationwide.

Others are less sanguine about the charter approach. School unions, for example, have been cautious in their support, often seeing charters as drawing funds away from resource-starved public school districts and diverting the discussion from how to fix public schools, which continue to serve the vast majority of American students.

At least part of the disagreement revolves around whether charter schools deliver on their promise to improve student outcomes. You might think this is a relatively easy proposition to evaluate—just compare whether charter school kids do better on tests than those in public schools. But any effort to compare performance is confounded by the fact that the kinds of parents who take steps to enroll their children in charter schools may be the kind of motivated and supportive parents whose children would have done just fine in any school system. (In the current study, charter school applicants do in fact have higher than average test scores even before they enroll. However, other analyses have seen charter school applicants with below-average scores, perhaps because kids struggling in the public system are more apt to look for other options.) And if the longer hours and additional school days that are a feature of many charter schools lead underperformers to drop out, the select group that remains may again be made up of those who would have tested well in any school environment.

But for at least a subset of charter schools, researchers can come fairly close to running a clinical trial where some applicants are enrolled at charters and others are left in the public system purely by chance. The reason is that many charter schools are oversubscribed, and their scarce spots are allocated through a lottery. So whether a particular student gets assigned a slot at the charter school is luck of the draw. (This randomness in gaining admission to sought-after charter schools has been documented recently in the films Waiting for Superman and The Lottery.)

Numerous studies have used this lottery method to analyze the impact of charter schools on standardized test scores, and by and large they report similar findings: Charters in rural or suburban areas don’t do any better than public schools, while in urban areas they are associated with greater test score improvements in math and language. But another important point from past studies is that there is enormous variation in the effectiveness of charter schools. There are some great ones but also some real duds.

The latest study focuses on six Boston schools that, in an earlier analysis, were found to produce large improvements in students’ performance in the statewide standardized test, the Massachusetts Comprehensive Assessment System. In the work released Wednesday, researchers assess whether these test score gains are actually putting students on a path to better lives or simply the result of charter schools’ effectiveness in “teaching to the test” for the MCAS. This is a significant concern for many charter school evaluations, since performance on the MCAS and other standardized tests are what determine whether schools keep their charters. The state doesn’t evaluate schools on whether their students perform well on SAT or AP tests, the high-stakes exams that are analyzed in the current study.

The study examines the college readiness of Boston public school students who applied to attend the six charter schools between 2002 and 2008, with projected graduation dates of 2006–2013. In just about every dimension that affects post-secondary education, students who got high lottery numbers (and hence were much more likely to enroll in a charter school) outperformed those assigned lower lottery numbers. Getting into a charter school doubled the likelihood of enrolling in Advanced Placement classes (the effects are much bigger for math and science than for English) and also doubled the chances that a student will score high enough on standardized tests to be eligible for state-financed college scholarships. While charter school students aren’t more likely to take the SAT, the ones who do perform better, mainly due to higher math scores.

The upshot of this improvement in college readiness is that, upon graduation, while charter and public school students are just as likely to go on to post-secondary education, charter students enroll at four-year colleges at much higher rates. A four-year college degree has historically meant a better job with a higher salary, making a spot in one of Boston’s charter schools a ticket to a better life for many students. (We’ll presumably know in a few years whether things actually turn out that way in the longer run for the cohort the researchers are following.)

Some charter school advocates will surely point to the new study as yet more evidence that public school districts should be replaced by a more decentralized approach to education, with a greater emphasis on charter schools. But that’s a very narrow interpretation of the mounting evidence of charter schools’ successes. Focusing on these successes glosses over the many cases where charter schools fail to outperform their public peers. For suburban districts in Massachusetts, for example, the numbers don’t favor the charter school advocates, in large part because suburban public schools in the state are pretty effective already. What’s more, lottery studies like the one out Wednesday can only be done at schools that are oversubscribed—and hence probably among the better charter schools around. As my Columbia Business School colleague and leading education researcher Jonah Rockoff puts it, saying charter schools are good is “like saying Italian restaurants are good places to eat—some are and some aren’t.” (In theory, underperforming charter schools lose their licenses, but it doesn’t always work out that way.)

The secret of many charter schools’ success isn’t a mystery: longer hours and additional school days, which are part of a “no excuses” philosophy that emphasizes frequent testing and often requires even longer days from charter school teachers. When public schools integrate these elements—as in a pilot project run by Harvard economist Roland Fryer in Houston, early evidence suggests those schools are seeing the same gains as high-performing charters.

But the success of charters doesn’t necessarily provide us any answers about what will improve education on a larger scale. KIPP, the nationwide network of charter schools and one of the great success stories of the charter movement, has 125 schools serving 41,000 students, making it less than one-twentieth the size of the New York City Department of Education. It isn’t clear that there are enough teachers willing to put in 10-hour days or enough parents who will force their kids to endure the extended hours of instruction. To this point, one recent study by MIT graduate student Christopher Walters argues that lack of demand by parents will be a significant limiting factor in the further expansion of charter schools.

Why wouldn’t parents try to get their children into charters that would get the kids on track to go to college? One explanation is that the debate about whether charter schools “work,” with its focus on testing and college placement, loses sight of the many reasons why people choose a school and what they value in an education. Charter schools can be set up for any purpose. Some focus on the arts, others emphasize cultural heritage (there are multiple Hmong charter schools in the Twin Cities alone); some are vocational, others rigorously academic. As Rockoff puts it, asking whether a particular school is good based on test scores or college placement is in many cases the wrong question: “To extend the restaurant metaphor, some people like Italian, others like Thai food. Similarly, many [charter] schools focus on tested subjects, while others might emphasize creative writing or the arts.” Not every charter school is right for every kid. Nor would it be an unmitigated good if all parents could segregate their children by their interests and beliefs.

In their short history, charter schools have shown enormous promise in improving the educations of many disadvantaged students. We shouldn’t lose sight of that, and we should be open to innovations coming out of charters that might be applied to a broader set of schools. But it’s naïve to think that charters will be the silver bullet that solves the education crisis in America.

From:http://www.slate.com/articles/news_and_politics/the_dismal_science/2013/05/do_charter_schools_work_a_new_study_of_boston_schools_says_yes.html