What does a "good school" mean?

At first glance, improving the US public school system seems simple: reward and replicate good schools; close bad schools. To accomplish this, good schools must be identified.  This, too, seems easy at first.  A good school is a school with good test scores.

Or is it?

When school performance is measured by any sort of standardized test, a strong correlation is always found with the percent of students living in poverty.  A representative example of this correlation is shown in the following Figure.  Here, the SAT scores of all Texas high schools that reported these numbers (that is, all schools where more than about 10 students took the exam) are shown versus their percent of economically disadvantaged students.  These data were taken from Texas AEIS reports (school report cards) obtained from the website of the Texas Education Agency.

This is just one example.  Many similar graphs could be made using state standardized tests from any of the 50 states, or ACT tests, or any other widely accepted standardized test.

Figure 1.

When newspapers report school performance, or when politicians or education reformers speak of it, they seldom refer to graphs like this.  Usually they will simply say that School A is a "high-performing" school, and school B is "under-performing."  The problem with this is that it leads to the conclusion that there is something wrong with how School B is run.  The administration must be bad, or the teachers, or the curriculum.  In fact, the only thing "wrong" with School B, relative to the other schools, is that it has a very high percentage of children living in poverty. 

Here's a simple analogy.  Factories A and B are both making a product, but they are not allowed to choose their suppliers of input materials.  Factory A is lucky and is getting input materials
from its supplier that are completely refined and ready to be put into production.  But Factory B is forced to get input materials from its supplier that are not completely refined, and therefore cause the output quality to be lower according to a rating system that is widely accepted.  Which factory makes the better product?  Factory A, of course.  With more suitable input materials, an average factory will make a better product.  But does that mean that Factory B is poorer at making the product?  No!  It is not possible to answer the question of which factory is better.  It could even be that Factory B actually has a better manager and better workers; it is impossible to say without controlling for the difference in input materials.  In the following figure, which uses the same numbers as before, this hypothetical scenario is shown by relabeling the axes:

Figure 2.

Returning to the real-life case of Figure 1, the regression line or "average" line represents the average performance of schools at a particular poverty level.  Schools A and B were chosen to both lie exactly on this line.  Some schools lie above the average line.  It would be far more meaningful to use this as a criterion for "high-performing," as the results suggest there is something about such schools that is making them "beat the odds."

Even this criterion, though, should be used with caution.  Poverty level is not the only predictor of school performance, as among students living in poverty, some will be more gifted and/or harder-working than others.  Thus, if a school has any ability (whether legally-sanctioned or not) to influence the selection of its students, this could lead to a high-poverty school that seems to "beat the odds."  For this reason, the percent of at-risk students (which is not the same as the percent in poverty) and the percent of special-education students should also be considered when comparing test scores across schools.

Despite this caveat, considering the percent of economically-disadvantaged students when comparing schools represents a huge improvement over simply comparing test scores.

This raises the question of why so many systems of rewarding schools still do not take poverty into account.  One egregious example is the National Blue Ribbon Schools Program, administered by the federal government's Department of Education.  According to the program webpage on eligibility, there are two ways that schools can be eligible for a Blue Ribbon award.  One is by being an "Exemplary Improving School."  For this category, a school must have at least 40% of its student body coming from a "disadvantaged background."  While this takes poverty into account to some extent, there is still a huge difference between schools with 40% and 100% economically disadvantaged students, yet this category is judging such schools relative to each other as if the playing field were level.  This defect, however, pales in comparison with that of the second category, the "Exemplary High Performing Schools."  According to the ED website, these schools qualify if "Regardless of the school's demographics or percentage of students from disadvantaged backgrounds, the school is high performing."

The major flaw with this is that it rewards "country club" schools - schools that have extremely low levels of poverty such as "School A" in the above Figure, but whose performance is around or less than average for their demographics.  What benefit is to be derived from rewarding such schools?  There is no reason to believe that their test scores are replicable, except by creating another student body with a very low poverty level.  

The criteria for both these categories of Blue Ribbon winners are extremely unfair to schools with high poverty levels that are performing better than average for their poverty level.

Rewarding "country club" schools is particularly problematic in open enrollment situations, or when school operators have freedom to influence the student demographics

Even in the traditional district school environment, rewarding "country club" schools accomplishes nothing constructive as far as improving our schools overall.  These schools cannot serve as "models" for other schools, until someone finds the magic key to eliminating child poverty.

When open enrollment allows parents to choose their school, or in the case of charter schools which have been found in some cases to create selective student bodies through methods that while not officially condoned are nevertheless possible and tolerated at present, the rewarding of "country club" schools becomes extremely harmful.  This is because it results in the diversion of large sums of money to school operators who have not necessarily demonstrated superior performance.

Officially, charter schools have to conduct a lottery in most states if the number of applicants exceeds the number of spaces.  In nearly every state (New Hampshire appears to be an exception) charter law prohibits the consideration of
academic performance, parent income, behavior (except for some extreme cases) or other factors when admitting students.  In practice, however, there are ways to select out unwanted students and get away with it.  Not all charter operators do this, but we have heard enough reports from different parts of the country to know that this does happen. 

Here are some tactics that charter operators can use (and easily get away with) to select for a more desirable student body:
  • psychological tactics (marginalizing parents or students, making them feel uncomfortable)
  • "counseling out" (telling parents "Another school may be a better fit for your child." "Our school may not be able to meet your child's needs" etc)
  • excessive use of in- or out-of-school suspension; other excessively punitive measures for small infractions
  • manipulation of grades or awards to favor certain students
  • admissions application forms that require students and parents to fill out lengthy questionnaires listing awards,accomplishments and interests; to divulge whether the student has an IEP; to list any medical problems; to state parents' occupation and employer; or to write essays
  • requirement of "placement test" before the student is admitted
  • excessive pressure to donate to the school (may make lower-income parents feel awkward)
  • requiring volunteer hours of service to the school (may be especially burdensome for lower-income or single parents)
Some readers may think it is impossible to apply such techniques without parents objecting publicly.  In fact, parents are very unlikely to complain.  If a child has been suspended numerous times, or appears to be doing poorly, and if the administration is telling the parent the child is "not fitting in" well, the parent may be very reluctant to bring such information into the public eye for fear of adverse consequences to their child or their child's record.  Also, it is virtually impossible for a parent to prove that their child was not admitted to a school because of deliberate exclusion.  No parent has access to data on the school's applicants, which they would need to show that lower-performing or perhaps lower-income students were systematically excluded.  Moreover, most government entities charged with oversight of charter schools will not get involved in such issues.  Further, parents who are unable to donate money or time to a school are far more likely to succumb to feelings of inadequacy or shame than to publicly question the school, especially when they see other parents willing and able to make such contributions.

The following figure shows just one example of how the demographics of a "school of choice" can change dramatically over time:

Figure 3.  Demographic changes in the enrollment of Magnolia Science Academy San Diego, a California charter school, over a 6-year period

While deliberate engineering of the student body by charter operators is one possible explanation for shifts in a charter school's demographics over time, or for it having a very different demographic from its resident district, it is not the only one.  Absent any outside interference, parents themselves create self-segregation in schools.  This happens in at least two ways.  One is that higher-income parents tend to gravitate towards schools where the students already, on average, are coming from higher-income families. 

Also, the desire of many parents to send their child to the "best" school, when not offset by any intervention, has the long-term effect of causing student bodies to self-segregate by student performance, as illustrated in the following Figure (which is an illustrative schematic that does not represent any actual school).   This effect does not, of course, continue without limit, since not all parents are willing or able to drive a longer distance, etc, to be able to choose a "better" school.

Figure 4.
The desire of parents for school choice is entirely understandable.  Naturally every parent wants the best for their child and they cannot be blamed for this.  However, the effects of this on the societal level must be understood.  Rewarding schools only because they have a higher-performing student body as the result of segregation by income and/or ability, whether the segregation occurred as a consequence of geographical location, through intentional "engineering" by the school operators, or through voluntary self-segregation by parents, is counterproductive.  The reason it is not helpful is that there is no evidence that these schools are actually doing a better job of educating their students.  In some cases, they may even be doing a worse job, but that may be overcome by the favorable demographics.

Of course parents love their "country club" school - but should the rest of us pay for it?

One point about charter schools that may seem confusing to outside observers is that in many cases, it is people outside the school who are critical, while the parents of students who attend the school support it almost fanatically, and react aggressively to any critics.  If the school were excluding certain students, or treating some students unfairly because they were less desirable, wouldn't parents be complaining?

This can be understood by considering that not all parents are in the same boat - and not all of them carry the same weight in the press or with politicians and the community.  A charter school can effectively exclude a number of special education or lower-performing students, while pleasing the parents of one award-winning, high-performing student, and still come out way ahead in public relations.  It is not easy for a parent to come forward to the public with a complaint that their lower-performing or special-education child was excluded.  Parents generally worry in such cases that their child will be stigmatized for life.  Such parents are far more likely to quietly slip away, looking for a school that has caring personnel and a system set up to actually handle their child's needs.  For the parent of the high-performing student, the situation is quite different.  This parent need not worry about any adverse consequences of publicizing their child's accomplishments.  Their public statement that the school is "excellent' carries huge weight with the community and with government officials.  If such parents are happy, the typical reasoning goes, surely the school must be doing something right.

And so it is very easy for a school to engage in, and succeed with, what might be termed "Country Club Collusion" - a collusion between a select group of parents and the administration to create a selective, "country club" school:

Figure 5.

The question might be raised, if the parents are happy, what is the problem here?  There are three main problems.  The first is that taxpayers are forced to pay for an elite school that while theoretically accessible to everyone in practice is not.  The second problem is that these schools are held up as shining examples of success in education reform, when in fact they have accomplished nothing that could not be done by simply replicating magnet schools.  The final problem arises when such schools are allowed to operate with extremely lax oversight.  The administration may be abusing public funds, diverting them in ways that are not benefiting the students.  As long as they offer the parents what they want most - a selective demographic - the parents will often remain very happy with the school despite any such abuses.  There is no reason, however, for the general public to be happy.  Their tax money could give better results for the public as a whole if used in other ways.

Should there be special schools for gifted or easier-to-educate students?

While it is unacceptable to intentionally segregate students by parental income, there are legitimate arguments that it may in some cases be beneficial to allow students to be separated by ability, and this is in fact already done in some districts that have magnet schools.  Parents of gifted students want their children to be challenged, not bored, in school.  It is also difficult to dismiss parents' concerns about having their child in the same classroom as an extremely disruptive or bullying student.  (An immediate caveat here is that in some cases a child may be both gifted and extremely disruptive or bullying.)

As understandable as such parents' desires may be, acceding to them can easily lead down a slippery slope.  In the context of our nation, which has huge disparities in income, a highly diverse population, and a history of racial tension,  it is difficult to ensure that separation be truly for acceptable reasons, and that it not morph into an excuse for segregation by race or income level.  Strict safeguards and diligent enforcement are needed to prevent abuse.  While every parent may wish that the one student in their child's class who is prone to bullying or disruption be simply removed, this does not answer the important question of where that student should then go.  Any classroom or school with a selective enrollment is, then, simply passing on some of society's most difficult problems to another classroom or school. 

Some would argue that separation by academic ability or performance is wrong under any circumstances, as it will inevitably result in unequal treatment, as it runs contrary to the egalitarian principles that we as a nation are supposed to hold, and as it deprives students of the benefits of interaction with other students who are at a different academic level.  While these arguments sound compelling, in practice, classrooms where a single teacher must simultaneously address students of widely disparate levels of preparation, ability or attention usually function very inefficiently, and result in bored or frustrated students.  Parents, too, are frustrated when they feel their child could make much more progress in a different environment. 

The general question of when and if it is appropriate to separate students by ability, performance or behavior is therefore complex, but it should be addressed honestly. Instead, it seems that there has been an attempt to bypass the legal and ethical difficulties of this question through a parallel system (charter schools) in which de facto segregation occurs, even though the law supposedly disallows it.  There is much evidence that many charter schools, particularly the ones held up as successful models, end up with a student body of easier-to-educate students, even though charter law in most states prohibits selective admission based on academic ability.  (Similarly, although charter law in every state prohibits any admission criteria based on race or ethnicity, this has not prevented de facto segregation from occurring along these lines, as evidenced by an April 2011 article in the Orlando Sentinel, "Florida charters less diverse than other public schools."At least one state, New Hampshire, explicitly allows aptitude to be considered for admission: "Chartered public schools may select pupils on the basis of aptitude, academic achievement, or need, provided that such selection is directly related to the academic goals of the school" (NH RSA 194-B:9(c)(1)). 

Regardless of whether many charter schools end up with an easier-to-educate student body through legally allowable selective admissions, or through intentional action by the school operators, or through a self-segregation process resulting from voluntary parental decisions, the key point is that it is happening, for whatever reason.  It is unlikely that any government intervention can change this as long as the charter school system remains in place.  Whether one sees it as right or wrong, then, a certain amount of separation of students by performance or ability is going to continue for the foreseeable future.

Given this, for parents to have true choice, and to ensure that the system operates on a level playing field, in any area where charter schools are operating, district schools should be given the latitude and resources to run their own magnet schools. 

In summary:

  1. A school should not be labeled as "excelling" by our government simply because it is producing good test scores.  Demographics, the percentages of at-risk and special-education students, and the academic level of the incoming student population must all be accounted for in comparing schools.  There should be an end to rewarding "country club" schools that succeed merely by virtue of more favorable demographics. 
  2. The US Department of Education needs to replace its current criteria for selecting National Blue Ribbon schools with another system that ensures a level playing field for comparisons.
  3. Politicians should look carefully at the evidence that some "high-performing" charter schools are in fact "country club" schools, before they set up these schools as examples for national education reform.
  4. There should be an honest, open debate about if, how and when our society will permit separation of students according to their performance.  The debate should not be bypassed by creating a parallel system where de facto separation occurs even though it is not legally sanctioned.
  5. Where charter schools are allowed to operate, districts should also be given the freedom to create magnet schools, to ensure fair competition and true choice for parents.
  6. The fact that a school is considered desirable by parents may be due to its selective enrollment and should not be taken as proof of exceptional competence of its operators.
  7. Our nation should think carefully about what is meant by the term "good school."