‘Best’ High Schools May Not Be as Good as You Think — What That Means for Parents, Policymakers

And who can blame them? In New York City, specialized schools, like Stuyvesant and Brooklyn Latin, earn top spots in national rankings. In Chicago, Walter Payton High School — which is also nationally renowned — accepted only about one in four students of the more than 16,000 who applied in 2014.

But what if the high school rat race is largely for naught?

That’s the provocative conclusion of a new study that examined students who attended public high schools in Chicago. Surprisingly, students at selective-enrollment schools didn’t seem to benefit academically compared with similar students at different schools.

“There is a lot of competition for students to get into selective-enrollment high schools,” said Marisa de la Torre, a co-author of the study and an associate director at the University of Chicago Consortium on School Research. “What the study says is, it’s not the end of the world if you cannot get into selective-enrollment high schools.”

However, the study found that students at selective schools were less likely to be suspended or to miss school and reported that they felt safer. In addition, students who attended high-scoring non-selective schools did see academically significant benefits relative to those at low-achieving schools.

The study highlights the challenge of measuring school quality and suggests that many parents and education policymakers are judging schools by the wrong metrics.

The study, conducted by four researchers affiliated with the consortium, was published this month in the peer-reviewed journal Educational Evaluation and Policy Analysis. The researchers examined ninth-graders who entered Chicago’s public high schools, including charter schools, between 2008 and 2011. Interestingly, about two thirds did not attend their zoned neighborhood high school in the 2011–12 school year.

The researchers divided schools into four groups: selective, top-tier, middle-tier and bottom-tier. The first group consisted of schools that admit students based largely on test scores. The latter three groups were ranked by their students’ ACT scores and high school graduation rates.

The study compared students against peers who attended different-tier schools but were otherwise similar based on traits including past test scores, degree of parental involvement and home neighborhood. This approach isn’t perfect, but it allows researchers to estimate the impact of schools while holding student characteristics constant.

When simply making raw comparisons between students at selective-enrollment versus other city schools, the differences appear stark: Students at selective schools scored more than seven points higher on the ACT, which has a maximum score of 36. Yet when researchers controlled for a variety of factors to isolate the effect of attending a selective school, the disparities all but vanished. Attending a selective-enrollment school led to only a statistically insignificant bump in the ACT of half a point.

The selective schools also seemed to have little or no effect on the likelihood of taking Advanced Placement classes, graduating from high school or enrolling and staying in college. Attending a selective-enrollment school actually led to a small dip in high school grade point averages.

Selective schools did, however, produce a variety of non-academic gains: Students had higher attendance and lower suspension rates, and they trusted their teachers more. Students also reported that their peers’ behavior was much better and that they felt safer in school — this suggests that insofar as selective schools are beneficial, it may be because of higher-achieving peers rather than better-quality instruction.

Again, to get these results, the study compared students at selective schools with similar students at different schools. Notably, though, the second group of students largely ended up at relatively high-achieving schools in the city — very few went to Chicago’s lowest-scoring schools.

Although students didn’t seem to benefit academically from the selective schools, this might be a reflection of the relative quality of the fallback schools rather than an indictment of the selective-enrollment schools.

The researchers also looked at the outcomes of three levels of non-selective schools in Chicago: top-tier vs. middle-tier, top-tier vs. bottom-tier, and middle-tier vs. bottom-tier.

Students in the top third of schools — those without selective admissions but with relatively high test scores and graduation rates — benefited academically, especially compared with similar students who attended bottom-tier schools. The differences between going to these top-tier schools and the lowest-tier ones were noticeable and evident across a variety of measures: ACT scores, high school graduation, attendance and suspension rates, and college enrollment and persistence rates.

The difference between attending a top-tier school and a middle-tier one was mixed: Attendance at the high-achieving schools led to gains on the ACT but produced small reductions in high school graduation rates, compared with middling schools. Differences in college outcomes generally favored top-tier schools but were usually small and not statistically significant. Similarly, students didn’t seem to make big gains from attending a mid-tier school vs. a bottom-tier one.

This suggests, unsurprisingly, that students are most likely to benefit when moving from a very low-achieving school to a high-achieving one.

The selective school results are somewhat startling: Chicago students received little academic benefit from going to what are widely seen as some of the best schools in the city and perhaps the country.

One possible explanation is that high schools just don’t matter a great deal in terms of academic outcomes — perhaps it’s really all about family circumstances and other outside factors, or maybe by the time a student gets to high school, it is simply too late to change his or her trajectory.

This, however, is belied by the impact of the top-tier schools in this study, as well as other research, including studies of Chicago high schools. One recent study showed that attending a Noble network high school — part of a chain of charter schools in Chicago — led to large gains in college attendance and persistence. An older study showed that charter high schools in Chicago had positive effects on high school graduation rates and college enrollment. Research in Texas showed that school quality had substantial effects on long-run outcomes, including college attendance and adult earning power, and that charter schools there had negative impacts on these measures. Students who won public high school choice lotteries in North Carolina’s Charlotte-Mecklenburg Schools were much more likely to complete college degrees than those who lost the lottery.

Still, this isn’t the first research to suggest that selective schools may not be as good as their reputations suggest. A study of New York City’s specialized high schools, including Stuyvesant, found that students who just made the exam cutoff for admission were no more likely to attend or complete college than students who just missed the cutoff score.

In other words, schools matter, but perception of the best schools may not line up with reality. One partial explanation, reflected in the latest Chicago study, is that the fallback school options for students considering attending a selective school are still quite high-achieving.

But another answer is that the common measurement of schools — raw test scores, unadjusted for differences in the level where students start — is misleading. (And this doesn’t even get into the limits of exclusively using test scores.) In a 2008 survey, an overwhelming majority of researchers said that schools should be evaluated instead based on how much their students grow academically over time.

One recent study suggested that a focus on growth can lead to better policies. Specifically, many schools in New Orleans and Baton Rouge, La., have been closed or taken over by charters in recent years. Students in New Orleans saw big test score gains because of this, but students in Baton Rouge actually lost ground. The difference? In New Orleans, the schools where students ended up were good at helping kids make progress, while the schools in Baton Rouge were not.

Yet many policymakers and parents don’t understand the problem with judging schools by simply looking at the percentage of students who score as proficient on state tests. Great Schools, a popular website for parents, rates schools in most states based exclusively on absolute test performance — though, to the site’s credit, it has introduced growth and other measures in a handful of states.

As long as the popular conception of a good school is based on a flawed measure, excellent schools that don’t screen students by test scores or real estate prices won’t get the credit they deserve — and schools with such careful selection will get far too much credit.