Analysis: Yes, National Media, Poor Schools With Big Challenges Sometimes Work Wonders

Years ago, I asked an older friend, Dave, to name the best school district in my home state of Massachusetts. Dave is a former teacher who now works as a consultant helping hundreds of districts improve their operations and staffing. I figured he would start listing communities like Wellesley and Weston: affluent, lily-white suburbs with sterling test results and college attendance rates.

But Dave’s answer surprised me. Those top students, he said, were like masterpieces that had been labored over for years. Their parents had read to them at bedtime, paid for math tutors, taken them to museums on the weekends. A painting by Picasso or Rembrandt looks great in the galleries of the Isabella Stewart Gardner Museum, sure, but it will also shine in a parking lot. You can learn only so much about the actual work of a museum or its staff simply by marveling at its collection.

As for Massachusetts’s best districts, Dave named a few cities in the South Shore and Western Massachusetts, areas that mainly served low-income kids. The schools in those places didn’t matriculate as many students to the Ivy League, but they helped them catch up to children in more affluent areas with far greater advantages. Through skill and dedication, they improved upon the raw materials they’d been given.

I hadn’t previously thought about education that way, and neither do a lot of people outside the field. There’s a prevalent assumption that schools located in upscale ZIP codes, often with spiffy facilities and extracurricular opportunities, are just better — that they hold the secret to higher graduation rates and SAT scores. But what if the truly great schools and districts are the ones that don’t benefit from enrolling the most fortunate students, and instead move mountains to improve the lives of kids who’d founder elsewhere?

***

It’s an old question in the education world, and one that’s received renewed attention since the release of a study by noted researcher Sean Reardon. A Stanford professor who focuses on poverty and inequality in education, Reardon burrowed down into testing data for 11,000 school districts to determine which ones helped students achieve the most growth between third and eighth grade. He found that the richest, whitest areas actually weren’t the best. The highest-flying district, in fact, was the Chicago Public Schools, where pupils’ test scores rose at four times the average rate (two-thirds of a grade level per year, compared with one-sixth) of schools across the country.

Chicago is not often listed among the Westons and Wellesleys of the world. Thirty years ago, then–Education Secretary Bill Bennett dubbed its school system the worst in the country. Unsurprisingly, Reardon’s study was well received by local media. Tennessee, which also received praise, won some good headlines as well, and a few outlets compared results for their local districts with the top performers.

Nationally, however, the commentary around Reardon’s research focused on his central premise: Rich communities that produce well-prepared third-graders don’t necessarily shepherd them to greater progress in the years that follow.

The New York Times’s data journalism division, The Upshot, ran the most celebrated analysis, a widely circulated visualization that allowed readers to search for academic growth in any district in the country. Reardon’s conclusion, authors Emily Badger and Kevin Quealy wrote, “tweaked conventional wisdom”: “Some urban and Southern districts are doing better than data typically suggests. Some wealthy ones don’t look that effective. Many poor school systems do.”

Mother Jones writer Kevin Drum emphasized that point in a long blog post, observing that “lots of poor school districts do pretty well.” In an interview with Reardon, Slate staff writer Isaac Chotiner called his findings “fascinating and surprising.”

Well, that all depends on your audience. Very few experts were likely surprised to read the study, and its conclusions seemed to escape the notice of much of the education community (to the extent that education media commentator Alexander Russo wondered on Twitter why the research hadn’t gotten more coverage).

Those experts may not all agree that Chicago is the highest-achieving urban district in the United States (some, like David Osborne in The 74, differ strongly on this point) but any graduate student in education could observe that schools tend to reflect the demographics of the communities they serve. Because of this, metrics like academic growth — tracking year-over-year jumps in standardized test scores, rather than simply looking at the scores themselves — give a fuller sense of school quality.

Perhaps the biggest moment in education this year occurred at Betsy DeVos’s Senate confirmation hearing, when Al Franken questioned the then-nominee — and who would have expected they’d be in their respective positions a year later? — about using student growth as an analytic measure rather than proficiency on tests. Matt Barnum’s analysis of that exchange was one of The 74’s most-read pieces of 2017!

You can’t understand current debates around education, which burn pretty hot, without understanding this phenomenon. Progressives sometimes look at the lamentable state of many high-poverty schools and assert that efforts to improve them will yield nothing in the absence of a comprehensive anti-poverty agenda. Education reformers reply that there’s no sense waiting for a world without poor families, and that concrete steps can be taken to improve the schools that serve them.

Those arguments don’t always filter down to the public, though. Many reporters still point to school “outputs” (the number of students who do well in class, graduate high school, and attend college) without considering “inputs” (family wealth, English proficiency, special education status). The same Upshot division that publicized Reardon’s findings on Chicago actually made this very mistake earlier this year, when it published a story on real-estate markets and education that (perhaps unconsciously) conflated high standardized test scores with school quality overall.

Embarrassingly, one of the researchers cited by the authors of that piece complained in the comments section that they hadn’t given full weight to socioeconomic circumstances when describing the effectiveness of schools. “To attribute test scores solely to ‘school quality’ ignores the powerful role that family background plays in shaping opportunity,” he wrote.

That researcher was Sean Reardon.

***

There is every reason to think that the biggest education research questions in 2018 will be the same ones we we pondered in 2017: school choice, accountability, equity, and racial/socioeconomic integration. Reardon, one of the most influential education experts in the world, will play a big role in guiding the conversation.

His work in this project would seem to bolster the reformers’ case: that major academic progress is possible, even in exceptionally deprived communities, with enough sweat and brainpower. It may be difficult to pinpoint exactly how Chicago has made such strides over the past few decades, though it has been a playground for reformers (like its former superintendent and later U.S. education secretary, Arne Duncan) since the 1990s.

Representatives from different camps will draw different lessons from Reardon’s analysis, and some will disagree with it altogether.

But those lessons can be of only limited use if news providers, and their customers, continue mistaking the paintings for the museum.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter