DeVos Clashes With Democrats Over Federal Protections in School Choice Budget Proposals

Weekend Education Reads: 12 Worthwhile Links About Students and Schools You May Have Missed

Wisconsin Lawmaker Tells Betsy DeVos That Vouchers Don’t Help Rural Kids. Research Shows He’s Right

You’re Not an ‘Interest Group’ Just Because You Believe School Funding Matters

How 2 Business-Savvy Nonprofits Are Breathing New Life Into Philadelphia’s Struggling Catholic Schools

State of American Pre-K: New Report Shows 1.5 Million Kids (and 1 in 20 3-Year-Olds) Enrolled

Trump Ed Budget Fleshes Out Choice Proposal, Justifies Deep Cuts

Mayoral Control Fight Heats Up: NYS Assembly Gives De Blasio 2 More Years, But Senate Demands More Data on School Spending

The Boys Wouldn’t Listen, So 9-Year-Old Girl Started Her Own Robotics Team — and Won Big

Candidates Are Drawing School Choice Battle Lines in Campaign to Succeed New Jersey Gov. Christie

Start School Later: New Study Shows That More Sunlight Before Classes Improves Test Scores

Facing Pressure From Conservatives, Texas Again Looks to Ban Transgender Bathroom Use

DeVos Emphasizes States’ Prerogative on School Choice, Gives No Details on Federal Expansion

Inside the School Lunch Affordability Gap: Too Affluent for Free Food, Can’t Afford $1.75 Meals

Long-Vacant NYC Courthouse Will Get New Life as Success Academy’s Second High School

1 in 5 Washington State HS Students Considered Suicide, 9% Attempted It, Shocking New Report Finds

Key Congressional Ally Backs Trump Admin’s School Choice Plan

Why the LA School Board Swung Pro-Reform: Did a Late Vote on ‘Charter Killer’ Bill Cost Board Chief His Job?

Federal Judge Denies NYC KIPP School’s Effort to Block Arbitration of Teacher Grievances

New Report Shows NYC’s Alternative to Charter Schools — Supported by De Blasio and UFT — Aren’t Getting Results

Research Shows Students Can Benefit When a School Closes — but Only If There Are Better Ones to Attend

Photo Credit: Getty Images

October 17, 2016

Talking Points

Researchers find that students in New Orleans, but not Baton Rouge, were helped by school closures

Sign Up for Our Newsletter

When schools close, students benefit if they end up attending a better school — but they’re hurt if they go to a worse school.

These findings — from a study released Monday by Tulane University’s Education Research Alliance — may seem obvious, but they’re important, as states have increasingly used school closures and takeovers as a turnaround strategy. In deciding whether to shutter a school, policymakers ought to consider how they’re measuring performance and whether new schools would be any better than those they replace.

Positive results in New Orleans; negative results in Baton Rouge

The research, conducted by Whitney Bross, Doug Harris and Lihan Liu, examined what happened to elementary and high school students in New Orleans and to high school students in Baton Rouge after their schools were closed or taken over by a charter school. In New Orleans, they examined 11 school closures and 15 charter takeovers; in Baton Rouge, three closures and two takeovers.

New Orleans has been at the heart of controversy after virtually all the city’s public schools became charters. Previous research found that the post–Hurricane Katrina package of changes — including the expansion of charter schools, a stringent system of test-based accountability and an infusion of new resources — led to large increases in test scores. Student mobility also declined, and another recent study found that students in New Orleans charters seemed to do better on measures of behavior.

The reforms included closing or replacing schools that were underperforming, as measured by test scores. To assess the impact, the researchers in this study compared students in closed or taken-over schools with students at similar schools that were not eliminated. They found that the strategy seems to have been effective in New Orleans, where students saw large increases in standardized test scores. But in Baton Rouge, the effects were the opposite: High school students were significantly harmed when their schools were shuttered.

Photo: Education Research Alliance

Effects on high school graduation rates mirrored the test score results: New Orleans students from closed or replaced schools were much more likely to complete high school, while Baton Rouge students were less so. In New Orleans, the effects on college entry were close to zero, while in Baton Rouge the impacts were more negative, but not statistically significant.1

Photo: Education Research Alliance

The research also found that the effects of closure or takeover tended to be more positive in elementary than high school, and when schools were phased out over time rather than closed immediately.

The study points out that the largest impact from the elimination of struggling schools may fall on future students who otherwise would have attended those schools — since such benefits (or harms) could accrue to multiple generations of students. These future students may stand to benefit in particular because they do not experience disruptive effects of closure, like having to switch schools.

Although they don’t have direct data on this, the authors conducted a simulation based on the closures’ effects on existing students to estimate the impact on theoretical future students. They conservatively estimate that between 25 percent and 40 percent of the test score gains caused by the New Orleans reform package can be attributed to the closure or replacement of low-performing schools.

Squaring the divergent results: quality of replacement schools matters

As this study notes, past research on school closures has produced mixed results, particularly on students directly impacted. However, some research has suggested negative effects on students in schools that receive an influx of new students due to closures. Little research exists regarding students who would have attended schools that were closed — but one study did find large, positive results in New York City.

Photo: Research compiled by Matt Barnum. Specific studies: Houston, NYC, Ohio, Milwaukee, Michigan, DC, Anonymous, Chicago

Even less research exists on charter school takeovers. One paper found no impact in Memphis, but another study found big test score gains from takeovers in Boston and New Orleans.

The Tulane report emphasizes the commonsense idea that these sort of interventions work only when new schools are markedly better than the ones closed or taken over. This squares with some previous studies, as well as with the Louisiana data: Replacement schools seemed to be much better in New Orleans, but not in Baton Rouge, and student gains varied based on the effectiveness of the school students moved to.

Photo: Education Research Alliance

The report notes, however, that there was “no clear and consistent pattern in how high school graduation and college entry effects relate to school quality improvement.” This suggests, consistent with past research, that measuring schools based solely on test scores doesn’t capture the entirety of their effect on student outcomes.

How to make school closures more effective

It’s clear that simply closing a “failing school” is no guarantee that students will benefit. In fact, in places like Baton Rouge and Milwaukee, school closures hurt the kids affected by them, probably due to a combination of the instability created by closures and the quality of the students’ new schools.

Policymakers considering closing schools should proceed with caution, unless there’s good reason to believe that students will end up at a more effective school. For instance, Michigan is considering large-scale closure of schools with low test scores, but Secretary of Education John King said in an interview with Chalkbeat Detroit, “The key question is: What are they being replaced with? What are the opportunities that will be available to students instead?”

The results also point to the importance of measuring school quality appropriately. In judging schools, some focus on raw proficiency rates, but most researchers recommend evaluating schools by students’ test score growth, since schools don’t control the starting point at which students come in. Focusing exclusively on test scores also excludes other important areas of performance.

Closing schools based on bad or limited data is unlikely to be effective and may be harmful.

Beyond student outcomes

On its own, research can’t answer whether school closures are a good idea.

The New Orleans data suggest that, at least in some circumstances, closures and takeovers can have significant positive effects on student outcomes, including test scores and high school graduation rates.

But this does not assess the broader impact of this strategy. Some argue that closing traditional public schools has negative effects on communities. In Chicago, for instance, the majority of schools closed by the city remain empty, becoming eyesores and magnets for crime. In New Orleans, the expansion of charters and closure of neighborhood schools led to a large decrease in the number of black teachers and bred significant community resentment.

Others have argued that closures are undemocratic, conducted in some places not by locally elected school boards but by state-appointed leaders. Indeed, in New Orleans, but not in Baton Rouge, the schools were largely under state control (though most were recently returned to community oversight).

Of course, the expansion of charter schools and replacement of less effective schools may also have certain benefits to local communities.

These potential costs and benefits may be real — even if they can’t be captured by research studies.


Footnote

1. The researchers distinguish between raw and adjusted college entry rates, with the latter accounting for changes in high school graduation rates, since an increase in high school graduates might reduce college entry. As the authors explain, additional high school graduates may be “less likely to go to college than those who would have graduated anyway, [which] would artificially reduce the interventions’ estimated effect on college entry.” Because of that, the adjusted college entry rate likely provides a “more realistic assessment of the college effect, because students who barely graduate from high school go to college at much lower rates than those who graduate more easily and without the help of school interventions.”