Our School’s Too White? Outraged Parents Vow to Lie About Child’s Race to Keep City From Removing Teachers

Ivanka Trump, Betsy DeVos Tout STEM Education to 200 Students at Air & Space Museum

Black Girls 6 Times as Likely to Be Suspended as Whites. ‘Let Her Learn’ Looks to Reverse the Trend

WATCH: 4,000 Kids Take Over NYC’s Radio City Music Hall With America’s Biggest Academic Pep Rally

Where Education Research, Politics & Policy Intersect: 3 States Reveal How Data Help Shape Their ESSA Plans

Tennessee Bets Big on Personalized Learning, Launching Pilot Program & Eyeing Big 2020 Goals

MUST-SEE: School Films Epic 12-Minute ‘Trolls’ Music Video to Lift Spirits of Sick 5-Year-Old Girl

In Uniting Students With Prospective Employers, the Whether Job Search App Wins SXSWedu Tech Competition

More HS Students Are Graduating, but These Key Indicators Prove Those Diplomas Are Worth Less Than Ever

Race & Class: Chicago Schools Sue State, Claim Minority Kids See 78 Cents Per Dollar Sent to White Schools

KIPP v. UFT: Charter Network Sues Union, Arguing It Doesn’t Represent School’s Teachers

Supreme Court Sets New Standard for Special Ed, Unanimously Rejects Minimal School Progress

D.C. Approves ESSA Accountability Plan That Emphasizes Testing Standards & Transparency

Q&A: Why Miami’s Superintendent, Once an Undocumented Immigrant, Is Banning ICE From His Schools

No Warrant, No Way In: NYC to Prevent ICE Agents From Entering Schools Without Warrants

In California, a New Push for Teacher Tenure Reform as State Bill Would Extend Probation to 3 Years

VIDEO REPLAY: The 74 Moderates a Conversation on America’s Misleading Charter School Narrative

In Pitching School Chiefs on ESSA Plans, 2 Congressmen See Wildly Different Futures for U.S. Education

Effective, Efficient — and Limited: DeVos Lays Out Her Vision of Federal Role to State Schools Chiefs

The Contender: Nick Melvoin’s Plan for Combating Misinformation — and Unseating LA’s School Board President

What Happens When a School Gets a Failing Grade? It Gets Better

June 1, 2016

Talking Points

Research shows that giving a school an ‘F’ spurs meaningful improvement

Sign Up for Our Newsletter

States will be required to give every school a “comprehensive, summative” score rating its performance under draft regulations proposed by the Department of Education for implementing the nation’s new education law.
State education departments typically provide a “data dashboard” for each school that includes a variety of metrics. If the new proposal goes into effect, they will have to combine those metrics into one total score — a letter grade, number, ranking, among other options — which some states and districts already do.
Some educators and analysts are skeptical that such an approach is supported by evidence. Josh Starr, the head of Phi Delta Kappa International and former Montgomery County Schools superintendent, tweeted as much. A report from the University of Colorado’s National Education Policy Center released last year claimed, that A–F school report cards “merit a failing grade.”

In fact, there is evidence that suggests giving grades to schools has an impact on student outcomes, at least for schools that get the lowest grade. According to research in New York City and Florida, these schools respond by improving their students’ test scores — and it doesn’t appear that gains are just the result of gaming the system or teaching to the test.

School grades lead to gains in New York City and Florida
In 2014, New York City removed letter grades from its school report cards in favor of a dashboard approach providing disaggregated ratings on a variety of measures, including test scores, strength of instruction, and quality of school environment.
Two earlier studies published in peer-reviewed journals showed that, under the old system, students in New York City schools that received an ‘F’ made somewhat larger than expected test score gains the following year.1 The researchers were able to determine the impact of the letter grade by comparing schools near the score cutoff between an F and a D or C — schools that are similar, with the only difference being how they were labeled.2
New research from University of Colorado at Colorado Springs Professor Marcus Winters, released by the Manhattan Institute, a conservative think tank, uses more recent data that once more shows students in F schools, under the old system, making significant test score improvements.3
Winters then looked at the new system, reconstructing the report cards using 2014 data and determine the grades schools would have received under the previous regime. Based on 2015 results, he finds that once letter grades stopped being used, the test score bump associated with them disappeared.
Devora Kaye, a spokesperson for the New York City Department of Education, said in an email, “Letter grades were misleading and oversimplified school quality which is why the new Snapshot evaluates schools using multiple measures and more data so families have a complete picture of a school.”
The research in New York City generally squares with multiple studies from Florida, which also show test scores increased after schools received an F.
Gaming or learning?
Some observers have questioned whether the gains were the result of greater learning — especially given a good deal of evidence that evaluation by test scores can lead to unintended consequences like cheating and teaching to the test.
We can’t say for certain but the research gives us reason to believe that the gains were at least in part educationally meaningful.
One of the New York studies showed increased parental satisfaction, measured by district-administered schools surveys, along with the rise in test scores. The schools tended to spend more time on direct instruction, which some research has found to be quite effective. On the other hand, student satisfaction in schools dropped as this shift occurred.
Another of the New York City studies found test scores gains persisted two years after schools received an F rating, suggesting that the improvement wasn’t solely the result of interventions like short-term test-taking strategies.
In perhaps the most comprehensive review of school grades, a Florida study found that after receiving a failing grade4, schools “appear to focus on low-performing students, lengthen the amount of time devoted to instruction, adopt different ways of organizing the day and learning environment of the students and teachers, increase resources available to teachers, and decrease principal control.” In response to accountability pressure, in other words, schools didn’t just start teaching to the test; they made significant changes. Moreover, test score gains persisted three years after the initial improvement.
Another study of Florida found that improvements on low-stakes exams were about half the size of those on the high-stakes tests used for accountability purposes. This suggests that gains may have resulted from some combination of “gaming” and meaningful improvement.
Unanswered questions, unintended consequences
This research should not be read as closing the case on whether giving letter grades to schools is a good idea.
Perhaps the biggest caveat is that studies have generally focused on the impact of grades on low-performing schools. It’s harder to know how the report card approach affects the system as a whole; that is, the research can show that F schools make more gains than D schools, but it doesn’t tell us the aggregate effect of using letter grades as opposed to a different system. (However, research has found that stringent accountability systems generally tend to improve overall student achievement.)
There also may be unintended consequences of school letter grades. For instance, New York City faced frequent complaints that letter grades bounced around significantly from year to year for no apparent reason; there is also evidence that the system reduced parental support for higher standards when they led to lower school grades.
In Florida, one study showed that a failing grade created a significant increase in teacher turnover — particularly among the most effective teachers. (The research also found that teachers who remain showed improvement, consistent with the studies that finding gains in student achievement.)
There are many factors to consider when designing an accountability system; reasonable people can disagree about how to weigh the costs and benefits of different approaches. It is clear, though, that by by some measures, students in low-performing schools appear to benefit from grading schools.
Disclosure: The Walton Family Foundation, which is a funder of The 74, also funded the Manhattan Institute study on New York City school accountability
Footnotes:

1. One study showed gains in both math and English with larger increases in math; the other showed gains in English, but no gains in math. (return to story)

2. Some may be wondering whether the gains are simply the result of “regression to the mean.” That’s quite unlikely because the researchers approach compared schools with relatively similar starting levels of achievement. (return to story)

3. The gains Winters finds are statistically significant but about half the size of those found in one of the previous studies. (return to story)

4. A failing letter grade also meant that students at the school had the opportunity to use a voucher to attend a private school. The study examined the combined impact of the letter grade and this ‘voucher threat’ — though other research suggests that most of the school improvement was due to stigma associated with the failing grade. (return to story)