Race & Class: Chicago Schools Sue State, Claim Minority Kids See 78 Cents Per Dollar Sent to White Schools

KIPP v. UFT: Charter Network Sues Union, Arguing It Doesn’t Represent School’s Teachers

Supreme Court Sets New Standard for Special Ed, Unanimously Rejects Minimal School Progress

D.C. Approves ESSA Accountability Plan That Emphasizes Testing Standards & Transparency

Q&A: Why Miami’s Superintendent, Once an Undocumented Immigrant, Is Banning ICE From His Schools

No Warrant, No Way In: NYC to Prevent ICE Agents From Entering Schools Without Warrants

In California, a New Push for Teacher Tenure Reform as State Bill Would Extend Probation to 3 Years

VIDEO REPLAY: The 74 Moderates a Conversation on America’s Misleading Charter School Narrative

In Pitching School Chiefs on ESSA Plans, 2 Congressmen See Wildly Different Futures for U.S. Education

Effective, Efficient — and Limited: DeVos Lays Out Her Vision of Federal Role to State Schools Chiefs

The Contender: Nick Melvoin’s Plan for Combating Misinformation — and Unseating LA’s School Board President

Is a Maryland Teachers Union Organizer Shepherding Weak ESSA Accountability Bill?

Principal of the Day: This North Carolina School Leader Makes House Calls to ALL 990 Students

Baltimore’s ‘Happy Teacher Revolution’: Educators Band Together to Help Each Other Thrive in the Classroom

Tennessee Legislators Seek State’s First Voucher Law With Targeted Program for Memphis Kids

New Research May Build Discrimination Case for Widely Used Principals Exam

With No Senate-Confirmed Appointees, Who’s Helping DeVos Run the Education Department?

After Heated Partisan Battle, Kentucky Is 44th State to Pass Charter Law

Impact of Weaker Unions in Wisconsin and Other States Much Clearer for Teachers Than Students

The Fifth-Grade ‘Future Chef’: A Grade-Schooler’s Comfort Food Gets Her Closer to National Cookoff

What if No Child Left Behind Worked and Nobody Realized it? Blame the Media.

September 30, 2015

Talking Points

No Child Left Behind increased student achievement, but nobody knows it because journalists won’t cover it.

The conventional wisdom on No Child Left Behind is wrong because the press ignored research

Sign Up for Our Newsletter

The Fact-Check is The Seventy Four’s ongoing series that examines the ways in which journalists, politicians and leaders misuse or misinterpret education data and research. See our complete Fact-Check archive.
If the public knows one thing about No Child Left Behind (NCLB), it’s that it probably didn’t work. It’s a widely held belief; the problem is it’s not true.
“The public perception,” says Stanford professor Tom Dee who has researched the law, “seems to be that No Child Left Behind has failed, but the available research evidence suggests it led to meaningful — but not transformational — changes in school performance.”
Unfortunately, the media has largely not addressed the crucial question of NCLB’s impact on students, favoring instead traditional he-said, she-said style reporting  or offering crude statistical analysis of its legacy.
Take a recent Washington Post news story on NCLB. Reporter Lyndsey Layton explains the law’s impact: “[NCLB’s] goals were later seen as unrealistic, and the law had unintended consequences: Many schools squeezed out art, science and other subjects to focus on math and reading; cheating scandals erupted; and some states lowered standards so their students would appear more proficient.”
There were in fact unintended consequences of the law, but the Post story doesn’t get around to how NCLB did when it came to intended consequence, namely, improving student learning. No research is cited; no researchers are quoted.
When I raised this on Twitter, Layton pointed to the National Assessment of Educational Progress (NAEP), a widely regarded, low-stakes national assessment, saying student progress has been “jagged” and “incremental.” From this, she implied that NCLB hadn’t improved academic achievement.
Education researchers Morgan Polikoff, a USC professor, and Steve Glazerman, a senior fellow at the research group Mathematica, quickly jumped in to explain that it’s inappropriate to use data that way.
In an interview, Polikoff said that simply looking at trends in test score data does not allow for a counterfactual — meaning there’s no way of knowing what the scores would have looked like in the absence of NCLB. That’s why sophisticated analyses are necessary to isolate NCLB’s impact.
Dee also said that NCLB can’t be judged by looking at overall NAEP scores , in part because some states already had NCLB-style accountability before the federal law required it, while others didn’t.
Similarly Matt Di Carlo, a senior research fellow for the Albert Shanker Institute, has written extensively about why using simplistic test score data is misguided for judging individual policies.
Instead, researchers must carefully construct control and treatment groups to determine a policy’s impact. In the case of NCLB that means, for example, comparing Catholic schools (not subject to the law) to public schools; states with accountability policies pre-NCLB to those that didn’t; schools just below a cutoff for NCLB sanctions to those just above the cutoff.
Polikoff and Dee agree that high-quality research on NCLB tends to find small but real gains in student achievement, particularly in math. Dee says the law produced “meaningful, important, and cost-effective improvements.”
On Twitter, Layton expressed surprise that anyone would think that NCLB had led to improved student achievement.
She and the Washington Post are not alone in ignoring the evidence. When the Senate passed a revision to NCLB in July among The New York Times, The Wall Street Journal, The Los Angeles Times, The Associated Press, USA Today, and The Atlantic, not a single outlet cited research about its impacts or spoke to an academic expert.
Instead they all covered the conflict. Teachers unions said this; civil rights groups said that. Republicans want less federal involvement; Democrats want more.
There were a couple exceptions: The Washington Post’s Wonkblog cited and quoted Stanford professor Sean Reardon. Vox had a solid write-up. NPR put together a decent summary of the research literature in 2014.
This is not a new tendency for our press corps, where false balance and conflict often  trump substance. Matt Yglesias recently wrote in Vox about how the media is paying more attention to Hillary Clinton’s email debacle than Jeb Bush’s tax plan.
In education the same problem exists: the boxing match is covered, the research is not. It’s charters vs. traditional public schools, reformers vs. unions, opt-out-ers vs. accountability hawks. Empirical evidence rarely sees the light of day.
Polikoff recently sent around a new, particularly rigorous research paper, which found positive outcomes from NCLB, to a group of education journalists. Not one of them covered it, he says.
“Part of the problem is it’s not a sexy thing,” Polikoff explains.  
He’s right: Getting into the weeds of different ways to evaluate NCLB — ya’ know, interrupted time series, regression discontinuities — makes readers’ eyes glaze over. But that is part of a reporter’s job: to render complex subject matter understandable so the public can make informed decisions
After all, taking research, and all its complexity, seriously is necessary if we have any hope of expanding effective education policies.