Max Eden is a senior fellow at the Manhattan Institute, specializing in education policy.
.@maxeden99: @nytimes central takeway of @BetsyDeVos and Detroit charter performance based on bad math
Sign Up for Our Newsletter
The campaign against Education Secretary–designate Betsy DeVos has been both predictable and extraordinary. It’s no surprise that the education establishment was perturbed by the selection of a school choice advocate, and opposition from interest groups is to be expected.
But in an era when the president of the United States has declared a “running war” on the media, accusing reporters of distorting facts to attack him, the work of one education journalist unfortunately lends some credence to that argument.
Liberal bias at the Times is less than a non-story; if anything, I’d argue a partisan press is healthy in a pluralistic democracy. But when America’s “paper of record” makes verifiably false claims, they must be checked and corrected. Here are two significant ones.
In a front-page June article titled “A Sea of Charter Schools in Detroit Leaves Students Adrift,” the Times education correspondent asserts that “half the charters perform only as well, or worse than, Detroit’s traditional public schools.”
That claim was echoed by a Times editorial and would be big, if true. DeVos was nominated based on her school choice advocacy. If that work helped foster charter schools that are worse than the worst-in-the-nation Detroit Public Schools, that would be profoundly troubling. But if Detroit’s charters are better (even if not as much better as we’d desire), then it’s a different story entirely.
Fortunately, they are better.
There are three key studies that compare Detroit’s charter and district schools: one from Stanford University, one from the center-right Mackinac Center and one from Excellent Schools Detroit (ESD), a local education nonprofit. As Jason Bedrick, a policy analyst at the Cato Institute’s Center for Educational Freedom, and I demonstrated in Education Next, all three show that charters significantly outperform district schools. Perplexed at how the Times reached the opposite conclusion, I reached out to Zernike.
Some critics assumed that Zernike was twisting data from the Stanford study, the presumptive source of district-to-charter comparisons. But Zernike informed me that she chose to use the ESD study after contacting the Stanford study’s author and determining that the data was too outdated for her purposes.
I asked why she chose the ESD data over the Mackinac Center’s. Mackinac grades schools using a complex regression taking into account students’ socioeconomic background. ESD grades on a combination of raw test scores, test-score growth and a school climate survey, but it doesn’t consider socioeconomic status.
She explained that Mackinac is “a partisan group that is pro–school choice and anti-DPS. ESD, despite how GLEP [the DeVos-backed Great Lakes Education Project] will characterize it, supported charters and traditional public schools, and the measures seemed broader.”
When I told her that sounded more like political than methodological reasoning, she countered, “It’s not politics, it’s methodology. I think graduation rate was the only thing Mackinac used to compare,” and added that she thinks the ESD data “do break down for demographics.” Wrong and wrong.
Now, it’s possible that she didn’t simply default to the politically congenial option without further scrutiny. Perhaps she just failed to properly recall the details several months later. Whatever the case, the ESD data also show charters outperforming district schools.
So, how did the Times national education correspondent reach the opposite conclusion?
Tier 1 and 2=excellent to good; Tier 3=average to weak; Tier 4=weak to failing. (Source: Excellent Schools Detroit)
Now, bear with me, here because it’s complicated and it makes no sense.
First she separated out K-8 district schools and high schools, calculating their respective average scores, weighted by student enrollment. She included high-performing selective-admissions district schools and excluded low-performing Detroit public schools that have been taken over by the state. (Neither decision is justifiable in a traditional-to-charter comparison.)
Then she saw that for both K-8 district schools and high schools, the (inflated) weighted average score was higher than the median charter school score, and concluded that “half the charters perform only as well, or worse than, Detroit’s traditional public schools.”
On the high school side, the unweighted average score of .33 is significantly lower than the weighted average of .41. It’s worth noting that the .41 is above the charter median score and the .33 is below it. So going by the weighted average was the only way to arrive at that result for high schools.
On the K-8 side, the weighted and unweighted averages are essentially equal. That average is indeed slightly higher than the median charter score, but it’s much higher than the district’s median score. So on K-8 schools, by her same faulty logic, it would also be accurate to say that “two thirds of the public schools perform only as well, or worse than, Detroit’s traditional public schools.”
If that sounds silly, it’s because comparing an average to a median is statistical nonsense. The “apples to oranges” metaphor is apt but insufficient here. Essentially, Zernike took a basket of apples, pulled out the rotten ones, kept the genetically modified ones, made statistically weighted applesauce, and plopped that applesauce in the middle of a row of organic oranges. Then she drew a false conclusion that’s become central to the case against Betsy DeVos’s nomination for secretary of education.
Personally, I doubt the mathematical mistakes were conscious or intentional. But what really matters is that the ESD, Mackinac and Stanford studies all show Detroit charters significantly outperforming traditional public schools.
The second claim also involves the Times’s editorial against DeVos, in this case lamenting that she funded charter advocacy efforts, “winning legislative changes that have “reduced oversight and accountability.” The editorial linked to a December article by Zernike covering a legislative debate on Detroit charter regulation wherein “Ms. DeVos pushed back on any regulation as too much regulation.”
Whatever the rhetorical merit of that editorial claim, it is flat false. In a Detroit News op-ed, to which the article later links, DeVos called for two additional regulations: A–F school accountability grades and default closure for failing schools, both charter and district. She certainly pushed back on some regulations as too much. But the bill that passed included the additional accountability regulations for which she advocated. In fact, the final legislation boosted Michigan’s accountability score on the National Alliance of Charter School Authorizers index.
Given the fact that the main subject of her article was a net increase in charter accountability, Zernike admits on Twitter that she’s “not sure what the ed board meant by that,” but notes that “MI legislation in 2011 (not June bill) did weaken oversight.” Zernike’s December article refers to the 2011 legislation in one passing sentence. Her June article noted that “the law repealed a longstanding requirement that the State Department of Education issue yearly reports monitoring charter school performance.” While true, that provision didn’t merit mention among the 12 key changes in the official legislative summary (five of which increased charter regulation).
It’s possible that the Times’s editorial was referring to that repealed reporting requirement from 2011 when it claimed that DeVos backed “legislative changes that have reduced oversight and accountability.” But that seems unlikely, given that the editorial linked to Zernike’s December article on the 2016 legislative debate and that piece doesn’t even mention the 2011 provision. It seems more likely that the editors honestly confused an increase in accountability that was smaller than some stakeholders wanted with an actual, absolute reduction. And given the reporting they relied on, it would be hard to blame them.
Education blogger Alexander Russo has skillfully outlined the “problematic media coverage” of Betsy DeVos, in which journalists have latched onto hyper-simplified story lines while ignoring complexities and eschewing nuanced criticism.
Whatever your take on DeVos or the media, everyone loses when the line between fact and falsehood is blurred beyond distinction. At a time when the president’s advisers proudly tout “alternative facts,” critical, fact-based reporting is more necessary than ever, especially from outlets with the weight and influence of The New York Times. Their readers, and America’s schoolchildren, deserve better. Correcting the record would be a good start.