Students who receive school vouchers to attend private schools in Ohio see significant drops in achievement, according to a study released today by the Fordham Institute, a conservative think tank. The study finds that public schools improved in response to competitive pressure from vouchers, however.
David Figlio, the study’s author, said the overall impact of the program was positive because many more students were affected by the competitive effect compared to the number of students who actually used vouchers.
Still, the negative finding for voucher participants is sure to spur debate. Until recently, research consistently found that programs allowing students to attend private schools with public dollars had no impact or small positive effects on test scores. But after two studies in Louisiana and another in Indiana, the Ohio research marks the fourth recent finding that vouchers can lower achievement among students using them.
The Louisiana findings led to a lengthy discussion among school choice supporters as to why studies have started to produce negative results. The latest research will surely spark further debate. Here are several potential explanations — some complementary, others contradictory, and still others unrelated.
It’s standardized tests
University of Arkansas professor Jay Greene has argued that standardized test scores are poor measures of school quality, pointing to studies of charter schools showing gains in achievement without increases in educational attainment. He contrasts these with research on vouchers finding they increase graduation rates and college attendance. (One of the cited studies, notably, finds that vouchers, had no impact on college enrollment and graduation, on students overall, but did have a positive impact for African-American students.)
This theory suggests that schools accepting vouchers have a positive effect on students that simply isn’t captured by standardized test scores.
However, there is evidence that test scores are related to long-term outcomes — though do not capture the full impact of schools. “My view about test scores is that they're one of many indicators of things we care about,” said Figlio. The Ohio study only looks at test scores, so can’t shed much light on this theory.
It’s the specific test
A related hypothesis is that standardized tests based on state standards will disadvantage private schools that may use curricula not aligned to those standards. Moreover, high stakes are often attached to state tests for public schools, creating incentives for ensuring instruction is tightly connected to tested material (whether that’s good or bad is a separate issue).
Giving credence to this idea is a Milwaukee study showing that test scores rose among voucher recipients after high-stakes accountability was introduced in private schools. On the other hand, the Louisiana study showed voucher students did equally poorly on test questions not closely aligned to standards as to those that were aligned.
Studies of Louisiana's voucher program have only examined its first couple years of existence. Although the effects were negative in both years, there was some modest relative improvement between years one and two. John White, the state’s superintendent of education, wrote, “Over time, as in public schools, the good [private schools] will separate from the bad, and policy makers and parents will respond to information at hand.”
The new Ohio research finds that negative results for recipients continue even five years into the program, however, with little if any improvement over time. Early growing pains don’t seem to explain the findings.
Still, the Louisiana program is distinct from Ohio’s and includes accountability provisions that rachet up in response to poor test scores, such as prohibitions on accepting new voucher-holding students. It will be important to see whether Louisiana voucher impacts follow a different trend.
Some argue that regulations intended to ensure voucher programs are effective and equitable — like bars on selective admissions, limits on how much tuition can be charged, testing mandates —may have the unintended consequence of deterring high-quality private schools from accepting voucher students. It is true that regulations seem to slightly reduce the number of participating private schools, but there’s not much evidence showing the effect this has on achievement; it’s also not clear whether this theory is sufficient to explain negative achievement effects.
Ohio is an interesting, and tough-to-interpret case. Although the program has a significant number of regulations, Chad Aldis — vice president for Ohio policy and advocacy at Fordham — said that the state’s private schools are already highly regulated and the voucher program doesn’t add too many new provisions. In his view, too much regulation is unlikely to explain the findings.
On the other hand, Ohio private schools are reequired to administer the state test for students receiving vouchers, which surveys find is a significant concern among private schools administrators. As of 2009–10, 39 percent of all private schools and 63 percent of urban private schools participate in Ohio’s voucher program.
It’s the improvement of public schools
One reason why newer studies find more negative results for vouchers may be that traditional public schools have improved, particularly as measured by standardized test scores. Mark Dynarski of the Brooking Institution has made this case, showing that public schools have made larger improvements on federal tests than private schools in recent years.
Research suggests that test-based accountability, like No Child Left Behind, has led to gains in math achievement among public schools. “There is mounting evidence that accountability pressure does improve the performance of relatively low-performing public schools,” said Figlio. Private schools, meanwhile, have not been subject to this and other policies changes that may have improved test scores.
If accountability for public schools has driven improvement, then the same may hold true for private schools. Again, the Milwaukee study showing gains on high-stakes tests in response to the introduction of accountability is relevant.
Aldis thinks that there should be greater transparency requirements to help families make more informed decisions; for instance, he argues that private schools should be required to publish and publicize student growth scores.
Private schools may not respond to accountability measures in the same way as public schools, though. Most saliently, private schools can simply opt out by declining to accept vouchers; public schools have no such option.
It’s the concept itself
Another hypothesis for the recent negative findings is that vouchers are simply not an effective policy for improving student achievement.
For instance, Christopher Lubienski and Sarah Theule Lubienski of the University of Illinois have found that public schools actually outperform private schools on standardized math tests when differences in student demographics are controlled for. If it is indeed true that public schools are actually better than private ones, expanding access to the latter is unlikely to be effective. These recent studies certainly provide some credence to this view.
However, the existence of competitive effects — in the Ohio study and many others — undermine it to some extent. So does the research suggesting vouchers can have positive impacts on educational attainment.
It’s still unclear
In many ways, the existing research and subsequent speculation lead to more questions than answers. Right now no one can say if one, several, or none of these explanations are correct.
The question going forward, though, perhaps should not be whether school vouchers “work.” But rather, when, where, and by what measure do vouchers succeed or fail? Ultimately, this is a more interesting and relevant question.
Disclosure: The Ohio study was funded by the Walton Family Foundation, which is also a funder of The 74.