8 Things We Learned From a Big New Texas Study on Charter Schools and Students’ Future Income

Texas charter schools have a quality problem…

The headline findings from the study are that charter schools in Texas, on average, have no impact on standardized test scores, have small positive impacts on college attendance and actually harm students’ earning potential later in life. For each year students attended a Texas charter school, they earned $163 less in their 20s than students who went to district schools. The study controlled for a number of factors in order to ensure fair comparisons between charter and district schools.

Like some other research, this study found differences by race: The impact on earnings ($41 per year) was small, positive and statistically insignificant for black and Hispanic students but large and negative (–$509 per year) for white and Asian students.

Notably, the findings are the opposite of research on Florida’s charters, which showed that they improve adult earnings.

…but they may be getting better.

There is some evidence from a separate study that Texas charter schools have improved over time, as measured by test scores. This was accomplished largely by reducing the number of low-performing charter schools. More research will be needed to see if charter schools continue to improve relative to traditional public schools in Texas.

“No-excuses” charter schools in Texas have only small effects on earnings…

Dobbie and Fryer estimated that attending “no-excuses” charter schools — defined as “schools that tend to have higher behavioral expectations, stricter disciplinary codes, uniform requirements and an extended school day and year” — increased adult earnings by about $100 for each year that a student attended such a school. These results were not statistically significant. Such schools include high-profile charters like KIPP, YES Prep and IDEA. (In contrast, charter schools not described as no-excuses had a statistically significant negative effect on income: –$322 per year.)

There are a number of caveats to this finding: One particularly important one is that students attending no-excuses charter schools were significantly more likely to attend an out-of-state college, which may affect the results, since the researchers have income data for adults in Texas only, not other states.

Dobbie and Fryer conducted a number of statistical tests to ensure the accuracy of their results. Depending on the statistical controls used, they found that at most, no-excuses schools raise earnings by $237 a year, which is still statistically insignificant.

…but it’s exactly what you’d expect from their test score gains.

No-excuses charters had statistically significant effects on standardized test scores as well as college attendance and persistence. Some coverage of the study has suggested that perhaps charter schools are good at raising test scores by, for example, teaching to the test without actually improving student learning or important non-cognitive skills.

While that can’t be ruled out, it is not consistent with the results. We know this because the researchers examined how all students’ test scores correlated with their income as adults: As expected, there was a significant association. Then they examined whether the income effect of a no-excuses charter was similar to what would be predicted by their test score gains. It was. This means the test score improvements did a good job of anticipating the no-excuses schools’ impact on longer-term outcomes.

The biggest caveat here is that the income effects were estimated with a great deal of statistical imprecision.

A school that lowers test scores is probably a struggling school…

There is a good deal of debate about the extent to which test scores are meaningful measures of school quality. This study found that across all Texas charter schools, those that had negative impacts on test scores also had negative impacts on students’ average earnings as adults. Crucially, this occurred in the context of Texas’s test-based accountability system that applied to charter schools. Many have pointed out that such a system can encourage cheating and teaching to the test, potentially corroding the usefulness of standardized assessment scores. This study, though, provides evidence that even in a high-stakes context, test scores have some value in predicting long-term outcomes.

…but a school that raises test scores may or may not be a good school.

Somewhat paradoxically, though, schools that were particularly effective at raising test scores had only small, statistically insignificant positive impacts on earnings. In tandem, these results suggest that test scores are good for identifying low-performing schools but not so good for singling out effective schools. On the other hand, schools’ impacts on high school graduation rates were a strong predictor of income effects across the board.

Education reform is likely to have only modest effects on poverty…

Many have argued that school reform is the best way to address poverty. But this study provides an extremely cautionary note for those who would use education policy or school choice as the chief means for reducing poverty.

Take a hypothetical student who attends a no-excuses Texas charter school from kindergarten through 12th grade, and take the highest estimate for those schools’ impacts on earnings. That student would make about $3,000 more per year in her mid-20s as a result of attending a charter.

For context, the federal poverty line is $11,880 for an individual and $24,000 for a family of four. The earnings bump is certainly meaningful and would surely be sufficient to raise some people out of poverty — but for many, it wouldn’t be nearly enough. And this is the best-case-scenario estimate, by these findings. In aggregate, Texas charters had a negative effect on income, suggesting on net that they slightly increased poverty rates.

…but schools do affect adult earnings.

The impact of schools, however, should not be downplayed. Dobbie and Fryer found that schools can influence student outcomes long after they graduate — or don’t. Schools can have significant and important effects on not just achievement but college attendance and adult income as well. It’s crucial that research continue to examine how and what schools — charter and otherwise — make a difference beyond short-term test score gains.