‘It Gave Us a Choice When We Didn’t Have One’: Private School Choice Participants Flood Capitol to Tell Their Stories

This Week in ESSA: Ohio, Montana, Oklahoma, New York Advance Plans as Group Promotes Evidence-Based Rules

Weingarten Speech Tying School Choice to Racism Sets Off Firestorm

House Committee Rejects Democrats’ Bid to Restore Education Funding, Protect Teacher Training

How’s This for a Yarn? School Bus Driver Crochets a Personalized Toy for Every Student on Her Route

A Phoenix Breakthrough: How 3 Massachusetts High Schools Are Helping Dropouts Become College-Bound Grads

Teacher Groups Frustrated With California ESSA Plan’s ‘Loose’ Definition of Ineffective Teachers

The 74 Interview: You Don’t Think Your Child Is Average & Harvard’s Todd Rose Doesn’t Either

Interactive: How Far Every State Has Gone to Update Education Policies Under the Every Student Succeeds Act

House Republicans Warn Education Dept. on ESSA Overreach as Democrats Lament Lack of Accountability Rules

More Attention to ELLs, Student Suspension, Fewer Test Days: NY Tweaks Its ESSA Plan

WATCH: These 100 HS Grads Made a Splash With Pomp, Circumstance — and a Jump in the Lake

12 Rhode Island Schools Vie for Chance to Become Their State’s 3 Personalized Learning Labs

Teacher Raises, Bathrooms, Vouchers: Texas Lawmakers Take Up Big School Fights in Special Legislative Session

Investigation: Forced Into Unneeded Remedial Classes, Some Community College Students Fail to Finish Degrees

A Summer Education Meltdown: Why Everyone in DC Is Mad About ESSA, Congress, Charters, Choice — or All of the Above

This Week’s ESSA News: Science Test Debate, a Career Readiness Blind Spot, and Massachusetts Has Work to Do

House Subcommittee Advances Education Funding Bill as Democrats Protest ‘Anti-Teacher’ Cuts

DeVos Hears of Sex Attacks at Colleges & K-12 Schools as Feds Weigh Changes to Title IX Evidence Rules

The $1,488 Back-to-School Bill: Backpack Index Tracks Rising Costs of Supplies, Fees for Band, Sports, Trips

74 Interview: Harvard Researcher David Deming Takes the Long View on Head Start, Integration

Photo Credit: Courtesy of Harvard University

May 10, 2017

Talking Points

.@ProfDavidDeming on the lasting benefits of Head Start, school accountability and integration

Harvard researcher David Deming: Head Start works but integration might not close the achievement gap

Sign Up for Our Newsletter

See previous 74 interviews, including former secretary of state Condoleezza Rice, Senate Education Committee Chairman Lamar Alexander, former education secretary Arne Duncan, and researcher Kirabo Jackson. Full archive here.

In education research, most studies look at the short-term effects of policy: whether a given reform — vouchers, early childhood education, school accountability — immediately helps kids, usually as measured by test scores.

But Harvard University professor David Deming plays the long game. Deming has produced several studies looking at the far-reaching effects of education policies on students, including their college attendance, adult earnings, and involvement in the criminal justice system.

In a recent interview with The 74, Deming talks about the lasting influence of Head Start, test-based accountability, and school integration. He also discusses how his research might impact the debates on early childhood education and implementation of the Every Student Succeeds Act.

The interview has been lightly edited for clarity and length.

The 74: Can you tell me a little bit about your work on Head Start? I think it’s obviously really important now — some people want to get rid of the program. What have you found?

David Deming: This was the very first real academic paper I ever wrote, actually, on Head Start, and this is a good lesson for aspiring graduate students. It started off as a replication of an older study about Head Start, finding that the impacts of Head Start, “fade-out” for African-American children who participate in the program — meaning there’s an initial impact on achievement and then by age 10 or so, there’s no difference between kids who are in Head Start and their siblings who were not in Head Start. So the initial impact in test scores fades out.

I was interested in this fade-out question as a graduate student, and so I said, I’m going to kind of replicate these findings and see if I can understand the fade-out patterns. When I got the data, I realized actually there was 15 more years of data since that paper was written and now these kids are 19, 20, 21 years old and I can look at longer-run outcomes, and so I did that.

What I found was that I was able to replicate the results, so that’s a win for science. I found that even though the impacts of Head Start faded out, there were long-run impacts on the things that we really care about, like high school graduation, college attendance, self-reported health, and then labor force participation, so whether you were working. The interesting thing about it was that it was actually the African-American kids and lower-income kids — kids whose mothers had lower academic achievement — they experienced the biggest benefits from Head Start and they had the most fade-out. It actually seemed like fade-out was somehow predicting longer impacts.

In the opposite of the way that you would expect?

My lesson from that is I don’t think we really know why test scores fade out, but in this case, I don’t really care, because test scores are only useful to the extent that they correspond to something in a kid’s life that we care about directly.

The bottom-line question is whether Head Start helps kids in the long run, and the answer is yes. My study is not the only one that finds that. I think that the evidence is pretty strong that early childhood interventions in general and Head Start in particular benefits low-income children and passes a benefit-cost test. Meaning the expenses of the program are less than if you try to sum up the total benefits, not just to kids but to society — that Head Start is a program that literally pays for itself.

When someone points to the most recent federal study saying the benefits are very short lived, that’s not a strong basis for rejecting the program, right?

Not by itself. Certainly we’d like to see bigger test score benefits. It is possible that either Head Start or the alternatives available to the kids who didn’t get Head Start have changed. Re-analyses of the Head Start impact study have looked at this. One is by Patrick Kline and Chris Walters and another by my colleagues Luke Miratrix, Avi Feller, and Lindsay Page, both papers answering the same question, which is: How do we understand the impact of Head Start relative to not being in any preschool, and then the impact of Head Start relative to the other available alternatives?

The benefits of going to Head Start relative to going nowhere are very large and do not fade out. However, when you compare kids doing the Head Start to, let’s say, another preschool alternative in the city, whether it’s a state pre-K program or another private pre-K, those benefits seem to be there too, but are a little bit smaller and do fade out.

The lesson there is that Head Start is much better for kids than nothing and Head Start might be a little bit better than other programs but might not and we’re not sure about that. That is not the way the results of the Head Start impact study have been reported.

Let me pick a nit with your study. You are comparing siblings, one who went to Head Start and one who didn’t. How were you able to ensure that there wasn’t some reason that one did and one didn’t and that may have caused differences in outcomes?

I’m not able to tell you that for sure — that’s the reality of it. Of course, selection into Head Start within the family is clearly not random in a sense that people are throwing darts at a board, but what we hope is that it was not related to other things that determine differences between siblings.

Here’s a story that, if you have two children and you really like one of them better and so every time you have a chance — let’s say you can only pick one kid to go to some program, you always pick the same kid. Then that’s going to make Head Start look better because the reason you put that kid in Head Start and not the other one is because you like them better.

Maybe you also sent that kid to summer camp; you are just a better parent to that kid.

Exactly. So I tried to account for that, by looking at variation across siblings in things that were determined prior to Head Start, like, did the kid who is in Head Start have higher test scores at a really young age? Did they have higher birth weight? Was the mom more likely stay-at-home then, or the dad? Were they more likely to be nursed? A lot of different things, and I found no systematic evidence that the kids who went to Head Start were different on the need dimension than the kids who weren’t.

Let’s talk about your work on accountability, which is a hot topic now under ESSA. Can you tells us about your study in Texas?

Sure. So, we were interested in the question of the longer-run impacts of school accountability.

Which have rarely, if at all, been studied before your research?

That’s right, and I think while it’s always important to look at the long-run impacts in everything you do in schools, it’s particularly important in the case of accountability, because there are ways of raising test scores that are associated with kids learning more and doing better, and there are ways of raising test scores that have strategic responses. There is a larger literature in economics and in the fields looking at all the ways that schools respond to accountability.

Many studies have found things ranging from making school lunches more nutritious to suspending kids strategically to keep them out, all that stuff, classifying kids for special education, all that stuff.

I think what a lot of people have done is, they said, “Well, we’ve found evidence that schools respond strategically to high-stakes testing,” and taken that by itself as evidence that accountability is bad. It could actually be the case that while some schools are responding strategically, they are also responding in ways that are good for kids — maybe you put your best teachers in the testing grade, and that’s unfortunate, but then you also hire extra teachers to tutor kids on the side, or you are better aligned in the curriculum with standards or whatever. Accountability is just a package of things, and you kind of want to say, on net, when we raise kids’ test scores, are they better off in the long run?

In Texas we looked at this — we compared schools that were close to the margin of being rated low-performance versus acceptable, and then acceptable versus recognized. Recognized is the gold star of schools that are doing well.

We actually compare schools to themselves in a future year when they were kind of safe from accountability pressure. There was a very large range in which schools are rated acceptable. If the pass rate of the lowest score in a subgroup was between, sometimes, 25 and 60 percent, the school was safe.

If you looked at your group of kids and you thought, “About half of them are going to pass,” probably nothing you are going to do is going to get them from 50 to 60 percent passing in one year, and probably nothing you do is going to get from 50 to 25 percent students passing. You are like, “OK, I’m facing some accountability pressure because everybody is, but not as much as the school down the street from me which is at 59 or at 26, where only a couple of kids passing or failing could make the difference for me.”

What happened was, in Texas, over time, the state ratcheted up the standards every year. The minimum threshold to be acceptable went from 30 to 35 to 40 to 45 percent passing. Every year it went up. What that means is a school could have the same population of kids to be safe one year and not safe for the next.

We said, “Let’s compare what happens in those schools when they are safe versus when they are not.” What we found was that for schools that were on the margin between being rated low-performing and acceptable, when those schools were on that margin, the test scores of low-scoring kids went up, which is consistent with many other studies. Then we followed those kids into adulthood. What we found was those kids were more likely to attend college, more likely to graduate college, and had higher earnings at age 25.



To summarize, schools that faced potential sanctions for a low-performing rating raised test scores in a way that improved long-run outcomes?
Yes, overall. Doesn’t mean they were also doing strategic responses, but overall these kids were better off. The impacts were there; they were statistically significant, but they weren’t huge.
Now let’s look at schools that were on the margin between acceptable and recognized. In that case, we found really not any evidence of test score gains, and we found that some low-scoring kids — these are low-scoring kids in high-scoring schools — those kids were worse off in the long run. We found pretty strong evidence that those kids were being strategically classified as eligible for special education. At that time, kids who were in special education did not count toward the school’s accountability area.
If you want to know why do we find this different pattern, one reason is that if you are a school that’s on this low-performing bubble, you would have to classify, like, 20 percent of your entire grade as special needs to make all the low-performing students not count.
For those schools, there is no easy way out — you have to really improve kids’ test scores. But if you are a school that’s high-scoring except for one or two student subgroups — these are schools that maybe only were 15 percent low-income or 15 percent black or whatever — and so you are talking about a small number of kids who are really making the school get below the rating. It’s tempting, I think, to just find a way to make those kids not count, because there was also a rule that the subgroup doesn’t count unless there were at least 30 kids in the group. So if you only have 25 low-income kids in your school, then the low-income group doesn’t count towards the rating.
All the states are designing new accountability systems under ESSA — what do you see as some of the lessons from your study being?
I think that there is a lot of evidence, not just from our study but others, that we should be kind of small-c conservative in the design of these systems. The more complicated you make it, the more incentives you create for strategic responses. I think really complicated systems and multiple targets and multiple ratings are kind of like technocratic exuberance. I think we really should scale back our ambitions with the accountability systems and focus on the most important challenges.
For me, that is making sure that the kids who have the greatest needs are being well served by the state. I think schools in higher-income communities are already accountable to parents. They already face competition because parents who have means can move to districts where they want. I don’t feel that the state has to intervene as much, whereas if you have schools and districts where parents don’t really have an exit option and don’t maybe have the social capital to know how to effectively lobby teachers and principals, then I think that’s where the state needs to play a stronger role. I’m not saying the state should target communities; I think we should basically have a bottom-line benchmark.
One way to think about this is that, instead of having an accountability system that says these are the best schools and kind of rate their rank, we should think about it like the way FDA does drug approvals. Like, I certify this school is safe for children, and certify a minimum standard and set that minimum standard so that most schools are going to meet it, that are functioning, but then when schools start to really fail, and fail students in a way that’s obvious — if just 20 percent of the kids are passing a basic test and they’re way below grade level and you keep telling the school to improve and they don’t — that’s when the state needs to step in.
If I were in charge of the accountability systems, I would focus on rigorous evaluation of schools that appear to be failing or appear to be unresponsive. One idea is to have a quantitative trigger so schools that fall below the benchmark pass rate of a test or an average score trigger a set of quantitative and qualitative evaluations. Trigger kind of value-added and growth-model stuff. Trigger school visits by inspectors like they do in the U.K., where you have somebody comprehensively come and provide additional resources to the school to improve. I think some successful accountability systems like the one we have in Massachusetts does that. They pair increased attention from the state with increases in resources.
It is not just hey, work harder, and cracking the whip, it’s also giving people the resources to do that. I think the cost to students of being in a school that’s truly not serving them is incredibly high, and the cost to taxpayers in society is incredibly high. We ought to really just narrow our focus, pare back our ambition, focus on the kids that really need our help. That’d be my lesson from, not just our study, but many other studies.
Can you tell me briefly about your research on schools and crime?
I’ve got a couple of different papers about crime. So, one is a paper that shows kids that won a lottery to attend a school they would have preferred to go to, made them much less likely to be arrested and incarcerated several years later.
It was in Charlotte. Both the studies were in Charlotte, actually.
The most striking, funny thing for me about that was it was true even for kids who didn’t graduate from high school. Among these “high-risk” kids that we looked at — these are kids who’ve have low prior test scores, come from very poor neighborhoods, have a history of misbehavior and school suspension — only 25 percent of those high-risk kids graduated from high school, many of them dropped out after ninth grade, and yet going to a different school had a huge impact on their later crime.
I think of school as the main social institution for kids at the age of peak criminality. When you think about it that way, it’s not surprising that the school context could have an important effect on crime.
The other study was about impact of school re-segregation in Charlotte. There we found that essentially when school boundaries were redrawn, kids who were on opposite sides of the boundary — that’s our kind of natural experiment — you and I live across the street from each other, the school zone get re-drawn, you go to a 20 percent black school, and I go to a 70 percent black school. Changes of that magnitude existed in Charlotte because basically the school district was under a court order to maintain racial balance in schools but the neighborhoods are really segregated. Once they ended the court order, things got a lot more segregated.
What we found was African-American boys who were assigned to attend schools with more African-American boys were much more likely to be arrested and incarcerated after the rezoning. The effect was very large. Attending a school with a 10 percent higher share of non-white students led to an increase in the chances that they’ll be incarcerated, like 1.4 percentage points, so it was a pretty big impact. And this is seven years later, so this is long after.
The other thing that was interesting about that paper is that we found pretty modest impacts on academic outcomes — not much, and actually it faded away over time, and so if you look at the black-white test gap in Charlotte today, it’s no different than it was in 2001. It’s exactly the same, and schools are much more segregated, but racial tensions are much worse in Charlotte. Issues between the police and communities of color are much worse.
The way I interpret that is that if you think of school integration as a policy tool for closing the black-white achievement gap, I don’t think we have a lot of evidence that is going to do that by itself. Because you look at these other studies, the studies of no-excuses charters, school-finance reforms. There are other things we see that have bigger impacts on kids than just integration by itself. However, that doesn’t mean integration doesn’t do anything. I think it’s about the non-academic outcomes: crime, civic participation, racial tolerance, all those things. If you look at the original justification for Brown v. Board of Education, it had nothing to do with achievement gaps.
It’s about, we want to bring society together, so that we can live together. Somewhere along the line, with the Coleman Report, we started to think about integration as a tool for closing achievement gaps. When actually to me, the first-order reason to integrate schools is so that we can all get along as a society and so that people can be more civically engaged and appreciate the perspectives of those who are different from them.
Another way to say it is, we have other tools for closing achievement gaps. Like, we can do separate-but-equal if we want to — that’s what some of these no-excuses charter schools are: really good schools that are highly segregated. To me that’s an incomplete solution, because schools aren’t just factories for producing achievement — they’re social institutions, they’re democratic institutions.