Research Suggests D.C.’s Tough Teacher Evaluation System Helped Students. 7 Big Lessons for Other Cities

Getty Images
There were cheers last week after the release of a new working paper by the National Bureau of Economic Research that showed students in Washington, D.C. benefited when low-performing teachers left the classroom.

Among those advocating for more rigorous instructional standards across the country, the report was held up as proof that Washington D.C. was right to implement tougher teacher evaluation policies, which required low performers to be fired.

The study is good news for D.C. school leaders, who faced fierce opposition in implementing a policy that rewarded good teachers while pushing ineffective ones out of the system. But policymakers across the country who might be thinking of replicating the city’s results elsewhere must read beyond the headlines to note what makes the D.C. success story so unique.

First some crucial background: Under current schools chancellor Kaya Henderson and her predecessor Michelle Rhee, D.C. Public Schools instituted a new evaluation system that considered gains in student test scores, along with observations by principals and master teachers from outside the school, in grading teachers. Instructors who received the lowest rating were fired, as were those who received the second lowest rating two years in a row; teachers with top ratings got bonuses and permanent salary bumps.

Past research has shown that the system pushed low performers to either leave the district voluntarily or improve if they stayed put. High performers who had the opportunity to receive a salary increase also got better and may have been more likely to remain teaching.1

This month’s study — by Melinda Adnot, Veronica Katz, and Jim Wyckoff of the University of Virginia, as well as Tom Dee of Stanford — goes further, showing that teacher turnover in D.C. had a net positive effect on student achievement. For researchers, this is a surprising finding, since past studies have generally suggested that turnover harms learning.

In D.C., when a teacher deemed low-performing left or was fired the gains to student learning were significant: “By comparison to almost any other intervention, these are very large improvements that are situated among some of the neediest students.”

Although the authors emphasize that this research is a broader assessment of the effect of turnover on student achievement — not of D.C.’s evaluation system per se — the past research (conducted by two of the same authors) showed that the district’s tough evaluation standards did in fact cause more low performers to leave the classroom. Paired together, the two studies suggest that D.C.’s evaluation system had positive effects.

Despite their noteworthy findings, the authors of the new research caution other school districts about rushing to fire underperforming teachers. “I’d be a little reluctant to tell every district, ‘Use whatever evaluation system you have and you will get these results,’” study coauthor Wyckcoff tells The 74. “But [D.C.] does provide a proof point that well-done evaluation systems can have positive effects on the composition of teacher quality.”

Key takeaways for other school districts:

1. Pay attention to who’s hired, not just who’s fired.

D.C. was able to improve teacher quality by replacing the teachers who left with better ones.2 That may sound obvious, but it’s a key insight because there was no guarantee that incoming teachers would be more effective, particularly since the risk of dismissal may have made the job less appealing.

But D.C. didn’t seem to run into this problem: Over the three years of the study, incoming teachers were always more effective than the ones they replaced. This might have been because D.C. paired aggressive dismissal policies with large bonuses for high performers, particularly those in high-poverty schools. Base salaries for D.C. teachers were also raised and are among the highest in the country. (Research elsewhere has found that high base salaries, as well as performance pay, can help recruit and retain teachers.)

Districts hoping to implement D.C.’s policy on the cheap might want to think again, and first invest in a professional salary for teachers.

2. For those who are low-performing, act swiftly.

D.C. has ensured that teachers who are identified as low-performing are swiftly dismissed — that may make it unique among districts. About half of low-performing teachers in D.C. leave voluntarily or are fired. Contrast this to New York, where just a single tenured teacher statewide has been dismissed due to the new evaluation system. In other parts of the country, the vast majority of teachers are identified as effective or highly effective.

Some worry that difficult-to-navigate dismissal processes have led to score inflation as principals avoid giving low rating to teachers who they can’t fire anyway. That doesn’t mean not trying to help struggling teachers get better, but D.C. has ensured that teachers who don’t perform well will be exited swiftly from the public schools if they don’t improve.

3. Use high quality tests — and monitor for cheating.

Critics of test-based teacher evaluations have long been concerned that standardized tests are not meaningful measures of student learning. A significant body of research suggests that test scores are important, but it’s unclear whether the usefulness of such measures lose value when high stakes are attached, because of cheating, teaching to the test, or other forms of gaming.

In fact, there were instances of cheating in D.C., though it was never clear how widespread the deceit went.

D.C. says it’s instituted procedures to prevent cheating; it also adopted a new Common Core-aligned standardized test, PARCC, which purports to measure critical thinking and depth of knowledge. It’s not clear whether either of these changes were helpful — the latest study doesn’t tell us anything on this point — but they show how crucial it is to get these practices right.

4. Focus on helping all teachers improve, not just firing bad ones.

D.C.’s evaluation system placed significant emphasis on giving teachers meaningful feedback to help them improve, and previous research suggests it was at least somewhat effective on this front. (Research from Cincinnati schools also finds that teachers get better in response to evaluation.) That means an evaluation system focused exclusively on smoking out and firing bad teachers is limited in its potential gains. It’s also possible the comprehensive nature of D.C.’s evaluation system helped the district get more effective teachers to replace the ones who left.

5. Teachers in different subject areas shouldn’t be lumped together.

Unlike many districts — which are often under relatively inflexible state teacher evaluation law — D.C. has different evaluation weights for teachers in tested grades and subjects (like 4th-8th grade math and reading) and teachers who aren’t (like gym or social studies). For the former group student achievement has accounted for 50 percent of their evaluation, but just 15 percent for the latter.3

This is likely a good practice. Many other districts have used local, teacher-created tests as a significant aspect of evaluation — up to 50 percent — but this approach has a limited research base. D.C. also does not use a school-wide measure — in which every teacher in the school gets a rating based on test scores across the board — for evaluation purposes. This is also wise because such group scores tend to push all teachers towards an average rating, while making it riskier to teach in low-performing schools.

6. Incorporate feedback, preserve political goodwill.

Since implementing the evaluation system in 2009 D.C. has gone through three mayors, three teachers union leaders, and two schools chancellors. But the evaluation system remains intact. Maybe D.C. got lucky, but aside from continued criticism from the teachers union, there doesn’t seem to be much opposition to the evaluation system. Contrast this to New York where one fifth of students opted out of state standardized tests and state tests have been removed from evaluation for the next several years.

Perhaps one of the reasons D.C. was able to maintain political support for tough evaluation, is that the district showed a willingness to tweak the evaluation system in response to stakeholder feedback. The district actually reduced the emphasis on standardized tests for evaluation purposes and put a pause on evaluation based on PARCC during the transition to the Common Core.

7. Focus on preserving teacher diversity.

Between 2003 and 2011 the teaching corps in D.C. has become significantly more white; this is concerning in its own right, as well as because research finds that student achievement increases when students of color are taught by teachers of color. The vast majority of D.C. students are black or Hispanic.

It’s not clear if the evaluation system, which was implemented in 2009, has had any effect on the racial make-up of teachers, but this may be an area where the district can improve, and where other districts can learn. D.C.’s evaluation system almost exclusively identified and dismissed low-performing teachers at high-poverty schools; those schools are often disproportionately staffed by teachers of color, which means those teachers are especially likely to face negative consequences of evaluation. Research from Chicago found that the observation aspect of the evaluation system gave teachers of color lower ratings on average.4

To be clear, this is not to suggest intentional discrimination in D.C. or elsewhere, but that care must be taken to ensure an evaluation system does not focus on high-poverty schools to the point that teachers of color are significantly reduced or that disadvantaged schools are seen as undesirable places to teach.

(Disclosure: In 2012, I worked as a summer intern at StudentsFirst, an organization founded by Michelle Rhee, the former chancellor of D.C. Public Schools.)

1. I say “may” because the results here weren’t statistically significant. (Back to story)
2. The study authors point out that D.C. may also have an advantage in recruiting new teachers because it is  a relatively attractive place to live. It’s not entirely clear if this point holds, however, because (1) more appealing places to live are also generally more expensive places to live, which certainly applies to D.C. (2) the question is not whether D.C. can recruit better teachers than other districts, but whether the difference between exiting low-performers and their replacements would be lower elsewhere than in D.C. This is an open empirical question, though recent research from Charlotte, North Carolina, also showed achievement gains to laying off ineffective teachers. (Back to story
3. This does not apply this or last school year because of the transition to the Common Core. Evaluations based on the state Common Core-aligned tests are on hold, and all teachers are evaluated just 15 percent on test scores. The 15 percent is based on a test, sometimes teacher-created, other than the state standardized assessment. (Back to story
4. The same pattern did not show up for measures of teachers’ impact on test scores, because there were statistical measures to control for student demographics. This suggests that test-based evaluation may help level the playing field and ensure teachers of color are not disproportionately dismissed due to evaluation. (Back to story

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Republish This Article

We want our stories to be shared as widely as possible — for free.

Please view The 74's republishing terms.

On The 74 Today