;)
Untangle Your Mind!
Sign up for our free newsletter and start your day with clear-headed reporting on the latest topics in education.
Those who follow education policy have surely heard these two mantras: “Teachers are the most important in-school factor affecting student achievement” and so-called “Non-cognitive skills, like grit, are incredibly important for students’ long-run success.”
Now a new working paper from Brown University researchers joins an emerging body of evidence combining these two separate strands to gauge how important teachers are in developing these other outcomes not so easily measured by a test.
The research has produced several important findings: Teachers do influence how their students develop these kinds of skills, like effort and determination, and some educators are better at it than others. But teachers who excel at improving test scores are not necessarily the same ones who are best at improving student attendance or happiness — and figuring out who those teachers will be based on background or training is not easy to do.
So if teaching a child how not to give up on solving a problem is as crucial as teaching them how to solve it, how do policymakers incorporate those teacher qualities into the white-hot debate over performance evaluations and the weight test scores get in that equation? Researchers aren’t so sure.
‘Grit’ and student achievement research combine
Economists have been measuring teachers’ effects on student test scores for decades, using a statistical model known as value-added that controls for student demographics like poverty. The findings have consistently shown that teachers matter a great deal and that some are better at improving student achievement than others.
This influenced policymakers focused on strengthening teacher quality and helped lead the vast majority of states to revamp teacher evaluation to include measures of student achievement.
Meanwhile, other researchers began examining the import of what is sometimes referred to as non-cognitive or non-academic skills (although many researchers have proposed different terms to describe these traits). Pennsylvania psychologist Angela Duckworth popularized the term “grit” — defined as “passion and perseverance for long-term goals” — with her research, beginning in 2005, finding that it predicted how well people fared across a variety of settings including school, the military, and the workplace.
Economist James Heckman also showed that skills that are not strictly academic affected how much money students would make later in life, how long they stayed in school, and whether they were likely to use drugs or become pregnant as teenagers.
Now some researchers began taking the logical next step: to what extent do teachers affect their students’ skills in these areas beyond test scores?
Teachers impact absences, suspensions — and happiness
To date, this new line of research has consistently found that teachers do, in fact, make a big difference for students beyond just test scores.
For instance, the new study from Matthew Kraft and Sarah Grace of Brown finds that teachers affect students’ determination and effort in class. Another a recent study by Kraft and David Blazar of Harvard showed that elementary school teachers had a significant effect on students’ classroom behavior, self-confidence and their happiness in class.
A study by Northwestern economist Kirabo Jackson looked at ninth-grade algebra and English teachers in North Carolina, finding they affected student absences, suspensions, course grades, and progress towards a diploma. The effects stayed with students over time, translating into gains in high school completion.
Other research has also found teachers can influence student attendance. (Interestingly, though, Kraft and Blazar’s study didn’t find evidence that teachers affected student absences.)
This research consistently shows significant variation from teacher to teacher, meaning some are better than others at improving students non-cognitive skills. But, with the exception of experience in a few cases, most studies suggest that easily observed teacher characteristics — like holding a master’s degree or having high test scores themselves — don’t seem to predict who will build non-academic traits in the classroom.
Complicating the test score debate
One important question this second wave research has started to address is whether a “good teacher” is a good teacher across multiple dimensions. The answer has been: not necessarily. Most studies suggest that being strong at one aspect of teaching is only slightly tied to excelling at others.
“It is certainly true that on average a teacher who raises test scores by a lot will tend to raise the non-cognitive factor to some extent … But it is not true that every teacher who raises the non-cognitive factor raises test scores and vice versa — far from [it],” Jackson, the Northwestern professor who studied North Carolina ninth-graders, said in an interview.
In some ways this complicates the polarizing debate about whether student test scores should be part of teacher evaluation — and gives credence to critics’ assertion that test scores don’t fully capture teacher effectiveness.
But Jackson says that dumping test-based evaluation isn’t necessarily the right takeaway.
“Test score [value-added] is informative and it's more informative than how many years [teachers] stay in the profession,” he said, alluding to traditional teacher salary schedules and some tenure systems, which reward longevity rather than measured effectiveness.
Kraft said that his research couldn’t render a verdict one way or another on the value-added debate.
Jackson’s study actually shows that a teacher’s value-added score was positively associated with students’ high school completion, though teachers’ combined effect on absences, suspensions, and grades was even more predictive.
“Are there better ways to measure teacher quality … than simply using test score value-added? Jackson said. “The [answer] there is absolutely yes.”
Put it in teachers evaluations? Not so fast.
So if researchers know that teachers affect non-cognitive skills, that the impact isn’t captured in test scores, and that it’s difficult to predict which teachers will be effective, one obvious policy takeaway is to start evaluating teachers in part based on these non-cognitive impacts.
In reality, though, it’s not that simple. “Any time we place incentives on measures it has the potential to distort the responses to that measure,” said Kraft.
This potential problem — sometimes referred to as Campbell’s Law — has long been seen in the form of cheating and other concerns about test-based evaluation, and may be even more difficult to overcome when measuring what are sometimes called softer skills. Students can be coached when filling out surveys, for instance, and teacher-reported attendance, GPA and behavior records might be easily manipulated.
“It's not obvious that measures of non-cognitive skill that I use [in research] are going to be easily implemented in a policy setting,” said Jackson.
Finding a way to make it work
Researchers agree that it’s crucial that policy — including school accountability and teacher evaluation systems — take the findings on non-cognitive into account. “It's important enough for long-term outcomes that we invest in trying to make this work,” Kraft said.
What that means is not entirely clear, but there are several options policymakers might consider.
One idea is simply investing in more evidence. While more scholars seem to be focusing on this issue, many unanswered questions remain, particularly on using non-cognitive outcomes for accountability purposes.
This is especially important considering that the new federal education law requires states to judge schools (and indirectly their teachers) by measures other than test scores. Different places will try different approaches with varying degrees of success, and it will be critical to look carefully at what’s working and what isn’t.
While it may be difficult to directly evaluating teachers based on non-cognitive measures, current teacher evaluation systems may already have proxies for these skills. Kraft and Blazar find that aspects of teachers’ classroom observation scores predict their effects on these non-academic outcomes. For instance, teachers who were highly rated by observers on providing emotional support also raised students’ happiness in class. To account for non-academic skills, it will be important for schools and districts to work to hone their evaluation tools.
Finally, these measures may be useful for low-stakes purposes designed to help teachers get better. At least for now, scores on non-cognitive measures could be calculated for teachers without the threat that they’ll be used for evaluation.
This would address the concerns about manipulation and would allow educators to attempt to improve their instruction based on strengths and weaknesses. For example, New York City is administering no-stakes student surveys solely to give teachers feedback and help them improve instruction.
“Using [non-cognitive measures] for high stakes is not the only way to use them,” said Kraft.
Help fund stories like this. Donate now!
