Shut Out of Schools Due to Pandemic, Many Education Researchers Say Their Work Is ‘In Shambles’
- .@HamiltonLauraS, at @RANDCorporation for 20 years, said she has never seen an #edresearch upheaval like the one brought on by #Covid-19. ‘It has changed everything that we do.'
- ‘Anybody who's frustrated that their data collection protocol on one more study on PD that none of us will trust is being disrupted, I think it shows just a profound failure of imagination.’ — @rickhess99
The ongoing COVID-19 pandemic has fundamentally changed how K-12 schooling works for virtually all of its players: students, parents, teachers, administrators and staff.
This fall, add one more group: education researchers.
Their work is also fundamentally changed, since they often study and rely on the very pieces of the education puzzle that are missing in most places: open access to school buildings, in-person teacher-student interactions, reliable attendance data and scores on big end-of-year state assessments.
As access to people and data dry up, uncertainty is overtaking an unseen, multimillion-dollar infrastructure built to guide schools, districts and even educational publishers to approaches that work.
Ongoing long-term studies “are now kind of in shambles,” said Douglas Harris, director of the National Center for Research on Education Access and Choice at Tulane University.
Many researchers, he said, have spent years designing and executing studies that require ongoing, close observations of students and teachers. Others require precise comparisons of curricula, observational studies or randomized controlled trials. COVID-19 closed down schools within days, locking them out. “The context changed so suddenly and unexpectedly,” he said. “Whatever the original question was, they can’t answer it now.”
He recalled a colleague who was studying a reading intervention. The research, begun in 2019, was to rely on this year’s results from state reading scores. But U.S. Education Secretary Betsy DeVos scuttled those plans last March when she gave states a one-year standardized testing waiver due to the pandemic.
“OK, well, there aren’t any outcomes,” Harris said. “And even if they had test scores, how would you interpret the effects of that intervention, given what happened with COVID? The meaning of that becomes much, much less significant.”
The overall effect, he said, is clear: “Most of the data this year, of any type, isn’t going to be very meaningful. Attendance data, discipline data, test score data — it’s just not going to be very informative. There are just going to be so many errors. And even if they are not errors per se, the interpretation is so muddled.”
While it may seem wonkish and esoteric to outsiders, the multimillion-dollar world of scholastic research would typically be humming in a school near you by now, quietly attempting to resolve many of the most vexing riddles in education: How do we close the achievement gap between white students and students of color, or between middle-class kids and their peers in low-income families? How do we isolate what makes teachers effective? And how do we turn around chronically low-performing schools?
‘It has changed everything we do’
Even if they’re able to keep collecting data this fall, researchers face another hurdle: overtaxed school personnel who are simply trying to figure out how to keep families happy.
Mark Schneider, director of the Institute of Education Sciences at the U.S. Department of Education, said many schools that are open have instituted blanket bans on non-school employees. “Research, even though it’s important, is not viewed by most as ‘essential,’” he said. So researchers can’t get waivers to allow them to work in school buildings. “I mean, do you blame anybody for limiting access to schools and kids?”
Teachers and administrators face challenges that are “all-consuming,” said Laura Hamilton, a behavioral scientist who directs the RAND Corporation’s Center for Social and Emotional Learning Research. “The idea of even engaging with external researchers, and just putting one more thing on their plates to worry about, is really challenging.”
Hamilton, who has worked at RAND for 20 years, said she had never seen such an upheaval as that wrought by COVID-19. “It has changed everything that we do.”
Elizabeth Albro, commissioner of the National Center for Education Research, one of four centers Schneider oversees, said that in many cases, universities themselves “are prohibiting people from doing in-person research unless it’s essential” such as working on a vaccine. “A lot of human-subject work has just been halted,” she said. Overall, about half of the center’s research efforts have been affected. Albro said she won’t know the financial implications of the “hibernation” in research until this spring or possibly even the spring of 2022 “because a lot of our projects are just reducing effort. They’re conserving their funds” in the hope that schools will reopen on a widespread basis in the fall of 2021.
She noted that Harris’s Tulane research center shifted gears amid the pandemic last May. He stopped simply analyzing past data and turned his attention to analyzing the messages of more than 3,500 school websites. The effort began about six weeks after most schools had shut down in-person instruction when they found that schools “seem to have responded most extensively in providing academic instruction to general education students,” apparently doing less for students with disabilities and English language learners, among other findings. They found that schools in the Midwest did the most for students, while schools in the South, as a group, did the least.
“Researchers are resilient,” she said. “And honestly, kids are resilient. So I think we have a lot we can learn around resilience in the context of this whole thing.”
V. Darleen Opfer, who directs RAND’s education and labor practice, estimated that about 30 of RAND’s 100 or so school research projects faced shutdown last March. Of those, just five have restarted. In the flurry of closures, she recalled, RAND’s funders said they’d wait to see whether the new school year brought a widespread return to in-person instruction. “Well, now we’re to next year, and that’s not going to happen,” she said.
In a few cases, she said, researchers began replacing in-person observations with digital surveys that push research queries to teachers’ mobile phones at the end of the school day, asking them to recall what they taught and how they taught it.
Schneider said the extra COVID-19 precautions are raising both the difficulty and expense of research. IES hands out about $90 million annually for research and manages a portfolio worth about $400 million.
Much of the work he funds is facing uncertainty, including the long-anticipated winter administration of the congressionally mandated National Assessment of Educational Progress, or NAEP, known informally as the “Nation’s Report Card.” Schneider said the institute asked Congress for $65 million to maintain next year’s full testing regimen of 500,000 students and 13,000 schools, since the institute must pay extra for safety procedures, more personnel and other costs. But the Aug. 28 deadline to make that happen passed without approval. So researchers now plan to test just 250,000 students at 6,500 schools.
“Everything gets divided in half — except the cost,” he said.
As for state tests, DeVos earlier this month told state school chiefs that they shouldn’t expect a second round of waivers this spring, but the cancellation of state assessments last spring struck a blow, Opfer said, leading people like her to rethink researchers’ reliance on the tests. That is part of a larger bid to make both schools and the research efforts that rely on them “more resilient” in the face of uncertainty, whether due to pandemics or other disruptions, “because it’s just going to happen again.”
Concerns like these aren’t isolated. In July, a group of researchers led by the University of Texas-Austin’s David DeMatthews found that inquiries of all kinds have been disrupted, “including clinical trials, case study and ethnographic inquiry in schools, and longitudinal studies using administrative or survey data.” Researchers using restricted-use data, which can contain information on sensitive topics such as students’ disabilities, mental health history, drug use or criminal behavior, have found themselves “locked out of secured labs.”
But even as their work dries up, DeMatthews and his colleagues suggested, researchers should step up to help schools improve remote teaching with targeted professional development “based on emergent needs.”
Harris, the Tulane scholar, said the pandemic has had less of an ill effect on his center’s research than on others, since he largely conducts “backward-looking” studies of school choice and other reforms in New Orleans. Much of his data, in other words, is already in hand. Actually, he said, the new school year may be a great time to study a few emerging choice-related topics.
“Homeschooling, virtual schooling, studying online tools — all of those things become much more relevant than they were in the past,” he said. “The relationship between parents and teachers generally, and how they interact, becomes a bigger topic.”
Reviewing grant proposals recently for a foundation, Harris said, he noticed renewed interest in work by longtime homeschooling scholars. Until now, he said, “they could never get any funding for homeschooling. Nobody was that interested.” Suddenly it’s newly relevant.
That’s actually how education research should work, he said. “We should be studying important problems — and the important problems have shifted.”
Rick Hess, who directs education policy studies for the American Enterprise Institute, said the pandemic has created “huge, important, valuable, pretty interesting questions on everything.”
Apart from those researchers who have forfeited long-term data, he said, he’s not concerned about lost progress. “Anybody who’s frustrated that their data collection protocol on one more study on PD [professional development] that none of us will trust is being disrupted, I think it shows just a profound failure of imagination.”
Instead, he said, in this moment researchers should be laser-focused on practical topics such as how to optimize online learning and how to help families create better home learning environments. Researchers, he said, “have a million questions” about issues online schooling, the roles teachers play and social-emotional learning.
Schneider, the IES director, said shifting existing projects midstream is not that simple for most researchers — especially those getting federal funding, since the research must be close to the original concept that was approved and within a researcher’s field of expertise.
“They can’t just say, ‘I can’t do my original research; here’s something more interesting, fun and different,’” he said. “So there are limits to what we can and will approve in terms of repositioning grants.”
But Schneider said the institute is helping researchers “reposition” when possible with a new — and newly funded — emphasis on remote learning. “I think we were all moving in that direction, but the COVID thing just created a catastrophe and potentially an opportunity to take everything we need to know about online learning to the next level.”Submit a Letter to the Editor