Exclusive: Independent Review of ESSA Plans Rates States Strong on Accountability, Weak on Counting All Kids
Bellwether Education Partners and the Collaborative for Student Success brought together more than 30 advocates from across the ideological spectrum to evaluate plans from the 16 states and Washington, D.C., which filed their submissions with the U.S. Department of Education in April.
Some states whose plans received high marks include Louisiana and New Mexico, while plans submitted by Michigan, Arizona, and Massachusetts were among those seen as weak in multiple areas. ESSA replaces No Child Left Behind, the former federal K-12 law, and was deliberately crafted to shift decision-making and power back to the states and away from the federal government.
Overall, the reviewers were pleased with the breadth of indicators states are using to evaluate schools.
“We’re definitely seeing states broaden their accountability systems beyond static proficiency rates in reading and math,” said Chad Aldeman, a principal at Bellwether Education Partners who led the review.
Some had measures in areas like science, social studies, and even physical education. All 17 jurisdictions that submitted in the first round had a measure on student growth, helpful for measuring schools’ progress with students who start school behind their peers.
“It’s not just a conversation about status and proficiency; it’s a conversation about both and the interplay between them as a much more dependable measure of a student’s progress,” said Gavin Payne, a consultant who previously worked for the Bill & Melinda Gates Foundation and as California’s chief deputy superintendent of public instruction.
The analysis is aimed both at helping states improve the plans that have been submitted and at serving as a resource for the majority of states, which will file theirs during a second review window in September.
A group of three to five generalist reviewers looked at each state’s plan, as did a reviewer focused on students with disabilities and one focused on English language learners. They rated each plan in nine areas, on a scale of 1 to 5.
The nine ratings areas were: goals, standards and assessments, indicators, academic progress, all students, identifying schools, supporting schools, exiting improvement status (improving enough to no longer need state interventions), and continuous improvement (learning from implementation and modifying the plan going forward).
Each state went through an “exhaustive” stakeholder review process, and this feedback will be one more piece to consider, said Chris Minnich, executive director of the Council of Chief State School Officers, which represents the state education leaders responsible for writing the ESSA plans.
“States are going to take this feedback. I know they’re going to take it seriously,” he said.
New Mexico had high ratings in the most categories, achieving a 5 in five of the nine categories. Reviewers praised it for its strong standards and assessment; simple, high-quality indicators of school success; and a clear list of possible interventions in underperforming schools, among other areas.
Louisiana also had high ratings, achieving a 5 in two categories.
In general, the foundation of Louisiana’s plan was “really strong,” said Gini Pupo-Walker, director of education policy and strategic growth at Conexión Américas, a nonprofit that works with Latinos, including on education, in Tennessee.
“For me, it was this great combination of high expectations, a lot of clarity, very easy-to-understand indicators, [and] an easy-to-understand school improvement scenario continuum,” she said.
She also praised the state’s use of an A–F letter grade for schools and accountability measures that shift to higher standards over the years.
“What’s an A today is not an A in three to five years, which is good,” she said.
Pupo-Walker did, however, say that Louisiana’s plan, like those of many other states, could be improved for English language learners.
The Louisiana plan embeds English language learners’ language proficiency test results in broader measures of test scores, so it’s hard to tell immediately how well schools are doing with those students. It also allows former English language learners, those who mastered English within the past four years, to count toward measures of current English language learners. That blending together could mask the poor performance of some ELLs, she said.
A representative of the Louisiana Department of Education referred to a column by 74 contributor Conor Williams praising the state for finding ways to include small populations of English learners across the state.
Each of the 17 plans submitted is different, of course, but reviewers generally said the plans were lacking detail, particularly in how they’ll make sure the performance of all students is counted in school accountability measures and how states will intervene in those that don’t perform well.
States will go further than what’s in the plans, Minnich of CCSSO said.
“I don’t think everything that’s in these plans is the entirety of what a state is going to do,” he said. “I don’t think it’s fair to think just the federal plan is going to be the only thing a state is going to do to help their kids.”
The Education Department will give feedback to states, though ESSA in most cases requires the department to approve state plans, making them eligible for federal education funds. Education Secretary Betsy DeVos has generally said she’ll take a hands-off approach so long as states abide by the law, though staff in her department gave what was seen as tougher-than-expected feedback to three states earlier this month.
Reviewers also cautioned that the state plans are turning to a “normative-based accountability” system — that is, how well schools are doing in relation to one another, rather than against a set benchmark.
Some of that is because ESSA requires states to identify the bottom 5 percent of schools for intervention, Aldeman said. But it also sets a standard where a school in that bottom 5 percent might not actually improve, but could move out of that intervention status so long as other schools dropped more.
Nevada’s plan is better, he said, in that it requires both a ranking improvement and some other objective criteria to exit that status.
“That combination, our peers felt, was a good way to balance both of those things,” he said.
Minnich also noted that the normative identification requirement could sweep up schools that have improved dramatically but remain in that bottom 5 percent, which could prove demoralizing for those schools.
“States are going to have to work on this, but the law itself is written in a normative way,” he noted.
Interventions in low-performing schools generally was a problem, both Aldeman and Payne said.
Many proposed generic “needs assessments” or professional development, suggesting they haven’t learned that “weak” interventions don’t improve schools, Aldeman said.
Payne noted that there’s no real “state of the art” in terms of turnaround, though it would be a helpful area for states to learn from one another.
“I think it’s honestly the hardest struggle. Some of the problems are so intractable, either from a policy perspective or an actual program perspective, it’s the hardest nut to crack,” he added.
Minnich, too, raised the issue of a lack of best practices in the area.
“If we knew exactly how to turn around schools, we would do it. It’s less important to me what words states write; it’s more important to me what states do to help these schools,” he said.
No state received the top score in “all students,” the measure of whether a state plan masks the performance of some subgroups of students or has adequate checks in place to ensure all students receive a high-quality education.
Advocates earlier this year said some state plans obscured the performance of children of color, English language learners, low-income kids, or students with disabilities. They did so by setting low expectations for those groups, or requiring them to be low-performing by every metric before intervening. Others set high “n-sizes,” a wonky term for the minimum number of students that would have to be in a particular subgroup before their results were separated out and reported for accountability purposes.
States should be trying to include as many students as possible without violating their privacy, Minnich said.
Many are doing so, and in some cases actually counting more students than they did under old accountability standards, he said.
“I don’t think it’s appropriate for us to just sort of universally say states aren’t doing as well on subgroups,” Minnich said.
Among specific states, Michigan had the lowest ratings, getting a 1 in seven of the nine categories. Four other states — Arizona, Connecticut, Maine, and Vermont — each received a 1 in four of the nine categories.
Michigan’s plan stood out for its lack of details — it even proposes three possible accountability systems, only one of which would meet ESSA’s requirements, without naming which one it plans to use, the reviewers said.
“As presented, Michigan’s plan is missing key elements that are required in order for the state to receive federal education funding,” the reviewers wrote in their final assessment.
The lack of detail and transparency in Michigan’s plan was “startling,” said Kerry Moll, vice president of policy and advocacy at Stand for Children.
On goals, for instance, the state aims to be among the top 10 in the country, but it doesn’t provide details of what that means or how it will get there, Moll said.
“On a surface level, they’re directionally right. I think they need to get perhaps more support,” she said.
Michigan Department of Education representatives did not respond to requests for comment. U.S. Education Secretary Betsy DeVos is from Michigan; her department has not released any public feedback on Michigan’s plan.
Massachusetts, too, was short on detail in a way that surprised many reviewers, given its long history as a top-performing state in public education.
“While there are a few bright spots, Massachusetts’ plan largely lacks the clarity and detail that are necessary to give the state the best opportunity to improve outcomes in the classroom,” Jim Cowen, executive director of the Collaborative for Student Success, said in a press release.
Massachusetts’s plan would measure growth by comparing students to one another rather than an objective standard, and growth and proficiency standards aren’t aligned to grade-level standards, the reviewers said. The plan lacks specifics on how low-performing schools will be identified, reviewers said.
Massachusetts is in the process of changing its assessment system and will set new baselines for grades 3–8 in English and math after this year’s results, with other standards to follow in later years, said Jacqueline Reis, a spokeswoman for the Massachusetts Department of Elementary and Secondary Education.
“While we’re still awaiting feedback from USED, we appreciate Bellwether’s analysis of both the strengths of our proposal and the areas where more detail might be useful. We look forward to continuous improvement of our ESSA state plan through the amendment process as allowed under ESSA and in partnership with Massachusetts educators and USED,” she said in an email.
Andy Rotherham co-founded Bellwether Education Partners. He sits on The 74’s board of directors and serves as one of the site’s senior editors.