Analysis: Seeking New Ways to Evaluate Charter Schools Serving the Highest-Needs Students — That Might Work for Others Shuttered by COVID-19

Updated May 9

With the coronavirus pandemic bringing the school year to a sudden halt before spring tests and high school graduations, school administrators and state policymakers are struggling with how to evaluate schools when there are big gaps in data.

Now imagine if this happened every year.

That’s the situation presented by alternative education campuses (AECs), schools that serve former dropouts, pregnant and parenting teens, formerly incarcerated students and others with extraordinary risks to successful learning. For those who operate and oversee such schools, conventional metrics such as state test scores and four-year graduation rates often say little about the actual quality of their programs.

Roughly 700 such campuses are found among the nation’s 3,000-plus charter schools. In recent months, work has accelerated on ways to find new forms of accountability for these out-of-the-box schools.

In February, before in-person meetings were replaced by Zoom, more than 40 charter school authorizers met in Columbus, Ohio; Atlanta; and Hayward, California, to look for new methods and metrics for AEC accountability. They’re part of a network created by A-GAME, a federally funded initiative housed at the National Charter Schools Institute in Michigan.

For public schools of all kinds, progress of students facing poverty, disability and English language deficits is routinely broken out in state and federal accountability reports. But AECs serve students whose lives and learning have already been seriously disrupted. Typically, they enroll for multiple years, far behind grade level proficiency and missing credits needed for high school graduation. Almost inevitably, their state test scores and graduation rates will lag well behind other schools’.

For charter schools, AECs pose a special challenge. While districts often provide alternative offerings in the form of special programs whose results may be submerged in broader averages, charter AECs are autonomous. Their charter contracts must be renewed periodically, with high-stakes decisions based on attainment of quantifiable goals. For these intensely mission-driven schools, a four-year graduation rate may look dismally low — unless you know that students arrived so far off track academically that they have no hope of graduating in four years. Test scores that would earn an automatic “F” on a state rating system might tip a school toward closure; but when many of the students hadn’t attended school or held a pencil in their hands for several years, the picture might look different.

A dearth of meaningful data can leave charter school authorizers in a quandary: Should they recommend a new charter when they can’t really justify it with hard evidence, or risk closing a school and leaving students adrift because the assessment system doesn’t fit their reality?

Data from Jim Griffin and Jody Ernst, founders of A-GAME partner Momentum Strategy & Research, offer some insight. Griffin and Ernst are former leaders of the Colorado League of Charter Schools, where they advised that state in drafting its pathbreaking alternative education framework. For most of the past decade, they have been collecting and sifting through statistics on how AEC students fare on state tests and nationally normed assessments such as NWEA. What they’ve discovered is that these students not only trail their peers on achievement measures but also show different growth patterns. Their findings put a dent in the argument that simply adding a growth factor to accountability frameworks solves the problem of staggered starts.

In 2018, the institute and Momentum won a three-year grant from the U.S. Department of Education’s Charter Schools Program, building on earlier work by the National Association of Charter School Authorizers. It’s being managed by Naomi Rubin DeVeaux, who was deputy executive director at the District of Columbia Public Charter School Board.

In its initial phase, the project produced a resource guide, “Measuring Quality,” offering recommendations from an initial 11-member National Authorizer Leadership Team on how to reframe accountability for the unique circumstances of alternative education campuses. The guide proposes narrowing the definition of AECs to those with a stated mission to serve highest-risk students and with a preponderance of students having at least one critical risk factor. It suggests that authorizers work directly with their AECs to create demanding goals, starting with broadly accepted indicators such as academic performance and career readiness, but setting targets that avoid pitting AECs against mainstream schools with traditional populations.

A key part of the project is creating a solid basis for apples-to-apples accountability. At present, only Ohio, Colorado, Arizona and D.C. use other AECs as the comparison set in their alternative frameworks. In response, Momentum is building out its national Alternative School, Performance and Policy Database, currently housing public data from more than 20 states and forming the backbone of the A–GAME data tool. It enables any state or authorizer to gauge how a given AEC is doing against a nationwide sample of schools serving similar youth.

Since alternative campuses work daily to help students surmount troubling situations, evaluating their success must go beyond academics. The guide suggests strategies and resources for measuring non-academic or social-emotional factors such as motivation, persistence and engagement. And whereas conventional accountability systems rely on a limited battery of tools to measure college and career readiness (AP and SAT scores, for example), the guide suggests diversifying that portfolio with assessments like Work Keys or the Armed Services Vocational Assessment Battery, metrics such as the percentage of students gaining an industry credential, and other measures that may be more directly germane to students’ intended paths.

Members of the now-expanded A-GAME network collectively oversee about 70 percent of the nation’s alternative charter schools. Following the regional meetings, they returned to work with individual schools to design new goals. Now, as educators have pivoted to online solutions, A-GAME is also helping network members share insights about supporting AECs in this unprecedented situation, since their students may be especially vulnerable to the effects of sudden dislocation.

Comprehensive state accountability systems that depend on uniform metrics and ratings have been showing strain for some time now. Perhaps this moment, when conventional measures have evaporated and millions of families have suddenly turned to intensely personalized instruction, might be an ideal time to look at how the charter sector is expanding the boundaries of school accountability.

As institute founder Jim Goenner notes, this work has implications far beyond charter schooling or, for that matter, “alternative” schooling. “At heart,” he says, “accountability should be about whether a school is living its mission. Every school district in the country is grappling with the fact that accountability in the ESSA era has become too standardized and rule-bound. But because chartering is all about pushing the envelope, the authorizers we’re working with actually have the freedom to explore new ways of doing things.

“I’m willing to bet that school districts will be eager to try some of these tools and strategies — not just for ‘alternative’ programs but for regular public schools that serve students who arrive with a wide variety of backgrounds and challenges.”

Nelson Smith is an adviser to the A-GAME project.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter