Carello: How Can Schools Know If Their Services and Supports Are Actually Driving Meaningful Student Learning? The ‘Active Ingredients’ Project Has Some Answers
No paywall. No pop-up ads. Keep the 74 free for everyone with a donation during our Fall Campaign.
“We have a partnership with a regional food bank,” a principal once told me on a school visit. “Each Friday, every eligible student gets a red backpack full of food for the weekend. … And when the parents show up (to refill the backpack each week), a reading specialist leads a 20-minute program on tactics for oral reading fluency and comprehension … and provides them with both a take home cheat sheet and additional materials.”
I was intrigued. This was an innovative approach to tackling food insecurity — a chronic problem in schools that serve economically disadvantaged students — but also to engaging parents in their children’s education. “What improvement in student reading are you seeing?” I asked.
Here is where my heart sank. The school had not been tracking any specific data, beyond anecdotes of grateful families. As a charter school authorizer, I love tales of happy parents and well-fed children, but more is needed to evaluate the success of critical supports: credible, replicable data and evidence. Out of this conversation arose what would become known as Active Ingredients: a project, launched in 2017 with funding from the Walton Family Foundation and the Bill and Melinda Gates Foundation, aimed at helping schools develop systems for measuring and reporting the success of supports — such as wraparound services, community partnerships, service projects and social-emotional learning — in driving meaningful student learning.
At its start, the State University of New York Charter Schools Institute convened authorizers, content experts, educators, leaders and policy advocates to explore the idea and create a draft framework that would guide the implementation of a pilot project. Schools wouldn’t necessarily have to create new Active Ingredients to take part. But while many already had supports such as wraparound services in place, they didn’t have clear desired student outcomes or credible ways to measure them. Through this project, we intended to provide schools with the ability to develop these measures and communicate them to their authorizer and other stakeholders in the language they understand: scientifically valid and replicable data.
The pilot project rolled out in 2019 with six authorizers from across the country: SUNY, Chicago Public Schools Office of Innovation and Incubation, Miami-Dade County Public Schools Office of Charter School Compliance and Support; School District of Osceola County, Florida; Thomas B. Fordham Institute; and Washington State Charter School Commission. Each invited one to three schools to participate.
SUNY’s Broome Street Academy Charter School, serving more than 300 at-risk high school-age students in Manhattan, was among the first to sign on. The school, which partners with the youth development organization The Door, chose to explore the relationship between student engagement (attendance and dropout prevention) and mental health services offered via counseling and social work. Broome Street, with assistance from SUNY as its authorizer, worked with an expert coach to assess its “conditions for success,” including developing and articulating a theory of change; identifying available resources and mission-aligned measures and intended outcomes; tracking, collecting and analyzing relevant data; and exploring ways to incorporate the Active Ingredients — and the lessons learned — into their plans.
Through the project, Broome Street learned that 85 percent of students are aware of mental health treatment and, importantly, that most were “counseling-ready.” This openness to social work has allowed the school to enhance its program by adding more consistent outreach and, through new survey tools, the ability to ask better and more thoughtful questions about students’ well-being. Together, this information and data shaped the school’s post-quarantine plans for expanding counseling and services to all students. Today, as a result of the project, more students are seeking support from a mental health professional on their own, on a regular basis.
The question we grapple with now, especially as the pandemic and cancellation of state exams have further underscored the importance of additional measures of school success, is, where do we go from here? And how do we give every charter school the opportunity that Broome and others have had to develop and integrate new credible and replicable measures of success?
The answer we settled on was finding a way to transform Active Ingredients from a small pilot project to a standard authorizing practice — and not just in the few states represented by the authorizers involved in Active Ingredients to date. To make this a reality, we will lean on one of our key partners, the National Association of Charter School Authorizers, which has been with us since the beginning and has agreed to take the lead on expanding the definitions of student success and school quality, nationally.
NACSA’s national reach and 20-plus-year history of working with authorizers and other policymakers enable it to build on the excellent work already done and expand the knowledge base and reach. Through tool and resource refinement, integration into core authorizing approaches, multiple learning and engagement opportunities, and access points for schools and authorizers, NACSA will build on the work piloted by SUNY and other authorizers to support broader efforts to expand definitions of quality.
Everything considered, I would say a pretty good result from one inspiring story about red backpacks.
Susie Miller Carello is executive director of the SUNY Charter Schools Institute.
Disclosure: The Walton Family Foundation and the Bill and Melinda Gates Foundation provide financial support to The 74.
Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter