How Good Are the Tests Teachers Give Their Students? Districts Need to Know

Help fund stories like this. Donate now!

At this critical juncture in K-12 education, it’s essential that schools invest in tools to better identify students’ learning needs so they can address pandemic recovery and chronic inequities. But while most districts use commercial interim assessments to guide them, far too little is known about the effectiveness of these tests.

Interim assessments are big business. The term covers a wide range of designs and purposes, but broadly, these are exams administered at different points in the school year to gauge student progress. Usage is widespread, with the heaviest reliance in urban districts — those that serve the most marginalized and vulnerable students. Many educators make instructional changes based on the results, decisions that can have profound and lasting effects on the trajectories of countless learners.

According to the RAND Corporation, three-quarters of English language arts and math teachers reported that their students had taken an interim assessment in the 2021-22 school year, and demand in this $1 billion-plus market is growing. But while states’ end-of-year exams are thoroughly peer-reviewed, no such process exists for interim assessments. Further, publishers share very little evidence to show that their products are standards-aligned or can improve student learning. For educators, this means interim assessments are a black box, with no third-party reviews of publishers’ marketing claims.

This was the very problem our organizations — EdReports, a nonprofit providing free reviews of instructional materials, and the National Center for the Improvement of Educational Assessment (Center for Assessment), an organization focused on improving the quality of educational assessment and accountability systems — sought to solve when we announced our plan to review commercial interim assessment products last year. 

Unlike EdReports’ reviews of K-12 instructional materials, for which products can be purchased independently, access to interim assessments requires publisher consent, because their test questions, reports and other tools are proprietary. Most publishers declined our invitation to participate in our new reviews. Two did agree, but then one pulled out. It simply wouldn’t have been meaningful to release a single review without context, so we had to bring the process to a halt. 

Particularly in the current moment, with districts making high-stakes instructional and budgetary decisions to try to accelerate post-COVID student learning, publicly available, independent reviews of interim assessments could have been a powerful resource. The impossibility of moving our reviews forward should be cause for concern. But by sharing what we’ve learned, we hope to inspire educators to demand greater transparency from publishers. Even without independent reviews, there’s a lot that districts can do to become critical consumers before purchasing interim assessments.

First, determine their needs:

• What are their instructional vision and overall goals for student learning in the relevant content area, and what should students therefore experience on a daily, weekly and monthly basis?
  
• What will assessments look like over the course of the school year? How will they work coherently with other instructional components to help educators understand and improve student learning?
  
• Based on the above, what do districts need in a commercial assessment product? What specific gap should it fill? If the district already has high-quality instructional materials, to what extent do their own assessments meet those needs?

Districts that do need a commercial product should get clear on what they want before looking at options:

• What is their main goal for the product? Do they want to evaluate school or district-level trends or help educators understand student progress in a specific learning area? While a publisher may claim that a product can do both these things equally well, in practice, that’s very challenging to achieve.
  
• What questions does the district expect the product to help answer, and what information is needed to answer those questions?
  
• How will the product meet the needs of its primary user? If it’s for teachers, how will the district know if it provides accurate information that educators can use to help students? What professional learning will users need in order to use the product in conjunction with instructional materials to support student learning effectively?
  
• How will the district know if the product is well-aligned to standards? What type of test questions should educators expect to see, and what evidence will confirm that the exams genuinely assess students’ understanding of the full depth of each standard? Districts should communicate their needs and ask for evidence. Equipped with a clear picture of their requirements, they can leverage their role as a current or potential customer to get the information and evidence they need.

Questions publishers should be able to answer include:

• What are the intended uses of your product, and what research supports those uses?
  
• How should assessment scores be interpreted, and what decisions can they inform? What evidence supports the idea that using the data in this way helps improve student outcomes?
  
• How were the product’s test questions evaluated, and were educators involved?
  
• Are all the test questions standards-aligned? If so, what evidence supports that claim?

In the absence of independent reviews, we encourage districts to take up the baton and exercise their purchasing power to press the assessment market for greater transparency. Students are counting on their teachers, administrators and educational leaders — they deserve evidence-based support to help them learn and grow.

Help fund stories like this. Donate now!