Explainer: Why Test Glitches Are a Much Bigger Problem Than You Think

While schoolchildren in Tennessee and Alaska are gone for summer break, adults in both states are stuck behind with a big mess on their hands. In both states, end-of-year statewide tests in reading, math, and science were cancelled because of technical and administrative problems on test day.

In Tennessee, widespread glitches with online assessments, followed by a scramble to procure paper tests in time, led Commissioner Candice McQueen to cancel testing for grades 3-8.

In Alaska, a rogue backhoe thousands of miles away in Kansas hit the fiber optics cables powering Alaska’s online testing server, leaving many of the state’s 130,000 students staring at blank screens on test day.

Neither state was alone in these difficulties: incidences of both technical and procurement problems with state K-12 tests were common this year.

Cancelling year-end assessments has complex and troubling implications. Information is missing on how well roughly 540,000 American students performed in school this year. No publically available information exists for any students in Alaskan public schools, and almost no data for Tennessee students in grades 3-8.

High school assessments in Tennessee went forward as usual, and most students in grades 3-8 were able to take the first part of their tests, but those results cannot be reliably reported and analyzed without the second half of the test.

Schools use the feedback provided by summative tests to target student and school improvement in a host of ways. Teachers may use the data to start vital conversations with students and their families about past performance and future goals. School leaders (when test results come back early enough) use the results to help make decisions about promoting and retaining students and then again for course placement when students arrive in September.

School and district leaders also use the data to analyze how different cohorts of students are progressing over time and how effectively teachers were able to help them grow. Teachers use the data to chart their own progress and continue to improve their practice.

Without summative data, all of these practices become significantly more challenging. Schools are left to fall back on data from student grades and educators' informal observations, which, while certainly valuable, do not necessarily paint a full picture of student mastery. Take, for instance, a student who ends the year with a C in English class but scores at advanced proficiency on her state English exam.

Based on just her grades, a school would probably place the student in remedial or general English in the coming grade. Administrators likely wouldn’t know (especially if the student were in a new school) that she earned A’s on her class assignments but received a C because she never handed in homework. Her test scores, taken together with her grades, would cue administrators to reconsider her record — and perhaps put her in a more appropriate place in Honors English the next year.

Schools at least have the benefit of some data on which to base their decisions. The state and federal governments, on the other hand, are left to fly nearly blind. Without test data, Alaska and Tennessee are both missing a foundation for state and federal accountability systems.

Tennessee has released guidance on how its state accountability system will be affected by and adapt to the lack of data: teacher evaluation systems will feed different data into their formulas, no schools will receive recognition for high performance, and a temporary hold will be placed on schools entering and exiting the bottom 5 percent of schools identified for targeted interventions.

Without this data, no schools will enter or exit Tennessee’s Achievement School District, no new schools will receive School Improvement Grants, and no schools will enter or exit the state’s district-within-a-district turnaround model, in which struggling schools remain within their districts but receive increased autonomy.

Alaska’s Department of Education and Early Development, by contrast, said in an email that “due to the timing of the implementation of ESSA [the most recent renewal of the Elementary and Secondary Education Act]…there was no plan to run a state accountability system based on the 2016 assessments.” The outcome for state accountability would have remained much the same with or without test data, in other words.  

Both states will also have to navigate federal accountability law which, requires that they test in reading and math yearly in grades 3-8, and once in high school, in addition to testing once at each level (upper elementary, middle school, and high school) in science.

Tennessee’s Department of Education says that they remain “in constant contact with the United States Department of Education to ensure that we are following federal law to the best of our ability.” Alaska has moved ahead with the formal process of seeking a retroactive waiver from the department for cancelling assessments.

The approval process may be complicated by recent legislation, currently awaiting signature by Alaska Governor Bill Walker, that suspends assessments in the state for an additional two years — with the caveat that testing will be reinstated if the Department of Education threatens to revoke federal education funding. The bill, which would prolong Alaska’s violation of federal education law to three consecutive years, places the request for retroactive leniency on shakier ground.

Non-government stakeholders are also at a loss without achievement data. Mendell Grinter, Tennessee’s State Director of Black Alliance for Educational Options, says “[Student achievement data] fits into our work in a lot of places. We go into communities to talk to families about high-quality schools, and at the start of that process we usually begin by looking at a school’s proficiency rates for the past few years, how it compares to a school just down the road or on the other side of town.

“Then we compare to how it should be performing relative to the national average,” she continued. “A lot of communities are shocked to learn that the school they love is a low-performing one.”

Because using assessment data is central to this process, Grinter says that for now they will simply have to rely on previous years’ data.

Perhaps, as U.S of Education Secretary John King has said, some amount of error is inevitable as states transition to new tests, new vendors, and new test formats. But we have built our systems of improvement, for individual students all the way up to national accountability policies, on assessment data. When errors occur and data are compromised, each level is affected and the rate of educational progress is slowed.