;)
Untangle Your Mind!
Sign up for our free newsletter and start your day with clear-headed reporting on the latest topics in education.
The hope had been simple: A new crop of computerized standardized tests would help schools across the country implement higher academic standards while also measuring their students’ critical thinking skills through interactive questions and exercises.
But as states have switched to digital exams in the last two years, students have faced persistent technical problems from slow load times to not being able to log on or save their answers.
The glitches have snagged regions using leading testing companies and Common Core-aligned tests as well as those who have developed their own exams with smaller companies. In New Jersey, the state Department of Education was forced to postpone testing for an entire day in April after testing giant Pearson inadvertently introduced an error into its software while Alaska cancelled its exams for some 73,000 students that same month after an internet outage disrupted service at the firm handling the test.
And while some might be experiencing a dose of nostalgia for the days of taking tests the traditional way,—with paper and No. 2 pencils—it’s unlikely computerized testing is going away anytime soon. The SAT is developing a digital version of its test to be piloted this fall while some schools can already offer an online version of the ACT to their students.
“I think some of the hiccups we’ve seen are inevitable because they are new assessments,” said Paige Kowalski, vice president of policy and advocacy at the Data Quality Campaign. “It’s going to take some time to smooth out.”
To get to the bottom of the widespread technical mishaps facing standardized testing, The 74 spoke with Tracy Weeks, executive director of The State Educational Technology Directors Association, a nonprofit that supports technology leaders in education. Before taking the helm of SETDA, Weeks was the chief academic and digital learning officer at the North Carolina Department of Public Instruction.
In an extensive interview with The 74, she talks about why schools, districts, states and testing companies have been so vulnerable to technical glitches, how they can prevent more problems and what the future holds for digital exams. The interview was lightly edited for clarity and length.
Tracy Weeks, executive director of The State Educational Technology Directors Association
Lesson One: Plan for the ‘Burst of Activity’
The 74: In Texas, technical issues caused public school students to lose their answers on more than 14,000 copies of the state’s STAAR exam. Students' responses on the test disappeared after they had logged out either voluntarily, due to inactivity or a lost internet connection. Are lost internet connections a common problem and what can be done about it?
Weeks: What typically happens when you do online testing is you really kinda touched on it … the bandwidth becomes very important. The challenge for schools, districts (and) states when planning out for these things is that …when you get to online assessments you get these spikes in usage. You’ve got a number of people all hitting the system at one time and you’ve just got these little peaks, these spikes that happen in your internet usage. Often it far exceeds what you may typically plan on for day-to-day instructional use, which is typically what most locations purchase for. And that, I think, has become a real challenge. The flip side of that is if you try to pay for what you think your absolute most would be then you’re probably overpaying most of the other days of the school year. So it’s a delicate balance of being able to try and figure out what you are truly going to actually need, how many kids are going to be taking the assessment simultaneously, and whether or not your broadband is really set to handle those little peak bursts of usage.
Are there strategies that schools and districts can take to address those concerns?
Well, I think there are some broadband plans that will allow a certain amount of burst usage to take place. I think that’s one of those conversations that needs to happen ahead of time with the provider. We see the same thing happen when schools suddenly shift to using some online or blended course work, where now you’ve got a whole roomful of kids all hitting high-end multimedia items at the same time. It really is a conversation with the broadband provider of, ‘This is what we’ll think we’ll use on average throughout the year but how are you going to handle those bursts, those little peak bursts of usage that happen in these specific situations.’
Lesson 2: Things Will Happen, So Be Flexible
In April, the Alaska Department of Education canceled the Alaska Measures of Progress (AMP) for grades three through 10 after a construction worker accidentally cut a fiber optic cable at the University of Kansas, where the state’s testing vendor, Achievement & Assessment Institute, was located. Even after the internet connection was restored to the university, students continued to have problems. Is something like that preventable?
I don’t think it is preventable. Things are going to happen. That’s the same as somebody could be doing utility work and cut the power line outside and you don’t have the lights on in the school and you can’t do any kind of testing. There are going to be freak events that certainly can happen whether we are talking about an online assessment, a paper-pencil assessment, what have you. That is definitely unfortunate when it happens, especially when we are talking about a high-stakes, end-of-year-type assessment, which are so carefully scheduled in schools … I think allowing schools and districts to have some flexibility in the timing of the testing is something we need to take careful consideration of. There are some states that limit testing only to be in the last five to 10 days of the school year, which can make sense instructionally but doesn’t really allow for much leeway. Dealing with something like a cut fiber optic line could take a couple of hours to repair … or it could take a day or two to repair and then where do you shift the schedules to at that point? Certainly I think we need to have a deal of flexibility built into the testing schedules that take place in the schools to accommodate some of these things.
Lesson 3: Go Easy On The System Updates
Sometimes it’s the company’s fault. In New Jersey, PARCC testing was delayed for a day after Pearson, the company administering the test, inadvertently introduced an error into the program preventing some students from logging on. The company said the glitch was introduced by a Pearson employee when it was making adjustments to the system the night before to improve its performance. What can be done in those instances?
We dealt with that in North Carolina and we also used Pearson … We got to the point where we wouldn’t allow major updates or sometimes any updates unless it was a critical thing that was needed to resolve another issue. We really tried to avoid any updates immediately preceding any assessments.
Lesson 4: Practice, Practice, Practice
What are the milestones states and districts should be meeting in their preparation for their annual assessments?
Even when kids are taking those bubble tests (there should be preparation). When you get those third-graders that are taking them for the first time they still have to go through the process of let’s sit down with actual bubble sheets and take some practice tests and actually bubble them. The question is whether your contract with the company covers any review time on the test and practice tests. That would really differ by contract with the company. Again that gets into … do you have flexibility built into the schedule if for some reason the testing did not go forward as expected? How are you going to reschedule this? How are you going to back this up? … If it were me and I we’re scheduling this and my only opportunity (to prepare) was (for) the end-of-the-year assessment, I would want to have conversations (with the testing company) about scheduling a practice session. The challenge there, Naomi, is that you’re talking about pulling kids aside away from instructional time. So that practice test needs to be folded into serious review of what they would be doing anyway. So it’s got to have instructional value for the kids, so we’re not pulling them away from valuable instructional time just to kick the tires on the infrastructure.
Lesson 5: Check your server space
Tennessee students taking the state’s online standardized test, TNReady, experienced slow load times on the first day as more students crashed on the first day of testing in February. Measurement Inc., the North Carolina-based company that developed the test, said at the time that the glitches were because there was not enough server space to handle the volume of students taking the test. Later a Chalkbeat Tennessee story highlighted many communication problems between the state and the company. They scrambled to print and ship versions of the test to schools. The state canceled the online test and opted for a paper-and-pencil version instead but the company could not get enough tests printed in time. What is a server?
It’s a computer that is sitting on a rack, in a room. On this rack might be 20, 30, 50, however big the rack is, servers or computers. Let me back up. On your laptop or on your home computer, you know that you buy your computer with different levels of memory and different processing speeds and different amounts of RAM. That controls how much you can do activity-wise on your computer. So these servers have a tremendous amount of computing power — more than any one person would need. They are meant for multiple folks to be hitting them at the same time. They can store databases. They can store applications … Back in 2000, if kids wanted to take a test (as a technology facilitator) I would load the application on each computer. And each student would sit down at that computer and take the assessment and it would only be saved on that computer because they didn’t have a network at that point. Everything was loaded and I would have to get them to save it. And we’d have to export it off of that computer somehow. Servers allow all of that to be consolidated into one space.
Is this vulnerable area for states and/or testing companies?
One of the things you have to be able to guarantee and make sure is, especially if it’s a high- stakes, end-of-year exam, is that the (test) items have to be kept secure. You just can’t have them on local servers that are unprotected …There is a different level of security that has to go into making sure that access can’t be gained ahead of time and it also means you have to be a bit more strategic about what servers (are used) and where they are actually hosted. Pearson has some that you do this thing called local cacheing. It kind of loads the test in locally so that each time the student refreshes they are not having to go back to Pearson’s server to refresh. It can refresh locally just in a certain time period and then it will all be gone so to speak. So there are some mechanisms in place to help balance the speed versus security issue. Otherwise, if every kid is having to send their stuff all the way back to wherever the servers are for Pearson or name your company that can actually slow down the system. So you are trying to balance speed and security at the same time.
Is cost an issue here? Is buying server space an expensive proposition?
It’s becoming less expensive. The challenge I know companies have is you’re not just buying one server. They usually have … two or three servers so that for some of these freak things like one server melts down you still (use) two others that can still handle (the load) … Sometimes they put multiple clients on the same server because (the testing company might say) you are not taking your test at the same time as the other place is going to be taking theirs so I’m also going to put them on a different part of the (same) server. Sometimes that can come into conflict because they didn’t know (the states) would be taking them at the same time and then one group overloaded it and it affected the performance of another place. The other place wasn’t doing anything out of the ordinary. They just happened to be on the same server as the folks who were causing the problem.
Lesson 6: Think Outside the Box
Do you have a sense of when states and companies might get a handle on how to administer computerized tests? Or will technological blips always be a problem?
I think you are really seeing a shift. We’re getting into what I like to refer to as learning in the digital age. Instead of digital learning, I talk about learning in the digital age, where we finally have the tools and the applications and more of a knowledge of how we can truly leverage these tools to do good things for kids … In some cases, rather than trying to fit these new systems to our old ways of doing things, it gives us the opportunity to rethink how we do things. For instance, do we really need to continue to have just testing at the end of the year because that’s what we have always done? Or with technology could we be assessing kids throughout the school year as they are learning and be able to document that they have mastered content and know content along the way? Then you can do some really interesting things like could kids progress at their own rates based on when they can demonstrate what they know? It allows you to consider new instructional paradigms rather than saying we’re still going to do this one-size-fits all education and test you at the end … I think we are just so early in that process. We’re still worrying about cut wires and poor (internet) access that day. These are real problems but I think there are enough folks who understand what the potential out there is for where we could be. Rather than use (technological glitches) as an excuse to not to do it, we’ve got to figure out better ways to plan for it and around it.
Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter
