News

Texting, Personalization, Free Computers for All: New Report Tells Where Ed Tech Works — and Fails

By Kate Stringer | September 13, 2017

Thousands of dollars spent outfitting a classroom with laptops might not improve student grades, while a simple series of text messages could inspire a student to attend college.

When it comes to finding education technology that works, it’s not always the most expensive interventions that lead to the best academic outcomes, reports the National Bureau of Economic Research in a new review of rigorous studies of education technology.

Unfortunately, those studies are few and far between, because research isn’t keeping pace with this rapidly growing industry. While NBER reported that the U.S. market for primary and secondary education is $8 billion, and ed tech is projected to grow to $252 billion globally by 2020, a review by The Joan Ganz Cooney Center at Sesame Workshop found that 77 percent of the most popular literacy apps did not provide information about the research behind them and two-thirds didn’t mention a guiding curriculum.

This leaves educators to sometimes make investments in technology based more on intuition than evidence, said Phil Oreopoulos, co-author of the review and professor of economics and public policy at the University of Toronto.

“The set of randomized control experiments was generally smaller than I would have thought, given the prevalence of technology out there,” he said.

In their literature review, researchers said they looked at all studies of education technology interventions that used either randomized control trials or regression discontinuity designs. Here’s what worked — and where more research is needed.

Personalization finds success with computer-assisted learning

One of the most promising uses of educational technology is learning software, especially programs geared toward improving math skills.

“It’s rare to find such consensus among a set of convincing studies that applied what we considered the gold standards of methodology for generating causal evidence,” Oreopoulos said. “There really is something to these interventions and these programs that are worth getting excited about.”

Of the 22 studies that looked at math, 15 reported positive effects, six reported no effects, and only one reported negative effects. A math curriculum in Texas that relied on computer software had one of the largest effect sizes of all the programs analyzed in NBER’s review, for its improvement of math scores for seventh- and eighth-graders.

Computer-assisted learning programs also highlight how personalization through technology can help students. A math homework program in Maine called ASSISTments gives students feedback and guidance as they solve problems. Teachers can then use the data to tailor their instruction toward student needs. This program in particular required only 30 to 40 minutes per week and had positive effects.

Eight reading and language programs were also studied, and while the results were not as robust as with math, half showed some positive effects.

Still, the research on computer-assisted learning programs focused on short-term effects. The report authors noted that more research is needed over a longer time frame to measure impact.

Texts can change actions and attitudes

Research on interventions aimed at influencing behaviors — like encouraging parents to read to their kids or spurring a student to complete the FAFSA — have shown positive results.

Moreover, some of these are as cheap and as simple as sending a text message. A preschool literacy program called READY4K! sent parents three texts each week suggesting activities for building their children’s vocabularies, such as taking their young student to the library or pointing out words that rhyme. This program showed improved learning outcomes as well as increased parent engagement in home and schools — and was estimated to cost less than a dollar per family.

Positive results were also found for programs that texted parents about student progress and attendance, though the more personalized these texts were, the more time was required from teachers or administrators to manage the programs. The review noted that texts may be so effective right now because they are a relatively new technology intervention, and that if parents are overwhelmed with texts, their effectiveness could decrease.

Social and emotional learning can also be influenced by mindset interventions that encourage students to alter their attitudes toward challenges. Twelve reviews found mostly positive results for online programs that gave students reading and writing prompts centered on facing adversity, rethinking setbacks, and overcoming feelings of isolation.

(The 74: New Tools to Fight ‘Summer Melt’: How One University’s Texting Campaign Is Keeping Incoming Freshmen on Track)

You get a computer! You get a computer!

Research shows that free computer distribution programs alone don’t lead to academic improvements for younger students, but they do have slightly positive effects on college students, perhaps because computers are more necessary for complex work. But the report cautioned that more research is needed before it can influence policy decisions.

When a community college in Northern California distributed free laptops to its low-income students, overall academics improved modestly, and computer skills rose significantly among minorities, women, lower-income students, and younger students. However, after seven years, no impact was found on student earnings.

For students in primary and secondary grades, studies revealed that providing free technology had no positive or negative effects on academics. Two studies from the Netherlands and Romania did find negative effects, which could have come from an increase in playing computer games. Studies of free technology distribution in the developing world also found little impact on learning outcomes.

However, the report pointed out that free computer distribution programs can help increase technology access for students who need it most. For example, only 67 percent of children who live in households with incomes less than $25,000 have access to computers. But when it comes to improving learning outcomes, the results are mixed.

Online learning

With over one-third of U.S. college students having taken at least one online course, there’s well-documented concern that online learning could replace teachers. But research shows that this would be unwise for education. Although more studies are needed, there is evidence that online-only courses at the college level can hurt rather than help student performance. At the high school level, a program aimed at credit recovery showed that online-only learning was not as helpful for students as in-person instruction.

“Though online learning courses have exploded in popularity over the last decade, there continues to be limited rigorous research to help us understand their effectiveness,” the report’s authors wrote.

However, when online learning at the college level is combined with in-person instruction, students perform just as well as their in-class peers, which suggests that blended learning provides possible cost-effective approaches, the authors concluded.

Submit a Letter to the Editor