AI Makes Quick Gains in Math, But Errors Still Worry Some Eyeing Reliability

Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter

While artificial intelligence has made remarkable gains in mathematics, its well-chronicled shortcomings in the subject continue to frustrate those keen on finding new ways to help kids learn. 

“Big picture, AI is not very good at math,” said Alex Kotran, co-founder and CEO of The AI Education Project. “Language models just predict the next word. You get mixed results using language models to do math. It’s not yet mature enough to where it can be trusted to be scaled.”

And even if it were to improve, critics worry it might hurt kids’ ability to try — and fail — on their own. Much would be lost, Kotran said, if “we get rid of productive struggle and we build this instinct where the first thing you do is go to AI for help.”

But students in the United States and the United Kingdom have a different view. A recent survey found 56% believe AI could go a long way in reducing math anxiety. 

Fifteen percent of the 1,500 16- to 18-year-old students surveyed said they had already experienced this relief themselves and slightly more than 1 in 5 said their math scores improved because of the technology. 

The survey also included 250 teachers. Sixty-one percent suggested students view AI as “a mentor or study partner rather than a crutch”, while nearly half “see value for students in using AI for help with the process of learning math concepts, rather than to give answers.” 

Nicole Paxton, principal of Mountain Vista Community School in Colorado Springs, said her teachers use AI in many ways. Tools like MagicSchoolAI analyze student responses to math prompts, with AI generating “specific, standards-aligned feedback for each student, focusing on their reasoning, accuracy, and math vocabulary.” 

Paxton said the tool highlights strengths and misconceptions, “which helps teachers give timely and targeted next steps.” The practice saves educators time so they can “more easily differentiate their re-teaching or follow-up, especially when addressing common errors across the class.”

Robin Lake, director of the Center on Reinventing Public Education, recently looked at the evidence base for using AI in math instruction, including whether it can help the “shocking number of students” with foundational skills’ gaps like those identified in a recent “Unlocking Algebra” study. 

The May 13 analysis by TNTP found that almost half of the students sampled started the class with only one-third of the concepts and skills needed from earlier grades. Lake said AI can be used by schools to identify children who are struggling — and, at least to some degree, by the students themselves. 

“AI can be very helpful in analyzing data and identifying gaps in student learning,” she said.

And, if a student wants to learn a mathematical concept in a different way than what they’ve experienced in class, she said, AI can provide a valuable alternative. 

“A lot of students are already doing this,” Lake said. 

More districts are training staff to use the technology, though many educators remain reluctant. Terrie Galanti, associate professor at the University of North Florida, said AI success in student learning depends on how teachers are prepared to use it. 

“AI can be more than an explainer or an answer giver,” said Galanti, who teaches secondary mathematics and STEM integration/computational thinking. “With thoughtful prompts, AI can become part of interactive, collaborative conversations to deepen mathematics understanding.” 

​​The National Council of Teachers of Mathematics said in a February 2024 position paper that teachers have long been accustomed to technological advances that change the way students learn. 

They had already adjusted to the availability of pocket calculators in the early ’80s and, more recently, to the widespread use of PhotoMath, a mobile app that recognizes and solves math problems. 

It notes that advancements in AI make teachers more, not less valuable, in student learning. 

Latrenda Knighten, the organization’s president, told Education Week in March that students will still need to rely upon their own discernment to solve mathematical problems — regardless of what tools become available.  

“We know that children learn math from being able to problem-solve, being able to use reasoning skills, critical thinking, having opportunities to collaborate with each other and talk about what they’re doing,” Knighten said. 

Irina Lyublinskaya, professor in the department of mathematics, science, and technology at Teachers College, Columbia University, distinguished between chatbots like ChatGPT and computational knowledge engines like WolframAlpha. She noted math specific AI-powered applications — including WolframAlpha and Symbolab — work very well. 

“AI chatbots can help students learn math, and they can help teachers to support students, but this is not about asking ChatGPT to solve a math problem,” she said. “I know of research-based initiatives that use AI to adapt learning materials to students’ learning styles and abilities and these definitely help students learn.”

One, she noted, was EvidenceB, developed by researchers and educators in Europe, and is now being tested in NYC. 

“Chatbots can be trained as teaching assistants or tutors that can provide students proper scaffolding and feedback, helping them to learn math the same way they would with a real person,” she said. 

Zachary A. Pardos is an associate professor of education at the University of California Berkeley where he studies adaptive learning and AI. He found, in a study conducted a year ago, that 25% of the answers provided by ChatGPT in algebra were incorrect. 

“That’s pretty high,” he noted. “Much higher than you would want.”

But the technology has improved since then. 

“With the right techniques — at least in algebra — from an error perspective, I feel it is ready for real-time intervention in math,” he said.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter