When the Homework Does the Thinking: AI Use Linked to Lower Test Scores in Classroom Study

Picture a student who sails through every homework problem, then freezes on exam day. That is not a hypothetical. It is what researchers just documented in a classroom experiment, and it should give every parent, teacher, and school administrator pause before handing students an AI chatbot and calling it a tutor.

The study, first reported by The Hechinger Report and highlighted by EdSource, followed nearly 1,000 high schoolers in Turkey. University of Pennsylvania researchers split the students into three groups while they worked through math problems. One group had access to ChatGPT. Another used a specially designed AI tutor built on ChatGPT. A third group worked without any AI help at all.

The results on practice problems looked like a win for technology. Students using ChatGPT solved 48% more problems correctly than their unassisted peers. Those using the AI tutor did even better, solving 127% more problems correctly. On paper, it looked like the chatbot was closing gaps and boosting achievement in real time.

Then came the tests. When the AI tools were taken away and students had to solve similar problems on their own, the picture flipped. Students who had used ChatGPT during practice scored 17% worse on tests than students who had practiced without any AI assistance. The students who worked independently throughout, meanwhile, performed about the same on tests as they had on practice problems. Their skills held steady. The AI users' skills, it turned out, were not really theirs to begin with.

The Crutch Problem

Researchers used a simple but telling word to describe what happened: crutch. A crutch helps you walk when you are injured, but if you lean on one long enough without doing the physical therapy underneath, your leg never gets stronger. That is roughly what seems to have happened here. Students leaned on ChatGPT to get through the practice problems, but the mental work, the actual math reasoning that builds durable skill, never happened. The researchers told The Hechinger Report that this kind of AI reliance can "substantially inhibit learning."

Think about how most of us learn to do long division, or balance an equation, or diagram a sentence. We get it wrong a dozen times before it clicks. That friction, frustrating as it is, is often where the learning actually lives. If a chatbot smooths over the friction by handing you the answer or walking you through every step, you get the right answer faster, but you may never build the underlying mental muscle you need when the chatbot is not there.

This matters because AI tutoring tools are being rolled out in schools across the country with real enthusiasm and genuine good intentions. Many districts see AI as a way to give every student personalized, round-the-clock academic support, something human tutors simply cannot scale to provide. That promise is real. The Penn study even found that the AI tutor version, which was designed with learning safeguards rather than just answer-generation, outperformed plain ChatGPT on practice problems. But "outperformed on practice" and "built lasting skill" turned out to be two very different things in this experiment.

It is worth being precise about what this single study does and does not tell us. It is one experiment, in one country, in one subject area: math. It does not prove that all AI tools harm all learning in all contexts. Other research on AI tutoring has shown more encouraging results when tools are designed specifically to prompt reasoning rather than supply answers, and when teachers build in enough unassisted practice time to let skills solidify. The distinction between an AI that does the thinking for you and one that coaches you through your own thinking appears to matter enormously.

Still, the test score gap here is not small. A 17% drop on assessments is the kind of number that shows up starkly on a report card or a state standardized test. For a student, that could mean the difference between passing and failing a class, or between qualifying for an advanced course and getting placed in remedial support. These are not abstract statistics. They are real academic and life trajectories shaped by a tool that felt, in the moment, like it was helping.

There is also an equity dimension worth sitting with. Schools serving low-income students are often the ones most eager to adopt AI tools, precisely because they lack the resources to provide one-on-one human tutoring. If those tools end up masking rather than building competency, the students who can least afford a widening achievement gap could be the ones most exposed to it. That is not a reason to abandon AI in classrooms. It is a reason to be far more careful about how these tools get designed and deployed.

Teachers and parents do not need to panic, but they should ask pointed questions. Is the AI tool prompting a student to explain their reasoning, or just generating the answer? Is there built-in unassisted practice time where students have to show what they actually retained? Are test results, not just homework completion rates, being tracked as the real measure of whether a tool is working?

Why It Matters

This study is a warning shot, not a verdict on AI in education overall. It tells us that ease of use and genuine learning are not the same thing, and that a tool making homework feel easier is not automatically making students smarter. As schools race to adopt AI, this kind of research deserves a seat at the table alongside the marketing promises, because the stakes are a generation of students whose foundational skills either get built or quietly erode while nobody is testing for it until it is too late.