The surprising effectiveness of having kids study why they failed

For a few weeks in the spring of 2016, nearly all the eighth graders at a small public school affiliated with Columbia University agreed to stay late after school to study math. They were preparing for a critical test, the New York State’s Regents examination in algebra. Half of the kids came from families that lived below the poverty line in Harlem and upper Manhattan. They attended a selective middle school and were advanced enough to be taking algebra in eighth instead of ninth grade. Many others were the children of Columbia professors, and none of them – rich or poor – really needed help passing the test.

But researchers set up a review class to test a theory about the best way to study for a test. For the first eight sessions, half the students had a traditional review class. They were given a Barron’s Regents review study guide with lots of practice problems. Their teachers worked through the first half of the problems, explaining how to solve them step by step.

The other half of their classmates studied the same algebra topics in a different way. They spent the first 45-minute session taking a mini practice test. They received no instruction and worked independently. The following day, their teachers went over the students’ errors. The students had four test-and-review cycles like this, for a total of four mini tests and four sessions of error review.

Then, the two groups swapped. The kids who had been taught via traditional, explicit instruction switched to reviewing the remaining algebra topics through their errors. And the kids who had been correcting their errors received eight sessions of traditional test prep. Their teachers taught both ways too, so that differences between the two modes couldn’t be attributed to a particular teacher. The following year, the same four teachers repeated the entire experiment with a fresh group of eighth graders. All told, 175 kids participated in the experiment.

Which method worked best?

On the surface, it was a tie. Students improved by about the same amount – 12% – whether they learned through explicit instruction or error review. Students had taken tests before and after the test prep course. Noting how much they improved on various algebra topics, researchers were able to trace those gains back to whether students learned that topic through explicit instruction or through their errors.

There was one big difference, however. Learning through errors was twice as powerful based on instructional time. Teachers had to teach all eight sessions in the traditional instruction condition, totaling 360 minutes of instructional time. But teachers only had to teach every other session when students learned through errors, adding up to only 180 minutes.

“You get more bang from your teacher buck, if you will, from the learning from errors condition,” said Janet Metcalfe, a psychologist at Columbia University who led the study, which was published online in the British Journal of Educational Psychology in January 2024. (The Hechinger Report is an independent news organization based at Teachers College, Columbia University, but has no relationship with the middle school or the researchers involved in the study.)

Of course, students might not see it that way. They still had to be in class for the full 360 minutes after school, with half the time spent taking practice tests in order to generate the errors.

The study is not a repudiation of explicit instruction. The students were also taking an algebra class during the school day where they had likely had a lot of explicit instruction and were already familiar with the concepts. Metcalfe said that having this background knowledge is critical for learning by errors to work. The students aren’t just guessing, but they’re making common mistakes.

“They’re just doing one little thing wrong,” said Metcalfe. “And once they understand what that one little thing is, and remember not to fall into a habit where they’ll make the same mistake, they can overcome it.”

Metcalfe offered the example of fractions. A student might mistakenly think that a large denominator means it’s a large number, but then remembers discussing the error in a review session and knows that a fraction with a large denominator might actually be a tiny number. The memory of discussing the error stops the student from making it again, she explained.

Learning from errors, however, was inconsistent. One of the four teachers produced more than twice the test score gains for students than a colleague. It’s not that this teacher was much better than the others. All four teachers produced almost identical test score gains when they taught explicitly how to solve problems. They were all good explainers.

But being a good explainer is not always the same thing as being a good teacher. Metcalfe and her team analyzed videos and transcripts of the review sessions to understand what the teachers were doing differently. And it turns out there are multiple ways to teach through errors.

The teacher who got the best results employed a sort of Socratic method. “Okay, you guys got this wrong? Why would somebody get this wrong?” recalled Metcalfe. “And he did very little lecturing, almost none.”

This teacher asked his students to talk about how they had solved the problem and why they did it that way. He asked them to talk about what they found difficult. Students would often explain their thinking to each other. Finally, the teacher would ask his students to come up with ideas on how to recognize and avoid such mistakes in the future. This teacher had a knack for maintaining a fast pace and getting through a lot of problems from the previous day’s mini test. His students’ test scores jumped by far more than 12% when he taught this way.

By contrast, the teacher who produced the lowest test score gains tended to lecture students on the correct way to solve the problems that they had gotten wrong. The focus was on the corrections, not the errors. His classes weren’t very interactive. His students’ test scores improved by only 6% instead of 12%. Still, on a per minute basis, he was as effective teaching through errors as he had been teaching traditionally.

Another teacher was extremely slow paced. “I was convinced when I was watching the teachers that the second teacher would have no success at all,” recalled Metcalfe. “He would take five minutes on one problem, and just let them mull over it.”

Her prediction was wrong. “His students did really well,” Metcalfe said, laughing. Like the star teacher, he didn’t lecture.

It’s worth emphasizing again that these were highly motivated, high-achieving students who cared about their Regents exam scores. This method might not work with less motivated students who are struggling in school.

Even with ideal students, it also seems like it takes a special teacher to pull off this kind of teaching. It reminds me of other progressive teaching approaches, from inquiry learning to project-based learning, for which researchers have documented remarkable results with masterful teachers. But maybe it’s asking too much of the average teacher to teach this way, thinking of questions on the fly that will magically steer students to the right answers. Should we be promoting ways of teaching that only a small minority of teachers can realistically do well?

My big takeaway from this study is for students. When preparing for a math exam, they should take a practice test, go over mistakes and make sure they understand why they made them.

This story about test prep was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Proof Points newsletter.