This is just one study with only a year or so of evidence. Bhatt says there’s a lot that researchers still need to figure out about mixing human tutors and technology to reduce costs without losing potency. This particular study had tutors working with students in a one-to-four ratio five days a week while using ed tech half the time. But $2,000 per student remains prohibitively expensive for most public schools, especially after $122 billion in federal pandemic recovery funds run out in 2024.
Bhatt is now studying how to further increase student-to-tutor ratios and time on ed tech to lower costs even more. She suspects that time needed with a human tutor varies by student and is currently partnering with schools in Illinois, Georgia and New Mexico to identify which students need more human attention and which need less.
Bhatt uses a metaphor of training for a 5K race. Most people can run this distance if they train in incremental baby steps. “If you showed up at my house every single day, watched me lace up my running shoes and ran with me, then I could definitely do it,” said Bhatt. “And there are some kids, you can just say, ‘Here’s the training schedule, please follow it.’ And that will work for them.” Bhatt is trying to figure out how much personal training each kid needs in math.
Another tutoring researcher, Philip Oreopoulos at the University of Toronto, is studying whether once-a-week Zoom tutoring sessions at home are sufficient for some students when combined with practice problems from Khan Academy, a nonprofit organization that provides free online learning.
Oreopoulos thinks the amount of tutoring a child needs might depend both on the child and the classroom teacher. In a separate study, Oreopoulos paired coaches with elementary and middle school teachers to help them differentiate instruction in their classrooms and assign different practice problems to different students on the Khan Academy website. He found that some teachers were far more successful at motivating students to do the practice work and their students’ math achievement gains were as strong as those seen in tutoring studies. Meanwhile, similar students taught by other teachers were less motivated to do the practice work. These students might need tutoring.
In the current University of Chicago study, researchers set up a tutoring lottery for almost all the ninth graders in six low-income schools, two in Chicago and four in New York City. (Roughly 10% of the students had severe disabilities or extreme absenteeism – attending school less than 25% of the time – and were excluded from the study.) A thousand students “won” the math lottery and were given an extra math class each day operated by the nonprofit tutoring organization Saga Education, whose tutoring program has produced strong results for students in several well-designed research studies. A thousand students “lost” the lottery and had another elective scheduled during this period. Everyone, both winners and losers, had a regular algebra class.
During the extra math block, about five or so tutors sat at tables in an ordinary classroom, each working with four students. The tutors worked closely with two students at a time using the Saga math curriculum, while the other two students worked on practice problems independently on ALEKS, a widely used computerized tutoring system developed by academic researchers and owned by McGraw-Hill. Each day the students switched: the ALEKS kids worked with the tutor and the tutored kids turned to ALEKS. The tutor sat with all four students together, monitoring that the ALEKS kids were on task.
This experiment started in the 2018-2019 school year and at the end of the year, the students who had this extra math block learned more than twice the amount of math than lottery losers who didn’t have this tutoring-and-ed-tech experience. More surprising, the math gains nearly matched what the researchers had found in a prior study of human tutoring alone, where tutors worked with only two students at a time and required twice as many tutors. In addition to higher scores on year-end math tests, students who received the extra math block also had higher math grades (by a fifth of a letter grade) and lower rates of failure in their algebra class. “It was remarkable,” Bhatt said.
A principal of one of the schools in the study, the High School of Fashion Industries in Manhattan, spoke at the briefing and said he continues to use Saga tutors, paying part of the tab from his own budget now that the study is over.
“One thing that they always survey students on is ‘Do you have an adult in the building that you can confide in and trust?’ You can’t underrate having that one ally in the building,” said Daryl Blank, the high school principal. “A lot of times for Saga students, it’s the Saga tutor who’s in that room, because they’re not just teaching them the math, the algebra, they’re just sort of looking out for them, cheering for them as an ally.”
The study was supposed to extend for two years, but the pandemic hit in the middle of the 2019-2020 school year and the experiment was cut short. Before schools closed, Bhatt said that midyear math grades were again higher among a second cohort of ninth grade students who had the extra math block. No standardized math assessments were administered that spring.
I have a jaundiced view of ed tech, based on the sheer number of studies that have shown null or very tiny results for students. I am concerned about replacing time with teachers and interacting with classmates with time staring at a computer screen with headphones in our own private bubbles. Maybe there is wisdom in incorporating work periods into the school day, when students do their practice work under the guidance of tutors and machines. But I’d hate to lose art and other electives to make room for it. These are tough decisions for school leaders to make.
