“The real key takeaway from the study,” said lead researcher Carly Robinson, is that just telling students about a tutoring service isn’t enough to make them use it. “And it happens even less for those students who we think probably need it the most,” said Robinson, a postdoctoral research associate at Brown University’s Annenberg Institute and a researcher at the National Student Support Accelerator, which is promoting the use of evidence-based tutoring at schools.
The students who did log in typically had no more than four tutoring sessions during the entire spring term. Only 26 of the 7,000 students used it three times or more a week, which is what experts are recommending. One student was a power user, logging on 168 times.
It’s unclear how much this optional tutoring helped students academically. Fewer students in the group that used the tutoring the most failed classes. Fifty-nine percent of the students who were nudged (along with their parents) passed all their courses without any Ds or Fs compared to 55 percent of the students who weren’t nudged to use the tutoring. Still, even with the availability of tutoring and the reminders, more than 40 percent of the students failed at least one class.
Students in the nudged group didn’t get higher grades than students in the control group who were not nudged. In both groups, the students who took advantage of at least one tutoring session did get better grades than those who never had a tutoring session. Math grades, for example, were more than a letter grade higher – an A versus a B minus. But researchers emphasized that’s not proof that the tutoring made the difference. It’s quite likely that students who were motivated to try a tutoring session were generally more motivated students and would have had higher grades regardless.
Schools are required to spend 20 percent of their $122 billion in federal pandemic recovery funds on helping students catch up academically. Education researchers and the U.S. Department of Education are calling for schools to set up tutoring programs, especially for the weakest students who fell the most behind during the pandemic. Strong scientific evidence of academic gains has come from a specific type of intensive tutoring that takes place three or more times a week and is often referred to as “high dosage.” Hallmarks of the proven programs are not just frequency, but working in-person with tutors using clear lesson plans, rather than merely helping with homework. And the sessions are scheduled during school hours, when attendance is required.
“Good tutoring also means working with the same tutor over time and building a relationship, which isn’t usually possible with an on-demand sort of support,” said Amanda Neitzel, assistant professor at the Center for Research and Reform in Education at Johns Hopkins University and research director for ProvenTutoring, a coalition of organizations that provide evidence-based tutoring programs.
Neitzel advises schools not to spend their pandemic recovery money on 24/7 online tutoring. “I think in most cases, no,” she said. “There is very little evidence to support this, and plenty of better alternatives.”
Several online tutoring companies have been marketing their 24/7 tutoring services to schools as “high dosage.” Paper has a webpage devoted to “high dosage” tutoring, correctly explaining the model that researchers are advocating, while presenting online products as “newer, more scalable high-dosage tutoring models.” Business has exploded since the pandemic hit. Paper currently has tutoring contracts with 300 school districts around the country, including Las Vegas, Boston and Atlanta, and statewide deals with Mississippi and Tennessee.
Paper was also the vendor of the online tutoring services in the California study. The company said that it “fully” agrees with the study’s findings and acknowledges that it needed to improve the usage rates of its tutoring services. Paper says that it has since adopted many changes to boost the number of students who log in. Younger students can now record their voices instead of text chatting, for example. And it says that nominating teachers who can “champion” their product in the district and share best practices has been effective in driving more use. However, Paper declined to disclose what it’s current usage rates are.
In an interview, Paper’s CEO Philip Cutler described his firm’s on-demand tutoring as an “enhanced” version of “high- dosage” tutoring. Better? Yes, he said, because it can serve more students.
“You need to show that there’s results,” said Cutler. He said that the kind of intensive tutoring that researchers are recommending is “valuable” but it can serve only “a handful of students.” “It doesn’t move the needle nationally,” he said.
Research-backed tutoring programs, by contrast, are difficult for schools to manage, from hiring and training tutors to finding classroom space for the tutoring sessions and rescheduling the school day to make time for it. It’s much easier for school leaders to pay for an online tutoring service that takes place outside the school walls.
Cutler admitted that he cannot yet point to proven results for his on-demand tutoring. He’s currently working with independent researchers at Learn Platform and McGill University to evaluate his product.
At first glance, on-demand online tutoring would seem to be more economical. Cutler said his company charges a flat fee of $40 to $80 per student, depending on the size of the school district, regardless of hours used or how many students log in. By contrast, evidence-based high-dosage tutoring can run $4,000 per student for the year. However, given the low usage seen in the California study, per-hour costs can be similar. (Here’s my back-of-the-envelope math: If a 10,000-student district pays $400,000, but only 20 percent of the students log in for four half-hour sessions each, then the district could end up paying $100 an hour for tutoring.).
One influential educator has some advice for administrators who are trying to figure out what to do. Terry Grier, the former schools superintendent of Houston and a mentor to school leaders around the country, said schools that want to offer on-demand tutoring should negotiate tighter deals and pay only for the hours used and only if student test scores increase. He said it’s “immoral” for schools to sign “blank contracts” without strings attached. In his own experience with “high-dosage” tutoring in Texas, he said that the in-person, intensive version was very effective, especially in math. He said he also tried online tutoring, but it didn’t work well. “Kids wouldn’t use it,” Grier said.
Online tutoring is still relatively new and these on-demand services may prove to be effective.
Tutoring companies describe impressive vetting processes and training programs for their tutors, who might be fantastic. I don’t know. In the California study, lead researcher Robinson noticed that online tutors could relieve teachers from having to answer every small question that students have so that they can spend time with students who need more help.
“I think there’s a place for this type of virtual on-demand tutoring,” said Robinson.
Cutler, Paper’s CEO, told me that some teachers are telling their students to submit their first drafts to a Paper tutor to work with students on revising their essays before turning them in. Using online tutors to build good editing habits sounds like a fantastic idea to me, but it might not help students make up for pandemic learning losses.
Meanwhile, the Aspire schools in California have reconsidered on-demand tutoring and many aren’t using it anymore. The schools that are have shifted to using the online tutors for special projects and as an additional resource when parents aren’t available for on-the-spot help with homework.
This story about on-demand tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.
