It’s “driving more accountability,” said Tal Havivi, senior director of industry partnerships at the International Society for Technology in Education, which connects educators and ed tech providers. While he’s unaware of other states doing anything similar at this scale, he said there’s a growing movement among school districts to write contracts that require ed tech providers to show results before they are paid.
That movement can’t grow fast enough, according Keith Krueger, CEO of the Consortium for School Networking, which represents school tech leaders. During the pandemic, school systems dramatically expanded the number of software products they used as companies offered free subscriptions for a limited time and the federal government showered districts with emergency funding, he said. But many of the products weren’t high quality.
“There’s a coming reckoning as the pandemic funding comes to an end over the next year,” Krueger said. “School districts will have to make choices.”
The Utah state legislature created the personalized learning program in response to concerns that students were falling behind in math. The project would identify software programs that showed evidence of improving student math performance and give free licenses to school districts that applied for them.
But at first, few teachers took note. Halfway through the project’s first school year, 2014-15, just 9% of licenses distributed were being used, said Clarence Ames, who coordinates the project for the STEM Action Center, created by the same legislation. So, starting in the second year, the center began requiring software companies to offer in-person instruction for teachers at each participating school before they were paid.
The STEM Action Center made other adjustments too. Because district-level administrators typically requested the software programs, school staff were often unaware of them or learned about them too late for teachers to receive training. So, the center began requiring that district leaders, district IT directors and school principals all sign off. The center also moved up the timeline for schools to get the software — from August to February — so teachers would have ample time to test the products before a new school year.
In addition, Ames rewrote ed tech contracts to require companies to return any unused license to the project for use the following school year. The system operates like a money-back guarantee, putting providers on the hook financially.
Because of these requirements, some companies opt out of partnering, said Ames. The onsite training is expensive. “It’s a challenge for us as an industry because it’s not something companies have typically done,” said Charles Ward, a vice president at ed tech company Derivita, based in Salt Lake City. “But I think that’s on us to figure out.”
At a time of increased scrutiny of ed tech, the results from the Utah effort are notable. Since the center retooled its approach, 100% of software licenses in participating districts are opened and used.
The state has also made progress in assessing which math software products correlate with improved student achievement. By collecting data for almost 10 years, the STEM Action team identified nine math tools that show a statistically significant impact on student outcomes.
For students using project-approved software, the gains have been real. A 2019 evaluation found that students who used such tools for half an hour or more per week were about 57% more likely to test proficient in math on state standardized math tests than a comparison group who didn’t use them.
During the pandemic, when learning went online and school districts elsewhere rushed to find proven tech tools to serve students, Utah had an advantage because of its approved provider list, said Ames. When the emergency hit, the state didn’t have to scramble to find vendors whose products showed evidence of success.
That may have shown up in test scores: Utah students’ fourth and eighth grade math scores on national-level tests fell during the pandemic, but the drops were smaller than those in most states. Ames is cautious about drawing conclusions but said the math software likely played a role in keeping Utah’s numbers from falling off a cliff.
But a lot depends on individual teachers: Those whose students more regularly use the software get better outcomes.
Heidi Watson, a math coach at North Park Elementary in the city of Tremonton, said the training on ed tech tools is invaluable. Using the program’s data, teachers can diagnose individual students’ challenges and more effectively work with them in small groups, she said. Teachers have also learned to refine their assignments — for example, by asking students to complete three modules rather than to spend 20 minutes with the software.
Some believe tech tools should minimize the role of teachers. A state leader once suggested moving entirely to software-driven learning to eliminate educators, calling them “the weak link,” Ames recalled. But if anything, Utah’s data suggests that despite the increasing sophistication of tech tools, educators are needed more than ever, Ames said. “100% of our data points to the fact that that is inaccurate,” he said of the argument that teachers have limited value. “The most important variable is the teacher, no matter what.”
Ames said he’s heard from some other states and districts inquiring about Utah’s model for managing ed tech. A few years ago, the Texas Education Agency adopted Utah’s practice of requiring participating school districts to use only agency-vetted software tools that show evidence of improving student outcomes on state tests.
Math teaching is going better for Pitts this fall. She just had her students take their first quiz on the software, and because she understands the program better, she’s better able to use those results to pinpoint the specific help each student needs. She also knows where on the company’s website to find guidance, including a feature that lets her access other teachers’ real-time tips on how they’re using it, which she didn’t know about last year.
Most important, she sees how the tool fits with her instruction. “It’s not teaching for you,” she said. “It’s a tool to support your teaching.”
