The evaluation, done by a team of seven scientists, is a response to a very public disagreement about the effectiveness of brain games, Simons says.
In October 2014, more than 70 scientists published an open letter objecting to marketing claims made by brain training companies. Pretty soon, another group, with more than 100 scientists, published a rebuttal saying brain training has a solid scientific base.
"So you had two consensus statements, each signed by many, many people, that came to essentially opposite conclusions," Simons says.
In an effort to clarify the issue, Simons and six other scientists reviewed more than 130 studies of brain games and other forms of cognitive training. The evaluation included studies of products from industry giant Lumosity, which has been a prominent sponsor of NPR and other public radio programming.
"We went through each paper and tried to look at the kind of evidence it provided," Simons says.
That meant asking questions like: How big was the study? Did it have an appropriate control group? Do the results support the marketing claims made by companies?
The scientists found that "many of the studies did not really adhere to what we think of as the best practices," Simons says.
Some of the studies included only a few participants. Others lacked adequate control groups or failed to account for the placebo effect, which causes people to improve on a test simply because they are trying harder or are more confident.
There were some good studies, Simons says. And they showed that brain games do help people get better at a specific task.
"You can practice, for example, scanning baggage at an airport and looking for a knife," he says. "And you get really, really good at spotting that knife."
But there was less evidence that people got better at related tasks, like spotting other suspicious items, Simons says. And there was no strong evidence that practicing a narrow skill led to overall improvements in memory or thinking.
That's disappointing, Simons says, because "what you want to do is be better able to function at work or at school."
The evaluation got a warm reception from at least some of the scientists who had signed the 2014 letter defending the science behind brain training.
"The evaluation was very even-handed and raised many excellent points," says George Rebok, a psychologist at Johns Hopkins University who has been involved in brain training research for the past 20 years. "It really helped raise the bar in terms of the level of science that we must aspire to."
Rebok, who says he has no ties to brain training companies, remains optimistic that the right program of brain exercises can improve mental functioning and delay the effects of aging.
One reason brain games haven't shown a clear benefit so far, he says, may be that they don't work the brain hard enough or over a long enough time period.
"It takes mental effort and practice to be able to see results," Rebok says. "If we can implement that long range, I think that there will be a big dividend eventually."
In the meantime, the brain training industry is facing scrutiny from the Federal Trade Commission.
In January, the company behind Lumosity agreed to pay a $2 million fine to settle FTC charges that it made unfounded claims about its brain training program.
And in May, a smaller brain training company called LearningRx agreed to pay a $200,000 fine to settle similar charges. Even so, the Learning Rx website still promises "A better brain at any age."
Copyright 2016 NPR. To see more, visit http://www.npr.org/.
9(MDAxOTAwOTE4MDEyMTkxMDAzNjczZDljZA004))
