No study alone provides definitive evidence, but a new paper in the Proceedings of the National Academy of Sciences makes some important points, that if confirmed, call into question many studies finding that brain training improves cognition. The study had two conditions. In one condition, participants were recruited using flyers that mentioned that researchers were seeking people for a brain training study. This flyer was representative of flyers typically used to recruit participants. Another group was recruited using flyers that did not mention brain training. When participants got to the lab, they were given an IQ test, and then participated in a sham brain training session (1 hour of working memory training). This training was too short to plausibly do anything with respect to IQ. IQ was then measured again. Participants who were recruited in such a way as to make them believe they were in a brain training study showed a 5 to 10 point increase in IQ, while the group that had no reason to suspect the study related to brain training did not show a reliable IQ increase (and the interaction term was significant).
The same exact training either resulted in an IQ boost or not. There are a few ways to interpret this finding, but all are problematic for previous brain training studies. One explanation is that the brain training flyer induced a belief in this group that they should improve, and that the IQ test picked up on a placebo effect. A second explanation is that one flyer attracted individuals more likely to believe that brain training is possible. Again, this would be a case of belief driving performance improvements rather than the training itself (again, a placebo effect). Another possibility is that instead of a placebo effect, participants cued into the nature of the study (brain training) wanted to produce good data for the experimenter, so they worked a little harder after training to be consistent with how they thought they were expected to perform (a demand characteristic explanation). The least likely explanation is that the differences between these conditions produced an actual increase in IQ for one group compared to the other.
This study has potentially important implications for how we think about brain training effects, and especially how we conduct brain training studies and recruit our participants. It's a nice study, though not perfect. Pre-registration would have been a plus for this study, as it generally increases confidence in reported results. However, data handling and analyses reported in the paper seem pretty straight-forward. Also, it would have been interesting to test whether the same manipulation might also have influenced IQ scores of a control group that performed a task other than working memory training. This gets at the question of whether certain active control groups can eliminate differential expectations. However, overall this study seems to provide evidence that the things we have been worrying about for a while (overt recruitment, placebo effects) do matter.