Jun 11 2010
More Trouble for Brain Training
Brain training is the idea that training in a specific task will improve brain function for a type of skill that will then transfer more broadly to other tasks. For example, a memory task with improve your overall memory and therefore improve your performance on a different memory task. There is now a multi-million dollar industry based upon this concept.
But as is typical, for-profit commercial claims tend to race ahead of the science. The evidence for this generalizability effect is weak at best. Those who play video games have some performance advantages over those who do not. There may be benefits to engaging in novel cognitive activities. But brain training products either do not work or have a minimal effect below the resolution of research to detect.
Now a recent study published in Nature add further evidence for lack of efficacy to brain-training products. They report:
Here we report the results of a six-week online study in which 11,430 participants trained several times each week on cognitive tasks designed to improve reasoning, memory, planning, visuospatial skills and attention. Although improvements were observed in every one of the cognitive tasks that were trained, no evidence was found for transfer effects to untrained tasks, even when those tasks were cognitively closely related.
This was an online study, so there may be some self-selection bias. But it is a fairly large study. If there were a significant effect it probably could have detected it.
One thing to note is that the subjects became better at the task at which they were training. This is universally true in such studies, even in other research areas, like stroke recovery, for example. There is a generic training effect – people become better at specific tasks at which they train. This has to be taken into consideration for any study that uses performance at baseline and after an intervention – subjects will get better at the task from training alone, even if the intervention had no effect at all. Keep an eye out for that training effect, because it can make any intervention seem to work.
But again – the question here is does training in one task create generic skills that transfer to other tasks that use those same skills? The answer appears to be no, at least not significantly. This is in line with other research.
I admit, however, that this does not entirely make sense to me – which means that the research is somehow misleading, or my concept of how the brain works is not accurate. It seems that skills should transfer. My concept, which is probably common, is that working out the brain is similar to working out a muscle. If you build muscle strength by lifting weights, that strength will transfer to other tasks, like doing pull ups. Of course, the brain is not a muscle, and cognitive ability is more complex than something as straightforward as how much force your muscles can generate. But still, this conceptual framework is compelling.
What does this research on the lack of efficacy of brain training mean, then? Assuming it is accurately reflecting underlying reality, it could mean that learning is very specific. The mechanism of memory and plasticity are such that they apply to a very narrow range of activity, not generic skills. Perhaps generic skills, like how good your memory is, are more inherent and not subject to training. But applying those base skills to a specific task is all about training. Perhaps this is the relationship between talent and skill. You are born with whatever talent you are ever going to have, but you have to learn how to apply it to fairly specific skills.
To add another dimension to this, some skills are more generally useful than others. Reading seems like a more useful skill than playing the kazoo. But this is different than the underlying cognitive substrate of learning one versus the other.
At this point the bottom line of this research is that expensive brain training products and programs are probably not worth the time and money. School systems should not invest their resources and their classroom time in such systems. Learn and teach those skills that serve some function for you themselves – they are practical or fun.
28 Responses to “More Trouble for Brain Training”
This reminds me of that study that came out back in 2007 about video games and surgeons. The study measured laproscopic skills, suturing capability, video games scores, and video game experience. They were able to find a significant correlation between gaming experience and rate of errors and speed of surgery. Specifically they showed that video game skill and past video game experience were high predictors of laproscopic skill. This seems to support the idea that there is transfer of function … and that video game playing is important for American youth …
Performance training is in-line with your perspective: specific training so those skills coupled with talent will be implicitly applied when under stress, as playing on a field with other professionals in front of vocal spectators.
I wonder if the study focused solely on memory-based or object recognition based tasks. I played a popular version of such a game on my Iphone, and most of the tasks were memorizing long sequences of symbols short-term and sorting objects quickly. If so, it makes sense.
Imagine training your brain to differentiate between different quantities of a certain object, like marbles. Assuming memory is not improvable, your only improvable skill is visually estimating marble quantity.
In a different type of exercise, like a video game, you practice skills like eye-hand-controller-screen coordination or user interface understanding, which will help you master other controllers with similar dynamics and similar user interfaces.
I am still following the video-game research – I think it’s equivocal. As much as I would love for there to be a generic benefit to playing my favorite video game, the research so far has mostly found a correlation, not proved caustion.
So surgeons who have the skills that make them good at laparoscopic surgery may also have the same skills that make them good at video games, hence they find video games fun and play them.
But I think user interface and translating hand movements to video control may be generic skills that are broadly applicable (like reading).
Perhaps we need to distinguish applicability from transferability.
The research is still sorting this out.
Maybe you need a different way of exercising or a catalyst for transfer.
Usually person would just trains their normal way of thinking. But what if those games/exercises suggest another way of thinking.
For example a game where you count birds. Usual approach would be to just estimate or count one-by-one. Eventually you’ll get faster and better but no significant improvement.
If you change your technique to counting-by-groups there is an significant improvement. (That means you group birds into 10 or more and start counting 10, 24, 37 and so on.) This teaches you a new way of thinking and maybe now it transfers to other things like adding/subtracting numbers.
Now a catalyst for a transfer would show how to apply this skill. For example 4*24 is same as counting 4 times group of 24 birds. So 24, 48, 72, 96.
“I admit, however, that this does not entirely make sense to me – which means that the research is somehow misleading, or my concept of how the brain works is not accurate.”
I see the problem being that these studies and products may say more about the efficacy of the training and the study designs than they do about the brain itself. If brain ‘training’ didn’t work then education in a broad sense wouldn’t work… unless we are arguing that education only works in a specific sense (which doesn’t appear to be true). I think we are not really testing for what we think we are when we design these studies, or we are attempting to say something broad about training the brain when our results may say something more narrow about a specific task.
So what does this imply for more complex training activities,such as learning a new language or learning to play a new musical instrument?
Also I recently heard that exercise has an important role in keeping the brain healthy,so has there been any research on the effects on learning during exercise?
I am confused how reading is a “generic skill.” Truly, once taught to read you can read anything, anywhere. But this is not a generic behavior, it is very specific. The things that you are reading consist of very specific stimuli, arranged in very specific patterns, and of which you have very specific training histories. If I give you a highly technical science article in a field you are unfamiliar with (no training history) sure you can read the words but they will be meaningless. I think reading is no different than walking in this respect, we don’t say walking is some sort of general behavior because you can walk at home, at the mall, and (with special equipment) on the moon. Similarly we read sci-fi, magazines, and science articles.
As for brain skill-transfer, I agree it is alluring and took a while for it to be trained out of me in grad school. It is far more likely that the brain only supports behavior that are functionally related to the environment, such things that have been already trained (either actively by someone or passively by the environment itself). If we teach a child all the math skills she will need to know, but we teach them all in a base-5 system, there is no reason to believe she will have any skills in 6+6 or 6×6. The skill to add and the skill to multiply will likely exist, but with out any history with the specific relations of those numbers no behavior will occur.
What about slowing the decay of Brain function in the elderly?
Do you know of any good studies about weather brain workouts (like countering the Stroop effect, adding large numbers or remembering word lists) have any effect on age-induced decrease in mental capacity?
Well this is depressing news. I have this dream of developing a video game or some sort of learning program that will teach critical thinking. So that once the skill is mastered narrowly in the game, it will be applied broadly in real life.
But this seems to indicate that it would be a useless attempt.
Xal – I tried to clarify this – generic in terms of applicability. It is a skill that gives you an enhanced ability to acquire more knowledge and skills. As opposed to playing the kazoo, which does not feed into much else.
We need to separate applicability from transferability.
@ ccbowers
” If brain ‘training’ didn’t work then education in a broad sense wouldn’t work… unless we are arguing that education only works in a specific sense (which doesn’t appear to be true) ”
Education doesn’t work in the sense that learning physics doesn’t help you learn english or biology. In fact, if skill transferability was a reality wouldn’t we expect students to be equally good at everything, more or less, by the time they were done? After all, all academic learning engages the same skill sets (critical thinking, structured learning, memorizing vocabulary, etc), but the majority of students do not develop equally. Even accounting for individual preference and differential levels of focus between studies, you do not generally see multi-talented individuals.
It seems to me that people are successful in things because of talents they have, such as memory or organizational ability, and not because of what they have learned in other areas of life, no matter how closely related.
What about n-back training which improves fluid intelligence outside of the game? There’s a research paper on n-back training improving fluid intelligence which acknowledges exactly what was discovered in the study referenced by Dr. Novella while showing n-back training is an exception and generally improves fluid intelligence. Here’s the abstract:
“Fluid intelligence (Gf) refers to the ability to reason and to solve
new problems independently of previously acquired knowledge.
Gf is critical for a wide variety of cognitive tasks, and it is
considered one of the most important factors in learning. Moreover,
Gf is closely related to professional and educational success,
especially in complex and demanding environments. Although
performance on tests of Gf can be improved through direct practice
on the tests themselves, there is no evidence that training on any
other regimen yields increased Gf in adults. Furthermore, there is
a long history of research into cognitive training showing that,
although performance on trained tasks can increase dramatically,
transfer of this learning to other tasks remains poor. Here, we
present evidence for transfer from training on a demanding working
memory task to measures of Gf. This transfer results even
though the trained task is entirely different from the intelligence
test itself. Furthermore, we demonstrate that the extent of gain in
intelligence critically depends on the amount of training: the more
training, the more improvement in Gf. That is, the training effect
is dosage-dependent. Thus, in contrast to many previous studies,
we conclude that it is possible to improve Gf without practicing the
testing tasks themselves, opening a wide range of applications.”Full paper: http://www.iapsych.com/articles/jaeggi2008.pdf
Free open source implementation: http://brainworkshop.sourceforge.net/“In fact, if skill transferability was a reality wouldn’t we expect students to be equally good at everything, more or less, by the time they were done?”
Not at all. There are other factors (interest, drive, work ethic, life circumstances, etc.) that come into play. I disagree with most of what you are saying. Talent is important for determining an aptitude for something, but there are very few things for which talent is enough. Do you really think only the most talented people always end up on top? I think the correlation is pretty weak. I would argue drive and determination is most important, assuming some talent is there.
“It seems to me that people are successful in things because of talents they have, such as memory or organizational ability, and not because of what they have learned in other areas of life, no matter how closely related.”
Its hard to separate the two sometimes because people tend to gravitate towards things for which they are more talented, but again you are downplaying the role that hard work plays. The most successful people are often the hardest working, in addition to being fairly talented.
“It seems to me that people are successful in things because of talents they have, such as memory or organizational ability, and not because of what they have learned in other areas of life, no matter how closely related.”
One major point that I didn’t mention is that you are confusing improvements in specific tasks (or even general tasks) with life successes. These are not the same thing, and may be completely dischordant for the reasons I stated before.
Steve
Out of curiosity does this support the research showing a correlation in dementia between spouses. I know it is correlational not causative. Does this enhance or dispute the preliminary findings that having a spouse with dementia increases your risk of dementia. It is difficult to draw any conclusions directly. I would think that this evidence would tend to support a hypothesis that lack of stimulus, IE: dementia in a spouse, would not be the reason for the statistical correlation. Even though it was reported as such. I have been racking my brain about that research trying to come up with a plausible origin other than statistical anomaly or environmental(Meaning lack of mental stimulation).Steve–
I can see a potential deep problem in the conclusions made in this post concerning the efficacy of “brain strength training.” As an avid video gamer in my earlier years, I realize that what I say may be a result of confirmation bias, but it makes enough logical sense that I believe it valid.
It seems intuitive that certain memory tasks use the same “brain muscles” (hippocampus, basal ganglia, etc.) and plenty of studies have demonstrated that specific types of memory can be very specifically localized within the brain. But a big thing that jumped out at me when reading this blog post was a video I saw by Richard Feyman, in which he discusses the different “ways of thinking” that people have (link: — a beautiful video!).
After watching that video and reading this post, I thought about times where a stranger in my building might ask how to get to, for example, the women’s rest room. In my head, I visualize a 3-D map of the building, and I usually have to suppress the urge to point diagonally upwards through the roof to where the actual location of the room is and say “right there.” Instead, I convert this information, based on my knowledge of the layout of the building, into a series of directions. This 3-D mapping is something I learned from playing Descent, a 3-D game in which spatial mapping involved 3-D maps. I firmly believe that this 3-D mapping was a specific skill I learned in the game and is DIRECTLY transferable to real-life scenarios. It’s not “related”–it IS the same skill, exactly. But if people didn’t follow visual maps in the same way I do–for example, updating lists of directions–then I can see how the skills might not transfer.
The problem I think is the assumption that skills learned in games are not really transferable to real life–I believe they very much are, and that some “general” skills, which can be learned, become applicable to many different circumstances. I would bet that major league pitchers would beat us in darts, that Halo players would have better aim with real rifles, that people who play computer pool would be better at real pool, that Starcraft players can better judge economic tradeoffs, and that runners can better tell how fast a car is moving–because the skills are the same.
I’ve kind of started rambling I guess, but my main point is that many skills acquired through “training” are not really specific to one thing. The skill itself might be specific in terms of how it’s described, but it’s still a skill that can be used in very large set of circumstances. I think that this study chose many skills that do not hold this feature (for example, matching windows to images that were in the window, or memorizing strings of digits), and as a result avoided using skills which can be specifically applied to different situations.
#Marshall your last point was sounded close to saying “the controls were to tight”. The suggestions you offered all had one thing in common, one common controllable factor+medley of other uncontrolled factors. For example: Halo players use a lot of spatial judging by shooting, jumping, throwing grenades and so on. You suggest they would be better at shooting rifles. There are more factors to shooting a rifle than spatial recognition and judgment. There are methods for pulling a trigger, breathing, etc.
I’m not saying this invalidates your hypothesis out of hand; I am saying those are holes in how you would control the experiment and a different question than was asked in the study. You now are testing against tons of previous experiences and potential knowledge. Yes you can tease this out through statistical methods, but this was neither the question nor hypothesis for this study.
This experiment kept it as clean as possible by having people do extremely simple tasks (like the games and products the study specifically questioning the efficacy of) and then see if it was applicable in a more general use of the specific skills the games were designed to work on (again what these products claim). This means that claims from games like Brain Age, Brain Academy and the myriad of other products spawned from their popularity (and other sources I’m sure) are not supported by this study.
To me it seems you have a problem with the scope of the experiment. You could now ask different questions such as, is there an effect from more complex or dynamic skills? Is it possible that there is an affect when you perform two different tasks that use multiple skills in the exact same manner (suggesting that the ability to link sets of skills has an effect rather than how proficient you are with the skill)? There are many things that could produce your theorized effects. This study is not one of them. The next one should be seeing if there is an effect in more complicated tasks (such as the ones you suggested) and then measuring. However if there is no effect there either, I would say that it would be a very unpromising avenue. =)
I just wonder if there is a “negative” correlation between training the brain in a particular task and improvements in other areas, I mean training your brain in task A makes you worse on task B.
Neurology is not my area but at least in computer neural networks (if they represent biological ones at all), training the network strongly on a particular input reduces the accuracy of the system for other inputs. Given the fixed number of neurons and synapses in brain, this also sounds logical to me. But again this is a very rough guess.
Do you know if there are studies showing such negative correlations?
Rikki-Tikki-Tavi, How can brain workouts reverse brain and vascular damage in the elderly? If you memorize a word list regardless of color context, or add a column of numbers quickly, its very possible you simply appear to others as an elderly idiot-savant. It still won’t help with practical daily living skills including making sensible financial or heathcare decisions. These workouts may simply be useful as a time filler for the elderly as social activities.
Marshall,
“It seems intuitive…I believe they very much are…I would bet that…”
This study converts “intuition”, “believing, and “betting” into what’s scientifically defensible. It’s not the last word (no single study can be), but it’s better then the notoriously unrelaible evidence from personal experience.
Another dubious anecdote:
I have very good 3D and map reading skills, but the only computer game I ever played is “Cosmo”. Maybe Cosmo was responsible. But maybe not.Marshall
“link: — a beautiful video!”
Thanks for that.
I spent a beautiful ten minutes listening to that video.The one about training is the fiction! ISn’t most of it a matter of executive function? The individual’s ability to systematically parse a learned skill and new situations relevant to its application? Isn’t efficiency at this a function of the skill level that IQ tests for?
@Marshall,
When transferring a physical skill out of the virtual realm, I think there is a disconnect. There was a recent article in Discover: ( http://discovermagazine.com/2010/apr/16-the-brain-athletes-are-geniuses ) about how we have to train ourselves to do a physical task well. It involves the repetitive physical action so that your brain knows exactly how to adjust your arm, your wrist, whatever.
So I think the halo player is unlikely to be a crack shot in real life, because while he has learned a great deal about the visual part of the task, he has not learned the muscular part of it.But to your point, because it involves a new skill, it would no longer be a direct transfer of learning. Unlike your 3-D map example, which is a direct transfer.
@HHC,
I am merely asking a question. I was not trying to imply an answer.>>How can…
I don’t know. That’s why I asked.
I study mechanical engineering with a focus on launch vehicles. I know nothing about neurology other than what I’ve read here.
By the way: You haven’t exactly answered my question, you just answered a straw man.
You implied that I asked about reversal of the damage, while I asked about reversal of the effects. And any way you put it, you have not stated any references. Can you give me some?Hi, I couldn’t find a contact admin link, so I am just leaving a comment. I used the search box to look something up, and my virus scanner alerted me, twice, with a Trojan virus warning.
Perhaps we need better training games. Ones that teach more general skills such as good organisation, deduction etc… rather than how many meaningless symbols you can remember after 10 seconds.
I think the distinction needs to be made between “learning a task” like the ability to memorise strings of numbers and “learning a skill” like the scientific method. It would seem that “learning a skill” IS transferable and “learning a task” is NOT transferable.
Here’s my conclusion (is this a hypothesis, or is this too strong a word?).
1. We are born – our DNA “pre-programs” a base level of (for want of a better word) “intelligence” and a tendency to be “better” at some tasks, more “driven” and/or have better concentration (obviously lots more to add here).
2. We grow up – we learn “skills” and become good at “tasks” that we repeat regularly.
3. We develop and grow our “abilities” throughout our life (my definition – the combination of DNA, skills and tasks). The development and direction of our tasks and our ability to utilise tasks is influenced by our DNA and “skills” (such as the scientific method).
4. Tasks are NOT transferable, but appropriate skills (such as the scientific method) provide the basis for tasks to be utilised cross functionally, influencing the direction and “growth” of our abilities.As a non-scientist, I am not sure if this actually makes sense or is missing the mark. Also, I have not read the original article, just Steve’s article and the 27 responses.
Leave a Reply
You must be logged in to post a comment.
Monday 14 June 2010
More Trouble for Brain Training
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif){kind=small}
via theness.com
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment