Select the appropriate answer:

BBC’s Brain Test Britain + Nature = ?

a) A well-researched, no nonsense brain training report
b) A landmark scientific refutation of cognitive training gains
c) Results of the largest study of brain exercise published by a respected journal
d) None of the above

It’s a trick question of course. While it would seem that the BBC’s reporting and fact-checking pedigree coupled with Nature magazine’s reputation for publishing serious scientific findings would add up to something significant, here it’s not the case.

With Brain Test Britain the BBC engaged scientists to design a battery of cognitive training tasks to test the claim that such training can improve not just performance on the tasks but also general cognitive ability. The BBC then used its not inconsiderable media exposure to engage a very large number of people (11,400 or so — which is still only about .06% of the population of Britain) in the training program.

Here’s the catch: The very premise of the study ignored existing, well-respected research on the kinds of brain training that can result in transfer to general cognitive ability. In particular:

1. The training was not frequent enough (3 days per week vs. 5 days per week in successful studies)

2. The training sessions were not time-intensive enough (10 minutes vs. 30 minutes)

3. The training tasks were varied instead of focusing on training that has previously shown transfer.

On average the participants in Brain Test Britain trained for about 250 minutes over the course of six weeks on a variety of tasks. In contrast, Mind Sparke Brain Fitness Pro’s initial training period requires about double the training time over just four weeks and focuses on a single task. It’s no wonder that there was no transfer.

Dual n-back working memory training places a very specific demand on the trainee. It is not just a task that places demands on working memory, it is a task that requires committed, focused, single-minded attention for a minute at a time. Each session includes twenty such spans back to back.

But perhaps most interesting stumble of all in the Brain Test Britain methodology was in its measurement of general cognitive ability. The study used a generally available cognitive assessment battery from Cambridge Brain Sciences http://www.cambridgebrainsciences.com/. (I’d encourage anyone interested to register and check out the tests. It’s free.)

I just went through the test battery, and in my humble opinion it’s really not a very good measure of the kinds of cognitive ability that would be valuable and useful in everyday life. Every test is time-pressured (not that this is a bad thing per se, but in everyday life much of our cognitive processing permits us some luxury of time and thought; especially tough problems.) And although there’s a matrix test, it’s not a particularly inspired one.

This astounded me: On the two reasoning tests (essentially these were fluid intelligence tests) I scored in the 19th and 25th percentiles respectively!! Whereas, my fluid intelligence on several full scale, well-normalized tests taken over the past 28 years has been above the 98th percentile and in some cases in the 99th percentile.

I would predict that a course of training with Brain Fitness Pro might help someone score better on the Cambridge Brain Sciences tests, but I wouldn’t expect this to bring them great satisfaction, nor to be an accurate measure of their true cognitive gains. Whereas a course of training with Brain Fitness Pro will translate into everyday benefits to someone’s thinking ability and problem-solving skills in the real world, as well as their scores on tests that truly measure intellectual ability.