'Tree Octopus' proves journos no smarter than 13-year-old Americans

We believe anything we read on the wire

During Saturday – at least to this writer, it may have started Friday in the lagging time-zones – a story started to take off on Twitter, news sites and blogs.

Picking up – either marginally re-written or verbatim – a wire release, journalists were gratified to discover that the Internet makes kids stupid. Specifically, the research (from the University of Connecticut’s New Literacies research team, led by professor Donald Leu) asked the children to research as real information a species, the Pacific Northwest Tree Octopus, which was actually a decade-old hoax.

The idea is that if the research subjects – “digital natives” – were any good at evaluating information on the Internet, they would catch the hoax. They didn’t, therefore the Internet is turning children into poor researchers.

And because the children believed the research materials, the researchers – and the credulous and lazy journalists that ran with the story – concluded that the Internet is destroying research skills.

Certainly, reporters’ research skills were tested. Almost nobody treated the story with even the mildest skepticism.

In a talk called The New Literacies of Online Reading (warning: it's a PowerPoint), Professor Leu says: “In our recent work, 96% of 7th graders (24/25) recommended The Pacific Northwest Tree Octopus Site as reliable to another classroom”.

To put this in context: a sample of 25 students is almost double the sample size that Andrew Wakefield used to prove the link between vaccination and autism.

Those reporting that “the Internet is making children stupid” also failed to ask:

“Was this the only task set for the children?” – this is important because if other tasks were included, they could impact the research. The children may have been put in a trusting frame of mind by other tasks; or they might have been asked to follow up the “Tree Octopus” information at the end of their attention span.

“Did anything about the research protocols, or the interactions between researchers and children, influence the results?” – For example, did the “Tree Octopus” information come after the researcher had been accepted as a trusted source by the children?

“Could any other factors affect the childrens’ research skills?” – For example, have they been taught well? What’s their educational and/or socio-economic background?

“How do you create a control to differentiate between the research skills of different student cohorts?” – Without a control, the research fails to tell us whether the problem is particular to research conducted using the Internet.

Finally, journalists could have asked whether their instinctive response to this story reflected their own selection bias, rather than the research outcomes.