Bishop Hill: Gonzo science and the Hockey Stick

Torturing the climate numbers until they confess

Interview In 2001 the IPCC published its Third Assessment report prominently featuring a graph that became "the logo of global warming". Previous historical reconstructions didn't show our modern warm climate as particularly anomalous. This was very different, and was hailed as a "call to action". Yet Michael Mann's studies were deeply flawed. Omit one or two proxies, for example, and the scary warming 'spike' disappears. Mann's model could produce hockey stick shapes using random data, such as baseball scores, or red noise. Critics alleged that Mann's choices of data and statistical tools all cooled the Medieval Warm Period, and emphasised late 20th Century warming.

A new book recounts how the 'Hockey Stick' model was created and more intriguingly, the political and institutional defence of the indefensible. (At one stage the Hockey Stick's defenders argued that trees on different continents had "teleconnections" with each other - a claim that wouldn't be out of place in a homeopathy brochure.)

Andrew Montford, a science publisher and blogger under the name Bishop Hill, has provided the storytelling to match the detective work and persistence of another blogger, Steve McIntyre, who dismantled the Stick. Andrew talked to us about The Hockey Stick Illusion and some of the key issues it highlights.

You say the IPCC needed the Hockey Stick to be true, and that's why it got such prominence. Was this the best they could do?

If you put any of the other major temperature reconstructions in place of the Hockey Stick I think it's true to say it doesn't look so frightening. They have a Medieval Warm Period that is within a whisker of the temperatures we have now. You can make them look scarier by overlaying the instrumental record over the 20th Century part of the graph - in fact, that's part of the reason why it looks scary. I show this in the book - in Briffa series [see below].

But [these reconstructions] wouldn't convince anybody there's anything going on there at all. So yes, they needed it to convince people; and they promoted it, they had policies to sell. And when it proved to be flawed they had to stand behind it, because otherwise, they'd look stupid.

Now to the story. The preparation of the original Hockey Stick turns out to be a small part of the saga. How the issues McIntyre raised were dealt with by the scientists and the establishment is the main story.

Yes, the book is a detective story of how Steve McIntyre worked out how the Hockey Stick got its remarkable shape, which was a combination of data that's not suitable and questionable statistical methodology. I didn't want to dwell on what happened upfront, it's about how the Hockey Stick met its demise, if you like.

So for the data: people might have heard of Mann's use of bristlecones, which are poor proxies for temperature.

Right, the theory is if you have the right trees you can use them as a thermometer - they'll grow more in a warm year than they will in a cool year. But not every tree. The theory is that trees at the northern limit or upper treeline on the sides of mountains will be sensitive to temperature. But they could be affected by water and nutrients. And there is some doubt as to whether there really is a temperature signal in the tree rings. It's certainly very noisy, if it's there at all.

And this is one of the things that comes out in the book. I spend quite a lot of time discussing the verification statistics: if the mathematical model derived from the tree rings actually is credible in statistical terms. Whether it looks as though the model can create temperatures of the past.

Keith Briffa's reconstruction showed no hockey stick

And this is something Mann went to great lengths, with the Hockey Stick, to keep from us?

Essentially statisticians have a range of measures of testing whether the numbers that fall out of their mathematical models match up to the real world. In the case of the Hockey Stick they used something called a verification period, where they reconstruct the temperatures for the second half of the 19th Century, based on the model, and compare that to the real temperatures, based on thermometers. That can be calculated using a range of measures, but the two that are important for the history of the Hockey Stick are R2 and the RE.

The R2 statistic is essentially a very standard statistical measure of how lines match up against each other, while the RE is a very obscure measure that is only used by climatologists, and has been very heavily criticised because its behaviour in different circumstances isn't really understood. It hasn't been studied by statisticians.

In the story of the Hockey Stick I spend quite a lot of time discussing a paper by two associates of Mann called Wahl and Amman, claiming they had replicated the Hockey Stick entirely. McIntyre did a lot of work looking at their study, and one of the things he concluded was that their model failed the R2 test. This meant it was not credible. So if Wahl and Amman's model was not credible, Mann's was not either.

McIntyre put a lot of pressure on the journal that published it to publish the data and methodology, and they eventually agreed. And it said the Hockey Stick passed that RE test.

There's another really amazing story of how McIntyre found it didn't pass the RE test at all, but there had been a rather dubious, ad hoc adjustment made to the procedure to make it look as if it passed the RE test. It really is the most extraordinary part of the story I think.

What did they do?

This is where it gets quite technical. With the RE statistic, the question is how high does it have to be before you decide your results are 'significant'. So you calculate a benchmark, which involves throwing random numbers into your model, instead of real data, to see how well that compares against the real temperatures.

What Wahl and Amman did in essence was not to take a random set of numbers, but a set of numbers that had been filtered in such a way that the benchmark for the RE became considerably lower, making the Hockey Stick look significant. This was a procedure that was new, no one had ever seen it before, and it defeated the basic object of the analysis, which was to compare what you got with what you could get with random numbers.

So the methodology remains secret, but Mann has gone on to apply the Hockey Stick to different situations. One of which is a Hockey Stick that didn't have tree proxies in it.

Mann has subsequently published new temperature reconstructions that all have Hockey Stick shapes, and these have been picked up by the media, and trumpeted as proof that the Hockey Stick despite its many flaws, was still OK.

There are many problems with these subsequent Hockey Sticks. The paper Mann did in 2008 is particularly famous because it was discovered that in one of the proxies he was using, which was a lake sediment series [varve] from Scandinavia, the uptick or blade of the Hockey Stick was due to agriculture disturbance of the sediment rather than anything climatic. But not only that, he'd got the segment upside-down anyway.

Broader topics

Other stories you might like

  • Running Windows 10? Microsoft is preparing to fire up the update engines

    Winter Windows Is Coming

    It's coming. Microsoft is preparing to start shoveling the latest version of Windows 10 down the throats of refuseniks still clinging to older incarnations.

    The Windows Update team gave the heads-up through its Twitter orifice last week. Windows 10 2004 was already on its last gasp, have had support terminated in December. 20H2, on the other hand, should be good to go until May this year.

    Continue reading
  • Throw away your Ethernet cables* because MediaTek says Wi-Fi 7 will replace them

    *Don't do this

    MediaTek claims to have given the world's first live demo of Wi-Fi 7, and said that the upcoming wireless technology will be able to challenge wired Ethernet for high-bandwidth applications, once available.

    The fabless Taiwanese chip firm said it is currently showcasing two Wi-Fi 7 demos to key customers and industry collaborators, in order to demonstrate the technology's super-fast speeds and low latency transmission.

    Based on the IEEE 802.11be standard, the draft version of which was published last year, Wi-Fi 7 is expected to provide speeds several times faster than Wi-Fi 6 kit, offering connections of at least 30Gbps and possibly up to 40Gbps.

    Continue reading
  • Windows box won't boot? SystemRescue 9 may help

    An ISO image you can burn or drop onto a USB key

    The latest version of an old friend of the jobbing support bod has delivered a new kernel to help with fixing Microsoft's finest.

    It used to be called the System Rescue CD, but who uses CDs any more? Enter SystemRescue, an ISO image that you can burn, or just drop onto your Ventoy USB key, and which may help you to fix a borked Windows box. Or a borked Linux box, come to that.

    SystemRescue 9 includes Linux kernel 5.15 and a minimal Xfce 4.16 desktop (which isn't loaded by default). There is a modest selection of GUI tools: Firefox, VNC and RDP clients and servers, and various connectivity tools – SSH, FTP, IRC. There's also some security-related stuff such as Yubikey setup, KeePass, token management, and so on. The main course is a bunch of the usual Linux tools for partitioning, formatting, copying, and imaging disks. You can check SMART status, mount LVM volumes, rsync files, and other handy stuff.

    Continue reading

Biting the hand that feeds IT © 1998–2022