Don't Be Distracted By Nit-Pickers: This Is Serious

David Rose's excellent investigative journalism in the Mail on Sunday has turned up a remarkable story of poor scientific practice, lack of openness and bias regarding the Karl et al (2015) paper in Science. This is the paper that was quickly dubbed the "pausebuster" as it was said to have removed any evidence of a "pause" or "hiatus" in the rate of increase of surface warming over the past 20 years or so contradicting the IPCC's own assessment of two years earlier.There was very little public criticism when the Karl paper came out although we now know, thanks to David Rose's piece, that that was far from the case within the scientific community.At the time of Karl paper critics pointed out that whatever the details of his new analysis was, it was clear that the authors had underestimated errors, and given that the claim of having eliminated the "pause" was premature.The Mail on Sunday piece is the most important piece of climate science journalism in a decade, opening a door on the hitherto closed world of internal NOAA discussions, revealing how scientific data can be massaged and timed, towards a predetermined end. It is rightly being taken very seriously by many people and will likely lead to further U.S. investigations.However, such is the nature of the climate debate that some are only concerned with the omission of a few words in an image caption. It's a clear case of deflection from the substantive part of the article. The problem is the caption to the graph which shows NOAA surface temperature data and HadCRUT4 temperature data post-1997. It should have said that NOAA was drawn relative to the average of its data between 1901 and 2000. HadCRUT4, however, takes its average or starting point from data between 1961 - 90. Whilst the data in the graph is accurate the caption was incomplete, until it was updated.It was seized upon by a few who made jokes about it for a while probably without stopping to think it was a caption error. But in the main it was done by those who prefer Twitter put-downs rather than anything more serious and lack the grace to look beyond a simple caption error to the bigger picture. Twitter is a wonderful way to disseminate information, but in some climate cases, in some hands, it is the shallow end of the pool for those who don't want much more than gags.To clarify matters let me show the data from NOAA and HadCRUT4 using the same baseline period.Fig 1 shows monthly data post-1997 (HadCRUT4 is black, NOAA red) and Fig 2 shows the annual data (HadCRUT4 is blue, NOAA is red and NASA GIS green). You can see that in the past decade NOAA has changed from being mostly cooler than HadCRUT4 to being mostly warmer. It might seem small but it is important or else why would Karl et al have made so much fuss about it and the media cover it so enthusiastically? The month-by-month accumulation of this difference adds up as seen in the graph of annual data.These graphs show that the introduction of the approach by Karl et al (2015) into NOAA's global temperature analysis has resulted in a larger rate of warming using what Dr Bates suggests in the Mail on Sunday is a dubious protocol.The changes made by Karl were influential but small and temporary and are turning out to be irrelevant anyway because of what the data is actually doing. Should anyone still think that the higher trends that include the years 2015 and 2016 are due to long-term global warming take a look at HadCRUT4, Fig 3, where you can see the deviation caused by the recent strong El Nino. Is there "no detectable sign of a "hiatus" or "pause" through to the present?" One can see the decline to 2014 temperatures and the return of the "pause." Some argue that HadCRUT4 has a coverage bias (the poles) that affects conclusions that can be drawn from it. However, the recent comments made about the Karl 2015 paper, and the subsequent changes made to the NOAA temperature dataset, have emphasised how consistent are the various global temperature datasets, and the graph above shows that HadCRUT4 is very similar to NOAA. Indeed the return of the "pause" is evident in NOAA data as well.Others are far more misleading. As we have posted many times before they will calculate a temperature gradient using a timespan that includes the recent very strong El Nino at its end. In this way they confuse a short-term intense temperature spike, known as weather, with long-term climatic change. This is comparing apples and oranges. Put the El Nino at the end of a timespan and the calculated gradient due to climatic change will be biased upwards by a temporary effect. This is highly misleading, and is akin to the bad judgement that lead some to start their analysis of trends in temperature data at the height of the 1998 El Nino.This latest development is one of scientific conduct and integrity. The global temperature datasets are among the most important datasets in the world. Billions of pounds rest on them. Politics and policies, jobs and industries, and the prosperity of billions are derived from them. What the Mail on Sunday's story, and its ongoing aftermath shows, is that what some scientists consider to be adequate stewardship of this data is not the case.It is not good enough to say that one believes the researchers would probably have released their computer code if they were asked for it, or to say the data was archived because it was available on an ftp site, or that procedures put in place were bypassed because they were inconvenient. In the Mail on Sunday article it is reported that some of the younger scientists at NOAA didn't even know that data had to be archived.Software runs our lives. The software that runs financial institutions is not perfect, but it is tested to industrial standards and protocols. So is the software used by health services and by engineers. I suspect that more people are involved in the production, testing and documentation of the software for a supercar than in some of the departments than produce these climate critical databases.Not long ago a professor of climate science at a British university involved in compiling such a climate database said he wouldn't know how to insert data into an Excel spreadsheet (Excel has been around for 24 years). Climategate revealed just how ad hoc, modified and ancient are some of the programs being used.Feedback: david.whitehouse@netzerowatch.com