CRU Email Hack - what do we know?

I happened to be online when The CRU email hack story broke and immediately downloaded a copy of the zip-file from a Russian server, all 63mb of it. Part of Friday was spent poking through the Mail folder, finding tantalising snippets, and being frustrated by the lack of context that made it difficult to interpret some of them.Then I had to go away and it was difficult to keep in touch with the developing story, so on Monday I started a long catch-up process that is hardly completed now.

The following are just a few notes. For a anyone who is still unaware of what is in these emails which have become so controversial, there is no better place to look than on Bishop Hill’s excellent blog here .

When unzipped, the file which mysteriously turned up in Russia amounts to nearly 170mb. Inside FOI2009.zip is a folder named FOIA containing two subdirectories: Mail and Documents.  Freedom of information seemed to have been very much on someones mind, either the person who named the main download file or Phil Jones of CRU who named the main folder within it or perhaps both. If Jones did give the main folder its name then I wonder why he call it FOIA (Freedom of Informaton Act) when a great deal of the contents does not refer to this piece of legislation? Certainly there is much material here that it would have been most embarrassing to have to release under the Act. Is it possible that Jones kept sensitive – perhaps even incriminating  – emails carefully in one place so that deleting a single folder would purge his computer of potentially dangerous material if danger loomed? This seems unlikely, as he would be aware that backups are likely to exist elsewhere beyond his reach.

The sizes of the two sub-folders within the FOIA folder are 9.7mb and 158mb.  The one called  ‘Mail’ contains 1073 files, but this does not represent the number of emails. Many contain strings of emails received and sent as well as copies of emails that are being referred to. So there are many more than 1073 emails in total.

I do not know how many words are involved, but assuming an average of a hundred words per email – and most are longer than that – that makes over a hundred thousand words. This is a vast amount of verbiage and most of it is ‘insider-speak’; exchanges between fellow professionals who know each other and the subject matter that is being discussed very well. Much of it is likely to be opaque to even a well-informed outsider. The task of making a detailed analysis of just the material in the Mail  folder is enormous and, even if this is split up between various people with the necessary expertise, it is likely to take weeks or months. In the meantime, what is emerging at the moment are a few obviously sensational quotes. Only a systematic analysis will fully reveal what is there.

The Documents folder has eleven sub-folders, most of them hierarchical, and the largest one, named Code, is over 30mb. That is an enormous amount of code. Already there are estimates on the net that suggest that it could run to millions of lines.

There are also over sixty files in the Document  folder, many of them large *.pdf and *.doc files some dealing with matters such as funding  and  advice on how to indoctrinate the public about climate change.  This includes a DEFRA publication that seems to owe a great deal to the infamous ‘Warm Words’, in which DEFRA explains that the ‘facts’ of climate science are unlikely to convince the public, but myths will be more far more effective.

So far, attention has focused on the Email folder, for obvious reasons. It’s not difficult to find quotations there that sound sensational. But most these are inevitably fragmentary statements with little context. There can be no doubt that Professor Phil Jones describing the death of a leading sceptic as ‘cheering’ is creepy. References to very obvious efforts to subvert the Freedom of Information Act in order to prevent sceptics seeing raw data are certainly suspect. Discussion of how to prevent inconvenient science reaching the peer reviewed journals, or being included in IPCC reports, is unprofessional and points to an unscrupulous culture where it is acceptable to use any means to maintain the illusion of consensus.

This suggests that the authors of the emails fear any challenge to their own work, even from people outside their own discipline who they are prepared to brand as nutcases, politically motivated idiots, troublemakers or merely incompetents.  This reflects badly on their confidence that  the work they have published is robust. Insulting references to anyone who does not agree with them reveals a mindset that one would not expect to encounter among senior scientists who are world leaders in their profession. Such people are usually self-confident adults.  Why do they not ignore criticisms if they are so obviously flawed? It is not convincing to claim that if they do so people will be misled, yet this is the justification that they share with each other. It would seem that there is much more than that at stake.

The speed with which CRU launched a damage limitation exercise when the news broke was astonishing and bears all the hallmark of very slick PR consultants. Almost as soon as the story broke, certain ‘dog whistle’ terms appeared in every statement from CRU’s defenders. So we were told that private emails had been stolen by a hacker who was part of a campaign by sceptics to sabotage the Copenhagen Summit. In spite of the devastatingly embarrassing content of the emails, it was immediately possible to spin CRU as the injured party, innocent victim of a vile conspiracy by extremists who want to destroy the planet.

A closer look at this raises a few problems. The emails were written by people employed in climate research by a publicly funded institution and concern their work. They were not, in any sense, private correspondence. It is assumed that their release was the work of a hacker, but it is equally possible that a whistle-blower within CRU leaked the files, perhaps because they were concerned about what was happening there. Such a person may not even have been sceptical about climate change, but deeply concerned about a culture that they considered to be unacceptable among scientists under any circumstance. And we all know what to think about dastardly saboteurs even if, without knowing who put that file on the Russian server or why they did it, there is no evidence that the Copenhagen Summit had anything to do with the timing.

It would be very interesting to know how hard CRU are trying to track down the culprit. From their point of view speculation about an evil hacker is far more convenient than discovering an honourable whistle-blower who has exposed malpractice. And what will CRU do if the culprit is found? Prosecution might well lead to a plea in mitigation that releasing the file was in the public interest. In such a case then the emails, and some of the people who wrote them, could be subject to forensic examination in a court of law. Even on the basis of a summary consideration of what some of those emails say, that would hardly be welcome to either their authors or to the institution that employs them.

Which raises another important point. Many of the correspondents are in other countries, particularly the USA. The press there are already talking about pressure for a Congressional inquiry, and if this were to happen it would be difficult to resist similar demands in the UK. Now representatives of CRU are saying that they would welcome an inquiry, and one can hardly blame them given the fix they are in. The fifth inquiry into the Iraq war has just started because the four previous ones were so effectively stage-managed that they failed to reach conclusions that, on the face of the evidence, seemed inevitable. What a great way to kick the whole affair into the long grass, perhaps forever.

Let’s end with a quote from Professor Phil Jones, apparently writing to John Christy on Tuesday, 5^th July 2005:

As you  know, I’m not political. If anything, I would like to see the climate change happen, so the science could be proved right, regardless of the consequences. This  isn’t being political, it is being selfish.

[Email 1120593115.txt  to John Christy20/11/2009]

Can you imagine an epidemiologist who has foretold a catastrophic Swine Flue pandemic saying the same kind of thing and keeping his job?