Climategate

Thoughts on the Coffs Harbour floods

By Rick Jelliffe
November 26, 2009 | Comments: 114

I am perversely enjoying Climategate: the conclusions that some people are reaching, with seemingly no actual evidence, that their opponents fudged their data looks like high farce.

Here is the quote from Professor Jones that is getting all the attention, from 1999:

I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (i.e., from 1981 onwards) and from 1961 for Keith's to hide the decline.

I don't know what US idiom is, but I do have an idea of what academic British idiom is, at my father's knee (who is still teaching at 82, my father, not his knee): here a trick being the art or knack of doing something skillfully or a clever or ingenious device or expedient; adroit technique. Not a deception.

But here is what our local climate change deniers were lapping up in a newspaper:

UNTIL last Friday, Wall Street's scammer Bernard Madoff was considered the biggest fraud in world history, having taken his greedy clients for an estimated $US64.8 billion.

Madoff's fraud has been put in the shade however by the extraordinary web of scandal which has been revealed by leaked emails from the Hadley Centre (University of East Anglia's Climate Research Unit) over the last few days, which certainly appear to demonstrate that a group of the most prominent scientists advising the United Nations on global warming have systematically manipulated data to support their argument that global warming is both genuine and caused by humans.

That's right: the biggest fraud in human history!

And here is what that Mike says about it, in 2009:

Mann said the "trick" Jones referred to was placing a chart of proxy temperature records, which ended in 1980, next to a line showing the temperature record collected by instruments from that time onward. "It's hardly anything you would call a trick," Mann said, adding that both charts were differentiated and clearly marked.

Yes, you got it: the 'adding' is not altering numbers, but adding a different graphic to the same page. It would be useful to actually see the page in question: was it actually ever even published or used anywhere?

UPDATE: It turns out the graph was published, though not in an academic article, but as cover art for a UN WHO document WMO Statement on the Status of the Global Climate in 1999.

Now that I have looked at it, I have modified my opinion a little.

I still think it is correct for that graph to use the real data rather than the proxies: it is a good graph. However the splice should have been called out graphically or in the text.

The rather mild conclusion made about the graph in the text, that 'against the background of the millenium as a whole, the 20th century was unusually warm' is true even if the spike above the normal is removed, and whether or not the segment of real data is distinguished graphically. So no use was made of the graphic relating to the splice. It is certainly not fraud let alone the biggest fraud in human history.

(Aside) However, if that is the graphic in question, I don't understandProfessor Mann's comment about it:

Mann said the "trick" Jones referred to was placing a chart of proxy temperature records, which ended in 1980, next to a line showing the temperature record collected by instruments from that time onward. "It's hardly anything you would call a trick," Mann said, adding that both charts were differentiated and clearly marked.


My Little Town

When I was a boy in the 1960s, one year my home town Coffs Harbour had a big flood: we walked over to the river to wonder, and it was a couple of feet (we still used feet then) higher than normal. And then in about 1996 we had a much bigger flood, a once-in-a-hundred years flood. Where I used to live was under a foot of water, and all the shops in the High Street.

This year (2009) Coffs has had 3 of these once-in-a-hundred years floods. Three. Plus two other floods. It is much wetter: on one day earlier in the month we got in one day about three times the average monthly November rainfall. My parents regularly get cut off, though they live on hill now.

We know so much more about the weather now: we know about the Southern Oscillation, La Niña, El Niño, interdecadal Pacific oscillation, and the Indian Ocean Dipole. But I have read no suggestion that these ultimately explain the changed climate in my home town in recent times.

In Australia, weather is a topic of much interest. We always say "Oh it has never been this hot/cold/wet/dry" but when we go back over the records we usually find it has been. (In Sydney we have six seasons rather than four, so some of the confusion comes from category error too, it must be admitted.) Our newspapers get disappointed when we don't break records, like some ghastly Olympics. We had the hottest September for 150 years too, by the way.

Watching our Senate's Question Time yesterday, our Minister for Climate Change, Senator Wong, was asked whether any water could be spared to rescue the estuary of the Murray River (one of the largest in the world). The Minister replied that the water situation was so bad, due to bad rain and evaporation, that the only way to save the river would be to deny people in cities and towns their drinking water. This looks like a fairly major environmental problem in plain sight.

Coffs Harbour has another claim to fame. It is where the Great Barrier Reef starts. However, the amounts of CO2 now in the water is causing some bleaching already, though it has not reached the levels it has in other places, where it prevent shell formation (a problem for other marine animals with shells too). In the last couple of months we had one famous marine biologist say (to the Royal Society) how we need to prevent human activity that would increase ocean acidification, while elsewhere another biologist saying the link to human activity had not been proven (to his satisfaction) but did so in the context that climate change was real. It seems pretty hard to find any scientists who deny recent climate change: though it is obviously a complex issue.

Out my window

So my problem with the climate-change deniers, is that when I look out my window, what I see fits in with the claims of the climate change scientists more than with the claims of the climate change sceptics. A freak weather event here or there is one thing: in Sydney we had our amazing dust storm recently, but unless it is repeated, I would consider it just an outlier. But when you get repeated occurrences of what had previously been outliers, such as my hometown's floods and reefs, it seems prudent to want to mitigate whatever contribution humans are making to it. [Update: Perhaps I wrote too soon about 'freak' weather events; we had another, smaller, dust storm on the Sunday after writing this.]

In fact, when look out my window, I see across Sydney Harbour, our oldest weather observatory, Observatory Hill. They have their readings since 1859 online. On that page, if you select the 30 year temperature averages, and compare how the red line has moved in comparison to the green, you can see that the recent average maxima seemed to have increased by three degrees or so compared to the 1800s.

The principle I see here is a basic one of conservatism: the rate of change must not greatly exceed society's (and the economy's) capacity to change (which is not to say that societies and economies do not sometimes have the capacity to change dramatically and fast!) I see conservatives apply this principle against action on climate change (protect our jobs!), but I think it is just as applicable in favour of mitigating actions.

I can understand that the extent of the impact of human activity can reasonably be debated; but I find it difficult to understand those who first insist that there can be no human impact, and then go from there to deny there is any climate change at all, or at least that there needs to be no real human response to climate change. Whether Rome is burning a little or a lot, and whether a person caused it or the Gods, and whether it will eventually burn anyway, we should at least stop fiddling and get out our buckets.

Update

I recommend Skeptical Science to readers. The site has a page on Climate gate and makes the comment:
when you look at Phil Jone's email in the context of the science discussed, it is not the schemings of a climate conspiracy but technical discussions of data handling techniques available in the peer reviewed literature.

In the skeptic blogosphere, there is a disproportionate preoccupation with one small aspect of climate science - proxy record reconstructions of past climate (or even worse, ad hominem attacks on the scientists who perform these proxy reconstructions). This serves to distract from the physical realities currently being observed. Humans are raising CO2 levels. We're observing an enhanced greenhouse effect. The planet is still accumulating heat. What are the consequences of our climate's energy imbalance? Sea levels rise is accelerating. Greenland ice loss is accelerating. Arctic ice loss is accelerating. Globally, glacier ice loss is accelerating. Antarctic ice loss is accelerating.

When you read through the many global warming skeptic arguments, a pattern emerges. Each skeptic argument misleads by focusing on one small piece of the puzzle while ignoring the broader picture. To focus on a few suggestive emails while ignoring the wealth of empirical evidence for manmade global warming is yet another repeat of this tactic.

Update 2

In the comments below, an indefatigable reader raises the issue of the adjustments of the CRU datasets. It is not very simple to get it all in one place: the HadCRUT3 anomoly dataset (2006-sh) is based on the CRUTEM3 dataset (for land entries) which in turn is based on multiple other datasets, and also updated from national meteorology organizations, primarily updating the previous version data which produced HadCRUT2 (2003-ish), which in turn was an update of the data that produced HadCRUT (1994-ish).

It is substantially similar to the independent GHCN datasets to the extent that they seem to get conflated or confused: close enough that the GHCN dataset is used to proxy the CRU data by partisans on both sides. (The extent of their similarity, see this entry by CRU critic Steve MacIntyre: note that under UK FOAI there is no necessity to provide information already available, as I understand it.)

Readers may be interested that the GHCN datasets are available here, with the adjustments applied in a nice adjustments file documented here (and some reading code here), and with links to explanatory papers here. (I am not sure the relationship between the GHCN adjustments and the CRU adjustments.)

For the HadCRUT3 dataset of temperature anomolies, the dataset can be downloaded here, and the explanatory paper is here. It is based on the CRUTEM3 dataset available here.

The missing metadata is in residual data in HadCRUT3 that comes from the original HadCRUT: according to the HarCRUT3 paper. It's approach is (as I understand it) to treat these adjustments, whether documented or not documented, statistically by estimating the "adjustment uncertainty" which feeds into the final results, when calculating certainty.

In other words, the HadCRUT3 results already (claim to) factor in (or out) the effects of the kinds of adjustments made: but these don't show up in the averages or the anomolies but in the uncertainty estimates.

By the way, for an update on the most recent science for the pro-climate change side, see The Copenhagen Diagnosis prepared here in Sydney at UNSW. (It is intended as a stopgap since the next IPCC report is not expected for 4 years.)

Update: April 2010

The British House of Commons equiry has found:

  • There was no evidence to challenge the "scientific consensus" that global warming is induced by human activities.
  • The balance of evidence "patently" failed to support the view that the phrases "trick" and "hide the decline" used by Jones in one email were part of a conspiracy to hide evidence that did not support his view.
  • On peer review, "the evidence we have seen does not suggest that Jones was trying to subvert the peer review process" and academics should not be criticised for "informal comments" on papers, MPs said.
  • On FOI The MPs expressed regret that the UK's deputy information commissioner had made a statement saying, in their words, that "at least some of the requested information should have been disclosed" without his office having conducted a formal investigation. However, they agreed that there was a prima facie case for the university to answer

The view in the press release of the MPs matches pretty much my conclusion, as an interested outsider:

The focus on Professor Jones and CRU has been largely misplaced. On the accusations relating to Professor Jones's refusal to share raw data and computer codes, the Committee considers that his actions were in line with common practice in the climate science community but that those practices need to change.

Fred Pearce's comments here.


You might also be interested in:

114 Comments

Analysis of the information that has been uncovered in the 'climategate' incident is still ongoing.

Fact is, if we are going to get the 'buckets' out, we better be sure that we're tackling the right problem.

The anthropogenic global warming theory is presented as a scientific consensus; than why making it so hard for people to get hold of the data & code?
If one things appears to become clear from 'climategate' is that there is considerable evidence that indicates that attempts were made by people(scientist) to actively destroy data & code, or otherwise absolutely prevent them being delivered to (the wrong) people who want to reproduce and test the theories.

You may enjoy this - I am worried about this.

There is about 30 years of precise temperature data, the other thousand years are minor/major 'reconstructions' (read: math formulas).
And the predictions made are computer models.
I'm a mathematician/computer guy myself - if life depended on my math/computer programs, I would welcome every opportunity to reaffirm that 'the thing works'.

Nick: "Ongoing" Don't you mean that the claims are so-far just made up?

"Actively destroy data and code"? The only thing I have read is one comment to delete some emails.

You say it is "presented as" scientific consensus, as if this were not the case. A 2004 review of all papers published in refereed journals between 1993 and 2003 on 'climate change' found:

The 928 papers were divided into six categories: explicit endorsement of the consensus position, evaluation of impacts, mitigation proposals, methods, paleoclimate analysis, and rejection of the consensus position. Of all the papers, 75% fell into the first three categories, either explicitly or implicitly accepting the consensus view; 25% dealt with methods or paleoclimate, taking no position on current anthropogenic climate change. Remarkably, none of the papers disagreed with the consensus position.

(For a recent follow-up to criticism of that study, see this survey.)

I suppose that there could be an elaborate conspiracy that actually there are all these scientists working in the area who want to put out denying papers, but every peer-reviewed journal is denying them space. I suppose it is possible...

However, I find this page sobering.

If putting a graph on a page is the worst fraud in human history, I suppose eager debunkers will be able find even worse things when they look in more detail: perhaps someone used Helvetica, or a graph had the dependent variable on the X axis! ;-) It shows a mentality.

However, I do agree that it would be better to have the data and any code open source. But there has never been a requirement for science to be open source to be credible, has there? Merely that the method be described in enough detail to allow replication, I thought.

More seriously, would your proposal be that all scientific data on all human-relevant subjects must be be open and sharable by researchers? For example, pharmaceutical experiments. Before or after they have published their findings? It seems a reasonable view, but it would certainly change the imperative to publish. Perhaps there should be a patent-like system, where they have the rights to their research and experimental results for several years, then their rights lapse.

I certainly agree that there should be every opportunity to reaffirm that the thing works: though who is going to rerun all the independence studies is a little unclear to me.

But when I see these alarming stories in the popular press, and then look at what the basis seems to be, and find there seems to be this ludicrous disconnect, it is hard not to draw the conclusion that this is not a subject where evidence will change the deniers' minds: some of them at least seem guilty of what some of them accuse the other side of: ideology.

And that was my comment on conservatism. There is a plausible component of climate change that may be due to human activity. Now it may be that this is co-inciding with a natural cycle too, but we cannot change the cycles: we can however change our activity. Either way, we need to try to mitigate climate change, to the extent of our knowledge, so that the climate change does not exceed our societies' and economies' capacity to change with it.

Rick: thanks for the thoughtful response

The amount of mails & docs & code that have been uncovered spans a decade, it will take a while before everything will be analyzed.

I'm not certain what 'made-up claims' you refer too, and I'm sure a lot of people are running away with this claiming ridiculous things, but a few things seem to be certain:
-that the CRU tried it utmost best to avoid conforming to a FOIA request it is obliged by law to abide to; even sending mails around to ask people to destroy e-mails
-that people inside CRU avoid peer review (and scientific work) by people 'not in the loop'; so in stead of welcoming critical review as advantageous, they see it as a problem
-that the science of compiling historic temperature data, and models, apparently requires the removal temperature data that 'doesn't fit'
-that the people of the CRU work with an AGW-confirming bias

I'm sure new things will be uncovered in the coming weeks.

About your conspiracy theory; Myself I do not believe in a such a big AGW conspiracy; in the sense that I do not think that this is a centrally controlled conspiracy by some dark group with a hidden agenda.
But as the AGW theory has become more and more a political issue, and that governments and scientific organization are more and more relying on the correctness of the theory, I'm sure that a large 'resistance' has developed against doubting AGW. And hence a 'force' to 'comply'.

About scientific peer-review; If major political and financial decisions are being made based on a particular scientific theory, then everyone needs to be able to take a look a this. Even those who my not be deemed 'worthy' to review the theory. And this means full access to the data, code and other scientific information used to reach these conclusions.

Nick: Looking through this, it seems that the chain of events is roughly

• Willis Eschenbach, apparently an construction manager for the Taunovo Bay Resort in Fiji made a UK FOIA request in 2005 for the CRU data sets
• The CRU responds that 98% of the data is available at other sources, and for at least some of the other 2% "we have signed agreements with many NMSs not to pass on the raw station data"
• Eschenbach requests specific and detailed lists
• CRU says their sources are merged so they cannot trace back to this information. "In short, we simply don’t have what you are requesting."
• Academics at CRU discuss not cooperating with requests from Steven McIntyre's Climate Audit website, which Eschenbach writes for?
• 2009 datasets are pirated including personal emails.

One of the emails quoted, to show their mindset is this:

"Quite frankly, Tom, having spent nearly 10 months of my life addressing the serious scientific flaws in the Douglass et al. IJoC paper, I am unwilling to waste more of my time fulfilling the intrusive and frivolous requests of Steven McIntyre.

"I believe that our community should no longer tolerate the behavior of Mr. McIntyre and his cronies. McIntyre has no interest in improving our scientific understanding of the nature and causes of climate change. He has no interest in rational scientific discourse. He deals in the currency of threats and intimidation. We should be able to conduct our scientific research without constant fear of an “audit” by Steven McIntyre; without having to weigh every word we write in every email we send to our scientific colleagues.

...

"One of the problems is that I’m caught in a real Catch-22 situation. At present, I’m damned and publicly vilified because I refused to provide McIntyre with the data he requested. But had I acceded to McIntyre’s initial request for climate model data, I’m convinced (based on the past experiences of Mike Mann, Phil, and Gavin) that I would have spent years of my scientific career dealing with demands for further explanations, additional data, Fortran code, etc. (Phil has been complying with FOIA requests from McIntyre and his cronies for over two years). And if I ever denied a single request for further information, McIntyre would have rubbed his hands gleefully and written: “You see – he’s guilty as charged!” on his website.


And what do we see in Climategate? It looks like exactly that. Personal emails between colleagues being used for all sorts of ludicrous interpretations. (The UK Environmental Information Regulations exempt the disclosure of internal communications, by the way. Other exceptions are if the information does not exist, or if has some IP, or if the request is manifestly unreasonable. Having formed the opinion that the requests were part of a vexatious campaign, rightly or wrongly, it is not surprising that the scientists would jack up.)

As one scientist (Steven Schneider) purportedly writes:

"This continuing pattern of harassment, as Ben rightly puts it in my opinion, in the name of due diligence is in my view an attempt to create a fishing expedition to find minor glitches or unexplained bits of code–which exist in nearly all our kinds of complex work–and then assert that the entire result is thus suspect.

And some news out today: University of East Anglia is working to get copyright permissions for the 2% to release it too. But they state:

"The publication of a selection of stolen data is the latest example of a sustained and, in some instances, a vexatious campaign which may have been designed to distract from reasoned debate about the nature of the urgent action which world governments must consider to mitigate, and adapt to, climate change. We are committed to furthering this debate despite being faced with difficult circumstances related to a criminal breach of our security systems and our concern to protect colleagues from the more extreme behaviour of some who have responded in irrational and unpleasant ways to the publication of personal information.

And this statement from Prof. Jones:

"In the frenzy of the past few days, the most vital issue is being overshadowed: we face enormous challenges ahead if we are to continue to live on this planet.

"One has to wonder if it is a coincidence that this email correspondence has been stolen and published at this time. This may be a concerted attempt to put a question mark over the science of climate change in the run-up to the Copenhagen talks.

"That the world is warming is based on a range of sources: not only temperature records but other indicators such as sea level rise, glacier retreat and less Arctic sea ice.

"Our global temperature series tallies with those of other, completely independent, groups of scientists working for NASA and the National Climate Data Center in the United States, among others. Even if you were to ignore our findings, theirs show the same results. The facts speak for themselves; there is no need for anyone to manipulate them.

"We have been bombarded by Freedom of Information requests to release the temperature data that are provided to us by meteorological services around the world via a large network of weather stations. This information is not ours to give without the permission of the meteorological services involved. We have responded to these Freedom of Information requests appropriately and with the knowledge and guidance of the Information Commissioner.

"We have stated that we hope to gain permission from each of these services to publish their data in the future and we are in the process of doing so.

"My colleagues and I accept that some of the published emails do not read well. I regret any upset or confusion caused as a result. Some were clearly written in the heat of the moment, others use colloquialisms frequently used between close colleagues.

"We are, and have always been, scrupulous in ensuring that our science publications are robust and honest.

The CRU's statement includes this:

"There is excellent agreement on the course of temperature change since 1881 between the data set that we contribute to (HadCRUT3) and two other, independent analyses of worldwide temperature measurements. There are no statistically significant differences between the warming trends in the three series since the start of the 20th century. The three independent global temperature data series have been assembled by:

• "CRU and the Met Office Hadley Centre (HadCRUT3) in the UK.
• "The National Climatic Data Center (NCDC) of the National Oceanographic and Atmospheric Administration (NOAA) in Asheville, NC, USA.
• "The Goddard Institute of Space Studies (GISS), part of the National Aeronautic and Space Administration (NASA) in New York.

"The warming shown by the HadCRUT3 series between the averages of the two periods (1850-99 and 2001-2005) was 0.76±0.19°C, and this is corroborated by the other two data sets.


...or one could admit that the CRU debacle is a disgrace for climate science, and start to work on building a better community around the AGW debate:

http://climateprogress.org/2009/11/27/%C2%AD-climategate-judith-curry-open-letter-to-graduate-students-young-scientists-climate-research-hacked-cru-emails/

(I find the PR that is coming out of the CRU since the hack/release not very reassuring: giving the impression that adhering to a FOAI request is optional, yapping about how the others all are unscientific charlatans and defending your research by saying that it matches someone else's results, so it must be true )

Nick: And one could also admit that the stealing and publicizing of private emails is a disgrace, and the whole thing has become exactly the kind of beat-up the scientists knew would happen, and a game they refused to play.

When you say that they must adhere to the UK FOAI, what about the exemptions under s39 for environmental information? (It has its own act.) What about the exemptions for commercial interest and in confidence information (for the 2% with NDAs)? Or the exemption for information available by other means (for the 98%)? Or the exemption for vexatious (repeated) requests? It looks like CRU had plenty of wiggle room, from a cursory look at the Act at least.

I'll leave the yapping comment.

But I don't get the comment that you don't believe that research results can be defended by saying they match other results. Isn't that the basis of science?

If all the research says there is global climate change, then that is the conclusion that science supports, regardless of its truth value. But in any case, I don't think it was Jones' point: not only are the CRU results replicated by the two other datasets, but also "That the world is warming is based on a range of sources: not only temperature records but other indicators such as sea level rise, glacier retreat and less Arctic sea ice."

What I will concede, however (and this is what these emails show), is that the need for more education and public discussion in this area has long overtaken the resources of the climate scientists and put them in the position of having to choose between core work they consider critical and urgent for humanity (gathering data, working on their models, testing hypotheses, publishing their work, figuring out plausible scenarios) and trying to get a world full of amateur climate scientists up-to-speed.

How very Phil-Jones of you to refuse to discuss all the other e-mails that were leaked. Some of which are discussions as to how to subvert the peer-review process (which these men have been accused of doing in June 1996, read the old e-mail from Seitz), others of which discuss how to subvert the influence of journals that do not agree with them.

How very hide-the-decline of you to refuse to discuss the full context of that first quote of yours, which taken in true context, describes a way of hiding the divergence of their paleoclimate data (tree-ring-data) with thermometers. This divergence is well known in the literature, and has no theoretical explanation. The decline was hidden to make their case stronger.

Yes yes, how very CRU of you to be so selective.

Jeremy #2: Don't confuse blowing off steam in emails to colleagues with conspiracy.

For example, take the supposed attempt to "subvert peer-review" by keeping two articles considered bad science from being included in the IPCC report: actually they were referenced, according to this article in Nature.

Even the blog of a website that hosts the nicked data now backs away from claims of conspiracy:

The evidence of the e-mails themselves may not show a major scientific conspiracy but it does show evidence of malfeasance such as conspiring to delete data/e-mails when they are requested under FoI legislation and nobbling journal editors

Err, what evidence? We again have blowing off steam, talking about (feeling like) deleting emails, but where is the evidence that this was actually done, yet alone a conspiracy?

Presumably hacking in and nicking private emails and data, exposing them to the world knowing that the most lurid interpretations will be made, in attempts to destroy the careers of Jones and the other academics, is not a malfeasance!

The most distressing thing I've heard about the CRU is that much of the original data of global temperature measurements from the past 75 years was thrown out when they moved to a new facility. What remains is the data that was "corrected" by the researchers -- but there's no way to know what they changed or why they changed it.

In other words, there's no way for anyone to reproduce the science because we only have the cooked data remaining.

Does this bother you Rich?

Phil: Yes, it would be much better if there was full tracing of all this. But we have to play with the hand we have been dealt.

The general approach of neutralizing data from stations where the numbers are known to be dominated by local factors (such as city warmth) seems reasonable. But it is one of the things being portrayed as fudging.

Lets say we are studying the effect of diet on male life expectancy. And half our sample population dies in a war at the age of twenty as soldiers. Of the remaining half, all the vegetarians live to 80 and all the meat eaters live to 90. If we don't remove the deaths due to war, we would say that vegetarianism contributes 5 years on average. But actual number we were looking for is 10 years. (This is not to say the 5 year figure is misleading, it may have some value to someone too.)

Removing or approximating or proxying data in this kind of situation is not bad science. It just means that they need to be careful to understand what the specific thing being measured by the numbers is, but they do anyway.

The question is not about playing with the hand that we are dealt; the question is whether or not there's a dirty dealer -- or perhaps the perception of a dirty dealer.

Given the volatility of this issue, the CRU should be bending over backwards to ensure that they are conducting business in as open and transparent a means as possible. From what I've seen, they're doing exactly the opposite. The deliberate destruction of data may tragically backfire -- giving countries exactly the excuse they want to not agree to CO2 emission caps. How much due diligence did they do before deciding to destroy that data? How hard did they work to find a home for those archives?

That's why I'm mystified why anyone on either side of this issue isn't outraged with the behavior of researchers at CRU. It appears to be unbelievably arrogant behavior.

I believe that you mis-characterized the debate currently happening in your blog entry.

Phil: For a start, your claim that data was destroyed seems to be wrong. CRU pre-processed their data to the point where it could be useful, but the raw data from the stations exists independently.

See this NYT for example.

I am not going to get all hot and bothered by fictitious conspiracies. Of course, examination will find flaws and infelicities. But data and archive storage used to be expensive (remember those days?), and only working data was kept.

What the anti-climate changers need to to do is to take the raw station data, massage it themselves in whatever ways they think would eliminate inappropriate data (and, yes, log all this scrupulously and have the funding to keep the records and the data), and then come up with credible alternative models that the general climate change scientific community can accept. Then compare those results with the results that other methodologies have given, and see what the consensus position.

This may seem to be a lot of work. It is: go for it! Perhaps the larger task would be redefining molecular science like the properties of CO2, which would seem to be necessary too.

The very article you reference notes the key issue: because they destroyed data it is problematic for someone to reproduce the science of the CRU.

I'm accusing nobody of a conspiracy; I am accusing them of doing bad science. It is unbelievably arrogant of this organization to have "value added" data in their research where there is not a clear audit of the steps they took to manipulate the data.

Someone blogging under the banner of ORA should understand this: openness and transparency.

You said: "What the anti-climate changers need to to do is to take the raw station data, massage it themselves in whatever ways they think would eliminate inappropriate data (and, yes, log all this scrupulously and have the funding to keep the records and the data), and then come up with credible alternative models that the general climate change scientific community can accept. "

Why didn't the CRU do that?

You seem to be questioning whether or not someone should do their science in such a way that someone else would be able to reproduce the results from the raw data.

Phil: Why didn't the CRU do that? Don't they say cost and lack of funding in the 80s as one of the reasons?

(And again, I don't think it is right to call failure to log the individual changes a "destruction of data". CRU has given the principles they use, haven't they? But I am not denying that it would be better have exact records on everything: I just don't think that it would make the slightest difference to some of the sceptics.)

I see Prof. Jones has now quit over this.

Correction: I should have written that Prof. Jones has stepped aside during the enquiry, not quit.

The CRU didn't make doing creating reproducible results -- scientific results -- a priority. Their work was contrary with the most basic expectations of the scientific method (from http://en.wikipedia.org/wiki/Scientific_method ):

"Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

In our sub-discussion you said, "I am not going to get all hot and bothered by fictitious conspiracies." Do you now understand that I am NOT talking about any conspiracies?

"I am not denying that it would be better have exact records on everything: I just don't think that it would make the slightest difference to some of the skeptics."

Agreed. Even if the scientists had been rigorous in their methodology, some would not have been swayed. That would not be skepticism; it would be an extreme point of view.

You should also note that some will be complacent about what these scientists did -- even after they realize that these scientists failed to follow the scientific method. That complacency would be another extreme point of view.

The far more interesting question is how the thinking public in the middle reacts when they realize what these scientists have done.

Phil: So when Jones refused to distribute the value-added data to outsiders and told people to start again with the raw data, wasn't he therefore (and perhaps paradoxically) acting scrupulously, since that micro-documented chain had been broken?

The value-added data values (eg adjusted for urban wrming) can all be identified without records: they are the ones that differ from the raw data. I would hope that there are papers detailing the rationale and method of the value-adding, enough that another team could apply the same methodology. I am not fussed that there are not individual records of changes, but there does need to be a description of the methodology enough to allow significant replication and criticism.

Personally, I'd like to see a sensitivity analysis of the effect of the value-adds on the global results, or with different weightings. Governments certainly should be spending up on this kind of thing, now that storage and supercomputers are cheap enough for it to be possible. What was feasible in the 1980s is not what is feasible now. If a sensitivity analysis showed that the changes made did not significantly alter the key results, then questions about pollution of the data would be moot.

The way that science is done is that you record the data and your methodologies to manipulate that data. You justify those manipulations -- and the original data -- scrupulously. Only by doing all those things is there any hope of getting reproducible results.

If you are doing physical science and you didn't go through those steps, you didn't do science. I am baffled why you're not outraged over the behavior of these scientists.

"Significant replication" is an oxymoron. It's like saying someone is "somewhat pregnant". Either the results are reproducible or they are not.

I have yet to see any of these scientists apologize for not doing scientific research.

I do not understand the atmosphere of complacency around what these scientists have done.

Phil: You are baffled why I am not outraged? Because emotional responses over technical discussions may help people's motivation but they do not help the rational discussion of a serious issue.

In fact, it is despicable to whip up emotional responses in an environment where people are being slandered, and I am glad that so many people on both sides do in fact resist the urge.

In a civilized difference of opinion, the cool heads on each side rope in the hot heads (there will always be some hot heads): venting in private is understandable and shouldn't make it into public action or debate (either by saying those things in public or by stealing/publicizing your opponent's private conversations.) I am not saying that anyone is perfect in this regard, of course.

I still haven't found any of the sensational claims made about Climategate to have any basis: no hiding, no deleting, no suppression of peer review, and so on. They were beat-ups, based on finding the most lurid interpretation of private emails between colleagues venting.

The other issue you mention is justifying manipulations: do you mean at the general level (which should indeed be mentioned in the scientific papers) or at the individual data item level? (From the Harry report, it seems the normalization vectors are all there, aren't they?)

But I don't want to deny that the CRU record-keeping and software engineering practices look substandard by what is now easily capable on *modern* systems: what may be justifiable disorganization for processes that are run once every few years with no public policy impact is not justifiable now.

I think you are wanting that the data, process and its intermediate results should be documented to such an extent that it could be replicable *with minimal effort* by armchair scientists on their home PC. That would certainly be a good future CRU policy, now that climate change is on the public policy radar. (I see the Australian BOM, for example, has lists of its observation stations with 'high quality' data.)

At the moment, the replication can only be done with extreme difficulty (even by them, hence the Harry log, which I blog about elsewhere) and cost: but it seems that it can be done.

The thing that is/was needed is/was for climate sceptics to give CRU and these other organizations funding grants to employ some independent software engineers to get the CRU data and systems into good shape, the code and processes documented, debugged and open sourced, with funded websites. This can be done independently of any issue of copyright etc, and the value-added data would, since they presumably are not under the NDAs of the 2% of the raw data, be made available open source.

Did any climate change deniers ever offer to fund their data requests? Honey is sweeter than vinegar.

Getting involved like that would be ten thousands times more productive than getting outraged.

I am indeed baffled by your complacency. They weren't doing science. Since they failed to keep the original data, there's no way to reproduce their research -- to double-chedk it.

If we're going to make trillion-dollar-plus decisions, I want it based on science. Getting out the word that these guys didn't do science is something everyone should know. If you don't think so, I am indeed mystified.

If you were actually concerned about sensationalism, you should be saying some things about the sensationalist statements in "An Inconvenient Truth".

If you're concerned about sensationalism, look at this report:

http://pajamasmedia.com/blog/climategate-imminent-demise-of-glaciers-due-to-a-typo/2/

The "2035" figure that's been reported by the IPCC for the loss of Himalayan glaciers was misread by the IPCC authors. The correct year is 2350. But if you googling around, you'll find a huge number of news sites reported the first number but have failed to report the 350-year error.

How about a blog entry that tracks when media sites start fixing their reporting on that -- and how many never fix their incorrect sensationalism?

I've seen nowhere that the CRU has even admitted they have a problem with creating reproducible science. I wouldn't guess that they would change their practices now.

If you want to get educated on this issue, I recommend looking at Charlie Martin's blogging on the topic. Charlie is a bright guy -- lots of ORA books on his shelf:

http://pajamasmedia.com/blog/climategate-imminent-demise-of-glaciers-due-to-a-typo/2/

http://pajamasmedia.com/blog/pielke-sr-climategate-emails-just-a-small-sample-of-a-broad-issue-pjm-exclusive/

http://pajamasmedia.com/blog/climategate-computer-codes-are-the-real-story/

http://pajamasmedia.com/blog/climategate-violating-the-social-contract-of-science/

http://pajamasmedia.com/blog/global-warminggate-what-does-it-mean/

Phil: You keep on repeating this claim that they destroyed data or didn't keep original data. But the raw data was kept (and/or is available from the original sources.) Which "original data" do you mean?

What I would characterize as "They are sensationalizing too, don't look at us look at them" is a pretty weak kind of response. The people saying "Their claims need to be examined" should hardly be surprised if their claims get examined too.

From those Martin links comes the Peilke link “Climate Change: The Need to Consider Human Forcings Besides Greenhouse Gas. And what does is say? I don't think there is any comfort for climate-change sceptics:

"Hypothesis 1: Human influence on climate variability and change is of minimal importance, and natural causes dominate climate variations and changes on all time scales. In coming decades, the human influence will continue to be minimal. ... "Because hypothesis 1 is not well supported, our scientific view is that human impacts do play a significant role within the climate system.

And from an interview reprinted on his blog

"Even if their (CRU) data analysis is excluded, it does not alter the findings that were reported in the 2007 I.P.C.C. report with respect to the surface temperature trends, since these other analyses provide a redundant check of their analyses over the last century.
...
"Humans are altering the climate in diverse ways, a variety of human climate forcings are significant, and the effects of these forcings need to be responded to, even if the climate did not warm.

Rick, Phil pointed out this exchange. I think you're mistaking a number of points, I presume out of ignorance (reading the sources is always good.) There are three major issues involved in the revelations from the Climategate files:

* first, that the CRU clique has been actively suborning the peer review process, preventing dissenting papers from being published, stepping outside of normal protocols to change IPCC editors, and violating the confidentiality and anonymity of peer review in order to ensure that only the bien pensant receive dissenting papers, thereby ensuring that they are "correctly" reviewed. This is discussed in a number of the emails. Also discussed in the emails is a number of situations in which they referred to members of the press — such as Andy Revkin and Richard Black — who are considered reliable and who are introduced into the inner conversations. This has, at this point, been extensively documented in various places, including the blogs of Pielke pere et fils. After the release, we also saw Michael Schlesing threatening Andy Revkin with the Big Cutoff if he wasn't careful about what he reported.

This, in itself, is the sort of misconduct that would normally receive a vigorous slap on both wrists, but it is misconduct.

* second, that the data was repeatedly manipulated in order to "tell the right story", not the story the data told. This has been coming out slowly, with this recent post by Steve McIntyre detailing one such series of emails. That sort of thing, scientifically, is very bad form.

* third, there has been an overt conspiracy to prevent data from being released under the Freedom of Information laws of the US and UK. This may not seem like much of an issue to you, but out in the rest of the world, it's multiple criminal offenses.

It's also proven to be the crack in the wall; since then, there are multiple studies, eg, the one by Richard Keen at CU, showing that the data has been "homogenized" mysteriously, introducing corrections that account for most of the observed warming; a clear demonstration that some of Pielke Jr's results were misrepresented in the IPCC report; clear evidence that the IPCC itself misrepresented the "2035" data, presenting it without citation, and asserting as fact something the named study didn't actually say; and there's been some flat out falsehoods passed around recently trying to cover up.

None of this means that "global warming is a hoax" — it's well-known there has been warming. The notion that some of it is anthropogenic still seems reasonably strong, and that CO2 is one among many anthropogenic forcings seems very likely.

On the other hand, at this point the validity of any studies based on the CRU data must be questioned, the evidence of at least some significant misconduct seems pretty unequivocal, the likelihood of actual criminality seems pretty significant.

Continuing to try to say "nothing to see here, move along" looks more and more like denial.

Charlie: A lengthier response is below.

The interesting thing in the composite graphs with the Briffa tree ring lines, such as in the MacIntyre blog entry you mention, is not the lines that are truncated or spliced (when real measurements are available) and should have been called out better, but the pre-1960 tree-ring line and the other lines.

Jeremy: Thank you for including me in your fantasy.

(If you want to bring something up some specific issue, like whatever Seitz reference you mean, please give the URL so that I and readers can respond, or enough text that we can Google.)

"Hiding with thermometers?" Frankly, that is the nuttiest comment I have ever had in this blog, and there have been lots of nutty comments over the years. Are you seriously saying estimates known to be factually wrong are just as important as exact, direct current measurements? Proxy data is used when direct data is not available. That if the tree rings say the most recent summer was around 40 degrees and the thermometer said it was 41 degrees, you want us to believe that that the 40 degree number somehow makes the 41 degree number misleading? As I said, it is farcical.

The point that there is no need to use proxy data (estimates of temperatures from tree rings) for recent times, where we have the direct temperature measurements.

Sure: that anomaly is of interest for tree-ring scientists to puzzle through, and for what it might improve estimates of older tree-rings but it does not alter the current real numbers. When the weather forecast says "clear" and it is raining outside, no-one can credibly says "oh, by saying it is raining now, you are disguising that the forecast says it is actually fine."

On the issue of peer review and subversion, we do in fact have a precedent: the fake science and campaigns that showed that nicotine was not addictive and that cigarettes do not cause cancer.

Now the people who defended tabacco were not all self-deluded, bad, mad, dangerous scum or people happy to make a career out of lingering painful deaths, though some were (and are): some people were just misleadable and were mislead. Some were out of their depth. Some found the prospect too unpleasant. For some, it did not match what they experienced. For some, they did not have access to information in a form that was persuasive. Some people have no broader horizon than how to survive in the coming year.

For some, the issue raised so many alarm bells (right to persuit of happiness, limits to science, government interference, inability to allow that someone is an expert merely on the say-so of others, and so on) that it became impossible to see the wood for the trees. And this is what is happening now, it seems to me, and what Prof. Jones and the others were complaining about.

But it raised so many alarm bells because the commercial interests campaigned to trigger them off: the Phillip Morris material online showed how easy it is to do.

There is no obligation for a serious scientist to spend an inordinate amount of time trying to convince nuts that their bogus theories and quibbles are wrong. There has to be a measure of engagement of course. But the talk of science operating by free criticism misses the point: elite scientists need to be mutually catalyzed primarily by scientists of similar calibre.

The other little comment I would make is that I read comments about the IPCC report and peer review. But the IPCC report was not a peer-review journal, but a report intended to convey the consensus view of the climate scientists: to integrate and synthesize information to give what is currently known about climate change. It should be the result of review, not a mechanism of peer-review: a report not a journal.

If Jones wasn't trying to 'trick' anyone then why did he HIDE THE DECLINE?

Joseph: I presume Jones didn't want the recent anomaly in the tree rings to confuse people, since the direct measurements were available for that period. So they were not distracted by a side issue.

And what do we see: conspiratorial obsession with this side-issue.

The graphs have multiple lines from different studies. Each one has to be somewhere on the Z-axis. It makes sense to me that you send the proxies to the back, or even stop showing the proxy data once you have direct measurements available. But the different data certainly needs to be graphically distinct or labelled or commented properly.

"Phil: You keep on repeating this claim that they destroyed data or didn't keep original data. But the raw data was kept (and/or is available from the original sources.) Which "original data" do you mean?"

I mean that they didn't do reproducible science. I mean that they failed to use the scientific method -- as described by the third paragraph of the Wikipedia entry http://en.wikipedia.org/wiki/Scientific_method . I've already included that paragraph in this discussion once.

There is no way for anyone to precisely reproduce their science; they destroyed their intermediate data.

" 'They are sensationalizing too, don't look at us look at them' "

Who are you quoting? I never ever said those words.

What I said was that I was baffled by your complacency in Climategate and that I want trillion-dollar-plus decisions based on science.

If you want to find sensationalism, look at "An Inconvenient Truth". Or look at current TV commercials showing polar bears falling from the sky -- and dying -- on city streets.

Look at the number of media outlets that made the 2035 error -- and the small number of media outlets that have corrected themselves on that.

"The people saying "Their claims need to be examined" should hardly be surprised if their claims get examined too."

Indeed. If you make a straw man "quote" from me, I will point it out.

What comments do you have about the 2035 typo?

Phil: I have rephrase that sentence, I was not intending make up a quote (how would that work, given that your text was immediately above, in any case?).

I don't think "They are sensationalizing too, don't look at us look at them" is a mischaracterization of what you wrote: "If you're concerned about sensationalism, look at this report". And "How about a blog entry that tracks when media sites start fixing their reporting on that -- and how many never fix their incorrect sensationalism?"

I am not sensationalizing.

Charlie is not sensationalizing.

That's why the contrived quotation is wrong.

Phil: You are a lot milder than many, and I appreciate that and don't doubt your good faith or integrity. And you may even turn out to be right in whole or part: time will tell! And I can understand if my collective use of "you" might be irksome, in that you may not want to be lumped with the nuts and oil/coal lobbyists just for sharing some of their concerns.

But you were sensationalizing: you repeated the loose claims about destruction of data and demanded that the appropriate response was to be outraged. Sorry, but demands for outrage evoke the opposite response in me, and it is not 'complacency.'

You are entitled to your opinion, but mine is not sensationalism.

These guys did science in a way that is not reproducible. There is no way to follow the chain of evidence and see the results they came up with, because they deliberately destroyed several steps in the chain. I'll repeat the pertinent from the wikipedia "scientific method" page now:

"Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

To risk shouting for a moment, it is a BASIC EXPECTATION of scientists to document, archive and share all data and methodology in research. If that's not done, it's not science. I don't know if there was a conspiracy in the background, but it certainly was irresponsible and grossly arrogant of those researchers to fail to meet those basic expectations of science.

Even if the researchers had met basic expectations in their methodology, some would not have been swayed. That would not be skepticism; it would be an extreme point of view.

Those who are complacent that the researchers failed to meet the basic expectations of science have a different extreme point of view. At this point in the discussion, it's pretty clear that you are complacent. Since you blog under the ORA banner, there's also a bit hypocrisy in your silence about CRU's failure to be open and transparent.

Rich, do you think there's something wrong with basic expectations enumerated in that Wikipedia article in order something to be science? I really would like an answer to that question. Thanks!

Phil: Could I just make clear that none of my blog entries are paid for or commissioned or edited by ORA. The only constraint from OReilly is that the blog be technical or (even better) technological rather than personal.

ORA editors would step in if there was some legal or other dispute, but otherwise they treat these blogs as a forum for authors (usually of books (typically theirs)): we are not corporate mouthpieces and ORA does not wish us to be.

The two things needed for more transparency are 1) specific funding and 2) fewer clowns making mischief. Show me the calls from inside the Climategate hubbuberie for more specific funding for CRU etc and for clowns to put down their cream pies, please. Outside those kind of practical steps, 'more transparency' is just a slogan.

The serious anti-climate-change sceptics should be pouncing on every daffy confumbulation (such as the latest 'smoking gun' I blog about elsewhere, where the values from a station 3000km away are used to question the Darwin values, just laughably inept and straw-grabbing), and every shot-gun rah rah cornball "Socialists are trying to steal my money", and every attempt to demonize individuals, since they do so much harm to the reasonable sceptics' cause IMHO.

By all means point out problems you see, some may indeed be right, but the anti-AGW side needs to get its house in order: they cannot win technical arguments by using obsolescent American political tactics of personal villification, sensationalism and making a hundred minor points into a major one.

As far as I can tell, the CRU failed to meet the basic expectations of the scientific method in their research. That is neither sensationalist nor political; it's simply looking at what they did and didn't do.

You failed to answer my question: is there something flawed in what the Wikipedia entry on the scientific method characterizes as the basic expectations of science?

Second question: do you honestly think the CRU actually met the basic expectations of science in their research? If so, please reconcile that opinion against these basic expectations:

"Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

I understand you are not on the staff of O'Reilly and Associates. I also understand that you are a strong personal advocate of openness and transparency, and I'm mystified by your apparent disconnect from those principles on this issue.

Phil: On the first question, I am not a philosopher of science, but I don't know that this "basic expectation" means that individual scientists must necessarily make data available to people they consider vexatious non-scientists as and when these people demand. (They may then be called to account for this, which is OK.) And I would be very surprised if it means that science is impossible where there are NDAs on source data (which are licensable from the source party.)

One the second question, it is obvious that this is a matter of degree.

The CRU has indeed "documented, archived and shared all data and methodology" to some extent: that there are the files to steal in the first place, the academic papers, and the discussions of NDAs etc shows this. But it is not the extent that other people want. And budget considerations has certainly lead to important programs and data being in a poor state of repair and demonstrability.

Now that this data is so important to decision-making, it is not a good enough state of affairs.

The CRU and other bodies with these large datasets, and indeed perhaps some new respectable institutions (not paid by Big Oil or Big Energy or the Big Miners or the free-lance ex-lobbyists of Big Tobacco shopping for a new sponsor) but perhaps representing some other bloc should be given specific funding to bring all their sets up to scratch, both in data and metadata.

Openness is not binary: open or hidden. The more society wants the openness of data, metadata and programs to extend from scientific peers to the populace, the more society will have to fork out for it.

The other way that this is a matter of degrees is that on the macro scale, another scientist can indeed take the same raw data sources CRU used (if they license them) and repeat the published CRU methodology (and whatever clever tricks --oops I mean techniques-- might be found in the source code) and get results that should concur statistically.

It is on the micro level, when talking of individual items of data, and exact replication of every step, that the documentation seems to run out, isn't it? But that can be fixed in the future by open sourcing, surely.

(By the way, it would seem to me that all science has to leave out some kinds of details that may turn out to be important (which brand of glass beaker was used, what was the humidity) in order to replicate some results, for any cutting edge research. The things documented and the data retained represent a theory about what may conceivably be salient for reproducing the results. That theory may be just as wrong as the theory being explored by the experiment.)

You don't need to be a science philosopher to answer my first question. All you have to do is understand how to apply the scientific method to physical sciences. And you don't even have to worry about what anyone demanded or didn't demand of the researchers of the CRU.

What I have pointed out is that the CRU researchers failed to perform their physical science research in a way that is reproducible. Neither proponents nor skeptics of their research caused them to do that; they failed all on their own.

If you think the CRU should have been given A Pass for that failure to meet the basic requirements of the scientific method, please explain why. If you can't, please say so.

"The CRU has indeed 'documented, archived and shared all data and methodology' to some extent."

"Shared all data and methodology -- to some extent" is a meaningless claim. Their data is either documented, archived, and available in a way that someone can reproduce their science, or it is not. Black or white.

This has nothing to do with anyone's point of view or their politics. And it is not sensationalist to claim they weren't doing science; it is a fact.

I am still mystified by your complacency on that fundamental point.

You don't need to be a science philosopher to answer my first question. All you have to do is understand how to apply the scientific method to physical sciences. And you don't even have to worry about what anyone demanded or didn't demand of the researchers of the CRU.

What I have pointed out is that the CRU researchers failed to perform their physical science research in a way that is reproducible. Neither proponents nor skeptics of their research caused them to do that; they failed all on their own.

If you think the CRU should have been given A Pass for that failure to meet the basic requirements of the scientific method, please explain why. If you can't, please say so.

"The CRU has indeed 'documented, archived and shared all data and methodology' to some extent."

"Shared all data and methodology -- to some extent" is a meaningless claim. Their data is either documented, archived, and available in a way that someone can reproduce their science, or it is not. Black or white.

This has nothing to do with anyone's point of view or their politics. And it is not sensationalist to claim they weren't doing science; it is a fact.

I am still mystified by your complacency on that fundamental point.

Phil: What comments do I have about the 2035 typo?

Well, that typos need to be fixed. I bet there are more where that came from too.

Does it really change the big picture? Don't tell me that the deniers' case is that really the whole AGW thing is at heart an accumulation of tens of thousands of typos by tens of thousands of scientists around the world?

For a cool picture from NASA of Himalayan glaciers, see
these

I predict the majority of the reports with the 300 year error will not be fixed. In particular, I doubt that CNN will correct their "science" story.

Would you like to track and blog about this for ORA?

You'll see all sorts of errors in "An Inconvenient Truth" also.

I watch little television, but I also saw the ad showing polar bears falling and dying in a city. More sensationalism.

If the actual truth were sufficient, why are we getting all of this sensationalist tripe? Why not just the cold facts?

Many are indeed in denial about the bad science of the CRU. They have paid little attention to what has been unearthed -- there is nothing that would shake their belief. Their belief seems to be based in something other than science.

That's the irony of your point of view: ORA is from a pedigree of openness and transparency. As the whistle-blower e-mails indicate, the CRU scientists had neither in their process. I would think that would be disturbing to someone who advocated transparency.

Perhaps if this were any issue other than global warming ....

Phil: By the way, I perhaps should say that I think the 2035 error looks like a good catch. But presumably the editorial policies that don't bother to correct old stories will work equally for both sides.

For example, take Eric Raymond's claims to have found evidence of manipulation of data in source code this week. And yet it turns out that the code to use this is commented out (i.e. it was more likely just some kind of exploration or fossil, anything else is mere speculation). Once these loose ideas are out there, it is impossible for anyone to unloose them.

You are right: an excellent way to judge the credibility of a source is to see how they react to their mistakes. The IPCC was responsible for releasing the incorrect 2035 year. Have they corrected that error? I haven't seen it.

The IPCC has a particular responsibility to be accurate, because global policy decisions are being made based on the output of that UN organization. And they are definitely being scrutinized: if they are not quick to correct their errors, it calls into question everything they claim.

What Eric Raymond blog entry are you referring to? Did you look through that comments of that entry and the rest of the blog to see if he noted the comments?

How is it that you know whether or not those comments were in place when the CRU used that code? Or even worse: perhaps the code was commented out when processing some data and not commented out with other data. Nobody on the outside knows. Consider: maybe even the CRU researchers don't know.

This goes to the heart of the issue with the CRU's research:

There is no reproducibility. There is insufficient documentation. Their archive has huge gaps. No record is available for the scrutiny of other scientists to verify their results by reproducing their research. Full disclosure was not practiced.

In short: they failed to do science.

Phil: The Eric Raymond blog entry is here.

"Nobody on the outside knows." Would you agree that where no-one knows but still makes claims, then they are just making things up?

Can we perhaps agree that you will use a symbol like "!!!!" to mean "They delete/remove/fail-to-retain/have-some-gaps-in/we-cannot-find-in-stolen intermediate data or metadata therefore it is not science according to Wikipedia" and I will use "????" to mean "but the original data and methods are published (and now code details), intermediate data could not be kept, and they should get funding to put it in the order it needs to be in now that it is so important". This will help prevent my poor readers getting RSI from unnecessary scrolling.

Thanks for the reference.

I spent 10-15 minutes looking through the blog article and the comments. What I'm left with is a more resonance for the point I was making: nobody knows what the heck the CRU was doing in the data it was manipulating and displaying.

An open, transparent, audit-able, and verifiable path between the raw data and the output of this lab is fundamentally absent. Science is what's needed from these people, and they are failing to do science.

This is vastly different from what the IPCC does. Eric Raymond allows everyone to comment on his blog postings. Anyone who wants to get the full picture can read them all right there.

The IPCC took 2350 and turned it into 2035. Even though the error has been known for over a week, they've done nothing to correct it. And, unlike Eric's blog, the IPCC has official sanction under our UN.

Where is the editor to fact-check their releases and ensure against typos? Where are the folks who listen to the feedback from the public in case something incorrect gets out? What the heck are these guys waiting for?

The 315-year error is turning out to be a black eye for the IPCC.

Phil: "Even though the error has been known for over a week, they've done nothing to correct it."

Err, maybe they have other things on their minds this week? Such as, say, the meeting in Copenhagen!

Where do you want them to change it? They cannot retroactively change old documents, so the place to change it is any new reports. And they would need to double check the chain to confirm it is indeed the error it is claimed to be.

By the way, I think it is pretty shakey to say that "nobody knows what the heck the CRU was doing in the data" when those nobodies are using stolen data amd we have no idea how complete it is? Was it taken from just one machine? Do other machines have different data? Is any data or programs in archives? It looks like huffing and puffing.

Bingo! The fact that Copenhagen is happening would be the precise reason that they should have issued a prompt correction. Burn a little midnight oil to get their report accurate.

We have sufficient openness and transparency in our society that anyone not shrouded in complacency already knows about the error. It costs the IPCC credibility to continue to have the wrong date published.

Things could well backfire. The fact that the IPCC is stonewalling their clear errors may give some countries a reason to not agree to action at this meeting. They might start presuming that the IPCC is being less than honest with the world, and this could be viewed as a smoking gun.

"Stolen data" is a spin. A different spin is "data provided by a whistle-blower". The whistleblower data has shown us that the CRU process is a failure unless the openly and transparently show the steps from raw data to published materials. Things look ugly because the whistle-blower has revealed what a cluster-mess the CRU has been doing. Had they been open and transparent with their data and processes from the start, there would be no climategate.

Eric Raymond is providing open and transparent discussions about the wistleblower data. Why do you seemingly equivocate that with the IPCC's clear 335-year error that they published and (AFAICT) haven't corrected?

The IPCC should issue a correction and distribute it exactly the same way they sent out the first report. This is not rocket science: how long should it take for them to verify that they made a typo in the first report?

"Err, maybe they have other things on their minds this week? Such as, say, the meeting in Copenhagen!"

Exactly how long would it take them to put out a release saying: "Oops. This report said 2035, but the actual paper we were citing said 2350."

We were discussing that an excellent way to judge a source is how adeptly to errors. An agency that can't post a quick correction an obvious error like this before the Copenhagen meeting leaves the impression that they are stonewalling. Such actions can lower the trust of everything the IPCC is saying -- their failure to respond promptly could indeed backfire.

How long would you give the IPCC to issue a correction?

"Where do you want them to change it?"

Re-issue an updated version of the original document with a paragraph at its beginning noting the error.

"They cannot retroactively change old documents, so the place to change it is any new reports. And they would need to double check the chain to confirm it is indeed the error it is claimed to be."

That last act is simple: they just need to look in the peer-reviewed journal they were citing.

"By the way, I think it is pretty shakey to say that 'nobody knows what the heck the CRU was doing in the data' when those nobodies are using stolen data amd we have no idea how complete it is?"

That is exactly what I have been trying to say! We have no idea what version of the code was applied to data to produce some particular result. What those whistleblowers have revealed is there was an extremely haphazard process happening. As Charlie notes, there's evidence that the CRU's motivation was massage their results until they "told the right story" rather than the story the data was actually showing.

The whistleblower data is revealing just how un-open and un-transparent the CRU process was.

"Was it taken from just one machine? Do other machines have different data? Is any data or programs in archives? It looks like huffing and puffing."

If the CRU had been open with their process -- if they had actually been doing science -- we would know the answers to that. But if they had been open from the start, there would have been no reason for whistle-blowers to take any action.

Did you notice that two major research organizations have gotten the message have announced an open-access data policy? See http://pajamasmedia.com/blog/climategate-and-the-first-rule-of-holes/ . Still waiting for the CRU ....

Phil: How long? Well, if it were one the agenda now it would take a few months for more important issues coming out of Copenhagen to get dealt with, then a couple of months for dealing with a lot of the scattergun crap being spewed out at the moment, then a couple of months for everyone to confirm it was correct, then however long it took to get to the next committee meeting (or however it is institutionally arranged.) So I six months would not be unrealistic.

As for a new version of an old report, that is simply not the way things work in any of these things. A new report gets issued, reflecting the current beliefs.

What many reputable publishers do, however, is have an errata website, where issues like this are put: even notices like "This information has been disputed by XXXX and will be checked." where XXXX is some reputable climate scientist. They should get funding for this.

So if people are making ludicrous speculations on not enough evidence, it is not their responsibility? Listen to yourself!

The most important thing for the IPCC to do is to have an environment of credibility and trust for the Copenhagen meeting.

What has happened in Copenhagen? I'm sure you know: http://blogs.wsj.com/environmentalcapital/2009/12/14/copenhagen-walkout-poor-countries-bail-on-climate-talks/

Did the IPCC's failure to promptly correct their 315-year gaffe have any impact on causing that walkout? I don't know. It is clear that developing nations have a fundamental issue of trust with developed countries; such an error could not have possibly helped. Anyone who is paying attention already knows about the error. Nobody is disputing the error. There can be no upside for the official mouthpiece to delay acknowledging it.

We'll see if the IPCC acknowledges their error by May 1, 2010. My last question: how will you react if they fail to correct their error by then?


Phil: So now they are back, does that mean they have decided that mistake (if it was) doesn't change anything, by the same logic?

I don't know what impact the walkout-and-return will have.

I have no idea how Obama could promise US payments to developing nations. That would constitute a treaty; those must be ratified by the Senate.

Bill Clinton signed on to the Kyoto Accord, but he never ever bothered to bring the agreement to a Senate vote.

It's your turn: How will you react if the IPCC fails to correct their 315-year error about himalayan ice within six months (May 1, 2010)?

You haven't answered. Again. I'll answer for you:

You won't react at all if the IPCC fails to correct their 315-year error by May 1, 2010. You don't really care if they ever correct their error. You're complacent about whether or not the IPCC reports the science accurately.

"Phil: So now [countries that walked out of Copenhagen] are back, does that mean they have decided that mistake (if it was) doesn't change anything, by the same logic?"

Copenhagen is now over; we don't have to speculate.No tangible agreement was crafted in Copenhagen. When you boil it down, the agreement that was made was to keep negotiating -- to keep talking.

China was clearly unwilling to enter any agreement that entailed monitoring of their emissions.

What did the 315-year gaffe -- and failure to correct it -- have on those third-world countries? Nobody can say for sure. The failure certainly did nothing to create an environment of trust between the UN agency and those countries.

Phil: From what I can see, it had absolutely no effect at all: Climategate and other sideshows (fake and real) were given the credibility they deserve: things for scientists to work through.

The third-world countries wanted more action on climate change not less. Was it any county's position that there was no climate change?

But they seemed to split between the countries who thought that they had a good chance of developing (India, China, South Africa) who didn't want limits that would impede their development on the one hand, and the countries who have basically given up any expectation that development (i.e. industrialization) will trickle down to them (Sudan) or the island states, who need finance.

At least Copenhagen gives the greenies a chance at the kind of nutty hyperbole that has been the skeptic's forte: I enjoyed Greenpeace's press release "The city of Copenhagen is a climate crime scene tonight, with the guilty
men and women fleeing to the airport in shame."

The discussion is continued at the bottom of the comments section.

Fallible is what all humans are and what they create.
I always think of how everything evolves; the earth, animals, humans, and even ideas. Newton’s law of universal gravitation was thought as gospel until Einstein’s general relativity was proven. Before Einstein’s theory was proven the consensus was that Newton’s law was the only truth. Given human fallibility I believe some day Einstein’s general relativity proven to be inconstant and a new theory will evolve.
That is way I can’t believe global warming is a valuable theory. History shows that climate change is part of earth’s habits. We have reliable weather data that has only be recorded for a few decades. We can only estimate on earth’s temperature and environmental factors and which years those readings apply to. So to truly be accurate in calculations only human recorded information must be used. When it comes to climate scientist can only observe and run manmade calculations, they cannot run scientific studies like other scientist to prove their theories. Let’s pretend that Newton also came out with theories on climate and current scientist are using his calculations for predictions and such but to be truly accurate they need to be using Einstein’s theory’s but Einstein’s theory’s have not be developed or created yet. All calculations using Newton’s theories would produce misleading and unreliable predictions but current scientist will take their data to heart until Einstein comes around and proves that Newton was on the right or wrong path.
The other thing that this article was trying to show is human observation. I can tell you my observation where I live and work that the overall area temperatures are cooler than I previous remember and our warm periods are not higher than usual. But that is just my observation and climate theory is solely based on observation.
Earth’s climate and environment (multiple forces) is too chaotic to fully grasp right now compared to gravitational forces (single force). You have more than just base chemicals in the air that control climate for areas on Earth. You have the sun, clouds, wind, geothermal anomalies, and even under water currents, such as the one that keeps UK warm. In theory UK should have the same climate and environment as Canada because they are roughly at the same Latitude but yet they do not.
Back to the observation... In theory for Global Warming to be 100% true all parts of the Earth need to be affected. My observations and someone else’s in the world observations should show continual anomalies in the climate and environment.
Time will tell if Global Warming is fact or fiction not equations or simulations. But one thing should be that politicians should not be using this theory, because it truly unproven that Global Warming is happening and if so what is causing it. A scientist ideas and actions effect evolution of their field, while a politician’s ideas and actions affect the citizens of their country from one generation to another.
The one thing I don’t get is how people can use the term climate change because the climate is always changing through the seasons and how people can use the terms climate change and global warming interchangeably since they are two different ideas. Climate change can be an increase or decrease of an area where global warming can only be an increase worldwide.

Michael: First, weather changes through the season, and the climate varies through the seasons, but that does not mean that climate cannot change: varying in different ways.

You can have global heating and local effects that are different: currents can flow different ways in different climates. In one place, it might be that storms or floods are more noticed. In another place, the change might be quite pleasant. In another place it may just seem that they don't have as many cooler summers than they used to.

For some good graphs on recent climate behaviours, see AccuWaether. The comment that only America experienced cooler than normal conditions this year (2009) is particularly interesting: it may explain why some Americans are the leading voices against the idea of global warming: it does not match their experience and they are extrapolating from their backyards to the rest of the world.

“Warmest on record”, is the phase I always see. How old is the earth and how far back do temperature records go ... 100 years? There is a dilemma, we don't know, the climate has “varied” through the plant's life time and will still vary until the end of the planet's life. Through out the life of the planet millions of life forms have been born and sent to extinction, and some day humans will be too, because of the evolution of the earth and not because of human interaction.

I was recently think about the Dodo bird and how humans were said to create the extinction of the bird. Man was just an predator, now what if another animal came around and was the predator against the bird? Since the bird didn't have any predators during it's evolution it would have mostly became extincted after the arrival of the predator, be it man or another animal. Just because right now people assume that humans are responsible for the recent climate change but it could just be the evolution of the planet or other factors.

The only two things we know is that the temperature has slowly fluctuating upwards and that there is quite a bit of CO2 in the atmosphere. With out real proof there is just a general observation correlation. Just like the world's total economy is still in a recession and the amount of news coverage on pirating has go up. Does that mean pirating has gone up and is it because of the recessions? General observation correlation.

I'm sorry but using a website that headlines John Stewart, a comedian, and Al Gore, an ex politician that will not even debate the global warming idea, makes the website look like it is scratching for information any where they can find it.

The one thing people keep forgetting is that the earth's climate & environment has changed / varied and never remained constant through out the planet's life.

Michael: One thing people keep forgetting? Sorry, I don't think that is true.

The climate scientists talk in terms of probabilities, not certainties. For example, in 2002, the IPCC said there was 66% chance of AGW. By 2007 they had strengthened this to 90%: 'very likely.'

To my way of thinking, even if the AGW probability is 50% global warming is worth acting on. And even if CO2 is the dominant issue, it is worth acting on a broader range of land-use and greenhouse gases, for their contributions.

And even if it works out as global cooling rather than warming (the probability seems to be that there would be some areas of cooling but more of warming), and even if there is a substantial natural component, it would be worth acting. Especially if climate systems have with local equilibria, where climate lurches rather than merges between states.

All these things suggest to me that it is prudent to take steps to mitigate the effects of climate change, as part of general ecology management, so that it does not happen (if it happens) faster than our social/economic capacity to cope with that change. Perhaps growing up on a farm in a beautiful valley so polluted by dioxins that people were not supposed to eat beef from cattle raised in our fields gives me a certain perspective.

Take one important mitigation issue alone: stopping forest/wetlands destruction and promoting reaforestation. That has many benefits apart from likely AGW: species diversity, smog reduction (if you have experienced the burnoffs in Indonesia/Malaysia you will understand this), sustainability, increased water retention (important in Australia with our river system crisis), maintaining spawning grounds, and so on.

Mitigation does not mean necessarily preventing any climate change, it means figuring out how to cope with it when it happens, and trying to make sure that if it does happen it does not outstrip our ability to cope with that change.

Many of the strongest sceptics are, despite disagreements on this issue, still very strong environmentalists, from what I have read. (There is the nutty fringe of ratbags operating out of conspiratorial political/culture wars, with robotic kneejerks too.) Their opposition to AGW is often based on extreme concern "no, these other issues are more important and should be looked at" it seems to me: I don't see why concerns of people who, for example, think that land management is a major issue that should or necessarily would be sidelined by attempts to bring carbon into the market system, should be ignored.

Charlie Martin (above) wrote:

"None of this means that "global warming is a hoax" ? it's well-known there has been warming. The notion that some of it is anthropogenic still seems reasonably strong, and that CO2 is one among many anthropogenic forcings seems very likely.

OK

"On the other hand, at this point the validity of any studies based on the CRU data must be questioned,

Well, any studies based on any single set of data must be validated anyway. Conclusions get their credibility from being supported by multiple studies. So what?

" the evidence of at least some significant misconduct seems pretty unequivocal,

Well, no scientist likes being rejected, so that at any time there is often one mob in the ascendency -- and another mob feeling hard done by -- is not new.

How much of this misconduct be proved and how much is just conjecture? (The Nature article points out that the offending papers were in fact referenced, for example. That rather deflates one of the arguments that they subverted peer review.) I have a very low thresh-hold for tolerating allegations: suspicions that may be appropriate for saying to confidants may be very inappropriate for broadcasting to the world.

" the likelihood of actual criminality seems pretty significant.

People should be careful of the UK libel laws. Someone merely saying they felt like doing something is very different from them actually doing it.

"Continuing to try to say "nothing to see here, move along" looks more and more like denial.

I think if you go through my blog, my "move on" is the very practical comment that to get improvement, funding must be provided.

Here is the problem I have: the anti-CRU "smoking gun" people are saying "You cannot trust the CRU because when we examine it we find bits here and there that are unreliable, this seems to be a pattern to the larger effort, plus they are nasty, and they have an ideology driving this."

But when I applied the same test to the various anti-CRU blogs (the ones I mention in this series of blogs) what do I find? Exactly the same. So I take their (implicit) advise and I don't trust them.

That ludicrous Raymond blog entry (and I have a lot of respect for him, in his area of expertise he is great), the awful Yambagate graphs of Eschenbach, the blowing out of proportion of bad coloring on a good graph, the claims of bad actions based on private emails between venting friends, the obvious grasping of straws, the apparant willingness by a large section of the anti-climate change brigade to accept any old codswallop as long as it is anti, does not inspire confidence.

When one lot of people says "They are bad scientists and bad people because they call us bad scientists and bad people because we call them bad scientists and bad people" then they need to be most scrupulous in their own science and conduct.

Now maybe I am guilty of just looking into the more lurid claims that make it into the papers. Maybe the lesson is that when anyone says "I've found a smoking gun" I should just be presume them to be idiots and not even bother taking them seriously. Maybe I should be looking at the calmer, more serious voices (err, you and Phil?), who will have a higher meat-to-nuttiness ratio.

It seems to me that there are three underlying issues here: how is a scientist supposed to deal with his peers when their views conflict (e.g. the debate with the Pielke's about whether AGW is mainly CO2 or mixed factor), how is a scientist support to deal with layman or non-specialists they regard as vexatious or nutty (e.g Eschenbach and so on), and what happens when their judgment (with the presumption of good faith we have to make) about who fits into each category is wrong?

Readers: There are a few links on this page to a site called Pyamas Media. I have just looked at it, and I let readers draw their on conclusion about a site peddling articles such as these:

Tiger's Score which includes the gem "African-American and Jewish-American women know all about how their men tend to lust blonde, how they often end up choosing White Gentile Goddesses as soon as they rise and leave the neighborhood." and finishes off "At least the Man in the White House has an African-American wife."

New Evidence of Saddam-Terrorism Links which quotes more people who say they have seen evidence but are ...err... still unable to produce it, including the same people who said there was evidence before the war and benefited from it, such as the ex-Prime Minister.

Orwellian Nightmare: Science Is Whatever ‘the Party’ Says It Is says "The Party — the political class of the world — does not want God to exist. Therefore, if the laws of physics and the laws of mathematics say He does, then the laws of physics and the laws of mathematics must be changed to whatever the Party wants." And this: "The Party wants the Earth to be warming, so that its members can establish their power over every aspect of our lives. The Earth has not warmed in a decade, in fact it has gotten colder. But the Party says warmer, and further, says that the warming is due to human addition of CO2 to the atmosphere.

I read the last article. It was quite enlightening: a physicist actually gets debated on colleges whether or not 2 + 2 really is 4. Apparently, there are quite a few people who grossly misunderstand Gödel's Incompleteness Theorem; he sets them (and the readers) straight in this short article. I learned something new: I'd never heard of Presburger arithmetic before reading that blog entry.

Even if all of those three blog entries were true gibberish, what does it say about the rest of the blog? What if someone on oreilly.com made a blog entry that set my bozo bit: should I then ignore not only that blog entry but every single posting here?

I presume it's clear to most readers here that the bloggers on oreilly.com are only loosely affiliated. I read the ones that are interesting to me and ignore the other ones.

It's also obvious to me that Pajamas Media operates in a similar loosely-organized fashion. Trying to paint an entire blog with a bozo bit because of something you personally found objectionable in some postings is a bit strange. It says far more about you than it does about that repository of a group of bloggers.

Thank you for letting the readers draw their own conclusions about that blog. However, there is another option: the readers are also free to draw no conclusion whatsoever about that blog.

Rick, have you ever heard of the "ad hominem circumstantial"? You might want to look it up: better understanding of it would let you practice it with greater skill.

As with O'Reilly, people who are published on PJM are only loosely affiliated with it; their opinions are their own. I think some of them are goofy; I think much the same of some ORA blogs.

Now, as to the other points, the most important issue in my mind, as someone trained as a scientist, it the corruption of peer review. Since Mike Hulme has already written that they made mistakes along those lines, and since it's clear from a number of the emails that, for example, the CRU authors were exchanging articles they received for peer review and discussing how to develop a unified front to suppress them, I think the case for its happening is very strong. This is real misconduct. As you say, this isn't unknown in other areas of science; what you don't mention is that other, extremely reputable, scientists have been reprimanded rather severely for it, as I suspect some of the IPCC authors will be.

On the libel issue, I think you might re-evaluate: "the likelihood of actual criminality seems pretty significant" is quite carefully phrased. Feel free to refer that sentence to Dr Jones' solicitors; I'm confident it will stand up. In any case, truth is a defense in both the UK and the US, and we have a thread of emails in which Dr Jones discusses FOI and in that context requests other to delete certain emails. We also have Mike Mann's confirmation he received that email, that he felt it was wrong to make that suggestion, and denying that he acted on it.

What really calls the IPCC results into question, though, is the clear political pressure felt by members of the IPCC group, such as Keith Briffa, to make sure that the data told the "right" story. Dr Pielke Sr has documented this in several cases; Hans von Storch, Eduardo Zorita, and others have also complained of the political pressures on publication. I have a new piece coming up soon (tomorrow, I hope, but that's a function of the webmasters) in which I discuss the truncated Briffa trendline. The truncation is considerably more serious, and more misleading, than you make it out to be. If the data had been included in the overall average, it would have dragged the trendline down significantly; if it had been shown in the figure, it would have conflicted with the story they wanted to tell — a fact the emails show was completely clear to the authors.

Maybe I'm becoming a cranky old man, but where I was trained, this would not have been acceptable. Science uses the data to find out the story; this was a case where they concluded on the story, and massaged the data to fit.

Add to that, on one hand, the fact that Roger Pielke Jr had his hurricane results misstated — actually had the conclusions completely misrepresented — on the one hand, the continued efforts by IPCC authors to use threats against members of the press to continue to control the message, and active prevarication about the basis of their conclusions, and I think the intellectually serious will have to conclude that there's something here that demands serious consideration, not dismissal.

Charlie: First, I did not bring up the issue of the how my viewpoint fits in with the hosting website, Phil did. And some of the views on Pyjamas Media are so extreme, nutty and borderline racist (or that is how it seems to this non-US reader: perhaps there are nuances in the language I don't get) I almost considered removing any links to it.

Second, if anyone at O'Reilly wrote the kinds of things that are in multiple articles on Pyjamas Media, I am sure that most OReilly authors would resign, not wanting to have their names associated with such a site. (In fact, many ORA authors probably are associated with ORA *because* it is not a site for non-technical/non-technological issues.) Because there are no political or general world commentary blogs on OReilly, no writer needs to feel associated with an agenda: Pyjamas Media is clearly quite different. I recommend you find a less cranky outlet.

Third, many climate-change sceptics or denialists frame the issue in a larger one of power grabs and conspiracies (or even attempts by greedy third-world-ers to get more money) which encourages this gotcha-seeking mentality, where bogus smoking guns are uncovered, only to be replaced in short order by the next equally bogus one. They claim that the CRU science is bad because it fits facts to an agenda, but they look to me like they do the same things themselves. What your viewpoint?


By the way, on libel, IANAL but I note that in the US it is possible that truth is not a defense if actual malice (the knowledge that a statement is false, or with reckless disregard for whether it is false) was involved.

And in the UK it seems, the burden of proof is on the defendant and a very high standard is set, there is no right of free speech, and the plaintiff doesn't need to prove actual damages. (There is some kind of public interest defense, but you would have to be a responsible journalist to get it, from what I read.)

And there are also fair reporting considerations: if I write "John stole money" it is somewhat different to if I write "Fred says John stole money".

What I said: "Since [Rick] blogs under the ORA banner, there's also a bit hypocrisy in your silence about CRU's failure to be open and transparent."

I presume that every single person who blogs or O'Reilly is for things open and transparent. It's true of any blogger I've ever read here. Based on your published biography ( http://www.oreillynet.com/pub/au/1712 ) you are clearly an advocate for open and transparent web standards and for the Wikipedia. Hooray! Those are good things!

Equivocating my statement with your ad hominem circumstantial attack of Charlie is rather strange. I would prefer to drop that conversation and talk about Climategate here. Please respond to my questions in my December 10, 2009 12:34 PM comment. Thanks!

Phil: I made responses to basically the same question several times, for example December 10, 2009 1:23 AM.

Not one of the people who have merely read the stolen private emails (and yes, I have read a good number of them) and partial archive from CRU is thereby in possession of all the facts, let alone people who rely on second-hand reports: they may be enough to make justify statements like "It looks like..." or "It worries me that..." or "I cannot see that..." but it is pure hubris, as far as I can see, to make any absolute statements like "This shows that the CRU was not doing science."

That some of the same people who make these overstretched comments also play so free and loose with terms like "destroyed" and "trick" is an alarm bell that says "sensationalist" or "fearmonger" and "don't buy in to any of it."

No. You failed to answer my question: is there something flawed in what the Wikipedia entry on the scientific method characterizes as the basic expectations of science?

You don't need to be a science philosopher to answer that question. All you have to do is understand how to apply the scientific method to physical sciences. And you don't even have to worry about what anyone demanded or didn't demand of the researchers of the CRU.

What I have pointed out is that the CRU researchers failed to perform their physical science research in a way that is reproducible. Neither proponents nor skeptics of their research caused them to do that; they failed all on their own.
If you think the CRU should have been given A Pass for that failure to meet the basic requirements of the scientific method, please explain why. If you can't, please say so.

You have never ever answered this question. Please answer it now. Thanks!

" That some of the same people who make these overstretched comments also play so free and loose with terms like 'destroyed' and 'trick' is an alarm bell that says 'sensationalist' or 'fearmonger' and 'don't buy in to any of it.' "

They destroyed data; their research is not reproducible. The CRU has failed to meet the basic expectations of science:

"Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

It is hardly fear-mongering to note that they failed to meet these measures; it is a statement of fact.

If they think they should be given A Pass -- a mulligan -- for failing to do science, please explain why. Explain why those standards should not apply to the Climate Research Unit of the University of East Anglia. If you can't explain why, please acknowledge that.

Phil: Oh, we are back to "destroyed data" again are we.

I thought we had established that they had not destroyed any raw data (how could they, it is not theirs to destroy but is available from stations), that they had published their methodology enough to allow another team to take the same raw data and derive their own version of the adjusted data, that the adjusted data exists and is available, and that (as far as we know) the specific programs that they used to do programmatic work still exist.

In Jones' words "We haven’t destroyed anything"

Elsewhere Jones comments


"If we have “lost” any data it is the following:

"1. Station series for sites that in the 1980s we deemed then to be affected by either urban biases or by numerous site moves, that were either not correctable or not worth doing as there were other series in the region.

"2. The original data for sites for which we made appropriate adjustments in the temperature data in the 1980s. We still have our adjusted data, of course, and these along with all other sites that didn’t need adjusting.

"3. Since the 1980s as colleagues and National Meteorological Services (NMSs) have produced adjusted series for regions and or countries, then we replaced the data we had with the better series."

The first case is not destroying but discarding. The second case is not destroying but discarding. The third case is not destroying but updating. And the fact that the CRU major findings have in fact been replicated three times independently rather undermines your argument, doesn't it?


[ASIDE] Speaking of destruction of data, I did a search on the CEI website (these are an organization mentioned by Jones in the link who have pushed the "destroyed data" idea) and I see a search under "smoking" and "tobacco" produced no hits on their site. This is rather strange, considering they were up to their eyeballs in trying to diss the science on tabacco: see here and here (and noting that ExxonMobil has pulled out from funding them: a bit too obvious perhaps.)

Charlie: I certainly agree that if circulating articles was against the peer-review rules, they were naughty, in absence of some justification.

(And I would expect the review would slap them on the knuckles for it, in proportion to their misconduct and the justification and intent to deceive: if they were acting in good faith, the remedy might just be not use them as peer reviewers again, or counsel them, and perhaps re-review recent submitted papers.)

Is this the issue that Pew Center made the following comment?:

"It is crucial to understand that the authors were not contemplating the suppression of an dissenting point of view. Rather they were reacting to what they considered scientific misconduct by authors ... and editors ... to circumvent the peer-review process so as to publish inferior papers that supported their own political agendas."

That puts in a rather different light, doesn't it?

But it doesn't change their science (to say it did would presumably be an ad hominem attack?)

More fallout after Climategate: Al Gore is no longer given A Pass for making sensationalist claims that can't be backed up by science:

http://www.timesonline.co.uk/tol/news/environment/copenhagen/article6956783.ece?utm_medium=twitter&utm_source=twitterfeed

From the Times story:

"Perhaps Mr Gore had felt the need to gild the lily to buttress resolve. But his speech was roundly criticised by members of the climate science community. 'This is an exaggeration that opens the science up to criticism from sceptics,' Professor Jim Overland, a leading oceanographer at the US National Oceanic and Atmospheric Administration said."

This rapid rebuke from a scientist is a welcome result of climategate. Unsubstantiated sensationalist claims about global warming were serving nobody in this debate.

The New York Times has a story about Gore's talk: http://www.nytimes.com/aponline/2009/12/14/science/AP-Climate-Gore.html?_r=1&scp=7&sq=al%20gore&st=cse

However, they have failed to pick up the inconvenient truth that the researcher he is citing is mystified where the claim came from. Back to the UK article:

"In his speech, Mr Gore told the conference: 'These figures are fresh. Some of the models suggest to Dr [Wieslav] Maslowski that there is a 75 per cent chance that the entire north polar ice cap, during the summer months, could be completely ice-free within five to seven years.' "

“It’s unclear to me how this figure was arrived at,” Dr Maslowski said. “I would never try to estimate likelihood at anything as exact as this.”

"Mr Gore’s office later admitted that the 75 per cent figure was one used by Dr Maslowksi as a 'ballpark figure' several years ago in a conversation with Mr Gore."

None of that is in the New York Times article! Apparently, a negative reaction from the scientist that Gore is citing isn't part of "all the news that's fit to print."

Now that the rebuke from the scientist himself has gotten wide coverage in The Drudge Report, it will be interesting to see if the New York Times alters their story.

Phil: So what? What Al Gore says is nothing to do with the CRU issue.

The far better question is what Al Gore's claims ever had to do with science.

When Gore is actually citing a scientific source, his reports are fine (provided he cites his source -- just like the Wikipedia). When he is exaggerating, sensationalizing, or just making up stuff, it serves no purpose.

What has changed is that Scientists are losing their tolerance for Mr. Gore's fiction. Dr. Maslowski called Al Gore on his sensationalist spin of his science -- as reported by the Times (UK). It's equally interesting that the New York Times failed to report that Maslowski's research was exaggerated by Gore. That speaks volumes about the NYT and the US media. CBS and Yahoo had similar watered-down stories. Fox's report included the Maslowski statement. Some of the media values accurate reporting on science; some is still complacent.

Scientists have been complacent about Al Gore's whoppers for a long time, but obviously the climate for accountability is changing. You think it has nothing to do with Climategate; that's interesting. Why do you think it's changing now?


Phil: Perhaps Al Gore is considered an important figure for Americans in this, but he is not a figurehead over here (either for the pros or antis). Please don't hijack the thread.

"Phil: Perhaps Al Gore is considered an important figure for Americans in this, but he is not a figurehead over here (either for the pros or antis). Please don't hijack the thread."

Rick: I've looked around and asked around. As far as I can tell, your claim that Al Gore is ignored by the rest of the world seems like utter nonsense.

There is no official journal of the Copenhagen 15 meeting, but the Cop 15 Post ( http://cop15post.com ) is the closest thing going. It's published by The Copenhagen Post ( http://www.cphpost.dk ).

PDFs of the daily are available for download. The 15 December issue ( http://cop15post.com/wp-content/uploads/2009/12/ISSUE-7.pdf ) covers Al Gore's talk in Copenhagen. It is a front-page article and it is the sole article on the front page.

Please defend your claim or retract it. Thank you!

Phil: You are just trolling. Or hyperventilating. Or inflating.

I didn't claim Al Gore was "ignored by the rest of the world", so I cannot retract it.

In Australia, our stars in this are more Senator Penny Wong, Minister Peter Garrett, Prof Plimer (who had such an embarrassing interview a couple of days ago: just came across as shifty), and Tim Flannery. There is a sprinkling of visitors from other countries who get some press when they are in town, such as Al Gore, but certainly not, as I wrote, a figurehead.

Phil: I have asked you not to hijack the thread with material about Al Gore that is clearly unrelated to Climategate in particular.

But you have continued to post about Al Gore so I will be removing any posting that mentions it in the future. You have your own blog to vent about US personalities and usual suspects.

"Phil: Oh, we are back to "destroyed data" again are we."

We are back to the failure of the Climate Research Unit to create research that is reproducible: a basic expectation of science.

You have failed to ever explain how anyone could do the following with the CRU's research (From the Wikipedia): "Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them.

"I thought we had established that they had not destroyed any raw data (how could they, it is not theirs to destroy but is available from stations),"

They threw out their archive of data they used to do their research. They made it problematic for any researcher to attempt to reproduce their results.

They not only threw out their original data, but they also do not have an archive which scrupulously documents what data they altered, what data they didn't alter, and how exactly they manipulated the data that they did alter.

Their failure to preserve those records makes it problematic for any researcher to reproduce and verify the results of the Climate Research Unit. Even if somebody were able to re-aggregate all the data that the CRU had acquired, there is no archive of the CRU's manipulations.

Again from the Wikipedia: "Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

Full disclosure is absent; the basic expectations of science were not met.

If you think the Climate Research Unit should be given A Pass -- a mulligan -- for failing to meet the basic expectations of science, please explain why.

"Phil: Oh, we are back to "destroyed data" again are we."

We are back to the failure of the Climate Research Unit to create research that is reproducible: a basic expectation of science.

You have failed to ever explain how anyone could do the following with the CRU's research (From the Wikipedia): "Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them.

"I thought we had established that they had not destroyed any raw data (how could they, it is not theirs to destroy but is available from stations),"

They threw out their archive of data they used to do their research. They made it problematic for any researcher to attempt to reproduce their results.

They not only threw out their original data, but they also do not have an archive which scrupulously documents what data they altered, what data they didn't alter, and how exactly they manipulated the data that they did alter.

Their failure to preserve those records makes it problematic for any researcher to reproduce and verify the results of the Climate Research Unit. Even if somebody were able to re-aggregate all the data that the CRU had acquired, there is no archive of the CRU's manipulations.

Again from the Wikipedia: "Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

Full disclosure is absent; the basic expectations of science were not met.

If you think the Climate Research Unit should be given A Pass -- a mulligan -- for failing to meet the basic expectations of science, please explain why.

Phil: Your premise (that data was destroyed) being wrong as far as I can tell, your conclusion (that the research can be replicated and therefore it is bad science) does not follow.

Rick: You didn't read what I said:

They not only threw out their original data, but they also do not have an archive which scrupulously documents what data they altered, what data they didn't alter, and how exactly they manipulated the data that they did alter.

Since there are not scrupulous records of what manipulations were done to what pieces of data, you need to spell out exactly how this is repeatable/verifiable science.

Blindly asserting something that you hope is true is not sufficient. The basic requirements of science mandate that all manipulations to the data were scrupulously documented. If Full Disclosure was not performed, then the basic requirements of science were not met.

Phil: So now we are back down to "not keeping scrupulous records" on individual decisions based on described methodologies (which is itself unproven, by the way: the stolen data cannot tell us what is or is not present in their complete systems.) How long before you inflate this back into "destroyed data"?

One interesting item in this regard is that Prof. Jones has said that they replace the datasets they adjusted with high quality adjusted sets from the originating countries as it becomes available. (And, for example, the Australian BoM does keep very detailed records and publishes the methodology and how it was applied to station data series.)

So *even if* CRU decided early that some logs of individual changes would not be necessary to record since they were applications of a described methodology etc. and a level of detail too fine to be significant, that adjusted data has progressively become less significant.

I would presume the extra attention will only spur this on further.

"Phil: They are only under an obligation to micro-document as much as they may reasonably consider to be significant."

No. If they expect their research to be viewed as science, they are under an obligation to meet science's basic expectations.

"Reasonably consider to be significant" is never described in the Wikipedia page when describing the scientific method. That would be like having a build for a large software system where some steps were done manually. What sense would it possibly make to not carefully document every step you did?

How could the CRU ensure reproducibility of their results if they didn't have scrupulous documentation?

"We can judge whether this theory (that the individual adjustments to the extent that some *may* be poorly documented did not exert a significant effect on the results) by testing the adjusted and unadjusted data."

This ignores the fundamental question: why would one be slipshod with their management of the data manipulations? You keep hinting that these guys should be given a waiver for the basic expectations of science.

Anyone who has looked at the HARRY_READ_ME file will get a taste of the "rigor" that the CRU was using with its manipulation scripts: http://www.climate-gate.org/cru/documents/HARRY_READ_ME.txt . It has the flavor of a high-schooler doing midnight hacks and not what should be a respected organization practicing science in a rigorous and reproducible fashion.

Have you read through the HARRY_READ_ME file?

"But to deny the validity of the conclusions based on adjustments which can be shown statistically to be neutral, and where the major results have been replicated multiple times, is not reasonable."

That's not what the ongoing research into the whistleblower files is showing. Major discrepancies from the CRU reports and reports from other research institutes are showing up. The darn hockey stick keeps disappearing!

Anyone interested should look at http://pajamasmedia.com/blog/climategate-faster-and-faster-the-dominos-fall/

One wonders if the AP will now reopen their analysis of the Climategate whistleblower files.

Rick: you're not thinking about the scientific method; your assertions are completely backwards. The onus is on the researchers to defend their science, not the other way around.

Did they demonstrate that they kept scrupulous records which scripts were applied to which data?

Where in their research did they document how the scripts were organized? Were they stored in a database?

How were updates to the scripts managed -- did they maintain version control on the script files? Is there any evidence in the Whistleblower Files that there was version control on the scripts?

DId they keep records which version of a script were applied to which data?

"So *even if* CRU decided early that some logs of individual changes would not be necessary to record since they were applications of a described methodology etc. and a level of detail too fine to be significant, that adjusted data has progressively become less significant."

Did Jones say that they took the manipulated data Carte Blanche from those countries? Did they get documentation on how the data was massaged, or did they just blindly accept the data?

That would be another thing that they would need to prove in their analysis.

"I would presume the extra attention will only spur this on further."

We shouldn't need statements from Jones to explain what was done in changing data of incremental versions of their research. All those should have been already published by the [currently-suspended] director and his team.

Do I need to spell this out for you again? The end-to-end chain of data for the all the data must be accounted for. And the metadata must also be managed.

The onus is on the CRU researchers to show that they followed protocols. If that was not done and scrupulously documented, their research is not science.

This is not political and it is not sensational. It is simply the way science works. You've said you are not a philosopher of science, but are you well-grounded in the fundamentals of the scientific method?

Phil: They are only under an obligation to micro-document as much as they may reasonably consider to be significant. As I said, what they document itself represents a theory about data.

We can judge whether this theory (that the individual adjustments to the extent that some *may* be poorly documented did not exert a significant effect on the results) by testing the adjusted and unadjusted data. And lo, the adjustments are neutral.

And lo the adjustments does not alter the trends for European data either. And they concur with those found in other people's versions of the data.

So you are getting excited about some adjustments that, it seems, were neutral.

It is perfectly reasonable of you to have a theory that actually this level of logging is of interest to their results. And that keeping these logs in good order is basic science housekeeping. And that for matters of public interest, these logs should be made public.

But to deny the validity of the conclusions based on adjustments which can be shown statistically to be neutral, and where the major results have been replicated multiple times, is not reasonable.

"Phil: They are only under an obligation to micro-document as much as they may reasonably consider to be significant."

No. If they expect their research to be viewed as science, they are under an obligation to meet science's basic expectations.

"Reasonably consider to be significant" is never described in the Wikipedia page when describing the scientific method. That would be like having a build for a large software system where some steps were done manually. What sense would it possibly make to not carefully document every step you did?

How could the CRU ensure reproducibility of their results if they didn't have scrupulous documentation?

"We can judge whether this theory (that the individual adjustments to the extent that some *may* be poorly documented did not exert a significant effect on the results) by testing the adjusted and unadjusted data."

This ignores the fundamental question: why would one be slipshod with their management of the data manipulations? You keep hinting that these guys should be given a waiver for the basic expectations of science.

Anyone who has looked at the HARRY_READ_ME file will get a taste of the "rigor" that the CRU was using with its manipulation scripts: http://www.climate-gate.org/cru/documents/HARRY_READ_ME.txt . It has the flavor of a high-schooler doing midnight hacks and not what should be a respected organization practicing science in a rigorous and reproducible fashion.

Have you read through the HARRY_READ_ME file?

"But to deny the validity of the conclusions based on adjustments which can be shown statistically to be neutral, and where the major results have been replicated multiple times, is not reasonable."

That's not what the ongoing research into the whistleblower files is showing. Major discrepancies from the CRU reports and reports from other research institutes are showing up. The darn hockey stick keeps disappearing!
Anyone interested should look at http://pajamasmedia.com/blog/climategate-faster-and-faster-the-dominos-fall/

One wonders if the AP will now reopen their analysis of the Climategate whistleblower files.

"Phil: From what I can see, it had absolutely no effect at all: Climategate and other sideshows (fake and real) were given the credibility they deserve: things for scientists to work through."

No agreement was achieved at the Copenhagen meeting. We have no idea why the developing nations failed to trust the Western countries, but you can bet their officials are far less complacent about the Climategate whistleblower files than you are. You have failed to respond to the latest report on what the bloggers have found: http://pajamasmedia.com/blog/climategate-faster-and-faster-the-dominos-fall/

Having the scientists that created this mess "work through" the problem is nonsensical. The researchers need to be held to account for their failure to meet the basic requirements of science.

"The third-world countries wanted more action on climate change not less."

I don't know what "action" means. Third-world countries were definitely interested in receiving more funds from the western world. Which third-world leader thought $1,000,000,000,000 (a trillion dollars!) in funds transfer would be appropriate?

"Was it any county's position that there was no climate change?"

I don't think officials were polled on that question.

"At least Copenhagen gives the greenies a chance at the kind of nutty hyperbole that has been the skeptic's forte: I enjoyed Greenpeace's press release 'The city of Copenhagen is a climate crime scene tonight, with the guilty
men and women fleeing to the airport in shame.' "

I recommend you ignore the extremists on any side. However, any attempt to lump the researchers and bloggers into the Climategate files with extremists is rather silly.

http://pajamasmedia.com/blog/climategate-faster-and-faster-the-dominos-fall/

Phil: The source is a press statement from an economic think tank in Russia. We have a lot of non-scientific think-tanks driving this, don't we, such as the ronin ex-tobacco lobbyists looking for new masters, such as the CEI?

The article also quotes Dr Keen, who seems to use unadjusted data and does not do geographic weighting: I am not suprised he gets different lines: it in no way refutes the IPCC lines, it seems to me, it just means we have to be clear about what the lines mean, and not read to much in. Which brings us back to the CRU point, that the lines are not as interesting as the margins for error of the points: Keen does not say whether his figures are within one or two standard deviations of the CRU figures, for example.

The article says "It ought to be said that these adjustments don’t necessarily prove the data has been purposefully fudged. Handling this kind of large data set requires using statistical techniques to try and make the data consistent." But then it goes ahead to make exactly those allegations anyway.

"Phil: The source is a press statement from an economic think tank in Russia. We have a lot of non-scientific think-tanks driving this, don't we, such as the ronin ex-tobacco lobbyists looking for new masters, such as the CEI?"

"Ronin ex-tobacco lobbyists" is a troll comment. This is not the first time you have used the "tobacco defense" in talking about Climategate. That dog don't hunt. Please stop.

There were actually four different discussions in the PJ Media article you are discussing. Rather than discuss the one that launches you into a non-sequitur debate, feel free to discuss any of the other three.

"The article also quotes Dr Keen, who seems to use unadjusted data and does not do geographic weighting: I am not suprised he gets different lines: it in no way refutes the IPCC lines, it seems to me, it just means we have to be clear about what the lines mean, and not read to much in. Which brings us back to the CRU point, that the lines are not as interesting as the margins for error of the points: Keen does not say whether his figures are within one or two standard deviations of the CRU figures, for example."

Nobody can know what the CRU did. Maybe even nobody at the CRU knows what they did. We've talked about the README_HARRY file several times and the disaster of data manipulation it indicates.

Did you at least scan through that file?

After doing that, can you honestly say that the CRU was doing real science?

"The article says 'It ought to be said that these adjustments don’t necessarily prove the data has been purposefully fudged.' "

Bingo. They failed to maintain precise enough records of what they did in order for anyone to know what they did.

Your mindset seems to be that someone must prove that the CRU did bad science. This is absolutely positively completely backwards. The onus is on the CRU to demonstrate that they did good science, and they are not doing that. The CRU has failed to ever account for the massively haphazard mismanagement they did with their data manipulation scripts. As Yoda would say: Science it is not.

If you don't learn anything else in this discussion, you should learn that. Please go read the Wikipedia article on the scientific method. Then read Galen's comments that were added earlier today to the thread.

"Handling this kind of large data set requires using statistical techniques to try and make the data consistent."

How interesting. Why exactly do you think they could not have been rigorous with their handing of data manipulations?

"But then it goes ahead to make exactly those allegations anyway."

No. This is absolutely totally backwards. The job of a scientist is to demonstrate that they're doing science. The CRU has failed to do that, and you're complacent about that failure.

Please go read Galen's comments. I am not formally a scientist, but I've learned enough to know what the difference. I'm sure you can, too.

Regarding your "looking outside the window", I don't think that skeptics argue that there is some climate change and that CO2 concentrations have gone up. What they point out is that the IPCC report fails to prove the link, but rather assumes it.
Furthermore, recent studies also indicate that the effect of CO2 was probably overestimated and other factors underestimated.

Such claims are not careless and seem worth addressing, rather than dismissing.

Readers: Sorry I am away on holidays this week. So no detailed responses from me.

Julien: Thanks for bringing that up.

Just to go back to what I wrote:

"I can understand that the extent of the impact of human activity can reasonably be debated; but I find it difficult to understand those who first insist that there can be no human impact, and then go from there to deny there is any climate change at all, or at least that there needs to be no real human response to climate change."

What I would like to see from the AGW sceptics who acknowledge some climate change, is actual positive programs on how to mitigate the effects: what our human response should be (I don't mean emotions, I mean policies.) For example, if we will get more El Nino Modoki in the future, what should our response be?

Mr. Rick Jelliffe: As someone who has read your discussion between you and Phil Earnhardt, I would like to simply repeat what Phil has said earlier:

-----------------

Rick: You didn't read what I said:

They not only threw out their original data, but they also do not have an archive which scrupulously documents what data they altered, what data they didn't alter, and how exactly they manipulated the data that they did alter.

Since there are not scrupulous records of what manipulations were done to what pieces of data, you need to spell out exactly how this is repeatable/verifiable science.

Blindly asserting something that you hope is true is not sufficient. The basic requirements of science mandate that all manipulations to the data were scrupulously documented. If Full Disclosure was not performed, then the basic requirements of science were not met.

-------------

Now, I don't know if you're a scientist or what, Rick. From your position I would say you are not. I am. And Phil is correct. You are not.

Period.

Or do you have some egregious excuse for their horrible behavior?

Any scientist that destroys his raw data destroys himself. His pronouncements are worthless; baseless.

Galen: I certainly do not hold myself out to be a scientist.

For "destroys his raw data" please see the other dozen times I have discussed this incorrect information. For the desirability and importance of keeping metadata, see my blog on XML for practical issues, and my general comment that CRU (and similar bodies, and I don't exclude contrarian climatologists) needs funding for this. For one statistical angle on whether adjustments introduced a bias, please Google "Are the CRU data suspect?" at RealClimate.

The dangers of, or limits to, drawing conclusions in numeric data where there are discontinuities and adjustments (including some whose particulars were not kept, even if though the method was described in papers in enough detail to allow replication) is not some kind of gotcha, but the basic issue dealt with by the HadCRUT3 paper.

Is repeatedly bringing up as objections points that were explicitly dealt with in the paper that explains the data, yet utterly ignoring that paper, really the hallmark of a credible scientist? Please allow my eyebrow to raise.

I take it that as a scientist you simply would not have made comments on the data without looking at the accompanying paper, of course: it would be worthless pronouncements indeed. But I am looking in vain at serious discussions of the statistical methods and approaches of that paper: whether standard deviations are an appropriate metric and so on.

Sorry, Rick, but you lose on this entire encounter. I get the distinct impression that you are either ignoring Phil Earnhardt's comments or you simply don't understand them--although there may yet be a third reason: I suspect you have so deeply drunk the Kool Aid from the Liberals, Warmers, and other nefarious denizens that you're with them. I've run into a few of these myself, and they're illogical, to the point of being stupid. That's right: with all the explanation Phil has outlined over and over and over, you just don't get it, won't get it, or because of a prior commitment, can't get it.

That's sad. But nothing I can do about it, and now that the "cat is out of the bag" about "global warming" (with which I agree, but not to the degree the AGW people hype it, and certainly not to the conclusion they espouse), there is nothing you can do to stop the exposure of the egregious efforts some have taken to distort the science. I have worked both as a scientist (BS and MS in geology) and as an engineer (BS and ME mining engineering) and in both positions I've been responsibile for projects that required a significant capital outlay ($1.5 billion (with a "B") on one project alone). And you know what? Before they signed off, they wanted my data, my thought processes, my justifications, my notes, even any napkins I may have written on at lunch.

EVERYTHING!

So, regardless of how you want to look at what the "scientists" did, their results aren't worth a DIME until they show everything to everybody--They're asking that we pony up $Trillions and don't have the data? They fudged the data? They hid the data? They hid methodology? We don't know--and YOU don't know. But there's a stench about this that I can smell in Peoria.

We are entitled to see every email, every paper, every calculation, every napkin. (Oh, and every travel expense and where all that grant money went to, too.)

Period.

But then again, since you don't work in the realm of science or engineering, you don't get it.

Just don't expect those of us who work as scientists and engineers to ever agree with you.

Ever.

PS. This is why roads, dams, skyscrapers, and aircraft carriers are built by scientists and engineers. You don't see Liberal Arts people doing it.

Gareth: Well, I don't think about comments in terms of winning or losing.

I don't see how you can say I am ignoring Phil's comment about Wikipedia when I have directly addressed it (multiple time.) And I still don't have an actual answer to that response (that the HadCRUT3 paper which explains the data directly and explicitly addresses the problem, expressing it as statistical uncertainty). Instead we get boilerplate about Wikipedia.

"Gareth: Well, I don't think about comments in terms of winning or losing."

There's a disconnect between saying you don't hold yourself to be a scientist and your apparent unwillingness to learn anything on the topic.

"I don't see how you can say I am ignoring Phil's comment about Wikipedia when I have directly addressed it (multiple time.)"

No. You have never ever reconciled the CRU's lack of management of their data-manipulation scripts with the Wikipedia's fundamental definition of science.

"And I still don't have an actual answer to that response (that the HadCRUT3 paper which explains the data directly and explicitly addresses the problem"

No. There is no explanation at all in that paper why the CRU failed to design their record-keeping so they kept track of their data-manipulation scripts.

"expressing it as statistical uncertainty). Instead we get boilerplate about Wikipedia."

Nonsense. The Wikipedia article clearly and succinctly expresses the Basic Requirements of science.

You have never ever reconciled the CRU's failure to meet those Basic Requirements.

The CRU has never reconciled their failure to meet the Basic Requirements of science, either.

"Galen: I certainly do not hold myself out to be a scientist."

It is not a prerequisite to understand why the CRU methodology was fundamentally flawed to disqualify it from being scientific research.

All you really need to do is to ponder the Wikipedia article on the scientific method and see if the CRU research qualifies.

For you to do this objectively, you might want to imagine that the CRU researchers were doing something other than global warming research.

"For 'destroys his raw data' please see the other dozen times I have discussed this incorrect information."

No, you have not. You have never ever given a justification for the CRU's gross mismanagement of their data manipulations over the years.

Let's break it down. From the Wikipedia definition:

"Among other facets shared by the various fields of inquiry is the conviction that the process be objective to reduce biased interpretations of the results."

Among other things, legitimate researchers would be highly suspect discarding any data (or metadata) because they presumed it would be "unimportant". Such decisions would be highly dangerous, because biases of the researchers could have have influenced those decisions.

"Another basic expectation is to document, archive and share all data and methodology"

It should be obvious to anyone looking at the whistleblower files that the CRU has failed to do this. The HARRY_READ_ME file is particularly instructive to see just how little control the CRU had on their methodology.

"so they are available for careful scrutiny by other scientists, thereby allowing other researchers the opportunity to verify results by attempting to reproduce them."

Unless the manipulations are under rigorous versioning, it is problematic for outsiders to scrutinize their methodology and get reproducible results.

"This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established."

Rick: do you now understand that the CRU scientists fundamentally failed to practice full disclosure?

"For the desirability and importance of keeping metadata, see my blog on XML"

Whether or not the researchers kept metadata is a side issue. What was needed was some way for them to preserve their data and manipulations in a way that their research was reproducible. They failed to do that.

"for practical issues, and my general comment that CRU (and similar bodies, and I don't exclude contrarian climatologists) needs funding for this."

All research needs funding to meet the Basic Requirements of science. Besides funding for the research, it is mandatory for researchers to ensure that they have sufficient funding to meet the Basic Requirements of science.

Science is objective and impartial in these requirements; it is required of all scientific research. The Climate Research Unit of the University of East Anglia was not given an exception to the Basic Requirements! They must follow them, or their research is not science.

"Is repeatedly bringing up as objections points that were explicitly dealt with in the paper that explains the data, yet utterly ignoring that paper, really the hallmark of a credible scientist?"

What does utterly ignoring the Basic Requirements of science do to your argument? You seem to understand that those requirements exist, but you seem to be implicitly give the CRU research A Pass on their research. That is mystifying.

"I take it that as a scientist you simply would not have made comments on the data without looking at the accompanying paper, of course: it would be worthless pronouncements indeed. But I am looking in vain at serious discussions of the statistical methods and approaches of that paper: whether standard deviations are an appropriate metric and so on."

You can't see the forest for the trees. Arguing about specific details is again ignoring the most fundamental question raised by climategate:

Please explain why you think that the CRU should be given A Pass for their failure to meet the Basic Requirements of science.

Phil wrote: "Please explain why you think that the CRU should be given A Pass for their failure to meet the Basic Requirements of science."

The question is whether the conclusions they draw from their data goes beyond what may legitimately be derived from that data. Your view is that if even a slight bit of the data chain is missing, the whole tree is poisoned and nothing can be used, it seems to me.

Their view, as expressed in the HadCRUT3 paper, is that where the data set must be derived from data in all sorts of conditions, provenances, multiple and shifting locations, changes in recording instruments, and unfortunate historical lapses in record-keeping (as the temperature data is, regardless of anything CRU has done), then those problems can be estimated statistically and expressed as statistical uncertainty.

This seems reasonable, documented, replicable, objective and discussable: the kinds of thing we want in science.

Is it your theory, then, that Wikipedia precludes the use of statistics in this fashion? I presume this does not go for all statistics in science, because otherwise there would not be much science left. It is not a "detail."

“Phil wrote: ‘Please explain why you think that the CRU should be given A Pass for their failure to meet the Basic Requirements of science.’

“The question is whether the conclusions they draw from their data goes beyond what may legitimately be derived from that data.”

No, that’s a distortion of the question I have asked, repeatedly, over the last six weeks. The specific question I am asking you to explain why any physical research project that mismanages its data manipulations as grossly as the CRU did should expect that research to qualify as science.

“Your view is that if even a slight bit of the data chain is missing, the whole tree is poisoned and nothing can be used, it seems to me.”

It’s hardly my personal view. It is the fundamental definition of science. If you scrupulously manage your data and metadata, they are available for the careful scrutiny of other science. If there is not management of the data and metadata, your research fails to meet the basic requirements of science.

I have referenced that fundamental definition of science over a half-dozen times in this discussion: http://en.wikipedia.org/wiki/Scientific_method . You have acknowledged that you do not hold yourself out to be a scientist, and I haven’t seen any indication that you have educated yourself on its fundamentals. The first three paragraphs of that article spell it out the fundamentals remarkably well. Please read them now.

Those rules for science are qualitative rules, not quantitative ones. There is no such thing as a little failure to keep your data and metadata in control. The reason for that is explained clearly in that Wikipedia article.

“Their view, as expressed in the HadCRUT3 paper [SNIP]”

Once again, that is not the question I am asking. I’m asking why they didn’t keep their data and metadata in control. The HARRY_READ_ME file clearly indicates that the CRU was haphazard in their applications of data manipulation scripts to their data.

I have yet to see any writeup where the CRU attempted to justify that categorical failure to manage their data manipulation scripts. It’s clearly not a lack of source code control tools: those have been available on UNIX systems since the early 1970s. It’s also not attributable to a lack of disk space: managing the actual manipulation scripts would have taken a negligible amount of space.

“This seems reasonable, documented, replicable, objective and discussable: the kinds of thing we want in science.”
Nowhere in the Wikipedia article does it say that these practices are a good idea. They are the basic requirements for science. Individuals and organizations are welcome to do research without rigor, but they forfeit the right to call such research science. There’s an excellent short paper: “The recently recognized failure of predictability in Newtonian dynamics” by Sir James Lighthill ( http://www.jstor.org/pss/2397780 ). Its beginning gives a great background story for what scientific rigor over time accomplished in the time of Newton.

“Is it your theory, then, that Wikipedia precludes the use of statistics in this fashion? [SNIP]”

No. I said no such thing. You’ve spent the entire posting trying to put words in my mouth. You did everything but answer the question I actually asked: Please explain why you think that the CRU should be given A Pass for their failure to meet the Basic Requirements of science. An example of their mismanagement is their failure to track which version of their data-manipulation scripts they applied to what data.

I see no reason to give them A Pass, but your messages here imply that you do. Please explicitly say: are you actually giving them A Pass for their lack of rigor?

If you are giving them A Pass, please reconcile why you are doing that against the Wikipedia’s definition of the Scientific Method. Thanks!


At issue is the fact that supporters of AGW are claiming that the climate is changing and reaching temperatures higher than any time in the past. This change needs to be stopped if us poor humans are to survive. What Climategate has shown is that much of the AGW claims may be solidily based on bad data, or flawed methodologies, etc.,

For instance the 1990 IPCC included a graphic showing a prominent medieval warming period and Little ice age. later versions of the IPCC reports focused attention on the Mann "hockeystick" graph that minimized the MWP & LIA and showed that from 1900 to present the temperature drastically increased (the blade of the stick). Bolstering the claim that AGW was the cause of the temperature increase.

Now the data for most of this is from proxies (since we didn't have temperature monitoring stations earlier than 1865 (I beleive)). The hide the trick issue is crucial here. The proxies discussed here is the use of tree-rings to derive temperatures. A model, formula, methodology, is developed that measures the width etc of the tree-rings and then calculates the corresponding past temperature.

In 1960 there is a divergence. The tree-ring proxies do not match the instrumented temperature. The tree-ring model shows temperatures lower than the recorded instrumented temperatures. While the "trick" is to use the real measured instrumented temperatures (from 1960 forward) this divergence should be seen as massively damaging to the use of tree-rings as temperature proxies.

If the tree-ring formulas don't accurately reflect the instrumented temperatures from 1960 on then how are we to beleive that these same formulas accurately reflect the temperatures in the past. Isn't it feasible that they may have been increased temperatures in the past that the tree-rings don't accurately reflect? If you are intectually honest the answer to that question must be yes that it is possible that the tree-ring formulas may not accurately derive the temperatures of the past.

As far as bad methodology. Mckitrick has documented fairly well the flaws in Mann (MBH98 & MBH99) that developed the "hockey stick" that is the poster child for showing global warming. This site contains his papers that show how Mann cherry picked data and how his formulas "forced" the hickey stick into being. http://climateaudit.org/multiproxy-pdfs/

Pay attention to MM2003 and MM2005 as these documents clearly lay out the flaws in the data crunching process that Mann used. Hell Mann even used tree-ring proxies that the authors of the tree-ring proxy study recommend not using because the sample size is too small and it are these flawed proxies used with Mann's flawed formulas that create the Hockey stick that purports to show that AGW is a dire situation that must be corrected.

"Gareth: Well, I don't think about comments in terms of winning or losing."

There's a disconnect between saying you don't hold yourself to be a scientist and your apparent unwillingness to learn anything on the topic.

"I don't see how you can say I am ignoring Phil's comment about Wikipedia when I have directly addressed it (multiple time.)"

No. You have never ever reconciled the CRU's lack of management of their data-manipulation scripts with the Wikipedia's fundamental definition of science.

"And I still don't have an actual answer to that response (that the HadCRUT3 paper which explains the data directly and explicitly addresses the problem"

No. There is no explanation at all in that paper why the CRU failed to design their record-keeping so they kept track of their data-manipulation scripts.

"expressing it as statistical uncertainty). Instead we get boilerplate about Wikipedia."

Nonsense. The Wikipedia article clearly and succinctly expresses the Basic Requirements of science.

You have never ever reconciled the CRU's failure to meet those Basic Requirements.

The CRU has never reconciled their failure to meet the Basic Requirements of science, either.

Phil: I am not going to be railroaded into taking a hard stand on something outside my expertise.

I think you are promoting the fallacy of the excluded middle, as if science and scientific practice does not progress. Where the records keeping fell short of the ideal, statistics was used to make good. It looks like you are trying to get evidence excluded on a technicality, like the lawyers on American court dramas.

I note that Wikipedia says "The term reproducible research was first proposed by Jon Claerbout at Stanford University and refers to the idea that the ultimate product of research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. necessary for reproduction of the results and building upon the research" and references a 1995 paper.

So the idea that *everything* computerized needs to be kept (and available at a button push) was not accepted wisdom nor the status quo until comparatively recently. (And see my blog on Climategate and XML on practicalities for this.) From what I can tell, it is increasingly required: for example, many scientific journals want to review all the information now.

And I also note an interesting quote here:

Statistical power was also a consideration as research moved out of a controlled experimental environment and into real-world data. An oceanographer noted:

Reproducibility is not a huge concern for us. Most of what we do is concerned with analysis of events in a statistical manner. More observations just get added to the statistical pile. If an observation shows a process happening differently than a previous observation, it does not negate the previous observation, it just indicates that a current system has more than one 'mean' state, or that not enough observations have been performed to achieve a statistical mean.

If you were writing things like "It is better to use the other data sets rather than HadCRUT3 because of these problems" then you would not be going much further than Prof Jones himself, by the way. But since these other data sets mainly replicate HadCRUT3, I suppose that would not serve your purpose would it?

Why exactly do you characterize the Wikipedia article about the scientific method as "boilerplate"?

Do you think the definition there is somehow inaccurate? Is there some other definition you would rather use?

It's disingenuous to simply assert there is some flaw with that definition and then fail to explain that assertion. I hope you appreciate the irony: that is exactly what the CRU got caught doing. Stonewalling doesn't work for science, and it doesn't work for a discussion about science.

Please explain yourself, or drop the nonsense claim. Thanks, Rick!

Second question: why did you launch the ad hominem circumstantial attack of Charlie Martin in your posting on December 11, 2009 9:31 PM? YOu have never explained that attack, either.

Phil: I wrote (in a new thread) "There are a few links on this page to a site called Pyamas Media. I have just looked at it, and I let readers draw their on conclusion about a site peddling articles such as these..."

I don't believe this is an ad hominem attack. But I am not going to let people put links to what appear to me to be racist and nutty political sites without alerting readers. I recommended Martin find a less cranky outlet.

"Phil: I wrote (in a new thread) 'There are a few links on this page to a site called Pyamas Media. I have just looked at it, and I let readers draw their on conclusion about a site peddling articles such as these...' "

And I followed up by looking at one of those articles -- pointing out that it was an imminently reasonable argument. You never followed up on that discussion.

"I don't believe this is an ad hominem attack."

Actually, it was an ad hominem circumstantial attack. It's an attack based on the perceived bias of a source. And you didn't even defend your claim.

"But I am not going to let people put links to what appear to me to be racist and nutty political sites without alerting readers."

That makes no sense. Even if your assessment of the website were defensible, what point did it have on Charlie's contribution to this discussion?

"I recommended Martin find a less cranky outlet."

I recommend you stop trying to make ad hominem circumstantial attacks. Argue based on the merit of the discussion.

Phil: I cannot apologize that I found PJM to be a site characterized by rather nutty and even vile content when I looked. (Others have noticed too.)

Nor can I apologize for apologizing to readers for disavowing it.

Nor can I apologize that you found one of my examples of nutty material to be quite reasonable. In an article whose main point is The Party wants the Earth to be warming, so that its members can establish their power over every aspect of our lives you only see the anecdote on Gödel's Incompleteness Theorem? Pullease...

I cannot apologize for an ad hominem attack because I don't believe this was a direct ad hominem attack (of any type): I specifically did not reference any commenter by name by arguments, and I did it in a different thread. I wrote it after being horrified by the content I read on that day.

That my original comment may *indirectly* reflect poorly on a commenter would require readers to do more detective work than I would expect them too. I do not want links to "extreme, nutty and borderline racist" websites in comments on my blog. I wouldn't want links to Nazi sites, to KKK sites, or to LaRouche sites, either. That commenter should consider moving to a different blog site if PJM does not improve.

(And I also make a general comment on credibility. An ad hominem attack does not alter the truth value of a logical proposition: however, whenever it is a matter of weighing up testimony--of credibility--it is completely legitimate to consider the circumstances. If a mental patient tells me they are Napoleon, they may in fact be Napoleon regardless of their circumstance: but their circumstances would reasonably disincline me from accepting their word without much more scrutiny and evidence; if a doctor told me the patient was in fact not Napolean, it would be reasonable to give their opinion more weight, where judgment had to be made. The onus would be on Napoleon.)

Golly, this is still going?

Rick, you might want to research the ad hominem (circumstantial). I used to teach this stuff — claiming that a particular point, or any article in PJM is flawed because it is published in association with other things you dislike is a textbook example of the ad hom circ.

(Frankly, I think some of the things about "power and control" are a little looney myself. But PJM isn't the product of one giant non-corporeal Group Mind; the opinions of each author are his or her own. Questioning one article on the basis of others not by the same author, or by commenters, is simply fallacious.)

In the mean time, it seems to me that your argument has been rather overtaken by events. Even neglecting the initial fallacy of global climate change being reflected by what you see out your window — somehow, it always seems weather is climate only when it appears to confirm climate change, never when it seems to contradict climate change — in the intervening three months there have been a number of further revelations:

* poor sourcing in the IPCC AR4 report, using nonreviewed advocacy cited as science — the underlying cause of the 2035 error, it now appears;

* admitted violations of the IPCC's rules, such as Dr Murari Lal's admission that the AR4 report was purposefully slanted to have a political effect (see the interview in the Daily Mail 24 Jan 2010);

* multiple independent studies showing an apparent selection bias in sites used in all three of the major historical temperature data sets (HADCRU, NCDC, and GHCN), and counter-intuitive urban heat island corrections that appear to correct rural sites to match urban, instead of the expected reverse;

* the admission on Phil Jones part (in an interview with the BBC) that he in fact isn't good at keeping records, can't reconstruct how some of the CRU dat set was computed, and that the strength of a number of the factors supporting CO2-forced AGW have been overstated, leaving the science unsettled and the question still open in his opinion;

* financial interests in continuing the march toward CO2 regulation on the part of advocates (Gore, David Blood, Richard Sandor, Maurice Strong, and others) that dwarf by a factor of hundreds to thousands the supposedly disqualifying financial interests of its critics.

Because of these facts, there have now been multiple calls by a number of climate scientists, many of them in the pro-AGW camp, for a radical restructuring of the IPCC and increasing pressure for its Chair, Rajendra Pachauri, to resign. The World Meteorological Organization and the UK Meteorological Office have called for a new historical climate data set, constructed openly, with fully peer-reviewed sources and methods. And a number of the assertions about the CRU's problems made in December are now confirmed by Dr Jones himself.

The argument will certainly continue; it ought to take into account the facts.

Charles: Yes, still going!

On your new point, which is the "the initial fallacy of global climate change being reflected by what you see out your window". First: if there is global climate change, there *must* be some windows out of which it will be visible, I think you will agree. Second: my point related the weight of evidence when there are opposing views, not syllogisms: note the use of "fits in...more". Third: the global warming theory says that we should expect these kinds of floods, the anti-global warming theory says we should not expect them: therefore the global warming theory is a better model at the moment (which does not mean it is correct, of course.)

On the other points:

* On the issue of PJM, what would you advise? If I find links to what I found to be an offensive, racist and nutty site, should I i) just remove all links, or ii) put a new comment up disavowing the links, but specifically do this out of the context of the thread where the links were, not mentioning indivudals by names, nor the particular arguments, and continuing the discussion in that thread on the issues raised? I chose ii).

* On the 2035 error, see my comments on link #comment-2337287 That such a tangential error is taken as somehow core, shows a willingness to misrepresent by some parties.

* On Dr Lai, the IPCC guidelines say that the document should "be neutral with respect to policy" (i.e. particular policy) not that it should not "impact policy-makers or encourage them to take some concrete action" (which is what Dr Lai said.) Indeed, the preface of the IPCC report explicitly says "This report is intended to assist governments and other decision-makers in the public and private sector in formulating and implementing appropriate responses to the threat of human-induced climate change." I am afraid this is pretty typical: you guys take words, find the most lurid interpretation, then push that. Dr Lai did not make an "admitted violation."


* On these "multiple independent studies" is science finally intruding in the argument? Please don't ask me to jump onto each bandwagon, I like to see what the informed discussion and responses are. I will be interested to see what AR5, the next IPCC survey, has to say about them. (B.t.w. I see the IPCC is having an independent review of their procedures preparatory to AR5, which sounds good.)

But I wrote originally that this is a complex issue.

* Prof Jones also has said that people who don't find his pioneering work satisfactory should look at the replications of the results (in NCDC and GHCN.) I will be interested in the result of the inquiry at UEA. But I have mentioned repeatedly that since the basis of the HADCUT3 work, as given in its explanatory paper, was to use statistical significance to overcome deficiencies in the data (of all causes) by feeding them into the uncertainty numbers, the idea that deficient data undermines the result is incredibly puzzling: HADCRUT3 *starts* from the position "our data and records have problems, what is knowable from it at what levels of uncertainty."

In any case, aren't you conflating two distinct issues: climate change and AGW? The HadCRUT and other data sets cannot prove AGW, because they measure an effect not a cause: all they can demonstrate is climate change.

* That you mention financial interest in the same post as complaining about supposed ad hominem attacks, suggests that you may need to look up your notes yourself.

But it is a total red herring in any case. Is merely mentioning the name of an American politician bogeyman supposed to be enough to make us in the rest of the world see how deluded we are? Is it your argument that the large coal/oil/carbon producing companies do not have an short-term economic interest in maintaining the status quo? Or that only the wealth of individuals matters, that somehow corporate interests (let alone ronin lobbyist firms) must not be involved?


http://www.timesonline.co.uk/tol/news/environment/article6994774.ece

"The 2007 report, which won the panel the Nobel Peace Prize, said that the probability of Himalayan glaciers 'disappearing by the year 2035 and perhaps sooner is very high'."

Their apology noted:

“In drafting the paragraph in question the clear and well-established standards of evidence, required by the IPCC procedures, were not applied properly,”

No kidding! By inserting that statement, they violated the IPCC's agreement of using only published science. Maybe the IPCC's Nobel Prize should be revoked.

The Copenhagen shoe got dropped this week:

"UN abandons climate change deadline" http://www.ft.com/cms/s/0/87479ee2-0600-11df-8c97-00144feabdc0.html :

"The timetable to reach a global deal to tackle climate change lay in tatters on Wednesday after the UN waived the first deadline of the process laid out at last month’s fractious Copenhagen summit."

What linkage did these two events have? Clearly, the IPCC's tendency for sensationalism over science has cost them credibility in the eyes of third-world nations -- several of which are downstream from those Himalayan glaciers.

We're still waiting for you to reconcile the CRU's failure to manage their data manipulation scripts against the fundamental definition of the scientific method.

You seemed unhappy with the Wikipedia's definition of that concept -- but never explained why. Is there some other definition you would prefer?

The IPCC Himalayan Glacier error and cover-up continues to paint a bleak picture for the errors of climate scientists: "UN climate change expert: there could be more errors in report" ( http://www.timesonline.co.uk/tol/news/environment/article6999051.ece?print=yes&randnum=1264343222952 )

The most telling quote in the article is from Pachauri. He was the scientist who was attributed with the 2035 number by a reporter -- and that number somehow wandered into the IPCC's report.

"Dr Pachauri also said he did not learn about the mistakes until they were reported in the media about 10 days ago, at which time he contacted other IPCC members. He denied keeping quiet about the errors to avoid disrupting the UN summit on climate change in Copenhagen, or discouraging funding for TERI’s own glacier programme."

"But he too admitted that it was 'really odd' that none of the world’s leading glaciologists had pointed out the mistakes to him earlier. “Frankly, it was a stupid error,” he said. “But no one brought it to my attention.”

Did none of those glaciologists read this seminal IPCC report? If so, did they not realize the numerous errors in it? Was it taboo for a scientist in the industry to be a whistle-blower against extreme and sensationalist global warming claims?

Complacency about global warming sensationalism seems to have been widespread. The end of that complacency is definitely a good thing! That is a most welcome change after climategate.

The rest of the article enumerates other errors in the report. It's rather distressing to think that trillion-dollar global policies could be influenced by such a sloppy report.

Have you read that Times article, Rick? Are you bothered by the obvious errors in that report?

Are you bothered that scientists failed to note out the errors in the IPCC report for years?

Phil: Of course there will be mistakes in a large report. It would be incredible if there were only one mistake in a 3000 page report. I hope more mistakes are found, and that many more problems will be found to belong in the 300 year time frame rather than the 35 year time frame.

But, as I have written to you before, please don't hijack the thread into other topics. If you want to write something about Climategate, please write it.

"Phil: Of course there will be mistakes in a large report. It would be incredible if there were only one mistake in a 3000 page report."

That sounds like a red herring. There were individual groups who were accountable -- or not accountable -- for their various sections of the report. Since the IPCC was chartered to use peer-reviewed scientific publications as its sole source, it should be easy to fact-check those sections to ensure that they were correct.

The second issue is the speed -- or lack of speed -- in correcting the errors. THe IPCC failed to take any action to even note the Himalayan error for months.

The IPCC is starting to get the message: the public won't tolerate complacency over errors in documents that could influence the public on trillion-dollwr policy decisions. They were much faster in correcting another error recently noted. See http://uk.reuters.com/article/idUKTRE61C1V920100213

"I hope more mistakes are found, and that many more problems will be found to belong in the 300 year time frame rather than the 35 year time frame."

The fundamental error was including things in the report that were not based on peer review.

Unfortunately, the damage has already been done. India has abandoned the IPCC and set up its own panel. From http://www.ibtimes.com/articles/20100205/india-ipcc-un-climate-change-global-warming.htm

"There is a fine line between climate science and climate evangelism," Ramesh said. "I am for climate science."

All I can say to that quote is "Amen!". ;-)

"But, as I have written to you before, please don't hijack the thread into other topics. If you want to write something about Climategate, please write it."

Sure. Climategate and the IPCC sloppiness are intimately related. It comes with no great surprise that all sorts of revelations are happening with the Climategate principal:

http://www.dailymail.co.uk/news/article-1250872/Climategate-U-turn-Astonishment-scientist-centre-global-warming-email-row-admits-data-organised.html

"Data for vital 'hockey stick graph' has gone missing"
"There has been no global warming since 1995"
"Warming periods have happened before - but NOT due to man-made changes"

He was interviewed by BBC reporter Harrabin:

"According to Mr Harrabin, colleagues of Professor Jones said ‘his office is piled high with paper, fragments from over the years, tens of thousands of pieces of paper, and they suspect what happened was he took in the raw data to a central database and then let the pieces of paper go because he never realised that 20 years later he would be held to account over them’."

Apparently, the professor thought he wouldn't be held to account for having the raw data to support the "hockey stick" graph. He seems to have no idea of how science works. Wow.

"Asked about whether he lost track of data, Professor Jones said: ‘There is some truth in that. We do have a trail of where the weather stations have come from but it’s probably not as good as it should be."

Again, that is an amazing understatement. If he can't maintain data integrity with the published reports, he is not doing science.

Do you defend Professor Jones's actions. What should be the consequences of his failure to meet the Basic Requirements of science in his management of the data?

Phil: First: The 2035 error is clearly being blown up out of proportion. The IPCC reports are all online. There are two wrong sentences in Vol 2 p 493. I invite readers to look at the online PDF and figure out whether the deletion of the two sentences has any impact on the rest of the subsection and the section as a whole: I don't see that it does.

For commentary on known errors so far, as compiled by IPCC fans, see here.

This makes the point that the error in Vol 2 Chapter 10 was in fact reported by an scientist who worked on Vol 1 Chapter 4: so the error was reported from inside the IPCC fold as it were.

Second, Vol 1 is the volume by climatologists, while volume 2 is being done by ecologists and so on. Their mistake was not referencing the Part 1 material. In insubstantial error in Volume 2 cannot be used to ignore the work of the climatologists in Volume 1, it seems to me.

And what do the climate scientists actually say about Himalayan glaciers? First a comment on ice formation:

At high and mid-latitudes, the hydrological cycle is determined by the annual cycle of air temperature, with accumulation dominating in winter and ablation in summer. In wide parts of the Himalaya most accumulation and ablation occur during summer (Fujita and Ageta, 2000), in the tropics ablation occurs year round and the seasonality in precipitation controls ccumulation (Kaser and Osmaston, 2002).

and second a comment on an effect of glacial retreat in certain terrain:
Formation of large and hazardous lakes is occurring as glacier termini retreat from prominent Little Ice Age moraines, especially in the steep Himalaya and Andes.

Again, I note that the 2035 error of Vol 2 is simply not contained in Vol 1, which is the synopsis of the climate science. It is not compromised or based on that slip up.

Third, on the Indian Government, I am surprised they did not have their own glacier body before now. Good for them. Of course no government can "rely" on any external report without checking it. The comment that it is a weakness of the IPCC that it didn't do original research is odd, since its purpose was to summarize.

Fourth, on Professor Jones, in my mind I am only weakly defending him: I am refusing to condemn him: for a start because it is not my expertise. But also, as I wrote above, practice seems to have tightened in the last 15 years: he was operating under the idea that replication means coarse-grain (i.e. anyone can take the station data, apply his described methods and come up with a result) but the world is expecting (and demanding) a fine-grain reproducibility, along the lines of Jon Claerbout. But I know a witch hunt when I see one.

Interested readers should look at the "Himalyagate" material in the IPCC. When you read reports like the one Phil mentions, which says, A primary claim of the report was the Himalayan glaciers could disappear by 2035 and you look at the report and see it is not a primary claim at all, it is just an erroneous fact whose removal changes nothing in the report, the thing is obviously just a beat-up.

All this sound and furry about Hockey Stick World I can't even take the time to follow any more. It's great to see such intense online debate here, but you see I've already formed an anti-AGW opinion based on a lack of even hand-waving criticism about a single cute little graphic I made that plots most of the continuous very-long-running thermometer stations. History can *not* be a Hockey Stick if sites spread across both Europe and North America fail to show *any* sign of that, going back 350 years. This is simple refutation of AGW theory, for if history is not a Hockey Stick then claims of future warming cannot even pass the laugh test any more.

http://i49.tinypic.com/rc93fa.jpg

Nik: Thanks for the graph. You should send it to a site which has scientists and enthusiasts who are capable of discussing it properly.

In my limited understanding, the AGW theory merely talks about net effects, not that everywhere will have uniform warming: some places may cool down. Some times, such in the Darwin (NT, Aust) record, the warming may happen as a lack of cooling at night rather than hotter days. So it is certainly possible that any arbitrary selection of data points might show less or more warming than the global averages or particular pockets.

So I accept that your graphs might *incline* you (but not me) against global warming, as one more thing on your personal scales as you weigh up evidence, if you think it is improbable that the superset of stations would have different values than your particular subset.

However, as I have mentioned before, the key to the HadCRUT3 data is that they base it on estimating uncertainty: what you would have to do is see whether your selection of numbers fit in the uncertainty range.

And, as I mention elsewhere, there is a difference between AGW and Climate Change: it is an extra step in modeling: weather records may suggest Climate Change with some level of uncertainty, climate modeling may suggest AGW to explain the climate change with some greater level of uncertainty: the use of that model to predict the future presumably has some even greater level of uncertainty.

I accept that people have different risk-taking proclivities or personalities here, consequently it does not surprise me that while I might think a 75% certainty of AGW is enough to warrant taking it extremely seriously, others might require a 99% certainty. (However, it seems there are some who require more than 100% certainty: they are against it on some kind of philosophical or perhaps religious/economic grounds.)

News Topics

Recommended for You

Got a Question?