Accurate worldwide temperature records only exist for 100 years or so. In order to estimate temperatures further back in the past, climate scientists use so-called 'proxy' data. A widely used example is tree rings - the assumption is that in warmer weather, trees grow faster and so have thicker rings. However, many other factors may influence tree growth, for example light and rainfall. So it is important to check whether the proxy data does indeed match the temperature as recorded by thermometers.
Take a look at the IPCC's AR4 figure 6.10b. The solid black line is the instrumental temperature record, according to the UK's Hadley Centre. One of the most striking things about this figure is how the 'proxy' data sets completely fail to match the instrumental temperatures in the second half of the 20th century. This is a huge problem for these 'reconstructions' of past climate. Since there is no match between the proxy data and instrumental temperatures in the overlap period, we can have no confidence at all that the 'proxy reconstruction' is an accurate representation of past temperatures. This gaping and obvious hole in the theory has been almost completely ignored by climate scientists and the IPCC. It is sometimes referred to as the 'divergence problem'.
Here is a blow-up of the period 1800-2000 from figure 6.10b, showing the divergence:
Astonishingly, the first and second drafts of AR4 chapter 6 did not mention the divergence problem at all. In response to IPCC reviewer questions, a paragraph was introduced into the final version (and therefore was not subject to any review). This section acknowledges the problem: "Several analyses of ring width and ring density chronologies, with otherwise wellestablished sensitivity to temperature, have shown that they do not emulate the general warming trend evident in instrumental temperature records over recent decades..." and also admits that "At this time there is no consensus on these issues..." and ends with the absurd statement that "...the possibility of investigating them further is restricted by the lack of recent tree ring data at most of the sites from which tree ring data discussed in this chapter were acquired." One might ask the obvious question of why this data is not available - tree ring data is easy to obtain, and given the importance of this question it is very curious that the scientists have not gone out to find it. Perhaps they are afraid of what they might find?
In some cases, climate scientists have even deleted some of their own data, in order to hide the inconvenient divergence problem. This is the case with the IPCC's BOS..2001 data set (From Briffa et al, Low frequency temperature variations from a northern tree ring density network, J. Geophys. Res 106, 2929-2941, 2001). In that paper, "The period after 1960 was not used to avoid bias in the regression coefficients that could be generated by an anomolous decline in tree density measurements over recent decades that is not forced by temperature". The real proxy data showed a sharp fall of 0.6 degrees over the period 1940-1980, so this was simply truncated for the IPCC AR4 report, on the grounds of it being 'anomolous'. By a remarkable coincidence, the lead author of this paper was also the lead author of AR4 chapter 6. The edited dataset is the light blue one in the graph above - if the series had not been truncated at 1960 it would show a more prominent continued downward trend.
There is much detailed discussion of this issue at Climate Audit, for example here, here and here. The deletion of proxy data was a major issue in the "climategate" emails, where the phrase "hide the decline" was used.
An alternative viewpoint on the divergence problem, suggested by Douglas Hoyt, is that the proxies are accurate and that the instrumental temperature record is in error, due to urban heat island effects.