Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product on Temperature Trends in the Lower Atmosphere (2005)

Chapter 2 asks the following two questions: (1) what kinds of atmospheric temperature variations can the current observing systems measure, and (2) what are their strengths and limitations, both spatially and temporally? The chapter concludes that most observing systems are generally able to quantify well the magnitudes of temperature change associated with shorter time scales, such as diurnal and seasonal cycles, quasi-biennial oscillation (QBO), El Niño Southern Oscillation (ENSO), and volcanic eruptions. However, for longer time scale changes, where the magnitudes of change are smaller and the stability requirements more rigorous, the observing systems face significant challenges to document climate variations and trends with the accuracy and representativeness that allows attribution of change to human causes to be reliably identified. Therefore, many sources of errors in climate temperature data records must be identified and eliminated or significantly reduced.

This chapter did a reasonably good job summarizing the main observing systems for measuring surface and upper air temperatures and showing what kinds of atmospheric temperature variations these observing systems can measure using Table 2.2. The strengths and limitations of these observing systems (the second question), however, are not presented as well, mainly because of a lack of quantitative information and some redundancy between Chapters 2 and 4. The chapter contains a fairly lengthy discussion of statistical issues associated with measuring trends in time series, but it omits some key issues such as autocorrelation. This discussion also seems out of place in Chapter 2 since the main treatment of “uncertainty” is in Chapter 4. The statistical discussion should be strengthened and possibly moved elsewhere in the document. One possibility is a self-contained appendix devoted to trends in time series.

1. In Chapter 4, the following is asked: what is our understanding of the contribution made by observational or methodological uncertainties to the previously reported vertical differences in temperature trends? The nature of the discussion in Chapters 2 and 4 needs to be focused to reduce redundancy and avoid omissions in both Chapters 2 and 4. Chapter 2 appears to be focused on answering Chapter 4’s question, rather than Chapter 2’s questions. In general, some of the topical divisions between Chapters 2 and 4 are

artificial, so some redundancy in material presented is inevitable. However, the committee suggests that Chapter 2 focus on the various observing systems and Chapter 4 focus on trends in the observations, as differentiated in the following:

Chapter 2 should focus on:

1. explaining the measuring systems and instrumentation, their accuracy and precision, and spatial temporal variability for global measurements of temperature;

2. addressing measurement issues both for surface temperature measurements and atmospheric temperature measurements;

3. addressing spatial and temporal sampling errors; and

4. discussing any particular geographic regions where measurement and retrieval errors are particularly large.

Chapter 4 should focus on:

1. errors associated with trends; and

2. assessing which of the bias errors in Chapter 2 could influence the trends, and why they do or do not do so.

The discussion related to trend estimation and uncertainties in Chapter 2 should be moved to Chapter 4. Text on reanalysis trends from lines 266-277 should be moved to Section 7 (“Reanalysis”) in Chapter 4. Also, Chapter 4 should add a section on “Methodological uncertainties” by including from Chapter 2 most of the text about linear trends in Section 2b (lines 385-460), discussions on structural uncertainty from lines 480-521, and the summary on “Errors or differences related to analysis or interpretation” from lines 584-601.

Alternatively, all material on trend estimation and uncertainties may be brought together in an appendix to the Temperature Trends report. In addition to the above material, discussion of statistical uncertainty in Chapter 3 (pages 39-40) could be included in the appendix.

2. Quantitative information is needed about the strengths and limitations of the observing systems. Specifically, quantitative discussion of the following sources of uncertainties should be included: accuracy and precision of the sensor, uncertainties in converting the fundamental measurement into temperature, and spatial and temporal sampling errors. There should be a summary of studies (with references) in which the different measurement types (e.g., radiosondes, active sensors, different satellite retrievals) have been intercompared and evaluated on a pixel level.

3. Increased discussion is needed on surface temperature measurements and trends, to parallel the detailed discussion provided on atmospheric temperatures. From reading this document, the impression is given that global surface temperature measurement is a solved problem, but this is not the case. Description of skin and bulk sea surface temperature (SST) in Chapter 4.5.1 should be moved to Chapter 2 and should reference the recent work of Chelton (2005). Errors associated with sea surface temperature measurement are not adequately covered in either Chapters 2 or 4. Discussion of microwave SST and blended infrared/microwave products should be included. Note, the

bulk SST is probably the suitable variable for trend estimation, but the skin SST values are needed to understand the variations in both bulk SST and atmospheric temperatures. Issues related to land-surface temperature measurement (skin versus screen) are not adequately addressed (see Jin and Dickinson, 2002).

4. Because we need to understand the processes contributing to the trends as well as measure the trends themselves, geographical regions having particularly large uncertainty should be addressed. For example, regional problems in surface temperature measurement should be discussed, including the Arctic Ocean and Southern Ocean, warm current regions, and the Indian Ocean.

5. Four and a half pages (pages 9-13) are devoted to “Reanalysis”. Uncertainties in reanalysis trends are nicely summarized, and it is shown how the data are used by National Centers for Environmental Prediction (NCEP)/National Center for Atmospheric Research (NCAR) reanalysis and European Center for Medium-Range Weather Forecasts (ECMWF) Reanalysis (ERA-40). The conclusion is that there are considerable uncertainties in reanalysis trends, so reanalysis results are downplayed and not used in drawing conclusions in this report. The committee agrees with the authors’ decision to deemphasize reanalysis data in the trend analyses in their report. However, this long discussion of reanalysis should be moved from Chapter 2 to page 19 in Chapter 3, where reanalysis temperature “data” are presented. Chapter 2 focuses on observing systems instead of particular datasets, and reanalysis products are not in fact “datasets”. In addition, the four and a half pages of reanalysis discussions seem overly long in comparison to the approximately two pages for surface air temperatures and approximately four pages for upper air temperatures.

6. Scientific justifications for future observing systems are listed in Chapter 6, such as why we need reference radiosondes, but this is not mentioned in Chapter 2. For example, after “no absolute standards” in line 512, one sentence can be inserted to state that reference instruments are needed for future networks, such as the global reference radiosonde network proposed by the Global Climate Observing System (GCOS).

7. The report states that two main methods are widely used for calculating trends: linear regression and a “nonparametric” method attributed to Gilbert (1987). In the statistics literature, a technique is said to be “robust” if it is insensitive to violations of the underlying assumptions (the presence of outliers is one example of how underlying assumptions could be violated). In this sense, linear (least squares) regression is not robust, though it is not clear that this is an issue in any of the climatic time series under discussion. Gilbert’s method does not seem to be widely used, but there are other methods (e.g., methods based on minimizing the sum of absolute deviations instead of squared deviations as in least squares) that have a large literature and should be referenced. These include the use of R functions to perform robust regression (Venables and Ripley, 2002), semi-parametric regression methods (Ruppert et al., 2003) and additive models (e.g., Hastie and Tibshirani, 1990). The distinction between least-squares and robust methods is not likely the main source of uncertainty in analyzing climatic time series. Non-linear trends are discussed on pages 18-19 of Chapter 2, though without making a clear-cut recommendation. It is self-evident that the trend is non-linear over any respectably long time interval, but nevertheless, fitting a linear trend could be the best thing to do if one is simply interested in coming up with one number to represent a trend over a stated time period. In the view of the committee, it is not unreasonable to use

linear trends in this kind of analysis, with two caveats: (i) it is important to remember that linear trends for different time periods will be different, and (ii) such linear trends should not be used for predicting future values. A further important issue is that when comparing observations and coupled model results, ENSO can appear in different sequences and magnitudes, making sampling a major issue. While linear removal of ENSO can ameliorate this problem, it is in fact impossible to remove all ENSO aspects even with multiple indices. As for ENSO (and similar) effects, the report discusses these on pages 18-19 of Chapter 2 but does not mention the most direct solution, that is, including ENSO (or other “natural variability” components) as additional covariates in the regression. There are arguments both for and against doing this, but comparing both analyses could be a useful reality check on the results.

8. The report barely mentions the issues of autocorrelation, i.e., the fact that correlations in time series could severely affect the estimation of a trend, especially in the calculation of standard error. Chapter 2 discusses error bars extensively without mentioning this issue. Chapter 3 mentions it tangentially, with discussion of error bars on lines 876-886 and a passing reference to the first-order autocorrelation in the captions of Tables 6.1 and 6.2, but with no details about the method. Given the importance of correct treatment of autocorrelation in the assessment of linear trends, this seems to be a major omission. The report should acknowledge that autocorrelation is a problem, as it is generally done incorrectly, and recommend how to properly account for its influence. The standard errors of estimated trends, allowing correctly for autocorrelation and other effects, are likely comparable to the “uncertainties” due to instrument shifts and effects of that nature quoted at numerous places in the report. This could lead to a quite different perspective on the relative importance of “structural” as opposed to simple statistical errors.

In fact, the method of Santer et al. (2000) seems to rely on the assumption that after subtracting trends, the time series is of AR1 form, which can indeed be characterized by the first-order autocorrelation. However, the AR1 assumption may not be correct and is certainly unnecessary as it is possible to fit a general ARMA (autoregressive, moving average) model with scarcely any more work. The “arima” function in the freely available R statistical package allows for fitting a linear regression component with ARMA errors, where the autoregressive and moving average components are of arbitrary order. The method is exact maximum likelihood, and standard errors are calculated for both the regression coefficients and the ARMA parameters. It should be noted that earlier versions of this method have been in use in the climatology literature for some time (Karl et al. 1996, 1998). Earlier discussions of time series approaches (e.g., including those based on fractional ARIMA models) have been given by Bloomfield (1992) and Bloomfield and Nychka (1992).

Another issue is whether to include an ENSO signal directly as a covariate in the analysis. In an analysis of annual hemispheric temperature averages, Smith et al. (2003) argued that inclusion of the Southern Oscillation Index as a covariate, though not having a great effect on the estimated trend, allows for specification of a lower-order AR model (AR1 rather than AR4) and in this sense simplifies the analysis. It would be of interest to see whether the same applies with the time series under discussion in this report.

Another method mentioned in Chapter 6 of the report is the adaption of methods from longitudinal data analysis (e.g. medical data in which individual subjects are

followed for some period of time) a book by Diggle et al. (1996) is mentioned in this respect. While it is conceivable that these methods could be adapted to the estimation of trends in climatological time series, it also seems unnecessary, given that the AR/ARMA/ARIMA approach is quite well established. Therefore, discussion of this method should be omitted.

9. Direct discussions with the authors of the report made it clear that they had given more consideration to statistical assessment of trends in time series than is apparent in the written report, but neveretheless, it was the strong view of the committee that the issues should be dealt with explicitly in the report. Based on the overall structure of the document, such discussion would logically belong with the “uncertainty” discussion in Chapter 4 rather than Chapter 2 but the authors might alternatively consider writing a separate appendix on the statistical issues associated with estimating trends in climatic time series.

10. There are insufficient bibliographic references to the technical aspects of temperature measurements and error determination and far too many references associated with climate variability and trends (these are more suitable to other chapters). Recent references (since NRC, 2000) should especially be included.

11. Cross evaluation and intercomparison of different technologies (including surface-based remote sensing) to measure temperature should be described.

1. Observations not used in this report should be mentioned, such as why Television Infrared Observation Satellite Program (TIROS) Operational Vertical Sounder/Infrared (TOVS/IR) was never used for trend analysis. This probably should be mentioned after line 193. A discussion of TOVS temperature profiles is needed. Note, the tuned regression type analysis used by the National Oceanic and Atmospheric Administration (NOAA) is not the only temperature available from TOVS. The Pathfinder effort and French 3I effort represent research-quality retrievals. John Bates at NOAA is in the process of doing a careful calibration of TOVS so that trends can be determined.

2. Table 2.2 is used to answer the first question of this chapter but provides insufficient emphasis on the long-term temperature changes due to anthropogenic effects (i.e., temperature trends), which is the sole focus of this report. It would be useful to add one column to list the “Outstanding issues” regarding specific variation, which includes inconsistencies among different datasets (or observing systems) and what future data are needed for better characterizing and understanding this variation. The column “Effect on trend estimates” needs more quantitative information if available, such as how much the temperature trends change before and after removing ENSO signals in the time series.

3. It appears that Table 2.1 and the text on pages 22-24 were used to try to answer the second question for this chapter. The information given here is too general and too qualitative. More quantitative information and some references should be given on pages 22-24. For example, the authors can summarize how insufficient spatial sampling of the radiosonde network affects the temperature trend from Agudelo and Curry (2004) and others. The authors can give more information about radiosonde errors in the upper troposphere and lower stratosphere—such as radiation errors, their magnitudes and

characteristics—errors in existing radiation corrections and how they affect the trends. Table 2.1 should include specific instruments and pixel size for satellite measurements. Humidity and wind measurements should be excluded from the table, although the authors may want to discuss how these measurements can be useful proxy diagnostics if measured carefully with climate-quality monitoring.

4. In lines 88-89, near-surface air temperatures over land are measured about 1.5-2 m above the ground level at official weather stations, rather than 1.5 m.

5. A reference is needed for this statement.

6. The caption for Figure 2.2 should state that the pressure levels at the y-axis are radiosonde “mandatory reporting levels”.

7. In lines 217-226, the reference for Global Positioning System-Radio Occultation (GPS-RO) is Kursinski et al. (1997). The comparison between GPS-RO and radiosonde data has shown that the GPS-RO soundings are of sufficiently high accuracy to differentiate performance among the various radiosonde types (Kuo et al., 2005). Also, the report should discuss the findings of Schroder et al (2003) on MSU versus GPS. In particular, Schroder et al. (2003) found that UAH T4 retrievals in the Arctic lower stratosphere in winter were biased relative to temperatures derived from GPS Radio Occultation measurements.

8. The statement “the method of calibrating a radiosonde before launch may introduce time-varying biases” in lines 543-544 needs clarification.

9. The references at the end of this review include several additional papers that should be considered for inclusion in Chapter 2 of the Temperature Trends report.