A new paper by Hausfather and his colleagues (including me) has just been published with the most comprehensive assessment of climate model projections since the 1970s. Bottom line? Once you fixed the small errors in the projected overrides, they performed remarkably well.
Climate models are central to our understanding of our future climate. They have also been frequently attacked by those who disdain climate change, who argue that since climate models are inevitably approximations, they have no predictive power, or even, that they are not even scientific.
In one upcoming article in Geophysical Research Letters, Zeke Hausfather, Henri Drake, Tristan Abbott and I examined how accurately climate models were able to project warming in the years after they were published. This is an extension of the comparisons we have been doing on RealClimate for many years, but with a broader scope and deeper analysis. We gathered all climate models published between 1970 and the mid-2000s which gave projections of both future warming and future concentrations of CO2 and other climate forcings – from Manabe (1970) and Mitchell (1970) to CMIP3 in IPCC 2007.
We found that climate models – even those published in the 1970s – performed remarkably well, with 14 of 17 projections statistically indistinguishable from what actually happened.
We evaluated these models both on the quality of the modeled warming relative to the observed warming after the models were published, and on the quality of the relationship between warming and CO2 (and other climate forcings) in models compare to observations (the implicit transient climate response) (see figure). The second approach is important because even if an old model had gotten all the physics correct, the projected future warming would be off if they assumed we would have 450ppm CO2 in 2020 (which some did!). Future emissions depend on the behavior of human society, not physical systems, and we can usefully separate the physics assessment of climate models from the trajectories of future concentrations.
However, it is not entirely obvious how to correct the forcing assumptions due to subtle issues related to the differing effectiveness of different forcings and, of course, the remaining uncertainty about the true value of the forcings. real (mainly related to the aerosol component). For forcing projections that were close to linear this didn’t make much difference, but for scenarios that were not (notably scenario C in Hansen et al (1988)) the correction does not work good.
There are a few other results that stand out, including the (infamous?) low sensitivity result in Rasool and Schneider (1971), which was mainly due to a lack of stratospheric adjustment and absorption of short waves of water vapor in their formulation. This was noted by Schneider (1975) and the calculation redone by Schneider and Thompson (1981) which turned out to be much more accurate. On the other hand, only Mitchell (1970) seems to have largely overestimated the TCR – even though he predicted the rise in temperature quite accurately (due to a compensation between too great sensitivity and an underestimation of the forcings). [Amusing aside, both Manabe’s and Mitchell’s 1970 projections appeared in a special volume on the Global Effects of Environmental Pollution, reporting on an 1968 AAAS workshop and edited by (the now-notorious) S. Fred Singer before he went off the deep end].
It should be noted that this comparison includes two types of climate models – those published before 1988 which are energy balance models of varying complexity, and those published afterward which are true GCMs and include atmospheric (and possibly ocean) dynamics. . Among the first models, the work of Sawyer (1972) stands out as being the most accurate in terms of temperature evolution and forcing, although this must be considered somewhat fortuitous.
The fact that both classes of climate models have been so successful in projecting future warming should increase our confidence that current climate models are doing it right for most of the right reasons. While there are still real uncertainties about future warming associated with climate sensitivity, we can confidently say that the rate of surface warming we experience today is roughly what past climate models predicted.
Gosh, maybe we know something about the climate after all!
Note: All data and code from this study are available here.
Z. Hausfather, HF Drake, T. Abbott, and GA Schmidt, “Evaluating the Performance of Past Climate Model Projections“, Geophysical Research Letters, flight. 47, 2020. http://dx.doi.org/10.1029/2019GL085378
SI Rasool and SH Schneider, “Atmospheric carbon dioxide and aerosols: effects of large increases on global climate”, Science, flight. 173, pages 138-141, 1971. http://dx.doi.org/10.1126/science.173.3992.138
SH Schneider, “On the carbon dioxide-climate confusion”, Journal of Atmospheric Sciences, flight. 32, pages 2060-2066, 1975. http://dx.doi.org/10.1175/1520-0469(1975)0322.0.CO;2
SH Schneider, and SL Thompson, “Atmospheric CO2 and climate: importance of the transient response”, Geophysical Research Journal, flight. 86, pp. 3135, 1981. http://dx.doi.org/10.1029/JC086iC04p03135
“Global Effects of Environmental Pollution”, 1970. http://dx.doi.org/10.1007/978-94-010-3290-2
JS SAWYER, “Human-Made Carbon Dioxide and the Greenhouse Effect”, Nature, flight. 239, p. 23-26, 1972. http://dx.doi.org/10.1038/239023a0