Universal metrics to compare the effectiveness of climate change adaptation projects

Adaptation to climate change is increasingly supported through international financing. In contrast to mitigation, where the effectiveness of policy action can be measured through the metric "tonnes of CO₂ equivalent reduced", no universally accepted metric for assessment of adaptation effectiveness exists. Without such a metric, adaptation finance vehicles such as the Adaptation Fund under the Kyoto Protocol encounter challenges when trying to compare the adaptive effect of ongoing or proposed projects in order to achieve an efficient allocation of their funds. The first experiences with adaptation funding show a tendency to use intermediate outcome indicators but no final impact metrics, similar to the state-of-art in development funding. This might lead to a backlash against adaptation funding by electorates in the North if the funding cannot show clear results. We discuss uncertainties encountered in applying these metrics, including the uncertain link between commonly reported outcome indicators and our metrics for saved wealth and health. The two metrics are tested by assessing five adaptation project proposals. Finally, we line out some ideas to handle these uncertainties, e.g. the use of regularly updated sectoral methodologies and agreed climate and economic models.

Current evaluation in multilateral adaptation funding
In contrast to climate change mitigation, the effectiveness of adaptation projects so far is not evaluated according to universally accepted metrics. As adaptation so far was the ?step child? of the climate regime, it was mainly treated as a by-product of development assistance and thus the evaluation procedures of development assistance projects have traditionally been used. These procedures are based on a "results chain", with a series of indicators from input to project impacts. The key feature of this chain is the ?attribution gap? between outcome and impact indicators, which is due to the effect of many non-project related influences on the level of the impact indicators. We find that all analyzed adaptation funds focus mainly on output and outcome indicators, with very few impact indicators being used.

Two potential impact indicators: Saved Wealth and Saved Health
To avoid a situation where adaptation funding is essentially distributed in a ?first come, first serve?
mode, it is imperative to agree on one or a set of impact indicators that allow a choice of adaptation projects according to their effectiveness per unit of money invested. A single, universal indicator is not possible due to the challenges in valuating protection of human lives and health. We therefore propose two main indicators: one assessing economic assets saved from destruction by climate change impacts (Saved Wealth) and one calculating human lives and health protected (Saved Health). The former indicator is a combination of two values, one of which is the absolute value of wealth saved, while the other looks at the relative wealth saved. This is done in order to take into account vulnerability, a major criteria for allocating scarce adaptation funding according to international agreements. The health indicator uses the concept of disability-adjusted life-years saved (DALYs). The indicators are calculated in a procedure that first estimates the expected wealth existing in the project area over time. Subsequently, the expected climate change damages during the lifetime of the adaptation project are calculated to set the baseline. A frequency distribution of damaging events is used for this purpose. Finally, the remaining damages after implementation or the project are assessed.

Further steps to operationalize the impact indicators
To reduce uncertainties that may make the indicator calculation irrelevant, coordinated approaches for the determination of wealth forecasts and damage frequency curves are required. A political agreement on the climate and economic models used would help. Here, the regulatory structure for the evaluation of emissions mitigation projects in the context of the Clean Development Mechanism could serve as example. As an initial step, the indicator systems of adaptation funds should be oriented versus impact indicators. Moreover, indicators should periodically assess the risk of maladaptation, especially for projects with a very long lifetime. The links between outcome and impact indicators should be studied in more detail.

A full copy of the working paper is available for download here.

The working paper was originally presented at the Colorado Conference on Earth System Governance, 17-20 May 2011, Colorado State University, & the 7th Meeting of the OECD Joint DAC-EPOC Task Team on Climate Change and Development Co-operation, 20-21 June 2011