Starting a new Climate-Eval study: Guidelines for climate mitigation evaluation

Last year, there was a lengthy discussion among some members of the community of practice on the possibility of writing guidelines for climate change mitigation evaluations. The preparation of the guidelines is the third Climate-Eval study, and will be conducted in the context of evaluating programs and projects which include the dual objectives of reducing greenhouse gas (GHG) emissions and sustainable development in developing countries and countries with economies in transition. The study will build on the concepts and tools developed in the previous Climate-Eval study "Meta-Evaluation of Mitigation Evaluations". The approach paper describes the purpose and process.

What will the guidelines do?

The guidelines will support project and program managers in need of evaluation; evaluators, evaluation administrators. They focus on defining standards and tools for mitigation evaluations in order to avoid redefining some aspects over and over. The guidelines can answer several key questions:

1. What are the necessary steps when evaluating climate change mitigation interventions?
2. How can the OECD DAC criteria (i.e., effectiveness, relevance, efficiency, impact and sustainability) be applied to climate mitigation projects and be operationalized to determine the success of such measures in relation to reducing GHG emissions and sustainable development?
3. What other criteria are applicable to climate mitigation programs?
4. What evaluation principles and reporting requirements should evaluators consider when evaluating climate change mitigation interventions?
5. What evaluation approaches and tools can assist the formative evaluation and ex-post measurement and calculation of GHG emission reduction?

The study will include a toolbox and an evaluation matrix to aid evaluators in identifying and using appropriate tools and guide questions when conducting their evaluations..

Proposed outline of the study

Guidance on many of these questions can only be given once the subject and purpose of the evaluation is adequately defined. Therefore, the guidelines have two parts: the first part focuses on conceptual challenges: why and what do we evaluate? These are questions linked to program theory and their implications for climate mitigation evaluations. In the first part, a hierarchy of climate mitigation intervention " from the largest to the most detailed level" will be developed to aid the application of theory-based evaluation approaches to the various levels. More details on the outline of the first part can be found here; comments are welcome!

The second part focuses on the more technical and methodological challenges including baselines identification and the operationalization and measurement of criteria and indicators. It is important, once the theory of change is clarified, evaluation questions have been formulated and the program logic is well defined, to operationalize the indicators which should be used to measure success in important parts of our overall transformation towards sustainable economies. Therefore, specific indicators are fitted to the theory of change developed in the first part, and the OECD DAC criteria will be further discussed for climate mitigation projects. Other suitable project and program criteria and indicators will be identified that are applicable to different types of climate mitigation interventions. These are necessary to reflect the extent to which climate mitigation goals, objectives and targets are being met by strategies, programs and projects.

Defining indicators comes along with defining measurement methods for the indicator and giving best practice examples. Climate mitigation evaluation could be much easier if clear guidance existed on measuring outcomes and impacts. Then evaluators would have to reinvent the wheel less often and interventions could be compared better.