Mainstreaming the environment into evaluations - can we walk the talk?

A lot has been said about the importance of the environmental dimension in evaluations. From the discussion of concepts like “triple bottom line” to the Agenda 2030 efforts have been made to strike a balance between the key goals of international development cooperation (social, environmental, economic) while evaluation should provide us with the evidence on the results of these efforts. In practice, however, there have been barriers to these efforts and a recent stock-taking exercise carried out by UNEG revealed a rather worrying status quo; with evaluation policies, guidance and reports covering environmental aspects to a very limited extent only.

With Agenda 2030 comes the principle of “indivisibility” and a strong call for policy coherence. The world needs to achieve all the goals and we cannot afford some of our efforts to be counterproductive to others. No more “silos” and sectoral tunnel visions. What does that mean for the environmental dimension of international development cooperation? According to UNEP about half of the 17 SDGs and their 169 targets are directly environmental in focus or address the sustainability of natural resources. Therefore, in principle, we can assume that the “planet” agenda is well represented in the global debate and action. But have we made this integrated vision our own?

A major role of the UN is to offer technical cooperation to developing countries. To make the projects and programmes effective they always come with a clear focus on a certain objective. Once this is in the design, it runs through the project cycle, from inception to evaluation. Let´s take a project that aims at improving the trade capacities of a country so that the benefits of international markets arrive at the poorest communities and contribute to sustainable development. The sector could be agro-industries, including small holder farmers as providers of raw materials to local industries. The key goal here is economic and social development. Of course, such a development initiative comes with environmental consequences. But the project itself is “soft” in nature (capacity building, training, institutional development, etc.), so the environmental safeguards do not trigger any major analysis of environmental impacts. Project implementation then happens with a focus to boost the envisaged economic activities. At the end, when the evaluation team arrives at the scene, there is a lot of good evidence to look at. People have been trained, first exports of local products were done, with some improvement in the income of beneficiaries. A good example of a successful project? Are we sure it can serve as model or ”pilot” for the whole country?

Let´s go back to the principle of indivisibility and Agenda 2030 for a moment. What happened on the environmental side of the project? Pulled by additional demand, the small holders are now organized differently. Plantations expanded into formerly forest covered areas and the traditional way of mixing crops has disappeared. They specialize on a single crop and the use of pesticides increased five-fold. The factories receiving the raw materials are located in off-grid areas, mainly using diesel fueled generators for their operations. At the level of the pilot project the environmental aspects are clearly evident. But if they if the same path of agro-industrial development is adopted broadly in the country we are contributing to a severe environmental impact. A typical trade-off between several SDGs.

So where could the evaluation make a difference? Is it the evaluators´ responsibility to discover the environmental issues? Can they be expected to assess the environmental consequences if the project did not produce any useful evidence on, for example, pesticide use? The answer is obvious: the environmental dimension needs to be mainstreamed into the project/programme cycle, from design to evaluation. That would be the ideal situation. As a second best approach and until that happens, evaluations need to come up with pragmatic solutions to best include the environmental aspects. From a recent review of evaluation reports of non-environmental projects at UNIDO we learned that in the few cases where environmental aspects are covered, it is mostly in the section on impact or as a risk to sustainability of the project outcomes. Interestingly it is not covered at all (in the sample analyzed) as a “cross-cutting issue”, a section which is nowadays dominated by gender and human rights aspects. We also saw that environmental issues are rarely analyzed in-depth and never quantified.

There are a number of reasons. First, efforts, including evaluation, usually are focused on the project activities and achievement of the intended objectives and goals. Unintended effects, such as possible environmental consequences, are usually not part of the logical framework or the theory of change that guides implementation, monitoring and evaluation. Second, evaluation budgets and competencies of evaluation teams are critical limitations for a solid assessment of environmental aspects. Third, importance of “indivisibility” or policy coherence for sustainable development has not yet been incorporated into the daily practice of evaluators.

Progress can be and needs to be made at all three fronts. Both the logical framework and the theory of change approaches recognize, in principle, the importance of external factors (risks, assumptions, drivers). Environmental aspects need to be included always as part of the “external” environment of non-environmental projects. Evaluation budgets need to be shifted in a way that cross-cutting analyses of programmes can look into environmental aspects. And last but not least, the new DAC criteria of coherence represents a real opportunity to insert a standardized assessment of the interactions between SDGs at the level of project and programme evaluations. It is now up to the evaluation community to adjust to and take advantage of the spirit of Agenda 2030 and to make more meaningful contributions to it.