Does Results-Based Aid Change Anything?

All the hype (and criticism) over foreign aid programs that pay for results got us wondering: how are these programs being implemented and are they effective?  In our new paper “Does Results-Based Aid Change Anything?” we looked at a subset of performance programs that pay governments in proportion to changes in development outcomes to see what  motivates the use of these approaches and how they work in practice. Instead of a revolution in aid, we found a cautious adaptation of traditional program approaches. 

We found very few programs that actually pay governments for outcomes

And when they do, these programs don’t work because governments want the money — they work by getting politicians and bureaucrats to pay attention to results. 

Our starting point was that it’s not all that useful to treat all types of performance-based programs as if they are trying to accomplish the same thing. The nature of governments differs from other categories of recipients in terms of resources, behaviors, and dynamics; and the responsibilities of funders and recipients change dramatically when payments are linked to outcomes rather than to completing activities or outputs.

So when we started this work in August 2013, we looked at the full universe of performance-based aid programs to identify case studies. We found many programs that pay businesses, NGOs, households, communities, and individuals for activities and outputs, but we could only identify six aid programs that paid governments for outcomes. Three of them were Norwegian programs to reduce deforestation, of which we chose to include Brazil’s Amazon Fund because it was the first and best documented. We included all three of the other programs — Gavi's Immunization Services Support (ISS) multicountry program, a secondary education program in Ethiopia financed by the UK Department for International Development (DFID), and a regional health program called Salud Mesoamérica 2015 administered by the Inter-American Development Bank (IDB).  Since that time, DFID, the World Bank, and Gavi have initiated other programs that might satisfy these criteria.

Most of the programs seem effective, but not necessarily for the reasons that are commonly put forward 

Each of the programs has achieved some success. Brazil’s deforestation rate has declined, more children are immunized, more students are completing Ethiopian secondary schools.  However, the reasons for success had little to do with governments responding to incentives in the sense of changing their behavior in order to get money. In reviewing these cases, we came to distinguish at least four theories for how these programs are supposed to work:

1. Pecuniary interests. Countries will change their priorities because they need the money promised by the RBA agreement.
2. Attention. Politicians and bureaucrats have limited time and attention. Because funds are linked to outcomes, politicians and bureaucrats will pay more attention to results and manage things differently than they would otherwise.
3. Accountability. RBA agreements make outcomes visible to citizens in funding and receiving countries, allowing them to hold their governments accountable for performance.
4. Recipient discretion. By linking payments to outcomes rather than inputs, funders give recipients wider latitude to design and implement strategies of their own making. Using this discretion is more compatible with responding to local knowledge, building local capacity, innovating, and adapting. 

Most of the cases we studied promised too little money to motivate change through pecuniary interests, lacked the transparency necessary to generate accountability to constituents, and involved funders too much in program design to be testing the effectiveness of recipient discretion. Rather, payments for outcomes seemed to increase the salience of results to people implementing programs — among staff in the funding and recipient agencies — making increased attention to outcomes the key mechanism for change. The one exception was the Amazon Fund where Brazil had full discretion over its domestic policies for reducing deforestation and where transparency has played a role in the domestic politics behind those efforts.

Few of the big concerns that are commonly raised about RBA have come to pass

Though we only had a small number of cases, none of them displayed the kinds of problems typically raised by critics. In particular, none of the agreements have been the subject of accusations regarding corruption and none appear to have distorted public allocation decisions, sacrificed long-term goals in the pursuit of short-term gains, or entailed costly verification and monitoring. 

RBA is a work in progress and COD Aid is largely untested

A large part of our interest in studying these cases was to learn whether programs with features related to our proposal for Cash on Delivery Aid (COD Aid) are showing success. What we found instead was that — with the exception of Norway’s performance agreements to reduce deforestation — very few funders are paying governments to achieve outcomes and, when they do, recipients still do not operate with full autonomy, and performance information is rarely communicated to the public in ways that could increase accountability.   

RBA does make results more visible to politicians and project implementers and this may be marginally better than traditional approaches that pay for inputs. However, the potentially transformative aspects of RBA that could come from greater recipient discretion and increased accountability to citizens have not been tested. Recipient countries need to demand, and funders need to offer, RBA programs that are more hands off and transparent if they are going to transform aid in these ways.