Diagnosis: The Problem, Updated

Management by way of top-down controls and targets sometimes gets in the way of aid donors’ aims, undermining project success. These unhelpful controls often stem from a need to account for performance; legislatures or executive boards induce agencies to exercise tight process controls and orient projects towards what is measurable and reportable.

My 2018 book, Navigation by Judgment, presents quantitative empirics from a database of 14,000 projects and qualitative evidence from eight case studies in support of this claim. I wrote the book in part because I imagined it would be useful to show aid agencies and authorizers empirics that demonstrated the problem—that the first step to change was recognition. But after talking about my book at aid agencies and authorizers, I am confident that while my 2018 diagnosis was right, my theory of change was all wrong.

The people who work at aid agencies—from field staff to senior managers—broadly already understand the nature of the problem. They have experienced first-hand the tension between reporting and performance, have chafed against constraints that prevent them from doing what in their professional judgment would be most likely to achieve their agency’s aims (which they largely share). They don’t need empirics to tell them what they live every day. Surprisingly often, authorizers agree. As one World Bank executive director put it to me, “I know that when I set targets, I don’t get what I want. But it’s the only way I can move the system at all, the only way I can steer; if I give up that lever, what do I have? Why am I here?”^[1]

This policy brief and underlying paper attempt to answer that question, laying out principles and actions that might better align aid agency management practices with the agency’s laudable goals.

Principles for Prescription: How to Make Aid Delivery Better by Rethinking Accountability Where Appropriate

Aid agencies often act as if counting and reporting things is all that “accountability” means. “Accounting-based accountability” is far from the sum total of accountability, or what accountability systems consist of in many successful organizations. The word “accountability” is not a magic potion; sprinkling it onto something does not, in and of itself, make that thing useful or good or welfare-enhancing.

The focus of many aid agencies is explicitly maximizing “value for money.” For those agencies, if what we call “accountability” is a practice or process that reduces the numerator (value) without changing the denominator (money), reducing that practice is, by definition, at odds with the agency’s explicit goals. Maximizing the value of every penny is not always done by maintaining the tightest possible oversight over the disposition of every penny. Donors can, and should, “do accountability differently” if they want to “do development differently.”

Below are five principles that, drawing on my conversation in agencies as well as my empirics and those of others, I believe would move agencies in the right direction.

1. Give Up the Delusion of Top-Down Fiat Control Where It Won’t Work

Where top-down controls and tight oversight work well, they can be quite effective in driving agency performance. But where the best possible control system is flawed (e.g., unpredictable environments and projects that are difficult to accurately and reliably measure without distorting effort), less control is likely to work better than implementing a flawed attempt at top-down control. This includes aid efforts in unpredictable environments (e.g., fragile states) and with tasks where measurement won’t work well (e.g., building a justice system, rather than a road). In these circumstances the data just isn’t good enough on its own to be an accurate, reliable basis for sanction or reward of staff or projects. It may seem from the outside like those at the top are in control, but that’s just a delusion—and most folks in management are, in fact, aware of this. The delusion is dangerous in that those controls have the power to keep those in the field from doing much of anything, by constraining them with tools that undermine the broader goals of projects.

2. Accept That Accurate Knowledge of Results Is Superior to the Reductive Façade of Inaccurate Metrics—Even If That Reduces “Success Rates”

Accountability is predicated on actually understanding what’s going on—which is different from generating numbers and reports that allow us to proceed as if we know what’s going on. Knowing more about what projects are doing and their effects sometimes means less reliance on reportable metrics. As many aid professionals agree across agencies and level of seniority, some projects succeed but have few ways of quantifying that success. Others meet their deliverables, hitting the target but missing the point—the current system makes some aid projects appear successful that may not, in fact, achieve their ends. Abandoning what’s widely understood to be a façade where inappropriate will help improve aid delivery. It’s hard to get better if you can’t understand what’s happened in the past.

3. Use Different Accountability Systems for Different Tasks, Environments, and People

 “One size fits all” works for accountability just about as well as it does everything else in the aid world—that is, poorly. Aid projects cover a dizzying array of tasks and recipient country environments. Accountability practices need to incorporate this basic fact. Agencies increasingly realize that routines need to be adapted for difficult contexts, like fragile states. But we need to go further—not just adding exceptions to a basically top-down accountability system where the focus is on reporting “up” the chain of command, but developing one focused much more on how to encourage and support agent judgment.

4. Accept that Changing Agency Practice Requires Constant Tending

There have been a number of laudable attempts at reforming aid agencies in recent years. These attempts generally teach us how very difficult it is to get reforms right. DFID’s Smart Rules, for example, aimed to empower staff in the field with greater flexibility and ability to make decisions (an example of a particularly laudable reform). Pairing this change with the creation of a “Senior Responsible Officer” (SRO) role has, it seems to me, led to a great deal of the Smart Rules’ potential benefits not being used, as SROs engage in conservative behavior in an attempt to minimize their personal risk. Getting agency reform right requires treating it as a process, not an action to be taken at one time; reform requires constant tending, and likely readjustment based on what is occurring in the actual behavior of field staff. Adaptive management—paying attention to what’s happening and adjusting accordingly—isn’t just good for projects; it’s good for the agency itself as well.

5. Move Towards a Default of Trust Where it Serves Agency Goals

Where agencies recognize the need to be led by those in the field, “accountability” can’t be all or even mostly about counting and reporting. Instead agencies would be better served by “trust-based accountability”; by assuming their agents were trustworthy until proven otherwise. This would shift agencies towards a focus on the selection processes, training, and support structures that would make trust work—putting HQ at the service of the field, rather than the other way around. A default towards trust does, inevitably, mean there will be instances where trust is abused. That’s a real cost, and thus change in this direction likely involves a more honest and open conversation with authorizers than is currently the case. Such a dialogue is in authorizers’ interests, too—a tight system of control may maximally prevent fraud. But in so doing it ensures many actions in service of an agency’s mission also go untaken, and thus authorizers’ objectives also remain unfulfilled.

Concrete Actions to Move Towards this Vision

People inside and outside aid agencies can take concrete actions to move the aid system in the right direction. These include:

Agency Leaders

• Start a pilot of Navigation by Judgment, or form a committee/explore same. Turn the reform lens your agency shines on others towards the mirror and pilot a reform of organizational practice on the agency itself.
• Commission analytic or diagnostic work on where metrics aren’t giving an accurate picture of success, and where/when/how rules and reporting are getting in the way of impact at your agency. Find out if indeed your agency has a problem, and the specifics of that problem.
• Talk to authorizers (politicians, board members) about when pursuing measurable and reportable results is undermining your ability to deliver results. Authorizers (Members of Parliament, Congress, executive board members, etc.) are, in fact, human beings. Many – I think probably most—also would like to see the money they spend translate into actual impact, not its façade.

Junior Staff

• Propose new things you think are right. Raise your hand and speak your mind (recognizing the long experience of others, of course). Ask questions that your relative youth and inexperience make it easier for you to ask; for example, “I know that I’m the new person, and there’s probably a good reason for this, but why can’t we just use a different kind of accountability framework here?”
• Learn the actual rules. The next time someone says, “You can’t do it that way,” maybe say “I’m sorry, I’m new here. Why not? What’s the rule that governs?” It’s a bureaucracy; if it’s not written down, it’s a custom, not a rule.

Mid-career Staff

• Challenge assumptions you think may not be accurate. The next time someone says in a design meeting, “Well, we need to have a measurable output,” push back if you think that’s not accurate.
• Pay attention to “pass through” to implementers. Flexibility at the top allows, but doesn’t guarantee, flexibility at the bottom. Think about where the flexibility (or constraint) inside your agency is affecting implementers, and what you may be able to do within the existing rules to give implementers more ability to use their judgment where appropriate.

Implementers

• Recognize your collective power. You’re where the work actually gets done, and your thoughtfulness and judgment are already built into the system—but are limited (to the program’s detriment, as almost every implementer I’ve ever spoken to agrees) by constraints that flow from your funders. Point them out, individually and collectively—that is, in concert/coordination with other implementing organizations.
• Don’t neglect your power in individual project management, either. As Pedro Pietro Martin recently put it in the context of managing adaptive programs, “We [implementers] need to find ways to tweak and leverage the current donor's system, their existing tools and processes, so that we can progress. One programme, one logframe, one risk matrix at a time. Until it becomes a trend. The system will not change by just being proved that it does not work, that it is wrong. It is required to demonstrate that it can work better... by making it work.”^[2]

This is but the tip of the recommendation iceberg. Please see the full paper for more, including more recommendations for the groups above, for authorizers, and for the rest of us both inside and outside the aid world.

Let’s Get Moving

The currently dominant accountability system in use in aid agencies gives a false picture of value for a wide range of projects where what is countable is not an accurate summary of the project as a whole. If this led only to inaccurate perceptions of success, however, that would, in fact, be an improvement. Theory and evidence strongly suggest that the primary way donors are asked to demonstrate “value for money”—with reportable, countable, accounting-based accountability reports of success—can, in fact, undermine the “value” part of that equation.

Judgment fails. All the time. In myriad ways. But “Will judgment always work?” is the wrong question. Fallible judgment ought be compared to also-fallible, and sometimes-delusional, top-down control. The question is when each of these two ultimately fallible broad strategies is going to lead to better results. In many circumstances, the answer to that question is “more judgment.” Where that’s the answer, agency management practice and accountability routines must change in practice (not just on paper) in order for agencies to realize performance gains.

I see, as I move through aid agencies, many dedicated professionals who genuinely care about their organization’s goals, who want to achieve the welfare impacts their organizations seek. Many are thwarted by their own organization’s rules and practices in executing their organization’s mission. I spent years working on a research project, and ultimately a book, because I thought the world needed evidence that there was a problem here. I now think I was wrong about that; the problem is widely recognized by insiders. What’s needed are solutions.

Will change be easy? No, of course not. But most people working in foreign aid have already chosen something other than the easiest path for their lives, because they care about the organization’s mission. We owe it not just to these professionals, but to the welfare of the world’s citizens, to do everything we can to build aid agencies worthy of the people who populate them.