Impact Evaluations and the 3ie: William Savedoff

Efforts to design better aid programs often are hampered by the failure to evaluate what works—and what doesn’t—in existing programs. Today, the International Initiative for Impact Evaluation and other important efforts are helping fill the evaluation gap.My guest this week is senior fellow Bill Savedoff. He was a member of the Center for Global Development’s 2004 Evaluation Gap Working Group, led by Ruth Levine, that urged and helped create a new institution for impact evaluation: the International Initiative for Impact Evaluation, or 3ie (“Triple I E”). Following a recent CGD speech by Esther Duflo on the importance of impact evaluation, I sat down with Bill to talk about how new impact evaluations are shaping development projects and policy.To put it simply, Bill describes impact evaluations as studies that try to attribute outcomes to specific interventions. Knowing what interventions work is important but understanding what made them work is just as valuable.“If you just measure the baseline of what happened, you don’t know what would have happened with that same intervention somewhere else,” says Bill.While reflecting on Esther Duflo’s  2011 Sabot Lecture at the Center, Bill tells me he is deeply impressed by her work at MIT’s Poverty Lab. Bill says Duflo’s micro-level studies have helped elevate and promote the practice of impact evaluations and are changing the focus of development economics in a positive way.“That’s what makes this stuff real,” says Bill. “It’s actually working in country, understanding where the data is coming from, and constructing studies that are really relevant to the context – I think that’s much more valued than it ever was before because of her.”In our final segment, I ask Bill about 3ie’s progress since it was created five years ago. Today, the eight million dollar institution is conducting 65 studies around the world to accumulate publicly accessible data on a wide variety of “big picture” development interventions.In the future, Bill tells me he hopes that more agencies and foundations begin using impact evaluations to scrutinize the efficacy of their programs. Bill highlights efforts at the U.S. Millennium Challenge Account and more recently, the U.S. Agency for International Development to increase impact evaluations. Both agencies are now members of the 3ie. But he worries that short-term budget focus in the U.S. Congress could stifle incentives for U.S. development agencies to share what they are learning from the impact evaluations.“Putting some of their resources to a third party, such as 3ie, is a way of saying we’re serious about learning what does and does not work and how it works,” says Bill. “I’m interested in seeing if some of these agencies are brave enough to publish studies that show things aren’t working as well as they think.”My thanks to Will McKitterick for his production assistance on the Wonkcast recording and for assistance in drafting this blog post. If you have iTunes, you can subscribe to get new episodes delivered straight to your computer every week.