The Evaluation Gap Hits Close to Home

"Drowning in Data" (subscription required), an article in the fall issue of Stanford Social Innovation Review, gets at the core of key problems plaguing nonprofit attempts at evaluation. CGD's Evaluation Gap Working Group found similar problems at the global level: confusion over different meanings of the word 'evaluation', difficulties and expense of attributing impact, ethical issues with randomization, and the failure of decision-makers to use evaluation evidence.The article recommends that nonprofits ditch "summative" (what others would call impact) evaluations. It makes sense that nonprofits shouldn't be asked to conduct lengthy impact evaluations at their own expense, especially when results of that evaluation might help the funder and the donor community at large do better the next time. However, this does NOT mean that impact evaluations should be dropped altogether.Knowing which types of interventions generate what impacts is critical to guiding decisions made by donors and governments in the U.S. and abroad, and these actors must take the lead in advancing such an evidence base. It would be a waste to attempt to achieve this kind of learning from every intervention, but as the article states, "there are times and places for summative evaluations." Yes, they may be "few and far between," but so few are currently being done, we have a long way to go before exhausting our opportunities.The article advises that certain programs be considered for impact evaluation: those that have a strong theory of change, are being implemented with fidelity, are transferable elsewhere, and generate confidence among the staff. These criteria should be taken into account by the Leading Edge Group--the designers of an international independent impact evaluation entity--when considering a priority agenda for evaluating the impact of development programs.