Rupert Scofield Reviews Due Diligence

Rupert Scofield, the co-founder, president, and CEO of FINCA, just reviewed my blog-linked book on his book-linked blog. He doesn't find my analysis very persuasive. Inevitably, the reaction is mutual.He says that I "[set] the statistical/methodological bar pretty high," so that the body of evidence on which I am willing to rely is thin; and I "[overreach] in the sense of drawing bold, sweeping conclusions."It is true that I admit only the best evidence. I explain why in chapter 6: almost every time I have poked into a non-randomized study, it proves wanting. (Not that there can't be high-quality non-randomized studies, ones that exploit natural experiments. The best attempt I've seen for microfinance was in Thailand.) Rupert does not respond directly to my judgments about the reliability of various forms of evidence. He presents almost no evidence that I'm too conservative in my assessment of the evidence.Meanwhile, I think he underestimates the power of the randomized studies:

Roodman sets the statistical/methodological bar pretty high---so high, in fact, you might say if there were an impact on revenues and empowerment, his approach wouldn’t detect it. At least that is my conclusion.

Then why have the microcredit studies found impacts on investment and profits, and the microsavings studies found impacts on income? Why did the study of the South African lender (see below) find it increasing employment?I disagree too that I overgeneralize from that thin evidence set. Consider this line from the book that he quotes: "On the limited high-quality evidence available so far, the average impact of microcredit on poverty is about zero. In contrast, the one high-quality study of savings does find economic gains." I mean that to say: "If forced to generalize from available evidence, here's what we should conclude." I think that's pretty much what it says. And I do think it is my job to generalize (responsibly). My work was funded by philanthropy on the faith that it would influence action. People who must act now, including Rupert and his employees, must do so on the evidence on hand now, and that always involves extrapolation---extrapolation from the contexts in which the studies were done to the contexts in which one is working. The practical question is not whether to generalize but how (even as we collect more evidence). That is where I try to help.But Rupert does not see things that way:

Four microcredit studies on a few thousand clients in three countries, and one study on savings in Kenya involving 122 people, and we can draw a conclusion like that for an industry of over 150 million clients? I would never make such an extrapolatory leap.

Well, maybe he doesn't, but the organization that bears his stamp as much as anyone else's does. Circa 2009, FINCA ran a "historic campaign to create 100,000 Village Banks…and lift millions out of poverty by 2010." (Did it happen?) And a FINCA report I blogged in 2010 says that:

Since the late 1980’s, hundreds of microcredit programs, collectively channeling several billions of dollars in loan capital, have already doubled the incomes of over 100 million of the world’s poorest families.

Are FINCA's strong claims about the efficacy of microfinance built on an evidence base more hefty and sturdy than mine? If so, Rupert does not describe it. Instead, he offers his own experience:

I saw many, many very poor people bootstrap themselves out of poverty on microcredit alone, and to this day using credit to “prime the pump” in a capital starved environment can be the only way a poor family can start or grow a business.

I respect his experience. But I don't think we should spend billions of dollars, especially on a potentially harmful intervention, based on such subjective experiences alone---no more than we should for new medicines. I am indeed "pretty rough on the evidence of our eyes and ears, arguing that we shouldn’t really believe women who say microfinance has changed their lives for the better just because they tell us so." Psychologists and behavioral economists have documented many foibles of human cognition over the last half century. That said, I give space in my book such "qualitative evidence"---and, when it comes to group microcredit, I find it disturbingly mixed.Rupert's main concern is not whether microfinance can reduce poverty, but whether it is being done wrong. He's worried about for-profit, predatory lenders giving the movement a bad name and doing real harm to the poor. I think he's right to be concerned about coercive practices such as the Smart Campaign is combating. At the same time, there are no certainties even here. He seems to miss that likely the most "predatory" lender in the randomized studies I review, a South African for-profit whose interest charges compound to 586%/year, is the only one to measurably reduce poverty. Those randomly offered these expensive credits more often got or retained their jobs and less often went to bed hungry. But I guess Rupert would not embrace this lender, nor the clientele (incomes of $6.50/household member/day), nor the economic context (jobs rather than enterprise being the upward route) as representative of microfinance as a whole.A final weakness in the argument here is a quote from the book taken out of context. A paragraph in chapter 6 thinks through an argument against reading much into the lack of poverty impact in the Hyderabad microcredit study. As Rupert quotes:

Perhaps the team failed to find impacts on poverty because the treatment groups had almost as much access to microcredit as the control groups.

Rupert interprets this line as a huge concession about the uselessness of the study. On the contrary, for reasons laid out in the next sentences, the paragraph quoted paragraph concludes oppositely.Not that I can't be stubborn and biased, but I think by nature I try to hear out all sides and share the thinking-through in public. That style brings the risk that my devil's advocacy will be mistaken for actual advocacy. But I hope that this judicial style in some small way elevates the public discourse about the impacts of microfinance and improves action going forward.