Beyond Regressions: Making Development Economics Better

Nick Lea, Deputy Chief Economist at DFID, had this to say at Oxford recently:  

What should be researched? … I have listened to about ten papers today. Every single paper is constrained by having to have a regression in it, and it’s really just an enormous, dolled up regression. The problems that Africa needs solutions to, most of them, are not amenable to closure by regression. There is simply not the data, and there is simply not the simplicity of causality. You have to find ways of making an intellectual input as economists on deeper problems that cannot be closed just by pressing a button on Stata [a popular statistical software]. Stata is constraining this profession, and it’s constraining its contribution to African development … limiting our ambition and actually making our work timid and sometimes irrelevant.

This happened at the Center for the Study of African Economies’ annual conference. Lea’s comments were interrupted by enthusiastic applause, and they’ve stayed with me. The fundamental critique here seems to be that economists are too bound to questions that can be answered with a significant amount of data, using the data to test hypotheses. Here are some reactions.

In defense of regressions:

• Lots of big, important development questions are examinable with regressions. In the conference where Lea was speaking, researchers presented regressions on trade and economic growth across Africa, how to increase farmers’ yields in Kenya, how to make sure kids stay in school in Mozambique, and how to keep kids alive in Peru. Of course, there are poorly executed regressions (as there are poorly executed examples of every other research method) and research varies in its direct applicability to the elimination of global poverty. But as I look over the wide array of economic development research happening today, what strikes me is how much of it is highly, directly relevant to improving people’s lives in one way or another.

• Regressions are the main way that economists prove themselves wrong. The late, great labor economist Alan Kreuger helped to push economics in a direction where—as Arindrajit Dube put it—“we consider the core theory to be falsifiable, and not something derivable from deduction alone.” Whether we are using randomized controlled trials or natural experiments (comparing the roll-out of distinctive policies in otherwise similar environments), regressions are the tool we use to see where our theory is right and where it isn’t.

• Economics already has a major tool in addition to its regressions: economic theory. Whether explicitly or implicitly, most empirical economics research is testing a prediction from economic theory. I once heard an economist lament the rise of randomized-controlled trials—one tool of empirical work—pointing out that there aren’t resources to test every combination of poverty alleviation tools. He was right. That’s why most trials and other empirical work are guided by theory. Good economists don’t recommend the testing of any random combination of interventions. They follow where the theory—and previous empirical evidence—points them. (Sometimes, when multiple factors are in play, the net effects from the theory are ambiguous. What helps sort out the reality in that context? Regressions.)

How can development economics be better?

It’s true that there are questions that can’t be answered with regressions. If we don’t have data, then we can’t run a regression. Back in 1955, when Simon Kuznets published “Economic Growth and Income Inequality” in the American Economic Review, he apologized in his conclusion for a paper that was “perhaps 5 per cent empirical information and 95 per cent speculation, some of it possibly tainted by wishful thinking.” His defense for “building an elaborate structure on such a shaky foundation”? “A deep interest in the subject.” Top economics journals may be less welcoming of such speculative work now, and perhaps there should be room for exploring topics for which there are few data. But at the same time, social scientists—including economists—are constantly innovating in their efforts to generate new data, from the macro (comparing student learning across the whole world) to the micro (analyzing transcripts of village meetings and improved ways to measure intimate partner violence).

So maybe the answer isn’t fewer regressions, but rather more and better data that allow more and better regressions.

Furthermore, no regression can answer the question of what will happen in the future if a government implements a particular policy. I remember once seeing an inquirer repeatedly try to get an esteemed economist to elaborate beyond the findings of his specific study. How would this apply if we took it to another country? What if we changed the program? He wouldn’t do it. He stuck to his expertise. In today’s world of abounding claims without any empirical basis, perhaps we should be glad when social scientists try to ground their policy advice as closely as possible in empirically established findings. That doesn’t mean there isn’t room for good economists to synthesize evidence into principles for application for other contexts, and Banerjee and Duflo did in Poor Economics and Acemoglu and Robinson did in Why Nations Fail. But the more our principles and policy advice are grounded in theory and empirics (regressions) together, the better off we’ll be.

Finally, regressions are better at explaining that X causes Y (or that X doesn’t cause Y, even though they’re correlated) than they are at explaining why X causes Y. Sometimes they can do it, but not always. This is where more use of mixed methods may significantly improve economic analysis, demonstrating the context needed for X to cause Y, the unintended consequences of X, and the mechanisms by which X had its impact.

Economics—and development economics specifically—is already a lot more than regressions. That doesn’t mean it can’t be better.

Bonus reading

• You can watch the video of Lea’s panel, which also included Emily Jones of Oxford and Albert Zeufack of the World Bank.

• You can find more videos from the conference, including Eliana La Ferrara’s keynote speech on “The Entertaining Way to Behavioral Change: Fighting HIV with MTV.”

• See the full program of the conference or read my microsummaries of 275+ papers.