Reinhart-Rogoff: The Problem and Solutions

This post orginally appeared in Business Standard.

Paraphrasing "jesting Pilate", "what is truth when academic superstars supposedly produce it?" is possibly the most important yet neglected question raised by the recent Reinhart-Rogoff (R-R) affair.

The essential facts of this episode are these: two Harvard professors, Carmen Reinhart and Kenneth Rogoff, used their academic research to become strong advocates in the policy realm, although not among fellow academics, of the need for fiscal austerity when economies approach a threshold level of debt-to-GDP ratio of 90 per cent. Their superstar reputations (deservedly earned through previous work) rendered their advocacy influential, even highly so, in the charged policy debates in the United States and all over Europe. The research - especially the critical and attention-grabbing finding of a sharp growth discontinuity at that 90 per cent threshold - was subsequently exposed as flawed.

The post-mortem frenzy of the R-R affair has focused on the implications for the austerity debate. That seems less interesting than the broader epistemological question. One response is to dismiss the question altogether as irrelevant. In some ways, the affair shows that the market for knowledge or the process of "scientific inquiry" actually worked. Claims were made based on evidence. The evidence was re-scrutinised and the claims were subsequently debunked. In fact, in a subtle but important way, there is an appealing self-regulatory quality to this market. The bigger the superstars and the more influential the claims, the greater the scrutiny that is elicited. And it is also possible that the embarrassment caused to Professors Reinhart and Rogoff (especially if their mistakes are seen as going beyond the merely sloppy) will make them and others more careful with future research.

The great physicist Max Planck said that science advances "one funeral at a time". Interpret "funeral" metaphorically and we have a validation not just of his morbid witticism but of the current process for generating knowledge itself.

But the R-R saga suggests there are two important caveats to this comforting conclusion, two ways in which this market may have failed. The first is what might be called the "halo effect". Ideally, every piece of research should be evaluated on its merits. But it is likely that the R-R research was subject to lower standards because they were, well, Professors Reinhart and Rogoff.

The second and perhaps more serious problem relates to the damage that is done meanwhile. Assume for the moment that research and ideas have influence. Assume, too, that influence is directly proportional to the reputation of the peddlers of ideas. Then, is a market that works after the fact good enough? Would society allow products to be sold without setting safety standards in advance on the grounds that manufacturers have an inbuilt incentive - via reputational losses - to ensure that their products are safe and do not end up killing people? In this view, Professors Reinhart and Rogoff were responsible for the real consequences of the austerity they advocated. Would society want this to be repeated?

Clearly, it would be silly to hold Professors Reinhart and Rogoff culpable for austerity and its consequences. Theirs was one among many contending voices on the austerity debate. Why single them out? But there is a problem that cannot be ignored. And this problem is especially acute in today's high-octane, high-frequency world where preliminary research comes out as "working papers", and is picked up and disseminated well in advance of the formal process of scrutiny that leads to eventual publication (or rejection) of this research. And these risks are greater, especially where the work of superstars is involved.

What should be done to minimise these risks? One possible solution might be the following. If superstars (say, Nobel laureates, John Bates Clark medal winners, and chaired professors at top universities) produce empirical research on important policy matters, they should voluntarily submit such research to higher evidentiary standards and to a process of scrutiny even before it is disseminated as preliminary research. For example, the National Bureau of Economic Research (NBER), which produces the most prestigious working paper series in the United States, could create a "Replication Office". Staffed by bright graduate students, superstars (many of whom are affiliated with the NBER) would send their empirical research to this office for pre-publication scrutiny. This would be a sort of noblesse oblige, a recognition that superstardom confers extra responsibility to ensure that not just superstars themselves but even others do not misuse their research.

In some ways, the R-R saga exemplifies the lesser of the dangers. Arguably, the greater one arises when superstars move beyond research into the world of pure policy advocacy, often outside their field of specialisation and expertise. In fact, this problem was anticipated long ago.

At the Nobel Banquet in 1974, Economics prize co-winner Friedrich von Hayek made a speech in which he suggested that the Nobel committee require its laureates to take "an oath of humility, a sort of Hippocratic oath, never to exceed in public pronouncements the limits of their competence". His explanation: "the Nobel Prize confers on an individual an authority which in economics no man ought to possess. This does not matter in the natural sciences. Here the influence exercised by an individual is chiefly an influence on his fellow experts; and they will soon cut him down to size if he exceeds his competence. But the influence of the economist that mainly matters is an influence over laymen: politicians, journalists, civil servants and the public generally. There is no reason why a man who has made a distinctive contribution to economic science should be omnicompetent on all problems of society - as the press tends to treat him till in the end he may himself be persuaded to believe."

If Keynes - von Hayek's intellectual doppelganger - was right when he famously concluded in his The General Theory of Employment, Interest and Money that: "But, soon or late, it is ideas, not vested interests, which are dangerous for good or evil." And if superstars are increasingly involved in the generation and dissemination of such ideas, perhaps they should undertake two commitments: subject their research to higher standards and early, pre-publication scrutiny; and, more controversially, strive to limit their policy advocacy to their specific fields of expertise. Caveat emptor carries risks in the market for ideas with superstars as key players. The time may be ripe for such a Hayek-Hippocrates oath to help minimise such risks.