Corrupting Perceptions: Why Transparency International’s Flagship Corruption Index Falls Short

This post was originally published in Foreign Policy.

For nearly 20 years, Transparency International has published a Corruption Perceptions Index that ranks countries (176 last year) according to how corrupt they are perceived to be by a small group of individuals. This approach was chosen in light of the difficulty of measuring actual corruption and the expense of running broad surveys. Transparency International does some great work, pushing issues of transparency and accountability up on the agenda, and there are very many serious and deeply committed people involved. The problem with the Index, however, can be found in the name. Perceptions are not facts, and in this case they may be an unhelpfully distorted reflection of the truth. 

Imagine a situation in which lawmakers in a lower-income country end up shutting major business figures out of a commodity deal. Suspicions of corruption are raised -- but, importantly, with zero evidence. There may have been foul play, but then again, there may not have been. As we all know, stuff happens and it's only natural to want to find someone to blame. But now, with the perception that corporate dealings aren't exactly kosher, that particular country, regardless of the absence of evidence, will be affected in its CPI ranking. 

Then let's say journalists pick up the original story ("Major companies shut out of commodity deal"). It could be due to crony capitalism, or it could just be what happened. To find out more, to whom do they turn to confirm their suspicions? The Index of course! A narrative then begins to develop. "Country X, which is identified by TI's Corruption Perceptions Index as one of the most corrupt countries on earth...." 

This in turn supports a much wider perception of Country X as corrupt. That consequently worsens country X's score in the next round of surveys. And so on. And so on.

That is, until, a new government decides to rebuild relations with major corporate players. Not necessarily out of any commitment to fighting corruption, and not necessarily with any effect on actual corruption, but out of a fierce desire to stop scoring so badly on that damned index which gets so widely reported. 

What's missing from this vicious cycle is not only evidence of actual corruption in Country X, but any information about how corruption is or isn't affecting the citizens of Country X.

You'd imagine that many of the staff and chapters of an organization producing such an index would protest internally; and I'm told they do. You'd imagine that even the original creator of the index, Johann Graf Lambsdorff, would call for an end to it -- and indeed he did, four years ago:

"In 1995 I invented the Corruption Perceptions Index and have orchestrated it ever since, putting TI on the spotlight of international attention. In August 2009 I have informed Cobus de Swardt, Managing Director of TI, that I am no longer available for doing the Corruption Perceptions Index."

The underreporting of Prof Lambsdorff's withdrawal from the Index -- though perhaps fostered by the absence of a detailed explanation for the decision -- stands in stark contrast to the ongoing coverage of the Index itself.

The Corruption Perceptions Index (CPI) is derived by aggregating 13 different perception surveys. There is a striking commonality in the people whose perceptions are actually assessed: "a group of country economists," "recognized country experts," "two experts per country," "experts based primarily in London (but also in New York, Hong Kong, Beijing and Shanghai) who are supported by a global network of in-country specialists," "staff and consultants," "over 100 in-house country specialists, who also draw on the expert opinions of in-country freelancers, clients and other contacts," "4,200 business executives," "100 business executives ... in each country," "staff," "100 business executives from 30 different countries/territories," "staff (experts)," "100 business executives per country/territory," and finally "over 2,000 experts and 66,000 other individuals from around the world have participated [to date]."

The point here is not that any of these underlying sources are bad, or wrong, or anything other than what they claim. The point is that in aggregating them, the result lacks any sense of diversity. The correlations between the various components run between 80 and 100 percent, because overwhelmingly the same kinds of people are being asked for their perceptions. Now clearly the opinions of an internationally focused elite, typically from a corporate background and perhaps a similar education, may be particular informative in certain circumstances; one or more of the component surveys might be a very useful source for a company considering setting up in business in a new jurisdiction. But do these sources, when aggregated, produce a broader truth about the public's experience of corruption in a given country? 

Earlier this year, University of Minnesota law professor Stuart Vincent Campbell made a number of criticisms about CPI as a tool, even for companies considering their duties under the Foreign Corrupt Practices Act. He gives the interesting example of Brazil, judged by the Corruption Perceptions Index as having a serious corruption problem.

Perhaps unsurprisingly, the view changes when the experiences of a group of a country's citizens are considered instead of restricting the view to elite perceptions only. Campbell writes that, in contrast to the CPI ranking which in 2010 put Brazil 69th, behind Italy and Rwanda, "The 2010 Global Corruption Barometer [based on a broader survey of Brazilian citizens] found that only 4 percent of Brazilians had paid a bribe, which is a lower percentage of bribe-givers than the survey found in the United States or any other country in Latin America." Interestingly, the Barometer is published by one of my favorite anti-corruption organizations: yes, Transparency International.

Now, the Global Corruption Barometer isn't perfect. Questions have been raised over the quality (and low cost) of some of the underlying national surveys, with only around 1,000 respondents per country. The most recent release attracted attention because of the absence of a score for China>, leading the organization to explain that they had been unable to find a national survey partner to work on this sensitive issue. However, Pew had surveyed more than 3,000 Chinese citizens on attitudes to corruption, among other things, in 2012 (and in 2008).

On the whole, however, the Barometer is a much more appealing approach than the CPI -- down to the name, which suggests an attempt to get a sense of the corruption climate, rather than the hard ranking the term "index" implies. The Barometer may not be perfect; but what it does is to take a more broadly representative group of citizens in order to try to understand better the actual experience of corruption in different countries.

When the Corruption Perceptions Index was launched in 1995, there were legitimate arguments for its construction. The cost, back then, of broad citizen surveys was probably prohibitive -- and certainly for an organization of TI's size then. Corruption is, almost by definition, hidden -- so perhaps impossible to measure with great accuracy. It's easy to see how the argument could have been made for using imperfect and narrow expert perception surveys to put some numbers on an important but under-reported phenomenon, and attract media attention to the issue.

But the world is different now. Transparency International is a global force, and has no shortage of media attention; in general the media emphasis on corruption in lower-income countries is, if anything disproportionate. The media can't be criticized too much for preferring simpler rankings and narratives -- and non-governmental organizations play on this. (Just to be on the safe side, I should note that one of the reasons I worked with the Tax Justice Network to create the Financial Secrecy Index of tax havens was to make the point that some major drivers of corruption begin at home, and to do so in a way that would help the story be reported.) But with the Global Corruption Barometer, TI has already created what looks to be -- or with greater investment, is capable of becoming -- a superior alternative to the Corruption Perceptions Index. This can do more than drive media profile; it can help to focus policy effort where it is most needed. 

The CPI embeds a powerful and misleading elite bias in popular perceptions of corruption, potentially contributing to a vicious cycle and at the same time incentivizing inappropriate policy responses. The index corrupts perceptions to the extent that it's hard to see a justification for its continuing publication. For the good of the organization, its important aims and the many people committed to its success, Transparency International should drop the Corruption Perceptions Index.