Measuring Domestic Resource Mobilization–Thoughts on a New Study by NORAD

“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” ― H. James Harrington

While there is debate in the management literature about the primacy of measurement in effecting change, there is no doubt that it is a necessary component of bringing about lasting change. The research and convening that CGD has been doing on domestic resource mobilization (DRM) have focused on political impediments to increased DRM in developing countries. With input from local researchers in five developing countries, we have spelled out the barriers to increased DRM and discussed them with ministers of finance from a wide range of African countries. And throughout our work, it has been evident that a critical component of the politics of DRM is performance measurement. While, in general, analysts have focused on tax-to-GDP ratio as a narrow gauge of performance, policymakers also need to benchmark their revenue systems both against good practices as well as performance of other countries to design and implement future reforms. And once a diagnosis and reforms are implemented, measurement will be critical to demonstrate that reform measures are having the desired impacts, both in increasing revenues and in making the tax system more efficient and equitable.

In fact, currently there are six DRM assessment tools that were introduced mostly in the past six years. To a lay person, they would appear to be too many, given that there are two more in the offing! The first one focuses on assessing the design of tax policies and is being jointly developed by the IMF and World Bank. The second is meant to assess Base Erosion and Profit Shifting activities and is being developed by German Development Cooperation and International Bureau of Fiscal Documentation. On top of the assessment instruments already in use, there are four data collection and reporting tools. Funded and supported by donors and international organizations, such as the IMF, OECD, and the World Bank, they are now widely used to diagnose weaknesses in different dimensions of a country’s tax system. 

Frode Lindseth of the Norwegian Tax Administration has just published a valuable inventory of these tools and frameworks. The work was undertaken to support the Norwegian Agency for Development and Cooperation (NORAD) in its DRM technical assistance to developing countries. Lindseth looked at the comparative advantages of the various tools and frameworks and the complexity of choices that developing country officials and their supporting partners face in their use.

Lindseth is appropriately objective and non-normative in his assessment of the diagnostic and measurement landscape and defers any recommendations of particular tools as “such a decision will depend on country specific circumstances.” But given our experience in working with developing countries and in managing DRM technical assistance, we take away several important lessons from his paper:

• No tool is complete in providing diagnostics, data collection, design advice, and monitoring. Countries are thus forced to choose several tools and marry them with their own processes to craft a strategy. Typically developing country tax administrations are strapped for resources, and choosing among various tools complicates their choices, especially when the complementarities and synergies across tools are not clearly demonstrated for the users. Neither do the assessment tools explain to countries how their use should be sequenced to minimize costs for developing countries’ tax administrations.
• Development partners should exploit existing tools rather than design their own. Where there are gaps, new tools should be complementary to existing efforts. Each of the five diagnostic tools grew out of individual donor and institutional needs, with only the Tax Administration Diagnostic Assessment Tool (TADAT), launched in 2015, representing an effort of donors to work together to a common framework. In 2016 the Platform of Collaboration on Tax (PCT) was established to bring together expertise and enhance cooperation among the four largest multilateral organizations engaged in DRM technical assistance. But two of the four members of PCT came up with new tools―the Tax Diamond and the Maturity Models after 2016. While these new tools may fill an important technical assistance design gap for World Bank teams and for the OECD advisory work, they have also introduced an element of complexity for developing countries in deciding where they should spend their scarce diagnostic resources. And very recently, the International Bureau of Fiscal Documentation (IBFD) has launched another assessment tool, TAx-Ray, claiming to have a more comprehensive methodology.
• Development partners and developing countries could build on Lindseth’s work to create a guide to the use of these tools. While each developing country can use Lindseth’s paper to make preliminary decisions about how to exploit the tools, tax administration officials need to understand technical assistance providers’ expectations of how impact will be measured and what tools partners see as most useful for diagnosis and evaluation. Currently, there is no consistent methodology or guidance across the donor community.
• The cost of using the various tools is difficult to estimate. While development partners can absorb much of the diagnostic cost, developing country tax administrations bear the brunt of reform design, implementation, evaluation, and data reporting, with the opportunity cost of taking tax officials away from their “regular work” being largely unmeasured. This makes any kind of cost effectiveness calculation difficult. It would be useful to have some cost estimates from countries that have used some of these tools in guiding others’ choices. Of course, it is almost impossible to assess how much it cost donors and international institutions to develop these tools.
• Reporting on which countries have used which tools is uneven. While country confidentiality concerns are understandable, it would no doubt be useful to developing countries undertaking assessments to know what other countries have done, if not the specific results. For example, the TADAT website gives this information in some detail, while Tax Diamond and the Maturity Models do not, although data are available on request. Some agreement could be reached amongst partners and countries as to what minimal information can be publicly divulged for the betterment of the DRM community.
• Considerable progress has been made to consolidate sources of data and make them consistent across countries, while trying to minimize reporting burden. As Lindseth suggests, this effort should continue, with the International Survey on Revenue Administration (ISORA) being the best data source on tax administration and USAID’s Collecting Taxes Database providing the most information for analyzing fiscal systems as whole. As with diagnostic tools, development partners and countries should eschew creating additional data sources that require input from member countries. Most of the countries providing data to the 2019 edition of the African Tax Outlook also did so to ISORA, both tools supported by broadly similar donors.
• The available data sets are now of sufficiently wide coverage, in terms of countries and variables, and duration to allow more sophisticated analysis of tax systems. Donors should support researchers, particularly in developing countries, to exploit these data and enrich the analytic bases for policy reform. Equally, the attention should be directed at improving the quality of the data to make it useful for policy design.
• We agree with Lindseth that the toolbox should be reviewed periodically to ensure complementarity and relevance of the various tools and whether there is scope for merging some of them. It would best that such an evaluation be undertaken jointly by partners, developing country representatives, and independent researchers. 

Lindseth’s work is a critical first step in helping the DRM community understand the tools in the DRM measurement toolbox. Consistent use of the right tools will ensure donor and developing countries marshal their limited reform resources well. The donor community and developing countries should build on this paper to that end. CGD will convene some of the key players in the months ahead to help move this work forward.