Another year, another attempt at harmonizing global health data collection. This time around, the effort comes from a multiagency working group comprised of representatives from donor agencies and international organizations, in collaboration with IHP+. The working group recently released a draft outcome statement of its year-long process to create a single harmonized and coherent regime for global health data collection, particularly for results reporting. Most excitingly, the announcement promises a list of 100 indicators carefully crafted to offer a "balanced and parsimonious set of core indicators" for global health monitoring and evaluation (consultation draft anticipated in the coming weeks).
While donors’ actions will ultimately speak louder than their proclamations (and none have actually officially endorsed this yet!), this statement and the TBA corresponding list of indicators is a very welcome first step. As we and others have often argued, the status quo – a mish mosh of hundreds and hundreds of uncoordinated and inconsistently defined indicators – is inefficient, ineffective, and unsustainable (see our previous blog posts and reports to this point here, here, and here, and findings of the PEPFAR Scientific Advisory Board Data Working Group here and here). Current practice attempts to measure everything of potential import to every different stakeholder with every possible disaggregation, resulting in fragmentation of resources and, ultimately, low-utility and poor-quality data.
If donors can really agree on a single list of priority indicators – and commit to drawing (exclusively?) from that list as the basis of grantees’ results reporting – the forthcoming reference list of core indicators could drastically reduce the burden for country governments and other implementers. Those same resources currently swallowed by duplicative and unnecessary data collection could then be invested in the core components of the ‘data revolution’ and the value for money agenda: vital registration systems, routine program monitoring (e.g. ‘administrative data’), robust independent verification, and independent national statistical agencies.
The forthcoming consultation draft of the reference list is praiseworthy not just for its contents, but also for the rationale and process behind its creation. The list results from over a year of systematic review by representatives from 19 different donor and international agencies, including PEPFAR, the CDC, USAID, DFID, and the Global Fund, complemented by input from low- and middle-income country governments and civil society organizations. The 100 ‘core’ indicators will be those that ultimately meet an explicit set of sensible criteria, including consistency with previous international agreements and compacts (e.g. the Millennium Development Goals), epidemiological utility, feasibility of measurement, and country-level buy-in. Indeed, the process for generating the list is highly reminiscent of a major recommendation handed down by the Data Working Group of PEPFAR’s Scientific Advisory Board (chaired by our CGD colleague Mead Over). This exercise is potentially even better than the initial DWG recommendation, since it includes not just internal harmonization within a single agency but instead global harmonization among all the biggest global health organizations – assuming they formally sign on.
Yet our excitement over this development is tempered by a few notable concerns. First, at 100 core indicators (plus an anticipated 100+ extra ‘additional’ indicators) calling the list ‘parsimonious’ seems like a stretch, particularly since they specify that these indicators are only intended for results reporting (e.g. excluding routine program/financial management purposes).
Second, the language of the declaration is purposely wishy-washy; signatories will commit only to using the list in a “normative, rather than prescriptive manner,” and to “focus results reporting” around the list of indicators (as opposed to drawing exclusively from it). As a result, holding any donor accountable against specific action items will be essentially impossible.
Third, the document reflects donors' apparent decision to follow the path of least resistance towards "contribution rather than attribution of results." With many actors working in every setting, it can indeed be difficult to divvy up 'credit' for results; nor is it constructive to set donors and country governments against each other in competing for their respective shares of progress. But a 'contribution' approach is also fraught with problems. As we argued in our More Health for the Money report, "attribution is important for determining what does and does not work—whether an intervention is effective. Even if the overall national program is seeing strong epidemiological progress, it is still wasteful to invest scarce resources in an ineffective component." And when actors are able to merely claim contribution for an undefined share of national progress, results reporting can be non-informative. For example, via vague reporting, the U.S. government seems to claim credit for the entire global decline in child and maternal mortality.
Still, the potential value of this declaration and reference list comes not from the documents themselves, but instead from the individual commitment of each agency to honor the spirit of the agreement and operationalize the list within its internal structures and M&E system, ideally while also tracking the results for which it alone can reasonably claim credit. We urge all donor agencies to formally endorse the document once finalized, and then to follow up with an action plan for adopting the reference list. For example, PEPFAR will need to update and harmonize its internal indicator reference list, potentially dropping unnecessary and extraneous indicators in the process; likewise, the Global Fund should describe how it will adopt this outcome statement and give specifics, particularly regarding how those changes with affect and improve current practice.
Will this attempt at harmonization help fix the global health M&E mess, or merely amount to just another flash in the pan? Only time will tell, but right now its in agencies' hands to step up. We'll certainly be following developments closely -- stay tuned!
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
