Source: Slide 21Based on these findings, both reports recommend that PEPFAR streamline its indicator list, and data collection and reporting process. This could be done by commissioning an external study on its indicators system, including a cost/benefit analysis of the value and cost of each indicator, and then using that report to identify a small set of “Tier 1” indicators that should be reported up to headquarters (as the IOM report recommends). Tier 1 indicators would stay consistent over time, be harmonized with existing global and/or national indicators, and form the basis of Congressional reporting on PEPFAR’s portfolio-wide achievements and epidemiological trends. 3. Require each future grantee and contractor to submit a standardized, realistic, and contractually binding “Data Management Plan” that conforms to specific PEPFAR guidance. Currently, most contractors are required to submit some sort of “performance monitoring plan” or a similar document as part of the grant proposal process. Typically, these documents outline a definition of each performance indicator; the source, method, frequency and schedule of data collection; and targets for each performance indicator. But because PEPFAR has not historically provided centralized guidance for the preparation of such documents, they have been limited in standardization, scope, enforceability, and utility beyond “indicators.” For example, such documents rarely account for the collection, archiving, and sharing of patient or facility-level data, such as on patient retention and outcomes. Accordingly, the IOM report recommends that PEPFAR “develop systems and processes for routine, active transfer and dissemination of knowledge both within and external to PEPFAR” (p. 594). One way to do this (as the DWG recommends) is for PEPFAR to produce clear, centralized guidance for Data Management Plans, which would focus on 1) high-quality data collection and reporting for PEPFAR’s essential indicators, and 2) implementers’ internal collection and management of patient-level data for their own purposes, including the expectations for public access to such data. 4. Expand the collection, application, and sharing of budget, expenditure and cost data through the Expenditure Analysis initiative The lack of available financial data for PEPFAR has been a longstanding frustration for those who want to understand the distribution and use of PEPFAR funds (see here, for example). But it’s not just a transparency issue – there’s some evidence that even internally, PEPFAR has not historically been able to track and manage its cost-structure below the prime partner level. In recent years, PEPFAR has worked proactively to better understand and improve the cost structure and cost-effectiveness of its programs, most ambitiously through the Expenditure Analysis (EA) initiative, which released a pilot report from its first six countries in July 2012. While EA itself was not a focus of the IOM report, the need for better financial data was repeatedly emphasized. The authors’ analysis itself was limited by data gaps in this area, for example on annual expenditures (p. 107) and PEPFAR’s prime and sub- implementing partners (p. 123). And the IOM reported implications of this limited data for program management and accountability: “partner country governments have been frustrated by not knowing where the PEPFAR money was going. Some interviewees wondered how the government could be expected to hold implementers accountable if they did not know where the money was going” (p. 399). Accordingly, the IOM recommends that PEPFAR collect and report financial data that is more closely aligned with programmatic data and program implementation (p. 92). Further, the DWG urged PEPFAR to share the relevant (albeit anonymized) underlying EA data more widely, particularly with partners and country governments, who must understand the financial implications and cost structure of HIV/AIDS programs if they hope to assume direct responsibility for program implementation. To be clear, this is not a comprehensive list of all IOM recommendations for knowledge management; beyond what’s discussed above, the IOM also considers issues such as PEPFAR’s evaluation policy and permitted research methodologies. Nonetheless, these four focus areas provide perhaps the most feasible, practical, and meaningful opportunities to improve PEPFAR’s collection, utilization, and dissemination of data and knowledge, ultimately helping us to better understand the HIV/AIDS epidemic and improve our programs. I am optimistic that this apparent consensus between the IOM and the DWG will lead to real action. Let’s get started!
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.