How Does HIV/AIDS Funding Affect a Country’s Health System?

This is a joint post with Amanda Glassman and Rachel Silverman. Recently, the American Journal of Tropical Medicine & Hygiene published a paper by Shepard et al. evaluating the impact of HIV/AIDS funding on Rwanda’s health system. The headline of the press release was catchy and assertive: “Six-year Study in Rwanda Finds Influx of HIV/AIDS Funding Does Not Undermine Health Care Services for Other Diseases. Study Addresses Long-standing Debate about Funding Imbalances for Global Diseases.” But after reading the report, we quickly assessed that a more accurate and appropriate press release headline for this paper would be “Some Differences Observed in General Healthcare Delivery between Facilities with and without HIV/AIDS Services in Rural Rwanda.” The study has serious limitations associated with its design and its generalizability that aren’t reflected in its catchy press release, and thus have unfortunately gone unrecognized. And because there is, in fact, an important and “long-standing debate about funding imbalances for global diseases” that this study does not sufficiently address, it’s important to examination the shortcomings of the study’s results:. 1. Internal Validity: Does the study do what it claims to do? No. Treatment was not randomly assigned, while matching and control strategies do not mitigate the effects generated by non-random assignment. As a result, the study’s current comparisons between the treatment and comparison groups are problematic in validly testing the proposed hypothesis. The paper analyzes a “randomly selected” intervention group of 25 health centers that provided HIV/AIDS services, which is then “perfectly matched” to a control group of 25 health centers that did not offer HIV/AIDS services. But in reality, the intervention group was “randomly” selected only in the sense that the authors chose to study them, not that the health centers in the intervention group were randomly assigned for treatment. Indeed, why were these health centers chosen to receive HIV/AIDS funding in the first place, back in 2002 or whenever? It’s quite possible that the centers were assigned to have HIV/AIDS funding because the centers were already more likely to have better outcomes. For example, centers that received funding may have had more and better (or better paid) doctors, or perhaps they were located in areas with higher population density, or with higher HIV/AIDS prevalence rates. Similarly, the authors note that, unlike the rural areas that were the subject of the study, all urban health centers in Rwanda provide HIV/AIDS services; this fact alone suggests that treatment (HIV/AIDS funding) was initially assigned based on facility characteristics rather than a random assignment in a representative list of centers. The authors attempt to address this issue by matching the 25 intervention health centers to 25 control health centers. But the authors match on just three characteristics – (1) health center ownership, (2) performance-based financing, and (3) district income in 2002; however, it is unclear that these were the criteria for initial assignment to treatment. Further, the authors do not provide any information to reassure us that the intervention group and control group were comparable on a range of relevant characteristics prior to treatment that might otherwise explain differential performance. 2. External Validity: How generalizable are study’s claims? Beyond the internal validity constraints, the generalizability of the study’s findings is very limited. The study—and particularly the press release—claims to measure the effects of HIV/AIDS funding on non-HIV/AIDS health services. Such a claim, however, ignores the numerous channels by which HIV/AIDS funding can affect a health system besides funding HIV/AIDS treatment in existing facilities; for example, HIV/AIDS funding can lead to technical assistance at the national level, newly built facilities operated by international NGOs or other foreign organizations, as well as health promotion and preventive care at the community level. But the authors’ indicator for HIV/AIDS funding is simply a binary categorization of whether a facility offered HIV/AIDS treatment or not. Moreover, the paper does not discuss the magnitude of funding, the funding source (PEPFAR or Global Fund vs. Ministry of Health disbursements), or whether the facility received an earmarked funding stream specifically for HIV/AIDS rather than general funds which it then elected to spend on HIV service provision. The narrowly focused study does not consider the wide array of other system level effects created by HIV/AIDS funding that have been raised in the previous literature. In particular, the study does not tell us anything about the effects of parallel NGO service delivery or the impact of new or dedicated facilities exclusively for HIV/AIDS, both of which are hot topics in the HIV/AIDS health systems debate. Indeed, in 2008, less than 5% of Rwanda’s PEPFAR funding was channeled through national institutions; the rest was delivered via a range of contractors, most of which were American NGOs or universities (Table 1). The paper makes no effort to address the consequences this funding arrangement and the presence of the 44 PEPFAR prime partners in Rwanda. Table 1: Top Planned Recipients of PEPFAR Funding for Rwanda (USD), FY2008 What’s more, this particular country (Rwanda) is likely to be an outlier among HIV/AIDS funding recipients due to its exceptional national healthcare system, high quality HIV/AIDS service delivery, and innovative health initiatives like community-based health insurance. According to the World Bank’s World Wide Governance Indicators for 2009, Rwanda ranked 7th out of 45 Sub-Saharan African countries for government effectiveness, scoring more than one standard deviation above the mean. Moreover, HIV/AIDS funding in Rwanda accounted for about a fifth of total health spending, a percentage higher than 30 other countries in sub-Saharan Africa. We understand that the authors likely suffered from significant data constraints; likewise, we recognize the enormous empirical challenges in demonstrating system-wide effects at the national level. Still, it remains important to carefully state qualify results and recognize the limitations of one’s research. Bottom line: The jury is still out on whether HIV/AIDS funding has displaced or improved efforts on other disease control priorities.  Let the debate about funding imbalances for global diseases continue…