The UK Aid Watchdog Review: Does the Government Accept Its Recommendations?

With the UK government reducing the aid budget, the impact and scrutiny on that spend will be even more important. One key element of this architecture is the Independent Commission for Aid Impact (ICAI)—whose remit the government is also currently reviewing.

One objective of the review is to ensure “timely, action-orientated recommendations” alongside its critical analysis. In this blog we analyse the government’s responses to ICAI’s 341 recommendations and their composition. We find the vast majority of recommendations were at least “partially accepted,” with just six substantively rejected. This may suggest that, rather than being impractical, ICAI’s recommendations may not be challenging enough.

We conclude with our recommendations for ICAI based on our recent submission of evidence, and note that the government’s newly planned legislative changes could provide broader opportunities to strengthen its independence and mandate.

Reviewing ICAI

The Independent Commission for Aid Impact (ICAI) has provided scrutiny of official development assistance (ODA) spent by the UK since 2011. The Foreign Secretary, Dominic Raab, has confirmed he will “reinforce” and “strengthen” ICAI, and has instigated a review covering ICAI’s purpose and remit, “ensuring that ICAI’s output includes timely, action-orientated recommendations as well as broad critical analysis” and options for “ministerial engagement”.

In terms of the wider review, our full evidence submission to the government on ICAI is here and we list our ten recommendations at the end of this blog. In brief, we think ICAI’s independence should at least be maintained: with ongoing links to parliament and no role for ministers in setting its agenda. We also believe that ICAI could create stronger incentives for officials and ministers to achieve impact with a focus on how actual results compare with those anticipated in original ministerial decisions based on business cases. ICAI’s new results reviews are a step in the right direction (for example, see ICAI’s recent review of DFID’s results in nutrition).

We have also found the traffic-light performance ratings that ICAI awards in its reviews to be a useful communication tool, allowing for comparisons across time and departments.

In addition to these points, we wanted to look further into the element of the review focusing on the practicality of ICAI’s recommendations and how “action-orientated” they are. To do so, we have examined whether ICAI’s recommendations are accepted, implemented, and focussed at the right level.

How does the government respond to ICAI’s recommendations?

Providing recommendations is already a central part of ICAI’s review process. Indeed, following these recommendations, relevant departments are required to provide a formal written response (accepting, partially accepting, or rejecting each recommendation). ICAI then follows-up to track implementation.

To look systematically at this, we have assembled a dataset compiling information on each ICAI recommendation to date, the UK government response, and ICAI’s follow-up reviews This dataset (available here) collates information on 348 of ICAI’s recommendations across 79 reviews between 2011 and 2020; coding the recommendations according to the government response, the level of the recommendation and any follow-up.

Considering all 348 of ICAI’s recommendations from 2011 to date (see chart 1), if we exclude those from recent reviews awaiting a response (7), the government has accepted 68 percent (231), partially accepted 28 percent (96), and rejected 4 percent (14).

Chart 1: Government response to ICAI recommendations

Image

Source: CGD analysis of UK government responses to ICAI reports

Perhaps the most striking finding is that the government rejected so few—just 14 of these recommendations over the period. Further investigation shows that in 8 of these cases the government rejected them as it had already implemented or begun implementing the recommended actions.

This means fewer than one in fifty recommendations (six, that comprise 2 percent) have been substantively rejected by the government. Given the very low rate of rejections, this could suggest that there is scope for ICAI’s recommendations to be more challenging. For instance, rarely has ICAI recommended that a poor-performing project should be terminated (though on a rare occasion when it did, the recommendation was rejected).

We also find that ICAI has consistently provided an average of four to five recommendations per review over the period. (The differences in the total number of recommendations ICAI provides each year has instead been driven by variation in the number of reviews it publishes e.g. thirteen reviews published in 2012, compared to six published in 2016).

The composition of ICAI recommendations balances policy and programme

We also wished to explore the composition, or level, of ICAI’s recommendations. To do so we coded each recommendation as either policy-level or programme-level based on our subjective assessment. The former includes recommendations concerning strategies, plans, priorities, and guidelines. The latter includes issues concerning management, implementation, procurement, staffing, monitoring, evaluation and learning.

Chart 2: Composition of ICAI recommendations

Image

Source: CGD analysis of ICAI reviews

We find that ICAI has provided a mix of both policy- and programme-level recommendations from 2011 to date. Out of its 348 recommendations, we classify 152 (44 percent) as policy-level, and 196 (56 percent) as programme-level. Since 2017 there has been a notable rebalancing towards policy-level recommendations, and it currently provides a fairly even split.

The implementation of ICAI recommendations is consistently tracked

Finally, we find that 37 of ICAI’s 79 reviews have a published follow-up to track the implementation of accepted recommendations. These follow-up reviews tend to be published a year after the initial ICAI review. ICAI has stepped up this practice since 2016: 28 of their 29 reviews published between 2016-19 already have a published follow-up, which helps to ensure implementation of its recommendations.

Conclusion: independence and challenge as an asset

A core focus of the UK government’s current review of ICAI is to ensure it provides action-oriented or practical recommendations. Our analysis suggests a reasonable balance between programme and policy recommendations and, with so few recommendations rejected, it is not clear they are impractical or too challenging to action. It could be that ministers feel unwilling to directly challenge ICAI’s recommendations and instead slow-pedal on implementing them—ICAI’s valuable yearly “follow-up” exercise suggests that 80 percent of recommendations were positively acted on.

In our previous work on ICAI, we heard that findings could sometimes be perceived as ad-hoc or subjective. If “impractical” is a euphemism for this problem, then asking ICAI to tie its findings more closely to the evidence base on effectiveness in a particular area would help—as would transparency in terms of ICAI’s interpretation of the literature (ICAI now routinely publishes its literature reviews). A focus on results and claims would ensure a practical focus on impact. With such wide variation in the impact aid can have, ministers and ICAI could work together sooner in the review process to make more fundamental shifts where evidence points to that need.

Still, fundamental to ICAI’s effectiveness is its independence and ability to raise challenging findings. Ministers already appoint the head of ICAI and set its budget—if “ministerial engagement” means less independence, then taxpayers, Government and aid recipients will be worse-off for it. The Government has signalled it will change the law to adjust the target to 0.5 percent. That legislation is also the basis for ICAI’s mandate. If parliamentarians are not persuaded by ministers’ review of ICAI, they could ensure its independence by making it a parliamentary body like the National Audit Office.