Recommended
As the COVID-19 pandemic continues to unfold, national decision-makers are drawing on information from a growing array of epidemiological models to inform their policy responses. With different models employing different approaches, it is not surprising that they can yield diverse and sometimes conflicting results. Since April 2020, a group of national governments, funders, and development partners supporting COVID-19 responses in low- and middle-income countries (LMICs) have been working together as the COVID-19 Multi-model Comparison Collaboration (CMCC) to support the use of epidemiological models during the COVID-19 pandemic. Today, the CMCC releases its first two outputs—an important step towards enhancing the informed use of models in COVID-related policymaking.
The CMCC approach
Over the past several months, technical experts on infectious disease modelling, COVID-19 modellers, and decision-makers (working on their own country response) worked together under the auspices of the CMCC to discuss the policy relevance, robustness, and usefulness of current models with the view of informing future modelling efforts.
To our knowledge, the CMCC is unique in its engagement across different stakeholders and geographies. With its focus on LMICs, it pays particular attention to the considerations and nuances of using models in resource-poor settings, from formulating the decision problem, to data availability and reliability, through to reporting model results for informing the policy dialogue. The CMCC’s working groups—a technical group, a modelling group, and a policy group—have been in constant dialogue with one another from the outset and this ongoing dialogue is reflected in the design of the CMCC’s outputs. An earlier CGD blog post provides more detail on the organization and structure of the CMCC.
Identifying key policy questions and comparing models
The past months of engagement have led to today’s release of two new global public goods to support the translation of evidence into policy in the pandemic response. The first is Guidance on Use of Modelling for Policy Responses to Covid-19, prepared by the CMCC’s Policy Group through engaging decision-makers across the world and within the CMCC. The guidance identifies high-priority policy questions and explores how (and what type of) models can be used to answer them. It is directed towards decision-makers (and their advisors), modellers, funders, and other researchers/technical staff supporting COVID-19 responses in LMICs. The guidance provides an introduction to infectious disease models; summarizes decision-makers’ experiences and intended uses of models; discusses how to determine a model’s fitness-of-purpose and how to report results; and highlights communication best practices and accountability in using models to inform policy responses.
The second CMCC output released today is the Technical Group’s Model Fitness-for-Purpose Assessment Report, the result of an engagement between a team of independent infectious disease modelling experts and seven modelling groups working on COVID-19 at the global level. The report gives an overview of the seven models’ objectives, methods, assumptions, and data sources, and offers guidance for assessing their fitness-for-purpose in context, framed as a dialogue between policymakers, analysts, and modellers themselves. It also offers recommendations for decision-makers and COVID-19 modellers for improving their engagement, ultimately with a view to better policy decisions.
Next steps
Acknowledging that decision-makers’ evidence needs for COVID-19 policy responses are likely to evolve over the course of the coming months, the CMCC has scoped a number of ways to build on the work conducted so far—these are summarised as non-mutually exclusive options in the table below. The option(s) to be pursued will be announced in due course.
| Option | Details |
|---|---|
| Conduct a more in-depth comparison of models | Prepare standardized scenarios, with decision-makers’ inputs, and run them across the models compared so far, potentially in a selection of settings, in order to understand the causes of divergence in model results. |
| Expand the scope of the model comparison | Replicate the comparison exercise with a focus on regional and national models, potentially working with modellers and decision-makers from a selection of settings to identify and develop bespoke models. |
| Integrate health economics in the model comparison | Compare models with a focus on the extent to which they integrate health economics considerations; develop guidance conducting health economics research in the context of COVID-19. |
| Health technology assessment (HTA) for COVID-19 commodities | Pilot approaches for the model-based economic evaluation of COVID-19 commodities, including tests, treatments and early HTA for vaccines. |
| Model non-COVID-19 excess deaths | Shift away from COVID-19-only models towards developing solutions for taking a whole of health approach and estimate the indirect health effects of COVID-19 in the context of specific policy scenarios; calculate the net health benefits of policy responses. |
The CMCC was convened with funding from the Bill & Melinda Gates Foundation and the Ministry of Higher Education, Science, Research and Innovation, Royal Thai Government. It brought together the international Decision Support Initiative hosted by the Center for Global Development, the World Bank, the World Health Organization, and the Thai Ministry of Public Health to provide management and technical expertise to deliver global knowledge products. Other partners include the Data 4 Sustainable Development Goals Partnership, the UK Department for International Development, the Norwegian Agency for Development Cooperation, the US Centers for Disease Control and Prevention, and the US Agency for International Development.
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
