What Does Success Look Like for a COVID-19 Vaccine? Improving Portfolio Level Understanding

Around the world COVID-19 and the policy responses to it are causing unprecedented disruptions of life and wellbeing. It has triggered what is likely to be a global economic recession on a scale that we have never experienced before, leaving the development of a vaccine our only reliable exit strategy, for now at least. As scientists and governments around the world race to develop a vaccine at scale, we take a look at what a portfolio approach entails and why it is so important.

Why it is useful to understand the overall portfolio

With the stakes being so high and unprecedented amounts of public monies “pushing” and “pulling” products along, portfolio risk analysis and hedging are increasingly being discussed, alongside research aiming to understand the optimal research and manufacturing strategy. Below, we discuss what risk analysis is and why it matters.

In the race to get to a COVID-19 vaccine(s), each vaccine candidate is not independent of the rest. They all rely on access to the same financial resource envelope, they may compete for manufacturing capabilities and raw materials, and their action mechanisms are often underpinned by the same scientific principles. If successful, they target the same demand pool and government budgets. With strong financing, scientific, production and market interdependencies, the probabilities of success of each product in a vaccines portfolio are, therefore, interlinked. A portfolio analysis is about understanding all the vaccines in the pipeline, and identifying risks of failure or areas that are over or under invested.

Such a wide analysis is useful in a number of ways:

• To push funders making early investment calls—as we have seen the US government do, and also CEPI, the Gates Foundation, and the recently announced initiative by a group of European countries.
• To those designing pull mechanisms at risk (rather than based on performance) such as the Gavi AMC (working with CEPI), or the Kremer proposal.
• To vaccine developers themselves, policymakers, and planners.
• To a more general policy audience that has a stake in the way taxpayer money is spent and of course in getting access to a successful vaccine.

A robust and precise portfolio analysis is (or ought to be), therefore, a highly valued global public good.

With little attention on performance-based pull and with the bulk of public money going towards early push and late stage volume commitments coupled with gigantic manufacturing scale up, picking some early winners and getting ready to produce them in large quantities may look like the world’s only way out. Understandably, it is important to be ready to ramp up capacity with some early investments so that we can manufacture enough vaccines for everyone including low- and middle-income countries (LMICs). But is arbitrarily picking winners the right approach?

Pressure to pick winners quickly might lead to a poor quality vaccine portfolio.

In order for the world to get the COVID-19 vaccine that it needs, a lot of things will need to fall into place. The vaccine candidates need to appear safe and effective in animal studies, in small trials in healthy volunteers, and then in large trials in representative groups of people, including the young, the elderly, and the sick. We need to manufacture the raw ingredients of the vaccines; to purify them*; to formulate and add adjuvants, stabilizers, and preservatives; to package and distribute; and deliver them to people. At present, most governments and companies look like they are trying to independently pick vaccines that will win this obstacle race, and perhaps boost their domestic biotechnology and pharmaceutical capabilities at the same time in ways that lack transparency or a clear scientific rationale.

The problem with this approach is that the obvious “winners” may be too similar to one another in terms of underlying technology, the way they are trying to make people immune, or the manufacturing capacity they require. But that’s too many eggs in too few baskets. We should include in the portfolio enough candidates which don’t look like obvious winners, but which won’t fail for the same reasons. Maybe they generate immunity in a different way? Maybe they rely on slower, but older and less risky, manufacturing? The world should be hedging its bets. After all, minimising the risk of getting nothing that can be used for wide population level prevention is just as important as maximising the speed of getting something.

Decision making is becoming increasingly centralised and antagonistic, with governments, technocrats and UN advisory panels, making funding and hence portfolio shaping decisions way “by committee” and beyond early grant-type decisions, including scaling up manufacturing capacity, all at risk to the public purse. There is good reason to think that with the pressure to pick winners fast the current portfolio is not optimal with what may be an unwarranted over-emphasis on vaccines that target the virus’ spike protein. Similarly problematic are the USA and China overly favouring their own domestic manufacturers.

So the combination of an understandable urgency to succeed, succeed now and succeed using a government’s own national (US, Chinese or UK) company, may end up reducing our chances of obtaining safe, efficacious vaccine(s) in sufficient quantities to serve global demand. 

While we wait for a vaccine, let’s improve the global COVID-19 policy response

Understanding how long it will take to get to a vaccine for wide scale population level use (in addition to risks, expected costs to taxpayers and likely economic outcome) is crucial in shaping the policy response including a gradual lifting of suppression measures and more general policy planning. The decisions that governments, industry and individuals make in the next few months will vary greatly depending on estimates of vaccine timelines. If we could say with a high degree of certainty that the world was likely to produce up to 8 billion doses of a new vaccine in the next twelve months, decisions about the health system, infrastructure, financial aid and on how to live our lives, would be very different than if developing and deploying a successful vaccine at scale was more than two or three years away. This is why it is so important to have a comprehensive portfolio analysis. While it would be impossible to be precise with estimates, it would give us the best chance of planning ahead. A rigorous portfolio analysis may facilitate better understanding of production scale-up timelines, and potential characteristics of the early entrants. These will help governments and global organisations (such as Gavi and WHO) plan for their distribution and rollout in a geographically and population sensitive way.

What does success look like?

A lot of the discussions about fast tracking a COVID-19 vaccine assume a linear process. The assumption is that we can get to the right product with a lot of carefully (albeit non transparently) allocated push funding coupled with pull in the form of manufacturing capacity at risk plus volume guarantees for the more advanced products (but as yet unregistered and unproven). Add on a global purchasing agreement under fair allocation rules set out by WHO or another norm-setting body and we stand a chance in winning the COVID-19 war globally. If the above sounds technically and politically complex, unfortunately, things are (even) more complex than that.

True we have a WHO TPP (Target Product Profile) setting out the minimum and maximum performance characteristics of a COVID-19 vaccine. However, success is hard to describe and conditional on population and sub-population characteristics, geographical variation and the epidemiological counterfactual on or two (or more) years into the future. A relatively less effective product which however does not require a cold chain may be ideal for Sub Saharan African countries but rejected by a High-Income Country (HIC) if a more efficacious alternative existed. A vaccine that is unsuitable for immunocompromised populations may be less appropriate for countries with large numbers of inadequately controlled HIV infections, for example. And what if the first entrant is a partially efficacious vaccine – e.g. one that is contraindicated for old people or people with major comorbidities but with an acceptable risk profile for younger health care workers? Or one that requires multiple doses or does not offer long term immunity and thus are less practical/more expensive for low income settings? Or, perhaps the worst nightmare of all, a vaccine fast tracked through trials ends up being unsafe?

A successful COVID-19 vaccine is not a binary state of affairs. Indeed, recent analysis shows that even at the minimum acceptable efficacy threshold set out by the WHO TPP, extremely high coverage rates will be needed to manage the outbreak even in a HIC setting; vaccine development is never going to be a shot on goal.

In another stream of work some of us have been reflecting on elements of value as a means of informing a genuine pull, crowding in private monies and insuring public payers against the probability of overpaying for suboptimal early choices. Such work can and ought to inform structured expert opinion elicitation which feeds into portfolio shaping tools. This way probability of success simulations could account for conditionalities and interdependencies and build off the various (as many as possible) characteristics of a successful product. Ultimately the value a vaccine confers to those receiving it or its benefit/risk ratio at individual and population levels must form part of the calculus for making vaccine investment decisions. 

To understand value and make a meaningful contribution to the ongoing discussions and actions, we think it is important to understand the vaccine portfolio. CGD is working with colleagues who specialise in vaccines, pharmaceutical portfolio forecasting, and running simulations. We are collecting data and hope to publish a portfolio analysis and our underpinning methods, soon. We are looking to speak to as many vaccine, manufacturing and supply chain experts as possible, to get the best estimates for probability of success. If you have such expertise or know somebody who does, please contact Anthony.