We Need Better Clinical Trial Systems before the Next Pandemic: The World Health Assembly’s Resolution Is a Good First Step

Yesterday at the annual World Health Assembly in Geneva, member states passed a resolution tabled by Argentina, Peru, and the UK that aims to strengthen clinical trials to improve public health. As COVID-19 has made abundantly clear, the evidence generated by high-quality clinical trials is essential to responding quickly and effectively to emerging global health threats. But while the goal of strengthening the world’s clinical trial capacity is laudable, and the resolution is a positive step in that direction, it does not, in its current form, go far enough in addressing the public health needs of low- and middle-income countries.

Clinical trials in the time of COVID

When health officials across the world first started to hear reports of an unusual outbreak of “pneumonia” in Wuhan, China, we knew nothing about the virus causing it nor much about the disease COVID-19. Scientists across the world worked to sequence the genome, establish how to prevent transmission, and, eventually, to discover an arsenal of tools to combat it. In January 2020 alone, 320 studies into COVID-19 therapies started, as tracked by the University of Oxford’s Bennett Institute for Applied Data Science.

During the first 10 months of the pandemic, trials were conducted solely within in 127 countries; there were another 256 trials were run in multiple countries. However, the size and quality of these studies were problematic. As the UK government’s chief medical advisor Chris Whitty said, many COVID-19 therapeutic trials were “insufficient in size, methodology and conduct, failing to generate the robust clinical evidence needed to inform decision making and practice change.”

The speed that studies were undertaken at, and answers were found, is unprecedented and commendable, but they still took time.

Unfortunately, the infection did not wait for the results from these studies, but instead increased exponentially over the following few months, racing through the world at unprecedented speed. Doctors and health systems were forced to guess what treatment would best protect people from COVID.

Figure 1. Number of trials launched into COVID-19 therapies in the first 10 months of 2020

Image

Data from University of Oxford’s Bennett Institute for Applied Data Science

Observational trials are usually much less rigorous than interventional ones, because it is difficult to identify what would have happened if the patient had not been given a specific drug. In contrast, standard practice in an interventional trial is to randomly allocate patients between a new treatment and either a placebo or the best existing treatment. Ideally interventional studies should have a large number of patients before clinical decisions are made based on their results. Yet few of the interventional trials enrolled more than a thousand people—just 8.4 percent. Most were far smaller, with a median interventional trial size of exactly 100. The median size for observational studies was 257. Most of these trials were small because they were testing drugs we knew were safe to humans.

This is a problem because you need a large number of people in a study to outweigh the randomness inherent in the world. Consider, for example, an illness that killed 2 percent of people. In a 100-person study, there is a 13 percent chance that no patients who would have died will have been enrolled in the study; there is a 60 percent chance that one of the control or treatment groups does not have any such patients. So even if the drug being tested prevented all deaths, it would be impossible to know if the difference between the treatment and control group was just random. To test how big a study needs to be to overcome randomness, researchers use power scores, which estimate how big a study needs to be to capture the effect being examined. To have an 80 percent chance of identifying a cure for the illness in this hypothetical, you would need 774 patients in a study. If instead the treatment only averted half of fatalities, 4,636 patients would be needed. There were just 87 trials in the first 10 months of COVID that included this many people. As the benefits to treatment become more incremental this size gets greater still. Researchers can often reduce the size of trials by measuring the impact of a treatment on risk factors or indicators that we know are linked to mortality.

The small trial size was a huge problem during COVID. For example, trial results released on March, 17, 2020 by professor Didier Raoult found that patients taking the malaria drug hydroxychloroquine became COVID-19 negative more quickly than the control group. In a world desperate for hope, the fact that just 24 people took part in this study was often overlooked (there were other methodological failings, and Raoult later faced disciplinary action). There were 349 interventional clinical trials during the first 10 months of 2020 that had 24 patients or fewer. Even if every other part of the study was well run, it is likely that there would always be at least one where chance meant that people assigned to the treatment group did far better than those in the control group.

Despite this, the French government authorised hydroxychloroquine as a treatment against COVID-19 and then US president Trump announced that he was taking it prophylactically. A global shortage of the drug ensued, causing problems for those who relied on it for other treatment. On April 30, after examining trials with a combined 6,000 patients that did not identify any medical benefit, the WHO announced that it was recommending against the use of hydroxychloroquine. A similar story can be told about Ivermectin, a deworming drug that only helps COVID-19 patients who also have worms.

As well as questions of the standard of study, there needs to be better coordination to reduce duplication and prioritise research towards areas of greatest need. Despite hydroxychloroquine being shown to not work in April 2020, by July there were still almost 200 trials examining it as a treatment (it has been suggested that the continued studies of hydroxychloroquine were politically driven).

There are many good examples of clinical trials that were big enough and provided hugely valuable information that helped reduce the case fatality rate of COVID-19. The largest trial run at the start of the pandemic was the Randomised Evaluation of COVID-19 Therapy (RECOVERY), which was started in the UK by scientists at University of Oxford but operates from 200 hospitals in six countries. This study has enrolled almost 48,000 people to date. At its height, almost one in four patients who went into an NHS hospital with COVID was included in this a trial. In four months the RECOVERY trial discovered the first successful treatment, dexamethasone, which the NHS estimates saved a million lives during the first year of the pandemic. The RECOVERY trial has completed tests on 10 potential treatments, playing a part in ruling out drugs such as hydroxychloroquine and identifying several that work. RECOVERY trials are still testing a further five treatments.

A shortcoming of the RECOVERY trials is that the vast majority of the patients recruited were in the UK, meaning the information was less useful in countries that differ greatly from the UK, particularly low- and middle-income countries. When the pandemic struck, hospitals had to be quickly recruited into studies; the speed at which COVID-19 trials were set up was unprecedented and, at a time when tens of thousands were dying every day, highly valuable. When the next pandemic hits, there would be real value in having similar architecture ready to go in many countries.

The WHA resolution on clinical trials

To that end, Argentina, Peru, and the UK proposed a resolution to the World Health Assembly to improve clinical trials. The resolution calls for the WHO’s Director-General to present a global action plan on clinical trials next year that:

• improves the standard of clinical trials and ensure they’re of adequate size
• increases the capacity for trials to be run quickly
• improves coordination and reducing duplication in trials, as well as ensuring they target illnesses of importance
• standardises data protocols, including the timely sharing of pre-publication data with health authorities, and where available and appropriate trials registering trials in a registry.
• ensure trials include all major population groups which an intervention is intended to benefit
• ensures trial results are quickly incorporated into clinical guidelines.

The resolution and Director-General’s global action plan still need to translate into national legislation for these lofty goals to be realized.

While I support the broad intention of the resolution, there are several problems with the specifics of its proposals that I hope to see addressed when the WHO Director-General gives his report next year.

The resolution calls for trial registrations on a registry, such as that run by the WHO. However, the resolution should have been stronger, mandating that trials are registered in advance, so that if the trial sponsors do not like the results, they cannot hide them. Despite WHO recommendations that all trials are registered, around half of all trial results are never published. Key information is often left out of registries and they are underfunded. In seeking better information sharing, the resolution should have looked to address these problems.

A second problem with the resolution in its current form is that it discusses the need to better ensure clinical trials are targeted at treatments for diseases of global, regional, and national importance, but it does not outline what these are or how they can be identified. The global research agenda is too driven by the global North—this is recognised in the resolution. If the WHO is going to get more involved in the international processes for clinical trials, then it should have a plan for addressing this imbalance.

The resolution should have looked at how clinical trial results are assessed and regulated. There is no global clinical trial regulator; instead international institutions often look to those the WHO has classed as stringent regulatory authorities (SRAs)—regulators the WHO believes meet a particularly high standard. There are currently 36 SRAs—30 come under the European Medicines Agency, and then Australia, Canada, Japan, Switzerland, UK, and US—and all are in very wealthy countries. These regulators are often used to judge quality in global health; for example, World Bank loans are available for COVID-19 vaccines which met certain conditions, including approval by an SRA (unless fully approved by WHO prequalification, which no vaccines were). However, an SRA’s mandate is to regulate pharmaceuticals for its own domestic market, and vaccine manufacturers not looking to sell in these markets often didn’t apply for this type of approval. If global systems are going to rely on a small number of regulators to judge products, then some of these need to be in LMICs. This should be included in discussions of standardising trial processes.

Despite these shortfalls, I am glad the World Health Assembly passed this resolution. There will be huge benefits to running clinical trials in a more organised way, in terms of speed, quality, and cost. A Wellcome Trust working group into clinical trial networks for antibiotics—which I led—estimated that the cost for clinical trials could be reduced by 23 percent per patient, and overall costs could be reduced by 40 to 60 percent if control groups could be shared between trials; this would also make trials faster. In short, the cost in both money and time of setting up clinical trials is substantial, from designing protocols, getting ethical approval, finding sites, and training people in the protocol, to name but a few. Whilst it will not be possible to design all of these processes for an unknown future pandemic, creating a framework and standard would see many of the same improvements.

Greater rules standardising trial protocols and data sharing could ensure that different studies are more easily comparable, and that those setting clinical guidelines have access to all of the relevant data, so that clinicians can make the best decisions possible, and patients can get the best care possible. A global action plan would mean that we move away from a world where a low-quality, 24-patient study can influence health decisions, and allow us to get evidence more quickly, so that as many lives as possible could be saved in the future.