The last weeks have seen headlines about the effectiveness of several new coronavirus vaccines. But the coverage at this point has been based on press releases – tiny snippets of results that have been pulled from clinical trials. This week, the real deluge of data will begin. The Food and Drug Administration analyzed thousands of pages of data for the BioNtech-Pfizer vaccine, in preparation for a one-day meeting on December 10 to decide whether to authorize its emergency use – and detailed summaries of those data could be released as early as Tuesday. Then the agency will repeat the process next week for the Moderna vaccine. The first publications of vaccine efficacy data in medical journals should also be released soon.
Experts and the news media will have a lot to digest. If the past is any guide, there will be a lot of error, misunderstanding and communication snafus in the coverage, as well as active disinformation campaigns on both sides to stimulate hype or spread fear. Here are three tricky or misleading claims to watch out for:
Misleading Claim 1:
The trials were so huge, all the results must be flawless. Or else, The trials included so few people who actually fell ill, the results must be unreliable.
We’ve been conditioned to think that the size of a study is pretty much the most important thing about it. This is how they are always described to us: “A study of 50,000 people revealed so and so.” But we shouldn’t be so easily impressed by the magnitude of the sound of an essay! For effectiveness, its power is still over the number of “events” that occur during the study – in this case, the number of people who have fallen ill with Covid.
Take the famous 1954 field test for the Salk polio vaccine, which included an incredible 1.8 million children! But the number of events used in the most critical analysis – a placebo comparison involving a subset of more than 400,000 children – was only 143. It is the number of children in the study that developed paralytic polio, and that was enough to be sure at all. important finding that the Salk vaccine worked. The BioNTech-Pfizer trial had 43,000 participants and 170 events; the Moderna trial had 30,000 participants and 196 events. These figures represent an effective means of obtaining urgent answers. They do not suggest excessive or risky cuts.
At the same time, it is important to remember that not all reported numbers will be based on a full analysis of all data. The BioNTech-Pfizer vaccine is said to be 95% effective; Moderna, 94.5 percent. These are the most important results. As more and more data comes out, it’s almost certain that finer – but potentially less reliable – analytics will make the news and be presented as if they are just as strong. We already saw this happen a few weeks ago, when a BioNTech-Pfizer press release reported that “efficacy in adults over 65 was over 94%”. Points of sale such as ABC News went ahead and conveyed this conclusion directly to their readers. But we cannot be as certain as possible of the overall effectiveness. When a little more data on the vaccine were published when it was authorized by the UK government Last Wednesday, he showed that there weren’t enough participants in the study’s oldest age group (over 75) to be so sure about this result. It is still an important discovery; it’s just a more tentative question. The same can apply to other statistics which will be communicated in the days to come. How effective is the vaccine in preventing infection, for example, or in protecting people with chronic illnesses from Covid? Don’t assume that side results like these will have the same level of confidence as the main ones.
Misleading Claim 2:
Now we know that the claims of “95% effectiveness” were exaggerated.
Be prepared to see a bunch of new effect estimates in the days and weeks ahead – and some of them could make vaccines less useful in preventing Covid than early reports. This is not a sign that early analyzes are unreliable. The 95% efficacy calculations were based on data from only those participants who took both shots of the vaccine, with some extra time for the most complete immunity to kick in. obtained Covid before this point is reached; some who dropped out of the study before receiving their second injection; and some who have stopped providing follow-up to researchers, so it won’t be clear if they’ve ever fallen ill. We will soon have analyzes that take these people into account, and they will show somewhat different levels of effectiveness. These can provide insight into how vaccines perform in practice, once deployed in our communities.