Meaning of Protective Efficacy from Vaccines

The other day I came across a debate in a vernacular channel where there was a reference to at least 60% protective efficacy from vaccines against COVID-19 even for some new variants of the virus even if one cannot say much about future variants. From the discussion, there seems to be a popular misconception in the understanding of protective efficacy. The purpose of this write-up is to address that misconception.

In popular understanding, 60% of protective efficacy from vaccines could mean that if 100 people take vaccines then 60 of them would be protected from the disease.

In actual practice, protective efficacy refers to relative risk reduction (RRR). Or,

RRR=1-RR.

RR is relative risk, that is, the ratio of the risk in the treatment group (proportion infected from among those who have been administered with the vaccine, R_t) with the risk in the control group (proportion infected from among those who have not been administered with the vaccine, R_c). In other words,

RR=R_t/R_c.

If the proportion of infected is equal in the two groups then a benchmark relative risk, R_tb/R_cb=1. RRR is a reduction from this benchmark. Hence, 60% of protective efficacy in the debate should have been identified with RRR%=RRR*100.

Now, what could this 60% of protective efficacy imply. If the treatment and control groups had 100 people each and if the number of people infected in the two groups are 2 and 5, respectively, or, R_t=2% and R_c=5%, then RRR=60%. This means that if there are 100 people each in the two groups then 2 could be infected from those vaccinated and 5 from those not vaccinated.

But, one would get the same value of RRR=60% with umpteen other possibilities where both the above-mentioned values of R_t and R_c are multiplied by a common factor, k. An extreme situation can be when R_t=40% and R_c=100%, which was the position taken by a panelist in the debate who conveyed that among those vaccinated 40% would be infected and among those unvaccinated everyone would be infected. This is an extreme scenario and in some sense a misrepresentation of facts.

It calls for the relevance of absolute risk reduction, ARR=R_c-R_t. A Lancet Microbe paper (see discussion in earlier blogs here and here), drawing from phase 3 trials of five COVID-19 vaccines indicated that RRR was in the range of 67%-95% while ARR was in the range of 0.84%-1.28%. And, in a population setting for which data was available in one case, ARR comes down to 0.46%.

Based on this, it is quite likely that RRR=60% is to be commensurate with ARR in the range of 0.2%-1.5%, that is, R_t=0.2% and R_c=0.5% where ARR=0.3%, or, if one wants to give some benefit of doubt then R_t=1.0% and R_c=2.5% where ARR=1.5%. By not doing this, the panelist amplified the risk by 40-200 times. This can create panic and fear among the public and should be avoided.

The role of the State in providing vaccination to people by considering it as a public good is a relevant matter. Equally important, in a democratic polity, as conveyed by another panelist in the debate, is the fact that that vaccines being administered are with Emergency Use Authorization and that it is a voluntary act by an individual  who has to weigh the information provided. An informed consent and right to refuse, after being provided with information that leaves questions unanswered, is not the same as hesitancy. For a complex, evolving and uncertain scenario, questions would be the first step for better science.

Comparing Deaths from COVID-19 among Uttarakhand Police in Two Periods

The Indian Express put up a write-up on incidences of COVID-19 and fatality among police personnel of Uttarakhand. It indicates that the police personnel of the state had started vaccination in January 2021 and more than 93% among them have been vaccinated. This is an impressive figure because for a disease with reproduction number or R₀ "R naught" around three (R₀=3), the proportions needed for herd immunity (including natural infection) should be around 67%.

Given this backdrop, it might be a matter of concern that in April and May of 2021 the number of police personnel inflicted by the disease in Uttarakhand were 2,382. However, it is comforting that from among these 2,204 have already recovered while five (5) of them, bless their souls, are no more with us. The proportion of death from those inflicted by the disease is 0.21% (which could increase if there are additional deaths from the remaining 173 who have not recovered). The write-up further points out that the severity of the disease and and the number of deaths were lower than previously observed.

For instance, the proportion of deaths from the start of the pandemic till March 2021 was 0.40% (that is, 08 deaths from among 1,982 inflicted by the disease). Even if one ignores the fact that we are comparing more than 13 months of data with two months and that there are 173 cases for which we still do not know the final outcome but consider them to have recovered, a proper comparison would be to estimate absolute risk reduction (ARR) and relative risk reduction (RRR) and their statistical significance for deaths among those inflicted by the disease in the two periods. 

The ARR and RRR estimates and their 95% confidence intervals are:

ARR%=0.19% (95% CI: -0.14%, 0.53%) and

RRR%=48.1% (95% CI: -58.7%, 83.0%). 

This indicates that when one compares the second period with the first, ARR in per cent is 0.19% and RRR in per cent is 48.1%, but more importantly both the estimates are not statistically significantly different from zero. This suggests that one should be cautious with the comparative statements between the two periods by using absolute numbers.

ARR is d1-d2 

95% Confidence Interval (CI) for ARR is ARR±1.96*Standard Error (SE) of ARR

SE of ARR is {[(d1*r1)/N1]+ [(d2*r2)/N2]}^(1/2)

RRR=1-RR; RR is d2/d1

95% CI for RRR is 1-95% CI of RR

95% CI of RR is {exp^[Ln(RR)±1.96 SE of RR]}; SE of RR is {[(r1/d1)/N1]+ [(r2/d2)/N2]}^(1/2)

In the above calculations, d  and r denote share of death and share of recovery from among those inflicted by the disease with subscripts 1 and 2 denoting periods 1 and 2, respectively, such that in each period their sums add up to unity, d1+r1=d2+r2=1. N1 and N2 denote number of people inflicted by the disease in periods 1 and 2 respectively.

The write-up also had mentioned that from among the family members of the police personnel 751 were inflicted by the disease and from among them there were 64 deaths. This is indeed an important sacrifice by their family members. The adverse effects, in terms of deaths, is much higher then that on the actual force and this should be an important concern.

One is also curious to know what would be the incidences and deaths by family members across the two time periods. Has that number increased in the second period then a question that comes to mind is the possibility of adverse impact having increased after a public health intervention among police personnel. A period wise break of the data will help us examine that.

Vaccine Effect in Trials and in Populations

In my blog How Not To Read Vaccine Efficacy based on a Lancet Microbe paper it was shown that five COVID-19 vaccines with emergency approval (that is, by bypassing certain regulatory requirement) had relative risk reduction (RRR=1-RR; RR=SDT/SDC, relative risk is the ratio of the shares exposed to disease in treatment and control arms) that ranged from 67% to 95% but, their absolute risk reduction (ARR=SDC-SDT) ranged from 0.84% to 1.28%.

The Pfizer-BioNTech (BNT162b2 mRNA) as per its phase 3 trial has an RRR of 95%. This immediately gives the impression that the proportions of the shares exposed to disease are 1% for treatment and 20% for control, but this could be 2% and 40% or any in other proportions that satisfy k times 1% and k times 20%.

If the proportions for SDT and SDC are 1% and 20%, respectively, then ARR will be 19%. But, ARR is actually 0.84%, which suggests that in the trial k=0.044. And, hence, SDT=0.04% and SDC=0.88%.

It is difficult to replicate this in a population, as design and methods cannot be the same as in a trial. However, following mass vaccination in Israel using Pfizer-BioNTech (BNT162b2 mRNA), a study shows that RRR is 94% (almost close to what was observed in the trial), but ARR was 0.046% (SDT=0.02% and SDC=0.48%) suggesting that k=0.024 is much lower than the trial.

A further decline in ARR means that the number of people that need to be vaccinated (NNV=100/ARR) to prevent one more person from being exposed to the disease is further increased by 83% from 119 in the trial to 217 in the specific population setting. In other words, the vaccine effect was further reduced in a population setting when compared to the trial.

The authorities associated with mass vaccination need to design studies to arrive at RRR and ARR, along with other aspects, in their population, as that would be of help in their public policy designing. The individual concerned should also have such information, along with other aspects, for them to take an informed decision.

How Not To Read Vaccine Efficacy

To address COVID-19 pandemic, many vaccines have received emergency approval (that is, by bypassing many regulatory requirements that are otherwise needed for vaccines or other therapeutic interventions). The third phase trial data for relative risk reduction (RRR) are referred to as efficacy of the vaccines. 

A popular perception is that if an individual takes the vaccine then the possibility of being afflicted by the disease gets reduced by this RRR%. This, however, is not the case. Then, what does this efficacy mean? From a lay perspective, RRR% is total risk in per cent (100%) minus relative risk in per cent (RR%).

Now, what is relative risk (RR) in a trial. It is the ratio of the share of those afflicted by the disease in the treatment arm of the trial (SDT=nt/Nt, number afflicted by the disease in the treatment arm of the trial divided by number of people in the treatment arm of the trial) to the share of those afflicted by the diseased in the control arm of the trial (SDC=nc/Nc, number afflicted by the disease in the control arm of the trial divided by number of people in the control arm of the trial). In other words, RR=SDT/SDC and when it is multiplied with 100 it gives us RR%.

The absolute risk reduction in per cent (ARR%=SDC%-SDT%) is the proportion of those afflicted by the disease in the control arm of the trial (SDC%) minus the proportion of those afflicted by the disease in the treatment arm of the trial (SDT%).   

A recent Lancet Microbe paper reports that for five COVID-19 vaccines that have received emergency approval the ARR% are as follows: 

1.28% Oxford-AstraZeneca (ChAdOx1 nCoV-19), 

1.24% Moderna-NIH (mRNA 1273), 

1.19% Johnson & Johnson (Ad26.COV2.S), 

0.93% Gamaleya (GamCovidVac) [Sputnik V], and 

0.84% Pfizer-BioNTech (BNT162b2 mRNA).  

However, the RRR% for these are:

66.84% Oxford-AstraZeneca (ChAdOx1 nCoV-19), 

94.08% Moderna-NIH (mRNA 1273), 

66.62% Johnson & Johnson (Ad26.COV2.S), 

90.97% Gamaleya (GamCovidVac) [Sputnik V], and 

95.02% Pfizer-BioNTech (BNT162b2 mRNA).

Note that the reporting of RRR% are at times in misleading ways, as a Lancet Infectious Diseases paper shows. An informed consent, among other things, should also provide this perspective to the vaccine recipients before they take a decision.