A Review of Vaccine Efficacy Measures

Sayan Dasgupta, PhD

Staff Scientist, Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, 1100 Fairview Ave N, M2-C200, Seattle, WA 98109, USA; E-mail: sdasgup2@fredhutch.org

INTRODUCTION

One of the crucial chapters in the history of science is the advancement of vaccine research and their impact on human longevity and health.¹ History of modern vaccination officially began with the discovery of smallpox immunization by Edward Jenner in the late 18^th century.² Since then, substantial progress has been achieved in the prevention of infectious diseases with inactivated vaccines, and a number of major disease agents have been controlled (most notably, smallpox, poliomyelitis, rabies, diphtheria, tetanus, per-tussis, measles, mumps, and rubella).³ Progress in vaccine research has resulted in a significant decrease in infection-associated morbidity and mortality, and as knowledge of microbiology and immunology began to grow through the 20^th century, the science of vaccinology continued to rapidly evolve, putting more emphasis on the importance of developing safe and effective strategies for infectious disease prevention in the 21^st century.⁴

One important discussion in the public health community is regarding how to optimally assess and measure the full public health value of preventive vaccines and incorporate that knowledge into the evidence-based decision-making process of vaccine licensure and recommendations for public health use.⁵ Thus, before each new vaccine is considered for licensure, the crucial question that needs to be addressed satisfactorily is “How well does the candidate vaccine prevent the disease?” This question, seemingly basic, often becomes quite complex to answer for health practitioners. One key step to remedy that is to enhance knowledge and understanding of vaccine epidemiology among health practitioners, as well as policy makers, public health experts, etc. It has also been argued that such knowledge will benefit the society through informed decision-making and improved vaccination coverage.⁶

Epidemiology is the study (scientific, systematic, data-driven) of the distribution and determinants (causes, risk factors) of health-related outcomes in specified populations, and the main application of this discipline is to advance best practices in public health. While vaccinology delves into understanding how vaccines work, epidemiology helps to ascertain the public health impact of a particular vaccine, and whether it is needed in the targeted population. Hence it is essential for understanding the implications of a vaccination program on the community and individuals. ‘Vaccine epidemiology’ thus builds a bridge between public health, basic medical sciences, and clinical medicine, and is aimed at maximizing the benefit of existing knowledge in these areas.⁶

This review article outlines the key concepts in vaccine epidemiology, such as risk reduction, odds ratio, vaccine efficacy and effectiveness, basic reproductive numbers, herd immunity, etc, that are key to understanding the impact of vaccines for public health researchers.

Vaccine Efficacy vs Vaccine Effectiveness

Although the terms ‘efficacy’ and ‘effectiveness’ are used interchangeably in everyday language, in the context of vaccine studies the terms have distinctly different meanings.⁷ ‘Efficacy’ is defined as the percentage by which the rate of the target disease is reduced among those who are vaccinated compared to those who are unvaccinated under ideal and controlled circumstances.⁸ Hence, efficacy is typically measured in the context of a placebo-controlled randomized clinical trial as the ‘per protocol’ efficacy (that is, only in individuals who followed the recommended schedule), as the intention is to establish the biologic performance capacity of the product under optimal conditions.⁷ On the other hand, ‘effectiveness’ measures the percent reduction in the rate of disease as efficacy, but in the context of a routine, real-world use of the vaccine. It generally differs in magnitude from the efficacy seen in controlled settings, as in routine use program implementation is more variable than in clinical trial settings. Thus, vaccine effectiveness is defined as the reduction in the incidence of the disease for those receiving the vaccine intervention in real world settings and can be further categorized into direct, indirect, total and overall effectiveness.

Vaccination can reduce transmission in a community beyond the direct protective effect in vaccinated individuals. Indirect effects of a vaccine are the effects in individuals who were not vaccinated, or at least those who were not vaccinated as part of the strategy of interest, due to an increase in the population level of immune protection, also known as herd immunity.^9,10 Herd immunity describes the collective immunological status of a population of hosts, as opposed to an individual host, with respect to a given pathogen. The total effects are the combined population-level effects of the vaccination strategy and the direct protective effects of vaccination in those individuals who received the vaccine. The overall effect of a vaccination strategy is the average effect in the population in those who did and did not receive the vaccine compared to if the population had not had the vaccination strategy.^11,12 Thus, the overall effectiveness considers benefits accrued by both vaccinated and unvaccinated individuals, and it is the measure most commonly used to evaluate the impact of a mass vaccination program at the population level.¹³ Establishing that there is a significant indirect effect can have important vaccine policy implications,¹¹ and hence, the indirect causal effects of vaccination in populations have been studied extensively in the literature (Figure 1).^14,15,16

Figure 1. Different Measures of Vaccine Effectiveness and the Respective Study Designs
for Evaluation of Each Measure Based on Comparison Populations. Population A and B are
Separated in Every Way to Ensure that there is no Interaction Between them with Respect
to Transmission Dynamics. In Population A, Some Individuals are Vaccinated, while Others are not. In population B, Nobody is Vaccinated¹⁷

Measures of Vaccine Epidemiology

Vaccine epidemiology is defined as the study of the effects and interactions of vaccines on the epidemiology of vaccine preventable diseases. Understanding the pattern of diseases and their interactions with vaccines by different demographic and socio-economic factors, including geographical, income status, urban/rural, gender variations, etc, is crucial for advancing the ultimate public health goal of disease moderation and eradication, and the key to this knowledge is based on the principles of epidemiology. In this section, we will review some important concepts in vaccine epidemiology (Figure 2).

Figure 2. A Hypothetical 2×2 Contingency Table where ‘a’ and ‘b’ Represent the Number
of Vaccinated Individuals with and without the Disease Respectively, and Similarly ‘c’ and ‘d’
Represent the Number of Unvaccinated Individuals with and without the Disease Respectively

Absolute Risk Reduction (ARR): The absolute difference in risk between the unvaccinated and vaccinated.

where 

are the incidence rates of the disease among the vaccinated and the unvaccinated respectively .The incidence difference I_u -I_v is also known as vaccine preventable disease incidence or vaccine-preventable disease incidence (VPDI).

Number Needed to Treat (NNT): The number of patients needed to vaccinate to prevent one additional disease outcome:

  NNT=1/ARR

Risk Ratio (RR): The ratio of risk between the vaccinated and the unvaccinated. 

Odds Ratio (OR): The ratio of odds of the disease in the vaccinated. and that of the disease in the unvaccinated.

Relative Risk Reduction (RRR) or Vaccine Efficacy (VE): It is also called ‘prevented fraction among the vaccinated’ as it measures the proportion of the disease incidence among vaccinated persons which was prevented by vaccination, or equivalently ‘preventable fraction among the unvaccinated’, as it measures the proportion of the disease incidence among unvaccinated persons which is theoretically preventable by vaccination.

Vaccine efficacy (VE) has been alternatively called rate fraction, etiologic fraction, and an attributable fraction. The expression describes the fraction of cases prevented by the vaccine. VPDI, in contrast to VE, is not a fraction, but an incidence. Mathematically VPDI is equal to I_u ×VE. This latter formulation emphasizes that VPDI encompasses both VE and the background incidence of the disease syndrome in question.¹⁸ Vaccine efficacy can sometimes fail to capture the complete public health impact of vaccines and can be relatively low when preventable disease burden is high. In this regard, measures beyond efficacy (like VPDI) may be more appropriate and could have a role for both vaccine licensure and policy recommendations.⁵

Community Effectiveness (CE): Incidence rates among vaccinated or unvaccinated compared to the total population. It is also called the prevented fraction in population, that is, proportion of new disease cases in the total population that have theoretically been prevented by vaccination.

where I_p is the incidence rate of the disease in the population. This measure is related to VE as

CE=VE × PPV

where PPV is the proportion of the population vaccinated or coverage.

Basic reproductive number (R₀): Measures the average number of secondary cases generated by one primary case in a susceptible population.^19,20 The magnitude of R0 can be ascertained by cross-sectional and longitudinal serological surveys. In general, for an epidemic to occur in a susceptible population R₀ must be >1, so the number of cases is increasing. On the other hand, if R₀<1, the disease dies out as we have a shrinking pool of infected individuals.⁶

Effective reproductive rate (R_e): The effective reproductive rate R_e estimates the average number of secondary cases per infectious case in a population made up of both susceptible and non-susceptible hosts.

R_e =R₀ x

where x is the fraction of the host population that is susceptible.

Herd immunity Threshold (HIT): Herd immunity occurs when a significant proportion of the population (or the herd) have been vaccinated, and this provides protection for unprotected individuals. The herd immunity threshold is the proportion of a population that needs to be immune in order for an infectious disease to become stable in that community. If this is reached, for example due to immunization, then each case leads to a single new case (that is, R_e=1) and the infection will become stable within the population.^21,22

CONCLUSION

At the post-licensure level, we have entered an era of vaccine evaluation where all aspects of the public health value of vaccines beyond efficacy should be assessed.^5,8 Evidence of the protection afforded by new vaccines in the context of real-world immunization programs is important for accelerating and sustaining their uptake globally.^7,23,24 Furthermore, analytic plan reporting for pre-licensure phase III pivotal trials should also include incidence rate reductions (VPDI) and number needed to treat (NNT) besides just VE.

The understanding of vaccine epidemiology has the potential to save additional lives from vaccine preventable diseases and improve health outcomes. Vaccine epidemiology should be part of key modules in the teaching of undergraduate and postgraduate medical students. Public-health program managers and policymakers should be trained in vaccine epidemiology through continued medical education and on-the-job training programs.⁶

1. Plotkin S. History of vaccination. Proc Natl Acad Sci U S A. 2014; 111(34): 12283-12287. doi: 10.1073/pnas.1400472111

2. Riedel S. Edward Jenner and the history of smallpox and vaccination. Proc (Bayl Univ Med Cent). 2005; 18(1): 21-25. doi: 10.1080/08998280.2005.11928028

3. Weinberg GA, Szilagyi PG. Vaccine epidemiology: Efficacy, effectiveness, and the translational research roadmap. J Infect Dis. 2010; 201(11): 1607-1610. doi: 10.1086/652404

4. Hsu JL. A brief history of vaccines: Smallpox to the present. S D Med. 2013; Spec: 33-37.

5. Saadatian-Elahi M, Horstick O, Breiman RF, et al. Beyond efficacy: The full public health impact of vaccines. Vaccine. 2016; 34(9): 1139-1147. doi: 10.1016/j.vaccine.2016.01.021

6. Lahariya C. Vaccine epidemiology: A review. J Family Med Prim Care. 2016; 5(1): 7-15. doi: 10.4103/2249-4863.184616

7. Verani JR, Baqui AH, Broome CV, et al. Case-control vaccine effectiveness studies: Preparation, design, and enrollment of cases and controls. Vaccine. 2017; 35(25): 3295-3302. doi: 10.1016/j.vaccine.2017.04.037

8. Wilder-Smith A, Longini I, Zuber P, et al. The public health value of vaccines beyond efficacy: Methods, measures and outcomes. BMC medicine. 2017; 15(1): 138. doi: 10.1186/s12916-017-0911-8

9. Fox JP, Elveback LR. Herd immunity–changing concepts. Viral Immunology and Immunopathology. 1975; 273-290. doi: 10.1016/B978-0-12-522050-7.50020-4

10. Fine PE. Herd immunity: History, theory, practice. Epidemiol Rev. 1993; 15(2): 265-302. doi: 10.1093/oxfordjournals.epirev.a036121

11. Halloran ME. The minicommunity design to assess indirect effects of vaccination. Epidemiol methods. 2012; 1(1): 83-105. doi: 10.1515/2161-962X.1008

12. Halloran ME, Longini IM. Struchiner CJ, Longini IM. Design and Analysis of Vaccine Studies. New York City, USA: Springer. 2010.

13. Shim E, Galvani AP. Distinguishing vaccine efficacy and effectiveness. Vaccine. 2012; 30(47): 6700-6705. doi: 10.1016/j.vaccine.2012.08.045

14. Halloran ME, Struchiner CJ. Causal inference in infectious diseases. Epidemiology. 1995; 6(2): 142-151. doi: 10.1097/00001648-199503000-00010

15. Hudgens MG, Halloran ME. Toward causal inference with interference. J Am Stat Assoc. 2008; 103(482): 832-842. doi: 10.1198/016214508000000292

16. Vanderweele TJ, Tchetgen Tchetgen EJ. Effect partitioning under interference in two-stage randomized vaccine trials. Stat Probab Lett. 2011; 81(7): 861-869. doi: 10.1016/j.spl.2011.02.019

17. Halloran ME, Struchiner CJ. Study designs for dependent happenings. Epidemiology. 1991; 2(5): 331-338. doi: 10.1097/00001648-199109000-00004

18. Gessner BD, Feikin DR. Vaccine preventable disease incidence as a complement to vaccine efficacy for setting vaccine policy. Vaccine. 2014; 32(26): 3133-3138. doi: 10.1016/j.vaccine.2014.04.019

19. Dietz K. The estimation of the basic reproduction number for infectious diseases. Stat Methods Med Res. 1993; 2(1): 23-41. doi: 10.1177/096228029300200103

20. Rothman KJ, Greenland S, Lash TL, Greenland S. Modern Epidemiology. Philadelphia, USA: Lippincott Williams & Wilkins. 2012.

21. Guerra FM, Bolotin S, Lim G, et al. The basic reproduction number (R0 ) of measles: A systematic review. Lancet Infect Dis. 2017; 17(12): e420-e428. doi: 10.1016/S1473-3099(17)30307-9

22. Fine P, Eames K, Heymann DL. Herd immunity: A rough guide. Clin Infect Dis. 2011; 52(7): 911-916. doi: 10.1093/cid/cir007

23. Hajjeh RA, Privor-Dumm L, Edmond K, et al. Supporting new vaccine introduction decisions: Lessons learned from the hib initiative experience. Vaccine. 2010; 28(43): 7123-7129. doi: 10.1016/j. vaccine.2010.07.028

24. Mahoney RT, Maynard JE. The introduction of new vaccines into developing countries. Vaccine. 1999; 17(7-8): 646-652. doi: 10.1016/s0264-410x(98)00246-1