Introduction

Effective vaccination strategies against the SARS-CoV-2 pandemic require a large proportion of the population to have the vaccine [1]. In December 2020, when the first vaccine was approved, about 37 to 45% of the adult population in Germany definitely intended to get vaccinated, according to population-wide national polls [2, 3]. In December 2021, vaccination rates in Germany approximated a plateau below 85% of people aged between 18 and 59 years, and below 90% of people 60 years and older, respectively [4]. The vaccinated population included a large share of those who were undecideds (who initially did not show a preference for or against vaccination) and skeptics (who more likely would not have had a vaccination initially but were not categorically opposed to it) before (in sum about 24 to 25% of the adult population between April and December 2020 [5, 6]). Before vaccines became available at the end of 2020, undecideds and skeptics sought information to make vaccination decisions [7, 8]. In fact, however, persuasive measures–including political, societal, moral, and social pressure–that do not serve to inform citizens played a substantial role in these decisions population-wide [9].

Health communication that informs and potentially convinces undecideds and skeptics about medical countermeasures during an emergency [10] represents an alternative to persuasive strategies that threaten trustworthiness and credibility of the communicator and of vaccinations [11]. According to international standards of evidence-based health care [12] and the patient protection law in Germany [13]–along with the ethical ideal of evidence-based decision-making [14]–every citizen should be enabled to weigh the possible benefits and harms of medical options on the basis of the best available evidence and to decide freely on this basis (informed decision-making) [15]. Accordingly, the information communicated should enable undecideds and skeptics to compare the benefits and harms of having or not having the vaccination.

The objective of our studies was to investigate how a public health information intervention (“mRNA COVID-19 vaccination fact boxes”) that was jointly developed by the Harding Center for Risk Literacy and the Robert Koch Institute (RKI) might shape the evaluation of COVID-19 vaccination by means of balanced information. Because the best available evidence clearly shows effectiveness (preventing COVID-19 and related hospitalization) and safety (with limited severe side effects), inferred benefit-harm ratios may be convincing. Thus, informed decision making, which considers individual preferences, more likely encourages vaccination and is expected to result in a net surplus of proponents over opponents at the population level. The expected surplus would be in line with the societal preference to increase the vaccinating share of the population in order to protect public health.

A “fact box” is a tabular or graphical version of a balance sheet [16] that summarizes the best available evidence on the benefits and harms of medical options and how likely these will occur [17, 18]. Fact boxes inform various health decisions, including those about medical treatments, cancer screenings, and vaccinations [17, 19, 20]. In contrast to regulation, incentives, and (invisible) nudges [21], fact boxes are not designed to enforce directed behavioral change [22]. They are boosts that have been shown to enable comprehension of medical risks and options and help acquire short-term knowledge thereof [23, 24].

We hypothesized that an intervention with a COVID-19 vaccination fact box not only conveys knowledge and corrects risk perceptions but also can support positive COVID-19 vaccination evaluations without employing persuasion. We conducted four studies to inform the further development of a COVID-19 vaccine fact box and to evaluate its impact on vaccination knowledge and evaluation. First, we assessed the relationship between vaccination knowledge and intention in Germany (Study 1) to determine whether such knowledge potentially contributes to the intention to vaccinate. To inform further development of a COVID-19 vaccination fact box, we identified information requirements and needs of the population in Germany (Study 2). Because accurate risk perception is a prerequisite for informed decision-making [25], Study 3 –conducted in parallel to study 2 –tested whether the fact box improves COVID-19 risk perception, as compared with conventional information. Study 4 evaluated whether different fact box formats are effective for enabling informed vaccination intentions. Studies 1, 2, and 4 were population-wide surveys, each sampling from a multi-level stratified sample from a pool of 72,000 panelists who belonged to the 25 million members of the German PAYBACK customer loyalty program (S1 Fig).

Study 1

To provide a foundation for our plan to promote COVID-19 vaccination knowledge, we assessed the relationship between vaccination knowledge and acceptance in Germany over a three-month period that included introduction of the first vaccines. According to the 5C model [26], which describes the five psychological antecedents of vaccination (confidence in vaccines, structural and psychological constraints, complacency in disease risk perception, collective responsibility in protecting others), the motivation to search for vaccination information is often linked to negative vaccination intention (calculation). On the other hand, vaccination knowledge can be linked to higher vaccination intentions [27] and can close the gap between intentions and actual vaccination decisions [28].

Results

The cross-sectional analysis of five assessments showed increasing vaccination intentions between November 2020 and February 2021 in Germany. The cumulative proportion of proponents (probably or definitely intending to get the vaccination) grew from 54.4% (T₀, end of Nov) to 65.1% (T₄, mid-February, including 2.2% already vaccinated) (Fig 1). Each assessment after T₀ made a proponent on average 1.1 times (odds ratio [OR]) more likely (95%-confidence interval [CI] [1.07, 1.12]), χ²(1) = 48.55, p < .001, although the proportion of those who probably intended to get vaccinated decreased (p < .001). At the same time, the proportion of undecideds declined (p = .044). The proportion of opponents (“definitely not”) and skeptics (“probably not”) decreased until actual vaccinations began (p < .001), but remained stable in January and February (p = .228), at about 12% and 9%, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Proportion of respondents according to their intention to get vaccinated against COVID-19 or indicating that they already had.

Independent samples were weighted at the time of their assessment. Error bars show 95% confidence intervals.

https://doi.org/10.1371/journal.pone.0274186.g001

Furthermore, between the end of December 2020 and the beginning of February 2021 (when only mRNA vaccines were available in Germany) the samples surveyed showed increasing vaccination knowledge, F(1,5034) = 40.81, p < .001, β_standardized = 0.09 (Fig 2). Over this period, those with low educational qualification (no formal qualification, primary, or secondary school) showed a similar increase in vaccination knowledge (from mean (M) = 4.40, standard error (SE) = 0.08 to M = 4.67, SE = 0.07; p < .001) compared with those with higher educational qualification (university entrance qualification) (from M = 4.68, SE = 0.04 to 4.96, SE = 0.03, p = .001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Number of correct items increased over two months according to respondents’ intention to get vaccinated against COVID-19.

The independent samples were weighted at the time of their assessment. Error bars show the standard error of the mean.

https://doi.org/10.1371/journal.pone.0274186.g002

More positive vaccination intentions were associated with higher knowledge scores (correlation coefficients r_T1 = .36 and r_T3 = .36, each p < .001). Among those who were undecided in February, little improvement in knowledge could be observed when compared with undecideds at the turn of the year (+0.1 correct out of 4 test items about vaccine-related uncertainty; compared with +0.3 among proponents).

Study 2

The development process of evidence-based health information ideally involves patients and consumers [33]. First, it is essential to identify and to consider objective informational requirements, as identified by experts [34]: What information do recipients require? What do they not know? What false beliefs do they hold? Second, subjective information needs must be elicited and considered, not only with regard to decision-making but also with regard to developing trustworthy information, which may be unrelated to the expert’s view on which information is crucial. On the basis of a secondary analysis of data on perceived reasons for and against vaccination, we assessed the respective information requirements and needs of the population in Germany.

Discussion

Our analysis of open-ended responses showed that confidence-related information needs concerning vaccine efficacy, safety, short- and long-term reactions and harms, and uncertainty are the most essential targets of vaccination communication. This is directly in line with information requirements of health information guidelines [35] and with the aim of building trust.

In parallel to Study 2, a pilot format of the fact box was tested in Study 3 with regard to its effect on risk perception. Insights from both studies were used to inform the further development of a COVID-19 vaccination fact box (from S2A, S2B Fig to Fig 3 and S4–S6 Figs, e.g. by collapsing typical vaccination reactions).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. English translation of the simple fact box for people between the ages of 18 and 60 from Study 4.

https://doi.org/10.1371/journal.pone.0274186.g003

Study 3

Accurate risk perceptions of having or not having an intervention is a prerequisite of informed choices [25]. Although higher disease risk perceptions can be associated with higher vaccination intentions [36–38], overstating the risk of COVID-19 in vaccination communication can not only create fears but can also be detrimental to trustworthiness of medical and scientific experts, which in turn predicts uptake [36]. By communicating transparently and correcting false risk perceptions, fact boxes theoretically can inform the public without undermining trust in the communicating institution. Our online study with a convenience sample controlled whether fact boxes–as evidence-based health information that helps understand the risks of medical options [23, 24]–improve COVID-19 risk perception.

Results

Studying the fact boxes decreased (Table 1) numeric COVID-19 risk perception (ANOVA interaction effect: F(1,357) = 10.05, p = .002, n_p² = 0.03) compared with control presentation, leading to more accurate estimates (see S3 Fig, p < .001). Only control presentation increased (Table 1) both personal fear (ANOVA interaction effect of time and presentation format, with F(1,357) = 4.17, p = .042, n_p² = 0.01) and perceived severity of developing COVID-19 (interaction, with F(1,357) = 19.90, P < .001, n_p² = 0.05).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Personal and social fear, subjective and numeric risk perception across conditions, diseases, pre- and post-presentation.

https://doi.org/10.1371/journal.pone.0274186.t001

More positive evaluations (F(1,357) = 12.55, p < .001, n_p² = 0.03) and increasing intentions to get vaccinated (F(1,357) = 7.63, p = .006, n_p² = 0.02) were shown as ANOVA main effects, but there were no format-specific interaction effects (S8 Table). Here, control information may have been as effective as fact boxes, albeit by promoting fear and perceived severity.

Study 4

Based on the insights of Studies 1, 2, and 3, Study 4 examined whether different fact box formats are effective for enabling informed vaccination intentions in a population-wide sample. Undecideds and skeptics needed and desired information (Study 2) to weigh potential benefits and harms [7]; otherwise they were hesitant to get vaccinated [8]. Study 4 tested whether both complex and simple fact boxes–based on the implemented version–would improve vaccination knowledge and modify vaccination evaluation or intention.

Results

First, compared with no treatment, the sum score of vaccination knowledge was higher after fact box presentations (ANOVA main effect, F(1,3101) = 36.58, p < .001, n_p² = 0.01). Respondents of both observed age groups (below 60 and 60 onward) recalled vaccine efficacy, safety, and related uncertainties differentially when presented with a fact box instead of nothing (S9 Table). Multiple logistic regressions including the factors presentation format and age group showed that respondents of both age groups, who were presented with the fact box instead of nothing, were more likely aware of the side effect of fatigue (Odds Ratio (OR) = 1.85 with a 95%-confidence interval (CI) [1.69, 2.01]). Compared to those without fact box, respondents with fact box presentation, aged 60 and above, more likely confirmed a potential risk of facial paresis (OR = 1.16 [1.04, 1.28]), and younger respondents more likely recalled vaccine efficacy (OR = 1.13 [1.02, 1.24]).

Finally, vaccine efficacy was (OR = 1.85 [1.53, 2.17], p < .001) more likely correctly inferred when a fact box was presented (logistic regression controlling for the level of education).

Second, logistic regressions (χ²(1) = 14.70, p < .001 and χ²(1) = 5.50, p = .020) revealed that fact boxes more likely prompted both any change (OR = 1.36 [1.20, 1.52], p < .001) and a positive change (OR = 1.25 [1.06, 1.44], p = .020) of the evaluation of the vaccination compared with no intervention. For a distinct consideration of the different formats: Whereas 18.6% of respondents without any intervention changed their evaluation of a COVID-19 vaccination positively, 20.3% did so if studying a complex fact box, and 24.2% if studying a simple fact box. At the same time, however, 14.5%, 19.8%, and 16.3%, respectively, evaluated the evaluation more negatively when asked a second time (at post-assessment). Compared with no intervention, presenting a simple fact box (χ²(1) = 9.52, p = .002) more likely led to a positive than to a non-positive change (OR = 1.40 [1.19, 1.62], p = .002) in the evaluation of the vaccination. This could not be confirmed for the complex box (χ²(1) = 0.90, p = .343).

The shift in vaccination evaluation after being presented simple fact boxes (+7.9%) could, to a substantial extent, be related to the skeptical and undecided respondents’ comprehension of the information presented. Those who drew correct inferences about vaccine efficacy after having seen simple fact boxes showed a positive change in evaluation, but not those who drew incorrect inferences (main effect of an univariate ANOVA, F(1,467) = 3.88, p < .050, n_p² = 0.01). Explorative sub-analyses highlighted that this effect is due to the younger skeptics and undecideds (F(1,387) = 5.65, p = .018, n_p² = 0.01), not to those aged 60 and above (F(1,76) = 0.04, p = .836). In contrast, younger skeptics’ and undecideds’ recall, with at least 80% of correct responses after information presentation, was not related to positive changes in evaluation (F(1,387) = 0.59, p = .445).

Taking into consideration participants with different levels of education, we compared simple and complex fact boxes with no treatment. Respondents with the lowest educational qualification (16.5% with no school degree or degree after 9 years of schooling) recalled less than those with higher qualifications (degree after 10 years, university entrance qualification) (ANOVA main effect of sum score, F(1,3086) = 11.44, p = .001, n_p² < 0.01). But no interaction indicated an education-related recall benefit over the control condition (F(2,3086) = 0.20, p = .820).

Additionally, we compared the two types of fact boxes. Recall of information from simple (59%) but not from complex fact boxes (61%) was significantly lower for respondents with low to moderate levels of education compared with those with higher levels (64% and 62%, respectively, ANOVA interaction effect, F(1,2048) = 4.10, p = .043, n_p² < 0.01). An additional analysis that split the low to moderately educated people according to their income median confirmed, across both age groups, that higher education, not income, was associated with recall of information from simple fact boxes (Fig 4A and 4B).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

Proportion of correct responses to five knowledge items according to different levels of education and household net income for respondents younger than 60 years of age (A) and 60 years of age and older (B). The sample is weighted. Error bars show the standard error of the mean.

https://doi.org/10.1371/journal.pone.0274186.g004

For complex fact boxes, the proportion of quick responders (below median response time of the control condition) among those with the lowest educational attainment was lower (7%; compared to those with higher education: 15%), which indicates that they needed more time for reading and/or for deciding to skip reading. In contrast, for simple fact boxes, the proportion of quick responders did not appear to depend on the level of education (in both groups about 11–12%).

General discussion

The association of vaccination knowledge and uptake implies one crucial mechanism with respect to the goal of an adequately immunized society. Besides showing a differential increase in vaccination knowledge in proponents, undecideds, and opponents around the start of COVID-19 vaccination in Germany, Study 1 showed increasing intentions to have the (mRNA) vaccinations in the early phase of vaccine distribution [2]. Study 2 found that both undecideds and skeptics lacked various pieces of information for making a decision, paired with false beliefs and a lack of trust in vaccine safety and efficacy. Study 3 revealed that a COVID-19 vaccination fact box improved disease risk perception without increasing COVID-19 fears (unlike in the control condition). Based on these findings, a previously implemented vaccination fact box was amended. Applied at the population level in Study 4, fact boxes boosted knowledge of undecideds and skeptics together with a more positive evaluation of the vaccination’s benefit-harm ratio. Studying simple or complex fact boxes instead of nothing was 1.3 times more likely to lead to any positive change in evaluation. In the case of complex fact boxes, however, any changes seem to be balanced.

Although a common factor (e.g. peer behaviors) admittedly may underlie information acquisition and intentions–while stable vaccination preferences may prompt differential information acquisition–knowledge about benefits and harms of COVID-19 vaccinations, which can reduce vaccine hesitancy [41], may lead to informed intentions for readers of fact boxes if they base their intentions on these facts. This mechanism, in line with the standards of health care systems in many democratic countries, also contributes to the trustworthiness of authorities engaged in vaccination risk communication. For example, in face of the fact that denying information gaps can undermine perceived credibility [42, 43], fact boxes contain epistemic uncertainty disclaimers (e.g. verbal statements or ranges as a result of incomplete knowledge about the risk numbers) [18]. Transparently informing the public about limitations of vaccinations, which considers individual preferences, does not necessarily reduce vaccination intentions [44], which can support the societal preference to increase the vaccinating share of the population. By contrast, persuasive communications, e.g. messages framed in relative risks, can increase vaccination intentions [45] but are both misinformative and incomprehensible [46], colliding with the rights and needs of undecided and skeptical citizens. In addition, incomprehensible information may provoke a backlash effect. Future research could investigate the causal link between the grade of comprehensibility of information about benefits and harms and people’s vaccination intentions.

Limitations of our study: In our population-wide sample, simple fact boxes appear to be more beneficial for those with higher education. This contradiction to the design intention could be due to a superficial reading of the simple box by participants with lower levels of education, as indicated by their brief reading times. Although information needs and requirements were elicited, the target group with lower educational attainment did not actively participate in the development process of the fact box intervention (e.g., Study 3 included a highly educated convenience sample, separate piloting was missing). However, the potential of shared decision-making interventions to reduce health inequities [47] is tied to the involvement of disadvantaged groups in their development [48]. If neglected, there is a risk of increasing health inequity [49], e.g. due to the heterogeneity in the audience’s ability to comprehend given information. Those who comprehend the least, potentially benefit the least [45], e.g. low-numerate breast cancer patients [50]. Also, cancer screening fact boxes let high-numerate people update their benefit perceptions more strongly than low-numerates [22]. This could indicate distorted interpreting of fact boxes. Thus, more research needs to examine under which conditions fact boxes can strengthen informed decision-making of disadvantaged groups and reduce health inequities. Furthermore, concepts such as critical health literacy or digital health literacy should be taken into account to examine determinants that influence who benefits or does not benefit from fact boxes and how this affects health inequities [51]. Additional factors associated with formal education can also lead to inequality (e.g. working and living conditions). Future studies should thus incorporate approaches such as the PROGRESS Plus framework, which describes inequity-generating factors at multiple levels.

An additional limitation of our work concerns the set of vaccination knowledge items that covered certain requirements of health information guidelines (benefit, harms, uncertainty) but were not a validated scale of vaccination knowledge.

The results are hardly transferable to the non-democratic contexts, because, alongside the differences in media, political handling of the pandemic, and access to vaccination, the needs and preferences for the content and design of health information depend on a population’s risk perception, numerical literacy, and the opportunity for informed decision-making. This suggests that the success of a complex public health intervention, e.g. the provision of fact boxes by a central health authority, depends on a number of factors related to the societal characteristics of a country or region.

We conclude that in Germany, the implementation of fact boxes [52] supports evidence-based communication and, thus, empowerment at the population level. Dissemination of the mRNA fact boxes via the RKI media channels included their webpage with about 130 million visits in 2020, an RKI Twitter account, and the cellphone app of the Permanent Vaccination Commission in Germany. Our figures imply that presenting the about 11 million undecided and skeptical adults under 60 years of age in Germany with a simple fact box for about 90 seconds would familiarize more than 600,000 people with vaccine efficacy. A majority of them would evaluate vaccinations more positively. Also, by abstaining from persuasion, reluctance and distrust concerning the sender can be prevented or alleviated. Loss of trustworthiness is a relevant risk of alternative interventions that are nontransparent (e.g., nudges), persuasive (e.g., advertisement), or that exert pressure (e.g., for authorized access to public activities).

Finally, informed decision-making is not a threat to the goal of an adequately immunized population. To achieve this, ensuring the right of each individual to decide about their own health might be sufficient, while other types of social contracts [53] could be less important. The standard of evidence-based health care, legally binding in Germany [12] and benchmark of a democratic society, would ensure responsible vaccination decisions and future commitment when refreshments of the vaccination might be necessary or when individuals might have to decide to get vaccinated against other diseases.

Supporting information

Acknowledgments

We thank our proofreader Rona Unrau for her critical eye and Carolin Harder for her coding of open-ended item responses. Furthermore, we thank Gudrun Keller for programming the COMPASS items.