- Email: [email protected]
On pins and needles: How vaccines are portrayed on Pinterest
Vaccine 33 (2015) 5051–5056
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
On pins and needles: How vaccines are portrayed on Pinterest Jeanine P.D. Guidry a,∗ , Kellie Carlyle a , Marcus Messner b , Yan Jin c a
Department of Social and Behavioral Health, Virginia Commonwealth University, United States Richard T. Robertson School of Media and Culture, Virginia Commonwealth University, United States c Grady College of Journalism and Mass Communication, University of Georgia, United States b
a r t i c l e
i n f o
Article history: Received 15 May 2015 Received in revised form 31 July 2015 Accepted 18 August 2015 Available online 28 August 2015 Keywords: Vaccinations Anti-vaccine Internet Social media Content analysis
a b s t r a c t Vaccination is an effective public health tool for reducing morbidity and mortality caused by infectious diseases. However, increasing numbers of parents question the safety of vaccines or refuse to vaccinate their children outright. The Internet is playing a significant role in the growing voice of the anti-vaccination movement as a growing number of people use the Internet to obtain health information, including information about vaccines. Given the role the Internet plays in providing vaccination-related communication, coupled with limited research in this area, this study focused on the social media platform Pinterest, analyzing 800 vaccine-related pins through a quantitative content analysis. The majority of the pins were anti-vaccine, and most were original posts as opposed to repins. Concerns about vaccine safety and side effects were oft-repeated themes, as was the concept of conspiracy theory. Pro-vaccine pins elicited consistently more engagement than anti-vaccine pins. Health educators and public health organizations should be aware of these dynamics, since a successful health communication campaign should start with an understanding of what and how publics communicate about the topic at hand. © 2015 Elsevier Ltd. All rights reserved.
1. Introduction Public voicing of anti-vaccination sentiment has been increasing [1]. Common themes are that vaccines are ineffective, useless, or dangerous [1]; children receive too many vaccines at once [2]; and mandatory vaccination infringes on civil liberties and parental rights [3,4]. Additionally, many parents believe that vaccinepreventable diseases no longer present a serious health risk because they lack personal experience with these diseases [5,6]. Given the significant public health implications of vaccine rejection (e.g., [7]), it is important to understand the content of vaccine-related information the public is using to make these decisions. Many people search for health information on the Internet and, although Internet searches lead to public health versus anti-vaccination websites with about equal likelihood [8,9], antivaccination messages are more commonly found on the Internet than in print or broadcast media [1]. Visiting anti-vaccination websites for 5–10 min can increase perceptions of vaccination risks and decrease intentions to vaccinate [23]. Parents who do not vaccinate their children are more likely to have searched for vaccination information online and from more sources in general than parents who fully vaccinate their children [8,10,11].
Anti-vaccination websites are likely to use narrative appeals, including personal stories accompanied by photos, which can increased risk perceptions and decrease vaccination intentions [12]. Across the few studies of vaccination representation on social media to date, results have been mixed. Briones and co-workers [4] found that most HPV-vaccine focused videos on YouTube were negative in tone and ‘liked’ more often by viewers than positive or neutral videos. An earlier study focused on general vaccinationthemed videos on YouTube found a higher percentage of positive videos (48% positive, 32% negative, and 20% ambiguous), although the negative videos in their study received more ratings and ‘likes’, and the content often included unsubstantiated medical claims [13]. Another study about the HPV vaccine among MySpace blog posts found a similar ratio of positive versus negative (52% positive, 43% negative, and 6% ambiguous). Finally, the sole Twitter study available indicated that the majority of tweets about vaccination promoted substantiated medical information [7]. These studies address more established social media platforms, whereas Pinterest is a newer visual social media platform that has not received much academic attention. 1.1. Research questions and hypotheses
∗ Corresponding author. Tel.: +1 804 683 4512. E-mail address: [email protected] (J.P.D. Guidry). http://dx.doi.org/10.1016/j.vaccine.2015.08.064 0264-410X/© 2015 Elsevier Ltd. All rights reserved.
There is a paucity of available research on the portrayal of vaccines and vaccinations on social media channels in general, and no
5052
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
studies to date have been published on Pinterest and vaccination representations. Visually focused social media platforms are widely seen as an area of continued growth, and users spend significantly more time per visit on Pinterest (14.2 min) than on most other social media platforms. With a third of online women in the US using Pinterest as of 2013 [14], and a majority of Americans searching for health information online, it is important to understand what vaccination information is available on such a popular social media platform. This study, therefore, addresses the following research questions: RQ1.
How are vaccinations portrayed on Pinterest?
RQ2. How do Pinterest users engage with vaccination content on the platform? The design of social and behavioral health interventions (such as a campaign to increase vaccinations) can best be done with a clear understanding of theories of behavior change [15, pp. 25–26]. The Health Belief Model (HBM), developed in the 1950s as a framework to explain health behaviors, is one of the most common theories used to study health-related behavior and has been used to predict several types of vaccine uptakes [15,16, pp. 45–46]. The HBM’s main constructs, as applied to vaccinations, are perceived susceptibility (what is the likelihood of contracting the disease), perceived severity (how serious is the disease, if contracted), perceived benefits (how effective is the vaccine in protecting against the disease), perceived barriers (what are the perceived costs of receiving the vaccine, such as vaccine safety and potential or feared side effects), and self-efficacy (belief that one can successfully take the steps to get vaccinated or get their kids vaccinated) [16,17]. Gerend and Shepherd [16], for example, found that perceived susceptibility and perceived barriers were significant predictors of HPV vaccine uptake among young adult women. Since no data is available on how HBM constructs are expressed in vaccine-focused Pinterest pins, research is needed to determine what the prevalent expressed constructs are, providing a baseline for communications and medical professionals who endeavor to address the issue of vaccinations on Pinterest. The third research question, therefore, is: RQ3. How are HBM constructs represented in vaccination-focused Pinterest pins? Kata [3] described two prominent content themes in the online anti-vaccination environment: conspiracy theories and the concept of violation of civil liberties. The majority of anti-vaccination websites frame the fight against vaccines as a struggle against government and pharmaceutical cover-ups as well as an exclusive pursuit of profits [18]. In the abovementioned study of HPVvaccine-focused YouTube videos by Briones et al. [17], researchers found that 20% of the videos addressed the concept of civil liberties, while a small percentage mentioned conspiracy theories. More broadly, there is a growing part of the population that mistrusts both government, the pharmaceutical industry, and public health officials with regard to vaccines and their safety [9]. In order to find out if either content theme is prevalent on Pinterest, the fourth research question is: RQ4. To what extent do vaccination-related Pinterest pins mention issues related to conspiracy theories and civil liberties? The available literature highlights the use of narratives in antivaccination materials [12]. Pinterest, with its visual focus, lends itself to the use of narratives. Since narratives are perceived as a potential strategy to communicate pro-vaccination messages, it is important to discover if anti-vaccine pins use narratives more frequently than statistical information on this social media channel, as well [8]. The hypothesis, therefore, is:
H1. Anti-vaccine messages on Pinterest will use narrative vaccination information more frequently, whereas pro-vaccine messages will use statistical representations more frequently. 2. Methodology This study analyzed information relating to vaccinations on Pinterest. Drawing upon existing literature, we used four keywords to select the Pinterest posts (hereafter identified as “pins”) for the sample: “vaccination,” “vaccine,” “vaccines,” and “vaccinate” [7,13,17,19]. Pins were sampled by choosing every fifth applicable pin for analysis. On March 19, 20, and 21 of 2014, each fifth pin for each keyword search was selected by scrolling down the page with search results. Each pin selected was captured with a screenshot and any links to connected websites noted, reaching a total of 200 pins per keyword for a total of 800 pins in the sample. Coding protocols for pins were developed, tested and implemented for the coding process. First, the pins were coded for a series of typical Pinterest characteristics, including timeframe of the post, poster identity (individual, web-based health organization, traditional (offline) health organization, other organization, and other), whether the pin originally was a repin, the number of repins, number of likes, number of comments, did the pin link to another website and if so, what type of website (traditional media website, social media site, blog, government agency website, official medical website like hospital or medical organization, other health-focused website, and other site). In addition to these characteristics, the pins were coded for a number of content-related variables (see Table 1 for a complete list). Two coders were trained to establish intercoder reliability. The first coder coded all of the posts (n = 800), while the second coder coded 10% of the posts (n = 80) for the study variables. After pre-testing and subsequent changes to the coding protocol, the intercoder reliability test with the ReCal statistical program showed Scott’s Pi [20] was on average .84.1 The individual coefficients were all considered to be reliable [24,25]. 3. Results The data collected through content analyses of 800 Pinterest pins was imported into and analyzed using Statistical Package for the Social Sciences version 22. Research question one asked how vaccinations are portrayed on Pinterest. The overarching stance of the pins was decidedly anti-vaccine: 74.0% (n = 592) fell in this category; 18% (n = 144) represented a pro-vaccine stance; 6.9% (n = 55) were neutral in tone; 1.1% (n = 9) undetermined (see Table 2 for complete results, and see Figs. 1–3 for example pins). A majority of the pins, 81.5% (n = 652), connected through the pin with a website external to Pinterest, 13.6% (n = 109) did not, and 4.9% (n = 39) pointed to a broken link. Of the 652 pins that actually pointed to another website, only 3.7% (n = 24) pointed to an official medical website (e.g., representing a hospital or a major medical organization). See Table 3 for a complete list of website types. Research question two asked how Pinterest users engage with vaccination content on the platform. Pinterest provides three ways for people to interact with pins: “repinning” (similar to Twitter’s “retweet” function), “liking”, and posting comments. Repinning was the most popular form of interaction: 69.6% (n = 557) of the
1 A complete list of the coefficients: .73 (image type), .89 (image stance), .77 (caption stance), .98 (link stance), .73 (juxtaposition image and caption), .82 (overall pin stance), .83 (story type), .98 (vaccine type), .85 (conspiracy theory), .87 (civil liberties), .83 (perceived benefits), .73 (perceived barriers), .84 (perceived susceptibility), .85 (perceived severity), and .85 (self-efficacy).
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056 Table 1 Pinterest content variables.
5053
Table 3 Website type.
Variable
Description
Website type
Percentage
Image type
• Primarily image • Primarily text • Mix of image and text • Infographic: graphic visual representations of information, data or knowledge intended to present complex information quickly and clearly
Blog Other social media platform (besides Pinterest) Government website Official medical website (hospital, clinic, major medical organization) Commercial website Other type of website
19.0% (n = 124) 10.1% (n = 66)
Image, caption, link stance
• Pro-vaccine (mentions of vaccines were positive in tone and/or expressed approval of vaccines) • Anti-vaccine (mentions of vaccines were negative in tone and/or expressed disapproval of vaccines) • Neutral
Story type
• Narrative: testimonials or anecdotes • Statistical: incidence rates related to vaccinations and vaccines
Conspiracy theory
Accusing a group or organization (e.g., the government, pharmaceutical companies, or doctors) of having caused or covered up vaccine adverse reactions or events: • Pharmaceutical companies and doctors benefit from illnesses caused by vaccines • Vaccination policies are motivated by profit • Vaccine information is withheld from the public • Doctors are ignorant or fearful of sanctions • Dangers of diseases are exaggerated to frighten parents
Civil liberties related
Dealing with a real or perceived violation of civil liberties: • Violating parental rights • Monitoring parents for adherence to vaccination schedules • Vaccine mandates perceived as excessive government control
Health Belief Model constructs
• Perceived benefits of the vaccine/vaccination • Perceived barriers to the vaccine/vaccination • Perceived susceptibility to contracting the disease the vaccine is supposed to protect from • Perceived severity of the disease the vaccine is supposed to protect from • Perceived self-efficacy
Type of vaccine
• DTP (diphtheria, tetanus, pertussis) • MMR (measles, mumps, rubella) • HPV (human papilloma virus) • Flu • Hepatitis B • Other • General vaccine
pins were repinned at least once. In contrast, 42.3% (n = 338) of the pins received at least one “like” and only 6.4% (n = 51) attracted at least one comment. The mean number of repins was 7.41, of likes 1.39, and of comments .24. The highest number of repins for one pin was 250, and the highest number of likes was 34. Research question three asked how the HBM constructs were represented in pins. The five primary variables of the Health Belief Model appear in varying degrees in the study sample (see Table 4 for complete results).
0.3% (n = 2) 3.7% (n = 24) 0.2% (n = 1) 66.7% (n = 435)
Table 4 Health Belief Model (HBM) constructs. Health Belief Model constructs
Percentage
Perceived benefits of vaccination
• High effectiveness: 16.5% (n = 132) • Low/no effectiveness: 20.3% (n = 162) • No information: 63.2% (n = 506)
Perceived barriers to vaccination
• High barriers: 59.8% (n = 478) • Low/no barriers: 3.8% (n = 30) • No information: 36.5% (n = 292)
Perceived susceptibility to the disease vaccine is supposed to protect from
• High: 8.9% (n = 71) • Low: 5.0% (n = 40) • No information: 86.1% (n = 689)
Perceived severity of the disease vaccine is supposed to protect from
• Severe: 4.6% (n = 37) • Not severe: 2.1% (n = 17) • No information: 93.3% (n = 746)
Perceived self-efficacy (ability to get a vaccine or get a vaccine for one’s children)
• Present: 0.4% (n = 3) • Absent: 99.6% (n = 797)
Research question four asked to what extent pins mention issues related to conspiracy theories and civil liberties. Conspiracy theory and concern for civil liberties were consistently two of the more popular themes. In this sample, 21.5% (n = 172) of pins mentioned the concept of a conspiracy on either the government’s or the pharmaceutical companies’ side, while only 4.8% (n = 38) did the same with the concept of civil liberties. Focusing only on the 592 pins with an anti-vaccine stance, 27.7% (n = 164) mentioned a conspiracy theory, while only 6.3% (n = 37) of the 592 pins addressed vaccines as a threat to civil liberties. Hypothesis one posited that anti-vaccine messages on Pinterest would use narrative vaccination information more frequently, whereas pro-vaccine messages would be more likely to use statistical representations. Of all 800 pins, 10.6% (n = 85) used narrative information, while 6.6% (n = 53) used primarily statistical information to communicate; 82.8% (n = 662) did not use either. Pro-vaccine pins use more statistical information (21.5%, N = 31) than narrative information (7.6%, N = 11). Anti-vaccine pins use more narrative information (11.3%, N = 67) than statistical information (3.0%, N = 18). Hypothesis one is supported. Specifically, there was a significant difference in the types of information used across anti- and pro-vaccine messages such that anti-vaccine pins used more narrative and pro-vaccine pins used more statistical representations (chi-square = 32.86(1), p < .001).
Table 2 Vaccine-related stance on Pinterest. Total Pro-vaccine Anti-vaccine Neutral Inconsistent Total
18.0% (n = 144) 74.0% (n = 592) 6.9% (n = 55) 1.1% (n = 9) 100% (n = 800)
Image
Caption
Link
11.6% (n = 93) 46.8% (n = 374) 41.6% (n = 333) N/A
17.6% (n = 141) 71.3% (n = 570) 10.4% (n = 83) N/A
19.2% (n = 125) 76.8% (n = 501) 4.0% (n = 26) N/A
100% (n = 800)
100% (n = 800)
100% (n = 652)
5054
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
Fig. 1. Example pin: Gardasil warning.
Fig. 2. Example pin: vaccine injury story.
4. Discussion This study is among the first to analyze the portrayal of vaccinations on the social media platform Pinterest. A quantitative content analysis of 800 pins showed that the majority of pins portray vaccinations in a negative light. Importantly, vaccination portrayals on YouTube were 74.7% pro-vaccine in Ache et al.’s [19] HPV vaccine study, 32% pro-vaccine in Briones et al.’s [17] HPV vaccine study, and now only 24.0% pro-vaccine in this general vaccine study. Although these studies vary in their methodologies and cannot be directly compared, this—together with the increase in public voicing of anti-vaccination sentiment—is a concerning trend in public sentiment that may be impacting the level of uptake of vaccines crucial to the continued prevention of morbidity and mortality due to infectious, vaccine preventable diseases. Considering the potential harmful health consequences for those who are not vaccinated, it is concerning that very few of the linked external websites refer to a government website like the Centers for Disease Control and Prevention or a medical website. Seemingly, organizations with the information and resources
to provide a counter-argument to the strong anti-vaccine presence on Pinterest do not participate in discussions on the platform as intensely as other content creators. Public health and medical organizations should increase their engagement with the public via social media to communicate the benefits and safety of vaccines [21]. While most pins in this study did not use narrative or statistical information, for the ones that did, pro-vaccine pins still featured more statistical information while anti-vaccine messages featured more narrative information. Several studies have noted that narrative information referring to adverse vaccination events will decrease vaccination intentions, as well as narratives having an overall stronger influence than statistical information [12]. As such, health educators should consider using more narrative information about the protective effects of vaccines. With respect to the HBM, perceived barriers appeared in more than half of the pins (59.7%), indicating a perception of great concern about vaccine safety and side effects. In contrast, there is an overwhelming lack of empirical evidence that vaccines are associated with serious health risks. The likelihood of vaccination uptake
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
5055
anti-vaccination movement, they do provide a foundation from which further interventions can be planned. 4.1. Limitations and future considerations There are a number of limitations and future considerations to this study. First of all, Pinterest does not list its pins chronologically and does not list an exact time stamp for each pin. This makes using a more conventional content analysis sampling method like a constructed two week time period virtually impossible. The lack of previous Pinterest studies means few examples are available and that there is lack of consensus in the field regarding best sampling practices for this medium. Furthermore, while this study focused on Pinterest, future studies in this field should consider Facebook and Instagram to study the same topic and build a more robust field of literature regarding the portrayal of vaccines and vaccinations on social media platforms. Finally, this study used the theoretical framework of the HBM to examine some of the psychosocial variables present in the portrayal of vaccines and vaccinations on Pinterest. The HBM is one of several models that has been applied to vaccination studies, and future studies should consider approaching this study concept from other social and behavioral health theories like the Theory of Reasoned Action and the Extended Parallel Processing Model [15]. We used the HBM because it is the most widely used model in studies on preventative behaviors such as vaccines. However, the content analysis showed the presence of fear messages, and the Extended Parallel Processing Model (EPPM) may be helpful to address that particular issues. The Theory of Reasoned Action (TRA) is particularly appropriate for audiences where social norms are very strong. Conflict of interest The authors have no conflict of interest to disclose. References
Fig. 3. Example pin: vaccines and autism.
decreases, however, as perceived barriers increase [22], making addressing vaccine safety and side effects concerns is of great importance. Finally, though civil liberties were not often addressed among the pins in this study, 21.6% of the pins did refer to the concept of conspiracy theory, including the belief that the government and the pharmaceutical industry are hiding “the truth about vaccines” or that their efforts in promoting vaccines are motivated exclusively by the pursuit of profit. These content themes have the potential to influence underlying perceptions of vaccination programs and should not be discounted. The success of vaccines in eradicating some, and greatly reducing morbidity and mortality in other, infectious diseases continues to be overshadowed by the growing anti-vaccine movement. Successful health communication campaigns should start with an understanding of the target audience’s current perceptions of the health issue at hand, including those communicated on more recently developed social media platforms like Pinterest. This quantitative content analysis of 800 vaccination-focused pins concluded that a significant majority of pins are anti-vaccine in stance and that potentially powerful pro-vaccine voices are not as present on Pinterest as their anti-vaccine counterparts. Perceived barriers, operationalized as vaccine side effects and safety, are of great concern on Pinterest. Health educators and public health campaigns should consider using more narrative information to tell the stories of lives saved through vaccines. While an increased social media presence, additional education, and a change in communication style by themselves are not likely to stem the tide of the
[1] Kata A. Anti-vaccine activists, Web 2.0, and the postmodern paradigm—an overview of tactics and tropes used online by the anti-vaccination movement. Vaccine 2012;30(25):3778–89, http://dx.doi.org/10.1016/j.vaccine.2011.11.112. [2] Paulussen TGW, Hoekstra F, Lanting CI, Buijs GB, Hirasing RA. Determinants of Dutch parents’ decisions to vaccinate their child. Vaccine 2006;24:644–51, http://dx.doi.org/10.1016/j.vaccine.2005.08.053. [3] Kata A. A postmodern Pandora’s box: anti-vaccination mison the Internet. Vaccine 2010;28:1709–16, information http://dx.doi.org/10.1016/j.vaccine.2009.12.022. [4] Madden K, Nan X, Briones RL, Waks L. Sorting through search results: a content analysis of HPV vaccine information online. Vaccine 2012;30:3741–6, http://dx.doi.org/10.1016/j.vaccine.2011.10.025. [5] Kitta A. Vaccinations and public concern in history: legend, rumor, and risk perception. New York, NY: Routledge; 2012. [6] Sugerman DE, Barskey AE, Delea MG, Ortega-Sanchez IR, Bi D, Ralston KJ, et al. Measles outbreak in a highly vaccinated population, San 351 Diego, 2008: role of the intentionally undervaccinated. Pediatrics 2010;125:747–55, http://dx.doi.org/10.1542/peds.2009-1653. [7] Love B, Himelboim I, Holton A, Stewart K. Twitter as a source of vaccination information: content drivers and what they are saying. Am J Infect Control 2013;41:568–70, http://dx.doi.org/10.1016/j.ajic.2012.10.016. [8] Betsch C, Brewer NT, Brocard P, Davies P, Gaissmaier W, Haase N, et al. Opportunities and challenges of Web 2.0 for vaccination decisions. Vaccine 2012;30:3727–33, http://dx.doi.org/10.1016/j.vaccine.2012.02.025. [9] Poland GA, Jacobson RM, Ovsyannikova IG. Trends affecting the future of vaccine development and delivery: the role of demographics, regulatory science, the anti-vaccine movement, and vaccinomics. Vaccine 2009;27:3240–4, http://dx.doi.org/10.1016/j.idc.2009.01.008. [10] Harmsen IA, Doorman GG, Mollema L, Ruiter RAC, Kok G, de Melker HE. Parental information-seeking behavior in childhood vaccinations. BMC Public Health 2013;13(1219). [11] Jones AM, Omer SB, Bednarczyk RA, Halsey NA, Moulton LH, Salmon DA. Parents’ source of vaccine information and impact on vaccine attitudes. Beliefs, and nonmedical exemptions. Adv Prev Med 2012;2012. [12] Betsch C, Ulshofer C, Renkewitz F, Betsch T. The influence of narrative v. statistical information on perceiving vaccination risks. Med Decis Mak 2011;31:742–53, http://dx.doi.org/10.1177/0272989X11400419.
5056
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
[13] Keelan J, Pavri-Garcia V, Tomlinson G, Wilson K. YouTube as a source of information on immunization: a content analysis. JAMA 2007;298:2482. [14] Smith C. By the numbers: 59 amazing Pinterest stats; 2014. Available from: http://expandedramblings.com/index.php/pinterest-stats/-.UzJHwdzEt8w [accessed 23.03.14]. [15] Glanz K, Rimer BK, Lewis ML. Health behavior and health education: theory, research, and practice. San Francisco, CA: Jossey-Bass; 2008. [16] Gerend M, Shepherd J. Predicting human papillomavirus vaccine uptake in young adult women: comparing the health belief model and theory of planned behavior. Ann Behav Med 2012;44:171–80. [17] Briones RL, Nan X, Madden K, Waks L. When vaccines go viral: an analysis of HPV vaccine coverage on YouTube. Health Commun 2012;27:478–85, http://dx.doi.org/10.1080/10410236.2011.610258. [18] Davies P, Chapman S, Leask J. Antivaccination activists on the world wide web. Arch Dis Child 2002;87:22–5. [19] Ache KA, Wallace LS. Human papillomavirus vaccination coverage on YouTube. Am J Prev Med 2008;35:389–92, http://dx.doi.org/10.1016/j.amepre.2008.06.029.
[20] Scott WA. Reliability of content analysis: the case of nominal scale coding. Public Opin Q 1955;19:321–5. [21] Poland GA, Jacobson RM. Understanding those who do not understand: a brief review of the anti-vaccine movement. Vaccine 2001;19:2440–5, http://dx.doi.org/10.1016/S0264-410X(00)00469-2. [22] Looijmans-van den Akker I, van Delden JJM, Verheij TJM, van Essen GA, van der Sande MAB, Hulscher ME, et al. Which determinants should be targeted to increase influenza vaccination uptake among health care workers in nursing homes? Vaccine 2009;27:4724–30, http://dx.doi.org/10.1016/j.vaccine.2009.05.013. [23] Betsch C, Renkewitz F, Betsch T, Ulshofer C. The influence of vaccine-critical websites on perceiving vaccination risks. J Health Psychol 2010;15:446, http://dx.doi.org/10.1177/1359105309353647. [24] Riffe D, Lacy S, Fico FG. Analyzing media messages: using quantitative content analysis in research. Mahwah, NJ: Lawrence Erlbaum Associates; 1998. [25] Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum Commun Res 2002;28(4):587–604.
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
On pins and needles: How vaccines are portrayed on Pinterest Jeanine P.D. Guidry a,∗ , Kellie Carlyle a , Marcus Messner b , Yan Jin c a
Department of Social and Behavioral Health, Virginia Commonwealth University, United States Richard T. Robertson School of Media and Culture, Virginia Commonwealth University, United States c Grady College of Journalism and Mass Communication, University of Georgia, United States b
a r t i c l e
i n f o
Article history: Received 15 May 2015 Received in revised form 31 July 2015 Accepted 18 August 2015 Available online 28 August 2015 Keywords: Vaccinations Anti-vaccine Internet Social media Content analysis
a b s t r a c t Vaccination is an effective public health tool for reducing morbidity and mortality caused by infectious diseases. However, increasing numbers of parents question the safety of vaccines or refuse to vaccinate their children outright. The Internet is playing a significant role in the growing voice of the anti-vaccination movement as a growing number of people use the Internet to obtain health information, including information about vaccines. Given the role the Internet plays in providing vaccination-related communication, coupled with limited research in this area, this study focused on the social media platform Pinterest, analyzing 800 vaccine-related pins through a quantitative content analysis. The majority of the pins were anti-vaccine, and most were original posts as opposed to repins. Concerns about vaccine safety and side effects were oft-repeated themes, as was the concept of conspiracy theory. Pro-vaccine pins elicited consistently more engagement than anti-vaccine pins. Health educators and public health organizations should be aware of these dynamics, since a successful health communication campaign should start with an understanding of what and how publics communicate about the topic at hand. © 2015 Elsevier Ltd. All rights reserved.
1. Introduction Public voicing of anti-vaccination sentiment has been increasing [1]. Common themes are that vaccines are ineffective, useless, or dangerous [1]; children receive too many vaccines at once [2]; and mandatory vaccination infringes on civil liberties and parental rights [3,4]. Additionally, many parents believe that vaccinepreventable diseases no longer present a serious health risk because they lack personal experience with these diseases [5,6]. Given the significant public health implications of vaccine rejection (e.g., [7]), it is important to understand the content of vaccine-related information the public is using to make these decisions. Many people search for health information on the Internet and, although Internet searches lead to public health versus anti-vaccination websites with about equal likelihood [8,9], antivaccination messages are more commonly found on the Internet than in print or broadcast media [1]. Visiting anti-vaccination websites for 5–10 min can increase perceptions of vaccination risks and decrease intentions to vaccinate [23]. Parents who do not vaccinate their children are more likely to have searched for vaccination information online and from more sources in general than parents who fully vaccinate their children [8,10,11].
Anti-vaccination websites are likely to use narrative appeals, including personal stories accompanied by photos, which can increased risk perceptions and decrease vaccination intentions [12]. Across the few studies of vaccination representation on social media to date, results have been mixed. Briones and co-workers [4] found that most HPV-vaccine focused videos on YouTube were negative in tone and ‘liked’ more often by viewers than positive or neutral videos. An earlier study focused on general vaccinationthemed videos on YouTube found a higher percentage of positive videos (48% positive, 32% negative, and 20% ambiguous), although the negative videos in their study received more ratings and ‘likes’, and the content often included unsubstantiated medical claims [13]. Another study about the HPV vaccine among MySpace blog posts found a similar ratio of positive versus negative (52% positive, 43% negative, and 6% ambiguous). Finally, the sole Twitter study available indicated that the majority of tweets about vaccination promoted substantiated medical information [7]. These studies address more established social media platforms, whereas Pinterest is a newer visual social media platform that has not received much academic attention. 1.1. Research questions and hypotheses
∗ Corresponding author. Tel.: +1 804 683 4512. E-mail address: [email protected] (J.P.D. Guidry). http://dx.doi.org/10.1016/j.vaccine.2015.08.064 0264-410X/© 2015 Elsevier Ltd. All rights reserved.
There is a paucity of available research on the portrayal of vaccines and vaccinations on social media channels in general, and no
5052
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
studies to date have been published on Pinterest and vaccination representations. Visually focused social media platforms are widely seen as an area of continued growth, and users spend significantly more time per visit on Pinterest (14.2 min) than on most other social media platforms. With a third of online women in the US using Pinterest as of 2013 [14], and a majority of Americans searching for health information online, it is important to understand what vaccination information is available on such a popular social media platform. This study, therefore, addresses the following research questions: RQ1.
How are vaccinations portrayed on Pinterest?
RQ2. How do Pinterest users engage with vaccination content on the platform? The design of social and behavioral health interventions (such as a campaign to increase vaccinations) can best be done with a clear understanding of theories of behavior change [15, pp. 25–26]. The Health Belief Model (HBM), developed in the 1950s as a framework to explain health behaviors, is one of the most common theories used to study health-related behavior and has been used to predict several types of vaccine uptakes [15,16, pp. 45–46]. The HBM’s main constructs, as applied to vaccinations, are perceived susceptibility (what is the likelihood of contracting the disease), perceived severity (how serious is the disease, if contracted), perceived benefits (how effective is the vaccine in protecting against the disease), perceived barriers (what are the perceived costs of receiving the vaccine, such as vaccine safety and potential or feared side effects), and self-efficacy (belief that one can successfully take the steps to get vaccinated or get their kids vaccinated) [16,17]. Gerend and Shepherd [16], for example, found that perceived susceptibility and perceived barriers were significant predictors of HPV vaccine uptake among young adult women. Since no data is available on how HBM constructs are expressed in vaccine-focused Pinterest pins, research is needed to determine what the prevalent expressed constructs are, providing a baseline for communications and medical professionals who endeavor to address the issue of vaccinations on Pinterest. The third research question, therefore, is: RQ3. How are HBM constructs represented in vaccination-focused Pinterest pins? Kata [3] described two prominent content themes in the online anti-vaccination environment: conspiracy theories and the concept of violation of civil liberties. The majority of anti-vaccination websites frame the fight against vaccines as a struggle against government and pharmaceutical cover-ups as well as an exclusive pursuit of profits [18]. In the abovementioned study of HPVvaccine-focused YouTube videos by Briones et al. [17], researchers found that 20% of the videos addressed the concept of civil liberties, while a small percentage mentioned conspiracy theories. More broadly, there is a growing part of the population that mistrusts both government, the pharmaceutical industry, and public health officials with regard to vaccines and their safety [9]. In order to find out if either content theme is prevalent on Pinterest, the fourth research question is: RQ4. To what extent do vaccination-related Pinterest pins mention issues related to conspiracy theories and civil liberties? The available literature highlights the use of narratives in antivaccination materials [12]. Pinterest, with its visual focus, lends itself to the use of narratives. Since narratives are perceived as a potential strategy to communicate pro-vaccination messages, it is important to discover if anti-vaccine pins use narratives more frequently than statistical information on this social media channel, as well [8]. The hypothesis, therefore, is:
H1. Anti-vaccine messages on Pinterest will use narrative vaccination information more frequently, whereas pro-vaccine messages will use statistical representations more frequently. 2. Methodology This study analyzed information relating to vaccinations on Pinterest. Drawing upon existing literature, we used four keywords to select the Pinterest posts (hereafter identified as “pins”) for the sample: “vaccination,” “vaccine,” “vaccines,” and “vaccinate” [7,13,17,19]. Pins were sampled by choosing every fifth applicable pin for analysis. On March 19, 20, and 21 of 2014, each fifth pin for each keyword search was selected by scrolling down the page with search results. Each pin selected was captured with a screenshot and any links to connected websites noted, reaching a total of 200 pins per keyword for a total of 800 pins in the sample. Coding protocols for pins were developed, tested and implemented for the coding process. First, the pins were coded for a series of typical Pinterest characteristics, including timeframe of the post, poster identity (individual, web-based health organization, traditional (offline) health organization, other organization, and other), whether the pin originally was a repin, the number of repins, number of likes, number of comments, did the pin link to another website and if so, what type of website (traditional media website, social media site, blog, government agency website, official medical website like hospital or medical organization, other health-focused website, and other site). In addition to these characteristics, the pins were coded for a number of content-related variables (see Table 1 for a complete list). Two coders were trained to establish intercoder reliability. The first coder coded all of the posts (n = 800), while the second coder coded 10% of the posts (n = 80) for the study variables. After pre-testing and subsequent changes to the coding protocol, the intercoder reliability test with the ReCal statistical program showed Scott’s Pi [20] was on average .84.1 The individual coefficients were all considered to be reliable [24,25]. 3. Results The data collected through content analyses of 800 Pinterest pins was imported into and analyzed using Statistical Package for the Social Sciences version 22. Research question one asked how vaccinations are portrayed on Pinterest. The overarching stance of the pins was decidedly anti-vaccine: 74.0% (n = 592) fell in this category; 18% (n = 144) represented a pro-vaccine stance; 6.9% (n = 55) were neutral in tone; 1.1% (n = 9) undetermined (see Table 2 for complete results, and see Figs. 1–3 for example pins). A majority of the pins, 81.5% (n = 652), connected through the pin with a website external to Pinterest, 13.6% (n = 109) did not, and 4.9% (n = 39) pointed to a broken link. Of the 652 pins that actually pointed to another website, only 3.7% (n = 24) pointed to an official medical website (e.g., representing a hospital or a major medical organization). See Table 3 for a complete list of website types. Research question two asked how Pinterest users engage with vaccination content on the platform. Pinterest provides three ways for people to interact with pins: “repinning” (similar to Twitter’s “retweet” function), “liking”, and posting comments. Repinning was the most popular form of interaction: 69.6% (n = 557) of the
1 A complete list of the coefficients: .73 (image type), .89 (image stance), .77 (caption stance), .98 (link stance), .73 (juxtaposition image and caption), .82 (overall pin stance), .83 (story type), .98 (vaccine type), .85 (conspiracy theory), .87 (civil liberties), .83 (perceived benefits), .73 (perceived barriers), .84 (perceived susceptibility), .85 (perceived severity), and .85 (self-efficacy).
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056 Table 1 Pinterest content variables.
5053
Table 3 Website type.
Variable
Description
Website type
Percentage
Image type
• Primarily image • Primarily text • Mix of image and text • Infographic: graphic visual representations of information, data or knowledge intended to present complex information quickly and clearly
Blog Other social media platform (besides Pinterest) Government website Official medical website (hospital, clinic, major medical organization) Commercial website Other type of website
19.0% (n = 124) 10.1% (n = 66)
Image, caption, link stance
• Pro-vaccine (mentions of vaccines were positive in tone and/or expressed approval of vaccines) • Anti-vaccine (mentions of vaccines were negative in tone and/or expressed disapproval of vaccines) • Neutral
Story type
• Narrative: testimonials or anecdotes • Statistical: incidence rates related to vaccinations and vaccines
Conspiracy theory
Accusing a group or organization (e.g., the government, pharmaceutical companies, or doctors) of having caused or covered up vaccine adverse reactions or events: • Pharmaceutical companies and doctors benefit from illnesses caused by vaccines • Vaccination policies are motivated by profit • Vaccine information is withheld from the public • Doctors are ignorant or fearful of sanctions • Dangers of diseases are exaggerated to frighten parents
Civil liberties related
Dealing with a real or perceived violation of civil liberties: • Violating parental rights • Monitoring parents for adherence to vaccination schedules • Vaccine mandates perceived as excessive government control
Health Belief Model constructs
• Perceived benefits of the vaccine/vaccination • Perceived barriers to the vaccine/vaccination • Perceived susceptibility to contracting the disease the vaccine is supposed to protect from • Perceived severity of the disease the vaccine is supposed to protect from • Perceived self-efficacy
Type of vaccine
• DTP (diphtheria, tetanus, pertussis) • MMR (measles, mumps, rubella) • HPV (human papilloma virus) • Flu • Hepatitis B • Other • General vaccine
pins were repinned at least once. In contrast, 42.3% (n = 338) of the pins received at least one “like” and only 6.4% (n = 51) attracted at least one comment. The mean number of repins was 7.41, of likes 1.39, and of comments .24. The highest number of repins for one pin was 250, and the highest number of likes was 34. Research question three asked how the HBM constructs were represented in pins. The five primary variables of the Health Belief Model appear in varying degrees in the study sample (see Table 4 for complete results).
0.3% (n = 2) 3.7% (n = 24) 0.2% (n = 1) 66.7% (n = 435)
Table 4 Health Belief Model (HBM) constructs. Health Belief Model constructs
Percentage
Perceived benefits of vaccination
• High effectiveness: 16.5% (n = 132) • Low/no effectiveness: 20.3% (n = 162) • No information: 63.2% (n = 506)
Perceived barriers to vaccination
• High barriers: 59.8% (n = 478) • Low/no barriers: 3.8% (n = 30) • No information: 36.5% (n = 292)
Perceived susceptibility to the disease vaccine is supposed to protect from
• High: 8.9% (n = 71) • Low: 5.0% (n = 40) • No information: 86.1% (n = 689)
Perceived severity of the disease vaccine is supposed to protect from
• Severe: 4.6% (n = 37) • Not severe: 2.1% (n = 17) • No information: 93.3% (n = 746)
Perceived self-efficacy (ability to get a vaccine or get a vaccine for one’s children)
• Present: 0.4% (n = 3) • Absent: 99.6% (n = 797)
Research question four asked to what extent pins mention issues related to conspiracy theories and civil liberties. Conspiracy theory and concern for civil liberties were consistently two of the more popular themes. In this sample, 21.5% (n = 172) of pins mentioned the concept of a conspiracy on either the government’s or the pharmaceutical companies’ side, while only 4.8% (n = 38) did the same with the concept of civil liberties. Focusing only on the 592 pins with an anti-vaccine stance, 27.7% (n = 164) mentioned a conspiracy theory, while only 6.3% (n = 37) of the 592 pins addressed vaccines as a threat to civil liberties. Hypothesis one posited that anti-vaccine messages on Pinterest would use narrative vaccination information more frequently, whereas pro-vaccine messages would be more likely to use statistical representations. Of all 800 pins, 10.6% (n = 85) used narrative information, while 6.6% (n = 53) used primarily statistical information to communicate; 82.8% (n = 662) did not use either. Pro-vaccine pins use more statistical information (21.5%, N = 31) than narrative information (7.6%, N = 11). Anti-vaccine pins use more narrative information (11.3%, N = 67) than statistical information (3.0%, N = 18). Hypothesis one is supported. Specifically, there was a significant difference in the types of information used across anti- and pro-vaccine messages such that anti-vaccine pins used more narrative and pro-vaccine pins used more statistical representations (chi-square = 32.86(1), p < .001).
Table 2 Vaccine-related stance on Pinterest. Total Pro-vaccine Anti-vaccine Neutral Inconsistent Total
18.0% (n = 144) 74.0% (n = 592) 6.9% (n = 55) 1.1% (n = 9) 100% (n = 800)
Image
Caption
Link
11.6% (n = 93) 46.8% (n = 374) 41.6% (n = 333) N/A
17.6% (n = 141) 71.3% (n = 570) 10.4% (n = 83) N/A
19.2% (n = 125) 76.8% (n = 501) 4.0% (n = 26) N/A
100% (n = 800)
100% (n = 800)
100% (n = 652)
5054
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
Fig. 1. Example pin: Gardasil warning.
Fig. 2. Example pin: vaccine injury story.
4. Discussion This study is among the first to analyze the portrayal of vaccinations on the social media platform Pinterest. A quantitative content analysis of 800 pins showed that the majority of pins portray vaccinations in a negative light. Importantly, vaccination portrayals on YouTube were 74.7% pro-vaccine in Ache et al.’s [19] HPV vaccine study, 32% pro-vaccine in Briones et al.’s [17] HPV vaccine study, and now only 24.0% pro-vaccine in this general vaccine study. Although these studies vary in their methodologies and cannot be directly compared, this—together with the increase in public voicing of anti-vaccination sentiment—is a concerning trend in public sentiment that may be impacting the level of uptake of vaccines crucial to the continued prevention of morbidity and mortality due to infectious, vaccine preventable diseases. Considering the potential harmful health consequences for those who are not vaccinated, it is concerning that very few of the linked external websites refer to a government website like the Centers for Disease Control and Prevention or a medical website. Seemingly, organizations with the information and resources
to provide a counter-argument to the strong anti-vaccine presence on Pinterest do not participate in discussions on the platform as intensely as other content creators. Public health and medical organizations should increase their engagement with the public via social media to communicate the benefits and safety of vaccines [21]. While most pins in this study did not use narrative or statistical information, for the ones that did, pro-vaccine pins still featured more statistical information while anti-vaccine messages featured more narrative information. Several studies have noted that narrative information referring to adverse vaccination events will decrease vaccination intentions, as well as narratives having an overall stronger influence than statistical information [12]. As such, health educators should consider using more narrative information about the protective effects of vaccines. With respect to the HBM, perceived barriers appeared in more than half of the pins (59.7%), indicating a perception of great concern about vaccine safety and side effects. In contrast, there is an overwhelming lack of empirical evidence that vaccines are associated with serious health risks. The likelihood of vaccination uptake
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
5055
anti-vaccination movement, they do provide a foundation from which further interventions can be planned. 4.1. Limitations and future considerations There are a number of limitations and future considerations to this study. First of all, Pinterest does not list its pins chronologically and does not list an exact time stamp for each pin. This makes using a more conventional content analysis sampling method like a constructed two week time period virtually impossible. The lack of previous Pinterest studies means few examples are available and that there is lack of consensus in the field regarding best sampling practices for this medium. Furthermore, while this study focused on Pinterest, future studies in this field should consider Facebook and Instagram to study the same topic and build a more robust field of literature regarding the portrayal of vaccines and vaccinations on social media platforms. Finally, this study used the theoretical framework of the HBM to examine some of the psychosocial variables present in the portrayal of vaccines and vaccinations on Pinterest. The HBM is one of several models that has been applied to vaccination studies, and future studies should consider approaching this study concept from other social and behavioral health theories like the Theory of Reasoned Action and the Extended Parallel Processing Model [15]. We used the HBM because it is the most widely used model in studies on preventative behaviors such as vaccines. However, the content analysis showed the presence of fear messages, and the Extended Parallel Processing Model (EPPM) may be helpful to address that particular issues. The Theory of Reasoned Action (TRA) is particularly appropriate for audiences where social norms are very strong. Conflict of interest The authors have no conflict of interest to disclose. References
Fig. 3. Example pin: vaccines and autism.
decreases, however, as perceived barriers increase [22], making addressing vaccine safety and side effects concerns is of great importance. Finally, though civil liberties were not often addressed among the pins in this study, 21.6% of the pins did refer to the concept of conspiracy theory, including the belief that the government and the pharmaceutical industry are hiding “the truth about vaccines” or that their efforts in promoting vaccines are motivated exclusively by the pursuit of profit. These content themes have the potential to influence underlying perceptions of vaccination programs and should not be discounted. The success of vaccines in eradicating some, and greatly reducing morbidity and mortality in other, infectious diseases continues to be overshadowed by the growing anti-vaccine movement. Successful health communication campaigns should start with an understanding of the target audience’s current perceptions of the health issue at hand, including those communicated on more recently developed social media platforms like Pinterest. This quantitative content analysis of 800 vaccination-focused pins concluded that a significant majority of pins are anti-vaccine in stance and that potentially powerful pro-vaccine voices are not as present on Pinterest as their anti-vaccine counterparts. Perceived barriers, operationalized as vaccine side effects and safety, are of great concern on Pinterest. Health educators and public health campaigns should consider using more narrative information to tell the stories of lives saved through vaccines. While an increased social media presence, additional education, and a change in communication style by themselves are not likely to stem the tide of the
[1] Kata A. Anti-vaccine activists, Web 2.0, and the postmodern paradigm—an overview of tactics and tropes used online by the anti-vaccination movement. Vaccine 2012;30(25):3778–89, http://dx.doi.org/10.1016/j.vaccine.2011.11.112. [2] Paulussen TGW, Hoekstra F, Lanting CI, Buijs GB, Hirasing RA. Determinants of Dutch parents’ decisions to vaccinate their child. Vaccine 2006;24:644–51, http://dx.doi.org/10.1016/j.vaccine.2005.08.053. [3] Kata A. A postmodern Pandora’s box: anti-vaccination mison the Internet. Vaccine 2010;28:1709–16, information http://dx.doi.org/10.1016/j.vaccine.2009.12.022. [4] Madden K, Nan X, Briones RL, Waks L. Sorting through search results: a content analysis of HPV vaccine information online. Vaccine 2012;30:3741–6, http://dx.doi.org/10.1016/j.vaccine.2011.10.025. [5] Kitta A. Vaccinations and public concern in history: legend, rumor, and risk perception. New York, NY: Routledge; 2012. [6] Sugerman DE, Barskey AE, Delea MG, Ortega-Sanchez IR, Bi D, Ralston KJ, et al. Measles outbreak in a highly vaccinated population, San 351 Diego, 2008: role of the intentionally undervaccinated. Pediatrics 2010;125:747–55, http://dx.doi.org/10.1542/peds.2009-1653. [7] Love B, Himelboim I, Holton A, Stewart K. Twitter as a source of vaccination information: content drivers and what they are saying. Am J Infect Control 2013;41:568–70, http://dx.doi.org/10.1016/j.ajic.2012.10.016. [8] Betsch C, Brewer NT, Brocard P, Davies P, Gaissmaier W, Haase N, et al. Opportunities and challenges of Web 2.0 for vaccination decisions. Vaccine 2012;30:3727–33, http://dx.doi.org/10.1016/j.vaccine.2012.02.025. [9] Poland GA, Jacobson RM, Ovsyannikova IG. Trends affecting the future of vaccine development and delivery: the role of demographics, regulatory science, the anti-vaccine movement, and vaccinomics. Vaccine 2009;27:3240–4, http://dx.doi.org/10.1016/j.idc.2009.01.008. [10] Harmsen IA, Doorman GG, Mollema L, Ruiter RAC, Kok G, de Melker HE. Parental information-seeking behavior in childhood vaccinations. BMC Public Health 2013;13(1219). [11] Jones AM, Omer SB, Bednarczyk RA, Halsey NA, Moulton LH, Salmon DA. Parents’ source of vaccine information and impact on vaccine attitudes. Beliefs, and nonmedical exemptions. Adv Prev Med 2012;2012. [12] Betsch C, Ulshofer C, Renkewitz F, Betsch T. The influence of narrative v. statistical information on perceiving vaccination risks. Med Decis Mak 2011;31:742–53, http://dx.doi.org/10.1177/0272989X11400419.
5056
J.P.D. Guidry et al. / Vaccine 33 (2015) 5051–5056
[13] Keelan J, Pavri-Garcia V, Tomlinson G, Wilson K. YouTube as a source of information on immunization: a content analysis. JAMA 2007;298:2482. [14] Smith C. By the numbers: 59 amazing Pinterest stats; 2014. Available from: http://expandedramblings.com/index.php/pinterest-stats/-.UzJHwdzEt8w [accessed 23.03.14]. [15] Glanz K, Rimer BK, Lewis ML. Health behavior and health education: theory, research, and practice. San Francisco, CA: Jossey-Bass; 2008. [16] Gerend M, Shepherd J. Predicting human papillomavirus vaccine uptake in young adult women: comparing the health belief model and theory of planned behavior. Ann Behav Med 2012;44:171–80. [17] Briones RL, Nan X, Madden K, Waks L. When vaccines go viral: an analysis of HPV vaccine coverage on YouTube. Health Commun 2012;27:478–85, http://dx.doi.org/10.1080/10410236.2011.610258. [18] Davies P, Chapman S, Leask J. Antivaccination activists on the world wide web. Arch Dis Child 2002;87:22–5. [19] Ache KA, Wallace LS. Human papillomavirus vaccination coverage on YouTube. Am J Prev Med 2008;35:389–92, http://dx.doi.org/10.1016/j.amepre.2008.06.029.
[20] Scott WA. Reliability of content analysis: the case of nominal scale coding. Public Opin Q 1955;19:321–5. [21] Poland GA, Jacobson RM. Understanding those who do not understand: a brief review of the anti-vaccine movement. Vaccine 2001;19:2440–5, http://dx.doi.org/10.1016/S0264-410X(00)00469-2. [22] Looijmans-van den Akker I, van Delden JJM, Verheij TJM, van Essen GA, van der Sande MAB, Hulscher ME, et al. Which determinants should be targeted to increase influenza vaccination uptake among health care workers in nursing homes? Vaccine 2009;27:4724–30, http://dx.doi.org/10.1016/j.vaccine.2009.05.013. [23] Betsch C, Renkewitz F, Betsch T, Ulshofer C. The influence of vaccine-critical websites on perceiving vaccination risks. J Health Psychol 2010;15:446, http://dx.doi.org/10.1177/1359105309353647. [24] Riffe D, Lacy S, Fico FG. Analyzing media messages: using quantitative content analysis in research. Mahwah, NJ: Lawrence Erlbaum Associates; 1998. [25] Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum Commun Res 2002;28(4):587–604.