- Email: [email protected]
Evaluation of Information Available on the Internet Regarding Anterior Cruciate Ligament Reconstruction
Current Concepts
Evaluation of Information Available on the Internet Regarding Anterior Cruciate Ligament Reconstruction Ian C. Duncan, M.D., Patrick W. Kane, M.D., Kevin A. Lawson, B.A., Steven B. Cohen, M.D., Michael G. Ciccotti, M.D., and Christopher C. Dodson, M.D.
Abstract: Searching the Internet is one of the most popular methods for acquiring information related to health. The Internet offers physicians and patients easy access to a wide range of medical material from anywhere in the world. For many patients, this information helps formulate decisions related to their health and health care. An important caveat is that virtually anything can be published on the Internet. Although academic publications require rigorous peer review, Internet websites have no regulatory body monitoring quality and content. With a lack of external regulation, the information retrieved may be incorrect or outdated. The Internet can be a valuable asset for educating patients, but because of significant variability physicians should be familiar with the quality of information available. This article discusses both the strengths and weaknesses of information available on the Internet regarding anterior cruciate ligament repair.
A
nterior cruciate ligament (ACL) rupture is the most frequent ligamentous injury of the knee,1 is common among young athletic populations, and predisposes the knee to subsequent injuries and the potential for early onset of osteoarthritis.2-5 Arthroscopic ACL reconstruction is a common, relatively safe, and effective method of restoring stability to the knee after injury, with more than 100,000 ACL reconstructions performed annually.6-10 Like any surgical procedure, a number of potential complications have been recognized that may affect functional outcome, such as postoperative bleeding, pain, infection, and repeated rupture.11-13 In addition, patients and their physicians must decide on the most appropriate graft selection and donor site. Surgical technique, graft choices, time for return to sport, postoperative complications, and risk of repeated rupture are a few examples of medical information patients seek before undergoing elective ACL reconstruction.14 From The Rothman Institute, Thomas Jefferson Sports Medicine Division, Philadelphia, Pennsylvania, U.S.A. The authors report that they have no conflicts of interest in the authorship and publication of this article. Received January 10, 2013; accepted February 8, 2013. Address correspondence to Christopher C. Dodson, M.D., The Rothman Institute, 925 Chestnut Street, 5th Floor, Philadelphia, PA 19010, U.S.A. E-mail: [email protected] Ó 2013 by the Arthroscopy Association of North America 0749-8063/1334/$36.00 http://dx.doi.org/10.1016/j.arthro.2013.02.008
Searching the Internet is one of the most popular methods for acquiring information related to health,15 and increasing numbers of patients are seeking this information online. More than 66 million Americans use the Internet daily.16 It offers physicians and patients access to a wide range of medical material almost instantaneously at any time and from anywhere in the world. For many patients, this information helps formulate decisions related to their health and health care.17 A critically important caveat is that virtually anything can be published on the Internet. With no regulatory body monitoring quality and content, there is easy circumvention of the peer-review process that is an essential feature of academic publications.18 With a lack of external regulation, the information retrieved may be incorrect or outdated. Several investigators have studied the use of the Internet as a medium for patients to derive health information.19-23 Beredjiklian et al.24 performed one of the first analyses of Internet information, on carpal tunnel syndrome, and found poor quality and limited scope.24 Another study by Sambandam et al.,25 the only similar study looking at a sports topic, found that the available information pertaining to knee arthroscopy was inadequate. Others, however, have concluded that the potential of Internet-derived medical information might be transformative to health care, with the possibility of better health outcomes and the more appropriate use of health care services.26 With these considerations in mind, the purpose of this study was to
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol 29, No 6 (June), 2013: pp 1101-1107
1101
1102
I. C. DUNCAN ET AL.
determine if patients can reliably use the Internet to obtain information about ACL reconstruction by evaluating the authorship and content of the websites. We hypothesized that websites reviewing ACL reconstruction are of higher quality than websites discussing other orthopaedic procedures.
Evaluating Websites To evaluate the quality of Internet information available for our topic, the search term ACL reconstruction was entered into 4 separate search engines (Google, Yahoo, Bing, and Ask). These 4 search engines accounted for greater than 92% of search engine use at the time of the investigation (Google, 83%; Yahoo, 6%; Bing, 3%, and Ask, <1%).27 The first 50 websites from each search engine were selected for evaluation in our study. All duplicate or overlapping websites within and among the search engines were discarded so that each website included was only evaluated once. Similarly, all advertisements as well as strictly video search results that contained no text were excluded. The search query began with the least popular search engine (Ask) and was continued from least to most popular, with Google being evaluated last, so more websites were evaluated from Google (since many of the first 50 sites were duplicates from the first 3 search engines). After each search engine was queried, a master list of 200 websites was created for individual evaluation and assessment. Each website was evaluated by one of 2 authors and verified by a third author during final analysis. There was 99.5% agreement in the categorization of each website. Authorship The authorship or sponsorship of each website was categorized into one of 6 categories: (1) academic included any university-affiliated physician or research group; (2) private indicated any physician or group that had no stated affiliation with any academic organization; (3) industry included any manufacturing or marketing company that advertised a product or service for profit; (4) news sources included any bulletin or article without an affiliation to any hospital or academic institution; (5) public education sites indicated any individual or organization with a noncommercial website designed to provide third-party information (e.g., Wikipedia and about.com); (6) blogs included any website that recounted an individual’s personal experiences in diary or journal form. Ability to Contact Author The ability to contact the author and directly schedule an appointment or consultation was analyzed for each website. An online appointment link, an email form for scheduling an appointment, or a listed telephone number were all considered positive for the ability to contact the author.
Discussion of Disorder Each website was reviewed for the presence or absence of a detailed explanation of an ACL injury. Specifically, websites had to either include an illustration of the knee with a depiction of an ACL injury or include a detailed description of both normal and abnormal knee anatomy. The description had to include tibia and femur as well as a description of the function of the ACL in a normal knee. A brief explanation of the consequences of an ACL tear on knee stability and function also had to be given. Surgical Technique A description of the surgical procedure of ACL reconstruction could be given in either diagram or paragraph form. A written description of the surgical technique had to at least include arthroscopic or surgical approach, graft use, and tibial and femoral fixation to be considered adequate. Complications The mention of risks and complications associated with ACL reconstruction was noted for each website. The risks that were analyzed included, but were not limited to, nerve damage, infection, persistent instability, complications from anesthesia, pain, bleeding, blood clots (including deep venous thrombosis and pulmonary embolism), and potential for revision surgery. A website needed to mention at least one of the complications listed to be considered positive for offering potential surgical risk. Other Treatment Options An inclusion of nonsurgical treatment options such as physical therapy, orthotic treatment, neuromuscular training, and so on was assessed for each website. Eligibility Each website was evaluated for a discussion of the factors that determine a patient’s eligibility for ACL reconstruction. Patient chronologic age as well as activity level and physiologic age needed to be mentioned to be considered positive for including eligibility criteria. Unreal Claims Three specific claims of a bloodless surgery, guaranteed pain relief, and guaranteed return to play or sport were examined separately for each website. Rehabilitation The postoperative rehabilitation regimen needed to be briefly mentioned for each individual website. A detailed description of specific rehabilitation protocols did not need to be included to be considered positive. Graft Selection A brief discussion on graft material used for ACL reconstruction was evaluated for each website. A
1103
ACL RECONSTRUCTION INFORMATION ON THE INTERNET
detailed comparison of the advantages and disadvantages of each graft type did not need to be given. However, at a minimum websites needed to at least mention allograft as well as bone-patellar-bone and hamstring autograft options to be considered as having adequately discussed graft choices. Other Injuries Websites were also analyzed for discussion of other potential injuries encountered in a knee with an ACL injury. Potential injuries included, but were not limited to, collateral ligament injury, posterolateral corner injury, meniscal injury, growth plate disturbance, bone contusion, and avulsion fracture. Although a detailed description of additional injury or its potential consequences on ACL reconstruction did not need to be given, a brief discussion of the possibility of other concomitant knee injuries did need to be included. Peer-Reviewed References Each website had to reference or cite a specific peerreviewed source for information provided regarding ACL reconstruction to be considered positive.
Results The comprehensive analysis of the top 50 websites from each search engine (200 total) can be found in Fig 1 and Tables 1 and 2. In regard to authorship, it was found that 45 websites (22.5%) were affiliated with an academic institution; 71 (35.5%) were authored by private physicians or physician groups with no academic affiliation; 9 (4.5%) were industry sponsored; 12 (6%) were news bulletins or media reports; 39
Table 1. Ability to Contact Author and Initiate Injury Discussion Website
Ability to Contact
Ask Bing Yahoo Google Total 81 (N ¼ 200)
Discussion Treatment of Disorder Options Eligibility
24 25 18 29 21 34 18 32 (40.5%) 120 (60%) 62
13 14 17 18 (31%) 58
Related Injuries
13 9 14 12 12 11 19 8 (29%) 41 (20.5%)
(19.5%) were public education websites; and 24 (12%) were personal blogs (Fig 1). A summary of the contents from the websites found using each search engine are shown in Tables 1 and 2. Eighty-one websites (40.5%) provided either information or direct access for scheduling an appointment or consultation with the website author. A detailed description of an ACL injury was given in 60% of websites (n ¼ 120). However, only 41 websites (20.5%) made any mention of potential concomitant pathologic processes of the knee with ACL injury. Similarly, although 125 websites (62.5%) offered an explanation of the surgical technique of ACL reconstruction, only 30% of websites discussed eligibility for surgery or other nonsurgical treatment options. Roughly half of all websites did discuss graft selection and postoperative rehabilitationd55% and 48.5%, respectively. Only 52 websites (26%) included peer-reviewed references in the information provided. Fortunately, only 3 websites (1.5%) made false claims of painless and bloodless surgery and guaranteed return to play after ACL reconstruction. Table 3 is a summary of other orthopaedic studies investigating the authorship and content of websites for various other orthopaedic injuries or procedures compared with the present study on Internet information on ACL reconstruction. Of all the studies included in Table 3, the websites found for ACL reconstruction had a higher proportion of academic, private physician, and public education authorship. Compared with a study on cervical disk arthroplasty, the percentage of websites discussing ability to contact the author, procedure eligibility, and complications were fairly uniform.23 Few studies looked at the use of peer-reviewed references for citation, but percentages were similar in the 2 studies that focused on the ACL (26% v 34%).20
Discussion
Fig 1. Website authorship for anterior cruciate ligament (ACL) reconstruction. Values represent percentage of total number of websites (N ¼ 200).
There have been multiple studies that have already looked at information available on the Internet for different orthopaedic specialties, but none have looked specifically at ACL reconstruction.19-23 One recent study reviewed the quality of information of the Internet relating to ACL injury in female athletes.20 This study focused on the readability of website content and
1104
I. C. DUNCAN ET AL.
Table 2. Explanation of Surgery, Postoperative Regimen, and References Website
Surgical Technique
Graft Selection
Complications
Rehabilitation
Peer-Reviewed References
Ask Bing Yahoo Google Total (N ¼ 200)
34 28 31 32 125 (62.5%)
29 26 27 28 110 (55%)
14 18 15 13 60 (30%)
23 23 25 26 97 (48.5%)
14 12 11 15 52 (26%)
used a nonvalidated grading system to assess the quality of websites reviewed. Our methodology was similar to that of many of the previous studies.25 Like the referenced articles, a binary list of items was noted as present or absent, rather than using an arbitrary nonvalidated grading scale. The only notable exception is that the current study used different search engines (Google, Yahoo, Ask, Bing), which according to a current US reference, account for greater than 93% of web searches in the United States.27 Previous articles on this topic have looked at several general criteria that are necessary in a good website, such as discussion of the surgical technique and complications and then some specific criteria regarding the particular surgery. Ideally, a high-quality patient information website should be written by a credible source, discuss the disorder, give treatment options and guidelines for choosing a treatment (such as the indications for surgery), provide at least a brief overview of the surgical procedure, at least mention complications, and, in today’s era of evidence-based medicine, cite peer-reviewed references. The specific information for ACL reconstruction that we believed improved the quality of the website included any discussion of other related injuries, such as pathologic processes of the meniscus, other ligament injuries, fractures, or chondral injuries, graft options, and rehabilitation. Anecdotally, there seem to be more patients coming into the office with a desire to get a certain type of graft, so websites were evaluated on whether or not they reviewed this information. The websites covered this topic with varying bias and thoroughness, but to facilitate binary data collection, at least mentioning the topic was considered positive. Although single-bundle versus double-bundle ACL reconstruction has been
a popular topic in the literature over the past 10 years, there has been decreased focus on it lately, with more of a specific emphasis on the concept of anatomic ACL reconstruction. Therefore, single-bundle versus doublebundle reconstruction was not included as an important topic to have on a website. Subjectively, very few websites discussed single-bundle versus double-bundle reconstruction or anatomic positioning of the tunnels. Regarding authorship, there were a large number of credible sources, which was consistent with similar investigations: 45 websites (22.5%) were affiliated with an academic institution and 39 (19.5%) were public education websites. Seventy-one (35.5%) of the websites were authored by private physicians or physician groups with no academic affiliation. Although private physicians and their groups often have very welldesigned websites, they are often biased because they are trying to solicit business; therefore, they are often not the most complete, especially regarding complications. They were noted to make false claims such as guaranteeing bloodless or painless surgery and return to sports after reconstruction. There were also a number of even less credible sources: 24 websites (12%) were personal blogs, 12 (6%) were news bulletins or media reports, and 9 (4.5%) were industry sponsored. In general, the websites reviewed in this study that discuss ACL reconstruction contained less industry bias and overall better quality than the other articles previously written on this topic.19-23 Although industry-sponsored websites are often well designed and eye-catching, one cannot ignore the underlying goal to solicit the consumer. ACL reconstruction has the fewest industry-sponsored websites, with only 9 websites (4.5%) compared with studies from other specialties such as cervical arthroplasty (13%), sports medicine (46%), ACL injuries in female patients (6%),
Table 3. ACL Reconstruction Results Compared With Other Similar Studies on Information Available on the Internet Topic
Academic/Private/ Public Education
Industry Author
ACL reconstruction Sports medicine21 ACL injuries in female athletes22 Cervical herniation24 Cervical disk arthroplasty25*
79 46 28 66 58
4.5 48 6 18 13
Ability to Contact
Eligibility
Complications
Peer Reference
40.5
29
30
26 34
65
19
35
Scoring No Yes Yes Yes No
*Methodology similar to that used in the present study. Blank spaces were categories not evaluated in that study. Values represent percentages.
ACL RECONSTRUCTION INFORMATION ON THE INTERNET
and cervical herniation. Interestingly, searches dealing with the ACL had the fewest number of commercial websites. Industry understandably invests less in producing websites for ACL reconstruction because the ACL implants are significantly less expensive than arthroplasty implants, and there seems to be less patient interest in knowing which ACL fixation implants are used compared with arthroplasty implants. Curiously, in comparison with the other studies, cervical arthroplasty in general had 18% industry-sponsored websites, whereas the study that focused on 10 common sports diagnoses had the highest percentage of industry websites (approximately 48%). A new authorship category that has not been previously evaluated is personal blogs (12%), likely because the number of these sites has grown significantly over the past few years. Blogs are a great format for an individual, or even a small group, to communicate their experiences. The opportunity for anyone to have a public forum to share their thoughts and feelings has led to their popularity. However, because blogs are not censored or screened, too much emphasis can be placed on an individual’s good or bad experience, and skewed perspectives can be conveyed. Certainly, patients with bad outcomes are more vocal and may persuade patients to use a different surgeon or graft or avoid a similar decision because they had a bad experience, even if that bad outcome is extremely rare. The ideal health information website is written from a credible source that provides a point of contact, such as a phone number or e-mail address, to have questions answered or to make an appointment with the author. Only 81 websites (40.5%) in this study provided this information. Another positive for ACL reconstruction websites is that they discussed eligibility and complications and referenced peer-reviewed sources approximately as much or more than similar articles (Table 3). Detailed description of an ACL injury was given in 120 websites (60%), but only 41 websites (20.5%) made any mention of potential concomitant pathologic processes of the knee in ACL injury (Table 1). Eighty-one websites (40.5%) provided either information or direct access for scheduling an appointment or consultation with the website author (Table 1). Similarly, although 125 websites (62.5%) offered an explanation of the surgical technique of ACL reconstruction (Table 3), only 30% of websites discussed eligibility for surgery or other treatment options such as nonoperative treatment (Table 1). Roughly half of all websites discussed graft selection and postoperative rehabilitationd55% and 48.5%, respectively (Table 2). Only 52 websites (26%) included peer-reviewed references in the information provided (Table 2). Fortunately, only 3 websites (1.5%) made false claims of painless and bloodless surgery and guaranteed return to play after ACL reconstruction. One concerning trend in these websites
1105
is that they discuss the benefits of a surgical procedure but leave out even a mention of complications, giving the perception that patients are being coerced into surgical treatments. To be more thorough, authors of websites should put more emphasis on presenting balanced pros and cons of a procedure so that patients can make truly educated decisions. Interestingly, most of the similar orthopaedic studies looking at information available on the Internet used different methods. Most used different search engines depending on when they were written and based on which search engines were most commonly used at the time. Several used a grading scale, but they were often different nonvalidated scales. Others used the HONcode, which is created and maintained by the Health on the Net Foundation, a third-party organization that certifies websites that follow its principles when disseminating medical information.28 Another study focusing on distal radius fractures analyzed the quality of information found on the Internet depending on the search term used.23 Overall, when comparing the results from this study with those of similar studies, there is slightly higher quality information available to patients researching ACL reconstruction. There are fewer industry-sponsored websites, a similar percentage of peer-reviewed sources, and a lower percentage of misleading claims. Despite most websites discussing ACL ruptures and treatment options, only a quarter reference peerreviewed articles. There are also a large number of websites that are essentially anecdotal stories of a single patient’s experience (mostly blogs), which may not be predictive of common outcomes. Fortunately, only a small percentage (1.5%) of the websites contained false claims. The Internet can be a valuable asset for educating patients, but physicians should be familiar with which websites contain high-quality information, so we can guide our patients to the appropriate resources. Limitations A weakness of this study is the relatively subjective nature in which the websites were graded. Although there was a list of criteria for an ideal website, which was noted for all items, one of the various grading systems used in some of the other similar studies was not used in this study. The decision not to use a grading scale was made after reviewing the literature and concluding that none of the grading systems had been adequately validated. Few of the other similar orthopaedic articles reviewed used the same system but rather subjectively created different grading scales. In this study, our goal was simply to note whether or not important topics were discussed on a website. If a validated scale was used, further differentiation between the websites that simply mention an important topic
1106
I. C. DUNCAN ET AL.
versus those that have thorough coverage of the same topic might have been possible. Another critique is that multiple authors did not review each website unless clarification was needed. This decision was made for 2 reasons: (1) the list of items noted on each website was very simple and straightforward and little confusion or ambiguity existed and (2) having a larger sampling of websites was determined to be more important than having multiple evaluations of the same website, so effort was divided so that as many websites as possible were reviewed.
Conclusions The quality of Internet information available to patients searching for ACL reconstruction appears mixed. Although a majority of websites offer a discussion of the condition as well as a description of the surgical procedure, less than half touch on equally important issues such as patient eligibility for this procedure and the potential complications. Fortunately, a very small percentage made false claims, such as painless surgery and a guaranteed return to activity. With an increasing number of patients using the Internet for health care education, it is important for physicians to be aware of the quality of information available.
References 1. Kamath GV, Redfern JC, Greis PE, Burke RT. Revision anterior cruciate ligament reconstruction. Am J Sports Med 2011;39:199-217. 2. Friel NA, Chu CR. The role of ACL injury in the development of posttraumatic knee osteoarthritis. Clin Sports Med 2013;32:1. 3. Frobell RB, Roos EM, Roos HP, Ranstam J, Lohmander LS. A randomized trial of treatment for acute anterior cruciate ligament tears. N Engl J Med 2010;363: 331-342. 4. Love H, Mohtadi N. Treatment of ACL tears in active young adults. Clin J Sports Med 2011;21:73. 5. Maffulli N, Longo UG, Gougoulias N, Loppini M, Denaro V. Long-term health outcomes of youth sports injuries. Br J Sports Med 2010;44:21-25. 6. Ardern CL, Taylor NF, Feller JA, Webster KE. Return-tosport outcomes at 2 to 7 years after anterior cruciate ligament reconstruction surgery. Am J Sports Med 2012;40: 41-48. 7. Brophy RH, Schmitz L, Wright RW, et al. Return to play and future ACL injury risk after ACL reconstruction in soccer athletes from the Multicenter Orthopaedic Outcomes Network (MOON) Group. Am J Sports Med 2012;40:2523-2529. 8. Ellis HB, Matheny LM, Briggs KK, Pennock AT, Steadman JR. Outcomes and revision rate after bonepatellar tendon-bone allograft versus autograft anterior cruciate ligament reconstruction in patients aged 18 years or younger with closed physes. Arthroscopy 2012;27:11291134.
9. Warner SJ, Smith MV, Wright RW, Matava MJ, Brophy RH. Sport-specific outcomes after anterior cruciate ligament reconstruction. Arthroscopy 2011;27: 1129-1134. 10. Vyas D, Rabuck SJ, Harner CD. Allograft anterior cruciate ligament reconstruction: Indications, techniques, and outcomes. J Orthop Sports Phys Ther 2012;42:196-207. 11. Richardson ML, Wahl CJ. Detached femoral bone plug following anterior cruciate ligament repair. Radiology Case 2011;64. 12. Tjoumakaris FP, Herz-Brown AL, Legath-Bowers A, Sennett BJ, Bernstein J. Complications in brief: Anterior cruciate ligament reconstruction. Clin Orthop Relat Res 2012:1-7. 13. Benner RW, Shelbourne KD, Freeman H. Infections and patellar tendon ruptures after anterior cruciate ligament reconstruction: A comparison of ipsilateral and contralateral patellar tendon autografts. Am J Sports Med 2011;39:519-525. 14. Cheung SC, Allen CR, Gallo RA, Ma CB, Feeley BT. Patients’ attitudes and factors in their selection of grafts for anterior cruciate ligament reconstruction. Knee 2012;19:49-54. 15. Cohen RA, Stussman B. Health Information Technology Use Among Men and Women Aged 18-64: Early Release of Estimates from the National Health Interview Survey, January-June 2009. Atlanta, GA: National Center for Health Statistics, Centers for Disease Control; 2010. 16. Koh HS, In Y, Kong CG, Won HY, Kim KH, Lee JH. Factors affecting patients’ graft choice in anterior cruciate ligament reconstruction. Clin Orthop Surg 2010;2:69-75. Epub 2010 May 4. 17. Groves ND, Humphreys HW, Williams AJ, Jones A. Effect of informational internet web pages on patients’ decisionmaking: Randomised controlled trial regarding choice of spinal or general anaesthesia for orthopaedic surgery. Anaesthesia 2010;65:277-282. 18. Lichtenfeld LJ. Can the beast be tamed? The woeful tale of accurate health information on the Internet. Ann Surg Oncol 2012:1-2. 19. Starman JS, Gettys FK, Capo JA, Fleischli JE, Norton HJ, Karunakar MA. Quality and content of Internet-based information for ten common orthopaedic sports medicine diagnoses. J Bone Joint Surg Am 2010;92:1612-1618. 20. Gosselin MM, Mulcahey MK, Feller E, Hulstyn MJ. Examining internet resources on gender differences in ACL injuries: What patients are reading. Knee. 2012 Dec 20. [Epub ahead of print.] 21. Dy CJ, Taylor SA, Patel RM, Kitay A, Roberts TR, Daluiski A. The effect of search term on the quality and accuracy of online information regarding distal radius fractures. J Hand Surg Am 2012;37:1882-1887. 22. Morr S, Shanti N, Carrer A, Kubeck J, Gerling MC. Quality of information concerning cervical disc herniation on the Internet. Spine J 2010;4:350-354. 23. Qureshi SA, Koehler SM, Lin JD, Bird J, Garcia RM, Hecht AC. An evaluation of information on the Internet of a new device: The cervical artificial disc replacement. Spine 2012;37:881-883. 24. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of
ACL RECONSTRUCTION INFORMATION ON THE INTERNET orthopaedic information on the Internet. The case of carpal tunnel syndrome. J Bone Joint Surg Am 2000; 82-A:1540-1543. 25. Sambandam SN, Ramasamy V, Priyanka P, Ilango B. Quality analysis of patient information about knee arthroscopy on the World Wide Web. Arthroscopy 2007;23: 509-513.e2.
1107
26. Bernard L, Parham L. The impact of web 2.0 on the doctorpatient relationship. J Law Med Ethics 2010;38:17-26. 27. Purcell K, Brenner J, Rainie L. Search Engine Use 2012. Washington, DC: Pew Internet & American Life Project, 2012. 28. Health on the Net Foundation. Available at: http://www .hon.ch/home1.html. Accessed January 23, 2012.
Evaluation of Information Available on the Internet Regarding Anterior Cruciate Ligament Reconstruction Ian C. Duncan, M.D., Patrick W. Kane, M.D., Kevin A. Lawson, B.A., Steven B. Cohen, M.D., Michael G. Ciccotti, M.D., and Christopher C. Dodson, M.D.
Abstract: Searching the Internet is one of the most popular methods for acquiring information related to health. The Internet offers physicians and patients easy access to a wide range of medical material from anywhere in the world. For many patients, this information helps formulate decisions related to their health and health care. An important caveat is that virtually anything can be published on the Internet. Although academic publications require rigorous peer review, Internet websites have no regulatory body monitoring quality and content. With a lack of external regulation, the information retrieved may be incorrect or outdated. The Internet can be a valuable asset for educating patients, but because of significant variability physicians should be familiar with the quality of information available. This article discusses both the strengths and weaknesses of information available on the Internet regarding anterior cruciate ligament repair.
A
nterior cruciate ligament (ACL) rupture is the most frequent ligamentous injury of the knee,1 is common among young athletic populations, and predisposes the knee to subsequent injuries and the potential for early onset of osteoarthritis.2-5 Arthroscopic ACL reconstruction is a common, relatively safe, and effective method of restoring stability to the knee after injury, with more than 100,000 ACL reconstructions performed annually.6-10 Like any surgical procedure, a number of potential complications have been recognized that may affect functional outcome, such as postoperative bleeding, pain, infection, and repeated rupture.11-13 In addition, patients and their physicians must decide on the most appropriate graft selection and donor site. Surgical technique, graft choices, time for return to sport, postoperative complications, and risk of repeated rupture are a few examples of medical information patients seek before undergoing elective ACL reconstruction.14 From The Rothman Institute, Thomas Jefferson Sports Medicine Division, Philadelphia, Pennsylvania, U.S.A. The authors report that they have no conflicts of interest in the authorship and publication of this article. Received January 10, 2013; accepted February 8, 2013. Address correspondence to Christopher C. Dodson, M.D., The Rothman Institute, 925 Chestnut Street, 5th Floor, Philadelphia, PA 19010, U.S.A. E-mail: [email protected] Ó 2013 by the Arthroscopy Association of North America 0749-8063/1334/$36.00 http://dx.doi.org/10.1016/j.arthro.2013.02.008
Searching the Internet is one of the most popular methods for acquiring information related to health,15 and increasing numbers of patients are seeking this information online. More than 66 million Americans use the Internet daily.16 It offers physicians and patients access to a wide range of medical material almost instantaneously at any time and from anywhere in the world. For many patients, this information helps formulate decisions related to their health and health care.17 A critically important caveat is that virtually anything can be published on the Internet. With no regulatory body monitoring quality and content, there is easy circumvention of the peer-review process that is an essential feature of academic publications.18 With a lack of external regulation, the information retrieved may be incorrect or outdated. Several investigators have studied the use of the Internet as a medium for patients to derive health information.19-23 Beredjiklian et al.24 performed one of the first analyses of Internet information, on carpal tunnel syndrome, and found poor quality and limited scope.24 Another study by Sambandam et al.,25 the only similar study looking at a sports topic, found that the available information pertaining to knee arthroscopy was inadequate. Others, however, have concluded that the potential of Internet-derived medical information might be transformative to health care, with the possibility of better health outcomes and the more appropriate use of health care services.26 With these considerations in mind, the purpose of this study was to
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol 29, No 6 (June), 2013: pp 1101-1107
1101
1102
I. C. DUNCAN ET AL.
determine if patients can reliably use the Internet to obtain information about ACL reconstruction by evaluating the authorship and content of the websites. We hypothesized that websites reviewing ACL reconstruction are of higher quality than websites discussing other orthopaedic procedures.
Evaluating Websites To evaluate the quality of Internet information available for our topic, the search term ACL reconstruction was entered into 4 separate search engines (Google, Yahoo, Bing, and Ask). These 4 search engines accounted for greater than 92% of search engine use at the time of the investigation (Google, 83%; Yahoo, 6%; Bing, 3%, and Ask, <1%).27 The first 50 websites from each search engine were selected for evaluation in our study. All duplicate or overlapping websites within and among the search engines were discarded so that each website included was only evaluated once. Similarly, all advertisements as well as strictly video search results that contained no text were excluded. The search query began with the least popular search engine (Ask) and was continued from least to most popular, with Google being evaluated last, so more websites were evaluated from Google (since many of the first 50 sites were duplicates from the first 3 search engines). After each search engine was queried, a master list of 200 websites was created for individual evaluation and assessment. Each website was evaluated by one of 2 authors and verified by a third author during final analysis. There was 99.5% agreement in the categorization of each website. Authorship The authorship or sponsorship of each website was categorized into one of 6 categories: (1) academic included any university-affiliated physician or research group; (2) private indicated any physician or group that had no stated affiliation with any academic organization; (3) industry included any manufacturing or marketing company that advertised a product or service for profit; (4) news sources included any bulletin or article without an affiliation to any hospital or academic institution; (5) public education sites indicated any individual or organization with a noncommercial website designed to provide third-party information (e.g., Wikipedia and about.com); (6) blogs included any website that recounted an individual’s personal experiences in diary or journal form. Ability to Contact Author The ability to contact the author and directly schedule an appointment or consultation was analyzed for each website. An online appointment link, an email form for scheduling an appointment, or a listed telephone number were all considered positive for the ability to contact the author.
Discussion of Disorder Each website was reviewed for the presence or absence of a detailed explanation of an ACL injury. Specifically, websites had to either include an illustration of the knee with a depiction of an ACL injury or include a detailed description of both normal and abnormal knee anatomy. The description had to include tibia and femur as well as a description of the function of the ACL in a normal knee. A brief explanation of the consequences of an ACL tear on knee stability and function also had to be given. Surgical Technique A description of the surgical procedure of ACL reconstruction could be given in either diagram or paragraph form. A written description of the surgical technique had to at least include arthroscopic or surgical approach, graft use, and tibial and femoral fixation to be considered adequate. Complications The mention of risks and complications associated with ACL reconstruction was noted for each website. The risks that were analyzed included, but were not limited to, nerve damage, infection, persistent instability, complications from anesthesia, pain, bleeding, blood clots (including deep venous thrombosis and pulmonary embolism), and potential for revision surgery. A website needed to mention at least one of the complications listed to be considered positive for offering potential surgical risk. Other Treatment Options An inclusion of nonsurgical treatment options such as physical therapy, orthotic treatment, neuromuscular training, and so on was assessed for each website. Eligibility Each website was evaluated for a discussion of the factors that determine a patient’s eligibility for ACL reconstruction. Patient chronologic age as well as activity level and physiologic age needed to be mentioned to be considered positive for including eligibility criteria. Unreal Claims Three specific claims of a bloodless surgery, guaranteed pain relief, and guaranteed return to play or sport were examined separately for each website. Rehabilitation The postoperative rehabilitation regimen needed to be briefly mentioned for each individual website. A detailed description of specific rehabilitation protocols did not need to be included to be considered positive. Graft Selection A brief discussion on graft material used for ACL reconstruction was evaluated for each website. A
1103
ACL RECONSTRUCTION INFORMATION ON THE INTERNET
detailed comparison of the advantages and disadvantages of each graft type did not need to be given. However, at a minimum websites needed to at least mention allograft as well as bone-patellar-bone and hamstring autograft options to be considered as having adequately discussed graft choices. Other Injuries Websites were also analyzed for discussion of other potential injuries encountered in a knee with an ACL injury. Potential injuries included, but were not limited to, collateral ligament injury, posterolateral corner injury, meniscal injury, growth plate disturbance, bone contusion, and avulsion fracture. Although a detailed description of additional injury or its potential consequences on ACL reconstruction did not need to be given, a brief discussion of the possibility of other concomitant knee injuries did need to be included. Peer-Reviewed References Each website had to reference or cite a specific peerreviewed source for information provided regarding ACL reconstruction to be considered positive.
Results The comprehensive analysis of the top 50 websites from each search engine (200 total) can be found in Fig 1 and Tables 1 and 2. In regard to authorship, it was found that 45 websites (22.5%) were affiliated with an academic institution; 71 (35.5%) were authored by private physicians or physician groups with no academic affiliation; 9 (4.5%) were industry sponsored; 12 (6%) were news bulletins or media reports; 39
Table 1. Ability to Contact Author and Initiate Injury Discussion Website
Ability to Contact
Ask Bing Yahoo Google Total 81 (N ¼ 200)
Discussion Treatment of Disorder Options Eligibility
24 25 18 29 21 34 18 32 (40.5%) 120 (60%) 62
13 14 17 18 (31%) 58
Related Injuries
13 9 14 12 12 11 19 8 (29%) 41 (20.5%)
(19.5%) were public education websites; and 24 (12%) were personal blogs (Fig 1). A summary of the contents from the websites found using each search engine are shown in Tables 1 and 2. Eighty-one websites (40.5%) provided either information or direct access for scheduling an appointment or consultation with the website author. A detailed description of an ACL injury was given in 60% of websites (n ¼ 120). However, only 41 websites (20.5%) made any mention of potential concomitant pathologic processes of the knee with ACL injury. Similarly, although 125 websites (62.5%) offered an explanation of the surgical technique of ACL reconstruction, only 30% of websites discussed eligibility for surgery or other nonsurgical treatment options. Roughly half of all websites did discuss graft selection and postoperative rehabilitationd55% and 48.5%, respectively. Only 52 websites (26%) included peer-reviewed references in the information provided. Fortunately, only 3 websites (1.5%) made false claims of painless and bloodless surgery and guaranteed return to play after ACL reconstruction. Table 3 is a summary of other orthopaedic studies investigating the authorship and content of websites for various other orthopaedic injuries or procedures compared with the present study on Internet information on ACL reconstruction. Of all the studies included in Table 3, the websites found for ACL reconstruction had a higher proportion of academic, private physician, and public education authorship. Compared with a study on cervical disk arthroplasty, the percentage of websites discussing ability to contact the author, procedure eligibility, and complications were fairly uniform.23 Few studies looked at the use of peer-reviewed references for citation, but percentages were similar in the 2 studies that focused on the ACL (26% v 34%).20
Discussion
Fig 1. Website authorship for anterior cruciate ligament (ACL) reconstruction. Values represent percentage of total number of websites (N ¼ 200).
There have been multiple studies that have already looked at information available on the Internet for different orthopaedic specialties, but none have looked specifically at ACL reconstruction.19-23 One recent study reviewed the quality of information of the Internet relating to ACL injury in female athletes.20 This study focused on the readability of website content and
1104
I. C. DUNCAN ET AL.
Table 2. Explanation of Surgery, Postoperative Regimen, and References Website
Surgical Technique
Graft Selection
Complications
Rehabilitation
Peer-Reviewed References
Ask Bing Yahoo Google Total (N ¼ 200)
34 28 31 32 125 (62.5%)
29 26 27 28 110 (55%)
14 18 15 13 60 (30%)
23 23 25 26 97 (48.5%)
14 12 11 15 52 (26%)
used a nonvalidated grading system to assess the quality of websites reviewed. Our methodology was similar to that of many of the previous studies.25 Like the referenced articles, a binary list of items was noted as present or absent, rather than using an arbitrary nonvalidated grading scale. The only notable exception is that the current study used different search engines (Google, Yahoo, Ask, Bing), which according to a current US reference, account for greater than 93% of web searches in the United States.27 Previous articles on this topic have looked at several general criteria that are necessary in a good website, such as discussion of the surgical technique and complications and then some specific criteria regarding the particular surgery. Ideally, a high-quality patient information website should be written by a credible source, discuss the disorder, give treatment options and guidelines for choosing a treatment (such as the indications for surgery), provide at least a brief overview of the surgical procedure, at least mention complications, and, in today’s era of evidence-based medicine, cite peer-reviewed references. The specific information for ACL reconstruction that we believed improved the quality of the website included any discussion of other related injuries, such as pathologic processes of the meniscus, other ligament injuries, fractures, or chondral injuries, graft options, and rehabilitation. Anecdotally, there seem to be more patients coming into the office with a desire to get a certain type of graft, so websites were evaluated on whether or not they reviewed this information. The websites covered this topic with varying bias and thoroughness, but to facilitate binary data collection, at least mentioning the topic was considered positive. Although single-bundle versus double-bundle ACL reconstruction has been
a popular topic in the literature over the past 10 years, there has been decreased focus on it lately, with more of a specific emphasis on the concept of anatomic ACL reconstruction. Therefore, single-bundle versus doublebundle reconstruction was not included as an important topic to have on a website. Subjectively, very few websites discussed single-bundle versus double-bundle reconstruction or anatomic positioning of the tunnels. Regarding authorship, there were a large number of credible sources, which was consistent with similar investigations: 45 websites (22.5%) were affiliated with an academic institution and 39 (19.5%) were public education websites. Seventy-one (35.5%) of the websites were authored by private physicians or physician groups with no academic affiliation. Although private physicians and their groups often have very welldesigned websites, they are often biased because they are trying to solicit business; therefore, they are often not the most complete, especially regarding complications. They were noted to make false claims such as guaranteeing bloodless or painless surgery and return to sports after reconstruction. There were also a number of even less credible sources: 24 websites (12%) were personal blogs, 12 (6%) were news bulletins or media reports, and 9 (4.5%) were industry sponsored. In general, the websites reviewed in this study that discuss ACL reconstruction contained less industry bias and overall better quality than the other articles previously written on this topic.19-23 Although industry-sponsored websites are often well designed and eye-catching, one cannot ignore the underlying goal to solicit the consumer. ACL reconstruction has the fewest industry-sponsored websites, with only 9 websites (4.5%) compared with studies from other specialties such as cervical arthroplasty (13%), sports medicine (46%), ACL injuries in female patients (6%),
Table 3. ACL Reconstruction Results Compared With Other Similar Studies on Information Available on the Internet Topic
Academic/Private/ Public Education
Industry Author
ACL reconstruction Sports medicine21 ACL injuries in female athletes22 Cervical herniation24 Cervical disk arthroplasty25*
79 46 28 66 58
4.5 48 6 18 13
Ability to Contact
Eligibility
Complications
Peer Reference
40.5
29
30
26 34
65
19
35
Scoring No Yes Yes Yes No
*Methodology similar to that used in the present study. Blank spaces were categories not evaluated in that study. Values represent percentages.
ACL RECONSTRUCTION INFORMATION ON THE INTERNET
and cervical herniation. Interestingly, searches dealing with the ACL had the fewest number of commercial websites. Industry understandably invests less in producing websites for ACL reconstruction because the ACL implants are significantly less expensive than arthroplasty implants, and there seems to be less patient interest in knowing which ACL fixation implants are used compared with arthroplasty implants. Curiously, in comparison with the other studies, cervical arthroplasty in general had 18% industry-sponsored websites, whereas the study that focused on 10 common sports diagnoses had the highest percentage of industry websites (approximately 48%). A new authorship category that has not been previously evaluated is personal blogs (12%), likely because the number of these sites has grown significantly over the past few years. Blogs are a great format for an individual, or even a small group, to communicate their experiences. The opportunity for anyone to have a public forum to share their thoughts and feelings has led to their popularity. However, because blogs are not censored or screened, too much emphasis can be placed on an individual’s good or bad experience, and skewed perspectives can be conveyed. Certainly, patients with bad outcomes are more vocal and may persuade patients to use a different surgeon or graft or avoid a similar decision because they had a bad experience, even if that bad outcome is extremely rare. The ideal health information website is written from a credible source that provides a point of contact, such as a phone number or e-mail address, to have questions answered or to make an appointment with the author. Only 81 websites (40.5%) in this study provided this information. Another positive for ACL reconstruction websites is that they discussed eligibility and complications and referenced peer-reviewed sources approximately as much or more than similar articles (Table 3). Detailed description of an ACL injury was given in 120 websites (60%), but only 41 websites (20.5%) made any mention of potential concomitant pathologic processes of the knee in ACL injury (Table 1). Eighty-one websites (40.5%) provided either information or direct access for scheduling an appointment or consultation with the website author (Table 1). Similarly, although 125 websites (62.5%) offered an explanation of the surgical technique of ACL reconstruction (Table 3), only 30% of websites discussed eligibility for surgery or other treatment options such as nonoperative treatment (Table 1). Roughly half of all websites discussed graft selection and postoperative rehabilitationd55% and 48.5%, respectively (Table 2). Only 52 websites (26%) included peer-reviewed references in the information provided (Table 2). Fortunately, only 3 websites (1.5%) made false claims of painless and bloodless surgery and guaranteed return to play after ACL reconstruction. One concerning trend in these websites
1105
is that they discuss the benefits of a surgical procedure but leave out even a mention of complications, giving the perception that patients are being coerced into surgical treatments. To be more thorough, authors of websites should put more emphasis on presenting balanced pros and cons of a procedure so that patients can make truly educated decisions. Interestingly, most of the similar orthopaedic studies looking at information available on the Internet used different methods. Most used different search engines depending on when they were written and based on which search engines were most commonly used at the time. Several used a grading scale, but they were often different nonvalidated scales. Others used the HONcode, which is created and maintained by the Health on the Net Foundation, a third-party organization that certifies websites that follow its principles when disseminating medical information.28 Another study focusing on distal radius fractures analyzed the quality of information found on the Internet depending on the search term used.23 Overall, when comparing the results from this study with those of similar studies, there is slightly higher quality information available to patients researching ACL reconstruction. There are fewer industry-sponsored websites, a similar percentage of peer-reviewed sources, and a lower percentage of misleading claims. Despite most websites discussing ACL ruptures and treatment options, only a quarter reference peerreviewed articles. There are also a large number of websites that are essentially anecdotal stories of a single patient’s experience (mostly blogs), which may not be predictive of common outcomes. Fortunately, only a small percentage (1.5%) of the websites contained false claims. The Internet can be a valuable asset for educating patients, but physicians should be familiar with which websites contain high-quality information, so we can guide our patients to the appropriate resources. Limitations A weakness of this study is the relatively subjective nature in which the websites were graded. Although there was a list of criteria for an ideal website, which was noted for all items, one of the various grading systems used in some of the other similar studies was not used in this study. The decision not to use a grading scale was made after reviewing the literature and concluding that none of the grading systems had been adequately validated. Few of the other similar orthopaedic articles reviewed used the same system but rather subjectively created different grading scales. In this study, our goal was simply to note whether or not important topics were discussed on a website. If a validated scale was used, further differentiation between the websites that simply mention an important topic
1106
I. C. DUNCAN ET AL.
versus those that have thorough coverage of the same topic might have been possible. Another critique is that multiple authors did not review each website unless clarification was needed. This decision was made for 2 reasons: (1) the list of items noted on each website was very simple and straightforward and little confusion or ambiguity existed and (2) having a larger sampling of websites was determined to be more important than having multiple evaluations of the same website, so effort was divided so that as many websites as possible were reviewed.
Conclusions The quality of Internet information available to patients searching for ACL reconstruction appears mixed. Although a majority of websites offer a discussion of the condition as well as a description of the surgical procedure, less than half touch on equally important issues such as patient eligibility for this procedure and the potential complications. Fortunately, a very small percentage made false claims, such as painless surgery and a guaranteed return to activity. With an increasing number of patients using the Internet for health care education, it is important for physicians to be aware of the quality of information available.
References 1. Kamath GV, Redfern JC, Greis PE, Burke RT. Revision anterior cruciate ligament reconstruction. Am J Sports Med 2011;39:199-217. 2. Friel NA, Chu CR. The role of ACL injury in the development of posttraumatic knee osteoarthritis. Clin Sports Med 2013;32:1. 3. Frobell RB, Roos EM, Roos HP, Ranstam J, Lohmander LS. A randomized trial of treatment for acute anterior cruciate ligament tears. N Engl J Med 2010;363: 331-342. 4. Love H, Mohtadi N. Treatment of ACL tears in active young adults. Clin J Sports Med 2011;21:73. 5. Maffulli N, Longo UG, Gougoulias N, Loppini M, Denaro V. Long-term health outcomes of youth sports injuries. Br J Sports Med 2010;44:21-25. 6. Ardern CL, Taylor NF, Feller JA, Webster KE. Return-tosport outcomes at 2 to 7 years after anterior cruciate ligament reconstruction surgery. Am J Sports Med 2012;40: 41-48. 7. Brophy RH, Schmitz L, Wright RW, et al. Return to play and future ACL injury risk after ACL reconstruction in soccer athletes from the Multicenter Orthopaedic Outcomes Network (MOON) Group. Am J Sports Med 2012;40:2523-2529. 8. Ellis HB, Matheny LM, Briggs KK, Pennock AT, Steadman JR. Outcomes and revision rate after bonepatellar tendon-bone allograft versus autograft anterior cruciate ligament reconstruction in patients aged 18 years or younger with closed physes. Arthroscopy 2012;27:11291134.
9. Warner SJ, Smith MV, Wright RW, Matava MJ, Brophy RH. Sport-specific outcomes after anterior cruciate ligament reconstruction. Arthroscopy 2011;27: 1129-1134. 10. Vyas D, Rabuck SJ, Harner CD. Allograft anterior cruciate ligament reconstruction: Indications, techniques, and outcomes. J Orthop Sports Phys Ther 2012;42:196-207. 11. Richardson ML, Wahl CJ. Detached femoral bone plug following anterior cruciate ligament repair. Radiology Case 2011;64. 12. Tjoumakaris FP, Herz-Brown AL, Legath-Bowers A, Sennett BJ, Bernstein J. Complications in brief: Anterior cruciate ligament reconstruction. Clin Orthop Relat Res 2012:1-7. 13. Benner RW, Shelbourne KD, Freeman H. Infections and patellar tendon ruptures after anterior cruciate ligament reconstruction: A comparison of ipsilateral and contralateral patellar tendon autografts. Am J Sports Med 2011;39:519-525. 14. Cheung SC, Allen CR, Gallo RA, Ma CB, Feeley BT. Patients’ attitudes and factors in their selection of grafts for anterior cruciate ligament reconstruction. Knee 2012;19:49-54. 15. Cohen RA, Stussman B. Health Information Technology Use Among Men and Women Aged 18-64: Early Release of Estimates from the National Health Interview Survey, January-June 2009. Atlanta, GA: National Center for Health Statistics, Centers for Disease Control; 2010. 16. Koh HS, In Y, Kong CG, Won HY, Kim KH, Lee JH. Factors affecting patients’ graft choice in anterior cruciate ligament reconstruction. Clin Orthop Surg 2010;2:69-75. Epub 2010 May 4. 17. Groves ND, Humphreys HW, Williams AJ, Jones A. Effect of informational internet web pages on patients’ decisionmaking: Randomised controlled trial regarding choice of spinal or general anaesthesia for orthopaedic surgery. Anaesthesia 2010;65:277-282. 18. Lichtenfeld LJ. Can the beast be tamed? The woeful tale of accurate health information on the Internet. Ann Surg Oncol 2012:1-2. 19. Starman JS, Gettys FK, Capo JA, Fleischli JE, Norton HJ, Karunakar MA. Quality and content of Internet-based information for ten common orthopaedic sports medicine diagnoses. J Bone Joint Surg Am 2010;92:1612-1618. 20. Gosselin MM, Mulcahey MK, Feller E, Hulstyn MJ. Examining internet resources on gender differences in ACL injuries: What patients are reading. Knee. 2012 Dec 20. [Epub ahead of print.] 21. Dy CJ, Taylor SA, Patel RM, Kitay A, Roberts TR, Daluiski A. The effect of search term on the quality and accuracy of online information regarding distal radius fractures. J Hand Surg Am 2012;37:1882-1887. 22. Morr S, Shanti N, Carrer A, Kubeck J, Gerling MC. Quality of information concerning cervical disc herniation on the Internet. Spine J 2010;4:350-354. 23. Qureshi SA, Koehler SM, Lin JD, Bird J, Garcia RM, Hecht AC. An evaluation of information on the Internet of a new device: The cervical artificial disc replacement. Spine 2012;37:881-883. 24. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of
ACL RECONSTRUCTION INFORMATION ON THE INTERNET orthopaedic information on the Internet. The case of carpal tunnel syndrome. J Bone Joint Surg Am 2000; 82-A:1540-1543. 25. Sambandam SN, Ramasamy V, Priyanka P, Ilango B. Quality analysis of patient information about knee arthroscopy on the World Wide Web. Arthroscopy 2007;23: 509-513.e2.
1107
26. Bernard L, Parham L. The impact of web 2.0 on the doctorpatient relationship. J Law Med Ethics 2010;38:17-26. 27. Purcell K, Brenner J, Rainie L. Search Engine Use 2012. Washington, DC: Pew Internet & American Life Project, 2012. 28. Health on the Net Foundation. Available at: http://www .hon.ch/home1.html. Accessed January 23, 2012.