- Email: [email protected]
Quality Analysis of Patient Information About Knee Arthroscopy on the World Wide Web
Quality Analysis of Patient Information About Knee Arthroscopy on the World Wide Web Senthil Nathan Sambandam, M.R.C.S., M.S., Vijayaraj Ramasamy, M.R.C.S., Priyanka Priyanka, M.B.B.S., and Balakrishnan Ilango, F.R.C.S.(Orth.)
Purpose: This study was designed to ascertain the quality of patient information available on the World Wide Web on the topic of knee arthroscopy. Methods: For the purpose of quality analysis, we used a pool of 232 search results obtained from 7 different search engines. We used a modified assessment questionnaire to assess the quality of these Web sites. This questionnaire was developed based on similar studies evaluating Web site quality and includes items on illustrations, accessibility, availability, accountability, and content of the Web site. We also compared results obtained with different search engines and tried to establish the best possible search strategy to attain the most relevant, authentic, and adequate information with minimum time consumption. For this purpose, we first compared 100 search results from the single most commonly used search engine (AltaVista) with the pooled sample containing 20 search results from each of the 7 different search engines. The search engines used were metasearch (Copernic and Mamma), general search (Google, AltaVista, and Yahoo), and health topic–related search engines (MedHunt and Healthfinder). The phrase “knee arthroscopy” was used as the search terminology. Results: Excluding the repetitions, there were 117 Web sites available for quality analysis. These sites were analyzed for accessibility, relevance, authenticity, adequacy, and accountability by use of a specially designed questionnaire. Our analysis showed that most of the sites providing patient information on knee arthroscopy contained outdated information, were inadequate, and were not accountable. Only 16 sites were found to be providing reasonably good patient information and hence can be recommended to patients. Understandably, most of these sites were from nonprofit organizations and educational institutions. Furthermore, our study revealed that using multiple search engines increases patients’ chances of obtaining more relevant information rather than using a single search engine. Conclusions: Our study shows the difficulties encountered by patients in obtaining information regarding knee arthroscopy and highlights the duty of knee surgeons in helping patients to identify the relevant and authentic information in the most efficient manner from the World Wide Web. Clinical Relevance: This study highlights the importance of the role of orthopaedic surgeons in helping their patients to identify the best possible information on the World Wide Web. Key Words: Patient information—Internet—World Wide Web—Quality analysis.
T
From Wrightington Hospital (S.N.S.) and Royal Albert Edward Infirmary (P.P.), Wigan, and Fairfield General Hospital (V.R.), Bury, England. The authors report no conflict of interest. Research was performed at Fairfield General Hospital, Bury, England. Address correspondence and reprint requests to Senthil Nathan Sambandam, M.R.C.S., M.S., 24 Dobson Close, Appley Bridge, Wigan WN69ES, England. E-mail: [email protected] © 2007 by the Arthroscopy Association of North America 0749-8063/07/2305-6340$32.00/0 doi:10.1016/j.arthro.2006.12.007 Note: To access the supplementary tables accompanying this report, visit the May issue of Arthroscopy at www. arthroscopyjournal.org.
he Internet has become a popular source of information. It was initially predicted in 1998 that there would be more than 720 million people all over world using the Internet by the end of 2005,1,2 and a recent report shows that more than 1 billion people use the Internet worldwide.3 This exponential increase in Web usage has made it possible for patients to obtain medical information that previously would have been difficult for anyone other than a physician or librarian to obtain. More than half of the individuals with Internet access go online to seek health-related information at least once per month.4 Although widespread Internet access has allowed the public to obtain a vast amount of medical information, there are very few rules, guidelines, and governing bodies to check the
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol 23, No 5 (May), 2007: pp 509-513
509
510
S. N. SAMBANDAM ET AL.
validity and reliability of Web information.4-7 This factor may lead to patients acquiring inadequate and inaccurate information that may affect treatment decisions, which is a matter of concern.8-10 A few reports have been published regarding the magnitude of this problem affecting orthopaedic patients seeking health information. Li and Irvin assessed the nature and quality of back pain–related information on the World Wide Web and found that the quality varied considerably, resulting in difficulties for patients to find useful information.11 Similarly, Beredjiklian et al.4 and Nogler et al.12 analyzed the quality of patient information–related Web resources on hand disorders and ankle and foot disorders, respectively. Beredjiklian et al. concluded that the information about carpal tunnel syndrome on the Internet is of limited quality and poor informational value. Nogler et al. assessed the patient information on foot and ankle disorders and concluded that more than 40% of the Web sites were commercial and advertisement-oriented. Surgical patients use the Internet to help in decision making and to gain more knowledge about the surgical treatment offered to them, as well as to prepare themselves for the proposed procedure.13-15 Knee arthroscopy is a commonly performed procedure, and currently, there is no exact information about the quality of Web resources providing patient information and there are also no recommended search strategies to obtain the best possible and relevant information needed. Hence we designed this study to determine the quality of patient information available on the World Wide Web on the topic of knee arthroscopy. We also compared results obtained with different search engines and tried to establish the best possible search strategy to obtain the most relevant and authentic information with minimum time consumption. Our hypotheses were that the quality of information available on the Web would be poor and that different search strategies would provide a different quantity and quality of information. METHODS Quality of Web Resources We used multiple search engines for the purpose of this study: metasearch (which does not create its own database of information but instead searches those of other engines), general search (which creates its own database of information), and health topic–related search engines. The metasearch engines used were Copernic and Mamma; the general search engines
used were Google, AltaVista, and Yahoo; and the health topic–related search engines were MedHunt and Healthfinder. “Knee arthroscopy” was used as the search terminology. The topic was typed in each individual search engine as a phrase. For the assessment of quality, we included the first 100 URLs from AltaVista and the first 20 search results from all other search engines. We analyzed the links as well. The first 2 links were included for quality analysis if there were more than 2 links. This yielded 232 URLs. Of these 232 URLs, those that were duplicates were excluded. Overall, there were 117 Web sites that were available for Web resource quality analysis. We performed quality analysis by using a modified quality assessment questionnaire that was designed based on methods used in previous studies (Table 1, online only, available at www.arthroscopyjournal.org).6,13,16-18 Each Web site was analyzed individually by 2 blinded observers using the assessment questionnaire. Each item on the questionnaire was clearly defined to the observer. For example, adequacy of content was analyzed under 6 subheadings and given a score of 0, 3, or 5. A score of 0 was given if there was no information at all on the subheading, a score of 3 was assigned if just some mention about the subheading was made, and a score 5 was given if the subheading was clearly explained. Overall, the information was considered just adequate if a Web site scored more than 3 in all subheadings under the adequacy analysis. Similarly, authenticity was defined as mentioned in Table 1 (online only, available at www.arthroscopyjournal. org). The final judgment of the observers was that the Web site was recommendable if it was an up-to-date, nonprofit, non-advertisement Web site and scored at least 3 on all subheadings considered under adequacy and at least 5 for the authenticity subheading. The mean interobserver reliability of the questionnaire was 0.87 (range, 0.62 to 0.98). Items such as adequacy of content (0.62) and authenticity of content (0.68) had an interobserver reliability of less than 0.70. Items such as overall assessment (0.94) and whether the observer would recommend the Web site (0.98) had a very high interobserver reliability. For those items with varied responses for each observer, a consensus was reached by discussion with the senior author, who was blinded with regard to the nature of the study. Search Strategy Evaluation Different strategies can be used to search for information regarding knee arthroscopy on the Web. Some patients may prefer to use a single search engine to
KNEE ARTHROSCOPY ON THE WORLD WIDE WEB make it time-efficient and less complex (strategy A).13 Others prefer to use the same search terminology in various search engines to obtain accurate information (strategy B).1,4,6,14 We tried to determine the better of these 2 strategies (A or B) in obtaining the required information. For strategy A, we included the first 100 search results obtained from a single commonly used search engine19-22—AltaVista. For strategy B, using the same search terminology, we used 3 general, 2 metasearch, and 2 health topic–related search engines. The top 20 results of each of the 7 search engines were included. We reviewed the search results obtained by both strategies for good-quality Web sites that can be recommended to patients. We report the odds ratios of obtaining a good-quality Web site with each search strategy. Statistics We used SPSS software (version 13; SPSS, Chicago, IL) for statistical analysis. We used descriptive statistics for quality analysis of Web sites. We used odds ratios to compare the efficiency of different search strategies and different search engines. RESULTS Quality Analysis Of 232 URLs, 117 were included for quality analysis. The remaining 115 URLs were excluded because they were repetitions. Of these, 10 sites were password-protected, no longer existed, or were not found. Information Update The exact time at which the site was last updated was available explicitly on only 27 sites. Among those 27, the median time since updating was 12 months (range, 1 to 84 months). Illustrations, Accessibility, Availability, Accountability, and Advertisement Only 23 URLs had some form of illustrations to help patients understand the information. Sixty-two sites were accountable for the information given by explicitly providing the ownership information of the Web site. Only 39 sites targeted mainly people with knee problems; 31 were for the general public. Of the remaining 47 Web sites, 22 were for health care providers. Of the 70 Web sites providing patient information about knee arthroscopy, only 5 were from
511
educational organizations and 4 were from professional organizations. Forty-seven Web sites were owned by for-profit organizations. Promotional messages could be seen on 40 Web sites. Most of these were promoting either their service (21 [30%]) or some product (9 [12%]). Content Analysis Of the 70 Web sites providing patient information about knee arthroscopy, we found that 62 URLs were inadequate and 8 were adequate. When individual components of information were considered for quality analysis, adequate information was found in the range of 1.5% to 12% of the Web sites for each of the components (diagnosis, 4; procedure, 7; alternative options, 4; postoperative protocol, 9; complications, 7; prognosis, 1). On only 3 Web sites was the information provided substantiated with some form of reference and hence was considered authentic. A Web site was considered to be a good site for patient information if the information provided was adequate in all aspects and also authentic. On this basis, we found that more than 75% of the sites were of poor quality. Only 1 Web site was found to have met all of the requirements to be considered as an ideal site to be recommended to patients. The remaining 16 sites were inadequate in some aspects but still can be recommended to patients as reliable and providing reasonably adequate information. Search Strategy Evaluation Pooling the data from both search strategies, 16 URLs were identified as recommendable. Strategy A yielded only 8 of these 16 Web sites (50% yield). Strategy B yielded 13 of these 16 URLs (80% yield). Hence quality analysis of Web sites obtained with each search strategy revealed that only 8 of 110 sites in AltaVista and 13 of 148 sites with the combined search engine strategy were found to be recommendable. Hence the odds ratio of obtaining adequate and authentic information by use of multiple search engines was 1.2 compared with the use of a single search engine strategy. DISCUSSION In our study we analyzed a pooled sample of 232 Web sites obtained by different search strategies. Excluding the repetitions, there were 117 sites for the final assessment. Only 70 of these 117 were found to provide some form of patient information and hence
512
S. N. SAMBANDAM ET AL.
were considered relevant. Among the 70 relevant sites, only 16 were found to be recommendable based on the adequacy, authenticity, and accountability of information provided. Hence our analysis shows that an enormous amount of information is available for patients on the Internet but at the same time raises concerns about the quality of information available and the difficulty encountered in obtaining the relevant information. With the increasing role of the Internet as a source of medical information for patients, it can be assumed that they can be misinformed or distracted with the valuable information being obscured by irrelevant, misleading information. The Internet continues to be the major source of health-related information. With the pace at which Web sites are proliferating, the quality of healthrelated information provided on the Web has become a major issue of concern in various specialties. In 1997 Sikorski and Peters23 reviewed a few Web sites on cancer treatments and tried to justify the reason for recommending those sites to patients. In 2001 Corpron and Lelli14 searched the Internet for patient information on intersex anomalies, especially ambiguous genitalia. They used 6 different general search engines and included the first 50 search results from each search engine for quality analysis. This is in contrast to our selection strategy because we used both metasearch engines and health search engines in addition to 3 commonly used general search engines. Their analysis revealed that of the 300 Web sites analyzed, only 45 were found to provide patient information, of which only 8 were unique sites and merely 5 Web sites were found to be recommendable (1.6%). Similar studies analyzing the quality of patient information available on the Internet have been performed on various orthopaedic topics.1,2,6,10,12,16,24-30 Those included were mostly some form of orthopaedic condition, such as ankle sprain, backache, carpal tunnel syndrome, hallux valgus, and congenital talipes equinovarus. However, patients very often seek Web information when they are offered different treatment options and have to select an option based on the information given to them by the health care provider. The studies exclusively analyzing the quality of information on orthopaedic procedures are very few.1,2 Mabrey1 in 2000 analyzed the Web resources for patient-oriented information on total hip replacement. He used different key words such as “hip replacement,” “total hip replacement,” and “hip surgery.” He used only metasearch engines, namely GO and AskJeeves. Instead of reporting the overall quality of information obtained, he performed a descriptive anal-
ysis of a few Web sites and made some conclusions based on those Web sites. Our study is unique in several aspects. We attempted to obtain the best sample of Web sites for quality analysis. To fulfill that objective, we used not just general search engines but also metasearch and health search engines. Furthermore, to obtain the most representative Web sites, we included the first 100 Web sites with all links from the most commonly used Web site (AltaVista)13 and the first 20 Web sites from all other search engines. This is in contrast to most of the previous studies, which either used a single search engine or used the first few Web sites from different search engines. Our study also designates 16 sites that can be recommended to patients (Table 2, online only, available at www.arthroscopyjournal.org). This is in contrast to some of the previous studies that just concluded that very few Web sites provide reliable information but failed to mention the URL addresses of those sites. We also tried to establish the most efficient search strategy in our study, and we found that using the first few search results of multiple search engines marginally increases the chances of obtaining the best possible information as compared with surfing all of the results of a single search engine. This is similar to the results obtained by Lawrence and Giles,31 who concluded in their study that combining results from different search engines yields 3.5 times more documents as compared with using a single search engine. However, we understand that our study has a few limitations. We based our study on Web sites provided by the most commonly used search engines and not on a roster of Web sites listed according to the overall usage by people interested in knee arthroscopy. We used a custom-made questionnaire based on the questionnaires used in similar studies. Because our questionnaire was not validated and its reliability was not checked, its use for similar purposes in other studies is difficult and debatable. Furthermore, because of the dynamic nature of some Web sites and the crosssectional nature of our study, it is difficult to generalize the information for future usage. In addition, our investigation was limited to knee arthroscopy and cannot be generalized to other orthopaedic procedures. CONCLUSIONS Our study shows that the Internet offers a wide variety of information about knee arthroscopy. However, indeed, the quality of information is limited. Most sites
KNEE ARTHROSCOPY ON THE WORLD WIDE WEB were not confined to conventional information, and even those that presented this did not describe it completely. Despite efforts to regulate the quality of information available on the Internet for patients, it is evident that the quality of information is still very poor. Even though it is the moral responsibility of the Web site owners to provide adequate, relevant, and authentic patient information without any bias, health care professionals are also responsible for helping their patients access the right health information. Health care professionals should suggest a few URL addresses providing reliable patient information. A few Web sites provide relevant patient information, such as that of the American Academy of Orthopaedic Surgeons (www.aaos.org), which lists information on more than 400 orthopaedic conditions, and can be recommended to patients. Orthopaedic surgeons can also provide in writing the appropriate search terminology and best search strategy31 to those patients who want to extend their search beyond those sites recommended by their surgeons. Furthermore, they should also help their patients identify the credibility of a Web site by providing them with the criteria suggested by various authors6,13,16-18 and governing bodies.5,7,32 REFERENCES 1. Mabrey JD. Survey of patient-oriented total hip replacement information on the World Wide Web. Clin Orthop Relat Res 2000:106-113. 2. Mathur S, Shanti N, Brkaric M, et al. Surfing for scoliosis: The quality of information available on the Internet. Spine 2005; 30:2695-700. 3. Miniwatts Marketing Group. Internet usage statistics—The big picture. World Internet users and population stats. 2006. Available from: URL: http://www.internetworldstats.com/stats.htm. Accessed December 21, 2005. 4. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome. J Bone Joint Surg Am 2000;82:1540-1543. 5. Winker MA, Flanagin A, Chi-Lum B, et al. Guidelines for medical and health information sites on the internet: Principles governing AMA web sites. American Medical Association. JAMA 2000;283:1600-1606. 6. Smart JM, Burling D. Radiology and the internet: A systematic review of patient information resources. Clin Radiol 2001;56: 867-870. 7. Scherrer J. Health On Net Foundation homepage. 1999. www. hon.ch/home.html 8. Bessell TL, McDonald S, Silagy CA, Anderson JN, Hiller JE, Sansom LN. Do Internet interventions for consumers cause more harm than good? A systematic review. Health Expect 2002;5:28-37. 9. Crocco AG, Villasis-Keever M, Jadad AR. Analysis of cases of harm associated with use of health information on the internet. JAMA 2002;287:2869-2871. 10. Shuyler KS, Knight KM. What are patients seeking when they turn to the Internet? Qualitative content analysis of questions asked by visitors to an orthopaedics Web site. J Med Internet Res 2003;5:e24.
513
11. Li L, Irvin E, Guzman J, Bombardier C. Surfing for back pain patients: The nature and quality of back pain information on the Internet. Spine 2001;26:545-557. 12. Nogler M, Wimmer C, Mayr E, Ofner D. The efficacy of using search engines in procuring information about orthopaedic foot and ankle problems from the World Wide Web. Foot Ankle Int 1999;20:322-325. 13. Chen LE, Minkes RK, Langer JC. Pediatric surgery on the Internet: Is the truth out there? J Pediatr Surg 2000;35:11791182. 14. Corpron CA, Lelli JL Jr. Evaluation of pediatric surgery information on the Internet. J Pediatr Surg 2001;36:1187-1189. 15. Murphy MA, Joyce WP. Information for surgical patients: Implications of the World Wide Web. Eur J Surg 2001;167: 728-733. 16. Aslam N, Bowyer D, Wainwright A, Theologis T, Benson M. Evaluation of Internet use by paediatric orthopaedic outpatients and the quality of information available. J Pediatr Orthop B 2005;14:129-133. 17. Simpson P, Oliver CW. Searching the internet for orthopaedic knowledge. J Bone Joint Surg Br 2004;86:1105-1107. 18. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor—Let the reader and viewer beware. JAMA 1997;277:1244-1245. 19. Burneo JG. An evaluation of the quality of epilepsy education on the Canadian World Wide Web. Epilepsy Behav 2006;8: 299-302. 20. Gimenez-Perez G, Caixas A, Gimenez-Palop O, GonzalezClemente JM, Mauricio D. Dissemination of “patient-oriented evidence that matters” on the Internet: The case of type 2 diabetes treatment. Diabet Med 2005;22:688-892. 21. Martin-Facklam M, Kostrzewa M, Martin P, Haefeli WE. Quality of drug information on the World Wide Web and strategies to improve pages with poor information quality. An intervention study on pages about sildenafil. Br J Clin Pharmacol 2004;57:80-85. 22. Styra R. The Internet’s impact on the practice of psychiatry. Can J Psychiatry 2004;49:5-11. 23. Sikorski R, Peters R. Oncology ASAP. Where to find reliable cancer information on the Internet. JAMA 1997;277:14311432. 24. Sproule JA, Tansey C, Burns B, Fenelon G. The Web: Friend or foe of the hand surgeon? Hand Surg 2003;8:181-185. 25. Groot D, ter Riet G, Khan KS, Misso K. Comparison of search strategies and quality of medical information of the Internet: A study relating to ankle sprain. Injury 2001;32:473-476. 26. Moshirfar A, Campbell JT, Khasraghi FA, Wenz JF Sr. Evaluating the quality of Internet-derived information on plantar fasciitis. Clin Orthop Relat Res 2004:60-63. 27. Greene DL, Appel AJ, Reinert SE, Palumbo MA. Lumbar disc herniation: Evaluation of information on the internet. Spine 2005;30:826-829. 28. McClune T, Burton AK, Waddell G. Whiplash associated disorders: A review of the literature to guide patient information and advice. Emerg Med J 2002;19:499-506. 29. Macario A, Schilling P, Rubio R, Bhalla A, Goodman S. What questions do patients undergoing lower extremity joint replacement surgery have? BMC Health Serv Res 2003;3:11. 30. Beall MS III, Golladay GJ, Greenfield ML, Hensinger RN, Biermann JS. Use of the Internet by pediatric orthopaedic outpatients. J Pediatr Orthop 2002;22:261-264. 31. Lawrence S, Giles CL. Searching the world wide web. Science 1998;280:98-100. 32. Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med 1998;28:603-610.
KNEE ARTHROSCOPY ON THE WORLD WIDE WEB TABLE 1.
513.e1
Assessment Tool for Web Site Quality Analysis
Search engine General
Metasearch Health topic Web site description URL address Date of search Position in search result Accessibility
Illustrations and pictures Duplicate Quality Date information was updated If yes, then how old? Authorship information availability If yes If yes
Promotional message What is being promoted? Target audience information Availability Type of target audience
Adequacy of content (0, no information; 3, some information; 5, adequate information) Diagnosis Procedure Alternate options Postoperative protocol Complications Prognosis Total Authenticity of content (0, no referencing at all; 3, good-quality Web sites referenced; 5, textbook referenced; 10, textbook and scientific articles referenced) Overall assessment Would you recommend the site?
Google AltaVista Yahoo Mamma Copernic MedHunt Healthfinder
Time search
Download time
Easy Page not found No longer exists Password-protected Y/N Y/N Y/N Y/N Easy Difficult General public Educational institution Instrument company Club Professional organization For-profit organization Other Y/N Product/technique/service/ advertisement Y/N Easy General public Targeted workers Health care providers Insurers Employers People with knee problems
Difficult
0 0 0 0 0 0
3 3 3 3 3 3
5 5 5 5 5 5
0
3
5
Poor Y/N
Fair
Good
10
513.e2
S. N. SAMBANDAM ET AL. TABLE 2.
Recommendable Web Sites
Exact URL address of Web page http://www.vh.org/adult/patient/orthopaedics/kneearthroscopy/ http://hcd2.bupa.co.uk/fact_sheets/html/arthroscopy.html http://www.nlm.nih.gov/medlineplus/tutorials/kneearthroscopy/ htm/index.htm http://surgerydoor.co.uk http://patienteducation.upmc.com/Pdf/KneeArthroscopy.pdf http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/ Arthroscopy?open http://www.priory.com/vet/stifle1.htm http://www.kneeguru.co.uk/kneegeeks/7922698801.html http://www.patient.co.uk/showdoc/27000470 http://www.themediweb.net/orthopaedics/infosheets_html/ inpatients/Knee.html http://www.yorkshirekneeclinic.co.uk/acl-reconstructionsurgery.htm http://www.orthopaedics.com/ortho/knees.html http://www.orthos.co.nz/kneejoint.html http://www.international-medical-tourism.com/arthroscopywith-ligament-reconstruction.html http://www.aaos.org.html http://www.niams.nih.gov/hi/topics/kneeprobs/kneeqa.html Strategy A http://www.vh.org/adult/patient/orthopaedics/kneearthroscopy/ http://hcd2.bupa.co.uk/fact_sheets/html/arthroscopy.html http://www.nlm.nih.gov/medlineplus/tutorials/kneearthroscopy/ htm/index.htm http://surgerydoor.co.uk http://patienteducation.upmc.com/Pdf/KneeArthroscopy.pdf http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/ Arthroscopy?open http://www.priory.com/vet/stifle1.htm http://www.kneeguru.co.uk/kneegeeks/7922698801.html http://www.orthopaedics.com/ortho/knees.html http://www.orthos.co.nz/kneejoint.html http://www.international-medical-tourism.com/arthroscopywith-ligament-reconstruction.html http://www.aaos.org.html http://www.niams.nih.gov/hi/topics/kneeprobs/kneeqa.html Strategy B http://hcd2.bupa.co.uk/fact_sheets/html/arthroscopy.html http://www.nlm.nih.gov/medlineplus/tutorials/kneearthroscopy/ htm/index.html http://www.priory.com/vet/stifle1.html http://www.kneeguru.co.uk/kneegeeks/7922698801.html http://surgerydoor.co.uk http://www.patient.co.uk/showdoc/27000470 http://www.themediweb.net/orthopaedics/infosheets_html/ inpatients/Knee.html http://www.yorkshirekneeclinic.co.uk/acl-reconstructionsurgery.htm http://www.vh.org/adult/patient/orthopaedics/kneearthroscopy/
Purpose: This study was designed to ascertain the quality of patient information available on the World Wide Web on the topic of knee arthroscopy. Methods: For the purpose of quality analysis, we used a pool of 232 search results obtained from 7 different search engines. We used a modified assessment questionnaire to assess the quality of these Web sites. This questionnaire was developed based on similar studies evaluating Web site quality and includes items on illustrations, accessibility, availability, accountability, and content of the Web site. We also compared results obtained with different search engines and tried to establish the best possible search strategy to attain the most relevant, authentic, and adequate information with minimum time consumption. For this purpose, we first compared 100 search results from the single most commonly used search engine (AltaVista) with the pooled sample containing 20 search results from each of the 7 different search engines. The search engines used were metasearch (Copernic and Mamma), general search (Google, AltaVista, and Yahoo), and health topic–related search engines (MedHunt and Healthfinder). The phrase “knee arthroscopy” was used as the search terminology. Results: Excluding the repetitions, there were 117 Web sites available for quality analysis. These sites were analyzed for accessibility, relevance, authenticity, adequacy, and accountability by use of a specially designed questionnaire. Our analysis showed that most of the sites providing patient information on knee arthroscopy contained outdated information, were inadequate, and were not accountable. Only 16 sites were found to be providing reasonably good patient information and hence can be recommended to patients. Understandably, most of these sites were from nonprofit organizations and educational institutions. Furthermore, our study revealed that using multiple search engines increases patients’ chances of obtaining more relevant information rather than using a single search engine. Conclusions: Our study shows the difficulties encountered by patients in obtaining information regarding knee arthroscopy and highlights the duty of knee surgeons in helping patients to identify the relevant and authentic information in the most efficient manner from the World Wide Web. Clinical Relevance: This study highlights the importance of the role of orthopaedic surgeons in helping their patients to identify the best possible information on the World Wide Web. Key Words: Patient information—Internet—World Wide Web—Quality analysis.
T
From Wrightington Hospital (S.N.S.) and Royal Albert Edward Infirmary (P.P.), Wigan, and Fairfield General Hospital (V.R.), Bury, England. The authors report no conflict of interest. Research was performed at Fairfield General Hospital, Bury, England. Address correspondence and reprint requests to Senthil Nathan Sambandam, M.R.C.S., M.S., 24 Dobson Close, Appley Bridge, Wigan WN69ES, England. E-mail: [email protected] © 2007 by the Arthroscopy Association of North America 0749-8063/07/2305-6340$32.00/0 doi:10.1016/j.arthro.2006.12.007 Note: To access the supplementary tables accompanying this report, visit the May issue of Arthroscopy at www. arthroscopyjournal.org.
he Internet has become a popular source of information. It was initially predicted in 1998 that there would be more than 720 million people all over world using the Internet by the end of 2005,1,2 and a recent report shows that more than 1 billion people use the Internet worldwide.3 This exponential increase in Web usage has made it possible for patients to obtain medical information that previously would have been difficult for anyone other than a physician or librarian to obtain. More than half of the individuals with Internet access go online to seek health-related information at least once per month.4 Although widespread Internet access has allowed the public to obtain a vast amount of medical information, there are very few rules, guidelines, and governing bodies to check the
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol 23, No 5 (May), 2007: pp 509-513
509
510
S. N. SAMBANDAM ET AL.
validity and reliability of Web information.4-7 This factor may lead to patients acquiring inadequate and inaccurate information that may affect treatment decisions, which is a matter of concern.8-10 A few reports have been published regarding the magnitude of this problem affecting orthopaedic patients seeking health information. Li and Irvin assessed the nature and quality of back pain–related information on the World Wide Web and found that the quality varied considerably, resulting in difficulties for patients to find useful information.11 Similarly, Beredjiklian et al.4 and Nogler et al.12 analyzed the quality of patient information–related Web resources on hand disorders and ankle and foot disorders, respectively. Beredjiklian et al. concluded that the information about carpal tunnel syndrome on the Internet is of limited quality and poor informational value. Nogler et al. assessed the patient information on foot and ankle disorders and concluded that more than 40% of the Web sites were commercial and advertisement-oriented. Surgical patients use the Internet to help in decision making and to gain more knowledge about the surgical treatment offered to them, as well as to prepare themselves for the proposed procedure.13-15 Knee arthroscopy is a commonly performed procedure, and currently, there is no exact information about the quality of Web resources providing patient information and there are also no recommended search strategies to obtain the best possible and relevant information needed. Hence we designed this study to determine the quality of patient information available on the World Wide Web on the topic of knee arthroscopy. We also compared results obtained with different search engines and tried to establish the best possible search strategy to obtain the most relevant and authentic information with minimum time consumption. Our hypotheses were that the quality of information available on the Web would be poor and that different search strategies would provide a different quantity and quality of information. METHODS Quality of Web Resources We used multiple search engines for the purpose of this study: metasearch (which does not create its own database of information but instead searches those of other engines), general search (which creates its own database of information), and health topic–related search engines. The metasearch engines used were Copernic and Mamma; the general search engines
used were Google, AltaVista, and Yahoo; and the health topic–related search engines were MedHunt and Healthfinder. “Knee arthroscopy” was used as the search terminology. The topic was typed in each individual search engine as a phrase. For the assessment of quality, we included the first 100 URLs from AltaVista and the first 20 search results from all other search engines. We analyzed the links as well. The first 2 links were included for quality analysis if there were more than 2 links. This yielded 232 URLs. Of these 232 URLs, those that were duplicates were excluded. Overall, there were 117 Web sites that were available for Web resource quality analysis. We performed quality analysis by using a modified quality assessment questionnaire that was designed based on methods used in previous studies (Table 1, online only, available at www.arthroscopyjournal.org).6,13,16-18 Each Web site was analyzed individually by 2 blinded observers using the assessment questionnaire. Each item on the questionnaire was clearly defined to the observer. For example, adequacy of content was analyzed under 6 subheadings and given a score of 0, 3, or 5. A score of 0 was given if there was no information at all on the subheading, a score of 3 was assigned if just some mention about the subheading was made, and a score 5 was given if the subheading was clearly explained. Overall, the information was considered just adequate if a Web site scored more than 3 in all subheadings under the adequacy analysis. Similarly, authenticity was defined as mentioned in Table 1 (online only, available at www.arthroscopyjournal. org). The final judgment of the observers was that the Web site was recommendable if it was an up-to-date, nonprofit, non-advertisement Web site and scored at least 3 on all subheadings considered under adequacy and at least 5 for the authenticity subheading. The mean interobserver reliability of the questionnaire was 0.87 (range, 0.62 to 0.98). Items such as adequacy of content (0.62) and authenticity of content (0.68) had an interobserver reliability of less than 0.70. Items such as overall assessment (0.94) and whether the observer would recommend the Web site (0.98) had a very high interobserver reliability. For those items with varied responses for each observer, a consensus was reached by discussion with the senior author, who was blinded with regard to the nature of the study. Search Strategy Evaluation Different strategies can be used to search for information regarding knee arthroscopy on the Web. Some patients may prefer to use a single search engine to
KNEE ARTHROSCOPY ON THE WORLD WIDE WEB make it time-efficient and less complex (strategy A).13 Others prefer to use the same search terminology in various search engines to obtain accurate information (strategy B).1,4,6,14 We tried to determine the better of these 2 strategies (A or B) in obtaining the required information. For strategy A, we included the first 100 search results obtained from a single commonly used search engine19-22—AltaVista. For strategy B, using the same search terminology, we used 3 general, 2 metasearch, and 2 health topic–related search engines. The top 20 results of each of the 7 search engines were included. We reviewed the search results obtained by both strategies for good-quality Web sites that can be recommended to patients. We report the odds ratios of obtaining a good-quality Web site with each search strategy. Statistics We used SPSS software (version 13; SPSS, Chicago, IL) for statistical analysis. We used descriptive statistics for quality analysis of Web sites. We used odds ratios to compare the efficiency of different search strategies and different search engines. RESULTS Quality Analysis Of 232 URLs, 117 were included for quality analysis. The remaining 115 URLs were excluded because they were repetitions. Of these, 10 sites were password-protected, no longer existed, or were not found. Information Update The exact time at which the site was last updated was available explicitly on only 27 sites. Among those 27, the median time since updating was 12 months (range, 1 to 84 months). Illustrations, Accessibility, Availability, Accountability, and Advertisement Only 23 URLs had some form of illustrations to help patients understand the information. Sixty-two sites were accountable for the information given by explicitly providing the ownership information of the Web site. Only 39 sites targeted mainly people with knee problems; 31 were for the general public. Of the remaining 47 Web sites, 22 were for health care providers. Of the 70 Web sites providing patient information about knee arthroscopy, only 5 were from
511
educational organizations and 4 were from professional organizations. Forty-seven Web sites were owned by for-profit organizations. Promotional messages could be seen on 40 Web sites. Most of these were promoting either their service (21 [30%]) or some product (9 [12%]). Content Analysis Of the 70 Web sites providing patient information about knee arthroscopy, we found that 62 URLs were inadequate and 8 were adequate. When individual components of information were considered for quality analysis, adequate information was found in the range of 1.5% to 12% of the Web sites for each of the components (diagnosis, 4; procedure, 7; alternative options, 4; postoperative protocol, 9; complications, 7; prognosis, 1). On only 3 Web sites was the information provided substantiated with some form of reference and hence was considered authentic. A Web site was considered to be a good site for patient information if the information provided was adequate in all aspects and also authentic. On this basis, we found that more than 75% of the sites were of poor quality. Only 1 Web site was found to have met all of the requirements to be considered as an ideal site to be recommended to patients. The remaining 16 sites were inadequate in some aspects but still can be recommended to patients as reliable and providing reasonably adequate information. Search Strategy Evaluation Pooling the data from both search strategies, 16 URLs were identified as recommendable. Strategy A yielded only 8 of these 16 Web sites (50% yield). Strategy B yielded 13 of these 16 URLs (80% yield). Hence quality analysis of Web sites obtained with each search strategy revealed that only 8 of 110 sites in AltaVista and 13 of 148 sites with the combined search engine strategy were found to be recommendable. Hence the odds ratio of obtaining adequate and authentic information by use of multiple search engines was 1.2 compared with the use of a single search engine strategy. DISCUSSION In our study we analyzed a pooled sample of 232 Web sites obtained by different search strategies. Excluding the repetitions, there were 117 sites for the final assessment. Only 70 of these 117 were found to provide some form of patient information and hence
512
S. N. SAMBANDAM ET AL.
were considered relevant. Among the 70 relevant sites, only 16 were found to be recommendable based on the adequacy, authenticity, and accountability of information provided. Hence our analysis shows that an enormous amount of information is available for patients on the Internet but at the same time raises concerns about the quality of information available and the difficulty encountered in obtaining the relevant information. With the increasing role of the Internet as a source of medical information for patients, it can be assumed that they can be misinformed or distracted with the valuable information being obscured by irrelevant, misleading information. The Internet continues to be the major source of health-related information. With the pace at which Web sites are proliferating, the quality of healthrelated information provided on the Web has become a major issue of concern in various specialties. In 1997 Sikorski and Peters23 reviewed a few Web sites on cancer treatments and tried to justify the reason for recommending those sites to patients. In 2001 Corpron and Lelli14 searched the Internet for patient information on intersex anomalies, especially ambiguous genitalia. They used 6 different general search engines and included the first 50 search results from each search engine for quality analysis. This is in contrast to our selection strategy because we used both metasearch engines and health search engines in addition to 3 commonly used general search engines. Their analysis revealed that of the 300 Web sites analyzed, only 45 were found to provide patient information, of which only 8 were unique sites and merely 5 Web sites were found to be recommendable (1.6%). Similar studies analyzing the quality of patient information available on the Internet have been performed on various orthopaedic topics.1,2,6,10,12,16,24-30 Those included were mostly some form of orthopaedic condition, such as ankle sprain, backache, carpal tunnel syndrome, hallux valgus, and congenital talipes equinovarus. However, patients very often seek Web information when they are offered different treatment options and have to select an option based on the information given to them by the health care provider. The studies exclusively analyzing the quality of information on orthopaedic procedures are very few.1,2 Mabrey1 in 2000 analyzed the Web resources for patient-oriented information on total hip replacement. He used different key words such as “hip replacement,” “total hip replacement,” and “hip surgery.” He used only metasearch engines, namely GO and AskJeeves. Instead of reporting the overall quality of information obtained, he performed a descriptive anal-
ysis of a few Web sites and made some conclusions based on those Web sites. Our study is unique in several aspects. We attempted to obtain the best sample of Web sites for quality analysis. To fulfill that objective, we used not just general search engines but also metasearch and health search engines. Furthermore, to obtain the most representative Web sites, we included the first 100 Web sites with all links from the most commonly used Web site (AltaVista)13 and the first 20 Web sites from all other search engines. This is in contrast to most of the previous studies, which either used a single search engine or used the first few Web sites from different search engines. Our study also designates 16 sites that can be recommended to patients (Table 2, online only, available at www.arthroscopyjournal.org). This is in contrast to some of the previous studies that just concluded that very few Web sites provide reliable information but failed to mention the URL addresses of those sites. We also tried to establish the most efficient search strategy in our study, and we found that using the first few search results of multiple search engines marginally increases the chances of obtaining the best possible information as compared with surfing all of the results of a single search engine. This is similar to the results obtained by Lawrence and Giles,31 who concluded in their study that combining results from different search engines yields 3.5 times more documents as compared with using a single search engine. However, we understand that our study has a few limitations. We based our study on Web sites provided by the most commonly used search engines and not on a roster of Web sites listed according to the overall usage by people interested in knee arthroscopy. We used a custom-made questionnaire based on the questionnaires used in similar studies. Because our questionnaire was not validated and its reliability was not checked, its use for similar purposes in other studies is difficult and debatable. Furthermore, because of the dynamic nature of some Web sites and the crosssectional nature of our study, it is difficult to generalize the information for future usage. In addition, our investigation was limited to knee arthroscopy and cannot be generalized to other orthopaedic procedures. CONCLUSIONS Our study shows that the Internet offers a wide variety of information about knee arthroscopy. However, indeed, the quality of information is limited. Most sites
KNEE ARTHROSCOPY ON THE WORLD WIDE WEB were not confined to conventional information, and even those that presented this did not describe it completely. Despite efforts to regulate the quality of information available on the Internet for patients, it is evident that the quality of information is still very poor. Even though it is the moral responsibility of the Web site owners to provide adequate, relevant, and authentic patient information without any bias, health care professionals are also responsible for helping their patients access the right health information. Health care professionals should suggest a few URL addresses providing reliable patient information. A few Web sites provide relevant patient information, such as that of the American Academy of Orthopaedic Surgeons (www.aaos.org), which lists information on more than 400 orthopaedic conditions, and can be recommended to patients. Orthopaedic surgeons can also provide in writing the appropriate search terminology and best search strategy31 to those patients who want to extend their search beyond those sites recommended by their surgeons. Furthermore, they should also help their patients identify the credibility of a Web site by providing them with the criteria suggested by various authors6,13,16-18 and governing bodies.5,7,32 REFERENCES 1. Mabrey JD. Survey of patient-oriented total hip replacement information on the World Wide Web. Clin Orthop Relat Res 2000:106-113. 2. Mathur S, Shanti N, Brkaric M, et al. Surfing for scoliosis: The quality of information available on the Internet. Spine 2005; 30:2695-700. 3. Miniwatts Marketing Group. Internet usage statistics—The big picture. World Internet users and population stats. 2006. Available from: URL: http://www.internetworldstats.com/stats.htm. Accessed December 21, 2005. 4. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome. J Bone Joint Surg Am 2000;82:1540-1543. 5. Winker MA, Flanagin A, Chi-Lum B, et al. Guidelines for medical and health information sites on the internet: Principles governing AMA web sites. American Medical Association. JAMA 2000;283:1600-1606. 6. Smart JM, Burling D. Radiology and the internet: A systematic review of patient information resources. Clin Radiol 2001;56: 867-870. 7. Scherrer J. Health On Net Foundation homepage. 1999. www. hon.ch/home.html 8. Bessell TL, McDonald S, Silagy CA, Anderson JN, Hiller JE, Sansom LN. Do Internet interventions for consumers cause more harm than good? A systematic review. Health Expect 2002;5:28-37. 9. Crocco AG, Villasis-Keever M, Jadad AR. Analysis of cases of harm associated with use of health information on the internet. JAMA 2002;287:2869-2871. 10. Shuyler KS, Knight KM. What are patients seeking when they turn to the Internet? Qualitative content analysis of questions asked by visitors to an orthopaedics Web site. J Med Internet Res 2003;5:e24.
513
11. Li L, Irvin E, Guzman J, Bombardier C. Surfing for back pain patients: The nature and quality of back pain information on the Internet. Spine 2001;26:545-557. 12. Nogler M, Wimmer C, Mayr E, Ofner D. The efficacy of using search engines in procuring information about orthopaedic foot and ankle problems from the World Wide Web. Foot Ankle Int 1999;20:322-325. 13. Chen LE, Minkes RK, Langer JC. Pediatric surgery on the Internet: Is the truth out there? J Pediatr Surg 2000;35:11791182. 14. Corpron CA, Lelli JL Jr. Evaluation of pediatric surgery information on the Internet. J Pediatr Surg 2001;36:1187-1189. 15. Murphy MA, Joyce WP. Information for surgical patients: Implications of the World Wide Web. Eur J Surg 2001;167: 728-733. 16. Aslam N, Bowyer D, Wainwright A, Theologis T, Benson M. Evaluation of Internet use by paediatric orthopaedic outpatients and the quality of information available. J Pediatr Orthop B 2005;14:129-133. 17. Simpson P, Oliver CW. Searching the internet for orthopaedic knowledge. J Bone Joint Surg Br 2004;86:1105-1107. 18. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor—Let the reader and viewer beware. JAMA 1997;277:1244-1245. 19. Burneo JG. An evaluation of the quality of epilepsy education on the Canadian World Wide Web. Epilepsy Behav 2006;8: 299-302. 20. Gimenez-Perez G, Caixas A, Gimenez-Palop O, GonzalezClemente JM, Mauricio D. Dissemination of “patient-oriented evidence that matters” on the Internet: The case of type 2 diabetes treatment. Diabet Med 2005;22:688-892. 21. Martin-Facklam M, Kostrzewa M, Martin P, Haefeli WE. Quality of drug information on the World Wide Web and strategies to improve pages with poor information quality. An intervention study on pages about sildenafil. Br J Clin Pharmacol 2004;57:80-85. 22. Styra R. The Internet’s impact on the practice of psychiatry. Can J Psychiatry 2004;49:5-11. 23. Sikorski R, Peters R. Oncology ASAP. Where to find reliable cancer information on the Internet. JAMA 1997;277:14311432. 24. Sproule JA, Tansey C, Burns B, Fenelon G. The Web: Friend or foe of the hand surgeon? Hand Surg 2003;8:181-185. 25. Groot D, ter Riet G, Khan KS, Misso K. Comparison of search strategies and quality of medical information of the Internet: A study relating to ankle sprain. Injury 2001;32:473-476. 26. Moshirfar A, Campbell JT, Khasraghi FA, Wenz JF Sr. Evaluating the quality of Internet-derived information on plantar fasciitis. Clin Orthop Relat Res 2004:60-63. 27. Greene DL, Appel AJ, Reinert SE, Palumbo MA. Lumbar disc herniation: Evaluation of information on the internet. Spine 2005;30:826-829. 28. McClune T, Burton AK, Waddell G. Whiplash associated disorders: A review of the literature to guide patient information and advice. Emerg Med J 2002;19:499-506. 29. Macario A, Schilling P, Rubio R, Bhalla A, Goodman S. What questions do patients undergoing lower extremity joint replacement surgery have? BMC Health Serv Res 2003;3:11. 30. Beall MS III, Golladay GJ, Greenfield ML, Hensinger RN, Biermann JS. Use of the Internet by pediatric orthopaedic outpatients. J Pediatr Orthop 2002;22:261-264. 31. Lawrence S, Giles CL. Searching the world wide web. Science 1998;280:98-100. 32. Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med 1998;28:603-610.
KNEE ARTHROSCOPY ON THE WORLD WIDE WEB TABLE 1.
513.e1
Assessment Tool for Web Site Quality Analysis
Search engine General
Metasearch Health topic Web site description URL address Date of search Position in search result Accessibility
Illustrations and pictures Duplicate Quality Date information was updated If yes, then how old? Authorship information availability If yes If yes
Promotional message What is being promoted? Target audience information Availability Type of target audience
Adequacy of content (0, no information; 3, some information; 5, adequate information) Diagnosis Procedure Alternate options Postoperative protocol Complications Prognosis Total Authenticity of content (0, no referencing at all; 3, good-quality Web sites referenced; 5, textbook referenced; 10, textbook and scientific articles referenced) Overall assessment Would you recommend the site?
Google AltaVista Yahoo Mamma Copernic MedHunt Healthfinder
Time search
Download time
Easy Page not found No longer exists Password-protected Y/N Y/N Y/N Y/N Easy Difficult General public Educational institution Instrument company Club Professional organization For-profit organization Other Y/N Product/technique/service/ advertisement Y/N Easy General public Targeted workers Health care providers Insurers Employers People with knee problems
Difficult
0 0 0 0 0 0
3 3 3 3 3 3
5 5 5 5 5 5
0
3
5
Poor Y/N
Fair
Good
10
513.e2
S. N. SAMBANDAM ET AL. TABLE 2.
Recommendable Web Sites
Exact URL address of Web page http://www.vh.org/adult/patient/orthopaedics/kneearthroscopy/ http://hcd2.bupa.co.uk/fact_sheets/html/arthroscopy.html http://www.nlm.nih.gov/medlineplus/tutorials/kneearthroscopy/ htm/index.htm http://surgerydoor.co.uk http://patienteducation.upmc.com/Pdf/KneeArthroscopy.pdf http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/ Arthroscopy?open http://www.priory.com/vet/stifle1.htm http://www.kneeguru.co.uk/kneegeeks/7922698801.html http://www.patient.co.uk/showdoc/27000470 http://www.themediweb.net/orthopaedics/infosheets_html/ inpatients/Knee.html http://www.yorkshirekneeclinic.co.uk/acl-reconstructionsurgery.htm http://www.orthopaedics.com/ortho/knees.html http://www.orthos.co.nz/kneejoint.html http://www.international-medical-tourism.com/arthroscopywith-ligament-reconstruction.html http://www.aaos.org.html http://www.niams.nih.gov/hi/topics/kneeprobs/kneeqa.html Strategy A http://www.vh.org/adult/patient/orthopaedics/kneearthroscopy/ http://hcd2.bupa.co.uk/fact_sheets/html/arthroscopy.html http://www.nlm.nih.gov/medlineplus/tutorials/kneearthroscopy/ htm/index.htm http://surgerydoor.co.uk http://patienteducation.upmc.com/Pdf/KneeArthroscopy.pdf http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/ Arthroscopy?open http://www.priory.com/vet/stifle1.htm http://www.kneeguru.co.uk/kneegeeks/7922698801.html http://www.orthopaedics.com/ortho/knees.html http://www.orthos.co.nz/kneejoint.html http://www.international-medical-tourism.com/arthroscopywith-ligament-reconstruction.html http://www.aaos.org.html http://www.niams.nih.gov/hi/topics/kneeprobs/kneeqa.html Strategy B http://hcd2.bupa.co.uk/fact_sheets/html/arthroscopy.html http://www.nlm.nih.gov/medlineplus/tutorials/kneearthroscopy/ htm/index.html http://www.priory.com/vet/stifle1.html http://www.kneeguru.co.uk/kneegeeks/7922698801.html http://surgerydoor.co.uk http://www.patient.co.uk/showdoc/27000470 http://www.themediweb.net/orthopaedics/infosheets_html/ inpatients/Knee.html http://www.yorkshirekneeclinic.co.uk/acl-reconstructionsurgery.htm http://www.vh.org/adult/patient/orthopaedics/kneearthroscopy/