- Email: [email protected]
Pediatric surgery on the internet: Is the truth out there?
Pediatric Surgery on the Internet: Is the Truth Out There? By Li Ern Chen, Robert K. Minkes, and Jacob C. Langer St Louis, Missouri
Background: The enormous amount of unmonitored medical information on the Internet prompted this investigation into the quality of pediatric surgery information on the Internet. Methods: The Internet was searched for information on diaphragmatic hernia (CDH), abdominal wall defects (AWD), pediatric inguinal hernia (IH), and pectus excavatum (PE). Websites were characterized, classified, and evaluated for completeness, accuracy and bias toward or against the medical profession. Results: A total of 141 websites were evaluated (NCDH ⫽ 37, NAWD ⫽ 49, NIH ⫽ 26, NPE ⫽ 29). A total of 59.6% targeted medical professionals, and 46.8% targeted the lay population. A total of 58.2% described symptoms and diagnosis. Etiology, pathology, surgery, postoperative course, and prognosis each were addressed by under 40%. A total of 58.2% were accountable for the information presented. A total of 93.1% were incomplete, 75.7% contained accurate informa-
S
INCE ITS CREATION in 1969, the Internet has experienced exponential growth both in its size and in the number of people who access it. As of April 1999, there were 83 million adults aged 16 years and older using the Internet in the United States.1 This includes over 40% of the US population aged 16 and older. Increasing ease of access guarantees continued growth in the number of people using the Internet. The Internet promises to be a major medium for the communication of information. In response to the creation and popularization of the Internet as a convenient medium of communication, there has been an explosion of pediatric health care information on the Internet. This proliferation of information has been paralleled by an increase in the number of people searching for such information.2 A recent study reported that in a 5-week period, a cluster of pediatric web sites was accessed by 5,216 individuals who requested over 36,000 links.3 The popularity of the Internet as a source of medical information is encouraging. However, the relationship between freedom of the press provided by the Internet and the accuracy of information presented using this medium may not be a harmonious one. The structure of the Internet makes it possible for anyone to play the roles of author, editor, and publisher simultaneously, leaving the quality of information on the Internet uncontrolled. This has led to the emergence of a large amount of
Journal of Pediatric Surgery, Vol 35, No 8 (August), 2000: pp 1179-1182
tion, and 97.7% were positive or neutral toward medical treatment. Among diagnoses, CDH had the highest percentage of websites owned by academic institutions. PE had the highest percentage of websites owned by lay people. PE websites also were the least accurate.
Conclusions: Internet information on pediatric surgery varies significantly in quality. Lay people own most websites targeted at the lay audience, and the information may not reflect the opinions of most pediatric surgeons. Increasing use of the Internet by parents seeking medical information warrants an organized approach to ensure complete and accurate information online. J Pediatr Surg 35:1179-1182. Copyright r 2000 by W.B. Saunders Company.
INDEX WORDS: Internet, websites, quality, surgery.
information on the Internet, which varies greatly in quality and accuracy.4 For example, McClung et al5 found that only 20% of websites containing information on childhood diarrhea conformed to the American Academy of Pediatrics recommendations for treatment. It is difficult for the lay person to distinguish accurate from inaccurate information. This lack of quality control makes it potentially dangerous for parents to seek medical information on the Internet. The current study was prompted by the coupling of these 2 phenomena: the increasing number of parents gleaning pediatric surgery information from the Internet and the unregulated nature of the enormous amount of information on the Internet. Our goal was to assess the quality of pediatric surgery information available to the lay person in terms of accountability for information presented, completeness, accuracy, and credibility.
From the Division of Pediatric Surgery, Washington University School of Medicine, St Louis, MO. Address reprint requests to Jacob C. Langer, MD, Chief, Division of Pediatric General Surgery, Hospital for Sick Children, 555 University Ave, Toronto, Ontario, Canada M5G 1X8. Copyright r 2000 by W.B. Saunders Company 0022-3468/00/3508-0007$03.00/0 doi:10.1053/jpsu.2000.8723
1179
1180
CHEN, MINKES, AND LANGER
MATERIALS AND METHODS
Table 2. Website Characteristics No.
Data Collection Alta Vista, the search engine with access to the largest number of web sites (30 million pages found on 275,600 servers) was used to conduct the Internet searches. Using the diagnosis as the search string, Internet searches for information on the key phrases ‘‘gastroschisis and omphalocele’’ (abdominal wall defects [AWD]), ‘‘congenital diaphragmatic hernia’’ (CDH), ‘‘pediatric inguinal hernia’’ (IH), and ‘‘pectus excavatum’’ (PE) were performed by a college student. Of the first 400 references for each diagnosis, those that were in English and had information on the diagnosis of interest were included in the study. Each website was described and classified according to a number of characteristics, which were chosen because they were felt to be important to the evaluation of medical sites. These included the site owner (academic institution, parenting resource, health website, medical doctor, commercial enterprise, or lay person); type of site (medical student or resident teaching file, abstract or article, support group, question and answer, or personal account); target audience (medical professionals or lay population); and content (symptoms and diagnosis, etiology and pathology, surgery, postoperative course, prognosis, or professional practice information). Accountability for posted information was assessed according to the disclosure of the following: author’s name, author’s credentials, evidence for claims or copyright, website owner and the date the content was posted.6 The quality of each site was evaluated by 2 pediatric surgeons according to completeness (on a scale of 0 to 2; 0 ⫽ incomplete, 2 ⫽ very complete), accuracy (on a scale of 0 to 2; 0 ⫽ inaccurate, 2 ⫽ very accurate), and bias toward or against the medical profession (on a scale of ⫺1 to ⫹1; ⫺1 ⫽ negative, 0 ⫽ neutral, ⫹1 ⫽ positive). Accuracy was defined as information that either is evidence based, or, in the absence of definitive evidence, generally accepted as true by the majority of pediatric surgeons.
Data Quantification and Analysis The 2 surgeons’ scores were combined to compute a composite completeness score (ranging from 0 to 4), accuracy score (ranging from 0 to 4), and bias score (ranging from ⫺2 to ⫹2) for each website. The information on a website was considered complete or accurate if it received a composite score of 3 or higher. The website was considered accountable for the information presented if it satisfied at least 4 of the 5 accountability criteria. Data analyses were computed using a standard SPSS program (SPSS Inc, Chicago, IL). A P value of less than .05 was considered statistically significant.
RESULTS
The results of the Internet search for information on each diagnosis are summarized in Table 1. A total of 141 websites were included in the study. Each search proTable 1. Internet Search Results
Diagnosis
AWD CDH IH PE
No. of References Generated
No. of Relevant Websites From First 400
1,010 49 1,138 37 452 26 557 29 Total number of relevant websites ⫽ 141
Site owner Academic institution Parenting resource Health website Medical doctor Commercial enterprise Lay person Target audience Medical professionals Lay population Type of site Teaching file Abstract or article Support group Question and answer Personal account Other Content Symptoms and diagnosis Etiology and pathology Surgery Postoperative course Prognosis Professional practice information Accountability criteria satisfied Yes No Information complete Yes No Information accurate Yes No Bias toward medical profession Positive Neutral Negative Referral to a reliable source of information Yes No
Percentage
58 8 50 6 4 15
41 6 36 4 3 11
84 66
60 47
34 39 2 12 20 34
24 28 1 9 14 24
82 47 50 31 33 31
58 33 36 22 23 22
82 59
58 42
9 122
7 93
90 29
76 24
62 66 3
47 50 3
28 103
21 79
NOTE. Some sites fit into more than 1 category, and the percentages, therefore, may add up to greater than 100%.
duced a large number of websites with no content on the diagnosis being searched, or were in a language other than English. Characteristics of the 141 websites are summarized in Table 2. Of the 141 websites, 59.6% targeted medical professionals, and 47% targeted the lay population. Symptoms and diagnosis were described by 58%. Etiology, pathology, surgery, postoperative course, and prognosis each were addressed by fewer than 40% of websites. Fifty-eight percent were accountable for the information presented, and 76% contained accurate information. Information was incomplete in 93%. A positive bias toward the medical profession was displayed by 47%, whereas 50% were neutral, and 3% were negative. Only 21% of the websites referred readers to a reliable source for further information.
PEDIATRIC SURGERY ON THE INTERNET
Personal accounts of experience with surgery appeared more often on websites that were owned by lay people (P ⬍ .001). Websites owned by lay people tended to be targeted at the lay population (P ⬍ .001), and websites owned by academic institutions were targeted at medical professionals (P ⬍ .01). Accountability Websites that were most accountable for the information that they posted were those owned by academic institutions (P ⬍ .001) and those that targeted medical professionals (P ⬍ .001). Websites owned by lay people and those that targeted the lay population were the least accountable for the information they posted (P ⬍ .01). Completeness Medical student or resident teaching files were the most complete, although this did not reach statistical significance (P ⫹ .073). Accuracy Websites owned by academic institutions significantly were more accurate (P ⬍ .05), as were websites targeted at medical professionals (P ⬍ .001). Websites owned by lay people were significantly less accurate (P ⬍ .001). Websites targeted at the lay population were also less accurate (P ⬍ .05). Bias A positive bias toward the medical profession was displayed by websites owned by both doctors (P ⬍ .01) and lay people (P ⫽ .001). As expected, websites with teaching files also displayed a positive bias toward the medical profession (P ⬍ .05). Reference to Reliable Source for Further Information Websites that targeted medical professionals were more likely to refer the reader on to a reliable source for further information (P ⬍ .001). Effect of Diagnosis There was wide variation in site ownership, content, and quality among diagnoses. PE websites had the highest percentage of websites owned by lay people (P ⬍ .01), and significantly more personal accounts or stories were found on these websites (P ⬍ .01). CDH websites had the highest percentage of websites owned by an academic institution (P ⬍ .05). Although CDH websites were the most accountable, and PE websites were the least accountable (Table 3), the difference was not statistically significant. Similarly, AWD websites were the most complete, and PE websites were the least complete (Table 3), but this difference also was not statistically significant. As also seen in Table 3, PE
1181
Table 3. Effect of Diagnosis on Accountability, Completeness, Accuracy, and Bias Toward the Medical Profession, Expressed as Mean Score for Each Category
Accountability Completeness Accuracy Bias
CDH
AWD
IH
PE
3.89 0.56 2.80 0.59
3.81 0.83 3.30 0.13*
3.57 0.76 3.33 0.92
3.44 0.34 2.27* 1.12*
*P ⬍ .05 compared with all other diagnoses.
websites were significantly less accurate than all other websites (P ⬍ .001) and were significantly more biased toward the medical profession than all other websites (P ⬍ .05). In contrast, AWD websites were significantly less positive (or more neutral) than all other websites (P ⬍ .05). Although there were individual sites that were antagonistic toward the medical profession in each diagnosis, none of the diagnoses exhibited a mean bias level that was negative. AWD websites were the most likely (P ⬍ .05), and IH websites were the least likely (P ⬍ .05) to refer readers on to a reliable source for further information. DISCUSSION
Our results show that there is an enormous amount of information on pediatric surgery available to the lay person on the Internet. However, our data also raise concern about the quality of this information. The first issue of concern is how understandable the information is to the average reader. A recent study suggested that average parents of pediatric patients at a university hospital were reading at a seventh to eighth grade reading level.7 Although not specifically measured in our study, others have estimated that the average patient education material on the Internet is written at a tenth grade level.8 This suggests that the majority of credible and accurate pediatric surgery information on the Internet is not comprehensible to many readers. The second issue of concern is the accuracy of the available information. Our study found that websites written by lay people tend to be targeted to other lay people and that the information on websites written by lay people also tends to be the least accurate. Most of the outcome measures used in our study to estimate accuracy, completeness, and accountability varied considerably among websites and was significantly affected by the pediatric surgery diagnosis in question. Although there is very little information that is biased against the medical profession, some of the information clearly is biased toward a particular approach, and some is blatantly commercial. A definitive description of this kind of bias was beyond the scope of this study. How can the quality of information available to our patients be controlled? The Health On the Net Foundation currently uses a Code of Conduct for medical websites
1182
CHEN, MINKES, AND LANGER
(HONcode) 9 to unify the quality of medical information on the Internet. The HONcode contains a set of 8 principles that govern the quality of health information on the Internet. Website owners who agree to abide by the principles may download the HONcode logo and display it on their website.10 The Foundation reviews sites bearing the HONcode logo on a regular basis and has mechanisms to identify sites bearing the HONcode logo that have not registered with the Foundation. We did not find the HONcode logo on any of the websites in our study, indicating a lack of awareness of the HONcode among website owners. Knowledge about the HONcode and other similar quality control mechanisms likely will increase with time. Another potential response to the variability in quality
would be an organized approach to providing a source of pediatric surgery information for the lay population through a designated web site. The AAP Section on Surgery or the American Pediatric Surgical Association could provide an online source for comprehensive and accurate information on pediatric surgery that is targeted at the lay population. However, until an organized approach to website quality control is established, it is important that pediatric surgeons warn parents about the variability in quality of the information on the Internet, stressing that not everything they read on the Internet is true or accurate. Giving parents basic, nonmedical criteria by which they can assess website credibility also will encourage safer browsing of the Internet.
REFERENCES 1. http://www.headcount.com: Internet usage around the world. accessed April 28, 1999 2. Eysenbach G, Diepgen TL: Patients looking for information on the Internet and seeking teleadvice. Arch Dermatol 135:151-156, 1999 3. Lehmann CU, Wang DJ, Kim GR, et al: Utilization of a pediatric link collection by health professionals and laypersons. Med Inform 23:53-62, 1998 4. Isaa¨cson M: Medical information on the Internet. J Travel Med 6:52, 1999 5. McClung HJ, Murray RD, Heitlinger LA: The Internet as a source for current patient information. Pediatrics 101:e2, 1998 6. Silberg WM, Lundberg GD, Musacchio RA: Assessing, control-
ling, and assuring the quality of medical information on the Internet. JAMA 277:1244-1245, 1997 7. Murphy PW: Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics 93:460-468, 1994 8. Graber MA, Roller CM, Kaeble B: Readability levels of patient education material on the World Wide Web. J Fam Pract 48:58-61, 1999 9. http://www.hon.ch/Conduct.html: HON Code of Conduct for Medical and Health Websites. accessed April 28, 1999 10. Boyer C, Selby M, Scherrer JR, et al: The Health On the Net Code of Conduct for medical and health websites. Comp Biol Med 28:603-610, 1998
Background: The enormous amount of unmonitored medical information on the Internet prompted this investigation into the quality of pediatric surgery information on the Internet. Methods: The Internet was searched for information on diaphragmatic hernia (CDH), abdominal wall defects (AWD), pediatric inguinal hernia (IH), and pectus excavatum (PE). Websites were characterized, classified, and evaluated for completeness, accuracy and bias toward or against the medical profession. Results: A total of 141 websites were evaluated (NCDH ⫽ 37, NAWD ⫽ 49, NIH ⫽ 26, NPE ⫽ 29). A total of 59.6% targeted medical professionals, and 46.8% targeted the lay population. A total of 58.2% described symptoms and diagnosis. Etiology, pathology, surgery, postoperative course, and prognosis each were addressed by under 40%. A total of 58.2% were accountable for the information presented. A total of 93.1% were incomplete, 75.7% contained accurate informa-
S
INCE ITS CREATION in 1969, the Internet has experienced exponential growth both in its size and in the number of people who access it. As of April 1999, there were 83 million adults aged 16 years and older using the Internet in the United States.1 This includes over 40% of the US population aged 16 and older. Increasing ease of access guarantees continued growth in the number of people using the Internet. The Internet promises to be a major medium for the communication of information. In response to the creation and popularization of the Internet as a convenient medium of communication, there has been an explosion of pediatric health care information on the Internet. This proliferation of information has been paralleled by an increase in the number of people searching for such information.2 A recent study reported that in a 5-week period, a cluster of pediatric web sites was accessed by 5,216 individuals who requested over 36,000 links.3 The popularity of the Internet as a source of medical information is encouraging. However, the relationship between freedom of the press provided by the Internet and the accuracy of information presented using this medium may not be a harmonious one. The structure of the Internet makes it possible for anyone to play the roles of author, editor, and publisher simultaneously, leaving the quality of information on the Internet uncontrolled. This has led to the emergence of a large amount of
Journal of Pediatric Surgery, Vol 35, No 8 (August), 2000: pp 1179-1182
tion, and 97.7% were positive or neutral toward medical treatment. Among diagnoses, CDH had the highest percentage of websites owned by academic institutions. PE had the highest percentage of websites owned by lay people. PE websites also were the least accurate.
Conclusions: Internet information on pediatric surgery varies significantly in quality. Lay people own most websites targeted at the lay audience, and the information may not reflect the opinions of most pediatric surgeons. Increasing use of the Internet by parents seeking medical information warrants an organized approach to ensure complete and accurate information online. J Pediatr Surg 35:1179-1182. Copyright r 2000 by W.B. Saunders Company.
INDEX WORDS: Internet, websites, quality, surgery.
information on the Internet, which varies greatly in quality and accuracy.4 For example, McClung et al5 found that only 20% of websites containing information on childhood diarrhea conformed to the American Academy of Pediatrics recommendations for treatment. It is difficult for the lay person to distinguish accurate from inaccurate information. This lack of quality control makes it potentially dangerous for parents to seek medical information on the Internet. The current study was prompted by the coupling of these 2 phenomena: the increasing number of parents gleaning pediatric surgery information from the Internet and the unregulated nature of the enormous amount of information on the Internet. Our goal was to assess the quality of pediatric surgery information available to the lay person in terms of accountability for information presented, completeness, accuracy, and credibility.
From the Division of Pediatric Surgery, Washington University School of Medicine, St Louis, MO. Address reprint requests to Jacob C. Langer, MD, Chief, Division of Pediatric General Surgery, Hospital for Sick Children, 555 University Ave, Toronto, Ontario, Canada M5G 1X8. Copyright r 2000 by W.B. Saunders Company 0022-3468/00/3508-0007$03.00/0 doi:10.1053/jpsu.2000.8723
1179
1180
CHEN, MINKES, AND LANGER
MATERIALS AND METHODS
Table 2. Website Characteristics No.
Data Collection Alta Vista, the search engine with access to the largest number of web sites (30 million pages found on 275,600 servers) was used to conduct the Internet searches. Using the diagnosis as the search string, Internet searches for information on the key phrases ‘‘gastroschisis and omphalocele’’ (abdominal wall defects [AWD]), ‘‘congenital diaphragmatic hernia’’ (CDH), ‘‘pediatric inguinal hernia’’ (IH), and ‘‘pectus excavatum’’ (PE) were performed by a college student. Of the first 400 references for each diagnosis, those that were in English and had information on the diagnosis of interest were included in the study. Each website was described and classified according to a number of characteristics, which were chosen because they were felt to be important to the evaluation of medical sites. These included the site owner (academic institution, parenting resource, health website, medical doctor, commercial enterprise, or lay person); type of site (medical student or resident teaching file, abstract or article, support group, question and answer, or personal account); target audience (medical professionals or lay population); and content (symptoms and diagnosis, etiology and pathology, surgery, postoperative course, prognosis, or professional practice information). Accountability for posted information was assessed according to the disclosure of the following: author’s name, author’s credentials, evidence for claims or copyright, website owner and the date the content was posted.6 The quality of each site was evaluated by 2 pediatric surgeons according to completeness (on a scale of 0 to 2; 0 ⫽ incomplete, 2 ⫽ very complete), accuracy (on a scale of 0 to 2; 0 ⫽ inaccurate, 2 ⫽ very accurate), and bias toward or against the medical profession (on a scale of ⫺1 to ⫹1; ⫺1 ⫽ negative, 0 ⫽ neutral, ⫹1 ⫽ positive). Accuracy was defined as information that either is evidence based, or, in the absence of definitive evidence, generally accepted as true by the majority of pediatric surgeons.
Data Quantification and Analysis The 2 surgeons’ scores were combined to compute a composite completeness score (ranging from 0 to 4), accuracy score (ranging from 0 to 4), and bias score (ranging from ⫺2 to ⫹2) for each website. The information on a website was considered complete or accurate if it received a composite score of 3 or higher. The website was considered accountable for the information presented if it satisfied at least 4 of the 5 accountability criteria. Data analyses were computed using a standard SPSS program (SPSS Inc, Chicago, IL). A P value of less than .05 was considered statistically significant.
RESULTS
The results of the Internet search for information on each diagnosis are summarized in Table 1. A total of 141 websites were included in the study. Each search proTable 1. Internet Search Results
Diagnosis
AWD CDH IH PE
No. of References Generated
No. of Relevant Websites From First 400
1,010 49 1,138 37 452 26 557 29 Total number of relevant websites ⫽ 141
Site owner Academic institution Parenting resource Health website Medical doctor Commercial enterprise Lay person Target audience Medical professionals Lay population Type of site Teaching file Abstract or article Support group Question and answer Personal account Other Content Symptoms and diagnosis Etiology and pathology Surgery Postoperative course Prognosis Professional practice information Accountability criteria satisfied Yes No Information complete Yes No Information accurate Yes No Bias toward medical profession Positive Neutral Negative Referral to a reliable source of information Yes No
Percentage
58 8 50 6 4 15
41 6 36 4 3 11
84 66
60 47
34 39 2 12 20 34
24 28 1 9 14 24
82 47 50 31 33 31
58 33 36 22 23 22
82 59
58 42
9 122
7 93
90 29
76 24
62 66 3
47 50 3
28 103
21 79
NOTE. Some sites fit into more than 1 category, and the percentages, therefore, may add up to greater than 100%.
duced a large number of websites with no content on the diagnosis being searched, or were in a language other than English. Characteristics of the 141 websites are summarized in Table 2. Of the 141 websites, 59.6% targeted medical professionals, and 47% targeted the lay population. Symptoms and diagnosis were described by 58%. Etiology, pathology, surgery, postoperative course, and prognosis each were addressed by fewer than 40% of websites. Fifty-eight percent were accountable for the information presented, and 76% contained accurate information. Information was incomplete in 93%. A positive bias toward the medical profession was displayed by 47%, whereas 50% were neutral, and 3% were negative. Only 21% of the websites referred readers to a reliable source for further information.
PEDIATRIC SURGERY ON THE INTERNET
Personal accounts of experience with surgery appeared more often on websites that were owned by lay people (P ⬍ .001). Websites owned by lay people tended to be targeted at the lay population (P ⬍ .001), and websites owned by academic institutions were targeted at medical professionals (P ⬍ .01). Accountability Websites that were most accountable for the information that they posted were those owned by academic institutions (P ⬍ .001) and those that targeted medical professionals (P ⬍ .001). Websites owned by lay people and those that targeted the lay population were the least accountable for the information they posted (P ⬍ .01). Completeness Medical student or resident teaching files were the most complete, although this did not reach statistical significance (P ⫹ .073). Accuracy Websites owned by academic institutions significantly were more accurate (P ⬍ .05), as were websites targeted at medical professionals (P ⬍ .001). Websites owned by lay people were significantly less accurate (P ⬍ .001). Websites targeted at the lay population were also less accurate (P ⬍ .05). Bias A positive bias toward the medical profession was displayed by websites owned by both doctors (P ⬍ .01) and lay people (P ⫽ .001). As expected, websites with teaching files also displayed a positive bias toward the medical profession (P ⬍ .05). Reference to Reliable Source for Further Information Websites that targeted medical professionals were more likely to refer the reader on to a reliable source for further information (P ⬍ .001). Effect of Diagnosis There was wide variation in site ownership, content, and quality among diagnoses. PE websites had the highest percentage of websites owned by lay people (P ⬍ .01), and significantly more personal accounts or stories were found on these websites (P ⬍ .01). CDH websites had the highest percentage of websites owned by an academic institution (P ⬍ .05). Although CDH websites were the most accountable, and PE websites were the least accountable (Table 3), the difference was not statistically significant. Similarly, AWD websites were the most complete, and PE websites were the least complete (Table 3), but this difference also was not statistically significant. As also seen in Table 3, PE
1181
Table 3. Effect of Diagnosis on Accountability, Completeness, Accuracy, and Bias Toward the Medical Profession, Expressed as Mean Score for Each Category
Accountability Completeness Accuracy Bias
CDH
AWD
IH
PE
3.89 0.56 2.80 0.59
3.81 0.83 3.30 0.13*
3.57 0.76 3.33 0.92
3.44 0.34 2.27* 1.12*
*P ⬍ .05 compared with all other diagnoses.
websites were significantly less accurate than all other websites (P ⬍ .001) and were significantly more biased toward the medical profession than all other websites (P ⬍ .05). In contrast, AWD websites were significantly less positive (or more neutral) than all other websites (P ⬍ .05). Although there were individual sites that were antagonistic toward the medical profession in each diagnosis, none of the diagnoses exhibited a mean bias level that was negative. AWD websites were the most likely (P ⬍ .05), and IH websites were the least likely (P ⬍ .05) to refer readers on to a reliable source for further information. DISCUSSION
Our results show that there is an enormous amount of information on pediatric surgery available to the lay person on the Internet. However, our data also raise concern about the quality of this information. The first issue of concern is how understandable the information is to the average reader. A recent study suggested that average parents of pediatric patients at a university hospital were reading at a seventh to eighth grade reading level.7 Although not specifically measured in our study, others have estimated that the average patient education material on the Internet is written at a tenth grade level.8 This suggests that the majority of credible and accurate pediatric surgery information on the Internet is not comprehensible to many readers. The second issue of concern is the accuracy of the available information. Our study found that websites written by lay people tend to be targeted to other lay people and that the information on websites written by lay people also tends to be the least accurate. Most of the outcome measures used in our study to estimate accuracy, completeness, and accountability varied considerably among websites and was significantly affected by the pediatric surgery diagnosis in question. Although there is very little information that is biased against the medical profession, some of the information clearly is biased toward a particular approach, and some is blatantly commercial. A definitive description of this kind of bias was beyond the scope of this study. How can the quality of information available to our patients be controlled? The Health On the Net Foundation currently uses a Code of Conduct for medical websites
1182
CHEN, MINKES, AND LANGER
(HONcode) 9 to unify the quality of medical information on the Internet. The HONcode contains a set of 8 principles that govern the quality of health information on the Internet. Website owners who agree to abide by the principles may download the HONcode logo and display it on their website.10 The Foundation reviews sites bearing the HONcode logo on a regular basis and has mechanisms to identify sites bearing the HONcode logo that have not registered with the Foundation. We did not find the HONcode logo on any of the websites in our study, indicating a lack of awareness of the HONcode among website owners. Knowledge about the HONcode and other similar quality control mechanisms likely will increase with time. Another potential response to the variability in quality
would be an organized approach to providing a source of pediatric surgery information for the lay population through a designated web site. The AAP Section on Surgery or the American Pediatric Surgical Association could provide an online source for comprehensive and accurate information on pediatric surgery that is targeted at the lay population. However, until an organized approach to website quality control is established, it is important that pediatric surgeons warn parents about the variability in quality of the information on the Internet, stressing that not everything they read on the Internet is true or accurate. Giving parents basic, nonmedical criteria by which they can assess website credibility also will encourage safer browsing of the Internet.
REFERENCES 1. http://www.headcount.com: Internet usage around the world. accessed April 28, 1999 2. Eysenbach G, Diepgen TL: Patients looking for information on the Internet and seeking teleadvice. Arch Dermatol 135:151-156, 1999 3. Lehmann CU, Wang DJ, Kim GR, et al: Utilization of a pediatric link collection by health professionals and laypersons. Med Inform 23:53-62, 1998 4. Isaa¨cson M: Medical information on the Internet. J Travel Med 6:52, 1999 5. McClung HJ, Murray RD, Heitlinger LA: The Internet as a source for current patient information. Pediatrics 101:e2, 1998 6. Silberg WM, Lundberg GD, Musacchio RA: Assessing, control-
ling, and assuring the quality of medical information on the Internet. JAMA 277:1244-1245, 1997 7. Murphy PW: Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics 93:460-468, 1994 8. Graber MA, Roller CM, Kaeble B: Readability levels of patient education material on the World Wide Web. J Fam Pract 48:58-61, 1999 9. http://www.hon.ch/Conduct.html: HON Code of Conduct for Medical and Health Websites. accessed April 28, 1999 10. Boyer C, Selby M, Scherrer JR, et al: The Health On the Net Code of Conduct for medical and health websites. Comp Biol Med 28:603-610, 1998