- Email: [email protected]
Burns first aid information on the Internet
Burns 32 (2006) 897–901 www.elsevier.com/locate/burns
Burns first aid information on the Internet§ G. Tiller, S. Rea *, R. Silla, F. Wood Burns Unit, Royal Perth Hospital, Wellington Street, Perth 6000, Australia Accepted 27 February 2006
Abstract The Internet is an increasingly important source of health-related information. However, the growth of the Internet and its use as a medical delivery tool should be viewed with caution. One of the key concerns is that although the volume of information is huge, the quality, accuracy and completeness of the information is questionable. The aim of this study was to evaluate burns first aid information on the Internet. The search term used was ‘first aid for burns’ and the first 25 hits from each search engine were analysed by one of the observers. We gathered basic information on the web sites—such as the country of origin, language in which the information was offered, accessibility, relevance and whether the site was commercial, organisational or academic. Quality and technicality of the web sites were assessed and scored. The mean quality score was 4.7/15 (31.5%) The mean technical score was 6.1 of 12 (51.1%). When the total score was categorised by percentage, none of the web sites ranked in the excellent category, 1 was very good, 4 were good, 6 were fair and the majority, 36, were poor. Based on the quality score only, two web sites were in the excellent category and two were very good. For technicality one web site was excellent and three were very good. This study has shown first aid information on the Internet is largely of poor quality, that the technical information provided is inadequate and that the sites include a significant amount of grossly inaccurate information. The few sites that contain excellent technical information make up a very small proportion of what is available. Therefore, the average Internet user may not encounter these resources, instead gaining knowledge from sites of questionable value. # 2006 Elsevier Ltd and ISBI. All rights reserved. Keywords: Burns; First aid; Internet; Search engines
1. Introduction Use of the Internet throughout the world is increasing, the Internet provides consumers with access to a vast expanse of information on virtually any topic. In Australia, between 2000 and 2005 the percentage of the population using the Internet rose by 115–68.4% [1]. The trend is similar worldwide with numbers of people using the Internet reaching beyond 1 billion at the end of 2005 representing a worldwide increase of 182% [1] over the same 5-year period. The Internet has also become an increasingly important source of health-related information [2]. In 2002 it was estimated that 80% of adult Internet users in the United § The aim of this study is to evaluate first aid information on the Internet with regard to its accessibility, relevance, quality and technical accuracy. * Corresponding author. Tel.: +61 8 404 894186; fax: +61 8 9224 7059. E-mail address: [email protected] (S. Rea).
0305-4179/$30.00 # 2006 Elsevier Ltd and ISBI. All rights reserved. doi:10.1016/j.burns.2006.02.020
States were accessing health-related information [3]. Similarly, a European Union (EU) Eurobarometer survey in 2003 found that approximately one quarter of Europeans use the Internet to access health-related information [4]. We have been warned however, that the growth of the Internet and it’s use as a medical delivery tool should be viewed both positively and critically [5]. The problem is that the volume of information is huge but the quality, accuracy and completeness of the information is questionable, not only in the field of medicine [5]. Several tools have been designed to specifically evaluate health-related sites but the reliability and validity of these is also unclear [6]. Burns are an international public health problem. In the United States 1.25 million people sustain burns each year [7]. In Australia the National Health Survey of 2001 indicated that patients who had received a recent burn or scald made up 8% of people reporting a recent injury and 1% of all Australians [8].
898
G. Tiller et al. / Burns 32 (2006) 897–901
Immediate and appropriate first aid following a burn injury has an impact on the burn outcome, decreasing morbidity and healthcare costs [9], and also influences cosmetic outcome [10]. For this reason it is essential that the population can appropriately administer first aid and for this to occur there needs to be widespread dissemination of reliable and accurate information. Previous studies of single medical conditions suggested that web based health information has limitations [11–13]. To date there has been no evaluation of burns first aid information on the Internet. If we aspire for the community to access the knowledge to administer appropriate burns first aid through the Internet we must be sure that the information available is holistic, accurate and of reliable quality.
2.2. Web site evaluation We gathered basic information on the web sites—such as the country of origin, language in which the information was offered, accessibility, relevance and whether the site was commercial, organisational or academic. Quality and technicality of the web sites were assessed using criteria developed by Boheack et al. [16] and partly by Health on the net code of conduct (HONcode) principles [17] which includes key criteria for assessing health-related web sites described by other authors [18]. The following quality elements were recorded; authorship, qualification, authors contact details, copyright, trademark, reference quantity, reference quality, ownership, responsiTable 2 The quality scoring system Quality
2. Methods 2.1. Identification of web sites In November 2005, over a 2-week period, we searched the world wide web using six popular search engines: Yahoo, Google, MSN, Excite, Alta Vista and Lycos. The search term used was ‘first aid for burns’ and the first 25 hits from each search engine were analysed by one of the observers. To ensure inter observer reliability we initially had each observer evaluate the first 25 sites generated by the MSN search engine and a Kappa score was calculated for a selection of data subsets (Table 1). The observers agreed that the evaluation would be based on the first page that appeared and that links to other locations would not be used unless the initial page was a table of contents [15]. For those sites with a table of contents the first link regarding burns first aid was selected. There was also agreement that sites would be classed as inaccessible if burns first aid information could not be accessed without re-entering the search term. Sites were classed as irrelevant if they provided information unrelated to burns first aid or if they only advertised items for sale. Table 1 Rates of inter observer reliability Information
N
Kappa statistica
Accessibility Relevance Origin of site Quality overall Technicality overall All information
25 25 25 154 182 411
1.000 1.000 0.583 0.694 0.870 0.837
a
Kappa scores less than 0.4 represent poor agreement, a kappa of 0.40– 0.75 represents intermediate to good agreement and a kappa greater than 0.75 represents excellent agreement beyond chance [14].
Authorship Name(s) identified No author identified
1 0
Qualification Healthcare professional Vague qualifications or personal exp No information/no experience
2 1 0
Authors contact address Telephone, e-mail or address No contact info
1 0
Copyright/trademark Present Absent
1 0
Reference quantity Attribution for all factual statements Attribution for some but not all No references
2 1 0
Reference quality Contacts provided for all references Contacts not readily accessible (only name and city) Contacts ambiguous or non functional
2 1 0
Ownership Ownership rights/sponsorship/organisation/company clearly stated No ownership information Responsibility Responsibilities as to the information provided are clearly stated No responsibility policy
1 0 1 0
Purpose Distinction made that the info is for commercial/educational or both Stated as educational but with potential for financial profit No statement of purpose
1 0
Original date Original date provided No date
1 0
Revised updates Any dates of revisal or frequency of updates stated No revision dates provided
1 0
2
G. Tiller et al. / Burns 32 (2006) 897–901
bility, original date and revised updates. The maximum quality score is 15. The scoring system is shown in Table 2. Technical criteria included the presence of background information, burning agent, first aid checklist, accuracy checklist and inaccuracy checklist. The first aid checklist contained the four essential components of burns first aid that have previously been recommended [10]; stop the burning process, cooling the burn, analgesia and covering the wound. Accuracy of the web sites was determined using some of the recommendations for first aid treatment of minor burns made by McCormack et al.; including run cold tap water directly onto the burn for at least 20 min, keep the rest of the patient warm; increase the ambient temperature to 25– 30 8C, remove wet clothing, cover unburnt areas (e.g., with a blanket), continue cooling throughout transport (fine mist spray or frequently changed soaked dressings), never ever use ice, starting first aid within 3 h after a burn is beneficial [19]. If web sites contained grossly inaccurate information the accuracy score was automatically zero. The maximum technical score was 12. The technical criteria and scoring system are shown in Table 3. The total possible score for each web site was 27. The scores from the quality and technical components along with the total score were converted to a percentage and based on these scores a label was assigned to each web site (80% = excellent, 70–79% = very good, 60–69% = good, 50–59% = fair, and <50% = poor) [16]. Table 3 The technical scoring system Technical Background information Any background information No background information
1 0
Types of burn Thermal, chemical, radiation, electrical Three or more types mentioned Two mentioned One mentioned None mentioned
3 2 1 0
First aid checklist Stop the burning process—stop drop and roll Cooling the burn—immersion or irrigation with running tepid water Analgesia—cool water, oral analgesia Covering the burn—cling film, clean cotton sheet, clear plastic bag
1 1
1 1 1 1
Grossly inaccurate, total accuracy score
0
Incorrect information Using ice Using butter/toothpaste
3. Results The total number of web sites evaluated was 150. The 150 were made up of 25 sites from each of the six search engines. Seventy-eight of the sites were duplicates resulting in 72 unique uniform resource locators (URL’s). Some web sites were seen a number of times both within and between search engines. One site, (http://www.healthy.net) was within the top 25 sites from the six search engines 11 times. An additional 7 URL’s were inaccessible and 18 were irrelevant leaving us 47 web sites to evaluate. It is important to note that for the purpose of our study each site was evaluated only once. Of these 47 web sites 32 (68.1%) were classed as organisational, 13 (27.7%) as commercial and 2 (4.3%) as academic. The average overall score of the web sites was 10.9/27 (40.2%) with a standard deviation of 4.1. The average quality score was 4.7/15 (31.5%) with a standard deviation of 3.5. The average technical score was 6.1 of 12 (51.1%) with a standard deviation of 1.7. When the total score was categorised by percentage none of the web sites ranked in the excellent category, 1 was very good, 4 were good, 6 were fair and the majority, 36, were poor. Based on the quality score only, two web sites were in the excellent category and two were very good. For technicality one web site was excellent and three were very good (Fig. 1). The HONcode logo was found in only 2 of the 47 sites (4.3%). Of the 47 web sites 37 (78.7%) had no author identified and the author was identified as a health professional in only 5 of the 47 sites (10.6%). References were not provided in 42 of 47 (89.4%) web sites. The four components of burns first aid were mentioned in varying frequencies. Stopping the burning process, including the concept of stop, drop and roll was mentioned in 17 of 47 (36.2%). Analgesia was mentioned in 13 of 47 (27.7%) and covering the burn in 41 of 47 (87.2%). Cooling the burn was the most frequently mentioned component of burns’ first aid, covered in 45 of 47 (95.7%) sites. In terms of accuracy of information, only 1 of 47 (2.1%) stated an appropriate water temperature. Three of 47 (6.4%)
1 1
Accuracy checklist Cold water at 5–258 Cold water no temperature specified For at least 20 min Within 3 h of injury
1 1
899
Fig. 1. The percent ranking of the web sites for the three scores.
900
G. Tiller et al. / Burns 32 (2006) 897–901
stated that the cooling should continue for at least 20 min and only 1 of 47 (2.1%) stated that cooling was beneficial within 3 h of injury. Gross inaccuracies were found in 16 of 47 (34.0%). Errors most often related to the advice to avoid cooling large or deep burns for example, ‘For third degree burns. After removing the heat source cover the area in a clean, dry dressing’ (http://www.homeofpoi.com). None of the web sites suggested that toothpaste or butter should be applied to the burn and only 1 of 47 (2.1%) mentioned that ice could be used as a cooling source.
4. Discussion Health information on the Internet has been evaluated in the context of a wide range of medical scenarios. The perceived need for these evaluations has evolved because of the ease at which information can be displayed on the Internet [15] and the absence of a formal process of review [11,15]. The conclusions from these studies have either been that good quality information is available, but is difficult to locate, and is small in number [20,21] or that information is of poor quality, and poor informational value [13]. As outlined, our analysis was based on the first 25 web sites from each of the search engines. We selected this small number of sites in an attempt to limit the frequency of duplicates within each search engine. Although this was largely the case, the problem was not eliminated as the major source of duplication arose between search engines. In retrospect 10 web sites from each engine may have been sufficient as typical web users do not look beyond the first page of search results [21]. There was some discrepancy between the observers with regard to the sites origin reflected in the lower kappa value. The authors felt that this arose because of the arbitrary distinction between organisational and commercial sites and the very small number of academic sites. It is interesting to note that none of the 25 sites from the MSN search engine, evaluated by both observers, were academic. The quality of web based health information has attracted considerable discussion and several tools have been formulated in an attempt to allow consumers to better understand the nature of the information they are accessing [22]. Health on The Net code of conduct [17] was developed by the health on the net foundation and is the oldest and perhaps best known, quality label [22]. It aims to standardise the quality of health information on the Internet by providing the HONcode logo, which owners display on their site, demonstrating their desire to provide information of a certain standard [17]. HONcode was one of the instruments evaluated by Jadad and Gagliardi who concluded that it was unclear wether these rating systems should exist, wether they measure what they claim to measure and wether they do more good than harm [6]. Of the 47 web sites we evaluated, only 2 (4.3%) displayed the HONcode symbol. One of the
two, although scoring highly in the quality component of our score failed to score highly in the technical component, which supports the suggestion that the value of such an assessment system is unclear. Interestingly, the only other site showing the HONcode symbol failed to score highly in the quality component of our assessment with a score of 6/15 (40.0%) corresponding with a quality ranking of poor. In an area such as burns first aid there is no standard upon which to base an assessment of accuracy and completeness of Internet information. For this reason we created our technical tool, consisting of completeness and accuracy checklists, from two different published criteria. The appropriate water temperature for cooling a burn is an area of controversy which prompted us to select a broad range, 5– 258 [19] in an attempt to encompass all temperatures other than iced water. Making use of an accuracy checklist highlighted the fact that although cooling with running water was mentioned in the majority of sites, how cold, how long and how soon was not. A review of minor burns patients by Rea et al. found that a significant number of patients had applied ice directly onto their wound. A smaller group had used other treatments, honey and toothpaste being the most common [23]. Our incorrect information checklist included mentioning ice or other treatments such as butter or toothpaste as adequate burns first aid. These remedies are known to have no effect on cooling and may contribute to tissue necrosis [9]. Given that these incorrect treatments have been demonstrated as commonly used remedies in the community it was interesting to note that our Internet search failed to locate this information. Other authors have commented that the public has a sub optimal understanding of cold water therapy for burns and that it is difficult to know why practices such as the application of butter exist [9]. One interesting point regarding inaccurate information was the commonly held belief that one should not apply cold water to large and deeper burns. While it is obviously important to seek medical attention as soon as possible in these situations, cooling has been shown to be of benefit to all burns patients regardless of size and depth [9,10]. The only caveat is the important need to avoid hypothermia [10]. It is appropriate to mention some of this study’s limitations. This study used a simple search strategy to locate burns first aid information using only one search term. Had more complex search strategies been used the information available for evaluation may have been different. Secondly, we used specific criteria to evaluate the web sites ensuring that some sites scored poorly overall even if their information was of excellent quality or technicality. For example, one web site (http://www.baysidehealth.org.au) had a technical score of 83% (10/12) and was the only site to mention an appropriate water temperature and that cooling was beneficial within 3 h of injury but scored only 7% (1/15) for quality, making its overall ranking poor. Thirdly, entering our search term in English may have limited the scope of information that was available for our assessment. We acknowledge that this
G. Tiller et al. / Burns 32 (2006) 897–901
excludes a large number of web sites including many professional web sites which provide burns first aid information in languages other than English. Establishing a recognised standard of burns first aid would be an important step towards improving the accuracy of information provided by websites. The standard should detail the key elements of the cooling process; ‘‘how long, how cold and how soon’’. As mentioned previously, the value of website quality codes is unclear. A standard of burns first aid care has the potential to increase the value of these codes as it could be incorporated as an additional criterion for meeting the standard. Access to websites providing quality information could be improved by extending the concept of quality codes. If an affiliation with a quality code was clearly displayed in the page description following a search engine query, this may help to locate sites of higher quality. Ideally, if quality codes could be recognised by search engines this would aid information filtration, a concept that would be applicable across all fields of medicine. Our study found that burns first aid information on the Internet is largely of poor quality, that the technical information provided is inadequate and that the sites include a significant amount of grossly inaccurate information. The few sites that contain excellent technical information make up a very small proportion of what is available. Therefore, the average Internet user may not encounter these resources, instead gaining knowledge from sites of questionable value.
Acknowledgements S. Rea is supported by a joint British Association of Plastic Surgeons/Royal College of Surgeons in Ireland Travelling Grant in Plastic Surgery. We acknowledge the support of Bess Fowler.
References [1] Internet World Stats, Usage and Population Statistics. [2] Berland GK, Elliott MN, Morales LS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001;285(20):2612–21. [3] McMullan M. Patients using the Internet to obtain health information: how this affects the patient-health professional relationship. Patient Educ Couns 2006.
901
[4] Survey of Online Health Information, European Commission. [5] Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor–Let the reader and viewer beware. JAMA 1997;277(15):1244–5. [6] Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA 1998;279(8):611–4. [7] Brigham PA, McLoughlin E. Burn incidence and medical care use in the United States: estimates, trends, and data sources. J Burn Care Rehabil 1996;17(2):95–107. [8] Rea S, Kuthubutheen B, Fowler B, Wood F. Burn first aid in Western Australia—do healthcare workers have the knowledge? Burns 2005. [9] Skinner A, Peat B. Burns treatment for children and adults: a study of initial burns first aid and hospital care. N Z Med J 2002;115(1163): U199. [10] Hudspith J, Rayatt S. First aid and treatment of minor burns. BMJ 2004;328(7454):1487–9. [11] Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ 2000;321(7275): 1511–5. [12] Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ 1997;314(7098):1875–9. [13] Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome.. J Bone Joint Surg Am 2000;82A(11):1540–3. [14] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159–74. [15] Jejurikar SS, Rovak JM, Kuzon Jr WM, et al. Evaluation of plastic surgery information on the Internet. Ann Plast Surg 2002;49(5):460–5. [16] Bohacek L, Gomez M, Fish JS. An evaluation of internet sites for burn scar management. J Burn Care Rehabil 2003;24(4):246–51 [discussion 245]. [17] Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med 1998;28(5):603–10. [18] Kim P, Eng TR, Deering MJ, Maxfield A. Published criteria for evaluating health related web sites: review. BMJ 1999;318(7184): 647–9. [19] McCormack RA, La Hei ER, Martin HC. First-aid management of minor burns in children: a prospective study of children presenting to the Children’s Hospital at Westmead, Sydney. Med J Aust 2003; 178(1):31–3. [20] Sandvik H. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ 1999;319(7201):29–32. [21] Isaac D, Cusimano MD, Sherman A, Chipman M. Child safety education and the world wide web: an evaluation of the content and quality of online resources. Inj Prev 2004;10(1):59–61. [22] Wilson P. How to find the good and avoid the bad or ugly: a short guide to tools for rating quality of health information on the internet. BMJ 2002;324(7337):598–602. [23] Rea SM, Wood FM. Minor burn injuries in adults presenting to the regional burns unit in Western Australia; a prospective descriptive study. Burns 2005.
Burns first aid information on the Internet§ G. Tiller, S. Rea *, R. Silla, F. Wood Burns Unit, Royal Perth Hospital, Wellington Street, Perth 6000, Australia Accepted 27 February 2006
Abstract The Internet is an increasingly important source of health-related information. However, the growth of the Internet and its use as a medical delivery tool should be viewed with caution. One of the key concerns is that although the volume of information is huge, the quality, accuracy and completeness of the information is questionable. The aim of this study was to evaluate burns first aid information on the Internet. The search term used was ‘first aid for burns’ and the first 25 hits from each search engine were analysed by one of the observers. We gathered basic information on the web sites—such as the country of origin, language in which the information was offered, accessibility, relevance and whether the site was commercial, organisational or academic. Quality and technicality of the web sites were assessed and scored. The mean quality score was 4.7/15 (31.5%) The mean technical score was 6.1 of 12 (51.1%). When the total score was categorised by percentage, none of the web sites ranked in the excellent category, 1 was very good, 4 were good, 6 were fair and the majority, 36, were poor. Based on the quality score only, two web sites were in the excellent category and two were very good. For technicality one web site was excellent and three were very good. This study has shown first aid information on the Internet is largely of poor quality, that the technical information provided is inadequate and that the sites include a significant amount of grossly inaccurate information. The few sites that contain excellent technical information make up a very small proportion of what is available. Therefore, the average Internet user may not encounter these resources, instead gaining knowledge from sites of questionable value. # 2006 Elsevier Ltd and ISBI. All rights reserved. Keywords: Burns; First aid; Internet; Search engines
1. Introduction Use of the Internet throughout the world is increasing, the Internet provides consumers with access to a vast expanse of information on virtually any topic. In Australia, between 2000 and 2005 the percentage of the population using the Internet rose by 115–68.4% [1]. The trend is similar worldwide with numbers of people using the Internet reaching beyond 1 billion at the end of 2005 representing a worldwide increase of 182% [1] over the same 5-year period. The Internet has also become an increasingly important source of health-related information [2]. In 2002 it was estimated that 80% of adult Internet users in the United § The aim of this study is to evaluate first aid information on the Internet with regard to its accessibility, relevance, quality and technical accuracy. * Corresponding author. Tel.: +61 8 404 894186; fax: +61 8 9224 7059. E-mail address: [email protected] (S. Rea).
0305-4179/$30.00 # 2006 Elsevier Ltd and ISBI. All rights reserved. doi:10.1016/j.burns.2006.02.020
States were accessing health-related information [3]. Similarly, a European Union (EU) Eurobarometer survey in 2003 found that approximately one quarter of Europeans use the Internet to access health-related information [4]. We have been warned however, that the growth of the Internet and it’s use as a medical delivery tool should be viewed both positively and critically [5]. The problem is that the volume of information is huge but the quality, accuracy and completeness of the information is questionable, not only in the field of medicine [5]. Several tools have been designed to specifically evaluate health-related sites but the reliability and validity of these is also unclear [6]. Burns are an international public health problem. In the United States 1.25 million people sustain burns each year [7]. In Australia the National Health Survey of 2001 indicated that patients who had received a recent burn or scald made up 8% of people reporting a recent injury and 1% of all Australians [8].
898
G. Tiller et al. / Burns 32 (2006) 897–901
Immediate and appropriate first aid following a burn injury has an impact on the burn outcome, decreasing morbidity and healthcare costs [9], and also influences cosmetic outcome [10]. For this reason it is essential that the population can appropriately administer first aid and for this to occur there needs to be widespread dissemination of reliable and accurate information. Previous studies of single medical conditions suggested that web based health information has limitations [11–13]. To date there has been no evaluation of burns first aid information on the Internet. If we aspire for the community to access the knowledge to administer appropriate burns first aid through the Internet we must be sure that the information available is holistic, accurate and of reliable quality.
2.2. Web site evaluation We gathered basic information on the web sites—such as the country of origin, language in which the information was offered, accessibility, relevance and whether the site was commercial, organisational or academic. Quality and technicality of the web sites were assessed using criteria developed by Boheack et al. [16] and partly by Health on the net code of conduct (HONcode) principles [17] which includes key criteria for assessing health-related web sites described by other authors [18]. The following quality elements were recorded; authorship, qualification, authors contact details, copyright, trademark, reference quantity, reference quality, ownership, responsiTable 2 The quality scoring system Quality
2. Methods 2.1. Identification of web sites In November 2005, over a 2-week period, we searched the world wide web using six popular search engines: Yahoo, Google, MSN, Excite, Alta Vista and Lycos. The search term used was ‘first aid for burns’ and the first 25 hits from each search engine were analysed by one of the observers. To ensure inter observer reliability we initially had each observer evaluate the first 25 sites generated by the MSN search engine and a Kappa score was calculated for a selection of data subsets (Table 1). The observers agreed that the evaluation would be based on the first page that appeared and that links to other locations would not be used unless the initial page was a table of contents [15]. For those sites with a table of contents the first link regarding burns first aid was selected. There was also agreement that sites would be classed as inaccessible if burns first aid information could not be accessed without re-entering the search term. Sites were classed as irrelevant if they provided information unrelated to burns first aid or if they only advertised items for sale. Table 1 Rates of inter observer reliability Information
N
Kappa statistica
Accessibility Relevance Origin of site Quality overall Technicality overall All information
25 25 25 154 182 411
1.000 1.000 0.583 0.694 0.870 0.837
a
Kappa scores less than 0.4 represent poor agreement, a kappa of 0.40– 0.75 represents intermediate to good agreement and a kappa greater than 0.75 represents excellent agreement beyond chance [14].
Authorship Name(s) identified No author identified
1 0
Qualification Healthcare professional Vague qualifications or personal exp No information/no experience
2 1 0
Authors contact address Telephone, e-mail or address No contact info
1 0
Copyright/trademark Present Absent
1 0
Reference quantity Attribution for all factual statements Attribution for some but not all No references
2 1 0
Reference quality Contacts provided for all references Contacts not readily accessible (only name and city) Contacts ambiguous or non functional
2 1 0
Ownership Ownership rights/sponsorship/organisation/company clearly stated No ownership information Responsibility Responsibilities as to the information provided are clearly stated No responsibility policy
1 0 1 0
Purpose Distinction made that the info is for commercial/educational or both Stated as educational but with potential for financial profit No statement of purpose
1 0
Original date Original date provided No date
1 0
Revised updates Any dates of revisal or frequency of updates stated No revision dates provided
1 0
2
G. Tiller et al. / Burns 32 (2006) 897–901
bility, original date and revised updates. The maximum quality score is 15. The scoring system is shown in Table 2. Technical criteria included the presence of background information, burning agent, first aid checklist, accuracy checklist and inaccuracy checklist. The first aid checklist contained the four essential components of burns first aid that have previously been recommended [10]; stop the burning process, cooling the burn, analgesia and covering the wound. Accuracy of the web sites was determined using some of the recommendations for first aid treatment of minor burns made by McCormack et al.; including run cold tap water directly onto the burn for at least 20 min, keep the rest of the patient warm; increase the ambient temperature to 25– 30 8C, remove wet clothing, cover unburnt areas (e.g., with a blanket), continue cooling throughout transport (fine mist spray or frequently changed soaked dressings), never ever use ice, starting first aid within 3 h after a burn is beneficial [19]. If web sites contained grossly inaccurate information the accuracy score was automatically zero. The maximum technical score was 12. The technical criteria and scoring system are shown in Table 3. The total possible score for each web site was 27. The scores from the quality and technical components along with the total score were converted to a percentage and based on these scores a label was assigned to each web site (80% = excellent, 70–79% = very good, 60–69% = good, 50–59% = fair, and <50% = poor) [16]. Table 3 The technical scoring system Technical Background information Any background information No background information
1 0
Types of burn Thermal, chemical, radiation, electrical Three or more types mentioned Two mentioned One mentioned None mentioned
3 2 1 0
First aid checklist Stop the burning process—stop drop and roll Cooling the burn—immersion or irrigation with running tepid water Analgesia—cool water, oral analgesia Covering the burn—cling film, clean cotton sheet, clear plastic bag
1 1
1 1 1 1
Grossly inaccurate, total accuracy score
0
Incorrect information Using ice Using butter/toothpaste
3. Results The total number of web sites evaluated was 150. The 150 were made up of 25 sites from each of the six search engines. Seventy-eight of the sites were duplicates resulting in 72 unique uniform resource locators (URL’s). Some web sites were seen a number of times both within and between search engines. One site, (http://www.healthy.net) was within the top 25 sites from the six search engines 11 times. An additional 7 URL’s were inaccessible and 18 were irrelevant leaving us 47 web sites to evaluate. It is important to note that for the purpose of our study each site was evaluated only once. Of these 47 web sites 32 (68.1%) were classed as organisational, 13 (27.7%) as commercial and 2 (4.3%) as academic. The average overall score of the web sites was 10.9/27 (40.2%) with a standard deviation of 4.1. The average quality score was 4.7/15 (31.5%) with a standard deviation of 3.5. The average technical score was 6.1 of 12 (51.1%) with a standard deviation of 1.7. When the total score was categorised by percentage none of the web sites ranked in the excellent category, 1 was very good, 4 were good, 6 were fair and the majority, 36, were poor. Based on the quality score only, two web sites were in the excellent category and two were very good. For technicality one web site was excellent and three were very good (Fig. 1). The HONcode logo was found in only 2 of the 47 sites (4.3%). Of the 47 web sites 37 (78.7%) had no author identified and the author was identified as a health professional in only 5 of the 47 sites (10.6%). References were not provided in 42 of 47 (89.4%) web sites. The four components of burns first aid were mentioned in varying frequencies. Stopping the burning process, including the concept of stop, drop and roll was mentioned in 17 of 47 (36.2%). Analgesia was mentioned in 13 of 47 (27.7%) and covering the burn in 41 of 47 (87.2%). Cooling the burn was the most frequently mentioned component of burns’ first aid, covered in 45 of 47 (95.7%) sites. In terms of accuracy of information, only 1 of 47 (2.1%) stated an appropriate water temperature. Three of 47 (6.4%)
1 1
Accuracy checklist Cold water at 5–258 Cold water no temperature specified For at least 20 min Within 3 h of injury
1 1
899
Fig. 1. The percent ranking of the web sites for the three scores.
900
G. Tiller et al. / Burns 32 (2006) 897–901
stated that the cooling should continue for at least 20 min and only 1 of 47 (2.1%) stated that cooling was beneficial within 3 h of injury. Gross inaccuracies were found in 16 of 47 (34.0%). Errors most often related to the advice to avoid cooling large or deep burns for example, ‘For third degree burns. After removing the heat source cover the area in a clean, dry dressing’ (http://www.homeofpoi.com). None of the web sites suggested that toothpaste or butter should be applied to the burn and only 1 of 47 (2.1%) mentioned that ice could be used as a cooling source.
4. Discussion Health information on the Internet has been evaluated in the context of a wide range of medical scenarios. The perceived need for these evaluations has evolved because of the ease at which information can be displayed on the Internet [15] and the absence of a formal process of review [11,15]. The conclusions from these studies have either been that good quality information is available, but is difficult to locate, and is small in number [20,21] or that information is of poor quality, and poor informational value [13]. As outlined, our analysis was based on the first 25 web sites from each of the search engines. We selected this small number of sites in an attempt to limit the frequency of duplicates within each search engine. Although this was largely the case, the problem was not eliminated as the major source of duplication arose between search engines. In retrospect 10 web sites from each engine may have been sufficient as typical web users do not look beyond the first page of search results [21]. There was some discrepancy between the observers with regard to the sites origin reflected in the lower kappa value. The authors felt that this arose because of the arbitrary distinction between organisational and commercial sites and the very small number of academic sites. It is interesting to note that none of the 25 sites from the MSN search engine, evaluated by both observers, were academic. The quality of web based health information has attracted considerable discussion and several tools have been formulated in an attempt to allow consumers to better understand the nature of the information they are accessing [22]. Health on The Net code of conduct [17] was developed by the health on the net foundation and is the oldest and perhaps best known, quality label [22]. It aims to standardise the quality of health information on the Internet by providing the HONcode logo, which owners display on their site, demonstrating their desire to provide information of a certain standard [17]. HONcode was one of the instruments evaluated by Jadad and Gagliardi who concluded that it was unclear wether these rating systems should exist, wether they measure what they claim to measure and wether they do more good than harm [6]. Of the 47 web sites we evaluated, only 2 (4.3%) displayed the HONcode symbol. One of the
two, although scoring highly in the quality component of our score failed to score highly in the technical component, which supports the suggestion that the value of such an assessment system is unclear. Interestingly, the only other site showing the HONcode symbol failed to score highly in the quality component of our assessment with a score of 6/15 (40.0%) corresponding with a quality ranking of poor. In an area such as burns first aid there is no standard upon which to base an assessment of accuracy and completeness of Internet information. For this reason we created our technical tool, consisting of completeness and accuracy checklists, from two different published criteria. The appropriate water temperature for cooling a burn is an area of controversy which prompted us to select a broad range, 5– 258 [19] in an attempt to encompass all temperatures other than iced water. Making use of an accuracy checklist highlighted the fact that although cooling with running water was mentioned in the majority of sites, how cold, how long and how soon was not. A review of minor burns patients by Rea et al. found that a significant number of patients had applied ice directly onto their wound. A smaller group had used other treatments, honey and toothpaste being the most common [23]. Our incorrect information checklist included mentioning ice or other treatments such as butter or toothpaste as adequate burns first aid. These remedies are known to have no effect on cooling and may contribute to tissue necrosis [9]. Given that these incorrect treatments have been demonstrated as commonly used remedies in the community it was interesting to note that our Internet search failed to locate this information. Other authors have commented that the public has a sub optimal understanding of cold water therapy for burns and that it is difficult to know why practices such as the application of butter exist [9]. One interesting point regarding inaccurate information was the commonly held belief that one should not apply cold water to large and deeper burns. While it is obviously important to seek medical attention as soon as possible in these situations, cooling has been shown to be of benefit to all burns patients regardless of size and depth [9,10]. The only caveat is the important need to avoid hypothermia [10]. It is appropriate to mention some of this study’s limitations. This study used a simple search strategy to locate burns first aid information using only one search term. Had more complex search strategies been used the information available for evaluation may have been different. Secondly, we used specific criteria to evaluate the web sites ensuring that some sites scored poorly overall even if their information was of excellent quality or technicality. For example, one web site (http://www.baysidehealth.org.au) had a technical score of 83% (10/12) and was the only site to mention an appropriate water temperature and that cooling was beneficial within 3 h of injury but scored only 7% (1/15) for quality, making its overall ranking poor. Thirdly, entering our search term in English may have limited the scope of information that was available for our assessment. We acknowledge that this
G. Tiller et al. / Burns 32 (2006) 897–901
excludes a large number of web sites including many professional web sites which provide burns first aid information in languages other than English. Establishing a recognised standard of burns first aid would be an important step towards improving the accuracy of information provided by websites. The standard should detail the key elements of the cooling process; ‘‘how long, how cold and how soon’’. As mentioned previously, the value of website quality codes is unclear. A standard of burns first aid care has the potential to increase the value of these codes as it could be incorporated as an additional criterion for meeting the standard. Access to websites providing quality information could be improved by extending the concept of quality codes. If an affiliation with a quality code was clearly displayed in the page description following a search engine query, this may help to locate sites of higher quality. Ideally, if quality codes could be recognised by search engines this would aid information filtration, a concept that would be applicable across all fields of medicine. Our study found that burns first aid information on the Internet is largely of poor quality, that the technical information provided is inadequate and that the sites include a significant amount of grossly inaccurate information. The few sites that contain excellent technical information make up a very small proportion of what is available. Therefore, the average Internet user may not encounter these resources, instead gaining knowledge from sites of questionable value.
Acknowledgements S. Rea is supported by a joint British Association of Plastic Surgeons/Royal College of Surgeons in Ireland Travelling Grant in Plastic Surgery. We acknowledge the support of Bess Fowler.
References [1] Internet World Stats, Usage and Population Statistics. [2] Berland GK, Elliott MN, Morales LS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001;285(20):2612–21. [3] McMullan M. Patients using the Internet to obtain health information: how this affects the patient-health professional relationship. Patient Educ Couns 2006.
901
[4] Survey of Online Health Information, European Commission. [5] Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor–Let the reader and viewer beware. JAMA 1997;277(15):1244–5. [6] Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA 1998;279(8):611–4. [7] Brigham PA, McLoughlin E. Burn incidence and medical care use in the United States: estimates, trends, and data sources. J Burn Care Rehabil 1996;17(2):95–107. [8] Rea S, Kuthubutheen B, Fowler B, Wood F. Burn first aid in Western Australia—do healthcare workers have the knowledge? Burns 2005. [9] Skinner A, Peat B. Burns treatment for children and adults: a study of initial burns first aid and hospital care. N Z Med J 2002;115(1163): U199. [10] Hudspith J, Rayatt S. First aid and treatment of minor burns. BMJ 2004;328(7454):1487–9. [11] Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ 2000;321(7275): 1511–5. [12] Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ 1997;314(7098):1875–9. [13] Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome.. J Bone Joint Surg Am 2000;82A(11):1540–3. [14] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159–74. [15] Jejurikar SS, Rovak JM, Kuzon Jr WM, et al. Evaluation of plastic surgery information on the Internet. Ann Plast Surg 2002;49(5):460–5. [16] Bohacek L, Gomez M, Fish JS. An evaluation of internet sites for burn scar management. J Burn Care Rehabil 2003;24(4):246–51 [discussion 245]. [17] Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med 1998;28(5):603–10. [18] Kim P, Eng TR, Deering MJ, Maxfield A. Published criteria for evaluating health related web sites: review. BMJ 1999;318(7184): 647–9. [19] McCormack RA, La Hei ER, Martin HC. First-aid management of minor burns in children: a prospective study of children presenting to the Children’s Hospital at Westmead, Sydney. Med J Aust 2003; 178(1):31–3. [20] Sandvik H. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ 1999;319(7201):29–32. [21] Isaac D, Cusimano MD, Sherman A, Chipman M. Child safety education and the world wide web: an evaluation of the content and quality of online resources. Inj Prev 2004;10(1):59–61. [22] Wilson P. How to find the good and avoid the bad or ugly: a short guide to tools for rating quality of health information on the internet. BMJ 2002;324(7337):598–602. [23] Rea SM, Wood FM. Minor burn injuries in adults presenting to the regional burns unit in Western Australia; a prospective descriptive study. Burns 2005.