- Email: [email protected]
Evaluation of the quality of self-education mammography material available for patients on the internet
Evaluation of the Quality of Self-Education Mammography Material Available for Patients on the Internet I Eric P. Tamm, MD, Bharat K. Raval, MD, Phan T. Huynh, MD
Rationale and Objectives. The Internet offers many advantages for educating patients but has no standards for publication. This limitation could negatively affect patient care. The purpose of this study was to evaluate the quality of information on mammography that a patient could find on the Internet. Materials and Methods. Three search utilities were used to research the term "mammography." For each utility, the first 50 addresses for Web pages were evaluated (or all the addresses, if fewer than 50 were returned). Web sites selected contained information that could guide an asymptomatic woman in deciding whether to undergo screening mammography. These sites were then evaluated for whether they indicated their sponsorship, authorship, the currency of information, and references. Sites were also noted if they advised women older than 50 years to undergo screening mammography at intervals of longer than 1 year or otherwise severely diminished the role of mammography. Results. Thirty-eight Web sites were identified. Ten indicated authorship, 29 indicated the currency of the information, and 27 provided references. All of the Web sites indicated sponsorship. Three sites recommended screening mammography at intervals of longer than 1 year for women 50 years of age or older. Two sites suggested that mammography is not substantially more sensitive than physical examination. Conclusion. Many Web sites do not meet the standards for disseminating information required in professional peer-reviewed journals. Some Web sites contain statements that might lead asymptomatic women over age 50 years to delay screening mammography or to undergo screening at intervals of longer than 1 year.
Key Words. Breast radiography; Computers, teaching aid; Internet.
Mammography is a unique area in radiology, in that patients can refer themselves directly for mammography. Screening m a m m o g r a p h y has been shown to decrease mortality by 33% in women 5 0 - 6 9 years of age. The American Cancer Society and the American College of Radiology currently advise annual screening m a m m o graphy for all women over age 40 years, but there has been controversy regarding w o m e n in the 4 0 - 4 9 - y e a r age group (1-4). Increasing patient awareness could increase compliance, but misinformation could delay when
Acad Radiol 2000; 7:137-141
1 From the Department of Radiology, University of Texas at Houston Medical School, 6431 Fannin, MSB 2.100, Houston, TX 77030. Received June 7, 1999; revision requested July 15; revision received August 16; accepted August 18. Address correspondence to E.P.T. © AUR, 2000
a woman chooses to undergo mammography. Therefore, the quality of information available to patients is a concern for radiologists, both because such information may motivate patient behavior and because patients may turn to radiologists to answer questions raised by the information they have found. A variety of health education resources for patients have become available on the Internet. Recent articles have raised concerns about the quality of such medical information (5-7). A Federal Trade Commission "Surf Day," dedicated to exploring health-related Web sites, found over 400 Web sites that were thought to promote questionable health products or services (5). Our objective was to evaluate the quality of information on mammography that a patient could find on the Internet. We sought to evaluate the results obtained with different search utilities and to assess the quality of information
137
provided by a sample of Web sites that contained information on mammography. A search utility is a computer program that allows computer users to enter search terms and returns addresses for Web pages that contain matching terms. Search utilities have become popular among computer users accessing the Internet, and the results they produce can steer the public toward, or away from, useful sources of information. A Web site is a collection of Web files and its associated data stored on a computer and readily accessed by other computers that have access to the Internet. Our focus was on recommendations for mammography for asymptomatic women over 50 years of age.
~IATERIALS AND METHOD.c In January 1998, the Internet information company Mecklermedia identified Yahoo, Infoseek, and Excite as the three most popular search utilities, based on the results of data acquired by two companies, Relevant Knowledge and Media Metrix (8). The two companies used software to monitor the activities of a group of Web users (6,000 for RelevantKnowledge and 10,000 for Media Metrix). At the time these data were pub-i lished, Netscape was not listed as a search utility by Mecklermedia. On March 16, 1998, we submitted the word "mammography" to each of these three search utilities. We then took the first 50 search results, which were addresses for Web pages (otherwise known as uniform resource locators, or URLs). If fewer than 50 addresses were produced by a query, we then evaluated all of the addresses. (The number 50 was arbitrarily chosen because we thought that the average computer user would probably not visit more than the first 50 addresses.)
ing organizations: commercial (selling medical products), health care provider (eg, hospital, medical school, private practice), government agency, health care association (other than health care provider), or "other."
Evaluation Criteria for Web Sites Each Web site was reviewed to determine whether it contained information that could guide an asymptomatic woman in deciding whether to undergo screening mammography. To evaluate Web sites fully, we reviewed the entire site identified by a search utility, not just the particular page identified. The Web sites found to have such content were then evaluated according to criteria suggested in a recent article as basicstandards for publicly distributed information on the Internet (7). These included (a) authorship (whether or not the author was clearly identified); (b) attribution (whether or not references were provided for the information for the article in question; (c) disclosure (whether a Web site indicated ownership); and (d) currency (whether or not the date of publication or last update was provided). Each Web site was also evaluated for the presence of information that advised mammography at intervals less fre-~ quent than yearly for women 50 years of age or older. This criterion was used because annual screening mammography for this age group is recommended by such organizations as the American Cancer Society (9). Because of controversy as to the role of screening mammography for women in the 40-49-year age group, we did not evaluate the information provided for this group (10). We also noted statements that denied or severely diminished the role of screening mammography in decreasing morbidity and mortality due to breast cancer in women. Evaluation was by consensus of three radiologists.
Evaluation Criteria for Search Engines Each search utility was evaluated quantitatively with regard to the following: (a) the total number of Web addresses found for the word "mammography"; (b) the number of addresses that were current (whether the page in question was still accessible at the found address); (c) the number of unique Web sites identified in the first 50 addresses found, or in all found addresses if the total was less than 50 (ie, the number of Web sites found after duplicate references to the same Web addresses had been removed); (d) the number of Web sites that provided guidance for an asymptomatic woman in deciding whether to undergo screening mammography; and (e) the distribution of Web sites found with respect to the types of sponsor-
138
Search Engines The Table lists the results from evaluation of the search engines. Figure 1 shows the distribution of Web sites by type of sponsor, for the first 50 addresses found for each search engine (or all found addresses if the total was less than 50).
Web Sites The three search utilities identified 38 Web sites with information on screening mammography that could influence an asymptomatic patient deciding whether to undergo mammography. Web sites were excluded if they merely
Results from Evaluation of Search Utilities
Finding
Yahoo
Infoseek
Excite
Total no. of Web addresses (Web pages) found No. of unique Web sites (nonduplicates) among first 50 found Web addresses (all if < 50) No. of out-of-date Web addresses No. of addresses for which Web site no longer existed No. of unique Web sites that had screening mammography information relevant to patients
40 25/40 (62.5)
10,402 45/50 (90)
8,490 34/50 (68)
3/40 1/40
5/50 3/50
6/50 0/50 26/45 (56)
5/25 (20)
10/34 (29)
Note.--Percentages in parentheses.
Figure 1. Distribution of sponsor types for Web site addresses found by Yahoo (dark gray bars), Excite (light gray bars), and Infoseek (white bars): health care providers, health care organizations (those not directly providing patient care), government agencies, commercial (companies selling mammography equipment), and other (not previously specified).
70%
60% 50% 40% 30% 20%
..........
J
I ~!ILII::]I
10% 0% health care health care government commercial provider assoc, agency 100% (38) 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Figure 2. Percentage of Web sites that indicated authorship, sources ("attribution," with separate category for full bibliographies), currency of information, and sponsoring organization (ownership) for their information on screening mammography. Numbers in parentheses indicate numbers of sites.
other
provided addresses ("links") to other Web sites containing such information. Web sites were excluded if they discussed mammography without providing guidance as to when women should seek screening mammography. Figure 2 shows the percentages of the 38 Web sites that indicated authorship, currency, and sponsorship and provided references for sources and indicated sponsorship. Figure 3 shows the distribution of sponsor types for these 38 Web sites. Of the 38 Web sites, five (13%) contained statements that were contrary to the criteria given above. Two sites recommended mammography be performed every 2-3 years, and one recommended mammography every 1-2 years for women over age 50 years; all three of these sites were located outside the United States. One article at one site suggested that physical examination was as sensitive as mammography and that mammography was unnecessary. One article at another site stated that the benefits of mammography versus those of breast self-examination are "not well established."
139
T A M M ET A L
Academic Radiology, Vol 7, No 3, March 2000
other s'~ A N /
Radiologists expect their professional literature to meet certain accepted standards for publication, including indications of authorship, the publishing organization, date of publication, and references. On the Internet, however, Web site developers can serve anonymously in multiple roles, usurping the positions of authors, editors, and publishers. Of the 38 Web sites that we found with information that could influence a woman's decision to undergo screening mammography, only 10 (26%) indicated authorship. These 38 sites also varied as to whether they provided other basic information (Fig 2); 29 (76%) indicated the timeliness of their information, while 27 (71%) provided references to the sources of their information. Five (13%) of the 38 sites contained information that could encourage a 50-year-old woman to delay undergoing screening mammography. Two suggested that mammography is not substantially more sensitive than physical examination. Patients exploring the Internet for health-related information should therefore judge Web sites by the same standards that radiologists require for their publications. Patients should also evaluate Web sites for the types of sponsoring organizations. In our research, 100% of the 38 Web sites indicated the sponsoring organization, with a variety of types of sponsors (Fig 3). Three sites, all located outside the United States in English-speaking countries, advised women over 50 years of age to undergo screening mammography at intervals of longer than 1' year. Because of the Internet's c@acity to reach Web sites anywhere in the world, and the tendency for English to be used internationally as the language for publication to the Internet, patients in the United States may cross "international boundaries" without realizing they have done so. They may then encounter information developed by a foreign Web site for the population of its country and mistakenly think it appropriate for them. Search utilities are popular means of finding Web sites. In searching for the term "mammography," we found very few sites that mentioned mammography only incidentally, but we did find many commercial sites that contained information only on mammography equipment. Our research also showed drastic differences between search utilities, both in terms of the numbers of sites found for the word "mammography" and in the distribution of types of sponsors for Web sites (Table, Fig 1). Fewer sites were found by Yahoo than by Excite and
140
health care provider 42%
commercial 5%
governm agenq 5% health care assoc.
24% Figure 3. Distribution of sponsor types for Web sites with information on screening mammography: health care providers, health care organizations (those not directly providing patient care), government agency, commercial (companies selling mammography equipment), and other (not previously specified).
Infoseek, probably because Excite and Infoseek are search engines with indexes created by computer programs, whereas Yahoo is actually a "catalog" of Web sites with indexes created by people. Surprisingly, there was very little overlap in the addresses found by the three search utilities. This would suggest that if one is researching a topic and not finding enough information, switching to a different search utility might produce substantially different results. Since we explored only one term, however, our results were too limited to permit further extrapolation. Nevertheless, similar findings have been described by the popular press (11). While we have listed above se.veral criteria for evaluating Web sites, it may be too demanding to expect the general public to follow these guidelines. A recent review of current mechanisms for rating medical sites showed them to be incompletely developed (12). Nationally known health organizations may therefore need to assume responsibility for evaluating Web sites and for disseminating the results to the general public, with the added challenge that, unlike journal articles, Web sites change over time. Individual radiologists should be proactive in patient education. They can advise patients to be critical about the information they review, inform them that information on the Internet can be misleading, and direct them to reliable sources of information. In conclusion, the Internet offers great opportunities for disseminating educational information to patients, but the information that is available varies greatly in its quality.
Academic Radiology, Vol 7, No 3, March 2000
M A M M O G R A P H Y A N D THE I N T E R N E T
IEFERENCE~ 1. The ACR and screening mammography: consensus and continued commitment. ACR Bull 1997; 53:4-9. 2. Feig SA, D'Orsi CJ, Hendrick RE, et al. American College of Radiology guidelines for breast cancer screening. A JR Am J Roentgeno11998; 171:29-33. 3. Leitch AM, Dodd GD, Costanza M, et al. American Cancer Society guidelines for the early detection of breast cancer: update 1997. CA Cancer J Clin 1997; 47:150-153. 4. Harris KM, Vogel VG. Breast cancer screening. Cancer Metastasis Rev 1997; 16:231-162. 5. Skolnick AA. Surf's up for health fraud investigators (news). JAMA 1997; 278:1725. 6. Skolnick AA. WHO considers regulating ads, sale of medical products on Internet (news). JAMA 1997; 278:1723-1724. 7. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lec-
8.
9.
10.
11.
tot et viewor--let the reader and viewer beware (editorial). JAMA 1997; 277:1244-1245. Sullivan D. Search engine status reports. Mecklermedia. January 1998. Available at: http://searchenginewatch.com/reports/index.html.Accessed February 20, 1998. Cady B, Steele G, Morrow M, et al. Evaluation of common breast problems: guidance for primary care providers. CA Cancer J Clin 1998; 48: 49-63. National Institutes of Health Consensus Development Panel. National Institutes of Health consensus development conference statement: breast cancer screening for women ages 40-49, January 21-23, 1997. J Natl Cancer Inst 1997; 89:1015-1026. Martinez M. Search tools comb the web...or at least part of it: start your engines. ABCNEWS.com. August 21, 1998. Available at: http:
//more.abcnews.go.com/sections/tech/DailyNews/search980820.html. Accessed August 25, 1998. 12. Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA 1998; 279:611-614.
141
Rationale and Objectives. The Internet offers many advantages for educating patients but has no standards for publication. This limitation could negatively affect patient care. The purpose of this study was to evaluate the quality of information on mammography that a patient could find on the Internet. Materials and Methods. Three search utilities were used to research the term "mammography." For each utility, the first 50 addresses for Web pages were evaluated (or all the addresses, if fewer than 50 were returned). Web sites selected contained information that could guide an asymptomatic woman in deciding whether to undergo screening mammography. These sites were then evaluated for whether they indicated their sponsorship, authorship, the currency of information, and references. Sites were also noted if they advised women older than 50 years to undergo screening mammography at intervals of longer than 1 year or otherwise severely diminished the role of mammography. Results. Thirty-eight Web sites were identified. Ten indicated authorship, 29 indicated the currency of the information, and 27 provided references. All of the Web sites indicated sponsorship. Three sites recommended screening mammography at intervals of longer than 1 year for women 50 years of age or older. Two sites suggested that mammography is not substantially more sensitive than physical examination. Conclusion. Many Web sites do not meet the standards for disseminating information required in professional peer-reviewed journals. Some Web sites contain statements that might lead asymptomatic women over age 50 years to delay screening mammography or to undergo screening at intervals of longer than 1 year.
Key Words. Breast radiography; Computers, teaching aid; Internet.
Mammography is a unique area in radiology, in that patients can refer themselves directly for mammography. Screening m a m m o g r a p h y has been shown to decrease mortality by 33% in women 5 0 - 6 9 years of age. The American Cancer Society and the American College of Radiology currently advise annual screening m a m m o graphy for all women over age 40 years, but there has been controversy regarding w o m e n in the 4 0 - 4 9 - y e a r age group (1-4). Increasing patient awareness could increase compliance, but misinformation could delay when
Acad Radiol 2000; 7:137-141
1 From the Department of Radiology, University of Texas at Houston Medical School, 6431 Fannin, MSB 2.100, Houston, TX 77030. Received June 7, 1999; revision requested July 15; revision received August 16; accepted August 18. Address correspondence to E.P.T. © AUR, 2000
a woman chooses to undergo mammography. Therefore, the quality of information available to patients is a concern for radiologists, both because such information may motivate patient behavior and because patients may turn to radiologists to answer questions raised by the information they have found. A variety of health education resources for patients have become available on the Internet. Recent articles have raised concerns about the quality of such medical information (5-7). A Federal Trade Commission "Surf Day," dedicated to exploring health-related Web sites, found over 400 Web sites that were thought to promote questionable health products or services (5). Our objective was to evaluate the quality of information on mammography that a patient could find on the Internet. We sought to evaluate the results obtained with different search utilities and to assess the quality of information
137
provided by a sample of Web sites that contained information on mammography. A search utility is a computer program that allows computer users to enter search terms and returns addresses for Web pages that contain matching terms. Search utilities have become popular among computer users accessing the Internet, and the results they produce can steer the public toward, or away from, useful sources of information. A Web site is a collection of Web files and its associated data stored on a computer and readily accessed by other computers that have access to the Internet. Our focus was on recommendations for mammography for asymptomatic women over 50 years of age.
~IATERIALS AND METHOD.c In January 1998, the Internet information company Mecklermedia identified Yahoo, Infoseek, and Excite as the three most popular search utilities, based on the results of data acquired by two companies, Relevant Knowledge and Media Metrix (8). The two companies used software to monitor the activities of a group of Web users (6,000 for RelevantKnowledge and 10,000 for Media Metrix). At the time these data were pub-i lished, Netscape was not listed as a search utility by Mecklermedia. On March 16, 1998, we submitted the word "mammography" to each of these three search utilities. We then took the first 50 search results, which were addresses for Web pages (otherwise known as uniform resource locators, or URLs). If fewer than 50 addresses were produced by a query, we then evaluated all of the addresses. (The number 50 was arbitrarily chosen because we thought that the average computer user would probably not visit more than the first 50 addresses.)
ing organizations: commercial (selling medical products), health care provider (eg, hospital, medical school, private practice), government agency, health care association (other than health care provider), or "other."
Evaluation Criteria for Web Sites Each Web site was reviewed to determine whether it contained information that could guide an asymptomatic woman in deciding whether to undergo screening mammography. To evaluate Web sites fully, we reviewed the entire site identified by a search utility, not just the particular page identified. The Web sites found to have such content were then evaluated according to criteria suggested in a recent article as basicstandards for publicly distributed information on the Internet (7). These included (a) authorship (whether or not the author was clearly identified); (b) attribution (whether or not references were provided for the information for the article in question; (c) disclosure (whether a Web site indicated ownership); and (d) currency (whether or not the date of publication or last update was provided). Each Web site was also evaluated for the presence of information that advised mammography at intervals less fre-~ quent than yearly for women 50 years of age or older. This criterion was used because annual screening mammography for this age group is recommended by such organizations as the American Cancer Society (9). Because of controversy as to the role of screening mammography for women in the 40-49-year age group, we did not evaluate the information provided for this group (10). We also noted statements that denied or severely diminished the role of screening mammography in decreasing morbidity and mortality due to breast cancer in women. Evaluation was by consensus of three radiologists.
Evaluation Criteria for Search Engines Each search utility was evaluated quantitatively with regard to the following: (a) the total number of Web addresses found for the word "mammography"; (b) the number of addresses that were current (whether the page in question was still accessible at the found address); (c) the number of unique Web sites identified in the first 50 addresses found, or in all found addresses if the total was less than 50 (ie, the number of Web sites found after duplicate references to the same Web addresses had been removed); (d) the number of Web sites that provided guidance for an asymptomatic woman in deciding whether to undergo screening mammography; and (e) the distribution of Web sites found with respect to the types of sponsor-
138
Search Engines The Table lists the results from evaluation of the search engines. Figure 1 shows the distribution of Web sites by type of sponsor, for the first 50 addresses found for each search engine (or all found addresses if the total was less than 50).
Web Sites The three search utilities identified 38 Web sites with information on screening mammography that could influence an asymptomatic patient deciding whether to undergo mammography. Web sites were excluded if they merely
Results from Evaluation of Search Utilities
Finding
Yahoo
Infoseek
Excite
Total no. of Web addresses (Web pages) found No. of unique Web sites (nonduplicates) among first 50 found Web addresses (all if < 50) No. of out-of-date Web addresses No. of addresses for which Web site no longer existed No. of unique Web sites that had screening mammography information relevant to patients
40 25/40 (62.5)
10,402 45/50 (90)
8,490 34/50 (68)
3/40 1/40
5/50 3/50
6/50 0/50 26/45 (56)
5/25 (20)
10/34 (29)
Note.--Percentages in parentheses.
Figure 1. Distribution of sponsor types for Web site addresses found by Yahoo (dark gray bars), Excite (light gray bars), and Infoseek (white bars): health care providers, health care organizations (those not directly providing patient care), government agencies, commercial (companies selling mammography equipment), and other (not previously specified).
70%
60% 50% 40% 30% 20%
..........
J
I ~!ILII::]I
10% 0% health care health care government commercial provider assoc, agency 100% (38) 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Figure 2. Percentage of Web sites that indicated authorship, sources ("attribution," with separate category for full bibliographies), currency of information, and sponsoring organization (ownership) for their information on screening mammography. Numbers in parentheses indicate numbers of sites.
other
provided addresses ("links") to other Web sites containing such information. Web sites were excluded if they discussed mammography without providing guidance as to when women should seek screening mammography. Figure 2 shows the percentages of the 38 Web sites that indicated authorship, currency, and sponsorship and provided references for sources and indicated sponsorship. Figure 3 shows the distribution of sponsor types for these 38 Web sites. Of the 38 Web sites, five (13%) contained statements that were contrary to the criteria given above. Two sites recommended mammography be performed every 2-3 years, and one recommended mammography every 1-2 years for women over age 50 years; all three of these sites were located outside the United States. One article at one site suggested that physical examination was as sensitive as mammography and that mammography was unnecessary. One article at another site stated that the benefits of mammography versus those of breast self-examination are "not well established."
139
T A M M ET A L
Academic Radiology, Vol 7, No 3, March 2000
other s'~ A N /
Radiologists expect their professional literature to meet certain accepted standards for publication, including indications of authorship, the publishing organization, date of publication, and references. On the Internet, however, Web site developers can serve anonymously in multiple roles, usurping the positions of authors, editors, and publishers. Of the 38 Web sites that we found with information that could influence a woman's decision to undergo screening mammography, only 10 (26%) indicated authorship. These 38 sites also varied as to whether they provided other basic information (Fig 2); 29 (76%) indicated the timeliness of their information, while 27 (71%) provided references to the sources of their information. Five (13%) of the 38 sites contained information that could encourage a 50-year-old woman to delay undergoing screening mammography. Two suggested that mammography is not substantially more sensitive than physical examination. Patients exploring the Internet for health-related information should therefore judge Web sites by the same standards that radiologists require for their publications. Patients should also evaluate Web sites for the types of sponsoring organizations. In our research, 100% of the 38 Web sites indicated the sponsoring organization, with a variety of types of sponsors (Fig 3). Three sites, all located outside the United States in English-speaking countries, advised women over 50 years of age to undergo screening mammography at intervals of longer than 1' year. Because of the Internet's c@acity to reach Web sites anywhere in the world, and the tendency for English to be used internationally as the language for publication to the Internet, patients in the United States may cross "international boundaries" without realizing they have done so. They may then encounter information developed by a foreign Web site for the population of its country and mistakenly think it appropriate for them. Search utilities are popular means of finding Web sites. In searching for the term "mammography," we found very few sites that mentioned mammography only incidentally, but we did find many commercial sites that contained information only on mammography equipment. Our research also showed drastic differences between search utilities, both in terms of the numbers of sites found for the word "mammography" and in the distribution of types of sponsors for Web sites (Table, Fig 1). Fewer sites were found by Yahoo than by Excite and
140
health care provider 42%
commercial 5%
governm agenq 5% health care assoc.
24% Figure 3. Distribution of sponsor types for Web sites with information on screening mammography: health care providers, health care organizations (those not directly providing patient care), government agency, commercial (companies selling mammography equipment), and other (not previously specified).
Infoseek, probably because Excite and Infoseek are search engines with indexes created by computer programs, whereas Yahoo is actually a "catalog" of Web sites with indexes created by people. Surprisingly, there was very little overlap in the addresses found by the three search utilities. This would suggest that if one is researching a topic and not finding enough information, switching to a different search utility might produce substantially different results. Since we explored only one term, however, our results were too limited to permit further extrapolation. Nevertheless, similar findings have been described by the popular press (11). While we have listed above se.veral criteria for evaluating Web sites, it may be too demanding to expect the general public to follow these guidelines. A recent review of current mechanisms for rating medical sites showed them to be incompletely developed (12). Nationally known health organizations may therefore need to assume responsibility for evaluating Web sites and for disseminating the results to the general public, with the added challenge that, unlike journal articles, Web sites change over time. Individual radiologists should be proactive in patient education. They can advise patients to be critical about the information they review, inform them that information on the Internet can be misleading, and direct them to reliable sources of information. In conclusion, the Internet offers great opportunities for disseminating educational information to patients, but the information that is available varies greatly in its quality.
Academic Radiology, Vol 7, No 3, March 2000
M A M M O G R A P H Y A N D THE I N T E R N E T
IEFERENCE~ 1. The ACR and screening mammography: consensus and continued commitment. ACR Bull 1997; 53:4-9. 2. Feig SA, D'Orsi CJ, Hendrick RE, et al. American College of Radiology guidelines for breast cancer screening. A JR Am J Roentgeno11998; 171:29-33. 3. Leitch AM, Dodd GD, Costanza M, et al. American Cancer Society guidelines for the early detection of breast cancer: update 1997. CA Cancer J Clin 1997; 47:150-153. 4. Harris KM, Vogel VG. Breast cancer screening. Cancer Metastasis Rev 1997; 16:231-162. 5. Skolnick AA. Surf's up for health fraud investigators (news). JAMA 1997; 278:1725. 6. Skolnick AA. WHO considers regulating ads, sale of medical products on Internet (news). JAMA 1997; 278:1723-1724. 7. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lec-
8.
9.
10.
11.
tot et viewor--let the reader and viewer beware (editorial). JAMA 1997; 277:1244-1245. Sullivan D. Search engine status reports. Mecklermedia. January 1998. Available at: http://searchenginewatch.com/reports/index.html.Accessed February 20, 1998. Cady B, Steele G, Morrow M, et al. Evaluation of common breast problems: guidance for primary care providers. CA Cancer J Clin 1998; 48: 49-63. National Institutes of Health Consensus Development Panel. National Institutes of Health consensus development conference statement: breast cancer screening for women ages 40-49, January 21-23, 1997. J Natl Cancer Inst 1997; 89:1015-1026. Martinez M. Search tools comb the web...or at least part of it: start your engines. ABCNEWS.com. August 21, 1998. Available at: http:
//more.abcnews.go.com/sections/tech/DailyNews/search980820.html. Accessed August 25, 1998. 12. Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA 1998; 279:611-614.
141