- Email: [email protected]
What's amiss with web search engines?
DISSECTING ROOM
The argument for the value of public hospitals is framed in terms of the general value of public institutions for which Opdycke cites four characteristics and shows how the municipal hospitals have displayed them. The first is inclusiveness: not only of those who were served, but in the geographical sense of being available for poor and underserved neighbourhoods, and also as employers. (Bellevue was the first in the city to hire female doctors, black doctors, black nurses, and male nurses.) Next is continuity: consistency of their mission through time and different eras, when other institutions changed their priorities, or the political and economic winds shifted. Then there is responsive-
ness: the “fundamental obligation to accommodate the needs, demands, and criticisms of the city’s many diverse groups”. And finally, there is visibility: both because they provide a prominent reminder of the ever-present needs of the poor, and because their existence demonstrates to the community that it shares a commitment to meeting the needs of all members of society. Does the argument convince? It would be hard to read this eloquently written, well-documented narrative in all its fascinating detail without recognising the importance of the roles that these hospitals have had in the development of New York City. It would be even harder to visualise a future so
What’s amiss with web search engines? Accessibility of Information on the Web (a study summary and more about web searching) http://wwwmetrics.com
A
bout 85% of web users rely on search engines to locate information, and several of the major engines rank consistently among the top sites accessed on the web. But, contrary to popular belief, search engines index only a small part of the web, are biased in what they do include, and are “really out of date”, says Steve Lawrence (NEC Research Institute, Princeton, NJ, USA), coauthor of a study on search engine coverage of the web (Nature 1999; 400: 107-09). The findings have implications for scientists and clinicians, who are increasingly using search engines to locate research of interest, and to access teaching materials, project information, and specialised databases. Lawrence and co-worker C Lee Giles, also of NEC, examined a random
sample of servers (computers that host websites) to estimate the number of pages on the publicly available web and their type of content. Then they gauged coverage of the pages by submitting 1050 scientific terms to 11 major fulltext search engines and analysing the responses (panel). None of the engines indexed more than a sixth of the web. Since overlap among the engines was low, together they covered about 42% of the estimated 800 million pages available; thus, use of a metasearch engine such as MetaCrawler (http://www.metacrawler.com), which combines results of multiple engines, can improve results, suggests Lawrence. The engines are also biased. Because they follow links from other pages to find new ones, sites with few links are
utopian-in which everyone has health insurance, all immigrants are welcome, and society’s outcasts are generously treated-that there would no longer be a need for the kind of places that the people of one of the world’s greatest cities, acting through their government, have sometimes grudgingly, sometimes wastefully, but always faithfully offered to all its citizens. Reader, unless you are the most unrepentant of social Darwinists, you will be, as I have been, convinced. Katherine S Lobach
Department of Pediatrics, Montefiore Medical Center, 111East 210th Street, Bronx, NY 10467, USA
unlikely to be found; new, unlinked web pages can take up to 6 months to appear on the listings. And search results are increasingly influenced by factors other than simple descriptions of a site’s contents; for example, many engines rank websites according to measures of “popularity”, such as the number of other sites that link to it, or the number of times a link to the site has been selected in previous searches. “We can see a trend where popular pages become more popular, while new, unlinked pages have an increasingly difficult time becoming visible in search-engine listings. This may delay or even prevent the widespread visibility of new highquality information”, warn Lawrence and Giles. Ivor Benjamin, coeditor of the cancer information website OncoLink (http://www.oncolink.com),worries that the article could discourage physicians from learning to use the web. Although the study was well done, “the slant on the interpretation is too alarmist”, he says. “I think doctors who have never used the web for research could conclude that it is virtually useless for iinding good, up-to-date information, and that definitely is not true.” He notes that the authors did not look at the effect of registering a new site directly with the search engines, instead of waiting for the engines to find it. “Submit the URLs yourself or have a service do it for you”, he advises, Although “imperfect”, search engines “will give you a sense of which are the key, expert-related sites on a given topic, especially if you search on several engines”, emphasises Benjamin. “You have to use them, get to know their strengths and weaknesses, and not think you’ve done a comprehensive search by using any single source.” Marilynn Larkin
[email protected] 260
THE LANCET * Vol354 -July 17,1999
The argument for the value of public hospitals is framed in terms of the general value of public institutions for which Opdycke cites four characteristics and shows how the municipal hospitals have displayed them. The first is inclusiveness: not only of those who were served, but in the geographical sense of being available for poor and underserved neighbourhoods, and also as employers. (Bellevue was the first in the city to hire female doctors, black doctors, black nurses, and male nurses.) Next is continuity: consistency of their mission through time and different eras, when other institutions changed their priorities, or the political and economic winds shifted. Then there is responsive-
ness: the “fundamental obligation to accommodate the needs, demands, and criticisms of the city’s many diverse groups”. And finally, there is visibility: both because they provide a prominent reminder of the ever-present needs of the poor, and because their existence demonstrates to the community that it shares a commitment to meeting the needs of all members of society. Does the argument convince? It would be hard to read this eloquently written, well-documented narrative in all its fascinating detail without recognising the importance of the roles that these hospitals have had in the development of New York City. It would be even harder to visualise a future so
What’s amiss with web search engines? Accessibility of Information on the Web (a study summary and more about web searching) http://wwwmetrics.com
A
bout 85% of web users rely on search engines to locate information, and several of the major engines rank consistently among the top sites accessed on the web. But, contrary to popular belief, search engines index only a small part of the web, are biased in what they do include, and are “really out of date”, says Steve Lawrence (NEC Research Institute, Princeton, NJ, USA), coauthor of a study on search engine coverage of the web (Nature 1999; 400: 107-09). The findings have implications for scientists and clinicians, who are increasingly using search engines to locate research of interest, and to access teaching materials, project information, and specialised databases. Lawrence and co-worker C Lee Giles, also of NEC, examined a random
sample of servers (computers that host websites) to estimate the number of pages on the publicly available web and their type of content. Then they gauged coverage of the pages by submitting 1050 scientific terms to 11 major fulltext search engines and analysing the responses (panel). None of the engines indexed more than a sixth of the web. Since overlap among the engines was low, together they covered about 42% of the estimated 800 million pages available; thus, use of a metasearch engine such as MetaCrawler (http://www.metacrawler.com), which combines results of multiple engines, can improve results, suggests Lawrence. The engines are also biased. Because they follow links from other pages to find new ones, sites with few links are
utopian-in which everyone has health insurance, all immigrants are welcome, and society’s outcasts are generously treated-that there would no longer be a need for the kind of places that the people of one of the world’s greatest cities, acting through their government, have sometimes grudgingly, sometimes wastefully, but always faithfully offered to all its citizens. Reader, unless you are the most unrepentant of social Darwinists, you will be, as I have been, convinced. Katherine S Lobach
Department of Pediatrics, Montefiore Medical Center, 111East 210th Street, Bronx, NY 10467, USA
unlikely to be found; new, unlinked web pages can take up to 6 months to appear on the listings. And search results are increasingly influenced by factors other than simple descriptions of a site’s contents; for example, many engines rank websites according to measures of “popularity”, such as the number of other sites that link to it, or the number of times a link to the site has been selected in previous searches. “We can see a trend where popular pages become more popular, while new, unlinked pages have an increasingly difficult time becoming visible in search-engine listings. This may delay or even prevent the widespread visibility of new highquality information”, warn Lawrence and Giles. Ivor Benjamin, coeditor of the cancer information website OncoLink (http://www.oncolink.com),worries that the article could discourage physicians from learning to use the web. Although the study was well done, “the slant on the interpretation is too alarmist”, he says. “I think doctors who have never used the web for research could conclude that it is virtually useless for iinding good, up-to-date information, and that definitely is not true.” He notes that the authors did not look at the effect of registering a new site directly with the search engines, instead of waiting for the engines to find it. “Submit the URLs yourself or have a service do it for you”, he advises, Although “imperfect”, search engines “will give you a sense of which are the key, expert-related sites on a given topic, especially if you search on several engines”, emphasises Benjamin. “You have to use them, get to know their strengths and weaknesses, and not think you’ve done a comprehensive search by using any single source.” Marilynn Larkin
[email protected] 260
THE LANCET * Vol354 -July 17,1999