- Email: [email protected]
Information retrieval in systematic reviews
Articles
Information Retrieval in Systematic Reviews Challenges in the Public Health Arena Chris C. Beahler, MLS, Jennifer J. Sundheim, BA, Naomi I. Trapp, BA Background: Information retrieval for systematic reviews in occupational injuries and other public health areas is much more elusive than retrieval for reviews in clinical medicine, due to the interdisciplinary nature of the field and the lack of a significant body of evaluative literature. Objective:
The objective of this study is to provide information about challenges and methodology in relevant literature retrieval for systematic reviews in the effectiveness of strategies to prevent occupational injury.
Methods:
Participants from Injury Control Research Centers and Agricultural Health and Safety Centers identified 12 areas of occupational injury and evaluated the effectiveness of interventions in each area. A systematic review of the literature was conducted, and results were critically reviewed and summarized.
Results:
The search strategy captured 41,871 abstracts or titles across all research topics. After screening, 1356 documents were identified as being potentially eligible studies. Relevant articles were also identified through gleaning references and contact with professionals in the field.
Conclusions: Literature reviews in the field of occupational injury cannot be limited to database searches. Much of the literature is not well-indexed, and librarians must employ information retrieval methods other than database searching to retrieve relevant literature in the field. Medical Subject Headings (MeSH): accidents-occupational, databases-bibliographic, information science, library service, medical library, MEDLINE, public health (Am J Prev Med 2000;18(4S):6 –10) © 2000 American Journal of Preventive Medicine
T
his special issue of the American Journal of Preventive Medicine presents the results of systematic reviews of the literature on the effectiveness of various strategies to prevent occupational injuries. These reviews were part of an occupational injury project coordinated by the Harborview Injury Prevention Research Center (HIPRC), and included participants from the Injury Control Research Centers (ICRCs), and the Division of Safety Research, Agricultural Safety and Health Centers. A systematic review is defined as “a review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyze data from studies that are included in the review.”1 Systematic reviews in the field of occupational injury presented a number of challenges
for literature searching and document retrieval. The purpose of this article is to discuss these challenges and the methods used to deal with them.
From the University of Washington Health Sciences Libraries (Beahler, Trapp), Seattle, Washington and the Harborview Injury Prevention and Research Center (Sundheim), Seattle, Washington Address correspondence and reprint requests to: Chris Beahler, University of Washington Health Sciences Libraries, University of Washington, Box 357155, Seattle, WA 98195-7155. E-mail: cbeahler @u.washington.edu.
●
6
Literature Searching Strategies Occupational injury spans a number of disciplines, including biomedical, occupational health, social science, education, criminal justice, agriculture, government, and business. A systematic review in the field must examine literature that appears in places other than the well-indexed biomedical databases MEDLINE and EMBASE. The types of databases that potentially include studies in occupational injury include: ●
● ●
Biomedical: MEDLINE, EMBASE, Nursing, and Allied Health Occupational health: National Institute for Occupational Safety and Health (NIOSHTIC), Transportation Research Information Services (TRIS) Business: ABI Inform Criminal justice and social science: Criminal Justice
Am J Prev Med 2000;18(4S) 0749-3797/00/$–see front matter © 2000 American Journal of Preventive Medicine • Published by Elsevier Science Inc. PII S0749-3797(00)00135-5
● ●
●
Periodicals Index, (CJPI) Sociofile, Educational Resources Information Center (ERIC), PsycINFO. Agricultural: Agris International, Agricola Government: National Technical Information Service (NTIS), Public Affairs Information Service (PAIS) General: Books in Print, Dissertation Abstracts, Expanded Academic Index
Search accuracy and reliability are impacted by the inclusion of a broad range of databases. This broad range includes both the subject matter and the organizational structure of the database. In a highly organized, indexed database, all entries have been analyzed by a professional indexer(s) who assigns relevant subject headings from a controlled vocabulary list. Indexing allows searching using this controlled vocabulary or subject headings, eliminating the need to include synonyms, equivalent conceptual terms, or alternate spelling in the search strategy. In the study on shiftwork,2 one MEDLINE subject heading is “work schedule tolerance,” defined as “physiological or psychological effects of periods of work which may be fixed or flexible, such as flextime, work shifts, and rotating shifts.” Using this subject heading in a search for relevant literature eliminates the need to incorporate all possible terminology, such as shiftwork, graveyard, night shift, swing shift, and rotating shift, as keywords since the subject headings usually suffice. Keyword searching is necessary when a database has not been indexed with subject headings. It is usually impossible to include all possible variations of a term or concept in a search, and keyword searching is not as effective as searching with subject headings from controlled vocabularies. A case in point is the “Effectiveness of crime prevention through environmental design” study.3 Searches in MEDLINE and EMBASE were conducted using subject headings, and then subsequent keyword searches. Search results indicated that the majority of relevant information available had been retrieved with subject headings. However, keyword searches in 14 other databases that were not well-indexed failed to retrieve a number of relevant citations. Further examination indicated that although we were searching for this information as pertinent to a “workplace violence” study, many databases defined this topic according to a different, subject-specific framework. Rather than focusing on workers as victims of workplace violence, some databases viewed the issue as profit-loss (business databases) or crime (criminal justice databases), not employee injury. When these varied points of view were incorporated into the search strategy, results improved. An additional challenge for information retrieval in the field of occupational injury is the scarcity of randomized controlled trials in the field, and the subsequent reliance on numerous other research methodologies. While randomized controlled trials are
acknowledged as the “gold standard” in clinical medicine reviews, they are uncommon in the field of injury prevention and almost nonexistent in the occupational injury literature. As a result, literature searches in these fields must be broad and far-ranging. In the field of farm safety, much of the evaluative literature consisted of conference proceedings, rather than articles published in peer-reviewed journals.4 But in the area of needlestick injury interventions, which is an area closer to clinical medicine than most of the others studied here, a number of randomized controlled trials do exist on the topic.5 As a result, the literature search could be limited to biomedical databases using a controlled vocabulary, and identified documents were easily located and retrieved.
References and Contacts Pearled references and professional contacts play a significant role in the location and acquisition of literature for a systematic review in a field such as occupational injury, where much of the research is located outside the peer-reviewed journal literature. A “pearl” or “pearled reference” refers to a piece of literature found by studying the body or bibliography of another document. The relevant study is “hidden” within the research like a pearl within the body of an oyster. Pearling references and making professional contacts finds the unindexed, unpublished, or difficult to obtain literature. As with other public health fields, references and contacts were particularly important for the occupational injury project due to the multidisciplinary nature of the topics and broad range of research methodologies examined. Of the 1356 journal articles, reports, conference proceedings, or books requested for review and obtained, a significant proportion was the result of pearled references or professional contacts. The percentage of the total literature located through contacts or references for each paper correlates with the degree to which a topic is clinical, or can be concisely defined. The research on carpal tunnel is one of the more clinical topics and peer-reviewed, biomedical databases such as MEDLINE and EMBASE index most of the referenced literature.6 In contrast, the farm safety education paper by DeRoo and Rautiainen,4 evaluates a broad-ranging topic that is difficult to define and spans several disciplines. The fact that much of the literature exists in conference proceedings, which are largely unindexed and difficult to obtain, further complicates literature searching. Conference proceedings accounted for 37% of the documents obtained for potential review in the farm safety study. To find many of the reports required manually thumbing through conference proceedings at several libraries. One of the authors, who manages a clearinghouse of agricultural information at the University of Iowa, located and Am J Prev Med 2000;18(4S)
7
supplied dozens of reports printed in the various conference proceedings from the National Institute of Farm Safety. The paper that evaluates environmental interventions in workplace violence3 again demonstrates the necessity of contacts and pearled references in multidisciplinary systematic reviews. Casteel and Peek-Asa3 obtained more than half of their literature from reviewing references and talking to contacts. Studies performed by police departments (New York City Police Department, Gainesville Police Department) and commercial organizations (Athena Research Organization, National Association of Convenience Stores, Touche Ross & Company) comprised a fair amount of this peripheral literature. Here again, many of the reports were not indexed in standard databases and, therefore, were difficult to identify without carefully examining references from other pertinent documents or talking to experts in the field. A number of relevant documents in this field are privately published, and cost can be a barrier to acquiring them without the assistance of professional contacts.
Retrieval of Full-Text Documents Once abstracts have been selected from the literature searches, relevant full-text documents are retrieved to determine whether they meet the criteria for the review. All requests for document retrieval were sent to the Document Services Department, the unit in the University of Washington Health Sciences Libraries that provides library-subsidized, full-text document retrieval for university departments. A document retrieval service is the preferred method for obtaining full-text articles for a systematic review because of the volume of work required and the unique nature of the citations not held in the local collection. Utilizing existing interlibrary borrowing relationships and experienced searchers increases the speed and effectiveness of document retrieval. Requests pass through a series of steps. Initially, the Document Services staff searches the University of Washington Libraries collection. If the document is not located within the collection, outside vendors are consulted. If the item still cannot be located, it moves into the interlibrary borrowing section.
Interlibrary Borrowing Documents with a National Library of Medicine (NLM) identification number are requested through DOCLINE (an electronic routing mechanism maintained by NLM). If the document has not been indexed by NLM, staff search the Online Computer Library Center (OCLC), a database with over 30 million catalog records and holdings information for materials owned by libraries all over the world. Finally, if the item cannot 8
be located or acquired through OCLC, staff searches the Internet looking for publisher, organization, author, and other relevant information about the document requested to facilitate its acquisition. Staff time and financial expense increase as the document request passes through the different levels of the process. A survey of interlibrary borrowing that took place in the Document Services department during the time period of this project indicated that 90% of the total requests filled for interlibrary borrowing came from DOCLINE, which provides a relatively straightforward and simple method for obtaining a document. In contrast, the statistics for the occupational injury studies during the same time period indicate that only 54% of the citations were filled by DOCLINE. The remaining 46% of the requests passed into the OCLC-Internet phase of the search. This phase requires more staff time and can be much more costly. Staff must find the appropriate balance between a comprehensive Internet search and a reasonable amount of time spent on each request. For example, in the list of requests for documents pertaining to carpal tunnel syndrome,6 one publication requested was a product pamphlet packaged with a particular brand of “squeegee.” The level of staff time involved in finding and requesting the pamphlet is extremely high. In this case, staff searched the Internet, did not find the company, and abandoned the search. Documents published by The National Association of Convenience Stores, city police departments and a humane society proved to be the most challenging to acquire. For these requests, staff used the Internet to locate contact information, and contacted the author or agency by email. The final phase of document retrieval highlighted the importance of cultivating good contacts. Although some contacts were unable to provide the requested item, they offered assistance through their topic-specific knowledge and connections. In this way, many documents were made available that would otherwise have been impossible to find.
Search Results The search strategy captured 41,871 abstracts or titles across all research topics. After review for subject relevance by the librarian heading the search, these abstracts or titles were screened independently by two subject experts using standardized criteria to identify potentially eligible articles. If an abstract or title met the screening criteria, a full copy of the study was retrieved; 1356 potentially eligible studies were identified (see Table 1). A standardized checklist was used by two independent raters to determine inclusion of the potentially eligible studies in the review. A total of 207 studies were selected for the 12 reviews. The authors abstracted each study, judged methodologic quality
American Journal of Preventive Medicine, Volume 18, Number 4S
Table 1. Literature selection results for all review articles
Author(s) Beahler et al.7 Frank2 Keifer8 Lincoln et al.6 Lipscomb9 Casteel et al.3,a DeRoo et al.4 Reed and Claunch10 Reynolds and Groves11 Rivara and Thompson12 Rogers and Goodno5 Runyan et al.13,a Segui-Gomez14 Totals
Total number of articles Total number initially selected of abstracts for (titles) review screened — 802 (2687) 263 (5735) 623 (2546) 671 (768) 808 (5628) 378 (9449) 118 (557) 277 (723) 200 (1255) 615 771 (5665) 208 (1124) 5734 (36,137) 41,871
— 79 129 310 110 110 118 108 53 53 76 137 73 1356
Total number of articles included in final review — 7 17 24 7 26 25 32 21 3 11 9 25 207
a
Abstracts and titles for the two workplace violence papers included only once.
and made the final decision on which articles to include in their reviews.
Conclusions Conducting a literature review in the field of occupational injury presents a number of challenges. Relevant literature exists outside the peer-reviewed scientific journals, and numerous other information sources must be searched. Additionally, very few randomized controlled trials are available, so a number of levels of research methodology must be explored. And finally, because so much of the information is located outside the indexed, scientific journal area, finding and retrieving relevant full-text documents is both time consuming and expensive. However, the results are of great importance, and provide vital information for allocation of resources, public policy making, and prevention of injuries to working people. Support for this project was provided by Injury Control Research Centers (ICRCs), the National Center for Injury Prevention and Control (NCIPC), Centers for Disease Control and Prevention, and the Division of Safety Research, the National Institute for Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention.
References 1. Murlow CD, Oxman AD, eds. Cochrane collaboration handbook (updated September 1997). In: The Cochrane Library [database on disk and CDROM]. The Cochrane collaboration. Oxford: Update Software, 1998, Issue 3.
2. Frank AL. Injuries related to shiftwork. Am J Prev Med 2000;18(suppl 4):33–36. 3. Casteel C, Peek-Asa C. Effectiveness of crime prevention through environmental design (CPTED) in reducing robberies. Am J Prev Med 2000; 18(suppl 4):99 –115. 4. DeRoo LA, Rautiainen RH. A systematic review of farm safety interventions. Am J Prev Med 2000;18(suppl 4):51– 62. 5. Rogers B, Goodno L. Evaluation of interventions to prevent needlestick injuries in health care occupations. Am J Prev Med 2000;18(suppl 4):90 – 98. 6. Lincoln AE, Vernick JS, Ogaitis S, Smith GS, Mitchell CS, Agnew J. Interventions for the primary prevention of work-related carpal tunnel syndrome. Am J Prev Med 2000;18(suppl 4):37–50. 7. Beahler CC, Sundheim JJ, Trapp NI. Information retrieval in systematic reviews: challenges in the public health arena. Am J Prev Med 2000; 18(suppl 4):6 –11. 8. Kiefer MC. Effectiveness of interventions in reducing pesticide overexposure and poisonings. Am J Prev Med 2000;18(suppl 4):80 – 89. 9. Lipscomb JH. Effectiveness of interventions to prevent work-related eye injuries. Am J Prev Med 2000;18(suppl 4):27–32. 10. Reed DB, Claunch DT. Nonfatal farm injury and disability to children: a systematic review. Am J Prev Med 2000;18(suppl 4):70 –79. 11. Reynolds SJ, Groves W. Effectiveness of roll-over protective structures in reducing farm tractor fatalities. Am J Prev Med 2000;18(suppl 4):63– 69. 12. Rivara FP, Thompson DC. Prevention of falls in the construction industry: evidence for program effectiveness. Am J Prev Med 2000;18(suppl 4):23– 26. 13. Runyan CW, Zakocs RC, Zwerling C. Administrative and behavioral interventions for workplace violence prevention. Am J Prev Med 2000;18(suppl 4):116 –127. 14. Segui-Gomez M. Evaluating worksite-based interventions that promote safety belt use. Am J Prev Med 2000;18(suppl 4):11–22.
Appendix The Databases Agricola 1970 –1999. Contains over 3 million records covering every major agricultural subject. Produced by the National Agricultural Library, this database contains articles, pamphlets, research reports and government documents on agricultural marketing, foods and feeds, farm technology, animal husbandry, and allied sciences. Updated monthly. Agris International 1975–1999. Covers agricultural and rural development literature on a worldwide basis. This database is produced by the Food and Agriculture Organization of the United Nations. Books in Print. Database containing 1.8 million records of books in print, out of print, and forthcoming from North American publishers. Criminal Justice Periodical Index (CJPI). Database with coverage of criminal justice journals and newsletters in the United States, Canada, and United Kingdom. Dissertation Abstracts 1861–1999. Indexing and abstracts for dissertations from North American and European universities. Contains 1.5 million entries. EMBASE 1974 –1999. Contains abstracts and citations to the international literature on drugs, pharmacology, and human medicine, including European and Asian publications. Articles appearing in EMBASE are largely peer reviewed. Am J Prev Med 2000;18(4S)
9
ERIC 1966 –1999. Educational Resources Information Center database. Indexes journal articles and documents in the field of education. Updated monthly.
from U.S. federal agencies, contractors or grantees; and research and development reports sponsored by governments of other industrialized nations.
Expanded Academic Index 1989 –1999. Indexing and abstracts for approximately 1500 scholarly and general interest periodicals covering major fields of study in the humanities, social sciences, and technology. Updated monthly.
Nursing and Allied Health 1982–1999. Provides coverage of the literature in nursing and 17 allied health disciplines. Over 700 journals are regularly indexed. Updated monthly.
Health Star 1975–1999. One of the National Library of Medicine’s bibliographic databases, providing access to the published literature of Health Services Technology, Administration and Research. NIOSHTIC 1973–1999. Covers occupational safety and health. Sources include about 160 core journals (about 33% of the database), non-core journals (33%), NIOSH publications (7%), conference and symposium proceedings (5%), and translations (5%). Subjects covered include: occupational health and safety programs, hazardous substances, toxicology, unsafe workplace environment, epidemiology, pathology and histology, occupational medicine, ergonomics, industrial hygiene, behavioral sciences, biochemistry, physiology and metabolism, chemistry, control technology, education and training, engineering, health physics, and safety. NTIS 1983–1999. Contains U.S. government-sponsored research; development and engineering analysis
10
PAIS International 1972–1999 (Public Affairs Information). Covers social and public policy literature from over 1400 periodicals and thousands of government documents, books, and other sources. Updated monthly. PsycINFO 1967–1999. Indexes and abstracts articles from over 1300 journals in the field of psychology. Updated monthly. Sociofile 1974 –1999. Contains journal article abstracts and citations for books, book reviews, conference papers, and dissertations published in Sociological Abstracts and SOPODA. Updated three times per year. TRIS (Transportation Research Information Services). Covers all surface transportation modes, air transport, and highway safety.
American Journal of Preventive Medicine, Volume 18, Number 4S
Information Retrieval in Systematic Reviews Challenges in the Public Health Arena Chris C. Beahler, MLS, Jennifer J. Sundheim, BA, Naomi I. Trapp, BA Background: Information retrieval for systematic reviews in occupational injuries and other public health areas is much more elusive than retrieval for reviews in clinical medicine, due to the interdisciplinary nature of the field and the lack of a significant body of evaluative literature. Objective:
The objective of this study is to provide information about challenges and methodology in relevant literature retrieval for systematic reviews in the effectiveness of strategies to prevent occupational injury.
Methods:
Participants from Injury Control Research Centers and Agricultural Health and Safety Centers identified 12 areas of occupational injury and evaluated the effectiveness of interventions in each area. A systematic review of the literature was conducted, and results were critically reviewed and summarized.
Results:
The search strategy captured 41,871 abstracts or titles across all research topics. After screening, 1356 documents were identified as being potentially eligible studies. Relevant articles were also identified through gleaning references and contact with professionals in the field.
Conclusions: Literature reviews in the field of occupational injury cannot be limited to database searches. Much of the literature is not well-indexed, and librarians must employ information retrieval methods other than database searching to retrieve relevant literature in the field. Medical Subject Headings (MeSH): accidents-occupational, databases-bibliographic, information science, library service, medical library, MEDLINE, public health (Am J Prev Med 2000;18(4S):6 –10) © 2000 American Journal of Preventive Medicine
T
his special issue of the American Journal of Preventive Medicine presents the results of systematic reviews of the literature on the effectiveness of various strategies to prevent occupational injuries. These reviews were part of an occupational injury project coordinated by the Harborview Injury Prevention Research Center (HIPRC), and included participants from the Injury Control Research Centers (ICRCs), and the Division of Safety Research, Agricultural Safety and Health Centers. A systematic review is defined as “a review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyze data from studies that are included in the review.”1 Systematic reviews in the field of occupational injury presented a number of challenges
for literature searching and document retrieval. The purpose of this article is to discuss these challenges and the methods used to deal with them.
From the University of Washington Health Sciences Libraries (Beahler, Trapp), Seattle, Washington and the Harborview Injury Prevention and Research Center (Sundheim), Seattle, Washington Address correspondence and reprint requests to: Chris Beahler, University of Washington Health Sciences Libraries, University of Washington, Box 357155, Seattle, WA 98195-7155. E-mail: cbeahler @u.washington.edu.
●
6
Literature Searching Strategies Occupational injury spans a number of disciplines, including biomedical, occupational health, social science, education, criminal justice, agriculture, government, and business. A systematic review in the field must examine literature that appears in places other than the well-indexed biomedical databases MEDLINE and EMBASE. The types of databases that potentially include studies in occupational injury include: ●
● ●
Biomedical: MEDLINE, EMBASE, Nursing, and Allied Health Occupational health: National Institute for Occupational Safety and Health (NIOSHTIC), Transportation Research Information Services (TRIS) Business: ABI Inform Criminal justice and social science: Criminal Justice
Am J Prev Med 2000;18(4S) 0749-3797/00/$–see front matter © 2000 American Journal of Preventive Medicine • Published by Elsevier Science Inc. PII S0749-3797(00)00135-5
● ●
●
Periodicals Index, (CJPI) Sociofile, Educational Resources Information Center (ERIC), PsycINFO. Agricultural: Agris International, Agricola Government: National Technical Information Service (NTIS), Public Affairs Information Service (PAIS) General: Books in Print, Dissertation Abstracts, Expanded Academic Index
Search accuracy and reliability are impacted by the inclusion of a broad range of databases. This broad range includes both the subject matter and the organizational structure of the database. In a highly organized, indexed database, all entries have been analyzed by a professional indexer(s) who assigns relevant subject headings from a controlled vocabulary list. Indexing allows searching using this controlled vocabulary or subject headings, eliminating the need to include synonyms, equivalent conceptual terms, or alternate spelling in the search strategy. In the study on shiftwork,2 one MEDLINE subject heading is “work schedule tolerance,” defined as “physiological or psychological effects of periods of work which may be fixed or flexible, such as flextime, work shifts, and rotating shifts.” Using this subject heading in a search for relevant literature eliminates the need to incorporate all possible terminology, such as shiftwork, graveyard, night shift, swing shift, and rotating shift, as keywords since the subject headings usually suffice. Keyword searching is necessary when a database has not been indexed with subject headings. It is usually impossible to include all possible variations of a term or concept in a search, and keyword searching is not as effective as searching with subject headings from controlled vocabularies. A case in point is the “Effectiveness of crime prevention through environmental design” study.3 Searches in MEDLINE and EMBASE were conducted using subject headings, and then subsequent keyword searches. Search results indicated that the majority of relevant information available had been retrieved with subject headings. However, keyword searches in 14 other databases that were not well-indexed failed to retrieve a number of relevant citations. Further examination indicated that although we were searching for this information as pertinent to a “workplace violence” study, many databases defined this topic according to a different, subject-specific framework. Rather than focusing on workers as victims of workplace violence, some databases viewed the issue as profit-loss (business databases) or crime (criminal justice databases), not employee injury. When these varied points of view were incorporated into the search strategy, results improved. An additional challenge for information retrieval in the field of occupational injury is the scarcity of randomized controlled trials in the field, and the subsequent reliance on numerous other research methodologies. While randomized controlled trials are
acknowledged as the “gold standard” in clinical medicine reviews, they are uncommon in the field of injury prevention and almost nonexistent in the occupational injury literature. As a result, literature searches in these fields must be broad and far-ranging. In the field of farm safety, much of the evaluative literature consisted of conference proceedings, rather than articles published in peer-reviewed journals.4 But in the area of needlestick injury interventions, which is an area closer to clinical medicine than most of the others studied here, a number of randomized controlled trials do exist on the topic.5 As a result, the literature search could be limited to biomedical databases using a controlled vocabulary, and identified documents were easily located and retrieved.
References and Contacts Pearled references and professional contacts play a significant role in the location and acquisition of literature for a systematic review in a field such as occupational injury, where much of the research is located outside the peer-reviewed journal literature. A “pearl” or “pearled reference” refers to a piece of literature found by studying the body or bibliography of another document. The relevant study is “hidden” within the research like a pearl within the body of an oyster. Pearling references and making professional contacts finds the unindexed, unpublished, or difficult to obtain literature. As with other public health fields, references and contacts were particularly important for the occupational injury project due to the multidisciplinary nature of the topics and broad range of research methodologies examined. Of the 1356 journal articles, reports, conference proceedings, or books requested for review and obtained, a significant proportion was the result of pearled references or professional contacts. The percentage of the total literature located through contacts or references for each paper correlates with the degree to which a topic is clinical, or can be concisely defined. The research on carpal tunnel is one of the more clinical topics and peer-reviewed, biomedical databases such as MEDLINE and EMBASE index most of the referenced literature.6 In contrast, the farm safety education paper by DeRoo and Rautiainen,4 evaluates a broad-ranging topic that is difficult to define and spans several disciplines. The fact that much of the literature exists in conference proceedings, which are largely unindexed and difficult to obtain, further complicates literature searching. Conference proceedings accounted for 37% of the documents obtained for potential review in the farm safety study. To find many of the reports required manually thumbing through conference proceedings at several libraries. One of the authors, who manages a clearinghouse of agricultural information at the University of Iowa, located and Am J Prev Med 2000;18(4S)
7
supplied dozens of reports printed in the various conference proceedings from the National Institute of Farm Safety. The paper that evaluates environmental interventions in workplace violence3 again demonstrates the necessity of contacts and pearled references in multidisciplinary systematic reviews. Casteel and Peek-Asa3 obtained more than half of their literature from reviewing references and talking to contacts. Studies performed by police departments (New York City Police Department, Gainesville Police Department) and commercial organizations (Athena Research Organization, National Association of Convenience Stores, Touche Ross & Company) comprised a fair amount of this peripheral literature. Here again, many of the reports were not indexed in standard databases and, therefore, were difficult to identify without carefully examining references from other pertinent documents or talking to experts in the field. A number of relevant documents in this field are privately published, and cost can be a barrier to acquiring them without the assistance of professional contacts.
Retrieval of Full-Text Documents Once abstracts have been selected from the literature searches, relevant full-text documents are retrieved to determine whether they meet the criteria for the review. All requests for document retrieval were sent to the Document Services Department, the unit in the University of Washington Health Sciences Libraries that provides library-subsidized, full-text document retrieval for university departments. A document retrieval service is the preferred method for obtaining full-text articles for a systematic review because of the volume of work required and the unique nature of the citations not held in the local collection. Utilizing existing interlibrary borrowing relationships and experienced searchers increases the speed and effectiveness of document retrieval. Requests pass through a series of steps. Initially, the Document Services staff searches the University of Washington Libraries collection. If the document is not located within the collection, outside vendors are consulted. If the item still cannot be located, it moves into the interlibrary borrowing section.
Interlibrary Borrowing Documents with a National Library of Medicine (NLM) identification number are requested through DOCLINE (an electronic routing mechanism maintained by NLM). If the document has not been indexed by NLM, staff search the Online Computer Library Center (OCLC), a database with over 30 million catalog records and holdings information for materials owned by libraries all over the world. Finally, if the item cannot 8
be located or acquired through OCLC, staff searches the Internet looking for publisher, organization, author, and other relevant information about the document requested to facilitate its acquisition. Staff time and financial expense increase as the document request passes through the different levels of the process. A survey of interlibrary borrowing that took place in the Document Services department during the time period of this project indicated that 90% of the total requests filled for interlibrary borrowing came from DOCLINE, which provides a relatively straightforward and simple method for obtaining a document. In contrast, the statistics for the occupational injury studies during the same time period indicate that only 54% of the citations were filled by DOCLINE. The remaining 46% of the requests passed into the OCLC-Internet phase of the search. This phase requires more staff time and can be much more costly. Staff must find the appropriate balance between a comprehensive Internet search and a reasonable amount of time spent on each request. For example, in the list of requests for documents pertaining to carpal tunnel syndrome,6 one publication requested was a product pamphlet packaged with a particular brand of “squeegee.” The level of staff time involved in finding and requesting the pamphlet is extremely high. In this case, staff searched the Internet, did not find the company, and abandoned the search. Documents published by The National Association of Convenience Stores, city police departments and a humane society proved to be the most challenging to acquire. For these requests, staff used the Internet to locate contact information, and contacted the author or agency by email. The final phase of document retrieval highlighted the importance of cultivating good contacts. Although some contacts were unable to provide the requested item, they offered assistance through their topic-specific knowledge and connections. In this way, many documents were made available that would otherwise have been impossible to find.
Search Results The search strategy captured 41,871 abstracts or titles across all research topics. After review for subject relevance by the librarian heading the search, these abstracts or titles were screened independently by two subject experts using standardized criteria to identify potentially eligible articles. If an abstract or title met the screening criteria, a full copy of the study was retrieved; 1356 potentially eligible studies were identified (see Table 1). A standardized checklist was used by two independent raters to determine inclusion of the potentially eligible studies in the review. A total of 207 studies were selected for the 12 reviews. The authors abstracted each study, judged methodologic quality
American Journal of Preventive Medicine, Volume 18, Number 4S
Table 1. Literature selection results for all review articles
Author(s) Beahler et al.7 Frank2 Keifer8 Lincoln et al.6 Lipscomb9 Casteel et al.3,a DeRoo et al.4 Reed and Claunch10 Reynolds and Groves11 Rivara and Thompson12 Rogers and Goodno5 Runyan et al.13,a Segui-Gomez14 Totals
Total number of articles Total number initially selected of abstracts for (titles) review screened — 802 (2687) 263 (5735) 623 (2546) 671 (768) 808 (5628) 378 (9449) 118 (557) 277 (723) 200 (1255) 615 771 (5665) 208 (1124) 5734 (36,137) 41,871
— 79 129 310 110 110 118 108 53 53 76 137 73 1356
Total number of articles included in final review — 7 17 24 7 26 25 32 21 3 11 9 25 207
a
Abstracts and titles for the two workplace violence papers included only once.
and made the final decision on which articles to include in their reviews.
Conclusions Conducting a literature review in the field of occupational injury presents a number of challenges. Relevant literature exists outside the peer-reviewed scientific journals, and numerous other information sources must be searched. Additionally, very few randomized controlled trials are available, so a number of levels of research methodology must be explored. And finally, because so much of the information is located outside the indexed, scientific journal area, finding and retrieving relevant full-text documents is both time consuming and expensive. However, the results are of great importance, and provide vital information for allocation of resources, public policy making, and prevention of injuries to working people. Support for this project was provided by Injury Control Research Centers (ICRCs), the National Center for Injury Prevention and Control (NCIPC), Centers for Disease Control and Prevention, and the Division of Safety Research, the National Institute for Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention.
References 1. Murlow CD, Oxman AD, eds. Cochrane collaboration handbook (updated September 1997). In: The Cochrane Library [database on disk and CDROM]. The Cochrane collaboration. Oxford: Update Software, 1998, Issue 3.
2. Frank AL. Injuries related to shiftwork. Am J Prev Med 2000;18(suppl 4):33–36. 3. Casteel C, Peek-Asa C. Effectiveness of crime prevention through environmental design (CPTED) in reducing robberies. Am J Prev Med 2000; 18(suppl 4):99 –115. 4. DeRoo LA, Rautiainen RH. A systematic review of farm safety interventions. Am J Prev Med 2000;18(suppl 4):51– 62. 5. Rogers B, Goodno L. Evaluation of interventions to prevent needlestick injuries in health care occupations. Am J Prev Med 2000;18(suppl 4):90 – 98. 6. Lincoln AE, Vernick JS, Ogaitis S, Smith GS, Mitchell CS, Agnew J. Interventions for the primary prevention of work-related carpal tunnel syndrome. Am J Prev Med 2000;18(suppl 4):37–50. 7. Beahler CC, Sundheim JJ, Trapp NI. Information retrieval in systematic reviews: challenges in the public health arena. Am J Prev Med 2000; 18(suppl 4):6 –11. 8. Kiefer MC. Effectiveness of interventions in reducing pesticide overexposure and poisonings. Am J Prev Med 2000;18(suppl 4):80 – 89. 9. Lipscomb JH. Effectiveness of interventions to prevent work-related eye injuries. Am J Prev Med 2000;18(suppl 4):27–32. 10. Reed DB, Claunch DT. Nonfatal farm injury and disability to children: a systematic review. Am J Prev Med 2000;18(suppl 4):70 –79. 11. Reynolds SJ, Groves W. Effectiveness of roll-over protective structures in reducing farm tractor fatalities. Am J Prev Med 2000;18(suppl 4):63– 69. 12. Rivara FP, Thompson DC. Prevention of falls in the construction industry: evidence for program effectiveness. Am J Prev Med 2000;18(suppl 4):23– 26. 13. Runyan CW, Zakocs RC, Zwerling C. Administrative and behavioral interventions for workplace violence prevention. Am J Prev Med 2000;18(suppl 4):116 –127. 14. Segui-Gomez M. Evaluating worksite-based interventions that promote safety belt use. Am J Prev Med 2000;18(suppl 4):11–22.
Appendix The Databases Agricola 1970 –1999. Contains over 3 million records covering every major agricultural subject. Produced by the National Agricultural Library, this database contains articles, pamphlets, research reports and government documents on agricultural marketing, foods and feeds, farm technology, animal husbandry, and allied sciences. Updated monthly. Agris International 1975–1999. Covers agricultural and rural development literature on a worldwide basis. This database is produced by the Food and Agriculture Organization of the United Nations. Books in Print. Database containing 1.8 million records of books in print, out of print, and forthcoming from North American publishers. Criminal Justice Periodical Index (CJPI). Database with coverage of criminal justice journals and newsletters in the United States, Canada, and United Kingdom. Dissertation Abstracts 1861–1999. Indexing and abstracts for dissertations from North American and European universities. Contains 1.5 million entries. EMBASE 1974 –1999. Contains abstracts and citations to the international literature on drugs, pharmacology, and human medicine, including European and Asian publications. Articles appearing in EMBASE are largely peer reviewed. Am J Prev Med 2000;18(4S)
9
ERIC 1966 –1999. Educational Resources Information Center database. Indexes journal articles and documents in the field of education. Updated monthly.
from U.S. federal agencies, contractors or grantees; and research and development reports sponsored by governments of other industrialized nations.
Expanded Academic Index 1989 –1999. Indexing and abstracts for approximately 1500 scholarly and general interest periodicals covering major fields of study in the humanities, social sciences, and technology. Updated monthly.
Nursing and Allied Health 1982–1999. Provides coverage of the literature in nursing and 17 allied health disciplines. Over 700 journals are regularly indexed. Updated monthly.
Health Star 1975–1999. One of the National Library of Medicine’s bibliographic databases, providing access to the published literature of Health Services Technology, Administration and Research. NIOSHTIC 1973–1999. Covers occupational safety and health. Sources include about 160 core journals (about 33% of the database), non-core journals (33%), NIOSH publications (7%), conference and symposium proceedings (5%), and translations (5%). Subjects covered include: occupational health and safety programs, hazardous substances, toxicology, unsafe workplace environment, epidemiology, pathology and histology, occupational medicine, ergonomics, industrial hygiene, behavioral sciences, biochemistry, physiology and metabolism, chemistry, control technology, education and training, engineering, health physics, and safety. NTIS 1983–1999. Contains U.S. government-sponsored research; development and engineering analysis
10
PAIS International 1972–1999 (Public Affairs Information). Covers social and public policy literature from over 1400 periodicals and thousands of government documents, books, and other sources. Updated monthly. PsycINFO 1967–1999. Indexes and abstracts articles from over 1300 journals in the field of psychology. Updated monthly. Sociofile 1974 –1999. Contains journal article abstracts and citations for books, book reviews, conference papers, and dissertations published in Sociological Abstracts and SOPODA. Updated three times per year. TRIS (Transportation Research Information Services). Covers all surface transportation modes, air transport, and highway safety.
American Journal of Preventive Medicine, Volume 18, Number 4S