- Email: [email protected]
Meta-analysis and public policy: Opportunity for nursing impact
Meta-analysis and Public Policy: Opportunity for Nursing Impact Vicki S. Conn, RIM, PhD Jane M. Armer, RN, PhD
Nurses must be prepared to use multiple and varied strategies to influence public policy so that optimal health outcomes can be achieved. Meta-analysis, a powerful research tool, is one such strategy that can be used to inform public policy.
E
arlY leaders in nursing, including Florence Nightingale, Lavinia Dock, and Lillian Wald, modeled political activism as an important component of nursing efforts to improve health and well-being. Despite the example set by these early leaders, widespread political involvement by nurses is yet to be realized) However, during the past decade there has been a heightened awareness among nurses that the forces shaping nursing practice are strongly influenced by public policy and that many health problems could be profoundly affected by public policy, z Nursing is uniquely poised to address many health problems that can be modified by public policy. Nurses already provide care to rnany underserved populations. 3 Health care policy makers are beginning to recognize the potential value in shifting emphasis from disease management to disease prevention and health promotion, which are traditional areas of nursing practice. Nurses are well positioned to solicit the support of important political constituencies because nurses share common goals with clients. Public health nurses are strategically and philosophically prepared to play a vital role in population.based efforts to enhance care. 4 Furthermore, many of the issues that afNurs Outlook 1996;44:267-71. Copyright © 1996 by Mosby-Year Book, Inc. 0029-6554/96/$5.00 + 0 3511170062
NURSING OUTLOOK
fect health require a holistic perspective, that is, a recognition that individuals are embedded in family and community syst e m s - a n o t h e r traditional nursing viewpoint. Clearly, nurses can make valuable contributions to public policy discussions and decisions. Nurses must be prepared to use multiple and varied strategies to influence public policy so that optimal health outcomes can be achieved. 5 Carefully conducted, well-designed research is one instrument that can be used to influence policy. This article will focus on the potential contribution of meta-analysis research methods to the process of informing public policy regarding health care issues. First, meta-analysis will be briefly described; next, the potential uses of meta-analysis at each stage of policy formation and evaluation will be presented; and finally, some realities of the political process will be reviewed. DESCRIPTION OF META-ANALYSIS Meta-analysis, which is often defined as an analysis of analyses, is a means of quantitatively summarizing the pooled results of multiple previously conducted primary studies. More specifically, meta-analysis involves the application of the research process to a group of individual primary studies when each study addresses the same fundamental question.
NOVEMBER/DECEMBER 1996
The initial stage of a meta-analysis is determination of the research question. After reading the available literature, the investigator makes important decisions about how narrow or broad to make the research question. The investigator then gathers subjects, with each primary research study (often one published research report) becoming one subject in the metaanalysis. Next, the investigator collects data from the primary study, such as the characteristics of any intervention, the study design, or the characteristics of the sample, and uses statistical analysis to summarize the findings. Finally, the metaanalysis is completed by interpreting the findings and suggesting future research and practice changes. Further descriptions of meta-analysis are readily available elsewhere for those unfamiliar with the process and those in-
Nursing is uniquely poised to address many health problems that can be modified by public policy. terested in meta-analysis statistical methodsk s Although the methodological decisions made in meta-analysis continue to generate considerable discussion, most social, psychological, and physical scientists accept the merit of this form of research. Some readers may be interested in several recent publications that identify limitations of meta-analysis. 9~1 META-ANALYSIS AND PUBLIC POLICY
History An early advocate of quantitatively summarizing studies, especially for use in makConn and Armer
267
Use of Meta-analysis at Different Stages of Public Policy * • • • •
Summarizes numerous primary studies May find ample effects in small sample studies Has the potential to find consistency amid apparent inconsistency Examines multiple outcomes Examines program effectiveness Does the program produce the desired outcome? For whom is the program effective? When is the program effective? Do circumstances exist in which the program is not effective? Does the program have characteristics that affect program effectiveness? Can cost be linked to effectiveness?
ing public policy decisions, was Richard Light. In 1970 the Department of Health, Education, and Welfare asked Light to determine whether the Head Start program worked. Light was confronted with contradictory findings from 13 studies. The prospect of drawing conclusions from these 13 studies was so problematic that he subsequently published a call for quantitative reviews of research findings.12 Policy makers frequently face a similar array of research findings that are often contradictory, rarely definitive, approach the problem from different angles, and are not helpful in making important policy decisions) z Although no research method can answer all policy questions, metaanalysis can provide useful information when multiple primary studies exist.
Strengths of Met&analysis as a Tool to Influence Public Policy Meta-analysis has many strengths that make it an attractive research strategy for
Meta-analysis can amplify effects that are too small to be observable in single studies and in studies with sma II samples.
informing public policy (Box 1), including the ability to summarize large quantities of data. It is the nature of researchers to want to conduct "just one more study" in the hope that it will provide a definitive answer to a research question; however, policy makers are often overwhelmed with the information already available. This sense of an overabundance of data is understandable, given the ex268
Conn and Armer
istence of approximately 25,000 health and health-related journals. 13 For example, t h e r e h a v e b e e n m o r e t h a n 100,000 studies of depression, lz The existence of a large number of primary studies has fueled efforts to use meta-analysis to summarize the results. Meta-analysis goes beyond the summaries made possible by narrative reviews of the literature and may, in fact, produce findings opposite to those produced when a "vote counting" system is used to review existing studies. In some literature reviews, vote counting consists of comparing the number of studies, or votes, that support a particular finding with the number that do not report a similar finding. Mann lz noted that vote counting may be particularly vulnerable to type II errors; this may be a contributing factor to the different findings obtained when vote counting methods as opposed to meta-analysis are applied to similar primary studies. Meta-analysis is an important tool in affecting public policy not only because it is capable of summarizing large amounts of data but also because it is able to make use of data from single studies and studies with small samples. Many nursing studies have been plagued by small samples, which often prevents the study from achieving adequate statistical power to reasonably test the study hypothesis. Meta-analysis can amplify effects that are too small to be observable in single studies and in studies with small samples5 z In addition, metaanalysis may reveal a consistent pattern in apparently contradictory studies .2 and is capable of examining multiple outcomes, whereas single research studies are often limited to one or two outcomes. These strengths of meta-analysis contribute to its potential value for influencing public policy.
Development Public policy has been defined as % course of action or inaction, either explicit or implicit, that is consciously or purposely chosen in the interest of addressing concerns or resolving some problem in a particular or specific way; policy-makers are those with legitimate power to make decisions and/or allocate scarce resources toward such ends. ''14 Stages of public policy development include identifying important problems, placing a problem on the policy agenda, formulating possible options to address the problem, adopting a particular policy, implementing the selected policy, and evaluating policy55'1~ Nurses may influence public policy at each of these stages of policy development through formal mechanisms such as testifying before committees, providing written statements at public hearings, or serving as experts to address components of proposed policies. Meta-analysis can provide information that nurses can use to encourage particular policy options. Planning an Effective M e t a - A n a l y sis. The same careful planning that precedes other forms of research is necessary
in meta-analysis. The nurse who is planning to conduct a meta-analysis to obtain information that can be used to influence health-related policy should consider forming a multi-disciplinary team to conduct the study. Because most health care problems amenable to policy are multifaceted and complex, expertise from members of
Any meta-analysis being conducted for the purpose of affecting public policy should include any cost data that are available, because cost is always an issue in public policy debates. III multiple health care disciplines is required to address the problem. 17 Even in the research planning stages, politically savvy nurses will think of ways to recruit allies; involvement of multiple disciplines will enhance the probability that interest groups associated with those disciplines will support the proposed policy. When
VOLUME 44 • NUMBER 6
NURSING OUTLOOK
planning a meta-analysis, it is also important to remember that most public policy changes occur in an incremental fashion over long periods of time) 7 The nurse researcher may want to modify research questions to produce answers that could become part of gradual changes in policy. In addition, any meta-analysis being conducted for the .purpose of affecting public policy should include any cost data that are available, because cost is always an issue in public policy debates. Meta-analysis at the stages of policy formation and evaluation. Meta-analysis can be useful at the first stages of policy development, when important health problems are identified and issues are placed on political agendas. Policy makers select and vote for issues on the basis of their perceptions that voting constituents are interested in particular topics. Meta-analyses that summarize the health care concerns of geographically dispersed, diverse populations help inform policy-makers about the public's interest in and support for particular topics and policies. Part of the process of exploring potential alternative policies to address problems is an examination of the effectiveness of programs or interventions. Compared with the time and cost involved in conducting additional primary studies, meta-analysis provides a more inexpensive and rapid means of gathering data that address effectiveness. The results of meta-analysis are often perceived as highly generalizable because the primary studies, which addressed similar or identical research questions, used different study designs, diverse eligibility criteria, varied methods of measuring outcomes, and perhaps divergent implementations of interventions.I8 Selection of the outcomes to examine through meta-analysis should reflect the concerns of policy makers. Policy makers are generally interested in cost of care, quality of care, and access to care. A metaanalysis that addresses components of each of these areas is particularly useful. Furthermore, the nurse researcher should determine which outcomes to study on the basis of the particular interests of groups likely to influence the policy process. There are 12,000 registered lobbyists in Washington, D.C. 19 The astute nurse will identify potential interest groups and plan a metaanalysis that will address the concerns of
NURSING OUTLOOK
those groups. For example, midwives have conducted timely and welt-planned studies that document both cost-effectiveness of care delivered and quality of care.Z° The ability to examine multiple outcomes is a reason meta-analysis is so attractive for policy planning. Meta-analysis can often go beyond the basic question about program or intervention effectiveness to examine the conditions under which programs are effective and the target populations for whom particular programs are effective. In addition, meta-analysis can identify interactions, such as those between types of programs and the types
The astute nurse will identify potential interest groups and plan a metaanalysis that will address the concerns of those groups.
of subjects exposed to the programs. Metaanalysis is valuable at the level of outlining policy alternatives because it can often determine if interventions or programs are differentially effective. Clearly identifying a target population for whom the intervention is effective is very helpful in policy development because scarce resources must be targeted to those most at risk and those most likely to benefit from policy initiatives. For example, efforts to reduce teenage pregnancy would benefit from information about the type of program that is most effective for 12- to 14year-old girls as opposed to 16- to 18-yearold girls. Meta-analysis can often provide this data if sufficient primary studies exist. A recent meta-analysis by Tobler provides a good example of the ability of metaanalysis to inform public policy. Tobler's meta-analysis revealed that, contrary to popular opinion, drug prevention programs were equally effective for junior high and high school students and for rural, suburban, and urban students, zl However, this meta-analysis found that the type of drug p r e v e n t i o n program leader (a mental health professional compared with teachers, peers, or others) dramatically affected the success of the program. The results of this meta-analysis suggest that funding of programs should be linked to use of mental health professionals as program lead-
NOVEMBER/DECEMBER1996
ers. Tobler's meta-analysis also found that peer-support-type programs were more effective than other programs, including the knowtedge-plus-affective-style programs c o m m o n l y adopted. Furthermore, the meta-analysis was able to examine interaction effects and found that alcohol prevention program effectiveness was related to the type of program, tobacco prevention program effectiveness was more independent of the type of program, and both were affected by the characteristics of the leader. 21 Policy makers are increasingly interested in the potential for differential impact of policies by gender or socioeconomic status and by cultural, ethnic, and racial groupings. Although most individual studies are unable to address these factors, many meta-analyses can include these factors as independent variables if sufficient studies have included diverse samples and reported data accordingly. Policy makers want to propose solutions that their voting constituencies will find acceptable. Studies that identify the acceptance by particular populations of alternative health care deliver,/- systems or providers, such as Armer's z2'z3recent work on receptivity to nurse practitioners among rural older adults, are needed as well. Nurse activists should continue involvement after policies have been adopted. Public policy has been compared to a hypothesis that should then be tested through evaluation. > Meta-analysis of multiple program site evaluations with diverse populations can provide evidence regarding the effectiveness of policy initiatives and the generalizability of the effectiveness.
Use of existing meta-analyses to in fluen ce pubfic policy In some cases, nurses may wish to use the results of meta-analysis conducted by others to influence public policy. Recent metaanalyses that could inform public policy are listed in Table 1. Nurses who use meta-analysis that they have not conducted themselves should carefully examine the meta-analysis with the understanding that investigators make decisions at each stage of the research process. For example, the effectiveness of condoms in prevention of HIV transmission was the focus of a recent meta-analysis. The analysis found a significant HIV transmission rate. Careful scrutiny revealed that
Conn and Armer 269
Title
First author
Year
Efficacy of BCG vaccine in the prevention of tuberculosis: Colditz 1994 Meta-analysis of the published literature Efficacy of screening mammography: A meta-analysis Kerlikowske 1995 Effects of rehabilitation exercise programmes on anxiety Kugler 1994 and depression in coronary patients: A meta-analysis Efficacy of pneumococcal vaccination in adults: Fine 1994 A meta-analysis of randomized controlled trials Relationships of measures of alcohol consumption Fillmore 1994 with alcohol-related problems in multiple studies: A research synthesis from collaborative alcohol-related longitudinal projects Environmental lead and children's intelligence: 1994 Pocock A systematic review of the epidemiological evidence A meta-analysis of randomized trials of prenatal Dolan-Mullen 1994 smoking cessation interventions Bendectin and birth defects: A meta-analysis of the 1994 McKeigue epidemiologic studies Does coffee drinking increase the risk of coronary 1994 Kawachi heart disease? Results from a meta-analysis How effective is drug abuse resistance education? 1994 Ennett A meta-analysis of Project DARE outcome evaluations Residential proximity to electricity transmission and 1994 Washburn distribution equipment and risk of childhood leukemia, lymphoma, and nervous system tumors: Systematic review, evaluation, and meta-analysis Review of dietary intervention studies: Effect on Truswell 1994 coronary events and on total mortality The effectiveness of the nicotine patch for Fiore 1994 smoking cessation: A meta-analysis Air pollution and daily mortality: A review and meta-analysis Schwartz 1994 the meta-analysis included studies of both method failure (condoms were consistently used) and user failure (the couple reported inconsistent use of condoms). Including user failure results is not a fair trial of condom effectiveness in preventing HIV transmission. As with any research, the reader and user of research must critically examine both the methods used and the results of the research. 6 The potential usefulness of meta-analysis to inform public policy decisions is enormous. This potential has been largely unrealized for several reasons. Meta-analysis has become widely accepted and used only within the past decade; thus nurses are often unable to find meta-analyses that provide the information needed. Sometimes not enough primary studies exist to conduct a meta-analysis. Most primary stud. ies have focused on effectiveness and have not collected or reported cost or access data. Regretfully, primary studies may lack 270
Conn and A r m e r
adequate rigor to be included in a metaanalysis, even in a meta-analysis that includes study quality as a variable. These limitations can provide the impetus for nurses to conduct research that has the potential to affect public policy.
HAVING AN IMPACT ON POLICYINFLUENCING GROUPS Determining public policy is part of the political process. Politics is the domain of practical decision-making, not universal truth. 25 Political decision making is about making decisions that are right for a specific issue, in a particular political climate, at a unique point in time. z5 Because policy making is a political process, nurses must remember that political processes have their own rationale. Many political decisions are values decisions. An important value consistently used by politicians to determine how they will vote on public health issues is that of individual free-
dora. 26 This seeming disregard for empirical findings can frustrate health professionals. For example, drug prevention programs often exclude tobacco and alcohol despite extensive data that tobacco and alcohol have more devastating public health consequences than all illegal drug use combined, z7 Rather than giving up attempts to tackle political issues because many political policy decisions seem to ignore empirical evidence, nurses can use data such as that derived from meta-analysis to provide correct information when information is important. Nurses need to use data in politically shrewd ways. Strategies for effective public policy advocacy are always important. 5 There is an old adage that all politics is local politics. Nurses intent on influencing policy makers should always consider the local ramifications of policies for elected officials and present findings from studies in light of local interests. Nurse scientists should be aware that whereas scientists want new data, policy-makers seek new allies. 26 Nurse activists are keenly aware t h a t different interest groups, even within nursing, value distinctly different aspects of particular policy issues2 Although the overall results of a meta-analysis will often be of interest to no one, the components of the results will be of considerable interest to different groups. Data should be presented in such a manner as to show how proposed policies will meet the needs of different interest groups. Meta-analysis methods will generally be of little interest to policy makers; rather, they are interested in a concise version of
Nurses intent on influencing policy makers should always consider the local ramifications of policies for elected officials and present findings from studies in light of local interests. the results. Nurses should provide graphic, brief summaries of findings, even those of complex meta-analysis studies. Verbal testimony should include only major points, and only a few crucial, striking numbers should be presented) ~ Policy makers are
VOLUME 44 • NUMBER 6
NURSING OUTLOOK
also i n f l u e n c e d by a n e c d o t a l e v i d e n c e , clinical examples, or case studies that add a h u m a n and e m o t i o n a l c o m p o n e n t to data that may otherwise seem cold and impersonal.
CONCLUSION Nursing's use of data to influence public policy is n o t a n e w d e v e l o p m e n t . T h e practice dates back to Florence N i g h t i n gale, w h o used statistical analysis and graphic r e p r e s e n t a t i o n to d o c u m e n t the relationship b e t w e e n d e a t h rates and imp r o v e m e n t s in care. zs T h e use of metaanalysis to influence public policy is n o t new, either. O n e of the earliest reported m e t a - a n a l y s e s was c o n d u c t e d by Karl Pearson, the n o t e d English statistician, who grouped data from British military studies and d e t e r m i n e d t h a t the popular practice of v a c c i n a t i o n against intestinal f e v e r was n o t effective, lz U s i n g m e t a analysis to influence public policy is one way nurses can c o n t i n u e our rich heritage of using data to improve important h e a l t h outcomes, z9 •
REFERENCES 1. Sapp M, Bliesmer M. A health promotion/ protection approach to meeting elders' health care needs through public policy and standards of care. In: Stanley M, Beare P, editors. Gerontological nursing. Philadelphia: FA Davis Company, 1995. 2. O'Neal D. Influencing policy in Washington, DC, through lobbying for nursing. AORN J 1994;60:478-81.
3. Anderson C. The enemy within. Nurs Outlook 1994;42:149-50. 4. Halbert T, Underwood J, Chambers L, Pleog J, Johnson N, Isaacs S. Population-based health promotion: a new agenda for public health nurses. Can J Public Health 1993;84:243-5. 5. Faherty B. Now is the time to advocate. Nurs Outlook !993;41:248-9. 6. Corm V, Armer J. Reading and using metaanalyses. Public Health Nurs 1994; 11:163-7. 7. Hedges L, Olkin I. Statistical methods for metaanalysis. New York: Academic Press, 1985. 8. Laird N, Mosteller E Some statistical methods for combining experimental studies. Int J Technol Assess Health Care 1990;6:5-30. 9. Eysenck H. Meta-analysis and its problems. Br Med J 1994;309:789-92. 10. Petitti D. Of babies and bathwater. Am J Epidemiol 1994;9:779-82. 11. Shapiro S. Meta-analysis/shxneta-analysis. Am J Epidemio11994;9:771-8. 12. Mann C. Can meta-analysis make policy? Science 1994;266:960-2. 13. Olkin I. Statistical and theoretical considerations in meta-analysis. J Clin Epidemiol 1995;48:133-46. 14. Moccia E The nurse as policymaker: toward a free and equal health care system. Nurs Health Care 1984;5(Nov):481-5. 15. Buerhaus E Teaching health care public policy. Nurs Health Care 1992; 13:304-9. 16. Melillo K. Preparing and presenting testimony: guidelines for the health care practitioner. Nuts Pract 1994; 19:59-64. 17. Nagelkerk J, Henry B. Leadership through policy research. J Nuts Adm 1991;21(5):20-4. 18. Mulrow C. Rationale for systematic reviews. Br Med J 1994;309:597-9. 19. Bocchino C, Wakefield M. Forces influencing public policy. Nuts Econ 1992;10:53-5. 20. Raudonis B, Griffith H. Model for integrating health services research and health care policy formation. Nurs Health Care 1991;12:32-6.
21. Tobler N. Drug prevention programs can work: research findings. J Addict Dis 1992; 11(3):1-28. 22. Armer J. A social survey: public perception of crises in health care and responses to advanced nursing role. Nurse Pract 1994; 15:18-20. 23. Armer J. Public response to the advanced nurse practice role in Missouri. Presented at 1995 National Conference on Rural Nursing, July 20-22, 1995, Bozeman, MT. 24. Stanley S. Crafting mental health policy. Nurs Clin North Am 1994;29:19-27. 25. Stevens B. Nursing, politics, and policy formulation. In: Wieczorek R, editor. Power, politics, and policy in nursing. New York: Springer Publishing, 1985:16-21. 26. Lowenstein S, Koziol-McLain J, Satterfield G, Orleans M. Facts versus values: why legislators vote against injury control laws. J Trauma 1993;35:786-93. 27. Mosher J, Yanagisako K. Public health, not social warfare: a public health approach to illegal drug policy. J Public Health Policy 1991; 12:278-321. 28. Goldwater M, Zusy M. Prescription for nurses: effective political action. St. Louis: MosbyYear Book, 1990. 29. McBride A. From the president: on being opportunistic. Nurs Outlook 1993;41:275-6.
VICKI S. CONN is a postdoctoral fellow at the College of Nursing, University of Iowa, and an associate professor and co-director of research at the School of Nursing, University of Missouri, Columbia. JANE M. ARMER is a postdoctoral fellow at the College of Nursing, University of Iowa, and an assistant professor and co-director of research at the School of Nursing, University of Missouri, Columbia.
Senior Scholar in Residence American Academy of Nursing and the Agency for Health Care Policy and Research Purpose of the Program: The American Academy of Nursing (AAN) and the Agency for Health Care Policy and Research (AHCPR) are seeking applicants for the position of Senior Scholar in Residence for 1997-1998. The purpose is to encourage a senior nurse scientist to develop areas of investigation that integrate clinical nursing care questions with critical issues of quality, costs, and access. The scholar will provide linkages between the private and public sectors in areas of research that affect health policy issues of national concern. Qualifications: Candidates should be experienced nurse investigators with an advanced degree in health services research or related area preferably with clinical experience in the delivery of primary care, or experience in per-
NURSING OUTLOOK
forming policy analyses related to primary care. Competitive candidates will have demonstrated research, writing, publication, and teaching experience. The Scholar must be able to conceptualize research probtems in a broad framework; develop and apply innovative techniques appropriate to a multidisciplinary framework; and synthesize findings into a pattern that takes into account the social, political, legal, and epidemiologic factors that affect the delivery of health care. Candidates should be prepared to return to an enhanced research career and facilitate the development of research in others. Interested applicants should submit a curricuIum vitae, descriptions of pertinent experience and interests, and a two-page letter of intent describing the project they would like to undertake during the Scholar in Residence program. A cover
NOVEMBER/DECEMBER1996
letter describing career objectives, experience, and future potential relating to the Senior scholar experience is desirable. The most highly qualified applicants may be interviewed. For further information, please contact Carolyn Clancy, MD, AHCPR, Center for Primary Care Research, Executive Office Center, Suite 502, 2102 East Jefferson Street, Rockville, MD 20852-4908. Phone: (301) 5941357, ext. 129; fax: (301) 594-3721; e-mail: cclancy@po3 .ahcpr.gov. Applications should be submitted to: Janet Heinrich, DrPH, RN, FAAN, Director, American Academy of Nursing, 600 Maryland Ave., SW, Washington, DC 20024-2571. Phone: (202) 651-7238; fax: (202) 554-2641; e-mail: jheinrich@ ANA.org. Applications are due by January 15, 1997.
Conn and Armer 271
Nurses must be prepared to use multiple and varied strategies to influence public policy so that optimal health outcomes can be achieved. Meta-analysis, a powerful research tool, is one such strategy that can be used to inform public policy.
E
arlY leaders in nursing, including Florence Nightingale, Lavinia Dock, and Lillian Wald, modeled political activism as an important component of nursing efforts to improve health and well-being. Despite the example set by these early leaders, widespread political involvement by nurses is yet to be realized) However, during the past decade there has been a heightened awareness among nurses that the forces shaping nursing practice are strongly influenced by public policy and that many health problems could be profoundly affected by public policy, z Nursing is uniquely poised to address many health problems that can be modified by public policy. Nurses already provide care to rnany underserved populations. 3 Health care policy makers are beginning to recognize the potential value in shifting emphasis from disease management to disease prevention and health promotion, which are traditional areas of nursing practice. Nurses are well positioned to solicit the support of important political constituencies because nurses share common goals with clients. Public health nurses are strategically and philosophically prepared to play a vital role in population.based efforts to enhance care. 4 Furthermore, many of the issues that afNurs Outlook 1996;44:267-71. Copyright © 1996 by Mosby-Year Book, Inc. 0029-6554/96/$5.00 + 0 3511170062
NURSING OUTLOOK
fect health require a holistic perspective, that is, a recognition that individuals are embedded in family and community syst e m s - a n o t h e r traditional nursing viewpoint. Clearly, nurses can make valuable contributions to public policy discussions and decisions. Nurses must be prepared to use multiple and varied strategies to influence public policy so that optimal health outcomes can be achieved. 5 Carefully conducted, well-designed research is one instrument that can be used to influence policy. This article will focus on the potential contribution of meta-analysis research methods to the process of informing public policy regarding health care issues. First, meta-analysis will be briefly described; next, the potential uses of meta-analysis at each stage of policy formation and evaluation will be presented; and finally, some realities of the political process will be reviewed. DESCRIPTION OF META-ANALYSIS Meta-analysis, which is often defined as an analysis of analyses, is a means of quantitatively summarizing the pooled results of multiple previously conducted primary studies. More specifically, meta-analysis involves the application of the research process to a group of individual primary studies when each study addresses the same fundamental question.
NOVEMBER/DECEMBER 1996
The initial stage of a meta-analysis is determination of the research question. After reading the available literature, the investigator makes important decisions about how narrow or broad to make the research question. The investigator then gathers subjects, with each primary research study (often one published research report) becoming one subject in the metaanalysis. Next, the investigator collects data from the primary study, such as the characteristics of any intervention, the study design, or the characteristics of the sample, and uses statistical analysis to summarize the findings. Finally, the metaanalysis is completed by interpreting the findings and suggesting future research and practice changes. Further descriptions of meta-analysis are readily available elsewhere for those unfamiliar with the process and those in-
Nursing is uniquely poised to address many health problems that can be modified by public policy. terested in meta-analysis statistical methodsk s Although the methodological decisions made in meta-analysis continue to generate considerable discussion, most social, psychological, and physical scientists accept the merit of this form of research. Some readers may be interested in several recent publications that identify limitations of meta-analysis. 9~1 META-ANALYSIS AND PUBLIC POLICY
History An early advocate of quantitatively summarizing studies, especially for use in makConn and Armer
267
Use of Meta-analysis at Different Stages of Public Policy * • • • •
Summarizes numerous primary studies May find ample effects in small sample studies Has the potential to find consistency amid apparent inconsistency Examines multiple outcomes Examines program effectiveness Does the program produce the desired outcome? For whom is the program effective? When is the program effective? Do circumstances exist in which the program is not effective? Does the program have characteristics that affect program effectiveness? Can cost be linked to effectiveness?
ing public policy decisions, was Richard Light. In 1970 the Department of Health, Education, and Welfare asked Light to determine whether the Head Start program worked. Light was confronted with contradictory findings from 13 studies. The prospect of drawing conclusions from these 13 studies was so problematic that he subsequently published a call for quantitative reviews of research findings.12 Policy makers frequently face a similar array of research findings that are often contradictory, rarely definitive, approach the problem from different angles, and are not helpful in making important policy decisions) z Although no research method can answer all policy questions, metaanalysis can provide useful information when multiple primary studies exist.
Strengths of Met&analysis as a Tool to Influence Public Policy Meta-analysis has many strengths that make it an attractive research strategy for
Meta-analysis can amplify effects that are too small to be observable in single studies and in studies with sma II samples.
informing public policy (Box 1), including the ability to summarize large quantities of data. It is the nature of researchers to want to conduct "just one more study" in the hope that it will provide a definitive answer to a research question; however, policy makers are often overwhelmed with the information already available. This sense of an overabundance of data is understandable, given the ex268
Conn and Armer
istence of approximately 25,000 health and health-related journals. 13 For example, t h e r e h a v e b e e n m o r e t h a n 100,000 studies of depression, lz The existence of a large number of primary studies has fueled efforts to use meta-analysis to summarize the results. Meta-analysis goes beyond the summaries made possible by narrative reviews of the literature and may, in fact, produce findings opposite to those produced when a "vote counting" system is used to review existing studies. In some literature reviews, vote counting consists of comparing the number of studies, or votes, that support a particular finding with the number that do not report a similar finding. Mann lz noted that vote counting may be particularly vulnerable to type II errors; this may be a contributing factor to the different findings obtained when vote counting methods as opposed to meta-analysis are applied to similar primary studies. Meta-analysis is an important tool in affecting public policy not only because it is capable of summarizing large amounts of data but also because it is able to make use of data from single studies and studies with small samples. Many nursing studies have been plagued by small samples, which often prevents the study from achieving adequate statistical power to reasonably test the study hypothesis. Meta-analysis can amplify effects that are too small to be observable in single studies and in studies with small samples5 z In addition, metaanalysis may reveal a consistent pattern in apparently contradictory studies .2 and is capable of examining multiple outcomes, whereas single research studies are often limited to one or two outcomes. These strengths of meta-analysis contribute to its potential value for influencing public policy.
Development Public policy has been defined as % course of action or inaction, either explicit or implicit, that is consciously or purposely chosen in the interest of addressing concerns or resolving some problem in a particular or specific way; policy-makers are those with legitimate power to make decisions and/or allocate scarce resources toward such ends. ''14 Stages of public policy development include identifying important problems, placing a problem on the policy agenda, formulating possible options to address the problem, adopting a particular policy, implementing the selected policy, and evaluating policy55'1~ Nurses may influence public policy at each of these stages of policy development through formal mechanisms such as testifying before committees, providing written statements at public hearings, or serving as experts to address components of proposed policies. Meta-analysis can provide information that nurses can use to encourage particular policy options. Planning an Effective M e t a - A n a l y sis. The same careful planning that precedes other forms of research is necessary
in meta-analysis. The nurse who is planning to conduct a meta-analysis to obtain information that can be used to influence health-related policy should consider forming a multi-disciplinary team to conduct the study. Because most health care problems amenable to policy are multifaceted and complex, expertise from members of
Any meta-analysis being conducted for the purpose of affecting public policy should include any cost data that are available, because cost is always an issue in public policy debates. III multiple health care disciplines is required to address the problem. 17 Even in the research planning stages, politically savvy nurses will think of ways to recruit allies; involvement of multiple disciplines will enhance the probability that interest groups associated with those disciplines will support the proposed policy. When
VOLUME 44 • NUMBER 6
NURSING OUTLOOK
planning a meta-analysis, it is also important to remember that most public policy changes occur in an incremental fashion over long periods of time) 7 The nurse researcher may want to modify research questions to produce answers that could become part of gradual changes in policy. In addition, any meta-analysis being conducted for the .purpose of affecting public policy should include any cost data that are available, because cost is always an issue in public policy debates. Meta-analysis at the stages of policy formation and evaluation. Meta-analysis can be useful at the first stages of policy development, when important health problems are identified and issues are placed on political agendas. Policy makers select and vote for issues on the basis of their perceptions that voting constituents are interested in particular topics. Meta-analyses that summarize the health care concerns of geographically dispersed, diverse populations help inform policy-makers about the public's interest in and support for particular topics and policies. Part of the process of exploring potential alternative policies to address problems is an examination of the effectiveness of programs or interventions. Compared with the time and cost involved in conducting additional primary studies, meta-analysis provides a more inexpensive and rapid means of gathering data that address effectiveness. The results of meta-analysis are often perceived as highly generalizable because the primary studies, which addressed similar or identical research questions, used different study designs, diverse eligibility criteria, varied methods of measuring outcomes, and perhaps divergent implementations of interventions.I8 Selection of the outcomes to examine through meta-analysis should reflect the concerns of policy makers. Policy makers are generally interested in cost of care, quality of care, and access to care. A metaanalysis that addresses components of each of these areas is particularly useful. Furthermore, the nurse researcher should determine which outcomes to study on the basis of the particular interests of groups likely to influence the policy process. There are 12,000 registered lobbyists in Washington, D.C. 19 The astute nurse will identify potential interest groups and plan a metaanalysis that will address the concerns of
NURSING OUTLOOK
those groups. For example, midwives have conducted timely and welt-planned studies that document both cost-effectiveness of care delivered and quality of care.Z° The ability to examine multiple outcomes is a reason meta-analysis is so attractive for policy planning. Meta-analysis can often go beyond the basic question about program or intervention effectiveness to examine the conditions under which programs are effective and the target populations for whom particular programs are effective. In addition, meta-analysis can identify interactions, such as those between types of programs and the types
The astute nurse will identify potential interest groups and plan a metaanalysis that will address the concerns of those groups.
of subjects exposed to the programs. Metaanalysis is valuable at the level of outlining policy alternatives because it can often determine if interventions or programs are differentially effective. Clearly identifying a target population for whom the intervention is effective is very helpful in policy development because scarce resources must be targeted to those most at risk and those most likely to benefit from policy initiatives. For example, efforts to reduce teenage pregnancy would benefit from information about the type of program that is most effective for 12- to 14year-old girls as opposed to 16- to 18-yearold girls. Meta-analysis can often provide this data if sufficient primary studies exist. A recent meta-analysis by Tobler provides a good example of the ability of metaanalysis to inform public policy. Tobler's meta-analysis revealed that, contrary to popular opinion, drug prevention programs were equally effective for junior high and high school students and for rural, suburban, and urban students, zl However, this meta-analysis found that the type of drug p r e v e n t i o n program leader (a mental health professional compared with teachers, peers, or others) dramatically affected the success of the program. The results of this meta-analysis suggest that funding of programs should be linked to use of mental health professionals as program lead-
NOVEMBER/DECEMBER1996
ers. Tobler's meta-analysis also found that peer-support-type programs were more effective than other programs, including the knowtedge-plus-affective-style programs c o m m o n l y adopted. Furthermore, the meta-analysis was able to examine interaction effects and found that alcohol prevention program effectiveness was related to the type of program, tobacco prevention program effectiveness was more independent of the type of program, and both were affected by the characteristics of the leader. 21 Policy makers are increasingly interested in the potential for differential impact of policies by gender or socioeconomic status and by cultural, ethnic, and racial groupings. Although most individual studies are unable to address these factors, many meta-analyses can include these factors as independent variables if sufficient studies have included diverse samples and reported data accordingly. Policy makers want to propose solutions that their voting constituencies will find acceptable. Studies that identify the acceptance by particular populations of alternative health care deliver,/- systems or providers, such as Armer's z2'z3recent work on receptivity to nurse practitioners among rural older adults, are needed as well. Nurse activists should continue involvement after policies have been adopted. Public policy has been compared to a hypothesis that should then be tested through evaluation. > Meta-analysis of multiple program site evaluations with diverse populations can provide evidence regarding the effectiveness of policy initiatives and the generalizability of the effectiveness.
Use of existing meta-analyses to in fluen ce pubfic policy In some cases, nurses may wish to use the results of meta-analysis conducted by others to influence public policy. Recent metaanalyses that could inform public policy are listed in Table 1. Nurses who use meta-analysis that they have not conducted themselves should carefully examine the meta-analysis with the understanding that investigators make decisions at each stage of the research process. For example, the effectiveness of condoms in prevention of HIV transmission was the focus of a recent meta-analysis. The analysis found a significant HIV transmission rate. Careful scrutiny revealed that
Conn and Armer 269
Title
First author
Year
Efficacy of BCG vaccine in the prevention of tuberculosis: Colditz 1994 Meta-analysis of the published literature Efficacy of screening mammography: A meta-analysis Kerlikowske 1995 Effects of rehabilitation exercise programmes on anxiety Kugler 1994 and depression in coronary patients: A meta-analysis Efficacy of pneumococcal vaccination in adults: Fine 1994 A meta-analysis of randomized controlled trials Relationships of measures of alcohol consumption Fillmore 1994 with alcohol-related problems in multiple studies: A research synthesis from collaborative alcohol-related longitudinal projects Environmental lead and children's intelligence: 1994 Pocock A systematic review of the epidemiological evidence A meta-analysis of randomized trials of prenatal Dolan-Mullen 1994 smoking cessation interventions Bendectin and birth defects: A meta-analysis of the 1994 McKeigue epidemiologic studies Does coffee drinking increase the risk of coronary 1994 Kawachi heart disease? Results from a meta-analysis How effective is drug abuse resistance education? 1994 Ennett A meta-analysis of Project DARE outcome evaluations Residential proximity to electricity transmission and 1994 Washburn distribution equipment and risk of childhood leukemia, lymphoma, and nervous system tumors: Systematic review, evaluation, and meta-analysis Review of dietary intervention studies: Effect on Truswell 1994 coronary events and on total mortality The effectiveness of the nicotine patch for Fiore 1994 smoking cessation: A meta-analysis Air pollution and daily mortality: A review and meta-analysis Schwartz 1994 the meta-analysis included studies of both method failure (condoms were consistently used) and user failure (the couple reported inconsistent use of condoms). Including user failure results is not a fair trial of condom effectiveness in preventing HIV transmission. As with any research, the reader and user of research must critically examine both the methods used and the results of the research. 6 The potential usefulness of meta-analysis to inform public policy decisions is enormous. This potential has been largely unrealized for several reasons. Meta-analysis has become widely accepted and used only within the past decade; thus nurses are often unable to find meta-analyses that provide the information needed. Sometimes not enough primary studies exist to conduct a meta-analysis. Most primary stud. ies have focused on effectiveness and have not collected or reported cost or access data. Regretfully, primary studies may lack 270
Conn and A r m e r
adequate rigor to be included in a metaanalysis, even in a meta-analysis that includes study quality as a variable. These limitations can provide the impetus for nurses to conduct research that has the potential to affect public policy.
HAVING AN IMPACT ON POLICYINFLUENCING GROUPS Determining public policy is part of the political process. Politics is the domain of practical decision-making, not universal truth. 25 Political decision making is about making decisions that are right for a specific issue, in a particular political climate, at a unique point in time. z5 Because policy making is a political process, nurses must remember that political processes have their own rationale. Many political decisions are values decisions. An important value consistently used by politicians to determine how they will vote on public health issues is that of individual free-
dora. 26 This seeming disregard for empirical findings can frustrate health professionals. For example, drug prevention programs often exclude tobacco and alcohol despite extensive data that tobacco and alcohol have more devastating public health consequences than all illegal drug use combined, z7 Rather than giving up attempts to tackle political issues because many political policy decisions seem to ignore empirical evidence, nurses can use data such as that derived from meta-analysis to provide correct information when information is important. Nurses need to use data in politically shrewd ways. Strategies for effective public policy advocacy are always important. 5 There is an old adage that all politics is local politics. Nurses intent on influencing policy makers should always consider the local ramifications of policies for elected officials and present findings from studies in light of local interests. Nurse scientists should be aware that whereas scientists want new data, policy-makers seek new allies. 26 Nurse activists are keenly aware t h a t different interest groups, even within nursing, value distinctly different aspects of particular policy issues2 Although the overall results of a meta-analysis will often be of interest to no one, the components of the results will be of considerable interest to different groups. Data should be presented in such a manner as to show how proposed policies will meet the needs of different interest groups. Meta-analysis methods will generally be of little interest to policy makers; rather, they are interested in a concise version of
Nurses intent on influencing policy makers should always consider the local ramifications of policies for elected officials and present findings from studies in light of local interests. the results. Nurses should provide graphic, brief summaries of findings, even those of complex meta-analysis studies. Verbal testimony should include only major points, and only a few crucial, striking numbers should be presented) ~ Policy makers are
VOLUME 44 • NUMBER 6
NURSING OUTLOOK
also i n f l u e n c e d by a n e c d o t a l e v i d e n c e , clinical examples, or case studies that add a h u m a n and e m o t i o n a l c o m p o n e n t to data that may otherwise seem cold and impersonal.
CONCLUSION Nursing's use of data to influence public policy is n o t a n e w d e v e l o p m e n t . T h e practice dates back to Florence N i g h t i n gale, w h o used statistical analysis and graphic r e p r e s e n t a t i o n to d o c u m e n t the relationship b e t w e e n d e a t h rates and imp r o v e m e n t s in care. zs T h e use of metaanalysis to influence public policy is n o t new, either. O n e of the earliest reported m e t a - a n a l y s e s was c o n d u c t e d by Karl Pearson, the n o t e d English statistician, who grouped data from British military studies and d e t e r m i n e d t h a t the popular practice of v a c c i n a t i o n against intestinal f e v e r was n o t effective, lz U s i n g m e t a analysis to influence public policy is one way nurses can c o n t i n u e our rich heritage of using data to improve important h e a l t h outcomes, z9 •
REFERENCES 1. Sapp M, Bliesmer M. A health promotion/ protection approach to meeting elders' health care needs through public policy and standards of care. In: Stanley M, Beare P, editors. Gerontological nursing. Philadelphia: FA Davis Company, 1995. 2. O'Neal D. Influencing policy in Washington, DC, through lobbying for nursing. AORN J 1994;60:478-81.
3. Anderson C. The enemy within. Nurs Outlook 1994;42:149-50. 4. Halbert T, Underwood J, Chambers L, Pleog J, Johnson N, Isaacs S. Population-based health promotion: a new agenda for public health nurses. Can J Public Health 1993;84:243-5. 5. Faherty B. Now is the time to advocate. Nurs Outlook !993;41:248-9. 6. Corm V, Armer J. Reading and using metaanalyses. Public Health Nurs 1994; 11:163-7. 7. Hedges L, Olkin I. Statistical methods for metaanalysis. New York: Academic Press, 1985. 8. Laird N, Mosteller E Some statistical methods for combining experimental studies. Int J Technol Assess Health Care 1990;6:5-30. 9. Eysenck H. Meta-analysis and its problems. Br Med J 1994;309:789-92. 10. Petitti D. Of babies and bathwater. Am J Epidemiol 1994;9:779-82. 11. Shapiro S. Meta-analysis/shxneta-analysis. Am J Epidemio11994;9:771-8. 12. Mann C. Can meta-analysis make policy? Science 1994;266:960-2. 13. Olkin I. Statistical and theoretical considerations in meta-analysis. J Clin Epidemiol 1995;48:133-46. 14. Moccia E The nurse as policymaker: toward a free and equal health care system. Nurs Health Care 1984;5(Nov):481-5. 15. Buerhaus E Teaching health care public policy. Nurs Health Care 1992; 13:304-9. 16. Melillo K. Preparing and presenting testimony: guidelines for the health care practitioner. Nuts Pract 1994; 19:59-64. 17. Nagelkerk J, Henry B. Leadership through policy research. J Nuts Adm 1991;21(5):20-4. 18. Mulrow C. Rationale for systematic reviews. Br Med J 1994;309:597-9. 19. Bocchino C, Wakefield M. Forces influencing public policy. Nuts Econ 1992;10:53-5. 20. Raudonis B, Griffith H. Model for integrating health services research and health care policy formation. Nurs Health Care 1991;12:32-6.
21. Tobler N. Drug prevention programs can work: research findings. J Addict Dis 1992; 11(3):1-28. 22. Armer J. A social survey: public perception of crises in health care and responses to advanced nursing role. Nurse Pract 1994; 15:18-20. 23. Armer J. Public response to the advanced nurse practice role in Missouri. Presented at 1995 National Conference on Rural Nursing, July 20-22, 1995, Bozeman, MT. 24. Stanley S. Crafting mental health policy. Nurs Clin North Am 1994;29:19-27. 25. Stevens B. Nursing, politics, and policy formulation. In: Wieczorek R, editor. Power, politics, and policy in nursing. New York: Springer Publishing, 1985:16-21. 26. Lowenstein S, Koziol-McLain J, Satterfield G, Orleans M. Facts versus values: why legislators vote against injury control laws. J Trauma 1993;35:786-93. 27. Mosher J, Yanagisako K. Public health, not social warfare: a public health approach to illegal drug policy. J Public Health Policy 1991; 12:278-321. 28. Goldwater M, Zusy M. Prescription for nurses: effective political action. St. Louis: MosbyYear Book, 1990. 29. McBride A. From the president: on being opportunistic. Nurs Outlook 1993;41:275-6.
VICKI S. CONN is a postdoctoral fellow at the College of Nursing, University of Iowa, and an associate professor and co-director of research at the School of Nursing, University of Missouri, Columbia. JANE M. ARMER is a postdoctoral fellow at the College of Nursing, University of Iowa, and an assistant professor and co-director of research at the School of Nursing, University of Missouri, Columbia.
Senior Scholar in Residence American Academy of Nursing and the Agency for Health Care Policy and Research Purpose of the Program: The American Academy of Nursing (AAN) and the Agency for Health Care Policy and Research (AHCPR) are seeking applicants for the position of Senior Scholar in Residence for 1997-1998. The purpose is to encourage a senior nurse scientist to develop areas of investigation that integrate clinical nursing care questions with critical issues of quality, costs, and access. The scholar will provide linkages between the private and public sectors in areas of research that affect health policy issues of national concern. Qualifications: Candidates should be experienced nurse investigators with an advanced degree in health services research or related area preferably with clinical experience in the delivery of primary care, or experience in per-
NURSING OUTLOOK
forming policy analyses related to primary care. Competitive candidates will have demonstrated research, writing, publication, and teaching experience. The Scholar must be able to conceptualize research probtems in a broad framework; develop and apply innovative techniques appropriate to a multidisciplinary framework; and synthesize findings into a pattern that takes into account the social, political, legal, and epidemiologic factors that affect the delivery of health care. Candidates should be prepared to return to an enhanced research career and facilitate the development of research in others. Interested applicants should submit a curricuIum vitae, descriptions of pertinent experience and interests, and a two-page letter of intent describing the project they would like to undertake during the Scholar in Residence program. A cover
NOVEMBER/DECEMBER1996
letter describing career objectives, experience, and future potential relating to the Senior scholar experience is desirable. The most highly qualified applicants may be interviewed. For further information, please contact Carolyn Clancy, MD, AHCPR, Center for Primary Care Research, Executive Office Center, Suite 502, 2102 East Jefferson Street, Rockville, MD 20852-4908. Phone: (301) 5941357, ext. 129; fax: (301) 594-3721; e-mail: cclancy@po3 .ahcpr.gov. Applications should be submitted to: Janet Heinrich, DrPH, RN, FAAN, Director, American Academy of Nursing, 600 Maryland Ave., SW, Washington, DC 20024-2571. Phone: (202) 651-7238; fax: (202) 554-2641; e-mail: jheinrich@ ANA.org. Applications are due by January 15, 1997.
Conn and Armer 271