Health policy issues and applications for evidence-based medicine and clinical practice guidelines

Health Policy 46 (1998) 1 – 19

Review

Health policy issues and applications for evidence-based medicine and clinical practice guidelines Kathleen N. Lohr a,*, Kristen Eleazer b, Josephine Mauskopf b a

Health Ser6ices and Policy Research, Health and Social Policy Di6ision, Research Triangle Institute, 3040 Cornwallis Road, P.O. Box 12194, Research Triangle Park, NC 27709 -2194, USA b Center for Economics Research, Research Triangle Institute, 3040 Cornwallis Road, P.O. Box 12194, Research Triangle Park, NC 27709 -2194, USA Received 27 May 1998; accepted 27 August 1998

Abstract Evidence-based medicine and clinical practice guidelines have become increasingly salient to the international health care community in the 1990s. Key issues in health policy in this period can be categorised as costs and access to care, quality of and satisfaction with care, accountability for value in health care, and public health and education. This paper presents a brief overview of evidence-based medicine and clinical practice guidelines and describes how they are likely to influence health policy. Evidence-based medicine focuses on the use of the best available clinical (efficacy) evidence to inform decisions about patient care; guidelines are statements systematically developed from efficacy and effectiveness research and clinical consensus for practitioners and patients to use in making decisions about appropriate care under different clinical circumstances. Both fields have developed methods for evaluating and synthesising available evidence about the outcomes of alternative health care interventions. They have clear implications for health policy analysts: greater reliance should be placed on scientific evidence, policy decisions should be derived systematically, and health care decisionmaking must allow for the active participation of health care providers, policy makers, and patients or their advocates. The methods and information generated from * Corresponding author. Tel.: +1 919 5416512; fax: + 1 919 5415945; e-mail: [email protected] 0168-8510/98/$ - see front matter © 1998 Published by Elsevier Science Ireland Ltd. All rights reserved. PII S0168-8510(98)00044-X

2

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

evidence-based guidelines efforts are critical inputs into health policy analysis and decisionmaking. © 1998 Published by Elsevier Science Ireland Ltd. All rights reserved. Keywords: Clinical practice guidelines; Decisionmaking; Effectiveness; Efficacy; Evidencebased medicine; Health policy; Patient care

1. Introduction As the 1990s draw to a close, evidence-based medicine (EBM), evidence-based practice, and clinical practice guidelines (CPGs) have become increasingly pertinent to the health care community. Although aimed chiefly at clinicians (especially physicians), these practices and products are of growing interest to policymakers, payers and purchasers, and patients. Because of their relatively recent appearance in the clinical and health care delivery worlds, these fields are not well described outside of fairly small circles, and their potential impact is not well understood. This paper: “ Reviews EBM and CPG concepts. “ Explores their commonalties and differences with respect to: Conceptual issues Use of scientific evidence and expert consensus methods Products and audiences Implementation and evaluation. “ Discusses applications that are relevant to health policymakers, health plan and sickness fund decisionmakers, purchasers and insurers, health professionals, patients and their families, and society at large. Health care decisionmaking can take place at very high levels of abstraction and policymaking or at quite detailed levels of specific programs or interventions for individual patients (Fig. 1). At the broadest social level lie questions about the overall level of a health care budget vis-a`-vis other socially desirable uses of a nation’s resources. This may be characterised as the ‘macro’ level of decisionmaking [1]. Second come questions of how resources ought to be allocated within fixed health budgets; these choices at the ‘meso’ level of decisionmaking involve population public health or individual health services, allocations among services, research and education, and allocations to one type of health disorder or one segment of the population or another. At the third, ‘micro’, level, decisionmaking focuses on care for individual patients, and administrators, clinicians, and patients must direct attention to the management of different clinical conditions. With respect to the use of prescription medications for a given disorder, for instance, authoritative guidelines or protocols may involve evidence or expert opinion about the use of pharmaceutical agents versus other, non-drug-based interventions (e.g. ‘watchful waiting’, surgery), use of one drug versus another within a given class, or use of a specific pharmaceutical compound but only in selected patients or settings. A thread that runs through most recent health policy analyses is that greater reliance must be placed on scientific evidence and less on ideology or expert opinion. A second theme is that guides to public policy, like guides to clinical

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

3

practice (or any other endeavor in health care), should be: systematic and logical; defensible (reliable and valid); practical and feasible; and clear and understandable to both experts and (within reason) laypersons alike. Third, health care decisionmaking must accommodate and facilitate the active participation of both health professionals and patients or, depending on the level of decisionmaking, health policymakers and consumer advocates. The context, thus, is the need to find more precise decision tools to aid in achieving optimal health care or health improvement for a given level of resources. It is in this context that the concepts and tools of EBM and CPGs should be understood.

2. EBM and CPGs

2.1. EBM and its counterparts In simplest terms, the EBM movement can be traced to a small group of Anglo-American medical researchers and educators who believed that medicine needed to be oriented more to the existing science and knowledge base than they perceived it was in the last quarter of this century. According to one of the leaders of this group, EBM is ‘‘the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients’’ ([2], p. 71). This definition, which calls for ‘‘…integrating individual clinical expertise with the best

Fig. 1. Levels of decisionmaking to which evidence-based practice and guidelines contribute.

4

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

available external clinical evidence from systematic research’’, is probably the most widely used and recognised in this family of concepts [3–7]. EBM is also described as an ‘‘approach to clinical problem solving that applies the best, relevant evidence from research [by and large from randomised clinical trials (RCTs)] to answer clinical questions’’ ([8], p. 59) and as a movement that promotes the adoption of effective innovations, delays the spread of unproven interventions, and prevents the use of ineffective technologies [9]. In evidence-based clinical practice, the clinician ‘‘uses the best evidence available, in consultation with the patient, to decide upon the option which suits that patient best’’ ([10], p. 9); it applies the evidence derived from studies of populations to the care of individuals — a clear link to epidemiology [11]. The chief difference between these related concepts appears to be the explicit involvement of the patient (‘in consultation with the patient’) in the clinical decisionmaking task. Increasingly, one encounters phrases such as evidence-based nursing (dentistry, etc.), evidence-based disease management and even evidence-based policy, evidencebased priority-setting and evidence-based rationing. These constructs obviously overlap, and to some degree they simply reflect adoption of a phrase now in vogue (i.e. ‘evidence-based’). In evidence-based policymaking (or evidence-based decisionmaking), public health professionals, administrators and policymakers apply evidence-based approaches to a group of patients or to populations (which can include both users and nonusers of health care) [12]. Systematically developed information is employed to re-engineer health systems, direct health services and policy research into productive avenues, ensure access to good care for all, maintain and improve the quality of care, foster appropriate innovation and diffusion of technology, and optimise the value of health care resources in addressing important health problems. Evidence-based policymaking (or its analogues) has somewhat less conceptual clarity or practical meaning than EBM as it is applied to medical education or clinical practice. Most health policy decisions in post-industrial nations in the past two to three decades have had some basis in data, but not necessarily data drawn from published sources and certainly not from RCTs. RCTs are rarely used for most policy- or population-level activities (in the USA, the celebrated RAND Health Insurance Experiment was a notable exception [13]), but reasonably good studies using other designs are often available; this fact underscores the need to broaden the acceptability of ‘effectiveness’ work and health services research once one departs from specific clinical questions. Also, policymaking is grounded far more in values and politics than is clinical practice; compared to patient care, it is constrained in more (and different) ways by the availability of resources and prevailing social opinion. In Europe, considerable attention has been directed recently at health care reform, as demand for health care rises and resources become increasingly constrained. These studies delve deeply into questions of policymaking and rationing and point to the need for hard data on costs and effectiveness [14,15], although they are not characterised as ‘evidence-based’ in the meaning intended above. Nevertheless, scientific evidence and quantitative analyses can be expected to play a greater

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

5

role in policymaking than in earlier years, whether it carries the ‘evidence-based’ descriptor or not.

2.2. CPGs In 1990, the Institute of Medicine (IOM) of the National Academy of Sciences in the USA defined CPGs as ‘‘systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances’’ ([16]; [17], p. 27). Four critical concepts are systematic development, assistance to both clinicians and patients, appropriateness of services, and specificity with respect to defined clinical problems. Other definitions of this and similar concepts (such as practice parameters, pathways, or protocols) from roughly the same period tend to embody one or more of these concepts (e.g. [18,19]). The IOM expert committees on guidelines established explicit criteria for judging the soundness of practice guidelines. The most important attribute identified was validity, which was defined in terms of (1) health and cost outcomes and (2) the strength of the relationship between clinical recommendations and the scientific evidence used to arrive at those recommendations. Thus, conceptually, practice guidelines are meant to have a strong basis in evidence, and the term ‘systematically’ implies that CPGs must rest on a clearly defined search for and assessment of relevant information. CPGs form a basis for: (1) improving knowledge by making clinicians aware of the recommendations; (2) changing attitudes about standards of care; (3) shifting practice patterns; and (4) enhancing patient outcomes. They can be an efficient way for clinicians, particularly generalists, to stay current with new developments, assuming that the guidelines initially contain the most recent information and that they are reviewed and updated with some regularity. To date, however, guidelines alone have had regrettably little impact in the absence of concrete efforts to translate them into tools usable in everyday practice; thus, although extremely important, by themselves they are an incomplete catalyst for change. CPGs also are used to inform health care policymakers and commissioning agencies about appropriate or inappropriate health interventions. In countries with a national health care system, administrators and policymakers can use guidelines to assign resources to areas where they are needed the most [20]. In countries with a private-based health care system, plan administrators and insurers can use CPGs to make decisions about what services to authorise (i.e. reimburse) for patients with given conditions. In addition, practice guidelines have been thought to be useful for legal matters, public and patient education, and priority-setting in research.

2.3. A note on origins One might speculate on why these movements have arisen in North America and the UK, and why they are now emerging so rapidly in Europe (and elsewhere). Among the factors conducive to their beginnings may have been:

6

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

“

The very high level of support for biomedical research (especially in the USA) and for health services research (although the funding for the latter is only a small fraction of the support for clinical research). “ The comparatively large numbers of physicians trained in health services research or holding, for example, master’s degrees in public health or business administration or doctoral degrees in law or other disciplines. “ The wide access to electronic literature databases such as those maintained by the US National Library of Medicine and to computer and communications technologies in general. “ The 15 – 20 years of preoccupation in the USA with the excessive costs of health care. “ A quarter-century or more of interests there in quality of care. The relationships of key leaders in Canada (McMaster University) and England (Oxford University), and the fact that Archie Cochrane and his landmark treatise in the early 1970s [21] was so heralded in the UK, also may have played a role. Reasons for the diffusion to Europe may well include the rapidly rising access to computer and communications technologies, a long-standing interest in some nations in technology assessment (which provides a natural bridge to evidencebased practice and guidelines), the fact that many of these nations are experiencing the same kinds of pressures on their health care budgets and a growing interest and expertise in measuring and improving quality of care, and the active proselytising of EBM proponents and the Cochrane Collaboration on the continent.

3. Commonalities and differences in EBM and CPGs

3.1. Issues for EBM and CPGs Various questions and criticisms have been directed at these fields throughout the 1990s [10,17,22 – 31]. Some points are technical or practical; others are more philosophical. Those that cut across both the EBM and the CPG movements include the following: “ EBM is noted for its marked emphasis on RCTs as a means of avoiding systematic bias in the evidence base. This preference may skew information, however, to those patient populations included in RCTs (often men, middleaged, white, and generally without co-existing conditions), leaving considerable gaps for the remainder of the patient population whom practitioners may see in everyday practice. That is, much of the RCT literature excludes patient populations or clinical problems encountered in primary care, leading practicing clinicians to be concerned about the relevance of evidence reports or practice guidelines based chiefly or solely on this knowledge base. Clinicians are often (probably mainly) faced with patients who are generally excluded from RCTs, including patients with multiple health problems, older patients, children and women. These factors create a conflict between internal and external validity, a demand for practice-based or medicine-based evidence [28], and a call to bring

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

“

“

“

“

“

“

“

7

more ‘effectiveness’ data (as contrasted with ‘efficacy’ data) into the process of developing systematic reviews, evidence reports, and guidelines. Both EBM and CPG activities emphasise patient outcomes—the elements of health status and quality of life that patients and their families value and the clinical or physiologic measures that health care providers value [10,32]. The evidence about outcomes in the published literature may be quite adequate or quite deficient, depending on the extent to which investigators appropriately select and measure a wide range of outcomes, report reasonably large effects, produce results that can be applied with an acceptable degree of confidence to population(s) or service(s) about which key decisions are needed, and provide some indication that the intervention does more good than harm. Literature databases are often incomplete, a fact that prohibits practitioners (or even expert librarians or researchers) from obtaining thorough information even from the largest databases. Relying on the published literature has its own hazards. Among them are: (1) publication bias, which is the tendency for research results showing significant differences, especially those favoring intervention over placebo in RCTs, to be published and for results showing no such differences not to be published; and (2) the ‘framing effect’ in which decisionmakers and purchasers may be influenced simply by the way(s) the data are presented. Cost-effectiveness, meta-analytic, and other pharmacoeconomic studies also come in for criticism. For example, despite their apparent statistical rigor, meta-analytic approaches may not sufficiently overcome clinical gaps in the trial data, because trial designs and endpoints are not homogeneous. Overall, these techniques may lead decisionmakers to believe that findings and conclusions have a level of scientific precision that is not really warranted. Some issues seem directed more at the EBM movement: EBM advocates draw a sharp distinction between research evidence (on the one hand) and clinical acumen and practitioner experience (on the other). To the extent this is too stark a line, they risk alienating the physician community, promoting a notion of statistical certainty that is untenable in the clinical setting, and ultimately undoing the good that evidence-based practices and guidelines can bring. The evidence-based movement is said to be at once evangelical and yet rooted in ‘narrow scientism’ [33]. In the eyes of the clinical professions, EBM is not all that new because they believe they have always practiced in accordance with ‘evidence’; indeed, it is the implication that they have not practiced in this fashion that so riles practitioners. EBM may increasingly involve the participation of ‘managers’ rather than physicians (or any health care clinicians), leading to skepticism on the part of health professionals and patients about its true purpose and value. In practical terms, and despite the success of individual medical education programs founded on evidence-based principles, one might well ask whether individual physicians can ever achieve the skills, or find the time and resources, to carry out the types of activities described in the main EBM textbooks. Other issues arise about guidelines:

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

8 “

Guidelines (especially older ones) may not be clearly written. Developers must avoid two problems: language that is so vague that the guideline does not provide useful information and language that is so specific that it prevents the practitioner from applying the information to an individual patient who may differ from the norm. Guidelines, with full recommendations that incorporate clinical expert judgments, may not be timely. Amassing the science base is likely to take essentially the same amount of time as it would in EBM activities, but guidelines require the further steps of evaluating that evidence and convening experts to reach some consensus about recommendations. Creating a complete guideline may take a year or more and, in that time, new evidence may become available. Guidelines may conflict even when they are based, ostensibly, on the same scientific knowledge. Conflicting sets of guidelines may emerge because of different rules for including or excluding literature, different types of expert clinicians on development panels, or ‘local adaptation’ of otherwise broadly accepted (e.g. national) guidelines. Guidelines are proliferating rapidly. When different health plans or insurance schemes impose many different guidelines at the same time, clinicians are in a difficult bind in trying to practice accordingly;1 this will be especially true when guidelines vary in reliability or conflict with each other.

“

“

“

3.2. Use of scientific e6idence Scientific evidence is of overriding importance to both fields. All other things equal, proponents would prefer to see evidence coming from multiple, well-designed RCTs. Unbiased, well-randomised RCTs with sufficient power are generally accepted as the gold standard for demonstrating the efficacy of a health care technology. The fields tend to part company on this point, however, because efficacy trials tend to provide inadequate information about the effectiveness of the technology in question. This difference between effectiveness and efficacy is an important one [32,36,37]. Efficacy requires that a clinical procedure achieve benefits to individuals in defined populations (often narrowly defined) when it is applied under ideal or optimal circumstances; this is the familiar terrain of RCTs. Effectiveness, by contrast, requires that a clinical procedure do more good than harm for the typical patient in ordinary or average settings and circumstances. One dilemma when evaluating evidence as inputs into clinical decisionmaking and guidelines, therefore, is that the 1

A decade ago, guidelines were sufficiently suspect in the general medical community that some means was needed by which to identify robust guidelines and design sound methods for development. To meet this need, the IOM committees [16,17] described a set of attributes of good guidelines that cover both their substantive content and their process of development. Recently, a consortium of researchers in Canada and several European countries (Denmark, Finland, France, Italy, The Netherlands, Spain and the UK) have used these concepts to develop and test an instrument for evaluating guidelines on characteristics such as reliability, clarity, validity, documentation, and development methods [34,35].

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

9

body of evidence is curtailed, sometimes radically, by accepting only efficacy studies as scientific truth.

3.3. Use of expert consensus and clinical experience/intuition Some years ago, the Evidence-based Medicine Working Group asserted that ‘‘EBM de-emphasizes intuition, unsystematic clinical experience, and pathophysiological rationale as sufficient grounds for clinical decision making’’ ([4], p. 2420). Practitioners strongly reacted to this statement and the attitude it conveyed. Today, proponents acknowledge the role of clinical knowledge and experience as ‘‘crucial and necessary parts of becoming a competent physician’’, and they emphasise the need for clinical intuition with respect to diagnosis and for treatment of conditions for which insufficient evidence is available to apply stricter evidence-based techniques [3]. Guidelines are meant to provide formal conclusions and recommendations about appropriate (inappropriate) and necessary (unnecessary) care for specified types of patients in specified types of practice settings. To get there, some form of expert or consensus panels is typically required to assess systematic reviews and evidence reports and arrive at the relevant clinical conclusions [17,38]. Thus, although the quality of the guideline may be determined in large part by the quality of the original evidence, the actual make-up and workings of the panel are not insignificant and may, in situations where evidence is weak or conflicting, be extremely important. Understanding the membership, viewpoints, and vested interests of these panels (or the professional associations that sponsor them), the methods by which they reach their recommendations on the basis of efficacy or effectiveness data (or both), and the tradeoffs they may be willing to make to achieve a specific goal is, similarly, very important. Guidelines are often best developed by multidisciplinary panels of experts who represent all the major types of clinicians who might have responsibility for the care of patients with the condition in question. CPG panels may comprise various combinations of generalist (primary care) and specialist physicians, nurses, and other health professionals such as optometrists or therapists. By contrast, EBM activities tend to be far less multidisciplinary. In addition, CPG developers, more than EBM practitioners, tend to see a more direct role for consumers and patients (or their representatives from voluntary associations and advocacy groups) in the development and dissemination processes.

3.4. Methods 3.4.1. Identifying literature and amassing information By and large, evidence-based and guidelines-oriented methods for searching the literature are similar. The first step is defining the clinical or research question. The ‘question’ will differ between, say, a single practitioner who is applying EBM in the care of his or her individual patients and a methodologist seeking to obtain quantitative information for a meta-analysis as background for a CPG program.

10

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

More important than this difference, however, is the need for both actors to be very precise in stating the problem they are addressing. The usual sources of information in these fields are basically the same. General medical literature databases such as MEDLINE or EMBASE and more specific databases such as AIDSLINE and PsycLIT all provide citations and some abstracts for medical journals. Other sources of information include the ACP Journal Club (a disk with selected internal medicine articles dating from 1990 that have been evaluated for scientific soundness and clinical applicability to the disease) and the Cochrane Collaboration (e.g. the Cochrane Database of Systematic Reviews). The Internet is a potential source for quick information; the Journal of the American Medical Association, the New England Journal of Medicine and the British Medical Journal publish key articles from the last several years on their web sites. MEDLINE and EMBASE include only a fraction of journals published worldwide, and most of the journals included are in English. To avoid publication bias and selection bias, experts counsel that including other sources of information is important. These sources may involve the ‘grey literature,’ non-English literature, references listed in primary sources, raw data from published trials, and studies that are in progress. Grey literature includes theses, internal reports, non-peer reviewed journals, and pharmaceutical industry files [39]. Regardless of the eventual sources tapped for a systematic review, evidence report, or literature search for a practice guideline, the search strategy should be thoroughly documented [17,40]. In the demand for accuracy on this score, the fields are in accord.

3.4.2. Assessing the science in the literature and grading the e6idence Although superficially convincing, published evidence may be slanted because of defects in study design, various other hard-to-detect biases and cofounders, and carelessness, and these problems impede accurate, objective assessment of the literature [41]. All parties emphasise the need for critical appraisal of the literature using explicit ‘grading’ systems. Among the approaches to grading either individual articles or the overall evidence are those of the Canadian Task Force on the Periodic Health Examination, the US Preventive Services Task Force, the Cochrane Collaboration, and the AHCPR guidelines programs and its newer evidence-based practice centers (EPCs), but more than 60 checklists, scales, or similar schemes for grading literature have been identified [42]. The approaches differ in terms of the weight given to efficacy vis-a`-vis effectiveness data. Some grading schemes define the quality of individual articles chiefly (if not exclusively) by study design (RCTs outweighing all other designs). Others create quantitative scales by which articles are graded on various domains such as problem studied, sampling, measurement, internal validity, external validity, construct validity, statistical conclusions, and justification for conclusions. Articles dealing with prevention, versus diagnostic tests, versus therapies, however, cannot easily be classified or graded with the same schema. Other classification or rating approaches relate to the ‘collective’ body of evidence. As cases in point: one ‘A-B-C’ system ranges from good research evidence

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

11

and some expert opinion (A) to mostly expert opinion and clinical experience but minimal research evidence (C). Another three-part list specifies: (1) clinical studies of good quality and consistency addressing the specific recommendation and including at least one RCT; (2) well-conducted clinical studies but without RCTs on the topic in question; and (3) information that does not include directly applicable, high quality clinical studies. AHCPR’s six grades, from highest quality to lowest, involve evidence from: (1) meta-analysis of RCTs; (2) at least one RCT; (3) at least one well-designed controlled study without randomisation; (4) at least one other type of well-designed quasi-experimental study; (5) well-designed non-experimental descriptive studies, such as comparative studies, correlation studies and case studies; and (6) expert committee reports or opinions or clinical experiences of respected authorities. Increasingly, experts recognise that study design alone is an insufficient guarantor of high quality data, and grading schemes based solely on study design are likely to become less acceptable over time. Choosing an appropriate grading system for a specific clinical question and literature (e.g. examining therapeutic questions versus studying diagnostic technologies) warrants careful attention for at least two reasons. First, the research base in health fields other than medicine may be very different and less oriented toward RCTs than medicine has traditionally been. Second, as evidence-based guidelines activities are increasingly aimed at diagnostic tests, epidemiologic issues, or health policy, reliance on RCTs as the gold standard becomes less tenable.

3.5. Products and audiences EBM products are, by definition, targeted at physicians (evidence-based nursing products are targeted at nurses, and so on). To the extent that this work is not multidisciplinary, evidence reports will not be deliberately aimed at related health professions. Rather, health professionals are encouraged to use ‘evidence-based’ approaches within their particular disciplines. CPG programs have been created in the past decade for many different health care fields other than medicine, including nursing, chiropractic care and dentistry [43,44]. Guidelines that are focused on clinical conditions involving an array of health workers, as well as guidelines that have been developed using a multidisciplinary panel, will aim for a broader audience. Guidelines may also be aimed at patients.2 When organisations representing persons with rare conditions are the impetus for guideline development, they will be either the primary consumer or the principal force behind dissemination of the guideline. In general, however, systematic reviews may be much less accessible to patient groups (or certainly to individual patients) than to professionals, except perhaps to those laypersons who have a sophisticated understanding of the field and/or access to the Internet. 2

The original AHCPR guidelines appeared in several formats: the full guideline, a clinical summary, and a patient booklet written in layperson’s language. At least some of the patient booklets were also produced in Spanish.

12

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

Avenues of dissemination include the peer-reviewed clinical literature, where articles are indexed and find their way into the various literature databases, and other print media (e.g. textbooks, patient booklets, the popular press). Broadcast media (both radio and television) and videos can also play a role in dissemination. Reviews and guidelines can be distributed via various electronic and computerbased means, such as CD-ROMs and Internet (web site) postings. Generally speaking, all these routes are relevant to both EBM and CPG products, but the approaches involving lay media are likely to be much more pertinent for guidelines than for systematic reviews and evidence reports.

3.6. Implementation and e6aluation Implementation and evaluation have been major issues for the CPG movement for years. The concerns include getting information to intended audiences; ensuring that practitioners read, understand, and use the materials and change their practice patterns as needed; and assessing whether the guideline has made a difference in the costs or quality of care. Recently, the Second International Conference on the Scientific Basis of Health Services [45] focused heavily on efforts in the USA, Canada, and numerous European nations to evaluate the impact of guidelines, and the first (1997) and second (1998) AHCPR conferences on Translating Evidence into Practice also tap these evaluation topics. The Cochrane Collaboration has created a new Group — the Cochrane Effective Practice and Organisation of Care Group (previously the Cochrane Collaboration on Effective Professional Practice — to help bridge the gap between evidence about effective and efficient health care delivery and its application in everyday settings.

4. Applications of evidence-based guidelines Regardless of the nation, four sets of health policy issues, which are discussed below, shape the concerns of policymakers, purchasers, payers and patients today: “ Costs of (and access to) care and decisions about coverage or reimbursement of services (e.g. pharmaceutical products and formularies). “ Quality of care, satisfaction with care, and accountability for value in health care. “ Public and professional education about health care, health policy, and science. “ Priority-setting in research. Generally speaking, these topics pertain to the bottom two levels in Fig. 1: allocation of resources to patient care and management, and allocation of patient care resources to specific interventions. Still, some aspects of allocation decisions within the health sector (e.g. between health care and research, or between health care and education) are also touched on. What we will denote hereafter the ‘evidence-based guidelines’ movement offers many tools and solutions to these problems. Other experts also argue that systematic reviews (as the phrase is used in the evidence-based practice community) can be useful adjuncts for decisionmaking

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

13

Fig. 2. The flow of activity in evidence-based guidelines.

by both policymakers and consumers, even if little has yet been done to evaluate their direct impact on such decisionmaking [46]. Fig. 2 depicts the flow of activities that underlie evidence-based guidelines. The starting point is biomedical and health services research and, then, publication of scientific findings. Evidence-based approaches involve development of authoritative evidence reports and dissemination of those reports to interested audiences and users groups. Other organisations may then develop and distribute full practice guidelines, and these efforts may include programs for implementing the guidelines and evaluating their impact. In all cases, identifying areas warranting further research is an important step that links back to clinical and health services research.

14

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

4.1. Costs of care Most troublesome for policymakers, payers, and patients alike, in all probability, are costs of care: that is, use of services by populations treated in the public sector or by members of health plans or sickness funds and the levels of expenditures on those services. Administrators of private health plans or sickness funds and of public health insurance programs, employers, insurers, and patients all have grappled in the past two decades with escalating health care costs and outlays, which are driven by population demographics (certainly age), medical technology, general price inflation, and other factors. Those accountable for managing such costs must decide what types of services a health benefit plan will cover and what level of reimbursement it will approve for those covered services. They must consider the efficiency of health systems, facilities, and service delivery organisations, because the nature of services that are (or are not) covered can have a substantial impact on how efficient such health care plans and delivery organisations are. Managers may focus on the bottom line of past or predicted outlays on health services. Moreover, they may lack the time, resources, or inclination either to confront directly the issue of the relative effectiveness (or cost-effectiveness) of services or to take account of information about new technologies (or new information about old technologies). This bottom-line orientation is insufficient for appropriate decisionmaking over the long run. More well-rounded decisionmaking can benefit from systematic reviews, evidence reports, and guidelines, as these yield readily accessible information on therapy versus therapy comparisons, on both efficacy and effectiveness of health interventions, and on the cost-effectiveness of different interventions. When such information comes from well-conducted meta-analyses, it can reflect a broader perspective than may be available solely from systematic literature reviews, and it may even permit some conclusions about patient populations or aspects of an illness not fully investigated in any one study. Application of decision analysis may foster greater consideration of patient preferences and values, which can be a significant factor in long-range health policy planning.

4.2. Quality of care Quality of care is ‘‘the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge’’ ([47], p. 21). It is fast emerging as the dominant contemporary issue as various public or private sector initiatives in many countries — but chiefly privatisation, competition and managed care strategies—begin to bring health care costs (or at least the rate of inflation in those costs) under control. Quality has three major components: ‘structural’ aspects of care such as credentials of physicians or hospitals; ‘processes’ of care that indicate what was done to or on behalf of patients; and ‘outcomes’ of care that reflect the short- or long-term results of services. It may also involve patient or consumer satisfaction with care.

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

15

Coupling quality of care directly with concerns about costs leads policymakers and purchasers to talk about ‘value’ for health care outlays—avoiding overuse of inappropriate or unnecessary care, ensuring timely provision of appropriate and needed care (i.e. avoiding underuse of services), and minimising misuse of services in the form of poor technical or interpersonal performance of health care professionals and providers [48]. Ultimately, value is measured in the outcomes that all patients experience; in practical terms, then, the real-life quality-of-care issues are ones of effectiveness, not efficacy. Health care professionals, administrators, and purchasers are concerned with the performance of individual practitioners, groups of clinicians, health plans and networks, and the health care sector in general. They may invoke concepts such as performance indicators, accountability, and audit or quality criteria to indicate that practitioners, facilities, and the like must answer for their performance and their stewardship of health care resources on behalf of both individual patients and society as a whole. In the quality-of-care arena, evidence-based guidelines serve several purposes. First, by definition, they provide information and recommendations for use by all types of health care professionals about appropriate and needed services for all aspects of patient management-screening and prevention, diagnosis, treatment, rehabilitation, and palliation. Such guidelines may be applied across many health care settings — inpatient or residential (e.g. hospitals, nursing homes), outpatient (e.g. offices, ambulatory clinics, and private homes), and emergency departments or clinics. Second, guidelines may also help physicians and other clinicians decide not to use services but rather to stay with ‘watchful waiting’. Third, public health officials may use some guidelines, such as those concerned with health promotion and disease prevention, to develop and implement community-based health programs. Fourth, evidence-based guidelines constitute a major element of quality assurance, quality improvement, medical audit, and similar activities. They can be converted into medical review criteria by which care can be assessed; they can also be the motivation for targeting particular kinds of services, providers, settings, or patient populations for in-depth review.

4.3. Education Evidence-based approaches are emerging as a basic component of the undergraduate and graduate training of many health professions. Although the practicalities of the methods can be questioned, little argument can be raised about the significance of insisting on a science base, instead of only received wisdom, in the clinical training of health professionals. Evidence-based guidelines also serve many educational purposes for other audiences. Policymakers, purchasers and insurers, providers of all types, patients, and the public at large benefit from easy access to evidence reports, technology assessments, guidelines, and similar documents produced in accordance with their particular needs and capacity to use and assimilate them. For example: authoritative guidelines on asthma management are of interest to primary care physicians,

16

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

nurses and patients and their families; guidelines about the management of highrisk pregnancy or labor and delivery after a previous caesarean section have value for both community and hospital-based physicians and nurse-midwives; and guidelines on management of depression or alcohol dependence are important to individuals in the health and social services sectors in addition to consumers and public health advocates. Publications from and about this field—whether in print or electronic form— serve wider educational purposes as well. At least indirectly, they inform a broad cross-section of a nation’s population about the benefits, advances, and limitations of science. In this way, they contribute to a better understanding within the body politic about the difficulties of allocating scarce resources to competing purposes, when claims are made that all these uses will improve the health and well-being of the population. Moreover, attention to these issues is increasing in Europe, not least because of the new interest in consumer access to health care information as part of the European Community’s framework for action in public health following the Amsterdam intergovernmental conference in 1997. Thus, those who speak of evidence-based policymaking or evidence-based rationing may well have these broad educational and consumer information goals partly in mind, even though they are likely thinking more of the day-to-day decisionmaking they face.

4.4. Research The hallmark of evidence-based guidelines is scientific evidence drawn from rigorous research of all types. Indirectly, research plays a fundamental role in setting health care priorities, in part by informing the development of robust evidence-based guidelines. The investments that are made today in clinical and health services research will make future choices about who should receive what kinds of care easier. Thus, although the core of evidence-based guidelines may be information generated from basic biomedical and clinical research, the information necessary to arrive at comprehensive, policy-relevant practice guidelines must also be derived from health services research. The fundamental basic science imperative may be to generate information about the efficacy of health care interventions, but the practical realities of policy and economic decisions call for knowledge about effectiveness. The interface between evidence-based guidelines and priorities for clinical investigation and health services research is of particular interest [49]. Often, priorities for guidelines or technology assessments are set in terms of models based on health burden, costs, and availability of evidence. Important topics for guidelines or assessments that are set aside for lack of evidence are obvious priorities for research. Furthermore, good evidence-based guidelines will explicitly indicate where information is lacking and point to the specific research questions that warrant support. Finally, those engaged in health policy, research, and delivery must continually and persuasively make the case for research, including health services

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

17

research. A weak research enterprise will be hard-pressed to provide the knowledge based required for difficult decisionmaking about health care priorities.

5. Concluding thoughts EBM and CPGs are related movements with increasingly greater salience to health care decisionmaking in many countries. EBM focuses on the use of the best available clinical evidence from systematic research to inform decisions about patient care; guidelines are systematically developed statements for use by practitioners and patients about appropriate care under different clinical circumstances. The fields have significant strengths and limitations in conceptual, methodologic and practical terms. On balance, they offer much to policymakers, purchasers, clinicians, patients and society at large for addressing key issues involving costs and access to care, quality of and satisfaction with care, accountability for value in health care and public health and education. The implications for health policy analysts are that they should place greater reliance on scientific evidence, attempt to arrive at policy decisions in systematic ways and allow for the active participation of health care providers, policymakers and patients or their advocates. The methods and information generated from evidence-based guidelines efforts, especially when that information reflects both the efficacy and effectiveness of health services, will increasingly be critical inputs into health policy analysis and decisionmaking.

Acknowledgements The project from which this paper originates was supported by Merck and Co., Inc. We are especially grateful to Jeffrey L. Sturchio and Chantal Dorange for their substantial encouragement and assistance throughout this project, particularly in providing European documents and an EMBASE literature search. We thank Steve Teutsch for his insightful comments on an earlier draft, and Merck and Co., Inc. staff members Judith Shevell and Sylvia Gassy for their help with the literature search. We also acknowledge the able secretarial assistance of Debra J. Bost, Linda Fonville and Teresa Gurley at Research Triangle Institute.

References [1] Borst-Eilers E. Health services research in a rapidly changing world (abstract and keynote presentation). Abstracts, 2nd International Conference, Scientific Basis of Health Services. Amsterdam, 5–8 October 1997, p. 5. [2] Sackett DL, Rosenberg WM, Muir-Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. British Medical Journal 1996;312(7023):71– 2. [3] Sackett DL, Richardson WS, Rosenberg WM, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. New York: Churchill Livingstone, 1992.

18

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

[4] Evidence-Based Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. Journal of the American Medical Association 1992;268(17):2420– 2425. [5] Winn RJ. The role of oncology clinical practice guidelines in the managed care era. Oncology 1995;9(Suppl. 11):177–83. [6] Grimshaw JM. Towards effective professional practice. Therapie 1996;51(3):223– 36. [7] Marwick C. Proponents gather to discuss practicing evidence-based medicine. Journal of the American Medical Association 1997;278(7):531– 2. [8] Sweeney K. Evidence and uncertainty. In: Marshall M, editor. Sense and Sensibility in Health Care. London: BMJ Publishing Group, 1996:59 – 86. [9] Haines A, Jones R. Implementing findings of research. British Medical Journal 1994;308(6942):1488–92. [10] Muir Gray JA. Evidence-based Healthcare. How to Make Health Policy and Management Decisions. London: Churchill Livingstone, 1997. [11] Fletcher RH, Fletcher SW, Wagner EH. Clinical Epidemiology: The Essentials. 2nd ed. Baltimore, MD: Williams and Wilkins, 1998. [12] Sackett DL. Applying overviews and meta-analyses at the bedside. Journal of Clinical Epidemiology 1995;48(1):61–70. [13] Newhouse JP and the Insurance Experiment Group. Free for All? Lessons from the RAND Health Insurance Experiment. Cambridge, MA: Harvard University Press, 1993. [14] Hoffmeyer UK, McCarthy TR, editors. Financing Health Care. A National Economic Research Associates study. Dordrecht: Kluwer, 1994. [15] Saltman RB, Figueras J. European Health Care Reform. Analysis of Current Strategies. WHO Regional Publications, European Series, No. 72. Copenhagen: World Health Organization, Regional Office for Europe, 1997. [16] Institute of Medicine. Clinical Practice Guidelines: Directions for a New Program. In: Field MJ, Lohr KN, editors. Washington, DC: National Academy Press. 1990. [17] Institute of Medicine. Guidelines for Clinical Practice: From Development to Use. In: Field MJ, Lohr KN, editors. Washington, DC: National Academy Press, 1992. [18] Woolf SH. Practice guidelines: a new reality in medicine. III. Impact on patient care. Archives of Internal Medicine 1993;153(23):2646– 55. [19] Woolf SH. Practice guidelines: what the family physician should know. American Family Physician 1995;51(6):1455–63. [20] Grimshaw JM, Hutchinson A. Clinical practice guidelines — do they enhance value for money in health care? British Medical Bulletin 1995;51(4):927– 40. [21] Cochrane A. Effectiveness and Efficiency. London: Nuffield Provincial Hospitals Trust, 1972. [22] Editorial. Evidence-based medicine, in its place. The Lancet 1992;346(8978):785. [23] Clarke MJ, Stewart LA. Systematic reviews of randomized controlled trials: the need for complete data. Journal of Evaluation in Clinical Practice 1995;1(1):119 – 26. [24] Benech I, Wilson AE, Dowell AC. Evidence-based practice in primary care: past, present and future. Journal of Evaluation in Clinical Practice 1996;2(4):249 – 63. [25] Hunter DJ. Rationing and evidence-based medicine. Journal of Evaluation in Clinical Practice 1996;2(1):5–8. [26] Polychronis A, Miles A, Bentley P. Evidence-based medicine: reference? dogma? neologism? new orthodoxy? Journal of Evaluation in Clinical Practice 1996;2(1):1 – 3. [27] Polychronis A, Miles A, Bentley P. The protagonists of ‘evidence-based medicine’: arrogant, seductive and controversial. Journal of Evaluation in Clinical Practice 1996;2(1):9 – 12. [28] Knottnerus JA, Dinant GJ. Medicine based evidence, a prerequisite for evidence based medicine. British Medical Journal 1997;315(7116):1109– 10. [29] Stradling J. Sleep apnoea and the misuse of evidence-based medicine. The Lancet 1997;349(9054):803–4. [30] Lohr KN. Guidelines for clinical practice: what they are and why they count. Journal of Law, Medicine and Ethics 1995;23(1):49–56. [31] Lohr KN. Guidelines for clinical practice: applications for primary care. International Journal for Quality in Health Care 1994;6(1):17 – 25.

K.N. Lohr et al. / Health Policy 46 (1998) 1–19

19

[32] Lohr KN. Outcome measurement: concepts and questions. Inquiry 1988;25:37 – 50. [33] Miles A, Bentley P, Polychronis A, Grey J. Evidence-based medicine: why all the fuss? This is why. Journal of Evaluation in Clinical Practice 1997;3(2):83 – 6. [34] Cluzeau F, Littlejohns P, Grimshaw J, Feder G, Moran S. Critically appraising guidelines. I: Testing the reliability and validity of an instrument. II. The quantity and quality of UK clinical guidelines for the management of asthma, coronary artery disease, breast cancer, lung cancer, and depression (abstracts and presentations). Abstracts, 2nd International Conference, Scientific Basis of Health Services, Amsterdam, 5–8 October 1997, pp. 76, 173. [35] Littlejohns P, Cluzeau F. Promoting the rigorous development of clinical guidelines in Europe through the creating of a common appraisal instrument (abstract and presentation). Abstracts, 2nd International Conference, Scientific Basis of Health Services, Amsterdam, 5 – 8 October 1997, p. 55. [36] Brook RH, Lohr KN. Efficacy, effectiveness, variations, and quality: boundary-crossing research. Medical Care 1985;23(5):710–22. [37] Detsky AS. Regional variation in medical care. New England Journal of Medicine 1995;333(9):589– 90. [38] Grimshaw J, Eccles M, Russell I. Developing clinically valid practice guidelines. Journal of Evaluation in Clinical Practice 1995;1(1):37 – 48. [39] Greenhalgh T. Towards a competency grid for evidence-based practice. Journal of Evaluation in Clinical Practice 1997;3(2):161–5. [40] Woolf SH. Practice guidelines: a new reality in medicine. I. Recent developments. Archives of Internal Medicine 1992;150(9):1811– 8. [41] Morgan WKC. On evidence, embellishment and efficacy. Journal of Evaluation in Clinical Practice 1997;3(2):117–22. [42] Valuck RJ, Vander Zanden J, Parker S, Weiss M, Malone D, Hope C. Development of Drug Specific Drug–Drug Interaction D.U.R. Criteria. Draft report developed for the United States Pharmacopeial Convention, Inc. Denver, Colorado: University of Colorado Health Sciences Center, School of Pharmacy, September 1997. [43] Meeker WC. The future impact of clinical practice guidelines. Journal of Manipulative and Physiological Therapeutics 1995;18(9):606– 10. [44] Goshorn J, Byers JF. CORE characteristics for survival in patient care redesign. AACN Clinical Issues 1997;8(2):236–45. [45] Abstracts 2nd International Conference, Scientific Basis of Health Services, 5 – 8 October 1997; 5th Annual Cochrane Colloquium, 8–12 October 1997, Amsterdam. [46] Bero LA, Jadad AR. How consumers and policymakers can use systematic reviews for decision making. Annals of Internal Medicine 1997;127(1):37– 42. [47] Institute of Medicine. In: Lohr KN, editor. Medicare: A Strategy for Quality Assurance, vol. 1. Washington, DC: National Academy Press, 1990. [48] Chassin MR. Assessing strategies for quality improvement. Health Affairs 1997;16(3):151– 61. [49] Lohr KN. The role of research in setting priorities for health care. Journal of Evaluation in Clinical Practice 1996;2(1):79–82.