- Email: [email protected]
The Importance of Outcomes Research in Pediatric Emergency Medicine
The Importance of Outcomes Research in Pediatric Emergency Medicine Carolyn M. Clancy, MD; Denise Dougherty, PhD; Elinor Walker Applying the methods and tools of outcomes research, ‘‘evaluation of the impact of health care on the health outcomes or ‘end results’ of patients and populations,’’ to the clinical domain of emergency services for children offers an important strategic opportunity for addressing the questions that confront all health care services: What works? For which patients? At what cost? From whose perspective? Although the important questions to address are extensive, much of the intersection between emergency services and outcomes research remains unexplored. Important challenges for researchers intrigued by the opportunities at this intersection of fields include: 1) clear definition of the scope of emergency services; 2) consideration of the appropriate end-point of emergency services—the entire episode of illness and/or services provided within the emergency setting; 3) selection and development of measures that incorporate children’s and families’ perspectives; and 4) a clear focus on linking research findings with strategies for improving outcomes and informed decision making. This essay will provide an overview of accomplishments and challenges from the field of outcomes research, suggest important opportunities for applying existing methods to emergency medical services for children, and identify potential career paths for current and future investigators. KEY WORDS:
outcomes; pediatric emergency medicine; research methods
Ambulatory Pediatrics 2002;2:293 300
PERSPECTIVE: WHAT IS EMERGENCY MEDICAL CARE FOR CHILDREN?
school, and even the police, as de facto first responders and/or agents of prevention.4 In a 1993 report, Emergency Medical Services for Children, the Institute of Medicine5 identified 7 basic responsibilities for emergency medical systems for children (EMSC):
One can view that segment of the healthcare system that centers on the emergency department in several different ways. On the one hand, it is the well-known safety net provider of ambulatory care to those without a medical home.1,2 On the other hand, in combination with the out-of-hospital emergency care system, it receives, stabilizes, and treats or refers people with specific conditions, such as trauma, or those with severe medical conditions, such as children with compromised respiration. It is necessary to see its emergency care role in a systems context, as pointed out by Fiser.3 From that perspective, emergency care is part of a cycle potentially involving all aspects of the care of a child, including not only resolution of a given episode of critical care but also the integration of the results of that episode into ongoing primary care, which then includes preventive care to prevent or mitigate future emergencies. Studying outcomes from this perspective also becomes a sequential and iterative process that must take into account the expected sequences of interventions and of expected results. Complicating the picture further is the role of the parent or caregiver, the
1) Identifying emergencies and the need for emergency care; 2) Ensuring access to the services of the system (eg, through 911 telephone service), with dispatch of personnel and equipment; 3) Providing appropriate prehospital care; 4) Transporting children to appropriate points of care; 5) Providing definitive treatment, including access to tertiary and rehabilitative care as needed; 6) Ensuring communication among emergency care providers and with other interested parties, including parents and primary care providers; and 7) Using information systems and feedback to assess and improve patient care and system performance and to point to areas for prevention efforts. This view of emergency care for children is often not reflected in the literature. For example, many articles and reports on outcomes research needs in ‘EMS’ address one of the above components, such as prehospital care, without acknowledging the entire spectrum of emergency services or addressing how a single aspect of emergency medical services fits within the health care system.6
From the Agency for Healthcare Research and Quality, US Department of Health and Human Services, Rockville, Md. The opinions expressed are the authors’ and are not intended to represent official policy of Agency for Healthcare Research and Quality or the Department of Health and Human Services. Address correspondence to Denise Dougherty, PhD, AHRQ, 2101 East Jefferson St, Rockville, MD 20852 (e-mail: ddougher@ahrq. gov). Received for publication October 1, 2001; accepted March 8, 2002. AMBULATORY PEDIATRICS Copyright q 2002 by Ambulatory Pediatric Association
BASIC STATISTICS ON EMSC Children are frequent users of emergency services. Weiss and colleagues estimated that there is, on average,
293
Volume 2, Number 4 Supplement July–August 2002
294
Clancy et al
one child or adolescent visit to an emergency department (ED) once every second.7 In 1996, 13% of children aged 0 to 17 years had an ED visit, according to the Agency for Healthcare Research and Quality (AHRQ) Medical Expenditure Panel Survey (MEPS).8 The visit rate was highest for children aged 0 to 4 years (15.9%) and aged 15 to 17 years (13.2%) and for children who were reported to be in fair or poor health status overall. A slightly greater percentage of white non-Hispanic children had an ED visit (14.4%) compared to black or Hispanic children. Emergency services are not cheap. Children with ED visits had an average of $500 in emergency room expenditures in 1996; each visit resulted in an average expenditure for ED care of $361.8 White children had higher average expenditures for ED visits. The ED is a frequent source of entry to inpatient care. In 1997, over 40% of children ages 1 through 14 years (equal to 707 000 children) who were discharged from inpatient hospitals had been admitted through the emergency room, according to ARHQ’s Healthcare Cost and Utilization Project (HCUP) data.8 In comparison, 33.6% of all hospital inpatient stays among adults ages 18 to 44 years originate in the ED. There is considerable variation in admissions to the hospital through the ED, ranging from just over 9% of all children’s hospital stays in Utah and Oregon to a high of 23.3% in New York.8 Why do children seek emergency care? Data from hospital EDs indicate that 15% of children under the age of 15 years seek care for conditions that are emergent, 32% seek care for urgent conditions, 14% for semiurgent conditions, and 11.4% for nonurgent conditions.*9 The percentage of visits that are judged emergent for Americans of all ages combined is slightly higher than that for children at 19%. Injury is the leading cause of ED visits among children aged 0 to 17 years (43%), followed by the category of ‘‘other’’ respiratory infections (13%), then otitis media (8%), then asthma (5%), and then intestinal infections (3%), according to AHRQ’s MEPS data.† The MEPS sample is too limited to analyze diagnoses by children’s age, but data from AHRQ’s HCUP data set of inpatient data (hospital discharge abstracts from 26 states) provide insights regarding how the type of illness changes with age. For children under the age of 1 year, the leading causes of inpatient admissions via the ED are related to respiratory conditions other than asthma, other perinatal conditions, and viral infections. In the next age grouping, 1–4 years old, asthma takes precedence as the most frequent cause of ED-related inpatient admission, followed by
*Definitions used in the survey are as follows: Emergency 5 a visit in which the patient should be seen in less than 15 minutes. Urgent 5 a visit in which the patient should be seen within 15–60 minutes. Semiurgent 5 a visit in which the patient should be seen within 60–120 minutes. Nonurgent 5 a visit in which the patient should be seen within 24 hours. †Unpublished data from the Medical Expenditure Panel Survey, analyzed January 2001.
AMBULATORY PEDIATRICS
pneumonia. Fluid and electrolyte imbalances are also important, as is epilepsy. Asthma is an even more substantial cause of ED-related admissions in the 5- to 9-year-old group, with pneumonia declining slightly as a cause. In this age group, however, appendicitis and injuries (upper and lower limb fractures) also increase in prevalence. For 10–14-year-olds, asthma and appendicitis become equally important as a cause of ED-related admissions, followed in importance by upper and lower limb fractures. In this age group, mental health problems make their first appearance as a leading cause of ED-related admissions, with affective disorders, other mental health problems, and poisoning by medications (likely an indicator of suicide attempt) accounting, in total, for 9% of ED-related inpatient admissions. In the 15–17-yearold group, reasons for ED-related admissions are extremely variable, with appendicitis dominating as a single diagnosis but with injuries and mental health problems also quite frequent. Our analysis of causes by major diagnostic categories gives another picture of the arguably most serious reasons for children’s entry into the ED and then into inpatient care. Available data on prehospital use of emergency medical services (EMS) for children are fairly consistent with the data on the serious conditions that result in inpatient admissions, except that in one analysis, seizures were estimated to account for 14% of prehospital EMS use.10 These prehospital estimates are not comparable to the hospital estimates. For example, the hospital estimates are based on nationally representative data, and the prehospital data are based on data available from possibly unrepresentative states and counties. The hospital data are for children ages 0 to 17 years and the prehospital estimates were made for children younger than 15 years of age. Further, the analyses used different definitions to categorize conditions. These data sources, although imperfect, strongly indicate that children seek a full range of emergency medical services for conditions that are not trivial and that it is critical that we strengthen our efforts to learn what works when children do encounter the need for emergency medical services. WHAT IS OUTCOMES RESEARCH? WHY OUTCOMES RESEARCH? Interest in assessing the outcomes or end results of health services reflects a convergence of several important forces in contemporary medicine. First is the growing burden of illness attributable to chronic illnesses, for which the therapeutic goal is often palliation, symptom relief, or improvement or maintenance of function. The correlation between these end-points and traditional metrics for assessing the success of clinical care (eg, mortality; physiologic measures or surrogates for physiology) is imperfect at best, so researchers have developed new approaches to assessing additional dimensions of outcomes. A second theme is increased recognition of the patient’s central role in decision making. For many clinical decisions, no one intervention is clearly associated with enhanced quality of
AMBULATORY PEDIATRICS
life or survival, or there are critical tradeoffs among different types of outcomes, such as length of survival, pain relief, or preservation of function, as exemplified by Barbara McNeil’s11 classic study about choices of treatment for laryngeal cancer. In other cases, interventions with substantial side effects offer modest but real survival benefits. A third issue is the limited generalizability or external validity of findings from clinical trials. Efficiency in research design and the need for internal validity has often resulted in restrictive inclusion criteria that limit broad applicability in clinical practice. In the past 15 years, mounting evidence of extensive variations in practice has captured the attention of policy makers and purchasers struggling to develop effective cost-containment programs without sacrificing quality of care. The most compelling explanatory hypothesis for practice variations is the consistent inverse relationship observed between practice variations and professional consensus. This relationship indicates that the existence of practice variation reflects both differences in practice style as well as an indication about where medical practice is limited by gaps in available evidence, though differences in patient populations, access to care, and local health care capacity also contribute. In the context of the US health care system, where supply-side regulation of health facilities has not been politically sustainable, a focus on developing better information on the effectiveness of interventions is considered to be a crucial strategy for increasing value in health care. At the same time, continued increases in health spending have led to new demands for accountability, referred to as the ‘‘third revolution’’ in health care. The widespread availability of inexpensive computing power, making examination of large databases feasible, has led to the use of existing data (usually from administrative claims) to conduct observational studies that include heterogeneous groups of patients, with an emphasis on quasi-experimental methods and nonrandomized designs. These methods have helped clarify the effects of clinical interventions in typical practice settings. The sometimes controversial assumption that statistical methods can be used to adjust for unmeasured selection bias does not negate the power of demonstrating the spectrum of care provided across multiple practice settings. Finally, until recently, the rate of increase in health costs had begun to decrease, resulting in concerns about possible threats to the quality of care. To measure the quality of care, purchasers, patients, and advocates recognize the need to evaluate the outcomes of care. In short, the focus of outcomes research, defined as inquiry that evaluates the impact of health care (including discrete interventions such as particular drugs, medical devices, and procedures as well as broader programmatic or system interventions) on the health outcomes of patients and populations, is pragmatic and closely linked with the information needs of health care decision makers. In addition, the establishment of a new federal agency in 1989, with a clear mandate to evaluate the outcomes of care, was linked with an explicit expectation that this re-
Outcomes Research in Pediatric Emergency Medicine
295
search would result in tangible improvements in clinical practice. INSIGHTS FROM THE FIRST DECADE OF OUTCOMES RESEARCH AT AHRQ Accomplishments In 1998 the Center for Outcomes and Effectiveness Research sponsored an evaluation of the first decade of investments in outcomes research supported by AHRQ (then the Agency for Health Care Policy and Research) for the purpose of identifying accomplishments and opportunities for future improvements.12 The full report, based on a survey of all principal investigators and indepth case studies of selected projects, can be obtained from AHRQ’s website (http://www.ahrq.gov). Responses to a survey of principal investigators demonstrated that the most salient findings identified by those researchers could be classified as descriptive epidemiology of medical practice. Although describing existing practice patterns is a necessary prerequisite to improving those practices, these responses also indicate a lack of concordance between policy makers’ and stakeholders’ expectations of outcomes research and those of the research community. In order to evaluate the impact of research investments on subsequent changes in practice and policy, a framework for assessing the impact of funded research was developed, one that included 4 levels of possible impact. Level 1 impacts are research findings published in peer-reviewed journals and clarification of testable hypotheses that merit further study. Level 2 impacts occur when specific research findings are used by one or more entities (‘change agents’), such as professional organizations, payors, patient organizations, accrediting organizations, the media, and others, to influence health policies. For example, evidence that beta-blockers reduce mortality in patients who have had an acute myocardial infarction was used by multiple organizations to increase the use of this effective intervention. Measurable changes in clinical practice are considered Level 3 impacts, and evidence that research has resulted in improvements in health outcomes is the highest level of impact, or a Level 4 impact. Not surprisingly, the report found that the majority of findings were consistent with Level 1 impact, such as peer-reviewed publications. Challenging Clinical Dogma In multiple areas, assessment of current clinical practice has resulted in important challenges to prevailing clinical ideas about how to manage specific clinical problems. For example, a study of otitis media in a pediatric Medicaid population showed that children with otitis media were more likely to receive a second course of antibiotics if they were initially given an expensive name-brand product compared with a generic product.13 This counter-intuitive finding helps to raise questions about one of the specific explanations given for using more recently developed drug therapies (ie, that failure rates are higher for generic antibiotics). This study result does not conclusively prove that cure rates are unequal for the 2 groups
296
Clancy et al
of drugs, since the same pattern of prescriptions would be observed if, for example, the clinicians who use expensive drugs are also more likely to prescribe a second course of therapy. The primary value of this study is confirmation in a large population (over 12 000 cases) that there is no evidence of worse outcomes with generic, narrow-spectrum antibiotics. Results from descriptive assessments of current clinical practice can clarify hypotheses and verify sufficient uncertainty to augment support for subsequent clinical trials. Improved Measures of Outcome Many health care interventions have important effects on patient function and well being but little effect in terms of death or disability. What matters for patients with cataracts is the impact of visual function on daily activities. Men with enlarged prostates may be more bothered by the potential risk of incontinence or impotence from surgery than by the need to get up at night to urinate. AHRQ’s outcomes research projects led in some instances to the development of new measures of outcomes, consistent with the development of shorter, valid tools that has progressed over the past 30 years, but more often these projects assessed the relationship between specific processes of care and existing outcome measures, both disease- or condition-specific and generic. A recent symposium cosponsored by AHRQ, the National Cancer Institute (NCI), and others on challenges in outcomes measurement is a reflection of the continued dynamism in this field. The VF-14, developed to help patients and clinicians understand who is most likely to benefit from cataract surgery, has been used in Manitoba to rationalize queues for elective surgery. A modification of this tool is now required by the National Eye Institute in all clinical trials. A related component of the focus on outcomes that matters to patients has been the recognition that for clinical decisions that involve 2 or more equally effective alternatives, the ultimate decision maker is the patient. Outcomes researchers have developed practical approaches for assessing the impact of alternative treatments, including no treatment, as well as methods for soliciting patient judgments regarding the value of those outcomes, so that individuals can select the option most consistent with their preferences. Shared decision-making programs have been developed for prostatic hypertrophy, breast cancer, the decision to screen for prostate cancer, back pain, and other conditions. Patients who participate in these programs have been shown to be more satisfied with their decisions and to report improved physical function compared with nonparticipants, even when their decisions as a group did not differ significantly. Finally, a critical accomplishment of the first decade of outcomes research was the development of methods and tools to assess outcomes of care when episodes of care involved multiple settings. Linking data for episodes of care that include hospital care, outpatient care, ED services, and care at other sites is difficult but important to identify what works and to identify specific opportunities to improve care and outcomes.
AMBULATORY PEDIATRICS
Lessons Learned The report, The Outcome of Outcomes Research at AHCPR, revealed a lack of alignment between policy makers’ expectations of outcomes research (improved outcomes at lower costs) and researchers’ perspectives on the most salient findings of their work (descriptions of current practice). This distinction underscores the need to link the conduct and focus of outcomes research with opportunities for improvement in care. Doing this well will result in a blurring of the distinction between research and clinical practice. Those projects that led to unambiguous improvements in care were those in which there was a collaboration established early in the research between the research team and relevant professional organizations (eg, cataracts and the American Academy of Opthalmology) or in which there was a clear vehicle to translate the findings into improvements in care (multiple projects resulted in findings used by Medicare’s Quality Improvement Organizations). Since the program was first created, it has become increasingly clear that knowledge of ‘what works’ is necessary but not sufficient to effect change; that knowledge must be linked with a supportive practice environment and incentives for change. AHRQ’s current focus on ‘‘Translating Research into Practice’’ is one outgrowth of this new understanding. Other insights include the need to reconsider the initial criteria used to prioritize topics for outcomes research. At the outset investigators were encouraged to evaluate conditions or procedures that were common, costly (either per unit or in the aggregate), and for which substantial variation in practice was known to exist. However, these criteria may have inadvertently underemphasized issues relevant to those populations who have traditionally received fewer services (women, children, and members of ethnic and racial minority populations). AHRQ’s research and other developments in the field have resulted in the proliferation of short, valid instruments to assess outcomes from the patient’s perspective. However, the lack of standardization makes comparisons across different measures that assess similar constructs impossible. In addition, the use of these tools in routine clinical practice remains the exception rather than the rule. Last, the initial premise underlying outcomes research—that existing data collected for another purpose can provide important insights about clinical care that will lead to improved care and outcomes—has proved to be overly optimistic. Almost all research projects in the first generation of AHRQ’s outcomes research endeavor shared discoveries regarding the limitations of administrative data, and all required substantial effort to augment administrative data to shed light on the critical clinical processes of care. OUTCOMES RESEARCH IN EMERGENCY SERVICES FOR CHILDREN This conference comes at an opportune time, as several groups have identified outcomes research as a critical need in EMSC.5,6,14–16 We conducted a brief Medline search to
AMBULATORY PEDIATRICS
identify recent examples of outcomes research in EMSC. Combining these results with findings from previous publications in this area, we identified numerous gaps in EMSC outcomes research and several areas in which there are unanswered questions.‡ We searched Medline for 1993 through 2000 using the terms ‘‘children or adolescents’’ and ‘‘outcomes assessment–health care.’’ The year 1993 was chosen because it was the year that the Institutes of Medicine (IOM) report on EMSC was published. Medline identified 59 abstracts, but only 11 met our (albeit subjective) definition of outcomes research; one of these was a review. Studies of Prehospital Care Four studies examined various aspects of prehospital care.17–20 This group included one study of the impact of the structure of EMSC services on outcomes, comparing pediatric and adult prehospital trauma care.§20, Another study looked at whether using lights and sirens during transport would affect outcomes.18 Two studies looked at medical procedures done in the prehospital period: ● Open versus closed-chest cardiopulmonary resuscitation for children in cardiac arrest19 ● Endotracheal intubation versus bag-valve-mask ventilation (BVM)17 Only Gausche’s study17 found a difference in outcomes. Studies of Hospital-Based ED Care In the ED setting, 2 studies found that pediatric trauma care resulted in better outcomes (including lower mortality) than adult trauma care.21,22 However, a review by Hulka23 suggested that no conclusions could be reached about the benefits of pediatric trauma systems over adult systems. Studies to improve the quality and efficiency of care in the ED found differing results. Hampers24 found that giving providers price information about tests resulted in lower test ordering with no untoward (or beneficial) effect on outcomes. Sucov25 tested provision of an ED-based asthma-specific care plan, but found no effect on health outcomes. Szilagyi26 tested an ED-based immunization program and found temporary improvement in immunization rates, but at great financial cost. Comparing these scant recent findings to the list of conditions that bring children to the ED indicates that there is ample room for an energetic outcomes research agenda. What should be addressed in the research agenda? At least for the planning stages, this conference has a preliminary focus on a particular set of conditions and a particular setting—the ED. The other groups that have called
‡We also used an additional reference list to double-check our search for EMSC outcomes research. That list yielded no additional publications. §The term structure refers to the resources and organizational arrangements that are in place to deliver care. Donabedian A. Evaluating the quality of medical care. Milbank Q. 1966;44:166–206.
Outcomes Research in Pediatric Emergency Medicine
297
for outcomes research in EMSC have also made suggestions. RESEARCH AGENDAS A 5-year project supported by the National Highway Traffic Safety Administration is developing a set of priorities for outcomes research in the prehospital setting. The first report from the project, called Emergency Medical Services Outcomes Project I, identified a set of highpriority tracer conditions as well as the aspects of outcomes with the most relevance to the condition.6 The intent was to identify conditions and outcomes for which prehospital services could potentially have substantial impact. The researchers also raised the prospect of using tracer conditions and a comprehensive set of outcomes as ways to measure the overall effectiveness of the prehospital system. For children, the 7 conditions identified as having the highest priority for prehospital EMSC outcomes research were minor trauma, major trauma, respiratory distress, airway obstruction, respiratory arrest, cardiac arrest, and seizure. Recommendations for outcomes to be studied varied by condition. For example, survival was the most highly ranked outcome for major trauma, airway obstruction, respiratory arrest, and cardiac arrest. For minor trauma, alleviation of discomfort and patient satisfaction were ranked most highly as outcomes to be studied and potentially improved. Taking a somewhat broader view across the spectrum of EMSC functions identified in the 1993 IOM report, including hospital services, others have identified a somewhat similar group of clinical topics as high priorities, including respiratory distress, multiple organ trauma, brain injury, shock, seizures, and pediatric resuscitation. The group also identified as clinical priorities the following14: asthma, poisoning, behavioral disorders, burns, and fevers. This group further identified the following cross-cutting areas as priorities related to outcomes research: ● Development and validation of outcome measures; ● Development and validation of injury and illness scoring scales; ● Injury prevention strategies and their outcomes; ● Prevention and relief of physical and emotional pain and suffering of patients; and ● Evaluation of the costs and effects of out-of-hospital EMS services as a priority for research.
OPPORTUNITIES AND CHALLENGES Based on our reviews, substantial resources are being devoted to develop research agendas for outcomes research in EMSC. To what has already been developed, we would add the importance of attention to a conceptual framework for EMSC outcomes as well as the importance of attention to perhaps unique methodological issues in EMSC outcomes research.
298
Clancy et al
AMBULATORY PEDIATRICS
Conceptual Framework
Challenges
The fragmentation of health care for Americans, including children, is a well-known phenomenon. Although there is undoubtedly a need for some site-specific research, it is somewhat disturbing to see various groups suggesting that at least some outcomes research focus exclusively on different parts of the EMS system, when coordination among systems and a combination of effects may have substantial impact on children’s outcomes for some conditions. Reconciling these perspectives represents a major challenge for assuring that the products of outcomes research in EMSC will result in outcomes improvement for children. Studying outcomes potentially affected by more than one setting is an undeniable challenge that researchers are just beginning to address by developing refinements of existing methods. For example, studies that include multiple settings—either settings linked to the ED as part of an episode of care or to multiple EDs—must include approaches to adjust for clustering of patients and providers within settings, such as hierarchical linear modeling.
Despite the need and the opportunities, the methodological and practical challenges of conducting outcomes research in EMSC should not be underestimated. Children themselves present some significant challenges as participants in health services research.30 In addition, the nature of EMS presents its own substantial challenges, some of which have been summarized in recent reviews. In a review by Mann and others31 of 284 articles on effectiveness of trauma systems (none involving children), only 42 were found to include an adequate comparison group. Twelve of these studies used expert panels retrospectively examining in-hospital injury deaths to determine their preventability. All had several sources of potential bias, and none examined prehospital deaths or other outcome measures. Similar deficiencies were found in 11 studies using comparisons with large injury registries to determine deaths that were expected, based on the Major Trauma Outcome Study severity scale (MTOS), versus those observed. Mann cites problems with the MTOS as well as overall study design and the limitations of registry data. Cairns and others14 point out that the research has tended to look at conditions in which either good or bad outcomes are relatively rare (eg, cardiac arrest vs straightforward wound care). Another example of methodological challenges is seen in Marianne Gausche’s research,17 which entailed an elaborate and painstaking randomized, controlled trial of the use by paramedics of intubation versus bag-valve-mask resuscitation in children. Because endotracheal intubation had already been adopted as the standard for children in that urban area, despite the lack of a scientific base demonstrating its effectiveness—Dr Gausche had to retrain emergency medical technicians to perform the older bag-valve-mask technique in order to conduct a fair comparison of the 2 methods. Even in adults, intubation has been shown to be fraught with errors.32
Research Priorities The previous efforts to define outcomes research agendas for children are to be applauded for highlighting the importance of patient perspectives such as discomfort, pain, symptom relief, functional outcomes, and satisfaction.6,10,14 For children, of course, the perspective of families or other primary caregivers is important.27 Addressing outcomes from other perspectives is important as well. Measures from the perspective of the payer might have to do with the appropriateness of decisions to admit, refer, or discharge and with the cost-effectiveness of both prehospital (state or locality as payer) and hospital-based care (insurance as typical payer). Developing and validating such measures clearly requires large numbers, multiple collaborating institutions, and carefully designed followup for a year or more after discharge. Studies employing this kind of follow-up are being conducted with adolescent trauma patients (by Holbrook) and for adult cardiac patients (by McNeil28). Others have highlighted the importance of developing valid, reliable, and feasible outcomes measures for outcomes research.14 One intermediate outcome indicator deserving of attention is timeliness, which is of importance to patients, payers, and providers. Extensive work on the importance of timeliness of interventions has been carried out in adults with chest pain, for which the anticipated success of the intervention (thrombolysis) is highly dependent on rapid recognition and dosing,29 but little has been done for children. Another area deserving of attention is the role of the family during the emergency episode. When should families be by their child’s side? Are there really good reasons for taking away this often essential source of support for any patient, particularly a young child? From a research perspective, what difference does it make to have family support?
FUNDING OPPORTUNITIES Research on outcomes in EMSC is emerging as an integral part of federal agencies’ child health services research agenda. We describe several recent and ongoing opportunities for researchers. AHRQ is one of 8 agencies cooperating to issue a program announcement (PA) detailing their interests in supporting research on EMSC (http://grants.nih.gov/grants/ guide/pa-files/PA-01–044.html). This PA indicates the Agency’s ongoing interest in supporting outcomes research as well as methodological research to improve the quality of EMSC. AHRQ also issued a new PA in June 2000 that strongly signals its interest in outcomes research related to children’s health care (http://grants.nih.gov/ grants/guide/pa-files/PA-00–111.html). In addition to the so-called ‘‘R01’’ application—for support of the traditional individual research project—the Agency also supports small project grants, limited to a total of $100 000 or less (total costs), one of whose aims
AMBULATORY PEDIATRICS
is to support new investigators. The AHRQ interests set out in the multiagency PA would also be welcome topics for small project grants. Further information is available on our website, www.ahrq.gov, under the Grant Announcements section. Finally, medical errors and patient safety are a new focus area for AHRQ and other federal agencies. Following upon the IOM report, To Err is Human, the media have been humming with patient-safety anecdotes and descriptions of hopeful counter-measures. A recently published article, led by a trainee, confirmed that pediatric medication error rates in hospitalized patients are common, underscoring the importance of efforts to reduce them.33 The IOM report also brought home that most of the available scientific evidence on medical errors was from inpatient hospitals, although they noted the near certainty of significant error rates in other settings and types of care. Some interventions to prevent medical errors among children in EDs, if not other parts of the EMSC system, are being tested, but certainly more are needed. As an indicator of interest, the EMSC program of the Maternal and Child Health Bureau (Health Resources and Services Administration) held an expert meeting on January 30–31, 2001, to explore issues to be taken into account in preparing guidance for prospective applicants for a research grant to support work on patient safety interventions, such as the Broselow color-coding system. The Broselow tape is a quick way to measure children to avoid egregious errors in medication dosing during medical emergencies and is now being used as the basis for a system of colorcoding all aspects of ED care. Finally, given the opportunities to enhance the state of the science in EMSC outcomes research, the area represents a fertile opportunity for new investigators and trainees. AHRQ supports dissertation grants, individual postdoctoral training awards, career development awards, and other training opportunities (http://www.ahrq.gov/fund/ training/trainix.htm).
Outcomes Research in Pediatric Emergency Medicine
8.
9.
10.
11.
12.
13.
14.
15. 16.
17.
18.
19.
20.
21.
22.
ACKNOWLEDGMENTS
23.
The authors would like to thank Anne Elixhauser and Joshua Thorpe of AHRQ for their assistance with data analysis.
24.
REFERENCES 1. Butler PA. Medicaid HMO enrollees in the emergency room: Use of nonemergency care. Med Care Res Rev. 1998;55:78–98. 2. Malone RE. Whither the almshouse? Overutilization and the role of the emergency department. J Health Politics Policy Law. 1998;23:795–832. 3. Fiser DH. Emergency medical services for children and pediatric subspecialty practice: a call to action. Pediatrics. 1995;96: 188–190. 4. Sinclair LM, Baker MD. Police involvement in pediatric prehospital care. Pediatrics. 1991;87:636–641. 5. Durch JS, Lohr KN, eds. Emergency Medical Services for Children. Washington, DC: National Academy Press; 1993. 6. Maio RF, Garrison HG, Spaite DW, et al. Emergency Medical Services Outcomes Project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33:423–432. 7. Weiss HB, Mathers LJ, Forjuoh SN, et al. Child and Adolescent Emergency Department Visit Databook. Pittsburgh, Pa: Center
25.
26.
27.
28.
29.
299
for Violence and Injury Control, Allegheny University of the Health Sciences; 1997. McCormick MC, Weinick RM, Elixhauser A, et al. Annual report on access to and utilization of health care for children and youth in the United States—2000. Ambul Pediatr. 2001;1:3–15. McCaig LF. National Hospital Ambulatory Medical Care Survey: 1998 Emergency Department Summary. Advance Data. 2000;313:1–24. Sapien RE, Fullerton L, Olson LM, et al. Disturbing trends: the epidemiology of pediatric emergency medical services use. Pediatr EMS. 1999;6:232–238. McNeil BJ, Weichselbaum R, Pauker SG. Speech and survival: tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med. 1981;305:982–987. Stryer D, Tunis S, Hubbard H, Clancy C. The outcomes of outcomes and effectiveness research: impacts and lessons from the first decade. Health Serv Res. 2000;35:977–993. Berman S, Byrns PJ, Bondy J, et al. Otitis media-related antibiotic prescribing patterns, outcomes, and expenditures in a pediatric medicaid population. Pediatrics. 1997;100:585–592. Cairns CB, Garrison HG, Hedges JR, et al. Development of new methods to assess the outcomes of emergency care. Ann Emerg Med. 1998;31:166–171. Ommaya AK, Simpson L, Walker E. More on assessing outcomes of emergency care. Ann Emerg Med. 1998;31:301–303. Seidel JS, Henderson D, Tittle S, et al. Priorities for research in emergency medical services for children: results of a consensus conference. J Emerg Nursing. 1999;25:12–16. Gausche M, Lewis RJ, Stratton SJ, et al. Effect of out-of-hospital pediatric endotracheal intubation on survival and neurological outcome: a controlled clinical trial. JAMA. 2000;283: 783–790. Kupas DF, Dula DJ, Pino BJ. Patient outcome using medical protocol to limit ‘‘lights and siren’’ transport. Prehospital Disaster Med. 1994;9:226–229. Sheikh A, Brogan T. Outcome and cost of open- and closedchest cardiopulmonary resuscitation in pediatric cardiac arrests. Pediatrics. 1994;93:392–398. Paul TR, Marias M, Pons PT, et al. Adult versus pediatric prehospital trauma care: is there a difference? J Trauma. 1999;47: 455–459. Hall JR, Reyes HM, Meller JL, et al. The outcome for children with blunt trauma is best at a pediatric trauma center. J Pediatr Surg. 1996;31:72–76. Potoka DA, Schall LC, Gardner MJ, et al. Impact of pediatric trauma centers on mortality in a statewide system. J Trauma. 2000;49:237–245. Hulka F. Pediatric trauma systems: critical distinctions. J Trauma. 1999;47(suppl 3):S85–S89. Hampers LC, Cha S, Gutglass DJ, et al. Fast track and the pediatric emergency department: resource utilization and patients outcomes. Acad Emerg Med. 1999;6:1153–1159. Sucov A, Veenema TG. Implementation of a disease-specific care plan changes clinician behaviors. Am J Emerg Med. 2000; 18:367–371. Szilagyi PG, Rodewald LE, Humiston SG, et al. Effect of two urban emergency department immunization programs on childhood immunization rates. Arch Pediatr Adolesc Med. 1997;151: 999–1006. Christakis DA, Johnston BD, Connell FA. Methodological issues in pediatric outcomes research. Ambul Pediatr. 2001;1:59– 62. Chertow GM, Normand SL, Silva LR, McNeil BJ. Survival after acute myocardial infarction in patients with end-stage renal disease: results from the cooperative cardiovascular project. Am J Kidney Dis. 2000;35:1044–1051. Schmid CH, D’Agostino RB, Griffith JL, et al. A logistic regression model when some events precede treatment: the effect of thrombolytic therapy for acute myocardial infarction on the risk of cardiac arrest. J Clin Epidemiol. 1997;50:1219–1229.
300
Clancy et al
30. Lohr KN, Dougherty D, Simpson L. Methodologic challenges in health services research in the pediatric population. Ambul Pediatr. 2001;1:36–38. 31. Mann NC, Mullins RJ, MacKenzie EJ, et al. Systematic review of published evidence regarding trauma system effectiveness. J Trauma Injury Infect Crit Care. 1999;47:S25–S33.
AMBULATORY PEDIATRICS 32. Li J, Murphy-Lavoie H, Bugas C, et al. Complications of emergency intubation with and without paralysis. Am J Emerg Med. 1999;17:141–143. 33. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001; 285:2114–2120.
outcomes; pediatric emergency medicine; research methods
Ambulatory Pediatrics 2002;2:293 300
PERSPECTIVE: WHAT IS EMERGENCY MEDICAL CARE FOR CHILDREN?
school, and even the police, as de facto first responders and/or agents of prevention.4 In a 1993 report, Emergency Medical Services for Children, the Institute of Medicine5 identified 7 basic responsibilities for emergency medical systems for children (EMSC):
One can view that segment of the healthcare system that centers on the emergency department in several different ways. On the one hand, it is the well-known safety net provider of ambulatory care to those without a medical home.1,2 On the other hand, in combination with the out-of-hospital emergency care system, it receives, stabilizes, and treats or refers people with specific conditions, such as trauma, or those with severe medical conditions, such as children with compromised respiration. It is necessary to see its emergency care role in a systems context, as pointed out by Fiser.3 From that perspective, emergency care is part of a cycle potentially involving all aspects of the care of a child, including not only resolution of a given episode of critical care but also the integration of the results of that episode into ongoing primary care, which then includes preventive care to prevent or mitigate future emergencies. Studying outcomes from this perspective also becomes a sequential and iterative process that must take into account the expected sequences of interventions and of expected results. Complicating the picture further is the role of the parent or caregiver, the
1) Identifying emergencies and the need for emergency care; 2) Ensuring access to the services of the system (eg, through 911 telephone service), with dispatch of personnel and equipment; 3) Providing appropriate prehospital care; 4) Transporting children to appropriate points of care; 5) Providing definitive treatment, including access to tertiary and rehabilitative care as needed; 6) Ensuring communication among emergency care providers and with other interested parties, including parents and primary care providers; and 7) Using information systems and feedback to assess and improve patient care and system performance and to point to areas for prevention efforts. This view of emergency care for children is often not reflected in the literature. For example, many articles and reports on outcomes research needs in ‘EMS’ address one of the above components, such as prehospital care, without acknowledging the entire spectrum of emergency services or addressing how a single aspect of emergency medical services fits within the health care system.6
From the Agency for Healthcare Research and Quality, US Department of Health and Human Services, Rockville, Md. The opinions expressed are the authors’ and are not intended to represent official policy of Agency for Healthcare Research and Quality or the Department of Health and Human Services. Address correspondence to Denise Dougherty, PhD, AHRQ, 2101 East Jefferson St, Rockville, MD 20852 (e-mail: ddougher@ahrq. gov). Received for publication October 1, 2001; accepted March 8, 2002. AMBULATORY PEDIATRICS Copyright q 2002 by Ambulatory Pediatric Association
BASIC STATISTICS ON EMSC Children are frequent users of emergency services. Weiss and colleagues estimated that there is, on average,
293
Volume 2, Number 4 Supplement July–August 2002
294
Clancy et al
one child or adolescent visit to an emergency department (ED) once every second.7 In 1996, 13% of children aged 0 to 17 years had an ED visit, according to the Agency for Healthcare Research and Quality (AHRQ) Medical Expenditure Panel Survey (MEPS).8 The visit rate was highest for children aged 0 to 4 years (15.9%) and aged 15 to 17 years (13.2%) and for children who were reported to be in fair or poor health status overall. A slightly greater percentage of white non-Hispanic children had an ED visit (14.4%) compared to black or Hispanic children. Emergency services are not cheap. Children with ED visits had an average of $500 in emergency room expenditures in 1996; each visit resulted in an average expenditure for ED care of $361.8 White children had higher average expenditures for ED visits. The ED is a frequent source of entry to inpatient care. In 1997, over 40% of children ages 1 through 14 years (equal to 707 000 children) who were discharged from inpatient hospitals had been admitted through the emergency room, according to ARHQ’s Healthcare Cost and Utilization Project (HCUP) data.8 In comparison, 33.6% of all hospital inpatient stays among adults ages 18 to 44 years originate in the ED. There is considerable variation in admissions to the hospital through the ED, ranging from just over 9% of all children’s hospital stays in Utah and Oregon to a high of 23.3% in New York.8 Why do children seek emergency care? Data from hospital EDs indicate that 15% of children under the age of 15 years seek care for conditions that are emergent, 32% seek care for urgent conditions, 14% for semiurgent conditions, and 11.4% for nonurgent conditions.*9 The percentage of visits that are judged emergent for Americans of all ages combined is slightly higher than that for children at 19%. Injury is the leading cause of ED visits among children aged 0 to 17 years (43%), followed by the category of ‘‘other’’ respiratory infections (13%), then otitis media (8%), then asthma (5%), and then intestinal infections (3%), according to AHRQ’s MEPS data.† The MEPS sample is too limited to analyze diagnoses by children’s age, but data from AHRQ’s HCUP data set of inpatient data (hospital discharge abstracts from 26 states) provide insights regarding how the type of illness changes with age. For children under the age of 1 year, the leading causes of inpatient admissions via the ED are related to respiratory conditions other than asthma, other perinatal conditions, and viral infections. In the next age grouping, 1–4 years old, asthma takes precedence as the most frequent cause of ED-related inpatient admission, followed by
*Definitions used in the survey are as follows: Emergency 5 a visit in which the patient should be seen in less than 15 minutes. Urgent 5 a visit in which the patient should be seen within 15–60 minutes. Semiurgent 5 a visit in which the patient should be seen within 60–120 minutes. Nonurgent 5 a visit in which the patient should be seen within 24 hours. †Unpublished data from the Medical Expenditure Panel Survey, analyzed January 2001.
AMBULATORY PEDIATRICS
pneumonia. Fluid and electrolyte imbalances are also important, as is epilepsy. Asthma is an even more substantial cause of ED-related admissions in the 5- to 9-year-old group, with pneumonia declining slightly as a cause. In this age group, however, appendicitis and injuries (upper and lower limb fractures) also increase in prevalence. For 10–14-year-olds, asthma and appendicitis become equally important as a cause of ED-related admissions, followed in importance by upper and lower limb fractures. In this age group, mental health problems make their first appearance as a leading cause of ED-related admissions, with affective disorders, other mental health problems, and poisoning by medications (likely an indicator of suicide attempt) accounting, in total, for 9% of ED-related inpatient admissions. In the 15–17-yearold group, reasons for ED-related admissions are extremely variable, with appendicitis dominating as a single diagnosis but with injuries and mental health problems also quite frequent. Our analysis of causes by major diagnostic categories gives another picture of the arguably most serious reasons for children’s entry into the ED and then into inpatient care. Available data on prehospital use of emergency medical services (EMS) for children are fairly consistent with the data on the serious conditions that result in inpatient admissions, except that in one analysis, seizures were estimated to account for 14% of prehospital EMS use.10 These prehospital estimates are not comparable to the hospital estimates. For example, the hospital estimates are based on nationally representative data, and the prehospital data are based on data available from possibly unrepresentative states and counties. The hospital data are for children ages 0 to 17 years and the prehospital estimates were made for children younger than 15 years of age. Further, the analyses used different definitions to categorize conditions. These data sources, although imperfect, strongly indicate that children seek a full range of emergency medical services for conditions that are not trivial and that it is critical that we strengthen our efforts to learn what works when children do encounter the need for emergency medical services. WHAT IS OUTCOMES RESEARCH? WHY OUTCOMES RESEARCH? Interest in assessing the outcomes or end results of health services reflects a convergence of several important forces in contemporary medicine. First is the growing burden of illness attributable to chronic illnesses, for which the therapeutic goal is often palliation, symptom relief, or improvement or maintenance of function. The correlation between these end-points and traditional metrics for assessing the success of clinical care (eg, mortality; physiologic measures or surrogates for physiology) is imperfect at best, so researchers have developed new approaches to assessing additional dimensions of outcomes. A second theme is increased recognition of the patient’s central role in decision making. For many clinical decisions, no one intervention is clearly associated with enhanced quality of
AMBULATORY PEDIATRICS
life or survival, or there are critical tradeoffs among different types of outcomes, such as length of survival, pain relief, or preservation of function, as exemplified by Barbara McNeil’s11 classic study about choices of treatment for laryngeal cancer. In other cases, interventions with substantial side effects offer modest but real survival benefits. A third issue is the limited generalizability or external validity of findings from clinical trials. Efficiency in research design and the need for internal validity has often resulted in restrictive inclusion criteria that limit broad applicability in clinical practice. In the past 15 years, mounting evidence of extensive variations in practice has captured the attention of policy makers and purchasers struggling to develop effective cost-containment programs without sacrificing quality of care. The most compelling explanatory hypothesis for practice variations is the consistent inverse relationship observed between practice variations and professional consensus. This relationship indicates that the existence of practice variation reflects both differences in practice style as well as an indication about where medical practice is limited by gaps in available evidence, though differences in patient populations, access to care, and local health care capacity also contribute. In the context of the US health care system, where supply-side regulation of health facilities has not been politically sustainable, a focus on developing better information on the effectiveness of interventions is considered to be a crucial strategy for increasing value in health care. At the same time, continued increases in health spending have led to new demands for accountability, referred to as the ‘‘third revolution’’ in health care. The widespread availability of inexpensive computing power, making examination of large databases feasible, has led to the use of existing data (usually from administrative claims) to conduct observational studies that include heterogeneous groups of patients, with an emphasis on quasi-experimental methods and nonrandomized designs. These methods have helped clarify the effects of clinical interventions in typical practice settings. The sometimes controversial assumption that statistical methods can be used to adjust for unmeasured selection bias does not negate the power of demonstrating the spectrum of care provided across multiple practice settings. Finally, until recently, the rate of increase in health costs had begun to decrease, resulting in concerns about possible threats to the quality of care. To measure the quality of care, purchasers, patients, and advocates recognize the need to evaluate the outcomes of care. In short, the focus of outcomes research, defined as inquiry that evaluates the impact of health care (including discrete interventions such as particular drugs, medical devices, and procedures as well as broader programmatic or system interventions) on the health outcomes of patients and populations, is pragmatic and closely linked with the information needs of health care decision makers. In addition, the establishment of a new federal agency in 1989, with a clear mandate to evaluate the outcomes of care, was linked with an explicit expectation that this re-
Outcomes Research in Pediatric Emergency Medicine
295
search would result in tangible improvements in clinical practice. INSIGHTS FROM THE FIRST DECADE OF OUTCOMES RESEARCH AT AHRQ Accomplishments In 1998 the Center for Outcomes and Effectiveness Research sponsored an evaluation of the first decade of investments in outcomes research supported by AHRQ (then the Agency for Health Care Policy and Research) for the purpose of identifying accomplishments and opportunities for future improvements.12 The full report, based on a survey of all principal investigators and indepth case studies of selected projects, can be obtained from AHRQ’s website (http://www.ahrq.gov). Responses to a survey of principal investigators demonstrated that the most salient findings identified by those researchers could be classified as descriptive epidemiology of medical practice. Although describing existing practice patterns is a necessary prerequisite to improving those practices, these responses also indicate a lack of concordance between policy makers’ and stakeholders’ expectations of outcomes research and those of the research community. In order to evaluate the impact of research investments on subsequent changes in practice and policy, a framework for assessing the impact of funded research was developed, one that included 4 levels of possible impact. Level 1 impacts are research findings published in peer-reviewed journals and clarification of testable hypotheses that merit further study. Level 2 impacts occur when specific research findings are used by one or more entities (‘change agents’), such as professional organizations, payors, patient organizations, accrediting organizations, the media, and others, to influence health policies. For example, evidence that beta-blockers reduce mortality in patients who have had an acute myocardial infarction was used by multiple organizations to increase the use of this effective intervention. Measurable changes in clinical practice are considered Level 3 impacts, and evidence that research has resulted in improvements in health outcomes is the highest level of impact, or a Level 4 impact. Not surprisingly, the report found that the majority of findings were consistent with Level 1 impact, such as peer-reviewed publications. Challenging Clinical Dogma In multiple areas, assessment of current clinical practice has resulted in important challenges to prevailing clinical ideas about how to manage specific clinical problems. For example, a study of otitis media in a pediatric Medicaid population showed that children with otitis media were more likely to receive a second course of antibiotics if they were initially given an expensive name-brand product compared with a generic product.13 This counter-intuitive finding helps to raise questions about one of the specific explanations given for using more recently developed drug therapies (ie, that failure rates are higher for generic antibiotics). This study result does not conclusively prove that cure rates are unequal for the 2 groups
296
Clancy et al
of drugs, since the same pattern of prescriptions would be observed if, for example, the clinicians who use expensive drugs are also more likely to prescribe a second course of therapy. The primary value of this study is confirmation in a large population (over 12 000 cases) that there is no evidence of worse outcomes with generic, narrow-spectrum antibiotics. Results from descriptive assessments of current clinical practice can clarify hypotheses and verify sufficient uncertainty to augment support for subsequent clinical trials. Improved Measures of Outcome Many health care interventions have important effects on patient function and well being but little effect in terms of death or disability. What matters for patients with cataracts is the impact of visual function on daily activities. Men with enlarged prostates may be more bothered by the potential risk of incontinence or impotence from surgery than by the need to get up at night to urinate. AHRQ’s outcomes research projects led in some instances to the development of new measures of outcomes, consistent with the development of shorter, valid tools that has progressed over the past 30 years, but more often these projects assessed the relationship between specific processes of care and existing outcome measures, both disease- or condition-specific and generic. A recent symposium cosponsored by AHRQ, the National Cancer Institute (NCI), and others on challenges in outcomes measurement is a reflection of the continued dynamism in this field. The VF-14, developed to help patients and clinicians understand who is most likely to benefit from cataract surgery, has been used in Manitoba to rationalize queues for elective surgery. A modification of this tool is now required by the National Eye Institute in all clinical trials. A related component of the focus on outcomes that matters to patients has been the recognition that for clinical decisions that involve 2 or more equally effective alternatives, the ultimate decision maker is the patient. Outcomes researchers have developed practical approaches for assessing the impact of alternative treatments, including no treatment, as well as methods for soliciting patient judgments regarding the value of those outcomes, so that individuals can select the option most consistent with their preferences. Shared decision-making programs have been developed for prostatic hypertrophy, breast cancer, the decision to screen for prostate cancer, back pain, and other conditions. Patients who participate in these programs have been shown to be more satisfied with their decisions and to report improved physical function compared with nonparticipants, even when their decisions as a group did not differ significantly. Finally, a critical accomplishment of the first decade of outcomes research was the development of methods and tools to assess outcomes of care when episodes of care involved multiple settings. Linking data for episodes of care that include hospital care, outpatient care, ED services, and care at other sites is difficult but important to identify what works and to identify specific opportunities to improve care and outcomes.
AMBULATORY PEDIATRICS
Lessons Learned The report, The Outcome of Outcomes Research at AHCPR, revealed a lack of alignment between policy makers’ expectations of outcomes research (improved outcomes at lower costs) and researchers’ perspectives on the most salient findings of their work (descriptions of current practice). This distinction underscores the need to link the conduct and focus of outcomes research with opportunities for improvement in care. Doing this well will result in a blurring of the distinction between research and clinical practice. Those projects that led to unambiguous improvements in care were those in which there was a collaboration established early in the research between the research team and relevant professional organizations (eg, cataracts and the American Academy of Opthalmology) or in which there was a clear vehicle to translate the findings into improvements in care (multiple projects resulted in findings used by Medicare’s Quality Improvement Organizations). Since the program was first created, it has become increasingly clear that knowledge of ‘what works’ is necessary but not sufficient to effect change; that knowledge must be linked with a supportive practice environment and incentives for change. AHRQ’s current focus on ‘‘Translating Research into Practice’’ is one outgrowth of this new understanding. Other insights include the need to reconsider the initial criteria used to prioritize topics for outcomes research. At the outset investigators were encouraged to evaluate conditions or procedures that were common, costly (either per unit or in the aggregate), and for which substantial variation in practice was known to exist. However, these criteria may have inadvertently underemphasized issues relevant to those populations who have traditionally received fewer services (women, children, and members of ethnic and racial minority populations). AHRQ’s research and other developments in the field have resulted in the proliferation of short, valid instruments to assess outcomes from the patient’s perspective. However, the lack of standardization makes comparisons across different measures that assess similar constructs impossible. In addition, the use of these tools in routine clinical practice remains the exception rather than the rule. Last, the initial premise underlying outcomes research—that existing data collected for another purpose can provide important insights about clinical care that will lead to improved care and outcomes—has proved to be overly optimistic. Almost all research projects in the first generation of AHRQ’s outcomes research endeavor shared discoveries regarding the limitations of administrative data, and all required substantial effort to augment administrative data to shed light on the critical clinical processes of care. OUTCOMES RESEARCH IN EMERGENCY SERVICES FOR CHILDREN This conference comes at an opportune time, as several groups have identified outcomes research as a critical need in EMSC.5,6,14–16 We conducted a brief Medline search to
AMBULATORY PEDIATRICS
identify recent examples of outcomes research in EMSC. Combining these results with findings from previous publications in this area, we identified numerous gaps in EMSC outcomes research and several areas in which there are unanswered questions.‡ We searched Medline for 1993 through 2000 using the terms ‘‘children or adolescents’’ and ‘‘outcomes assessment–health care.’’ The year 1993 was chosen because it was the year that the Institutes of Medicine (IOM) report on EMSC was published. Medline identified 59 abstracts, but only 11 met our (albeit subjective) definition of outcomes research; one of these was a review. Studies of Prehospital Care Four studies examined various aspects of prehospital care.17–20 This group included one study of the impact of the structure of EMSC services on outcomes, comparing pediatric and adult prehospital trauma care.§20, Another study looked at whether using lights and sirens during transport would affect outcomes.18 Two studies looked at medical procedures done in the prehospital period: ● Open versus closed-chest cardiopulmonary resuscitation for children in cardiac arrest19 ● Endotracheal intubation versus bag-valve-mask ventilation (BVM)17 Only Gausche’s study17 found a difference in outcomes. Studies of Hospital-Based ED Care In the ED setting, 2 studies found that pediatric trauma care resulted in better outcomes (including lower mortality) than adult trauma care.21,22 However, a review by Hulka23 suggested that no conclusions could be reached about the benefits of pediatric trauma systems over adult systems. Studies to improve the quality and efficiency of care in the ED found differing results. Hampers24 found that giving providers price information about tests resulted in lower test ordering with no untoward (or beneficial) effect on outcomes. Sucov25 tested provision of an ED-based asthma-specific care plan, but found no effect on health outcomes. Szilagyi26 tested an ED-based immunization program and found temporary improvement in immunization rates, but at great financial cost. Comparing these scant recent findings to the list of conditions that bring children to the ED indicates that there is ample room for an energetic outcomes research agenda. What should be addressed in the research agenda? At least for the planning stages, this conference has a preliminary focus on a particular set of conditions and a particular setting—the ED. The other groups that have called
‡We also used an additional reference list to double-check our search for EMSC outcomes research. That list yielded no additional publications. §The term structure refers to the resources and organizational arrangements that are in place to deliver care. Donabedian A. Evaluating the quality of medical care. Milbank Q. 1966;44:166–206.
Outcomes Research in Pediatric Emergency Medicine
297
for outcomes research in EMSC have also made suggestions. RESEARCH AGENDAS A 5-year project supported by the National Highway Traffic Safety Administration is developing a set of priorities for outcomes research in the prehospital setting. The first report from the project, called Emergency Medical Services Outcomes Project I, identified a set of highpriority tracer conditions as well as the aspects of outcomes with the most relevance to the condition.6 The intent was to identify conditions and outcomes for which prehospital services could potentially have substantial impact. The researchers also raised the prospect of using tracer conditions and a comprehensive set of outcomes as ways to measure the overall effectiveness of the prehospital system. For children, the 7 conditions identified as having the highest priority for prehospital EMSC outcomes research were minor trauma, major trauma, respiratory distress, airway obstruction, respiratory arrest, cardiac arrest, and seizure. Recommendations for outcomes to be studied varied by condition. For example, survival was the most highly ranked outcome for major trauma, airway obstruction, respiratory arrest, and cardiac arrest. For minor trauma, alleviation of discomfort and patient satisfaction were ranked most highly as outcomes to be studied and potentially improved. Taking a somewhat broader view across the spectrum of EMSC functions identified in the 1993 IOM report, including hospital services, others have identified a somewhat similar group of clinical topics as high priorities, including respiratory distress, multiple organ trauma, brain injury, shock, seizures, and pediatric resuscitation. The group also identified as clinical priorities the following14: asthma, poisoning, behavioral disorders, burns, and fevers. This group further identified the following cross-cutting areas as priorities related to outcomes research: ● Development and validation of outcome measures; ● Development and validation of injury and illness scoring scales; ● Injury prevention strategies and their outcomes; ● Prevention and relief of physical and emotional pain and suffering of patients; and ● Evaluation of the costs and effects of out-of-hospital EMS services as a priority for research.
OPPORTUNITIES AND CHALLENGES Based on our reviews, substantial resources are being devoted to develop research agendas for outcomes research in EMSC. To what has already been developed, we would add the importance of attention to a conceptual framework for EMSC outcomes as well as the importance of attention to perhaps unique methodological issues in EMSC outcomes research.
298
Clancy et al
AMBULATORY PEDIATRICS
Conceptual Framework
Challenges
The fragmentation of health care for Americans, including children, is a well-known phenomenon. Although there is undoubtedly a need for some site-specific research, it is somewhat disturbing to see various groups suggesting that at least some outcomes research focus exclusively on different parts of the EMS system, when coordination among systems and a combination of effects may have substantial impact on children’s outcomes for some conditions. Reconciling these perspectives represents a major challenge for assuring that the products of outcomes research in EMSC will result in outcomes improvement for children. Studying outcomes potentially affected by more than one setting is an undeniable challenge that researchers are just beginning to address by developing refinements of existing methods. For example, studies that include multiple settings—either settings linked to the ED as part of an episode of care or to multiple EDs—must include approaches to adjust for clustering of patients and providers within settings, such as hierarchical linear modeling.
Despite the need and the opportunities, the methodological and practical challenges of conducting outcomes research in EMSC should not be underestimated. Children themselves present some significant challenges as participants in health services research.30 In addition, the nature of EMS presents its own substantial challenges, some of which have been summarized in recent reviews. In a review by Mann and others31 of 284 articles on effectiveness of trauma systems (none involving children), only 42 were found to include an adequate comparison group. Twelve of these studies used expert panels retrospectively examining in-hospital injury deaths to determine their preventability. All had several sources of potential bias, and none examined prehospital deaths or other outcome measures. Similar deficiencies were found in 11 studies using comparisons with large injury registries to determine deaths that were expected, based on the Major Trauma Outcome Study severity scale (MTOS), versus those observed. Mann cites problems with the MTOS as well as overall study design and the limitations of registry data. Cairns and others14 point out that the research has tended to look at conditions in which either good or bad outcomes are relatively rare (eg, cardiac arrest vs straightforward wound care). Another example of methodological challenges is seen in Marianne Gausche’s research,17 which entailed an elaborate and painstaking randomized, controlled trial of the use by paramedics of intubation versus bag-valve-mask resuscitation in children. Because endotracheal intubation had already been adopted as the standard for children in that urban area, despite the lack of a scientific base demonstrating its effectiveness—Dr Gausche had to retrain emergency medical technicians to perform the older bag-valve-mask technique in order to conduct a fair comparison of the 2 methods. Even in adults, intubation has been shown to be fraught with errors.32
Research Priorities The previous efforts to define outcomes research agendas for children are to be applauded for highlighting the importance of patient perspectives such as discomfort, pain, symptom relief, functional outcomes, and satisfaction.6,10,14 For children, of course, the perspective of families or other primary caregivers is important.27 Addressing outcomes from other perspectives is important as well. Measures from the perspective of the payer might have to do with the appropriateness of decisions to admit, refer, or discharge and with the cost-effectiveness of both prehospital (state or locality as payer) and hospital-based care (insurance as typical payer). Developing and validating such measures clearly requires large numbers, multiple collaborating institutions, and carefully designed followup for a year or more after discharge. Studies employing this kind of follow-up are being conducted with adolescent trauma patients (by Holbrook) and for adult cardiac patients (by McNeil28). Others have highlighted the importance of developing valid, reliable, and feasible outcomes measures for outcomes research.14 One intermediate outcome indicator deserving of attention is timeliness, which is of importance to patients, payers, and providers. Extensive work on the importance of timeliness of interventions has been carried out in adults with chest pain, for which the anticipated success of the intervention (thrombolysis) is highly dependent on rapid recognition and dosing,29 but little has been done for children. Another area deserving of attention is the role of the family during the emergency episode. When should families be by their child’s side? Are there really good reasons for taking away this often essential source of support for any patient, particularly a young child? From a research perspective, what difference does it make to have family support?
FUNDING OPPORTUNITIES Research on outcomes in EMSC is emerging as an integral part of federal agencies’ child health services research agenda. We describe several recent and ongoing opportunities for researchers. AHRQ is one of 8 agencies cooperating to issue a program announcement (PA) detailing their interests in supporting research on EMSC (http://grants.nih.gov/grants/ guide/pa-files/PA-01–044.html). This PA indicates the Agency’s ongoing interest in supporting outcomes research as well as methodological research to improve the quality of EMSC. AHRQ also issued a new PA in June 2000 that strongly signals its interest in outcomes research related to children’s health care (http://grants.nih.gov/ grants/guide/pa-files/PA-00–111.html). In addition to the so-called ‘‘R01’’ application—for support of the traditional individual research project—the Agency also supports small project grants, limited to a total of $100 000 or less (total costs), one of whose aims
AMBULATORY PEDIATRICS
is to support new investigators. The AHRQ interests set out in the multiagency PA would also be welcome topics for small project grants. Further information is available on our website, www.ahrq.gov, under the Grant Announcements section. Finally, medical errors and patient safety are a new focus area for AHRQ and other federal agencies. Following upon the IOM report, To Err is Human, the media have been humming with patient-safety anecdotes and descriptions of hopeful counter-measures. A recently published article, led by a trainee, confirmed that pediatric medication error rates in hospitalized patients are common, underscoring the importance of efforts to reduce them.33 The IOM report also brought home that most of the available scientific evidence on medical errors was from inpatient hospitals, although they noted the near certainty of significant error rates in other settings and types of care. Some interventions to prevent medical errors among children in EDs, if not other parts of the EMSC system, are being tested, but certainly more are needed. As an indicator of interest, the EMSC program of the Maternal and Child Health Bureau (Health Resources and Services Administration) held an expert meeting on January 30–31, 2001, to explore issues to be taken into account in preparing guidance for prospective applicants for a research grant to support work on patient safety interventions, such as the Broselow color-coding system. The Broselow tape is a quick way to measure children to avoid egregious errors in medication dosing during medical emergencies and is now being used as the basis for a system of colorcoding all aspects of ED care. Finally, given the opportunities to enhance the state of the science in EMSC outcomes research, the area represents a fertile opportunity for new investigators and trainees. AHRQ supports dissertation grants, individual postdoctoral training awards, career development awards, and other training opportunities (http://www.ahrq.gov/fund/ training/trainix.htm).
Outcomes Research in Pediatric Emergency Medicine
8.
9.
10.
11.
12.
13.
14.
15. 16.
17.
18.
19.
20.
21.
22.
ACKNOWLEDGMENTS
23.
The authors would like to thank Anne Elixhauser and Joshua Thorpe of AHRQ for their assistance with data analysis.
24.
REFERENCES 1. Butler PA. Medicaid HMO enrollees in the emergency room: Use of nonemergency care. Med Care Res Rev. 1998;55:78–98. 2. Malone RE. Whither the almshouse? Overutilization and the role of the emergency department. J Health Politics Policy Law. 1998;23:795–832. 3. Fiser DH. Emergency medical services for children and pediatric subspecialty practice: a call to action. Pediatrics. 1995;96: 188–190. 4. Sinclair LM, Baker MD. Police involvement in pediatric prehospital care. Pediatrics. 1991;87:636–641. 5. Durch JS, Lohr KN, eds. Emergency Medical Services for Children. Washington, DC: National Academy Press; 1993. 6. Maio RF, Garrison HG, Spaite DW, et al. Emergency Medical Services Outcomes Project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33:423–432. 7. Weiss HB, Mathers LJ, Forjuoh SN, et al. Child and Adolescent Emergency Department Visit Databook. Pittsburgh, Pa: Center
25.
26.
27.
28.
29.
299
for Violence and Injury Control, Allegheny University of the Health Sciences; 1997. McCormick MC, Weinick RM, Elixhauser A, et al. Annual report on access to and utilization of health care for children and youth in the United States—2000. Ambul Pediatr. 2001;1:3–15. McCaig LF. National Hospital Ambulatory Medical Care Survey: 1998 Emergency Department Summary. Advance Data. 2000;313:1–24. Sapien RE, Fullerton L, Olson LM, et al. Disturbing trends: the epidemiology of pediatric emergency medical services use. Pediatr EMS. 1999;6:232–238. McNeil BJ, Weichselbaum R, Pauker SG. Speech and survival: tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med. 1981;305:982–987. Stryer D, Tunis S, Hubbard H, Clancy C. The outcomes of outcomes and effectiveness research: impacts and lessons from the first decade. Health Serv Res. 2000;35:977–993. Berman S, Byrns PJ, Bondy J, et al. Otitis media-related antibiotic prescribing patterns, outcomes, and expenditures in a pediatric medicaid population. Pediatrics. 1997;100:585–592. Cairns CB, Garrison HG, Hedges JR, et al. Development of new methods to assess the outcomes of emergency care. Ann Emerg Med. 1998;31:166–171. Ommaya AK, Simpson L, Walker E. More on assessing outcomes of emergency care. Ann Emerg Med. 1998;31:301–303. Seidel JS, Henderson D, Tittle S, et al. Priorities for research in emergency medical services for children: results of a consensus conference. J Emerg Nursing. 1999;25:12–16. Gausche M, Lewis RJ, Stratton SJ, et al. Effect of out-of-hospital pediatric endotracheal intubation on survival and neurological outcome: a controlled clinical trial. JAMA. 2000;283: 783–790. Kupas DF, Dula DJ, Pino BJ. Patient outcome using medical protocol to limit ‘‘lights and siren’’ transport. Prehospital Disaster Med. 1994;9:226–229. Sheikh A, Brogan T. Outcome and cost of open- and closedchest cardiopulmonary resuscitation in pediatric cardiac arrests. Pediatrics. 1994;93:392–398. Paul TR, Marias M, Pons PT, et al. Adult versus pediatric prehospital trauma care: is there a difference? J Trauma. 1999;47: 455–459. Hall JR, Reyes HM, Meller JL, et al. The outcome for children with blunt trauma is best at a pediatric trauma center. J Pediatr Surg. 1996;31:72–76. Potoka DA, Schall LC, Gardner MJ, et al. Impact of pediatric trauma centers on mortality in a statewide system. J Trauma. 2000;49:237–245. Hulka F. Pediatric trauma systems: critical distinctions. J Trauma. 1999;47(suppl 3):S85–S89. Hampers LC, Cha S, Gutglass DJ, et al. Fast track and the pediatric emergency department: resource utilization and patients outcomes. Acad Emerg Med. 1999;6:1153–1159. Sucov A, Veenema TG. Implementation of a disease-specific care plan changes clinician behaviors. Am J Emerg Med. 2000; 18:367–371. Szilagyi PG, Rodewald LE, Humiston SG, et al. Effect of two urban emergency department immunization programs on childhood immunization rates. Arch Pediatr Adolesc Med. 1997;151: 999–1006. Christakis DA, Johnston BD, Connell FA. Methodological issues in pediatric outcomes research. Ambul Pediatr. 2001;1:59– 62. Chertow GM, Normand SL, Silva LR, McNeil BJ. Survival after acute myocardial infarction in patients with end-stage renal disease: results from the cooperative cardiovascular project. Am J Kidney Dis. 2000;35:1044–1051. Schmid CH, D’Agostino RB, Griffith JL, et al. A logistic regression model when some events precede treatment: the effect of thrombolytic therapy for acute myocardial infarction on the risk of cardiac arrest. J Clin Epidemiol. 1997;50:1219–1229.
300
Clancy et al
30. Lohr KN, Dougherty D, Simpson L. Methodologic challenges in health services research in the pediatric population. Ambul Pediatr. 2001;1:36–38. 31. Mann NC, Mullins RJ, MacKenzie EJ, et al. Systematic review of published evidence regarding trauma system effectiveness. J Trauma Injury Infect Crit Care. 1999;47:S25–S33.
AMBULATORY PEDIATRICS 32. Li J, Murphy-Lavoie H, Bugas C, et al. Complications of emergency intubation with and without paralysis. Am J Emerg Med. 1999;17:141–143. 33. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001; 285:2114–2120.