- Email: [email protected]
Traumatic Brain Injury Rehabilitation Comparative Effectiveness Research: Introduction to the Traumatic Brain Injury–Practice Based Evidence Archives Supplement
Archives of Physical Medicine and Rehabilitation journal homepage: www.archives-pmr.org Archives of Physical Medicine and Rehabilitation 2015;96(8 Suppl 3):S173-7
INTRODUCTION
Traumatic Brain Injury Rehabilitation Comparative Effectiveness Research: Introduction to the Traumatic Brain InjuryePractice Based Evidence Archives Supplement Susan D. Horn, PhD,a John D. Corrigan, PhD,b Marcel P. Dijkers, PhDc From the aInstitute for Clinical Outcomes Research, International Severity Information Systems, Salt Lake City, UT; bDepartment of Physical Medicine and Rehabilitation, Ohio State University, Columbus, OH; and cDepartment of Rehabilitation Medicine, Icahn School of Medicine at Mount Sinai, New York, NY. Current affiliation for Horn, Department of Population Health Sciences, University of Utah School of Medicine, Salt Lake City, UT.
Abstract This supplement of the Archives of Physical Medicine and Rehabilitation is devoted to the Traumatic Brain InjuryePractice Based Evidence study, the first practice-based evidence study, to our knowledge, of traumatic brain injury rehabilitation. The purpose of this preface is to place this study in the broader context of comparative effectiveness research and introduce the articles in the supplement. Archives of Physical Medicine and Rehabilitation 2015;96(8 Suppl 3):S173-7 ª 2015 by the American Congress of Rehabilitation Medicine
This supplement of the Archives of Physical Medicine and Rehabilitation offers an initial set of results and analyses of the TBIPBE study, funded by the National Institutes of Health, National Institute on Disability and Rehabilitation Research, and Ontario Neurotrauma Foundation. It is the first practice-based evidence study, to our knowledge, of traumatic brain injury (TBI) rehabilitation. Practicing therapists provided detailed documentation of therapy sessions, and this information was combined with medical record abstracted data, downloaded medication data, surveys of patients’ postdischarge outcomes, and clinician profiles of their An audio podcast accompanies this article. Listen at www.archives-pmr.org.
Supported by the National Institutes of Health, National Center for Medical Rehabilitation Research (grant no. 1R01HD050439-01); National Institute on Disability and Rehabilitation Research (grant no. H133A080023); and Ontario Neurotrauma Foundation (grant no. 2007-ABIISIS-525). The opinions contained in this article are those of the authors and should not be construed as an official statement from the National Institutes of Health, National Center for Medical Rehabilitation Research; National Institute on Disability and Rehabilitation Research; or Ontario Neurotrauma Foundation. Publication of this article was supported by the American Congress of Rehabilitation Medicine. Disclosures: none.
training and experience in rehabilitation, in general, and with TBI specifically. The purpose of this preface is to place this study in the broader context of comparative effectiveness research (CER) and introduce the articles in the supplement, which offer conclusions based on what we believe is the richest dataset on TBI rehabilitation ever assembled.
The Problem The Centers for Disease Control and Prevention (CDC) estimated that in 2010 approximately 2.5 million people in the United States sustained a TBI.1 Of these, 2.2 million had emergency department visits, 280,000 were hospitalized and survived, and 52,000 died.1,2 The Brain Injury Association of America and researchers at the CDC estimated that 3.1 to 5.3 million people in the United States live with long-term disability arising from physical, emotional, or cognitive consequences of TBI.3-5 Although the personal and familial consequences of TBI are devastating, so are the financial implications. In 2010, the CDC estimated that the annual cost to society of TBI for direct medical care, rehabilitation, and indirect costs (eg, lost productivity) totaled >$76.5 billion, with 90% of these costs attributed to severe injury.6
0003-9993/15/$36 - see front matter ª 2015 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2015.03.027
S174
S.D. Horn et al
Inpatient rehabilitation after TBI was started experimentally in the 1970s, but it now is received by growing numbers, not just adolescents and young adults who were the recipients originally, but increasingly by adult and older adult patients. Cuthbert et al7 recently reported that from 2008 to 2010, on average, 20,000 people in the United States aged >15 years received rehabilitation for a primary diagnosis of TBI. Although there have been hundreds of publications on aspects of TBI inpatient rehabilitation, the process is still largely a black box: we do not know what services are being offered, with what results, if they are individually or in combination, and how this might differ by severity of injury, nature and seriousness of patient deficits, or general social, psychological, and health characteristics of patients. Substantial attention has been given to describing etiologies of TBI and characterizing its consequences. Far less effort has been paid to studying interventions, particularly in acute rehabilitation. The 1999 National Institutes of Health consensus statement8 recommended that rehabilitation programs for persons with moderate and severe TBI be interdisciplinary and comprehensive. However, the authors of this statement admitted that scientific evidence for effectiveness of comprehensive rehabilitation was limited to uncontrolled studies and 1 nonrandomized controlled trial.8 A more recent review of TBI studies concluded that more research is needed to identify which interdisciplinary rehabilitation programs are promising practices and provide the greatest improvement in outcomes and which patient subgroups benefit the most from various forms of interdisciplinary rehabilitation.9 Over a decade ago, Chesnut et al conducted a systematic review of evidence available to answer several questions related to interventions for TBI, including whether intensity of acute inpatient rehabilitation is related to outcome.10 They lamented the lack of high-quality research, finding only 1 quasi-experimental comparison of inpatient rehabilitation effectiveness and 4 observational studies of intensity effects.10-14 Chesnut concluded that the single quasi-experimental study of effectiveness suggested that acute rehabilitation may make a difference; however, there was little evidence that therapeutic intensity, as measured by hours of treatment, is related to this beneficial effect. Spontaneous recovery after TBI is known to be significant and confounds conclusions in this area. As a possible explanation for this lack of relation between intensity of treatment and outcomes, Chesnut observed that specific impairments and comorbidities were not taken into account. Results from a practice-based evidence study of stroke rehabilitation that took into account impairments and comorbidities found that hours of treatment per week did not predict outcomes; however, accounting for time spent doing specific activities per week (eg, gait, upper-extremity control, problemsolving) significantly improved prediction of outcomes.15
CER on TBI rehabilitation CER has received much attention recently. The Agency for Healthcare Research and Quality describes CER as:
.designed to inform health-care decisions by providing evidence on the effectiveness, benefits, and harms of different treatment options. The evidence is generated from research studies that compare drugs, medical devices, tests, surgeries, or ways to deliver health care. Comparative effectiveness research requires the development, expansion, and use of a variety of data sources and methods to conduct timely and relevant research and disseminate the results in a form that is quickly usable by clinicians, patients, policymakers, and health plans and other payers.16 The Institute of Medicine has a similar definition.17 Most existing medical research focuses on benefits or harms of new medications or other single-component interventions, using randomized controlled trial (RCT) designs involving an academic researcher (rather than community-active clinicians), a small homogeneous sample of patients, and (often) outcomes that are a proxy for the real-world outcomes in which patients are interested. The results often are not applicable to many patients with the specified condition of interest. These explanatory studies (also known as efficacy studies) differ in a number of very important aspects from CER (known as effectiveness studies).18 Briefly, RCT study patients usually have no or just minor comorbidities in addition to the index disorder being studied, in contrast with many patients being treated by primary care physicians or most specialists. In RCTs, interventions must adhere exactly to the treatment regimen (protocol) or the participant will be considered nonevaluable, in contrast with routine care in which patients are prescribed a treatment regimen but may not follow instructions meticulously. RCTs are powered to test a single (usually short-term) outcome selected as the primary outcome, in contrast with real-world care in which longer-term outcomes and >1 outcome are of interest to physicians and their patients. The result of this highly controlled but narrowly focused research is that patients and clinicians still do not have answers to questions (eg, Does it work for someone with my health issues? Is it any better than the treatment I am currently receiving? What are the long-term effects? What are the side effects and likelihood and seriousness of each side effect?).19 The CER approach addresses these limitations in a number of ways, including conducting systematic reviews and comparing 2 treatments for their effectiveness. Systematic reviews analyze existing research reports and determine whether there is information relevant for treatment decision-making. These reviews, however, may be limited by existing research, mostly RCT-type studies, where only 1 treatment is compared at a time with routine care or a placebo often for a short duration of time using a proxy for the desired outcome. In contrast, CER primary research evaluates multiple treatments for outcomes that are relevant to a more typical, real-world patient population. This approach involves new, prospective data collection or mining of administrative and clinical databases using new statistical techniques (eg, propensity scores, instrumental variables, severity of illness measures) to address selection bias and confounding by lack of comparability of patients who receive different treatments. Practice-based evidence studies are a type of CER that allows comparison of treatment differences that arise naturally in the course of treatment provided in different facilities by diverse professionals.
List of abbreviations: CDC CER RCT TBI TBI-PBE
Centers for Disease Control and Prevention comparative effectiveness research randomized controlled trial traumatic brain injury Traumatic Brain InjuryePractice Based Evidence
Practice-based evidence study methodology for TBI rehabilitation Rehabilitation is an example of a complex intervention with many different components delivered in an individualized manner; www.archives-pmr.org
Preface to Traumatic Brain Injury Supplement therefore, it is impossible to use a single RCT, or even a series of successive RCTs, to determine the optimal timing, duration, content, and combination of all the therapies that may be used. Most previous rehabilitation research on interventions or management strategies has been based on retrospective data abstracted from clinical and administrative records and contains only those data elements that are readily available and exportable from rehabilitation center medical record systems. Data are aggregated to achieve larger samples, as is done in the TBI Model Systems’ National Database.20 However, the medical record often includes only procedure codes used for billing and does not contain measures designed to capture the active ingredients of treatments provided by various disciplines. In addition, the observational research in rehabilitation that has been published commonly lacks information on comparators and often uses weak, incomplete, and/ or untested outcome measures that happen to be available.19 Practice-based evidence research has addressed the shortcomings of previous research by developing a methodology that can handle multiple differences in rehabilitation programs resulting from a large number of therapists, at diverse sites, customizing many aspects of their treatment program, and treating patients with various levels of severity of illness and injury.18 Practicebased evidence studies in rehabilitation (eg, stroke,15 orthopedic joint replacement,21 spinal cord injury22) have the following characteristics19: (1) they are prospective, multisite observational studies that capture practice variations existing both between and within sites; (2) sites enroll large numbers of consecutively admitted patients, which permit analysis of subgroups of patients and cross-validation of findings; (3) treatment data are either abstracted from the medical record or provided by point of care documentation of therapies that are given during each treatment session (additional data may be collected through downloads of, for example, medication information; (4) outcome data are abstracted from the medical record, supplemented with information obtained from patient interviews after various spans of time postdischarge; (5) clinicians (organized into a clinical project team) and former patients are involved in design of the study, including creating the study design, developing the content of point of care documentation forms to describe the care that they provide, and generating content of the follow-up interview; (6) analysis of this comprehensive database is led by the clinicians and former patients who helped design the study and contributed data on rehabilitation treatments administered, with a focus on basic effectiveness questions (eg, which type of treatment, administered for how long, during what phase of the patient’s admission, produces the best outcomes?); and (7) because clinicians and former patients are involved in analysis and reporting, this partially closes the circle of knowledge translation: clinicians and former patients are involved in initial design through final dissemination. Various experts have argued whether practice-based evidence studies can prove efficacy and effectiveness of treatment alternatives, or whether their findings need to be confirmed by an RCT. However one views this question, one conclusion is clear: practice-based evidence studies are superior to RCTs when it comes to external validity (generalizability)23: (1) placebos are replaced by real-life treatments (eg, alternative approaches being used by therapists); (2) narrowly defined patient samples using multiple inclusion/exclusion criteria are replaced by a sample including every patient who consents to be observed (and because there are no required treatment regimens that may pose varying risks, most patients consent); (3) surrogate outcome measures are www.archives-pmr.org
S175 replaced by real-life outcomes (eg, function, participation, longterm use of health-care resources); and (4) to mirror clinical reality, limitations on treatment setting or minimal level of training of therapists are not used in recruiting sites or treatment providers. Therefore, practice-based evidence studies address all the requirements of well-designed CER. Findings have the potential to influence decisions of patients, clinicians, and administrators. Practice-based evidence research is an outcomes-directed, model-building methodology that has established effective and efficient treatments in other diagnostic groups and clinical settings.24,25 Strengths of the practice-based evidence approach are the extensive array of data collected, their level of detail, and as a result, the breadth of issues that can be examined in testing of a priori or post hoc hypotheses. Practice-based evidence research uses detailed descriptions of actual rehabilitation practices to examine the relations among patient characteristics, content of therapy, and their associations with rehabilitation outcomes. The practice-based evidence approach does not disrupt the routines of the treatment setting in the way an RCT does. It offers a naturalistic view of rehabilitation treatment by examining what actually happens in the care process, not altering the treatment regimen to evaluate the efficacy of a particular intervention.18 The practice-based evidence approach also offers the advantage of large numbers of patients that often cannot be attained in an RCT constrained by stringent selection criteria and/or costs involved in identifying, treating, and measuring patients. It controls for patient differences by taking into account important clinical covariates (eg, severity of illness, injury severity, comorbidities, functional status). This inclusiveness allows greater generalizability of findings. In addition, detailed data on interventions allow researchers to focus on the most meaningful level of resolution regarding the types of care rendereddconsistent with current knowledge and insights offered by clinical participants. Therefore, the practicebased evidence approach can answer study questions and hypotheses initially at a basic level, but it also allows drilling down into the data with the help of additional insights offered by the interdisciplinary clinical project team. Practice-based evidence methodology allows important statistical associations to be identified. Although causality cannot firmly and definitely be established, alternate hypotheses regarding possible cause and effect can be tested using the large number of available variables to identify mediating and moderating influences on outcomes. Results of these analyses can be used to eliminate most alternative explanations about causality and to generate specific additional analytic questions. Because of the multivariable methods used in practice-based evidence studies to determine factors significantly associated with outcomes, findings are based on strong statistical association. In the past, when changes in practice were made based on findings of a practice-based evidence study, evaluation of the effects of the change demonstrated the correctness of the practice-based evidence study’s conclusions.24,25 This supplement offers an initial set of results from the TBI-PBE study. Funding by the National Institutes of Health, National Institute on Disability and Rehabilitation Research, and Ontario Neurotrauma Foundation made possible what we believe is the richest dataset on TBI rehabilitation ever assembled. The analyses reported in this supplement only scratch the surface and hopefully will only be the beginning of an extensive analysis and reporting process. The TBI-PBE study enrolled 2205 individuals with TBI receiving initial inpatient rehabilitation at 9 rehabilitation centers across the United States and 1 in Canada over a 2.5-year time
S176 period. Three of the 10 study sites were part of the TBI Model Systems, funded by the National Institute on Disability and Rehabilitation Research. This 6-year study developed treatment taxonomies for each rehabilitation discipline and used paper and web-based electronic data capture systems to document the interventions occurring each time a clinician had a treatment session with a consented patient, or a nurse interacted with or on behalf of the patient. This resulted in >1000 clinicians providing detailed information about hundreds of specific treatment activities in >350,000 encounters/treatment sessions. Extensive medical record abstracting documented patient and injury characteristics, severity of principal and comorbid conditions, and ancillary treatments (eg, medications, nutritional support) to augment information collected on therapy sessions. In addition, in-depth follow-up interviews (with patients or, where appropriate, their proxies) were conducted at 3 and 9 months postdischarge to capture outcomes in the first year postinjury for most participants. This supplement includes 12 articles, starting with an introductory article describing in detail how the practice-based evidence methodology was applied in this investigation of TBI rehabilitation and the measures and methods used in the project. The subsequent articles aim to answer 1 of the following questions: What patient and injury characteristics contributed to patient outcomes? What clinical events occurred in the course of treatment that may have mediated outcomes? What treatments were used in response to what clinical problems? Where various treatment options were available, which one(s) is best, as demonstrated by superior outcomes achieved (taking into account TBI severity and other patient deficits and strengths that may affect outcomes)? We believe the findings will provide guidance in developing guidelines for clinical decision-making and other evidence-based practices.
Keywords Brain injuries; Comparative effectiveness research; Rehabilitation
Corresponding author Susan D. Horn, PhD, University of Utah School of Medicine, Department of Population Health Sciences, Health System Innovation and Research Program, Williams Bldg, Rm 1N461, 295 Chipeta Way, Salt Lake City, Utah 84108. E-mail address: susan. [email protected].
Acknowledgments We thank the clinical and research staff at each of the 10 inpatient rehabilitation facilities represented in the study for their contributions. The study center directors included the following: John D. Corrigan, PhD, and Jennifer Bogner, PhD (Department of Physical Medicine and Rehabilitation, Ohio State University, Columbus, OH); Nora Cullen, MD (Toronto Rehabilitation Institute, Toronto, ON, Canada); Cynthia L. Beaulieu, PhD (Brooks Rehabilitation Hospital, Jacksonville, FL); Flora M. Hammond, MD (Carolinas Rehabilitation, Charlotte, NC [now at Indiana University]); David K. Ryser, MD (Neuro Specialty Rehabilitation Unit, Intermountain Medical Center, Salt Lake City, UT); Murray E. Brandstater, MD (Loma Linda University Medical Center, Loma Linda, CA); Marcel P. Dijkers, PhD (Mount Sinai Medical
S.D. Horn et al Center, New York, NY); William Garmoe, PhD (Medstar National Rehabilitation Hospital, Washington, DC); James A. Young, MD (Physical Medicine and Rehabilitation, Rush University Medical Center, Chicago, IL); and Ronald T. Seel, PhD (Crawford Research Institute, Shepherd Center, Atlanta, GA). We also thank Michael Watkiss for contribution in data collector training and support during data collection and Randall J. Smout for contribution in data cleaning and analysis.
References 1. Centers for Disease Control and Prevention. Traumatic brain injury in the United States: fact sheet. 2014. Available at: http://www.cdc.gov/ TraumaticBrainInjury/get_the_facts.html. Accessed March 6, 2014. 2. Faul M, Xu L, Wald MM, Coronado VG. Traumatic brain injury in the United States: emergency department visits, hospitalizations, and deaths. Atlanta: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control Centers for Disease Control; 2010. 3. Brain Injury Association of America. Brain injury facts: brain injury statistics. 2012. Available at: http://www.biausa.org/ LiteratureRetrieve.aspx?IDZ104992. Accessed March 6, 2014. 4. Thurman D, Alverson C, Dunn K, Guerrero J, Sniezek J. Traumatic brain injury in the United States: a public health perspective. J Head Trauma Rehabil 1999;14:602-15. 5. Selassie AW, Zaloshnja E, Langlois JA, Miller T, Jones P, Steiner C. Incidence of long-term disability following traumatic brain injury hospitalization in the United States, 2003. J Head Trauma Rehabil 2008;23:123-31. 6. Finkelstein E, Corso P, Miller T, editors. The incidence and economic burden of injuries in the United States. New York: Oxford Univ Pr; 2006. 7. Cuthbert JP, Corrigan JD, Whiteneck GG, et al. Extension of the representativeness of the Traumatic Brain Injury Model Systems National Database: 2001 to 2010. J Head Trauma Rehabil 2012;27: E15-27. 8. Consensus conference. Rehabilitation of persons with traumatic brain injury. NIH Consensus Development Panel on Rehabilitation of Persons With Traumatic Brain Injury. JAMA 1999;282:974-82. 9. Brasure M, Lamberty GH, Sayer NA, et al. Multidisciplinary rehabilitation programs for moderate to severe traumatic brain injury in adults: future research needs: identification of future research needs from comparative effectiveness review no. 72 [Internet]. 2013. Available at: http://www.ncbi. nlm.nih.gov/books/NBK143629/. Accessed March 6, 2014. 10. Chesnut R, Carney N, Maynard H, Patterson P, Mann NC, Helfand M. Evidence report: rehabilitation for traumatic brain injury. Rockville: Agency for Health Care Policy and Research; 1999. AHCPR Pub No 99-E005. 11. Aronow HU. Rehabilitation effectiveness with severe brain injury: translating research into policy. J Head Trauma Rehabil 1987;2: 24-36. 12. Heinemann AW, Hamilton B, Linacre JM, Wright BD, Granger C. Functional status and therapeutic intensity during inpatient rehabilitation. Am J Phys Med Rehabil 1995;74:315-26. 13. Blackerby WF. Intensity of rehabilitation and length of stay. Brain Inj 1990;4:167-73. 14. Spivack G, Spettell CM, Ellis DW, Ross SE. Effects of intensity of treatment and length of stay on rehabilitation outcomes. Brain Inj 1992;6:419-34. 15. Horn SD, DeJong G, Smout RJ, Gassaway J, James R, Conroy B. Stroke rehabilitation patients, practice, and outcomes: is earlier and more aggressive therapy better? Arch Phys Med Rehabil 2005;86(12 Suppl 2):S101-14. 16. Agency for Healthcare Research and Quality. What is comparative effectiveness research. Available at: http://www.effectivehealthcare.
www.archives-pmr.org
Preface to Traumatic Brain Injury Supplement
17. 18.
19.
20.
21.
ahrq.gov/index.cfm/what-is-comparative-effectiveness-research1/. Accessed August 31, 2014. Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med 2009;151:203-5. Horn SD, DeJong G, Deutscher D. Practice-based evidence research in rehabilitation: an alternative to randomized clinical trials and traditional observational studies. Arch Phys Med Rehabil 2012; 93(8 Suppl):S127-37. Dijkers MP, Whiteneck GG, Gassaway J. CER, PBE, SCIRehab, NIDRR, and other important abbreviations. Arch Phys Med Rehabil 2013;94(4 Suppl):S61-6. Dijkers MP, Harrison-Felix C, Marwitz JH. The traumatic brain injury model systems: history and contributions to clinical service and research. J Head Trauma Rehabil 2010;25:81-91. DeJong G, Hsieh C, Gassaway J, et al. Characterizing rehabilitation services for patients with knee and hip replacement in skilled nursing
www.archives-pmr.org
S177
22.
23.
24. 25.
facilities and inpatient rehabilitation facilities. Arch Phys Med Rehabil 2009;90:1269-83. Whiteneck GG, Gassaway J. SCIRehab uses practice-based evidence methodology to associate patient and treatment characteristics with outcomes. Arch Phys Med Rehabil 2013;94(4 Suppl): S67-74. Dijkers MP. External validity in research on rehabilitation interventions: issues for knowledge translation. Austin: SEDL, National Center for the Dissemination of Disability Research; 2011. Focus Technical Brief 33. Hudak S, Sharkey S, Engleman M, Horn SD, Spector W, Limcangco R. Pressure ulcer plan is working. Provider 2008;34:34-9. Olsho LE, Spector WD, Williams CS, Rhodes W, Fink RV, Limcangco R, Hurd D. Evaluation of AHRQ’s on-time pressure ulcer prevention program: a facilitator-assisted clinical decision support intervention for nursing homes. Med Care 2014;52:258-66.
INTRODUCTION
Traumatic Brain Injury Rehabilitation Comparative Effectiveness Research: Introduction to the Traumatic Brain InjuryePractice Based Evidence Archives Supplement Susan D. Horn, PhD,a John D. Corrigan, PhD,b Marcel P. Dijkers, PhDc From the aInstitute for Clinical Outcomes Research, International Severity Information Systems, Salt Lake City, UT; bDepartment of Physical Medicine and Rehabilitation, Ohio State University, Columbus, OH; and cDepartment of Rehabilitation Medicine, Icahn School of Medicine at Mount Sinai, New York, NY. Current affiliation for Horn, Department of Population Health Sciences, University of Utah School of Medicine, Salt Lake City, UT.
Abstract This supplement of the Archives of Physical Medicine and Rehabilitation is devoted to the Traumatic Brain InjuryePractice Based Evidence study, the first practice-based evidence study, to our knowledge, of traumatic brain injury rehabilitation. The purpose of this preface is to place this study in the broader context of comparative effectiveness research and introduce the articles in the supplement. Archives of Physical Medicine and Rehabilitation 2015;96(8 Suppl 3):S173-7 ª 2015 by the American Congress of Rehabilitation Medicine
This supplement of the Archives of Physical Medicine and Rehabilitation offers an initial set of results and analyses of the TBIPBE study, funded by the National Institutes of Health, National Institute on Disability and Rehabilitation Research, and Ontario Neurotrauma Foundation. It is the first practice-based evidence study, to our knowledge, of traumatic brain injury (TBI) rehabilitation. Practicing therapists provided detailed documentation of therapy sessions, and this information was combined with medical record abstracted data, downloaded medication data, surveys of patients’ postdischarge outcomes, and clinician profiles of their An audio podcast accompanies this article. Listen at www.archives-pmr.org.
Supported by the National Institutes of Health, National Center for Medical Rehabilitation Research (grant no. 1R01HD050439-01); National Institute on Disability and Rehabilitation Research (grant no. H133A080023); and Ontario Neurotrauma Foundation (grant no. 2007-ABIISIS-525). The opinions contained in this article are those of the authors and should not be construed as an official statement from the National Institutes of Health, National Center for Medical Rehabilitation Research; National Institute on Disability and Rehabilitation Research; or Ontario Neurotrauma Foundation. Publication of this article was supported by the American Congress of Rehabilitation Medicine. Disclosures: none.
training and experience in rehabilitation, in general, and with TBI specifically. The purpose of this preface is to place this study in the broader context of comparative effectiveness research (CER) and introduce the articles in the supplement, which offer conclusions based on what we believe is the richest dataset on TBI rehabilitation ever assembled.
The Problem The Centers for Disease Control and Prevention (CDC) estimated that in 2010 approximately 2.5 million people in the United States sustained a TBI.1 Of these, 2.2 million had emergency department visits, 280,000 were hospitalized and survived, and 52,000 died.1,2 The Brain Injury Association of America and researchers at the CDC estimated that 3.1 to 5.3 million people in the United States live with long-term disability arising from physical, emotional, or cognitive consequences of TBI.3-5 Although the personal and familial consequences of TBI are devastating, so are the financial implications. In 2010, the CDC estimated that the annual cost to society of TBI for direct medical care, rehabilitation, and indirect costs (eg, lost productivity) totaled >$76.5 billion, with 90% of these costs attributed to severe injury.6
0003-9993/15/$36 - see front matter ª 2015 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2015.03.027
S174
S.D. Horn et al
Inpatient rehabilitation after TBI was started experimentally in the 1970s, but it now is received by growing numbers, not just adolescents and young adults who were the recipients originally, but increasingly by adult and older adult patients. Cuthbert et al7 recently reported that from 2008 to 2010, on average, 20,000 people in the United States aged >15 years received rehabilitation for a primary diagnosis of TBI. Although there have been hundreds of publications on aspects of TBI inpatient rehabilitation, the process is still largely a black box: we do not know what services are being offered, with what results, if they are individually or in combination, and how this might differ by severity of injury, nature and seriousness of patient deficits, or general social, psychological, and health characteristics of patients. Substantial attention has been given to describing etiologies of TBI and characterizing its consequences. Far less effort has been paid to studying interventions, particularly in acute rehabilitation. The 1999 National Institutes of Health consensus statement8 recommended that rehabilitation programs for persons with moderate and severe TBI be interdisciplinary and comprehensive. However, the authors of this statement admitted that scientific evidence for effectiveness of comprehensive rehabilitation was limited to uncontrolled studies and 1 nonrandomized controlled trial.8 A more recent review of TBI studies concluded that more research is needed to identify which interdisciplinary rehabilitation programs are promising practices and provide the greatest improvement in outcomes and which patient subgroups benefit the most from various forms of interdisciplinary rehabilitation.9 Over a decade ago, Chesnut et al conducted a systematic review of evidence available to answer several questions related to interventions for TBI, including whether intensity of acute inpatient rehabilitation is related to outcome.10 They lamented the lack of high-quality research, finding only 1 quasi-experimental comparison of inpatient rehabilitation effectiveness and 4 observational studies of intensity effects.10-14 Chesnut concluded that the single quasi-experimental study of effectiveness suggested that acute rehabilitation may make a difference; however, there was little evidence that therapeutic intensity, as measured by hours of treatment, is related to this beneficial effect. Spontaneous recovery after TBI is known to be significant and confounds conclusions in this area. As a possible explanation for this lack of relation between intensity of treatment and outcomes, Chesnut observed that specific impairments and comorbidities were not taken into account. Results from a practice-based evidence study of stroke rehabilitation that took into account impairments and comorbidities found that hours of treatment per week did not predict outcomes; however, accounting for time spent doing specific activities per week (eg, gait, upper-extremity control, problemsolving) significantly improved prediction of outcomes.15
CER on TBI rehabilitation CER has received much attention recently. The Agency for Healthcare Research and Quality describes CER as:
.designed to inform health-care decisions by providing evidence on the effectiveness, benefits, and harms of different treatment options. The evidence is generated from research studies that compare drugs, medical devices, tests, surgeries, or ways to deliver health care. Comparative effectiveness research requires the development, expansion, and use of a variety of data sources and methods to conduct timely and relevant research and disseminate the results in a form that is quickly usable by clinicians, patients, policymakers, and health plans and other payers.16 The Institute of Medicine has a similar definition.17 Most existing medical research focuses on benefits or harms of new medications or other single-component interventions, using randomized controlled trial (RCT) designs involving an academic researcher (rather than community-active clinicians), a small homogeneous sample of patients, and (often) outcomes that are a proxy for the real-world outcomes in which patients are interested. The results often are not applicable to many patients with the specified condition of interest. These explanatory studies (also known as efficacy studies) differ in a number of very important aspects from CER (known as effectiveness studies).18 Briefly, RCT study patients usually have no or just minor comorbidities in addition to the index disorder being studied, in contrast with many patients being treated by primary care physicians or most specialists. In RCTs, interventions must adhere exactly to the treatment regimen (protocol) or the participant will be considered nonevaluable, in contrast with routine care in which patients are prescribed a treatment regimen but may not follow instructions meticulously. RCTs are powered to test a single (usually short-term) outcome selected as the primary outcome, in contrast with real-world care in which longer-term outcomes and >1 outcome are of interest to physicians and their patients. The result of this highly controlled but narrowly focused research is that patients and clinicians still do not have answers to questions (eg, Does it work for someone with my health issues? Is it any better than the treatment I am currently receiving? What are the long-term effects? What are the side effects and likelihood and seriousness of each side effect?).19 The CER approach addresses these limitations in a number of ways, including conducting systematic reviews and comparing 2 treatments for their effectiveness. Systematic reviews analyze existing research reports and determine whether there is information relevant for treatment decision-making. These reviews, however, may be limited by existing research, mostly RCT-type studies, where only 1 treatment is compared at a time with routine care or a placebo often for a short duration of time using a proxy for the desired outcome. In contrast, CER primary research evaluates multiple treatments for outcomes that are relevant to a more typical, real-world patient population. This approach involves new, prospective data collection or mining of administrative and clinical databases using new statistical techniques (eg, propensity scores, instrumental variables, severity of illness measures) to address selection bias and confounding by lack of comparability of patients who receive different treatments. Practice-based evidence studies are a type of CER that allows comparison of treatment differences that arise naturally in the course of treatment provided in different facilities by diverse professionals.
List of abbreviations: CDC CER RCT TBI TBI-PBE
Centers for Disease Control and Prevention comparative effectiveness research randomized controlled trial traumatic brain injury Traumatic Brain InjuryePractice Based Evidence
Practice-based evidence study methodology for TBI rehabilitation Rehabilitation is an example of a complex intervention with many different components delivered in an individualized manner; www.archives-pmr.org
Preface to Traumatic Brain Injury Supplement therefore, it is impossible to use a single RCT, or even a series of successive RCTs, to determine the optimal timing, duration, content, and combination of all the therapies that may be used. Most previous rehabilitation research on interventions or management strategies has been based on retrospective data abstracted from clinical and administrative records and contains only those data elements that are readily available and exportable from rehabilitation center medical record systems. Data are aggregated to achieve larger samples, as is done in the TBI Model Systems’ National Database.20 However, the medical record often includes only procedure codes used for billing and does not contain measures designed to capture the active ingredients of treatments provided by various disciplines. In addition, the observational research in rehabilitation that has been published commonly lacks information on comparators and often uses weak, incomplete, and/ or untested outcome measures that happen to be available.19 Practice-based evidence research has addressed the shortcomings of previous research by developing a methodology that can handle multiple differences in rehabilitation programs resulting from a large number of therapists, at diverse sites, customizing many aspects of their treatment program, and treating patients with various levels of severity of illness and injury.18 Practicebased evidence studies in rehabilitation (eg, stroke,15 orthopedic joint replacement,21 spinal cord injury22) have the following characteristics19: (1) they are prospective, multisite observational studies that capture practice variations existing both between and within sites; (2) sites enroll large numbers of consecutively admitted patients, which permit analysis of subgroups of patients and cross-validation of findings; (3) treatment data are either abstracted from the medical record or provided by point of care documentation of therapies that are given during each treatment session (additional data may be collected through downloads of, for example, medication information; (4) outcome data are abstracted from the medical record, supplemented with information obtained from patient interviews after various spans of time postdischarge; (5) clinicians (organized into a clinical project team) and former patients are involved in design of the study, including creating the study design, developing the content of point of care documentation forms to describe the care that they provide, and generating content of the follow-up interview; (6) analysis of this comprehensive database is led by the clinicians and former patients who helped design the study and contributed data on rehabilitation treatments administered, with a focus on basic effectiveness questions (eg, which type of treatment, administered for how long, during what phase of the patient’s admission, produces the best outcomes?); and (7) because clinicians and former patients are involved in analysis and reporting, this partially closes the circle of knowledge translation: clinicians and former patients are involved in initial design through final dissemination. Various experts have argued whether practice-based evidence studies can prove efficacy and effectiveness of treatment alternatives, or whether their findings need to be confirmed by an RCT. However one views this question, one conclusion is clear: practice-based evidence studies are superior to RCTs when it comes to external validity (generalizability)23: (1) placebos are replaced by real-life treatments (eg, alternative approaches being used by therapists); (2) narrowly defined patient samples using multiple inclusion/exclusion criteria are replaced by a sample including every patient who consents to be observed (and because there are no required treatment regimens that may pose varying risks, most patients consent); (3) surrogate outcome measures are www.archives-pmr.org
S175 replaced by real-life outcomes (eg, function, participation, longterm use of health-care resources); and (4) to mirror clinical reality, limitations on treatment setting or minimal level of training of therapists are not used in recruiting sites or treatment providers. Therefore, practice-based evidence studies address all the requirements of well-designed CER. Findings have the potential to influence decisions of patients, clinicians, and administrators. Practice-based evidence research is an outcomes-directed, model-building methodology that has established effective and efficient treatments in other diagnostic groups and clinical settings.24,25 Strengths of the practice-based evidence approach are the extensive array of data collected, their level of detail, and as a result, the breadth of issues that can be examined in testing of a priori or post hoc hypotheses. Practice-based evidence research uses detailed descriptions of actual rehabilitation practices to examine the relations among patient characteristics, content of therapy, and their associations with rehabilitation outcomes. The practice-based evidence approach does not disrupt the routines of the treatment setting in the way an RCT does. It offers a naturalistic view of rehabilitation treatment by examining what actually happens in the care process, not altering the treatment regimen to evaluate the efficacy of a particular intervention.18 The practice-based evidence approach also offers the advantage of large numbers of patients that often cannot be attained in an RCT constrained by stringent selection criteria and/or costs involved in identifying, treating, and measuring patients. It controls for patient differences by taking into account important clinical covariates (eg, severity of illness, injury severity, comorbidities, functional status). This inclusiveness allows greater generalizability of findings. In addition, detailed data on interventions allow researchers to focus on the most meaningful level of resolution regarding the types of care rendereddconsistent with current knowledge and insights offered by clinical participants. Therefore, the practicebased evidence approach can answer study questions and hypotheses initially at a basic level, but it also allows drilling down into the data with the help of additional insights offered by the interdisciplinary clinical project team. Practice-based evidence methodology allows important statistical associations to be identified. Although causality cannot firmly and definitely be established, alternate hypotheses regarding possible cause and effect can be tested using the large number of available variables to identify mediating and moderating influences on outcomes. Results of these analyses can be used to eliminate most alternative explanations about causality and to generate specific additional analytic questions. Because of the multivariable methods used in practice-based evidence studies to determine factors significantly associated with outcomes, findings are based on strong statistical association. In the past, when changes in practice were made based on findings of a practice-based evidence study, evaluation of the effects of the change demonstrated the correctness of the practice-based evidence study’s conclusions.24,25 This supplement offers an initial set of results from the TBI-PBE study. Funding by the National Institutes of Health, National Institute on Disability and Rehabilitation Research, and Ontario Neurotrauma Foundation made possible what we believe is the richest dataset on TBI rehabilitation ever assembled. The analyses reported in this supplement only scratch the surface and hopefully will only be the beginning of an extensive analysis and reporting process. The TBI-PBE study enrolled 2205 individuals with TBI receiving initial inpatient rehabilitation at 9 rehabilitation centers across the United States and 1 in Canada over a 2.5-year time
S176 period. Three of the 10 study sites were part of the TBI Model Systems, funded by the National Institute on Disability and Rehabilitation Research. This 6-year study developed treatment taxonomies for each rehabilitation discipline and used paper and web-based electronic data capture systems to document the interventions occurring each time a clinician had a treatment session with a consented patient, or a nurse interacted with or on behalf of the patient. This resulted in >1000 clinicians providing detailed information about hundreds of specific treatment activities in >350,000 encounters/treatment sessions. Extensive medical record abstracting documented patient and injury characteristics, severity of principal and comorbid conditions, and ancillary treatments (eg, medications, nutritional support) to augment information collected on therapy sessions. In addition, in-depth follow-up interviews (with patients or, where appropriate, their proxies) were conducted at 3 and 9 months postdischarge to capture outcomes in the first year postinjury for most participants. This supplement includes 12 articles, starting with an introductory article describing in detail how the practice-based evidence methodology was applied in this investigation of TBI rehabilitation and the measures and methods used in the project. The subsequent articles aim to answer 1 of the following questions: What patient and injury characteristics contributed to patient outcomes? What clinical events occurred in the course of treatment that may have mediated outcomes? What treatments were used in response to what clinical problems? Where various treatment options were available, which one(s) is best, as demonstrated by superior outcomes achieved (taking into account TBI severity and other patient deficits and strengths that may affect outcomes)? We believe the findings will provide guidance in developing guidelines for clinical decision-making and other evidence-based practices.
Keywords Brain injuries; Comparative effectiveness research; Rehabilitation
Corresponding author Susan D. Horn, PhD, University of Utah School of Medicine, Department of Population Health Sciences, Health System Innovation and Research Program, Williams Bldg, Rm 1N461, 295 Chipeta Way, Salt Lake City, Utah 84108. E-mail address: susan. [email protected].
Acknowledgments We thank the clinical and research staff at each of the 10 inpatient rehabilitation facilities represented in the study for their contributions. The study center directors included the following: John D. Corrigan, PhD, and Jennifer Bogner, PhD (Department of Physical Medicine and Rehabilitation, Ohio State University, Columbus, OH); Nora Cullen, MD (Toronto Rehabilitation Institute, Toronto, ON, Canada); Cynthia L. Beaulieu, PhD (Brooks Rehabilitation Hospital, Jacksonville, FL); Flora M. Hammond, MD (Carolinas Rehabilitation, Charlotte, NC [now at Indiana University]); David K. Ryser, MD (Neuro Specialty Rehabilitation Unit, Intermountain Medical Center, Salt Lake City, UT); Murray E. Brandstater, MD (Loma Linda University Medical Center, Loma Linda, CA); Marcel P. Dijkers, PhD (Mount Sinai Medical
S.D. Horn et al Center, New York, NY); William Garmoe, PhD (Medstar National Rehabilitation Hospital, Washington, DC); James A. Young, MD (Physical Medicine and Rehabilitation, Rush University Medical Center, Chicago, IL); and Ronald T. Seel, PhD (Crawford Research Institute, Shepherd Center, Atlanta, GA). We also thank Michael Watkiss for contribution in data collector training and support during data collection and Randall J. Smout for contribution in data cleaning and analysis.
References 1. Centers for Disease Control and Prevention. Traumatic brain injury in the United States: fact sheet. 2014. Available at: http://www.cdc.gov/ TraumaticBrainInjury/get_the_facts.html. Accessed March 6, 2014. 2. Faul M, Xu L, Wald MM, Coronado VG. Traumatic brain injury in the United States: emergency department visits, hospitalizations, and deaths. Atlanta: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control Centers for Disease Control; 2010. 3. Brain Injury Association of America. Brain injury facts: brain injury statistics. 2012. Available at: http://www.biausa.org/ LiteratureRetrieve.aspx?IDZ104992. Accessed March 6, 2014. 4. Thurman D, Alverson C, Dunn K, Guerrero J, Sniezek J. Traumatic brain injury in the United States: a public health perspective. J Head Trauma Rehabil 1999;14:602-15. 5. Selassie AW, Zaloshnja E, Langlois JA, Miller T, Jones P, Steiner C. Incidence of long-term disability following traumatic brain injury hospitalization in the United States, 2003. J Head Trauma Rehabil 2008;23:123-31. 6. Finkelstein E, Corso P, Miller T, editors. The incidence and economic burden of injuries in the United States. New York: Oxford Univ Pr; 2006. 7. Cuthbert JP, Corrigan JD, Whiteneck GG, et al. Extension of the representativeness of the Traumatic Brain Injury Model Systems National Database: 2001 to 2010. J Head Trauma Rehabil 2012;27: E15-27. 8. Consensus conference. Rehabilitation of persons with traumatic brain injury. NIH Consensus Development Panel on Rehabilitation of Persons With Traumatic Brain Injury. JAMA 1999;282:974-82. 9. Brasure M, Lamberty GH, Sayer NA, et al. Multidisciplinary rehabilitation programs for moderate to severe traumatic brain injury in adults: future research needs: identification of future research needs from comparative effectiveness review no. 72 [Internet]. 2013. Available at: http://www.ncbi. nlm.nih.gov/books/NBK143629/. Accessed March 6, 2014. 10. Chesnut R, Carney N, Maynard H, Patterson P, Mann NC, Helfand M. Evidence report: rehabilitation for traumatic brain injury. Rockville: Agency for Health Care Policy and Research; 1999. AHCPR Pub No 99-E005. 11. Aronow HU. Rehabilitation effectiveness with severe brain injury: translating research into policy. J Head Trauma Rehabil 1987;2: 24-36. 12. Heinemann AW, Hamilton B, Linacre JM, Wright BD, Granger C. Functional status and therapeutic intensity during inpatient rehabilitation. Am J Phys Med Rehabil 1995;74:315-26. 13. Blackerby WF. Intensity of rehabilitation and length of stay. Brain Inj 1990;4:167-73. 14. Spivack G, Spettell CM, Ellis DW, Ross SE. Effects of intensity of treatment and length of stay on rehabilitation outcomes. Brain Inj 1992;6:419-34. 15. Horn SD, DeJong G, Smout RJ, Gassaway J, James R, Conroy B. Stroke rehabilitation patients, practice, and outcomes: is earlier and more aggressive therapy better? Arch Phys Med Rehabil 2005;86(12 Suppl 2):S101-14. 16. Agency for Healthcare Research and Quality. What is comparative effectiveness research. Available at: http://www.effectivehealthcare.
www.archives-pmr.org
Preface to Traumatic Brain Injury Supplement
17. 18.
19.
20.
21.
ahrq.gov/index.cfm/what-is-comparative-effectiveness-research1/. Accessed August 31, 2014. Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med 2009;151:203-5. Horn SD, DeJong G, Deutscher D. Practice-based evidence research in rehabilitation: an alternative to randomized clinical trials and traditional observational studies. Arch Phys Med Rehabil 2012; 93(8 Suppl):S127-37. Dijkers MP, Whiteneck GG, Gassaway J. CER, PBE, SCIRehab, NIDRR, and other important abbreviations. Arch Phys Med Rehabil 2013;94(4 Suppl):S61-6. Dijkers MP, Harrison-Felix C, Marwitz JH. The traumatic brain injury model systems: history and contributions to clinical service and research. J Head Trauma Rehabil 2010;25:81-91. DeJong G, Hsieh C, Gassaway J, et al. Characterizing rehabilitation services for patients with knee and hip replacement in skilled nursing
www.archives-pmr.org
S177
22.
23.
24. 25.
facilities and inpatient rehabilitation facilities. Arch Phys Med Rehabil 2009;90:1269-83. Whiteneck GG, Gassaway J. SCIRehab uses practice-based evidence methodology to associate patient and treatment characteristics with outcomes. Arch Phys Med Rehabil 2013;94(4 Suppl): S67-74. Dijkers MP. External validity in research on rehabilitation interventions: issues for knowledge translation. Austin: SEDL, National Center for the Dissemination of Disability Research; 2011. Focus Technical Brief 33. Hudak S, Sharkey S, Engleman M, Horn SD, Spector W, Limcangco R. Pressure ulcer plan is working. Provider 2008;34:34-9. Olsho LE, Spector WD, Williams CS, Rhodes W, Fink RV, Limcangco R, Hurd D. Evaluation of AHRQ’s on-time pressure ulcer prevention program: a facilitator-assisted clinical decision support intervention for nursing homes. Med Care 2014;52:258-66.