- Email: [email protected]
Predictors of Follow-Up Completeness in Longitudinal Research on Traumatic Brain Injury: Findings From the National Institute on Disability and Rehabilitation Research Traumatic Brain Injury Model Systems Program
Archives of Physical Medicine and Rehabilitation journal homepage: www.archives-pmr.org Archives of Physical Medicine and Rehabilitation 2014;95:633-41
ORIGINAL ARTICLE
Predictors of Follow-Up Completeness in Longitudinal Research on Traumatic Brain Injury: Findings From the National Institute on Disability and Rehabilitation Research Traumatic Brain Injury Model Systems Program Jason W. Krellman, PhD,a Stephanie A. Kolakowsky-Hayner, PhD,b Lisa Spielman, PhD,a Marcel Dijkers, PhD,a Flora M. Hammond, MD,c Jennifer Bogner, PhD,d Tessa Hart, PhD,e Joshua B. Cantor, PhD,a Theodore Tsaousides, PhDa From the aDepartment of Rehabilitation Medicine, Icahn School of Medicine at Mount Sinai, New York, NY; bSanta Clara Valley Medical Center, Rehabilitation Research Center, San Jose, CA; cIndiana University School of Medicine, Indianapolis, IN; dDepartment of Physical Medicine and Rehabilitation, Ohio State University, Columbus, OH; and eMoss Rehabilitation Research Institute, Elkins Park, PA.
Abstract Objective: To identify baseline participant variables in the domains of demographics, medical/psychosocial history, injury characteristics, and postinjury functional status associated with longitudinal follow-up completeness in persons with traumatic brain injury (TBI) using the TBI Model Systems (TBIMS) National Database (NDB). Design: Exhaustive chi-square automatic interaction detection was used to identify factors that classified participants according to level of followup completeness. Setting: Retrospective analysis of a multi-center longitudinal database. Participants: Individuals (NZ8249) enrolled in the TBIMS NDB between 1989 and 2009 who were eligible for at least the first (year 1) followup up to the fifth (year 15) follow-up. Interventions: None. Main Outcome Measures: Follow-up completeness as defined by 6 different longitudinal response patterns (LRPs): completing all follow-ups, wave nonresponse, dropping out, completing no follow-ups without formally withdrawing, formally withdrawing before completing any followups, and formally withdrawing after completing some follow-ups. Results: Completing all follow-ups was associated with higher levels of education, living with parents or others, and having acute care payer data entered in the NDB. Subgroups more vulnerable to loss to follow-up (LTFU) included those with less education, racial/ethnic minority backgrounds, those with better motor functioning on rehabilitation discharge, and those for whom baseline data on education, employment, and acute care payer were not collected. No subgroups were found to be more likely to have the LRPs of dropping out or formal withdrawal. Conclusions: These data identify subgroups in which retention strategies beyond those most commonly used might reduce LTFU in longitudinal studies of persons with TBI, such as the TBIMS, and suggest future investigations into factors associated with missing baseline data. Archives of Physical Medicine and Rehabilitation 2014;95:633-41 ª 2014 by the American Congress of Rehabilitation Medicine
Supported by the National Institute on Disability and Rehabilitation Research, U.S. Department of Education (grants nos. H133P050004, H133B980013, and H133A070038 [Mt. Sinai]; H133A070038 [Santa Clara]; H133A120035 and H133A070042 [Indiana University]; H133A070029 and H133A120086 [Ohio State], and H133A070040 and H133A120037 [Moss]) and the Centers for Disease Control and Prevention (grant no. 1R49CE001171-01 [Mt. Sinai]). No commercial party having a direct financial interest in the results of the research supporting this article has conferred or will confer a benefit on the authors or on any organization with which the authors are associated.
Longitudinal studies are vital to understanding long-term outcomes after traumatic brain injury (TBI). However, 33% to 50% of participants in these studies have historically been lost to follow-up (LTFU).1,2 Participants who are lost nearly always differ systematically from those retained,1,2 which threatens both the internal and external validity of longitudinal outcome studies3 and promotes
0003-9993/14/$36 - see front matter ª 2014 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2013.10.016
634
J.W. Krellman et al
biased conclusions about outcomes after TBI. Indeed, factors generally predictive of LTFU (eg, low socioeconomic status [SES]) often adversely affect outcomes,4 which suggests that systematic LTFU might yield outcome data that are positively biased. Such bias can be reduced by preventing LTFU (eg, by increasing the frequency of participant contact or using sophisticated tracking systems5,6) or by using statistical methods that compensate for loss after it has occurred. However, statistical procedures cannot always reliably estimate or eliminate the impact of LTFU on outcome data.7 Prevention techniques are, therefore, more desirable1 but require what Greenland8 termed subject matter knowledge, such as data regarding which participants are more vulnerable to LTFU. Relatively few investigations have focused on factors predictive of LTFU in longitudinal studies of individuals with TBI. A study by Corrigan et al1 suggested that those with a history of substance abuse, with higher motor function at rehabilitation discharge, and from “socioeconomically disadvantaged groups” (racial and ethnic minorities and those who had little education, were unemployed at time of injury, or were dependent on public funding for acute care) were more frequently lost at 1 and 2 year postinjury follow-ups in 3 large longitudinal studies. Of the 3 datasets analyzed by Corrigan, the TBI Model Systems (TBIMS) National Database (NDB) includes the largest number of participants, is the longest running, and has provided the basis for the most analyses, yielding approximately 350 published studies thus far.9 The NDB has experienced LTFU since its inception, as do all longitudinal studies, and instituted targets for follow-up completeness in 2006.10 However, between 17% and 23% of participants in the NDB are still lost depending on the specific follow-up point.11 The goal of the current study was to determine participantrelated factors predictive of LTFU in the longitudinal study of persons with TBI using the current TBIMS NDB. Previous work in this area has sought to identify factors associated with either response or nonresponse at 1 or 2 follow-ups, but we aimed to identify factors associated with a number of different longitudinal response patterns (LRPs) (eg, completing all follow-ups, skipping some follow-ups) up to a maximum of 5 follow-ups. This approach allowed us to investigate factors associated with LTFU across the more lengthy sequence of follow-ups needed to best characterize long-term outcomes, thus enhancing the potential usefulness of our findings. The NDB was ideal for our purposes because of the large number of available participants, volume of data collected, and number of follow-up data points available, all of which suggest that findings in the NDB would be applicable to many other longitudinal studies of persons with TBI and could inform followup practices to maximize participant retention. In addition, there is value in identifying factors associated with LTFU in the NDB for its own sake because of its significant contribution to the outcomes
List of abbreviations: CHAID HS LRP LTFU NDB PTA SES TBI TBIMS
chi-square automatic interaction detection high school longitudinal response pattern loss to follow-up National Database posttraumatic amnesia socioeconomic status traumatic brain injury Traumatic Brain Injury Model Systems
literature, evidence of selective LTFU1,12 in the database, and the association between previously established predictors of LTFU in the NDB and outcomes.13-15 The overarching goal of the present study was to determine whether various LRPs could be predicted early in study participation and to identify participant characteristics that might signal the need for implementation of specific longitudinal retention strategies. Ideally, these strategies would maximize retention and, therefore, enhance the validity and applicability of longitudinal studies of persons with TBI.
Methods TBI Model Systems An extensive description of the TBIMS and its NDB is reported in Dijkers et al.10 Further information regarding inclusion criteria and data included in the NDB is available on the TBMIS National Data and Statistics Center website.11 Briefly, participants must have sustained a moderate-severe TBI as evidenced by a Glasgow Coma Scale score of <13 in the emergency department, loss of consciousness >30 minutes, posttraumatic amnesia (PTA) >24 hours, and/or structural abnormalities indicative of TBI on neuroimaging; must be 16 years old; and must have received acute care and rehabilitation from a participating medical center. Informed consent is obtained from the participant or, if unable, a family member or legal guardian. Data collected just prior to rehabilitation discharge (baseline) include case mix, acute care, and rehabilitation information obtained through review of emergency and acute care medical records and an interview with the participant or proxy if the participant is unable to respond reliably. Review of rehabilitation records is used to calculate scores on functional status measures (eg, FIM16 and Disability Rating Scale17). Follow-up data on outcomes are currently collected 1, 2, 5, and 10 years postinjury and every fifth year thereafter. The follow-up survey is completed in-person or via telephone or mail with the participant or a proxy if the participant is unable to respond.
Analyzed sample At the time of our analyses, the NDB included 9310 participants enrolled at 22 centers that are currently (16 centers) or previously (6 centers) participating in the TBIMS. Participants whose LRP, which will be subsequently discussed, could not be determined from the data were excluded from the analysis. These included participants whose first (year 1) follow-up was not yet due (nZ589) and those incarcerated during 1 follow-ups (nZ251) because incarcerated participants are not contacted. Participants who died before completing any follow-ups (nZ207) were also excluded. Only 14 participants were eligible for follow-up at year 20 and were also excluded. The sample analyzed included all individuals enrolled in the TBIMS between 1989 and 2009 and who were eligible for at least the first (year 1) follow-up up to the fifth (year 15) follow-up (NZ8249).
Longitudinal response pattern LRP was the outcome variable in our analyses. Table 1 contains numbers and examples of each LRP included as a possible www.archives-pmr.org
Follow-up in traumatic brain injury research
635
outcome. Based on the LRP, participants were classified as 1 of 6 kinds of responders: (1) completers who completed all follow-ups; (2) wave nonresponders who completed follow-up(s) but also skipped follow-up(s); (3) dropouts who completed follow-up(s) but then did not complete subsequent follow-up(s) without formally withdrawing; (4) late withdrawers who formally withdrew after completing 1 follow-ups; (5) withdrawers throughout who formally withdrew without completing any follow-ups; and (6) nonresponders who did not complete any follow-ups but did not formally withdraw. Follow-ups scheduled during periods when a TBIMS center was defunded were not considered when determining participants’ LRP because follow-up contact is not attempted during such periods. The number of individuals affected by defunding was small (a minimum of 0.5% of those eligible for a year 15 follow-up and a maximum of 8% of those eligible for a year 5 follow-up). Indeed, preliminary analyses did not suggest an association between whether a participant was affected by defunding in the past and their LRP.
Predictors Selected baseline variables were recoded based on the distribution of participants within subgroups or on categories used in previous analyses of the NDB18 and then used as candidate predictors of LRPs. Predictors fell into 4 broad categories: (1) demographics, including acute care payer (public/private/other), age, marital status (married/unmarried), race (white/black/other), sex, and education at injury (
Table 1
LRPs in the TBIMS NDB with examples Follow-Up Status (year no.)
Response Pattern
n
%
1
2
5
10
15
Completed all Wave nonresponse* Dropout* Late withdrawal* Withdrawal throughout Nonresponse
5588 1023 651 175 113 699
67.7 12.4 7.9 2.1 1.4 8.5
O O O O W X
O X O O W X
O X X W W X
O O X W W X
O O X W W X
Abbreviations: W, participant withdrew; X, follow-up not completed; O, follow-up completed. * The pattern of these LRPs can vary among participants. The examples provided reflect one possible pattern.
www.archives-pmr.org
psychiatric and substance abuse populations19,20 and level of education in persons with spinal cord injury21) whereas others are conceptually similar to variables previously associated with LTFU. For example, condition-specific indicators of severity have been associated with LTFU in non-TBI populations (eg, immunologic status in human immunodeficiency virus/acquired immunodeficiency syndrome22 and length of hospital stay in burn injury23); therefore, we included duration of PTA, an indicator of TBI severity,24 as a candidate predictor.
Statistical procedures Statistical analyses were conducted using SPSS version 19.0 with SPSS Answer Tree 3.0 software.a Exhaustive chi-square automatic interaction detection (CHAID)25 was used to identify characteristics that classified participants as having a specific LRP. Exhaustive CHAID involves repeatedly merging, splitting, and stopping categories of predictor variables to create a decision tree that can identify group membership based on the relative association between each significant predictor and outcome. Continuous variables are first transformed into categorical variables by dividing continuous distributions into categories with a roughly equal number of observations. Categories of predictors that are least significantly different with respect to the dependent variable are then merged. If certain categories are not statistically different, those categories are merged and the step repeated until statistically different categories of each predictor are determined. This exhaustive repetition is an improvement over the original version of CHAID, which was developed by Biggs et al,26 wherein merging ends when all remaining categories are statistically different, which can result in misidentification of the optimal split for a variable. Chi-square tests are then performed to determine the relation between each predictor and outcome, and a Bonferroni-adjusted P value is calculated for each predictor, which yields a classification tree organized according to the degree of association between predictor and outcome (eg, the most significant predictor of the outcome represents the first split in the data, and the least significant represents the last split). Stopping occurs when statistically different categories of the least significant predictor are determined. At that point, no further splits are performed, which results in a terminal node of the classification tree. The CHAID technique was chosen because the analysis is designed to determine variables that identify group membership; allows both predictors and the dependent variable to be nominal, ordinal, or interval; and because predictor variables need not be measured on the same scale (eg, nominal or ordinal). CHAID considers missing data to be a valid category (eg, sex: male/ female/missing); therefore, the absence of a value for a particular variable can be considered a value in itself.
Results For categorical criterion variables, such as the LRP, the risk estimate is the proportion of cases in the sample incorrectly classified or the error rate. The overall correct classification of the CHAID decision tree was 68.7% for the entire sample (31.3% error rate), and correct classification of individuals with the most frequent LRP (completed all) was 98%.The most significant predictor of the LRP was education at time of injury (c2Z956.26, dfZ10, P<.000).
636 The CHAID decision tree has been split into 3 figures based on the first split in the data by education at the time of injury. Figure 1 represents the far left portion of the decision tree predicting the LRP for individuals who completed an HS education or less than an HS education at the time of injury (node 1) and its subsequent splits. Figure 2 represents the central portion of the decision tree predicting the LRP for individuals who completed more than an HS education (node 2) and its subsequent splits. Figure 3 represents the far right portion of the decision tree predicting the LRP for those individuals who were missing baseline education data (node 3) and its subsequent splits. The CHAID decision tree consists of 23 groups of participants; each group had different combinations of values on significant predictor variables (and, therefore, each represented by a node) across 3 levels with 15 end groups (terminal nodes).
J.W. Krellman et al Subgroups with a statistically greater likelihood of having a specific LRP are those which are represented by terminal nodes with an index percentage above a cutoff selected a priori. Index percentage is the proportion of participants in a given node with the targeted LRP relative to the number of participants with that LRP in the overall sample. For our analyses, we considered significant terminal nodes to be those with index percentages 120%. That is, a subgroup was considered significantly more likely to have a specific LRP if the number of participants in that subgroup with the LRP was 120% than the number of participants with the LRP in the overall sample. Using that criterion, several subgroups of participants were more likely to have one of the following LRPs: completed all, wave nonresponse, or nonresponse. No subgroups were more likely to have the LRP of dropout, late withdrawal, or withdrawal throughout.
Fig 1 CHAID decision tree predicting the LRP for those with an HS education or less and subsequent splits. Abbreviations: Adj., adjusted; FUP01, year 1 follow-up.
www.archives-pmr.org
Follow-up in traumatic brain injury research
637
Fig 2 CHAID decision tree predicting the LRP for those with greater than an HS education and subsequent splits. Abbreviations: Adj., adjusted; FUP01, year 1 follow-up; Rehab, rehabilitation.
A written interpretation of the terminal node representing the subgroup with the highest percentage of completers, wave nonresponders, and nonresponders is subsequently provided, and table 2 contains subgroup characteristics, subgroup total, node number, and index percentage for all terminal nodes representing the subgroups considered more likely to have a specific LRP. For the LRP of completed all, 2 subgroups had index percentages >120%, the greater of which was for those with more than an HS education, whose acute care primary payer was entered in the NDB, and who were living with parents after inpatient rehabilitation discharge (node 14). For the LRP of wave nonresponse, 7 subgroups had index percentages >120%, the greatest of which was for black individuals who were missing data on education level at time of injury and were classified as independent based on the FIM motor score (node 21). For the LRP of nonresponse, 7 subgroups had index percentages >120%, the greatest of which was for those www.archives-pmr.org
with an HS education or less and for whom acute care primary payer data were missing from the NDB.
Discussion Our analyses revealed subgroups more likely to have specific LRPs in the TBIMS NDB. Completers were more likely to have higher levels of education, acute care payer data entered in the NDB, and live with parents or others. Wave nonresponders were more likely to have less education or have no baseline education data in the NDB, have no acute care payer data in the NDB, be physically independent, and be from a racial/ethnic minority background; however, white participants with missing education data and who were <41 years old were also more likely to be wave nonresponders. Nonresponders were also more likely to be
638 J.W. Krellman et al
www.archives-pmr.org
Fig 3 CHAID decision tree predicting the LRP for those missing baseline education data and subsequent splits. Abbreviations: Adj., adjusted; FIMMOT, FIM motor score; FUP01, year 1 follow-up; PHI, protected health information.
Follow-up in traumatic brain injury research Table 2
639
Subgroups associated with LRP in CHAID analyses
Response Pattern
Associated Subgroups
n
Node
Index (%)
Completed all
>HS education Acute care payer entered in NDB Living with parents at rehabilitation discharge >HS education Acute care payer entered in NDB Living with others at rehabilitation discharge Education missing Black No helper required on FIM motor Education missing Other race Unmarried or marital status missing Education missing Black Dependent or missing FIM motor score Education missing Other race Married Education missing White <41y of age HS education Acute care payer missing >41y of age HS education Acute care payer missing >HS education Other or missing acute care payer Education missing Other race Unmarried or marital status missing Education missing Black No helper required on FIM Education missing White >41y of age Education missing Other race Married Education missing Black Dependent or missing FIM motor score
654
14
124.6
370
15
120.9
249
21
401.6
177
22
305.2
343
20
223.3
99
23
219.9
856
18
218.5
451
19
126.9
69
5
530.2
79
7
268.9
177
22
226.7
249
21
142.2
451
19
141.3
99
23
131.4
343
20
120.4
Wave nonresponse
Nonresponse
physically independent and have less education or have missing baseline education data, but those with more education were also more likely to be nonresponders if they were missing acute care payer data. Nonresponders were also more likely to be from racial/ ethnic minority backgrounds; however, white participants were also more likely to be nonresponders if baseline education data were missing and they were >41 years. The fact that completers have more education might suggest higher SES among these individuals. Indeed, low SES has been associated with LTFU in the work of Corrigan et al1 and in analyses of other longitudinal databases.19,21,23,27 In addition, completers’ greater likelihood of living with parents or others suggests that supervision and/or social support associated with cohabitation might be protective against LTFU. www.archives-pmr.org
Unlike completers, wave nonresponders and nonresponders tended to belong to historically socioeconomically disadvantaged groups (ie, those with less education, those with ethnic/racial minority backgrounds), which is consistent with previous work.1,19,21,23,27 Physical independence was also predictive of both nonresponse and wave nonresponse, again in accord with Corrigan’s work1 showing that severe motor deficits mitigate LTFU risk. Perhaps physically mobile individuals are more difficult to follow because they have less stable living situations, lower likelihood of having caregivers, and/or more responsibilities within their communities. Interestingly, having acute care payer data in the NDB was associated with completing all follow-ups whether that payer was private (eg, commercial insurance) or public (eg, state-based financial assistance), suggesting that this finding was not related to
640 SES. Conversely, participants missing baseline education, acute care payer, or baseline employment status data were more likely to be wave nonresponders or nonresponders. Subgroups with missing data often had other characteristics associated with LTFU (eg, less education, minority race).1,19,21,23,27 However, missing data was also predictive of LTFU in subgroups with traits typically not associated with LTFU in this study or others (eg, more education, white race),1,19,21,23,27 suggesting that missing baseline datadand perhaps subsequent LTFUdin these individuals reflect the quality and/or consistency of data collection methods and not the characteristics of the participants themselves. Indeed, collecting acute care payer and baseline education and employment data requires only direct questioning of the participant, a reliable proxy, and/or the hospital billing department. These data might be missing because participants were too impaired to respond and no reliable proxy was available or because the data collection center could not or did not attempt to obtain information not recorded in the initial data collection (eg, by following up with a participant after level of consciousness improved or completing baseline data forms using information obtained at follow-up). Standard follow-up practices have been mandated by the TBIMS, but additional strategies to enhance data quality and followup completeness have been recommended (eg, periodic check-ins between mandated follow-ups and maintaining extensive records of alternative contact information for participants and proxies).28 Such optional strategies have been shown to improve retention even in populations traditionally among the most challenging to retain, such as substance abusers.29-31 Future analyses should examine the association between use of optional follow-up methods and rates of LTFU in the TBIMS NDB and other longitudinal studies. Further, our data suggest particular subgroups for which these methods might be especially useful; for example, evening or weekend calls might be a preferred strategy for contacting individuals with intact motor functioning who might be less available during traditional working hours whereas participants with low SES might be more easily tracked by regularly searching public records.
J.W. Krellman et al be lost, yielding a different LRP. Future analyses might classify participants’ LRPs while accounting for the number of follow-ups they are eligible to complete. Third, we were able to correctly classify most of those with the LRP of completed all, but correct classification of those with other LRPs was much less accurate, which was likely because of the relatively low frequency of most other LRPs in the NDB and the high degree of association between missing baseline data and LTFU. Therefore, future analyses aimed at developing highly accurate predictive models of LTFU might consider classifying participants according to fewer LRPs than used here (eg, completed all vs nonresponse and/or wave nonresponse) and/or not including missing data as a predictor. Finally, there is no empirically derived cutoff to determine which exhaustive CHAID index percentages are meaningful. Our data were consistent in that subgroups with index percentages around the cutoff of 120% were mostly similar to those with percentages even 2 or 3 times as high. However, the absence of a validated, stringent cutoff means that some findings, especially those supported by index scores around the cutoff, might not be meaningful.
Conclusions Specific demographic, psychosocial, and postinjury functioning factors are predictive of LRPs in the TBIMS NDB. These data provide rehabilitation researchers with subject matter knowledge about risk factors for LTFU among persons with TBI and provide a basis for the addition of specific retention strategies for different subgroups at risk for LTFU. The ultimate aim is to increase retention of participants and reduce biases in data that threaten the validity and applicability of longitudinal outcome studies of persons with TBI.
Supplier a. SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL 60606.
Keywords Study limitations Several other factors that might predict LTFU could not be included in our analysis because relevant data are not contained in the NDB. These include the specific follow-up practices (eg, retention strategies, participation incentives) used by different sites, the degree of coordination between research and clinical care follow-ups, participants’ level of rehabilitation program compliance, and mood symptoms in participants during rehabilitation. The association between these factors and LTFU should be investigated in future studies, and relevant data could be considered for future inclusion in the NDB. In regard to limitations of our analysis, including all TBIMS participants eligible for follow-up in our analyses let us examine a broader range of response patterns than previous studies, but our findings might not be as generalizable as those obtained from a more circumscribed analysis. Nesting future analyses of the NDB within a center could yield data on geographic factors (eg, community setting and demographic populations served) that might modify the association between participant characteristics and LTFU. Second, our analysis did not define the LRP based on the number of follow-ups for which participants were eligible; therefore, an individual who completed 1 of 1 follow-up was classified identically to another who completed 5 of 5 follow-ups. The former participant could eventually
Brain injuries; Bias (epidemiology); Follow-up studies; Lost to follow-up; Outcome assessment (health care); Rehabilitation
Corresponding author Jason W. Krellman, PhD, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1163, New York, NY 10029-6547. E-mail address: [email protected].
Acknowledgments We thank Wayne Gordon, PhD, for acting as a consultant on the design of this study, Dave Mellick, MA, and Chris Cusack, BA, of the TBIMS National Data and Statistical Center for their assistance with using the TBIMS NDB, and Michael Nguyen, MS, and Jennifer Oswald, BA, for consultation and information about issues relevant to TBIMS NDB data collection.
References 1. Corrigan JD, Harrison-Felix C, Bogner J, Dijkers M, Terrill MS, Whiteneck G. Systematic bias in traumatic brain injury outcome
www.archives-pmr.org
Follow-up in traumatic brain injury research
2.
3.
4.
5.
6. 7.
8. 9.
10.
11.
12.
13.
14.
15.
16.
studies because of loss to follow-up. Arch Phys Med Rehabil 2003;84: 153-60. Levin HS, Gary HE Jr, Eisenberg HM, et al. Neurobehavioral outcome 1 year after severe head injury. Experience of the Traumatic Coma Data Bank. J Neurosurg 1990;73:699-709. Touloumi G, Babiker AG, Pocock SJ, Darbyshire JH. Impact of missing data due to drop-outs on estimators for rates of change in longitudinal studies: a simulation study. Stat Med 2001;20:3715-28. Corrigan JD, Bogner JA, Mysiw WJ, Clinchot D, Fugate L. Systematic bias in outcome studies of persons with traumatic brain injury. Arch Phys Med Rehabil 1997;78:132-7. Bell KR, Hammond F, Hart T, Bickett AK, Temkin NR, Dikmen S. Participant recruitment and retention in rehabilitation research. Am J Phys Med Rehabil 2008;87:330-8. Hunt JR, White E. Retaining and tracking cohort study members. Epidemiol Rev 1998;20:57-70. Kristman VL, Manno M, Cote P. Methods to account for attrition in longitudinal data: do they work? A simulation study. Eur J Epidemiol 2005;20:657-62. Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol 1977;106:184-7. University of Washington. Model Systems Knowledge Translation Center. Available at: http://msktc.washington.edu/research/searchpubs. asp?modelsysZtbi. Accessed September 21, 2011. Dijkers MP, Harrison-Felix C, Marwitz JH. The traumatic brain injury model systems: history and contributions to clinical service and research. J Head Trauma Rehabil 2010;25:81-91. Traumatic Brain Injury Model Systems National Data and Statistical Center. 2012 TBI Model Systems presentation. Available at: https:// www.tbindsc.org/StaticFiles/Documents/2012%20TBIMS%20Slide%20 Presentation.pdf. Accessed January 10, 2013. Harrison-Felix C, Dijkers M. Predictors of loss to follow-up in a longitudinal study of persons with traumatic brain injury. Detroit: Rehabilitation Institute of Michigan Del Harder Research Day; 1998. Hoofien D, Vakil E, Gilboa A, Donovick PJ, Barak O. Comparison of the predictive power of socio-economic variables, severity of injury and age on long-term outcome of traumatic brain injury: samplespecific variables versus factors as predictors. Brain Inj 2002;16:9-27. Arango-Lasprilla JC, Rosenthal M, Deluca J, et al. Traumatic brain injury and functional outcomes: does minority status matter? Brain Inj 2007;21:701-8. Gary KW, Arango-Lasprilla JC, Stevens LF. Do racial/ethnic differences exist in post-injury outcomes after TBI? A comprehensive review of the literature. Brain Inj 2009;23:775-89. Keith RA, Granger CV, Hamilton BB, Sherwin FS. The functional independence measure: a new tool for rehabilitation. Adv Clin Rehabil 1987;1:6-18.
www.archives-pmr.org
641 17. Hall K, Cope DN, Rappaport M. Glasgow Outcome Scale and Disability Rating Scale: comparative usefulness in following recovery in traumatic head injury. Arch Phys Med Rehabil 1985;66:35-7. 18. Bushnik T, Englander J, Wright J, Kolakowsky-Hayner SA. Traumatic brain injury with and without late posttraumatic seizures: what are the impacts in the post-acute phase: a NIDRR Traumatic Brain Injury Model Systems study. J Head Trauma Rehabil 2012;27:E36-44. 19. Bergman P, Ahlberg G, Forsell Y, Lundberg I. Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates. Int J Soc Psychiatry 2010;56:119-32. 20. Claus RE, Kindleberger LR, Dugan MC. Predictors of attrition in a longitudinal study of substance abusers. J Psychoactive Drugs 2002; 34:69-74. 21. de Groot S, Haisma JA, Post MW, van Asbeck FW, van der Woude LH. Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study. J Rehabil Med 2009;41:382-9. 22. Brown DM, Thorne JE, Foster GL, et al. Factors affecting attrition in a longitudinal study of patients with AIDS. AIDS Care 2006;18: 821-9. 23. Holavanahalli RK, Lezotte DC, Hayes MP, et al. Profile of patients lost to follow-up in the burn injury rehabilitation model systems’ longitudinal database. J Burn Care Res 2006;27:703-12. 24. Malec JF, Brown AW, Leibson CL, et al. The mayo classification system for traumatic brain injury severity. J Neurotrauma 2007;24: 1417-24. 25. Kass GV. An exploratory technique for investigating large quantities of categorical data. Appl Statis 1980;29:119. 26. Biggs D, deVille B, Suen E. A method of choosing multiway partitions for classification and decision trees. J Appl Statist 1991;18:49. 27. Brilleman SL, Pachana NA, Dobson AJ. The impact of attrition on the representativeness of cohort studies of older people. BMC Med Res Methodol 2010;10:71. 28. Traumatic Brain Injury Model Systems National Data and Statistical Center. Guidelines and strategies for maximizing follow-up. Available at: https://www.tbindsc.org/StaticFiles/SOP/105a%20-%20Guidelines %20and%20Strategies%20for%20Maximizing%20Followup.pdf. Accessed January 11, 2011. 29. Scott CK. A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers. Drug Alcohol Depend 2004;74:21-36. 30. Scott CK, Sonis J, Creamer M, Dennis ML. Maximizing follow-up in longitudinal studies of traumatized populations. J Trauma Stress 2006; 19:757-69. 31. Kleschinsky JH, Bosworth LB, Nelson SE, Walsh EK, Shaffer HJ. Persistence pays off: follow-up methods for difficult-to-track longitudinal samples. J Stud Alcohol Drugs 2009;70:751-61.
ORIGINAL ARTICLE
Predictors of Follow-Up Completeness in Longitudinal Research on Traumatic Brain Injury: Findings From the National Institute on Disability and Rehabilitation Research Traumatic Brain Injury Model Systems Program Jason W. Krellman, PhD,a Stephanie A. Kolakowsky-Hayner, PhD,b Lisa Spielman, PhD,a Marcel Dijkers, PhD,a Flora M. Hammond, MD,c Jennifer Bogner, PhD,d Tessa Hart, PhD,e Joshua B. Cantor, PhD,a Theodore Tsaousides, PhDa From the aDepartment of Rehabilitation Medicine, Icahn School of Medicine at Mount Sinai, New York, NY; bSanta Clara Valley Medical Center, Rehabilitation Research Center, San Jose, CA; cIndiana University School of Medicine, Indianapolis, IN; dDepartment of Physical Medicine and Rehabilitation, Ohio State University, Columbus, OH; and eMoss Rehabilitation Research Institute, Elkins Park, PA.
Abstract Objective: To identify baseline participant variables in the domains of demographics, medical/psychosocial history, injury characteristics, and postinjury functional status associated with longitudinal follow-up completeness in persons with traumatic brain injury (TBI) using the TBI Model Systems (TBIMS) National Database (NDB). Design: Exhaustive chi-square automatic interaction detection was used to identify factors that classified participants according to level of followup completeness. Setting: Retrospective analysis of a multi-center longitudinal database. Participants: Individuals (NZ8249) enrolled in the TBIMS NDB between 1989 and 2009 who were eligible for at least the first (year 1) followup up to the fifth (year 15) follow-up. Interventions: None. Main Outcome Measures: Follow-up completeness as defined by 6 different longitudinal response patterns (LRPs): completing all follow-ups, wave nonresponse, dropping out, completing no follow-ups without formally withdrawing, formally withdrawing before completing any followups, and formally withdrawing after completing some follow-ups. Results: Completing all follow-ups was associated with higher levels of education, living with parents or others, and having acute care payer data entered in the NDB. Subgroups more vulnerable to loss to follow-up (LTFU) included those with less education, racial/ethnic minority backgrounds, those with better motor functioning on rehabilitation discharge, and those for whom baseline data on education, employment, and acute care payer were not collected. No subgroups were found to be more likely to have the LRPs of dropping out or formal withdrawal. Conclusions: These data identify subgroups in which retention strategies beyond those most commonly used might reduce LTFU in longitudinal studies of persons with TBI, such as the TBIMS, and suggest future investigations into factors associated with missing baseline data. Archives of Physical Medicine and Rehabilitation 2014;95:633-41 ª 2014 by the American Congress of Rehabilitation Medicine
Supported by the National Institute on Disability and Rehabilitation Research, U.S. Department of Education (grants nos. H133P050004, H133B980013, and H133A070038 [Mt. Sinai]; H133A070038 [Santa Clara]; H133A120035 and H133A070042 [Indiana University]; H133A070029 and H133A120086 [Ohio State], and H133A070040 and H133A120037 [Moss]) and the Centers for Disease Control and Prevention (grant no. 1R49CE001171-01 [Mt. Sinai]). No commercial party having a direct financial interest in the results of the research supporting this article has conferred or will confer a benefit on the authors or on any organization with which the authors are associated.
Longitudinal studies are vital to understanding long-term outcomes after traumatic brain injury (TBI). However, 33% to 50% of participants in these studies have historically been lost to follow-up (LTFU).1,2 Participants who are lost nearly always differ systematically from those retained,1,2 which threatens both the internal and external validity of longitudinal outcome studies3 and promotes
0003-9993/14/$36 - see front matter ª 2014 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2013.10.016
634
J.W. Krellman et al
biased conclusions about outcomes after TBI. Indeed, factors generally predictive of LTFU (eg, low socioeconomic status [SES]) often adversely affect outcomes,4 which suggests that systematic LTFU might yield outcome data that are positively biased. Such bias can be reduced by preventing LTFU (eg, by increasing the frequency of participant contact or using sophisticated tracking systems5,6) or by using statistical methods that compensate for loss after it has occurred. However, statistical procedures cannot always reliably estimate or eliminate the impact of LTFU on outcome data.7 Prevention techniques are, therefore, more desirable1 but require what Greenland8 termed subject matter knowledge, such as data regarding which participants are more vulnerable to LTFU. Relatively few investigations have focused on factors predictive of LTFU in longitudinal studies of individuals with TBI. A study by Corrigan et al1 suggested that those with a history of substance abuse, with higher motor function at rehabilitation discharge, and from “socioeconomically disadvantaged groups” (racial and ethnic minorities and those who had little education, were unemployed at time of injury, or were dependent on public funding for acute care) were more frequently lost at 1 and 2 year postinjury follow-ups in 3 large longitudinal studies. Of the 3 datasets analyzed by Corrigan, the TBI Model Systems (TBIMS) National Database (NDB) includes the largest number of participants, is the longest running, and has provided the basis for the most analyses, yielding approximately 350 published studies thus far.9 The NDB has experienced LTFU since its inception, as do all longitudinal studies, and instituted targets for follow-up completeness in 2006.10 However, between 17% and 23% of participants in the NDB are still lost depending on the specific follow-up point.11 The goal of the current study was to determine participantrelated factors predictive of LTFU in the longitudinal study of persons with TBI using the current TBIMS NDB. Previous work in this area has sought to identify factors associated with either response or nonresponse at 1 or 2 follow-ups, but we aimed to identify factors associated with a number of different longitudinal response patterns (LRPs) (eg, completing all follow-ups, skipping some follow-ups) up to a maximum of 5 follow-ups. This approach allowed us to investigate factors associated with LTFU across the more lengthy sequence of follow-ups needed to best characterize long-term outcomes, thus enhancing the potential usefulness of our findings. The NDB was ideal for our purposes because of the large number of available participants, volume of data collected, and number of follow-up data points available, all of which suggest that findings in the NDB would be applicable to many other longitudinal studies of persons with TBI and could inform followup practices to maximize participant retention. In addition, there is value in identifying factors associated with LTFU in the NDB for its own sake because of its significant contribution to the outcomes
List of abbreviations: CHAID HS LRP LTFU NDB PTA SES TBI TBIMS
chi-square automatic interaction detection high school longitudinal response pattern loss to follow-up National Database posttraumatic amnesia socioeconomic status traumatic brain injury Traumatic Brain Injury Model Systems
literature, evidence of selective LTFU1,12 in the database, and the association between previously established predictors of LTFU in the NDB and outcomes.13-15 The overarching goal of the present study was to determine whether various LRPs could be predicted early in study participation and to identify participant characteristics that might signal the need for implementation of specific longitudinal retention strategies. Ideally, these strategies would maximize retention and, therefore, enhance the validity and applicability of longitudinal studies of persons with TBI.
Methods TBI Model Systems An extensive description of the TBIMS and its NDB is reported in Dijkers et al.10 Further information regarding inclusion criteria and data included in the NDB is available on the TBMIS National Data and Statistics Center website.11 Briefly, participants must have sustained a moderate-severe TBI as evidenced by a Glasgow Coma Scale score of <13 in the emergency department, loss of consciousness >30 minutes, posttraumatic amnesia (PTA) >24 hours, and/or structural abnormalities indicative of TBI on neuroimaging; must be 16 years old; and must have received acute care and rehabilitation from a participating medical center. Informed consent is obtained from the participant or, if unable, a family member or legal guardian. Data collected just prior to rehabilitation discharge (baseline) include case mix, acute care, and rehabilitation information obtained through review of emergency and acute care medical records and an interview with the participant or proxy if the participant is unable to respond reliably. Review of rehabilitation records is used to calculate scores on functional status measures (eg, FIM16 and Disability Rating Scale17). Follow-up data on outcomes are currently collected 1, 2, 5, and 10 years postinjury and every fifth year thereafter. The follow-up survey is completed in-person or via telephone or mail with the participant or a proxy if the participant is unable to respond.
Analyzed sample At the time of our analyses, the NDB included 9310 participants enrolled at 22 centers that are currently (16 centers) or previously (6 centers) participating in the TBIMS. Participants whose LRP, which will be subsequently discussed, could not be determined from the data were excluded from the analysis. These included participants whose first (year 1) follow-up was not yet due (nZ589) and those incarcerated during 1 follow-ups (nZ251) because incarcerated participants are not contacted. Participants who died before completing any follow-ups (nZ207) were also excluded. Only 14 participants were eligible for follow-up at year 20 and were also excluded. The sample analyzed included all individuals enrolled in the TBIMS between 1989 and 2009 and who were eligible for at least the first (year 1) follow-up up to the fifth (year 15) follow-up (NZ8249).
Longitudinal response pattern LRP was the outcome variable in our analyses. Table 1 contains numbers and examples of each LRP included as a possible www.archives-pmr.org
Follow-up in traumatic brain injury research
635
outcome. Based on the LRP, participants were classified as 1 of 6 kinds of responders: (1) completers who completed all follow-ups; (2) wave nonresponders who completed follow-up(s) but also skipped follow-up(s); (3) dropouts who completed follow-up(s) but then did not complete subsequent follow-up(s) without formally withdrawing; (4) late withdrawers who formally withdrew after completing 1 follow-ups; (5) withdrawers throughout who formally withdrew without completing any follow-ups; and (6) nonresponders who did not complete any follow-ups but did not formally withdraw. Follow-ups scheduled during periods when a TBIMS center was defunded were not considered when determining participants’ LRP because follow-up contact is not attempted during such periods. The number of individuals affected by defunding was small (a minimum of 0.5% of those eligible for a year 15 follow-up and a maximum of 8% of those eligible for a year 5 follow-up). Indeed, preliminary analyses did not suggest an association between whether a participant was affected by defunding in the past and their LRP.
Predictors Selected baseline variables were recoded based on the distribution of participants within subgroups or on categories used in previous analyses of the NDB18 and then used as candidate predictors of LRPs. Predictors fell into 4 broad categories: (1) demographics, including acute care payer (public/private/other), age, marital status (married/unmarried), race (white/black/other), sex, and education at injury (
Table 1
LRPs in the TBIMS NDB with examples Follow-Up Status (year no.)
Response Pattern
n
%
1
2
5
10
15
Completed all Wave nonresponse* Dropout* Late withdrawal* Withdrawal throughout Nonresponse
5588 1023 651 175 113 699
67.7 12.4 7.9 2.1 1.4 8.5
O O O O W X
O X O O W X
O X X W W X
O O X W W X
O O X W W X
Abbreviations: W, participant withdrew; X, follow-up not completed; O, follow-up completed. * The pattern of these LRPs can vary among participants. The examples provided reflect one possible pattern.
www.archives-pmr.org
psychiatric and substance abuse populations19,20 and level of education in persons with spinal cord injury21) whereas others are conceptually similar to variables previously associated with LTFU. For example, condition-specific indicators of severity have been associated with LTFU in non-TBI populations (eg, immunologic status in human immunodeficiency virus/acquired immunodeficiency syndrome22 and length of hospital stay in burn injury23); therefore, we included duration of PTA, an indicator of TBI severity,24 as a candidate predictor.
Statistical procedures Statistical analyses were conducted using SPSS version 19.0 with SPSS Answer Tree 3.0 software.a Exhaustive chi-square automatic interaction detection (CHAID)25 was used to identify characteristics that classified participants as having a specific LRP. Exhaustive CHAID involves repeatedly merging, splitting, and stopping categories of predictor variables to create a decision tree that can identify group membership based on the relative association between each significant predictor and outcome. Continuous variables are first transformed into categorical variables by dividing continuous distributions into categories with a roughly equal number of observations. Categories of predictors that are least significantly different with respect to the dependent variable are then merged. If certain categories are not statistically different, those categories are merged and the step repeated until statistically different categories of each predictor are determined. This exhaustive repetition is an improvement over the original version of CHAID, which was developed by Biggs et al,26 wherein merging ends when all remaining categories are statistically different, which can result in misidentification of the optimal split for a variable. Chi-square tests are then performed to determine the relation between each predictor and outcome, and a Bonferroni-adjusted P value is calculated for each predictor, which yields a classification tree organized according to the degree of association between predictor and outcome (eg, the most significant predictor of the outcome represents the first split in the data, and the least significant represents the last split). Stopping occurs when statistically different categories of the least significant predictor are determined. At that point, no further splits are performed, which results in a terminal node of the classification tree. The CHAID technique was chosen because the analysis is designed to determine variables that identify group membership; allows both predictors and the dependent variable to be nominal, ordinal, or interval; and because predictor variables need not be measured on the same scale (eg, nominal or ordinal). CHAID considers missing data to be a valid category (eg, sex: male/ female/missing); therefore, the absence of a value for a particular variable can be considered a value in itself.
Results For categorical criterion variables, such as the LRP, the risk estimate is the proportion of cases in the sample incorrectly classified or the error rate. The overall correct classification of the CHAID decision tree was 68.7% for the entire sample (31.3% error rate), and correct classification of individuals with the most frequent LRP (completed all) was 98%.The most significant predictor of the LRP was education at time of injury (c2Z956.26, dfZ10, P<.000).
636 The CHAID decision tree has been split into 3 figures based on the first split in the data by education at the time of injury. Figure 1 represents the far left portion of the decision tree predicting the LRP for individuals who completed an HS education or less than an HS education at the time of injury (node 1) and its subsequent splits. Figure 2 represents the central portion of the decision tree predicting the LRP for individuals who completed more than an HS education (node 2) and its subsequent splits. Figure 3 represents the far right portion of the decision tree predicting the LRP for those individuals who were missing baseline education data (node 3) and its subsequent splits. The CHAID decision tree consists of 23 groups of participants; each group had different combinations of values on significant predictor variables (and, therefore, each represented by a node) across 3 levels with 15 end groups (terminal nodes).
J.W. Krellman et al Subgroups with a statistically greater likelihood of having a specific LRP are those which are represented by terminal nodes with an index percentage above a cutoff selected a priori. Index percentage is the proportion of participants in a given node with the targeted LRP relative to the number of participants with that LRP in the overall sample. For our analyses, we considered significant terminal nodes to be those with index percentages 120%. That is, a subgroup was considered significantly more likely to have a specific LRP if the number of participants in that subgroup with the LRP was 120% than the number of participants with the LRP in the overall sample. Using that criterion, several subgroups of participants were more likely to have one of the following LRPs: completed all, wave nonresponse, or nonresponse. No subgroups were more likely to have the LRP of dropout, late withdrawal, or withdrawal throughout.
Fig 1 CHAID decision tree predicting the LRP for those with an HS education or less and subsequent splits. Abbreviations: Adj., adjusted; FUP01, year 1 follow-up.
www.archives-pmr.org
Follow-up in traumatic brain injury research
637
Fig 2 CHAID decision tree predicting the LRP for those with greater than an HS education and subsequent splits. Abbreviations: Adj., adjusted; FUP01, year 1 follow-up; Rehab, rehabilitation.
A written interpretation of the terminal node representing the subgroup with the highest percentage of completers, wave nonresponders, and nonresponders is subsequently provided, and table 2 contains subgroup characteristics, subgroup total, node number, and index percentage for all terminal nodes representing the subgroups considered more likely to have a specific LRP. For the LRP of completed all, 2 subgroups had index percentages >120%, the greater of which was for those with more than an HS education, whose acute care primary payer was entered in the NDB, and who were living with parents after inpatient rehabilitation discharge (node 14). For the LRP of wave nonresponse, 7 subgroups had index percentages >120%, the greatest of which was for black individuals who were missing data on education level at time of injury and were classified as independent based on the FIM motor score (node 21). For the LRP of nonresponse, 7 subgroups had index percentages >120%, the greatest of which was for those www.archives-pmr.org
with an HS education or less and for whom acute care primary payer data were missing from the NDB.
Discussion Our analyses revealed subgroups more likely to have specific LRPs in the TBIMS NDB. Completers were more likely to have higher levels of education, acute care payer data entered in the NDB, and live with parents or others. Wave nonresponders were more likely to have less education or have no baseline education data in the NDB, have no acute care payer data in the NDB, be physically independent, and be from a racial/ethnic minority background; however, white participants with missing education data and who were <41 years old were also more likely to be wave nonresponders. Nonresponders were also more likely to be
638 J.W. Krellman et al
www.archives-pmr.org
Fig 3 CHAID decision tree predicting the LRP for those missing baseline education data and subsequent splits. Abbreviations: Adj., adjusted; FIMMOT, FIM motor score; FUP01, year 1 follow-up; PHI, protected health information.
Follow-up in traumatic brain injury research Table 2
639
Subgroups associated with LRP in CHAID analyses
Response Pattern
Associated Subgroups
n
Node
Index (%)
Completed all
>HS education Acute care payer entered in NDB Living with parents at rehabilitation discharge >HS education Acute care payer entered in NDB Living with others at rehabilitation discharge Education missing Black No helper required on FIM motor Education missing Other race Unmarried or marital status missing Education missing Black Dependent or missing FIM motor score Education missing Other race Married Education missing White <41y of age HS education Acute care payer missing >41y of age HS education Acute care payer missing >HS education Other or missing acute care payer Education missing Other race Unmarried or marital status missing Education missing Black No helper required on FIM Education missing White >41y of age Education missing Other race Married Education missing Black Dependent or missing FIM motor score
654
14
124.6
370
15
120.9
249
21
401.6
177
22
305.2
343
20
223.3
99
23
219.9
856
18
218.5
451
19
126.9
69
5
530.2
79
7
268.9
177
22
226.7
249
21
142.2
451
19
141.3
99
23
131.4
343
20
120.4
Wave nonresponse
Nonresponse
physically independent and have less education or have missing baseline education data, but those with more education were also more likely to be nonresponders if they were missing acute care payer data. Nonresponders were also more likely to be from racial/ ethnic minority backgrounds; however, white participants were also more likely to be nonresponders if baseline education data were missing and they were >41 years. The fact that completers have more education might suggest higher SES among these individuals. Indeed, low SES has been associated with LTFU in the work of Corrigan et al1 and in analyses of other longitudinal databases.19,21,23,27 In addition, completers’ greater likelihood of living with parents or others suggests that supervision and/or social support associated with cohabitation might be protective against LTFU. www.archives-pmr.org
Unlike completers, wave nonresponders and nonresponders tended to belong to historically socioeconomically disadvantaged groups (ie, those with less education, those with ethnic/racial minority backgrounds), which is consistent with previous work.1,19,21,23,27 Physical independence was also predictive of both nonresponse and wave nonresponse, again in accord with Corrigan’s work1 showing that severe motor deficits mitigate LTFU risk. Perhaps physically mobile individuals are more difficult to follow because they have less stable living situations, lower likelihood of having caregivers, and/or more responsibilities within their communities. Interestingly, having acute care payer data in the NDB was associated with completing all follow-ups whether that payer was private (eg, commercial insurance) or public (eg, state-based financial assistance), suggesting that this finding was not related to
640 SES. Conversely, participants missing baseline education, acute care payer, or baseline employment status data were more likely to be wave nonresponders or nonresponders. Subgroups with missing data often had other characteristics associated with LTFU (eg, less education, minority race).1,19,21,23,27 However, missing data was also predictive of LTFU in subgroups with traits typically not associated with LTFU in this study or others (eg, more education, white race),1,19,21,23,27 suggesting that missing baseline datadand perhaps subsequent LTFUdin these individuals reflect the quality and/or consistency of data collection methods and not the characteristics of the participants themselves. Indeed, collecting acute care payer and baseline education and employment data requires only direct questioning of the participant, a reliable proxy, and/or the hospital billing department. These data might be missing because participants were too impaired to respond and no reliable proxy was available or because the data collection center could not or did not attempt to obtain information not recorded in the initial data collection (eg, by following up with a participant after level of consciousness improved or completing baseline data forms using information obtained at follow-up). Standard follow-up practices have been mandated by the TBIMS, but additional strategies to enhance data quality and followup completeness have been recommended (eg, periodic check-ins between mandated follow-ups and maintaining extensive records of alternative contact information for participants and proxies).28 Such optional strategies have been shown to improve retention even in populations traditionally among the most challenging to retain, such as substance abusers.29-31 Future analyses should examine the association between use of optional follow-up methods and rates of LTFU in the TBIMS NDB and other longitudinal studies. Further, our data suggest particular subgroups for which these methods might be especially useful; for example, evening or weekend calls might be a preferred strategy for contacting individuals with intact motor functioning who might be less available during traditional working hours whereas participants with low SES might be more easily tracked by regularly searching public records.
J.W. Krellman et al be lost, yielding a different LRP. Future analyses might classify participants’ LRPs while accounting for the number of follow-ups they are eligible to complete. Third, we were able to correctly classify most of those with the LRP of completed all, but correct classification of those with other LRPs was much less accurate, which was likely because of the relatively low frequency of most other LRPs in the NDB and the high degree of association between missing baseline data and LTFU. Therefore, future analyses aimed at developing highly accurate predictive models of LTFU might consider classifying participants according to fewer LRPs than used here (eg, completed all vs nonresponse and/or wave nonresponse) and/or not including missing data as a predictor. Finally, there is no empirically derived cutoff to determine which exhaustive CHAID index percentages are meaningful. Our data were consistent in that subgroups with index percentages around the cutoff of 120% were mostly similar to those with percentages even 2 or 3 times as high. However, the absence of a validated, stringent cutoff means that some findings, especially those supported by index scores around the cutoff, might not be meaningful.
Conclusions Specific demographic, psychosocial, and postinjury functioning factors are predictive of LRPs in the TBIMS NDB. These data provide rehabilitation researchers with subject matter knowledge about risk factors for LTFU among persons with TBI and provide a basis for the addition of specific retention strategies for different subgroups at risk for LTFU. The ultimate aim is to increase retention of participants and reduce biases in data that threaten the validity and applicability of longitudinal outcome studies of persons with TBI.
Supplier a. SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL 60606.
Keywords Study limitations Several other factors that might predict LTFU could not be included in our analysis because relevant data are not contained in the NDB. These include the specific follow-up practices (eg, retention strategies, participation incentives) used by different sites, the degree of coordination between research and clinical care follow-ups, participants’ level of rehabilitation program compliance, and mood symptoms in participants during rehabilitation. The association between these factors and LTFU should be investigated in future studies, and relevant data could be considered for future inclusion in the NDB. In regard to limitations of our analysis, including all TBIMS participants eligible for follow-up in our analyses let us examine a broader range of response patterns than previous studies, but our findings might not be as generalizable as those obtained from a more circumscribed analysis. Nesting future analyses of the NDB within a center could yield data on geographic factors (eg, community setting and demographic populations served) that might modify the association between participant characteristics and LTFU. Second, our analysis did not define the LRP based on the number of follow-ups for which participants were eligible; therefore, an individual who completed 1 of 1 follow-up was classified identically to another who completed 5 of 5 follow-ups. The former participant could eventually
Brain injuries; Bias (epidemiology); Follow-up studies; Lost to follow-up; Outcome assessment (health care); Rehabilitation
Corresponding author Jason W. Krellman, PhD, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1163, New York, NY 10029-6547. E-mail address: [email protected].
Acknowledgments We thank Wayne Gordon, PhD, for acting as a consultant on the design of this study, Dave Mellick, MA, and Chris Cusack, BA, of the TBIMS National Data and Statistical Center for their assistance with using the TBIMS NDB, and Michael Nguyen, MS, and Jennifer Oswald, BA, for consultation and information about issues relevant to TBIMS NDB data collection.
References 1. Corrigan JD, Harrison-Felix C, Bogner J, Dijkers M, Terrill MS, Whiteneck G. Systematic bias in traumatic brain injury outcome
www.archives-pmr.org
Follow-up in traumatic brain injury research
2.
3.
4.
5.
6. 7.
8. 9.
10.
11.
12.
13.
14.
15.
16.
studies because of loss to follow-up. Arch Phys Med Rehabil 2003;84: 153-60. Levin HS, Gary HE Jr, Eisenberg HM, et al. Neurobehavioral outcome 1 year after severe head injury. Experience of the Traumatic Coma Data Bank. J Neurosurg 1990;73:699-709. Touloumi G, Babiker AG, Pocock SJ, Darbyshire JH. Impact of missing data due to drop-outs on estimators for rates of change in longitudinal studies: a simulation study. Stat Med 2001;20:3715-28. Corrigan JD, Bogner JA, Mysiw WJ, Clinchot D, Fugate L. Systematic bias in outcome studies of persons with traumatic brain injury. Arch Phys Med Rehabil 1997;78:132-7. Bell KR, Hammond F, Hart T, Bickett AK, Temkin NR, Dikmen S. Participant recruitment and retention in rehabilitation research. Am J Phys Med Rehabil 2008;87:330-8. Hunt JR, White E. Retaining and tracking cohort study members. Epidemiol Rev 1998;20:57-70. Kristman VL, Manno M, Cote P. Methods to account for attrition in longitudinal data: do they work? A simulation study. Eur J Epidemiol 2005;20:657-62. Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol 1977;106:184-7. University of Washington. Model Systems Knowledge Translation Center. Available at: http://msktc.washington.edu/research/searchpubs. asp?modelsysZtbi. Accessed September 21, 2011. Dijkers MP, Harrison-Felix C, Marwitz JH. The traumatic brain injury model systems: history and contributions to clinical service and research. J Head Trauma Rehabil 2010;25:81-91. Traumatic Brain Injury Model Systems National Data and Statistical Center. 2012 TBI Model Systems presentation. Available at: https:// www.tbindsc.org/StaticFiles/Documents/2012%20TBIMS%20Slide%20 Presentation.pdf. Accessed January 10, 2013. Harrison-Felix C, Dijkers M. Predictors of loss to follow-up in a longitudinal study of persons with traumatic brain injury. Detroit: Rehabilitation Institute of Michigan Del Harder Research Day; 1998. Hoofien D, Vakil E, Gilboa A, Donovick PJ, Barak O. Comparison of the predictive power of socio-economic variables, severity of injury and age on long-term outcome of traumatic brain injury: samplespecific variables versus factors as predictors. Brain Inj 2002;16:9-27. Arango-Lasprilla JC, Rosenthal M, Deluca J, et al. Traumatic brain injury and functional outcomes: does minority status matter? Brain Inj 2007;21:701-8. Gary KW, Arango-Lasprilla JC, Stevens LF. Do racial/ethnic differences exist in post-injury outcomes after TBI? A comprehensive review of the literature. Brain Inj 2009;23:775-89. Keith RA, Granger CV, Hamilton BB, Sherwin FS. The functional independence measure: a new tool for rehabilitation. Adv Clin Rehabil 1987;1:6-18.
www.archives-pmr.org
641 17. Hall K, Cope DN, Rappaport M. Glasgow Outcome Scale and Disability Rating Scale: comparative usefulness in following recovery in traumatic head injury. Arch Phys Med Rehabil 1985;66:35-7. 18. Bushnik T, Englander J, Wright J, Kolakowsky-Hayner SA. Traumatic brain injury with and without late posttraumatic seizures: what are the impacts in the post-acute phase: a NIDRR Traumatic Brain Injury Model Systems study. J Head Trauma Rehabil 2012;27:E36-44. 19. Bergman P, Ahlberg G, Forsell Y, Lundberg I. Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates. Int J Soc Psychiatry 2010;56:119-32. 20. Claus RE, Kindleberger LR, Dugan MC. Predictors of attrition in a longitudinal study of substance abusers. J Psychoactive Drugs 2002; 34:69-74. 21. de Groot S, Haisma JA, Post MW, van Asbeck FW, van der Woude LH. Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study. J Rehabil Med 2009;41:382-9. 22. Brown DM, Thorne JE, Foster GL, et al. Factors affecting attrition in a longitudinal study of patients with AIDS. AIDS Care 2006;18: 821-9. 23. Holavanahalli RK, Lezotte DC, Hayes MP, et al. Profile of patients lost to follow-up in the burn injury rehabilitation model systems’ longitudinal database. J Burn Care Res 2006;27:703-12. 24. Malec JF, Brown AW, Leibson CL, et al. The mayo classification system for traumatic brain injury severity. J Neurotrauma 2007;24: 1417-24. 25. Kass GV. An exploratory technique for investigating large quantities of categorical data. Appl Statis 1980;29:119. 26. Biggs D, deVille B, Suen E. A method of choosing multiway partitions for classification and decision trees. J Appl Statist 1991;18:49. 27. Brilleman SL, Pachana NA, Dobson AJ. The impact of attrition on the representativeness of cohort studies of older people. BMC Med Res Methodol 2010;10:71. 28. Traumatic Brain Injury Model Systems National Data and Statistical Center. Guidelines and strategies for maximizing follow-up. Available at: https://www.tbindsc.org/StaticFiles/SOP/105a%20-%20Guidelines %20and%20Strategies%20for%20Maximizing%20Followup.pdf. Accessed January 11, 2011. 29. Scott CK. A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers. Drug Alcohol Depend 2004;74:21-36. 30. Scott CK, Sonis J, Creamer M, Dennis ML. Maximizing follow-up in longitudinal studies of traumatized populations. J Trauma Stress 2006; 19:757-69. 31. Kleschinsky JH, Bosworth LB, Nelson SE, Walsh EK, Shaffer HJ. Persistence pays off: follow-up methods for difficult-to-track longitudinal samples. J Stud Alcohol Drugs 2009;70:751-61.