- Email: [email protected]
Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review
G Model
ARTICLE IN PRESS
JSAMS-1272; No. of Pages 7
Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
Journal of Science and Medicine in Sport journal homepage: www.elsevier.com/locate/jsams
Review
Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review Erin K. Howie ∗ , Leon M. Straker School of Physiotherapy and Exercise Science, Curtin University, Australia
a r t i c l e
i n f o
Article history: Received 14 August 2015 Received in revised form 28 December 2015 Accepted 30 December 2015 Available online xxx Keywords: Sample size Research design Follow-up studies
a b s t r a c t Objectives: The purpose of this brief review was to describe the missingness, from both attrition and non-compliance, during physical activity randomized controlled trials among children which have used accelerometers to measure physical activity. Design: Systematic review. Methods: Using a previously published search strategy, an updated search of the literature was performed in the MEDLINE database for articles published from 1996 to February 2015 identifying physical activity RCTs in children (ages 2–18) measuring physical activity using accelerometers. Rates of attrition and non-compliance were extracted from identified articles. Twenty-three independent studies provided complete attrition and non-compliance data and were included. Results: The mean attrition rate was 11.5% (SD 10.1%, range 0–30.9%). The mean accelerometer noncompliance rate at baseline was 22.7% (SD 16.4%, range 1.7–67.8%) and 29.6% (SD 19.4%, range 3.3–70.1%) at follow-up. The mean total study missingness was 37.4% (SD 20.2%, range 3.3–75.4%) and ranged from 3.3% to 75.4%. There was large variation in how missingness was accounted for between studies. There were no statistically significant differences in missingness between study characteristics including sample size, participant age, intervention setting, duration of follow-up, whether physical activity was the primary outcome, and weartime compliance criteria. Conclusions: Missingness is common among randomized controlled trials using accelerometry in children and is currently handled inconsistently. Researchers must plan for high levels of missingness in study design and account for missingness in reporting and analyses of trial outcomes. © 2016 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.
1. Introduction To improve poor physical activity levels of children, high quality research trials are needed to evaluate intervention strategies.1 To improve the quality and transparency of randomized controlled trial (RCT) reporting, the Consolidated Standards of Reporting Trials (CONSORT) statement was developed in 1996.2 In physical activity intervention trials, it is commonly anticipated that all participants who begin the trial may not provide complete data at follow-up, creating ‘missingness’ in trials. The CONSORT checklist requires a flow diagram including the number of participants at each stage including the number of participants randomly assigned to treatments and those included in the final analysis. According to
∗ Corresponding author. E-mail address: [email protected] (E.K. Howie).
the CONSORT website, over 585 journals have endorsed CONSORT reporting guidelines. Thus missingness should be reported for all RCTs to enable readers to judge the quality of the evidence reported. There are two components of missingness in trials: attrition and non-compliance. For the purposes of this study, attrition is defined as those participants who entered the study and who did not remain in the study at follow-up. It is important to note, that as part of this definition, missingness does not include whether a participant followed the intervention protocol, otherwise known as adherence, but whether or not they were still available for measurement at follow-up. In addition to attrition, there is non-compliance with measurement, in this case accelerometer protocols. Noncompliance is defined as participants who remained in the study at follow-up but did not participate in outcome assessment or did not provide valid physical activity outcome data. Together, attrition and non-compliance create missingness, or those who entered the study but were not included in the final results of the study.
http://dx.doi.org/10.1016/j.jsams.2015.12.520 1440-2440/© 2016 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
G Model JSAMS-1272; No. of Pages 7
ARTICLE IN PRESS E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
2
Accelerometers are currently and commonly being used for objective measurement of physical activity in children, and thus are often the primary outcome of physical activity trials. However, noncompliance with accelerometer protocols is known to be high,3,4 though limited information is available on precise estimates. As objective measures, accelerometers require substantial participant burden compared to the majority of self-report measures. While protocols vary, participants are usually required to provide three to four days of 8 to 10 h per day to have data considered valid for youth.5 In a recent meta-analysis of controlled trials that measured physical activity using accelerometers among children,6 the median “losses to follow-up” was 11% and ranged from zero to 46%. It is unclear, however, how attrition and non-compliance specifically contributed to ‘losses to follow-up’. Additionally, issues of how researchers dealt with the missingness and if it varied by study characteristics was not explored. The combination of attrition and non-compliance with accelerometer protocols may be resulting in a significant percentage of study samples missing from the findings of physical activity studies and thus findings are at risk of biased study findings. The purpose of this review was to examine the missingness, both attrition and non-compliance, during physical activity RCTs among children which have used accelerometers to measure physical activity.
2. Methods Using the published search strategy from the EarlyBird 54 review of physical activity RCTs with objective measures of physical activity,6 an updated search of the literature was performed in the MEDLINE database. Articles were searched that had been published from February 2015 to 1996, when CONSORT reporting guidelines were established. Reference lists were cross-checked. To be included, articles had to be the primary report of the RCT, excluding duplicate or secondary analyses articles. Study design was restricted to randomized controlled trials as they are currently considered the highest quality study design and should follow CONSORT reporting guidelines, thus increasing the chance of required information being reported. Participants needed to be between the ages of 2 and eighteen. To reduce the inclusion of preliminary studies or small studies where higher than normal effort may be applied to decrease attrition and non-compliance, studies with an n < 50 were excluded. The intervention duration had to be a minimum of two weeks, thus providing a minimum duration of two weeks between baseline and follow-up measures. This was to allow for the natural process of attrition. The reported outcome (primary or secondary) of the study had to include child physical activity as measured by accelerometer. The accelerometer had to be used for measuring full day physical activity (excluding studies that only measured school day physical activity where research staff supervised accelerometer wear at school thus create an artificial weartime compliance). The same children had to be measured at two time points as part of the experimental design andtudies where a random sample of children were selected at each time point were excluded. These two previous points would lead to different samples used to calculate attrition and non-compliance, thus precluding the ability to calculate total missingness from these studies. To calculate the attrition, non-compliance and overall missingness, four numbers were extracted from the articles. • N available at baseline = participants present for baseline testing and randomized (not excluded).
• N with valid PA at baseline = participants with valid accelerometer data at baseline as defined by author. • N available at follow-up = participants who were available at the first follow-up assessment following completion of intervention delivery. If not reported, taken as measure with the highest n at follow-up. • N with valid PA at follow-up = participants with valid accelerometer wear at follow-up, only of those available at follow-up. Technical failure of accelerometers resulted in non-valid data and these participants were not included as having valid data. If the published study included some, but not complete noncompliance data, the corresponding authors were contacted and given the opportunity to provide the data. Five authors were contacted and two authors provided additional data (De Craemer, Verloigne). Four variables to assess missingness were calculated from the extracted variables using the following formulas: (1) Attrition (%) = 100 − (N available at follow-up/N available at baseline). (2) Non-compliance at baseline (%) = 100 − (N with valid PA at baseline − N available at baseline). (3) Non-compliance at follow-up (%) = 100 − (N with valid PA at follow-up − N available at follow-up). (4) Missingness (%) = 100 − (N with valid PA at follow-up − N available at baseline). The relationships between the study characteristics of sample size, participant age, intervention setting, intervention duration, whether physical activity was a primary or secondary outcome and compliance criteria (total hours needed for wear ranging from 12 h (2 days with a minimum of 6 h) to 53 h (4 days with 800 min per day) with missingness were examined using Spearman correlations for continuous data or ANOVA between stratified categorical variables. 3. Results A total of 8699 articles were retrieved and titles were screened. 272 abstracts were reviewed of which 100 full text articles were examined. 21 of the articles were excluded for using physical activity measures other than accelerometry, 15 studies were nonrandomized, 9 did only measured partial day physical activity using accelerometers, 9 measured PA in a only a subsample of participants, 5 were not the report of the primary study outcome, 3 had no measure of physical activity, 2 did not measure PA in the same children at two time-points, and one did not provide enough information to be assessed. Thirty-five original studies were eligible. Of the 35 eligible studies, twenty-three studies provided complete attrition and non-compliance data and were included in the analysis. One study only reported the total n at baseline. Seven studies did not provide compliance data at either timepoint (only reported attrition data). Four studies were missing baseline compliance data. One study reported compliance as “not enough to assess” but did not provide numbers. One study did not report follow-up numbers and the cited Appendix was unable to be obtained. Characteristics of included studies can be seen in Table 1. The average sample size was 517 (SD 565, range 60–2221). Sixteen of the studies had accelerometer measured physical activity as a primary outcome. Three studies were conducted with young children (<6 years of age), 17 were conducted with children (ages 6 to 11 years), and three were conducted with adolescents (11+). The majority (n = 13) of the interventions were conducted in a school setting. The duration ranged from 6 weeks to two school years with a mean of 34 weeks.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
Setting
location
Age (years)
Total n
Accelerometer criteria
PA outcome
Duration of intervention
Intervention
Baranowski et al.7
2011
Individual
US
10 to 12
153
4 days, 800 min/day
Primary
Home video game intervention
Catenacci et al.8
2014
Family
US
8 to 12
131 (98 families)
4 (1 weekend day), 10 h/day
Primary
Variable (9 sessions at own pace) 12 weeks
Cliff et al.9
2011
Individual
Australia
5 to 9
165
4 days, 10 h/day
Primary
6 months
De Craemer et al.10
2014
School
Belgium
4 to 6
472 (27 schools)
2 week days, 6 h/day
Primary
24 weeks
Dewar et al.11
2014
School
Australia
12 to 14
357
3 days (1 weekend), 10 h/day
Primary
12 months
Fairclough et al.12
2013
School
UK
10 to 11
318 (12 schools)
Secondary
20 weeks
Fitzgibbon et al.13
2013
Preschool
US
3 to 5
147 (4 preschools)
3 days, 540 min/week days: 480 min/weekend days 4 days, 8 h/day
Primary
14 weeks
Grydeland et al.14
2013
School
Norway
11 to 12
2165 (37 schools)
3 days (1 weekend day), 8 h/day
Primary
20 months
Hughes et al.15
2008
individual
UK
5 to 11
143
Secondary
26 weeks
Jago et al.16
2012
School
UK
11 to 12
210
Primary
9 weeks
Kipping et al.17
2014
School
UK
8 to 9
2221
3 days (1 weekend day), 6 h/day 3 weekdays, 500 min/day 3 days, 8 h/day
Primary
1 school year
Kriemler et al.18
2010
School
Switzerland
6 to 7
502 (15 schools)
2 weekdays, 12 h/day
Primary
1 school year
Maddison et al.19
2011
Individual
New Zealand
10 to 14
322
4 days (1 weekend day), 10 h/day
Secondary
24 weeks
America on the Move, small changes, messages delivered via printed workbook or internet HIKCUPS, 3 conditions: (1) 10 weeks of small group PA sessions (2) parent diet intervention (3) both Toy Box, teacher training to implement 1 h/week, PA component using ToyBox materials, resources, newsletters NEAT (Nutrition and Enjoyable Activity for Teen Girls), multi-component intervention targeting sports, lunchtime PA, nutrition workshops CHANGE!, weekly lesson plans and homework activities Hip Hop to Health, 3 day/week nutrition and aerobic lessons, parents 6 weekly classes HEIA (Health In Adolescents), multi-component intervention including lessons, exercise breaks, posters, active commuting campaigns, activity box 8, 1-to-1 behavioural program sessions delivered by pediatric dieticians Bristol Girls Dance Project, after-school dance classes AFLY5 (Active for Life Year 5), lesson plans and training for classroom teachers, homework activities KISS (Kinder-Sportstudie), multicomponent intervention including 2 additional PE lessons per week, PA breaks, PA homework Home active video games
ARTICLE IN PRESS
Date
G Model
JSAMS-1272; No. of Pages 7
Author
E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
Table 1 Intervention characteristics of included studies.
3
G Model
JSAMS-1272; No. of Pages 7
4
Date
Setting
location
Age (years)
Total n
Accelerometer criteria
PA outcome
Duration of intervention
Intervention
Magnusson et al.20
2011
School
Iceland
6 to 7
320 (6 schools)
Primary
2 years
Maloney et al.21
2008
Individual
US
7 to 8
60
Primary
28 weeks
Multi-component intervention including teacher workshops, equipment, additional PE Active video games (Dance Dance Revolution) at home
Meinhardt et al.22
2013
Individual
Switzerland
10 to 14
102
2 weekdays, 10 hrs/day and ≥85% of 6 h school day Valid data for >80% of waking hours or missing weekend day Not reported
Primary
1 school year
Patrick et al.23
2006
Individual
US
11 to 15
878
3 days, 10 h/day
Secondary
12 months
Puder et al.24
2011
School
Switzerland
4 to 5
652 (40 classes)
3 days (1 weekend day), 6 h
Secondary
1 school year
Smith et al.25
2014
School
Australia
12 to 14
361 (14 schools)
3 weekdays (1 weekend day for weekend analysis), 10 h/day
Secondary
20 weeks
Toftager et al.26
2014
School
Denmark
11 to 13
1348 (14 schools)
3 days, 10 h/day
Primary
2 years
Trost et al.27
2014
Family
US
8 to 12
75 (11 sites)
3 days, 9 h/day
Primary
16 weeks
Verloigne et al.28
2012
School
Belgium
10 to 11
740 (10 schools)
Primary
6 weeks
Wafa et al.29
2011
Individual
Malaysia
7 to 11
107
2 weekdays, 10 hrs/day: 1 weekend day, 8 h/day 4 days, 10 h/day
Secondary
26 weeks
Strength training PE lessons compared to regular PE PACE+ (Patient-Centred Assessment and Counselling For Exercise + Nutrition), primary-cared based including manual, telephone calls, mail Multicomponent curriculum including 4 × 45 min PA sessions, two workshops for teachers, environmental changes ATLAS (Active Teen Leaders Avoiding Screen-time), multicomponent intervention including teacher training, parent newsletters, enhanced school sport SPACE, multicomponent intervention with 11 components to change physical and organizational environment JOIN for ME, pediatric weight management program, 60 minute weekly sessions in parent-child small groups UP4FUN, 1–2 lessons per week conducted by classroom teachers, newsletter MASCOT (Malaysian Childhood Obesity Treatment Trial), 8 1-h group sessions
ARTICLE IN PRESS
Author
E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
Table 1 (Continued)
G Model
ARTICLE IN PRESS
JSAMS-1272; No. of Pages 7
E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
5
Table 2 Attrition and non-compliance sorted by percentage of missingness. Study
n (Available at baseline)
n (Valid PA at baseline)
n (Available at follow-up)
n (Valid PA at follow-up)
Attrition (%)
Non-compliance (baseline) (%)
Non-compliance (follow-up) (%)
Missingness (%)
Maloney et al.21 Catenacci et al.8 Meinhardt et al.22 Baranowski et al.7 Jago et al.16 Magnusson et al.20 Puder et al.24 Toftager et al.26 Kriemler et al.18 Verloigne et al.28 Wafa et al.29 Trost et al.27 Maddison et al.19 Kipping et al.17 Grydeland et al.14 Cliff et al.9 Fairclough et al.12 De Craemer et al.10 Hughes et al.15 Patrick et al.23 Smith et al.25 Fitzgibbon et al.13 Dewar et al.11
60 98 102 157 210 269 652 1348 502 740 107 75 322 2221 1580 165 318 1150 134 819 361 146 357
59 92 96 150 203 196 542 1233 352 566 86 53 297 1289 1129 137 280 653* 117 330 240 70 221
60 96 102 144 197 254 632 1159 498 732* 80 60 259 2121 1464 114 230 1074 97 690 293 143 294
58 90 93 138 181 224 484 912 335 479 65 44 187 1252 892 87 170 599* 67 330 120 46 88
0 2 0 8 6 6 3 14 1 1 25 20 20 5 7 31 28 7 28 16 19 2 18
2 6 6 4 3 27 17 9 30 24 20 29 8 42 29 17 12 43 13 60 34 52 38
3 6 9 4 8 12 23 21 33 35 19 27 28 41 39 24 26 44 31 52 59 68 70
3 8 9 12 14 17 26 32 33 35 39 41 42 44 44 47 47 48 50 60 67 68 75
*
Additional unpublished information was supplied by authors.
Individual study missingness results can be seen in Table 2. The mean attrition was 11.5% (SD 10.1%, range 0–30.9%). The mean accelerometer non-compliance at baseline was 22.7% (SD 16.4%, range 1.7–67.8%) and 29.6% (SD 19.4%, range 3.3%, 70.1%) at followup. The mean missingness was 37.4% (SD 20.2%) and ranged from 3.3% to 75.4%. Eleven studies did not report any attrition analyses. Ten compared baseline demographics or baseline values of outcome variables between those who completed from those who dropped out or did not have complete data. One study compared drop-out rates between the intervention and control groups. One study used a statistical test to assess whether the data was missing at random. Of the statistical methods for addressing missing data, 8 of the studies used complete case analysis (for 5 this was not reported by authors but inferred from n used in analyses). Three studies used last observation carried forward. Four studies used mixed models and stated this method was used to account for missing data. Five studies used mixed models, which can account for missing data, but did not specify how missing data was accounted for (i.e. only complete cases used or all available data). One study conducted a sensitivity analysis with those with non-valid accelerometry data (the primary analysis was inferred to be complete cases). One used a method that included an indicator variable for individuals with missing data. One study did not report any techniques for addressing missing data and no inferences were able to be made. There were no statistically significant differences in missingness between any of the study characteristics. However, due to limited power from small sample sizes, the mean estimates are presented below. There was no correlation between the number of participants entering the study and the amount of missingness (Spearman’s rho = .35, p = .10). When study size was stratified by sample size, studies with 200–500 participants had the greatest percentage of missingness (43.5%, n = 6), followed by studies with greater than 500 participant (40.2%, n = 8), and studies with less than 200 participants (31.0%, n = 9). Studies with adolescents had the greatest percentage of missingness (52.9%, n = 3), followed by young children (47.4%, n = 3), and children (33.0%, n = 17).
Studies conducted in a school setting had the highest level of missingness (42.2%, n = 13) followed by individual settings (32.8%, n = 8) and family settings (24.7%, n = 2). There was no correlation between the duration of follow-up and the amount of missingness (r = .20, p = .37). When the duration was stratified, interventions longer than 52 weeks had the highest missingness (42.2%, n = 7) followed by interventions less than 24 weeks (37.8% missingness, n = 8) and 24 to 52 weeks (33.0%, n = 8). Missingness was lower in studies where accelerometer measured PA was a primary outcome (33.2%, n = 16) compared to those where it was not (41.7%, n = 7). There was no correlation between valid wear criteria and amount of missingness (rho = .06, p = .81), compliance at baseline (.34, p = .14)), or compliance at follow-up (−.005, p = .982). Only four studies reported providing incentives for decreasing missingness as either cash incentives or regular catch up visits. Three studies used rewards as part of the intervention to decrease attrition. One study explicitly stated no financial incentives were used while the remaining did not report on the use of incentives. 4. Discussion Overall, there was a high amount of missingness in RCTs measuring physical activity using accelerometers in children. There are several implications of this missingness that researchers need to plan for and address when conducting research in order to improve the quality of intervention studies. Before intervention, researchers must use valid estimates of missingness in order to have sufficient participants at follow-up with complete data for adequate power. This is critical to maximise effectiveness of grant funding. Traditionally, missingness has been estimated using an assumption of 20% attrition.30 A brief review of sample size calculations from protocols published in the past year for trials using accelerometers with children have shown that researchers estimated between 10% 31 and 30% 32 attrition, with many estimating 20 to 25%,33,34 or no account for attrition.35 Given the current review found an average missingness of 37%, this would be an appropriate figure to use in future research planning. However, several other factors such as accelerometer type, protocol, population and setting may need to be considered.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
G Model JSAMS-1272; No. of Pages 7
ARTICLE IN PRESS E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
6
Underestimating missingness may result in inadequate power due to a smaller than anticipated analysis sample size and thus true effects of successful interventions may be missed. During an intervention, methods are needed to reduce attrition and non-compliance. Few studies included in this review reported providing incentives to reduce missingness, with monetary compensation, reminder phone calls, and visits with research staff being the strategies reported. Other strategies such as reminder phone calls or logs may help to improve compliance.36 The method for distributing and collecting the accelerometers may also influence compliance, but this was unable to be examined in the current study due to homogeneity of the distribution in-person by research staff. Regular reminder SMS messages may be a low-cost strategy, though the effectiveness of these needs to be tested. Recent studies have suggested that 24-h wear protocols or wrist-worn devices may increase weartime,37,38 which may decrease the number of days needed for reliable estimates. After the trial is completed, missingness must be addressed through attrition and outcome analyses. The randomness of missingness should be assessed by, at a minimum, comparing baseline demographic or outcome variables between those available at follow-up and those missing. Reporting these comparisons will help to describe any potential for bias from unique characteristics of those with missing data. Additionally, attrition rates between sub-populations within a sample should be compared to identify potentially underrepresented sub-populations. High missingness in certain sub-populations may require over sampling of these underrepresented populations in future studies. As there is likely to be high missingness, appropriate statistical methods should be used for handling missingness in outcome analyses, such as multiple imputation for complete data or partial data (i.e. missing epochs) or maximum likelihood estimation, which may help to reduce bias from missing data.39 However, these methods rely on the assumption that the data is missing at random, which may be unlikely. Solely using complete case analysis may bias the findings by excluding underrepresented populations with higher attrition rates.40 Missingness, including non-compliance with accelerometers, should be reported in accordance with CONSORT guidelines. This was a brief technical review and was limited to RCTs with the purpose of illustrating the existence of non-compliance, attrition, and missingness, though many other study designs are likely to have issues of missingness and thus should follow similar steps to plan for and address missingness. RCT studies have a higher probability of reporting the necessary information for extraction, as they should report attrition and non-compliance in accordance CONSORT recommendations. The included RCT’s represent multiple settings and a large range of number of participants which illustrates the presence of missingness across differing study characteristics. Missingness appears to be common among RCTs using accelerometry in children. A higher level of missingness alone should not be considered a major methodological flaw limiting its contribution to the evidence base, as this would result in bias and may further reduce available evidence in difficult to access populations. However, how the researchers anticipate, transparently report, and account for missingness should all be included when assessing the quality of a study and the confidence in its findings.
Practical implications • Missingness needs to be adequately anticipated in the powering of intervention studies using accelerometers, using a figure close to 37%. • As missingness is highly prevalent, attempts to minimize missingness should be included in study procedures.
• When analysing and reporting results, missingness should be accounted for and reported. Appendix A. Search strategy (1) (Randomised or controlled or trial or RCT or intervention).mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier]. (2) (Child* or youth or adolescen* or juvenile or teen* or infant* or boy*).tw. (3) (Physical activity or activ* or exercis* or training or sedentary or obesity or overweight or BMI).tw. (4) (Objective* or acceleromet* or activity monitor or actigraph or MTI or CSA or actical or actiheart or tritrac or unidimensional or triaxial or MVPA).tw. (5) 1 and 2 and 3 and 4. References 1. van Sluijs EM, McMinn AM, Griffin SJ. Effectiveness of interventions to promote physical activity in children and adolescents: systematic review of controlled trials. Br J Sports Med 2008; 42(8):653–657. 2. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research ed) 2010; 340:c332. 3. Lee PH, Macfarlane DJ, Lam TH. Factors associated with participant compliance in studies using accelerometers. Gait Posture 2013; 38(4):912–917. 4. Belton S, O’Brien W, Wickel EE et al. Patterns of noncompliance in adolescent field-based accelerometer research. J Phys Act Health 2013; 10(8):1181–1185. 5. Cain KL, Sallis JF, Conway TL et al. Using accelerometers in youth physical activity studies: a review of methods. J Phys Act Health 2013; 10(3):437–450. 6. Metcalf B, Henley W, Wilkin T. Effectiveness of intervention on physical activity of children: systematic review and meta-analysis of controlled trials with objectively measured outcomes (EarlyBird 54). BMJ (Clinical Research ed) 2012; 345:e5888. 7. Baranowski T, Baranowski J, Thompson D et al. Video game play, child diet, and physical activity behavior change a randomized clinical trial. Am J Prev Med 2011; 40(1):33–38. 8. Catenacci VA, Barrett C, Odgen L et al. Changes in physical activity and sedentary behavior in a randomized trial of an internet-based versus workbook-based family intervention study. J Phys Act Health 2014; 11(2):348–358. 9. Cliff DP, Okely AD, Morgan PJ et al. Movement skills and physical activity in obese children: randomized controlled trial. Med Sci Sports Exerc 2011; 43(1):90–100. 10. De Craemer M, De Decker E, Verloigne M et al. The effect of a kindergartenbased, family-involved intervention on objectively measured physical activity in Belgian preschool boys and girls of high and low SES: the ToyBox-study. Int J Behav Nutr Phys Act 2014; 11(1):38. 11. Dewar DL, Morgan PJ, Plotnikoff RC et al. Exploring changes in physical activity, sedentary behaviors and hypothesized mediators in the NEAT girls group randomized controlled trial. J Sci Med Sport 2014; 17(1):39–46. 12. Fairclough SJ, Hackett AF, Davies IG et al. Promoting healthy weight in primary school children through physical activity and nutrition education: a pragmatic evaluation of the CHANGE! randomised intervention study. BMC Public Health 2013; 13:626. 13. Fitzgibbon ML, Stolley MR, Schiffer L et al. Family-based hip-hop to health: outcome results. Obesity (Silver Spring) 2013; 21(2):274–283. 14. Grydeland M, Bergh IH, Bjelland M et al. Intervention effects on physical activity: the HEIA study—a cluster randomized controlled trial. Int J Behav Nutr Phys Act 2013; 10:17. 15. Hughes AR, Stewart L, Chapple J et al. Randomized, controlled trial of a best-practice individualized behavioral program for treatment of childhood overweight: Scottish Childhood Overweight Treatment Trial (SCOTT). Pediatrics 2008; 121(3):e539–e546. 16. Jago R, Sebire SJ, Cooper AR et al. Bristol girls dance project feasibility trial: outcome and process evaluation results. Int J Behav Nutr Phys Act 2012; 9:83. 17. Kipping RR, Howe LD, Jago R et al. Effect of intervention aimed at increasing physical activity, reducing sedentary behaviour, and increasing fruit and vegetable consumption in children: active for Life Year 5 (AFLY5) school based cluster randomised controlled trial. BMJ (Clinical Research ed) 2014; 348:g3256. 18. Kriemler S, Zahner L, Schindler C et al. Effect of school based physical activity programme (KISS) on fitness and adiposity in primary schoolchildren: cluster randomised controlled trial. BMJ (Clinical Research ed) 2010; 340:c785. 19. Maddison R, Foley L, Ni Mhurchu C et al. Effects of active video games on body composition: a randomized controlled trial. Am J Clin Nutr 2011; 94(1):156–163. 20. Magnusson KT, Sigurgeirsson I, Sveinsson T et al. Assessment of a two-year school-based physical activity intervention among 7–9-year-old children. Int J Behav Nutr Phys Act 2011; 8:138.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
G Model JSAMS-1272; No. of Pages 7
ARTICLE IN PRESS E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
21. Maloney AE, Bethea TC, Kelsey KS et al. A pilot of a video game (DDR) to promote physical activity and decrease sedentary screen time. Obesity (Silver Spring) 2008; 16(9):2074–2080. 22. Meinhardt U, Witassek F, Petro R et al. Strength training and physical activity in boys: a randomized trial. Pediatrics 2013; 132(6):1105–1111. 23. Patrick K, Calfas KJ, Norman GJ et al. Randomized controlled trial of a primary care and home-based intervention for physical activity and nutrition behaviors: PACE+ for adolescents. Arch Pediatr Adolesc Med 2006; 160(2):128–136. 24. Puder JJ, Marques-Vidal P, Schindler C et al. Effect of multidimensional lifestyle intervention on fitness and adiposity in predominantly migrant preschool children (Ballabeina): cluster randomised controlled trial. BMJ (Clinical Research ed) 2011; 343:d6195. 25. Smith JJ, Morgan PJ, Plotnikoff RC et al. Smart-phone obesity prevention trial for adolescent boys in low-income communities: the ATLAS RCT. Pediatrics 2014; 134(3):e723–e731. 26. Toftager M, Christiansen LB, Ersboll AK et al. Intervention effects on adolescent physical activity in the multicomponent SPACE study: a cluster randomized controlled trial. PLoS ONE 2014; 9(6):e99369. 27. Trost SG, Sundal D, Foster GD et al. Effects of a pediatric weight management program with and without active video games a randomized trial. JAMA Pediatr 2014; 168(5):407–413. 28. Verloigne M, Bere E, Van Lippevelde W et al. The effect of the UP4FUN pilot intervention on objectively measured sedentary time and physical activity in 10–12 year old children in Belgium: the ENERGY-project. BMC Public Health 2012; 12:805. 29. Wafa SW, Talib RA, Hamzaid NH et al. Randomized controlled trial of a good practice approach to treatment of childhood obesity in Malaysia: Malaysian Childhood Obesity Treatment Trial (MASCOT). Int J Pediatr Obes 2011; 6(2–2):e62–e69. 30. Karlson CW, Rapoff MA. Attrition in randomized controlled trials for pediatric chronic conditions. J Pediatr Psychol 2009; 34(7):782–793.
7
31. Guagliano JM, Lonsdale C, Kolt GS et al. Increasing girls’ physical activity during an organised youth sport basketball program: a randomised controlled trial protocol. BMC Public Health 2014; 14:383. 32. Larsen KT, Huang T, Moller NC et al. Effectiveness of a one-year multi-component day-camp intervention for overweight children: study protocol of the Odense overweight intervention study (OOIS). BMC Public Health 2014; 14:313. 33. O’Malley G, Clarke M, Burls A et al. A smartphone intervention for adolescent obesity: study protocol for a randomised controlled non-inferiority trial. Trials 2014; 15:43. 34. Wells NM, Myers BM, Henderson Jr CR. Study protocol: effects of school gardens on children’s physical activity. Arch Public Health 2014; 72(1):43. 35. Riley N, Lubans DR, Holmes K et al. Rationale and study protocol of the EASY Minds (Encouraging Activity to Stimulate Young Minds) program: cluster randomized controlled trial of a primary school-based physical activity integration program for mathematics. BMC Public Health 2014; 14:816. 36. Ward DS, Evenson KR, Vaughn A et al. Accelerometer use in physical activity: best practices and research recommendations. Med Sci Sports Exerc 2005; 37(11 Suppl):S582–S588. 37. Fairclough SJ, Noonan R, Rowlands AV et al. Wear compliance and activity in children wearing wrist and hip-mounted accelerometers. Med Sci Sports Exerc 2015; 48:245–253. 38. Tudor-Locke C, Barreira TV, Schuna Jr JM et al. Improving wear time compliance with a 24-hour waist-worn accelerometer protocol in the International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE). Int J Behav Nutr Phys Act 2015; 12:11. 39. Little RJ, Rubin DB. Statistical analysis with missing data, 2nd edition Hoboken, New Jersey, John Wiley & Sons, 2002. 40. Amico KR. Percent total attrition: a poor metric for study rigor in hosted intervention designs. Am J Public Health 2009; 99(9):1567–1575.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
ARTICLE IN PRESS
JSAMS-1272; No. of Pages 7
Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
Journal of Science and Medicine in Sport journal homepage: www.elsevier.com/locate/jsams
Review
Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review Erin K. Howie ∗ , Leon M. Straker School of Physiotherapy and Exercise Science, Curtin University, Australia
a r t i c l e
i n f o
Article history: Received 14 August 2015 Received in revised form 28 December 2015 Accepted 30 December 2015 Available online xxx Keywords: Sample size Research design Follow-up studies
a b s t r a c t Objectives: The purpose of this brief review was to describe the missingness, from both attrition and non-compliance, during physical activity randomized controlled trials among children which have used accelerometers to measure physical activity. Design: Systematic review. Methods: Using a previously published search strategy, an updated search of the literature was performed in the MEDLINE database for articles published from 1996 to February 2015 identifying physical activity RCTs in children (ages 2–18) measuring physical activity using accelerometers. Rates of attrition and non-compliance were extracted from identified articles. Twenty-three independent studies provided complete attrition and non-compliance data and were included. Results: The mean attrition rate was 11.5% (SD 10.1%, range 0–30.9%). The mean accelerometer noncompliance rate at baseline was 22.7% (SD 16.4%, range 1.7–67.8%) and 29.6% (SD 19.4%, range 3.3–70.1%) at follow-up. The mean total study missingness was 37.4% (SD 20.2%, range 3.3–75.4%) and ranged from 3.3% to 75.4%. There was large variation in how missingness was accounted for between studies. There were no statistically significant differences in missingness between study characteristics including sample size, participant age, intervention setting, duration of follow-up, whether physical activity was the primary outcome, and weartime compliance criteria. Conclusions: Missingness is common among randomized controlled trials using accelerometry in children and is currently handled inconsistently. Researchers must plan for high levels of missingness in study design and account for missingness in reporting and analyses of trial outcomes. © 2016 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.
1. Introduction To improve poor physical activity levels of children, high quality research trials are needed to evaluate intervention strategies.1 To improve the quality and transparency of randomized controlled trial (RCT) reporting, the Consolidated Standards of Reporting Trials (CONSORT) statement was developed in 1996.2 In physical activity intervention trials, it is commonly anticipated that all participants who begin the trial may not provide complete data at follow-up, creating ‘missingness’ in trials. The CONSORT checklist requires a flow diagram including the number of participants at each stage including the number of participants randomly assigned to treatments and those included in the final analysis. According to
∗ Corresponding author. E-mail address: [email protected] (E.K. Howie).
the CONSORT website, over 585 journals have endorsed CONSORT reporting guidelines. Thus missingness should be reported for all RCTs to enable readers to judge the quality of the evidence reported. There are two components of missingness in trials: attrition and non-compliance. For the purposes of this study, attrition is defined as those participants who entered the study and who did not remain in the study at follow-up. It is important to note, that as part of this definition, missingness does not include whether a participant followed the intervention protocol, otherwise known as adherence, but whether or not they were still available for measurement at follow-up. In addition to attrition, there is non-compliance with measurement, in this case accelerometer protocols. Noncompliance is defined as participants who remained in the study at follow-up but did not participate in outcome assessment or did not provide valid physical activity outcome data. Together, attrition and non-compliance create missingness, or those who entered the study but were not included in the final results of the study.
http://dx.doi.org/10.1016/j.jsams.2015.12.520 1440-2440/© 2016 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
G Model JSAMS-1272; No. of Pages 7
ARTICLE IN PRESS E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
2
Accelerometers are currently and commonly being used for objective measurement of physical activity in children, and thus are often the primary outcome of physical activity trials. However, noncompliance with accelerometer protocols is known to be high,3,4 though limited information is available on precise estimates. As objective measures, accelerometers require substantial participant burden compared to the majority of self-report measures. While protocols vary, participants are usually required to provide three to four days of 8 to 10 h per day to have data considered valid for youth.5 In a recent meta-analysis of controlled trials that measured physical activity using accelerometers among children,6 the median “losses to follow-up” was 11% and ranged from zero to 46%. It is unclear, however, how attrition and non-compliance specifically contributed to ‘losses to follow-up’. Additionally, issues of how researchers dealt with the missingness and if it varied by study characteristics was not explored. The combination of attrition and non-compliance with accelerometer protocols may be resulting in a significant percentage of study samples missing from the findings of physical activity studies and thus findings are at risk of biased study findings. The purpose of this review was to examine the missingness, both attrition and non-compliance, during physical activity RCTs among children which have used accelerometers to measure physical activity.
2. Methods Using the published search strategy from the EarlyBird 54 review of physical activity RCTs with objective measures of physical activity,6 an updated search of the literature was performed in the MEDLINE database. Articles were searched that had been published from February 2015 to 1996, when CONSORT reporting guidelines were established. Reference lists were cross-checked. To be included, articles had to be the primary report of the RCT, excluding duplicate or secondary analyses articles. Study design was restricted to randomized controlled trials as they are currently considered the highest quality study design and should follow CONSORT reporting guidelines, thus increasing the chance of required information being reported. Participants needed to be between the ages of 2 and eighteen. To reduce the inclusion of preliminary studies or small studies where higher than normal effort may be applied to decrease attrition and non-compliance, studies with an n < 50 were excluded. The intervention duration had to be a minimum of two weeks, thus providing a minimum duration of two weeks between baseline and follow-up measures. This was to allow for the natural process of attrition. The reported outcome (primary or secondary) of the study had to include child physical activity as measured by accelerometer. The accelerometer had to be used for measuring full day physical activity (excluding studies that only measured school day physical activity where research staff supervised accelerometer wear at school thus create an artificial weartime compliance). The same children had to be measured at two time points as part of the experimental design andtudies where a random sample of children were selected at each time point were excluded. These two previous points would lead to different samples used to calculate attrition and non-compliance, thus precluding the ability to calculate total missingness from these studies. To calculate the attrition, non-compliance and overall missingness, four numbers were extracted from the articles. • N available at baseline = participants present for baseline testing and randomized (not excluded).
• N with valid PA at baseline = participants with valid accelerometer data at baseline as defined by author. • N available at follow-up = participants who were available at the first follow-up assessment following completion of intervention delivery. If not reported, taken as measure with the highest n at follow-up. • N with valid PA at follow-up = participants with valid accelerometer wear at follow-up, only of those available at follow-up. Technical failure of accelerometers resulted in non-valid data and these participants were not included as having valid data. If the published study included some, but not complete noncompliance data, the corresponding authors were contacted and given the opportunity to provide the data. Five authors were contacted and two authors provided additional data (De Craemer, Verloigne). Four variables to assess missingness were calculated from the extracted variables using the following formulas: (1) Attrition (%) = 100 − (N available at follow-up/N available at baseline). (2) Non-compliance at baseline (%) = 100 − (N with valid PA at baseline − N available at baseline). (3) Non-compliance at follow-up (%) = 100 − (N with valid PA at follow-up − N available at follow-up). (4) Missingness (%) = 100 − (N with valid PA at follow-up − N available at baseline). The relationships between the study characteristics of sample size, participant age, intervention setting, intervention duration, whether physical activity was a primary or secondary outcome and compliance criteria (total hours needed for wear ranging from 12 h (2 days with a minimum of 6 h) to 53 h (4 days with 800 min per day) with missingness were examined using Spearman correlations for continuous data or ANOVA between stratified categorical variables. 3. Results A total of 8699 articles were retrieved and titles were screened. 272 abstracts were reviewed of which 100 full text articles were examined. 21 of the articles were excluded for using physical activity measures other than accelerometry, 15 studies were nonrandomized, 9 did only measured partial day physical activity using accelerometers, 9 measured PA in a only a subsample of participants, 5 were not the report of the primary study outcome, 3 had no measure of physical activity, 2 did not measure PA in the same children at two time-points, and one did not provide enough information to be assessed. Thirty-five original studies were eligible. Of the 35 eligible studies, twenty-three studies provided complete attrition and non-compliance data and were included in the analysis. One study only reported the total n at baseline. Seven studies did not provide compliance data at either timepoint (only reported attrition data). Four studies were missing baseline compliance data. One study reported compliance as “not enough to assess” but did not provide numbers. One study did not report follow-up numbers and the cited Appendix was unable to be obtained. Characteristics of included studies can be seen in Table 1. The average sample size was 517 (SD 565, range 60–2221). Sixteen of the studies had accelerometer measured physical activity as a primary outcome. Three studies were conducted with young children (<6 years of age), 17 were conducted with children (ages 6 to 11 years), and three were conducted with adolescents (11+). The majority (n = 13) of the interventions were conducted in a school setting. The duration ranged from 6 weeks to two school years with a mean of 34 weeks.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
Setting
location
Age (years)
Total n
Accelerometer criteria
PA outcome
Duration of intervention
Intervention
Baranowski et al.7
2011
Individual
US
10 to 12
153
4 days, 800 min/day
Primary
Home video game intervention
Catenacci et al.8
2014
Family
US
8 to 12
131 (98 families)
4 (1 weekend day), 10 h/day
Primary
Variable (9 sessions at own pace) 12 weeks
Cliff et al.9
2011
Individual
Australia
5 to 9
165
4 days, 10 h/day
Primary
6 months
De Craemer et al.10
2014
School
Belgium
4 to 6
472 (27 schools)
2 week days, 6 h/day
Primary
24 weeks
Dewar et al.11
2014
School
Australia
12 to 14
357
3 days (1 weekend), 10 h/day
Primary
12 months
Fairclough et al.12
2013
School
UK
10 to 11
318 (12 schools)
Secondary
20 weeks
Fitzgibbon et al.13
2013
Preschool
US
3 to 5
147 (4 preschools)
3 days, 540 min/week days: 480 min/weekend days 4 days, 8 h/day
Primary
14 weeks
Grydeland et al.14
2013
School
Norway
11 to 12
2165 (37 schools)
3 days (1 weekend day), 8 h/day
Primary
20 months
Hughes et al.15
2008
individual
UK
5 to 11
143
Secondary
26 weeks
Jago et al.16
2012
School
UK
11 to 12
210
Primary
9 weeks
Kipping et al.17
2014
School
UK
8 to 9
2221
3 days (1 weekend day), 6 h/day 3 weekdays, 500 min/day 3 days, 8 h/day
Primary
1 school year
Kriemler et al.18
2010
School
Switzerland
6 to 7
502 (15 schools)
2 weekdays, 12 h/day
Primary
1 school year
Maddison et al.19
2011
Individual
New Zealand
10 to 14
322
4 days (1 weekend day), 10 h/day
Secondary
24 weeks
America on the Move, small changes, messages delivered via printed workbook or internet HIKCUPS, 3 conditions: (1) 10 weeks of small group PA sessions (2) parent diet intervention (3) both Toy Box, teacher training to implement 1 h/week, PA component using ToyBox materials, resources, newsletters NEAT (Nutrition and Enjoyable Activity for Teen Girls), multi-component intervention targeting sports, lunchtime PA, nutrition workshops CHANGE!, weekly lesson plans and homework activities Hip Hop to Health, 3 day/week nutrition and aerobic lessons, parents 6 weekly classes HEIA (Health In Adolescents), multi-component intervention including lessons, exercise breaks, posters, active commuting campaigns, activity box 8, 1-to-1 behavioural program sessions delivered by pediatric dieticians Bristol Girls Dance Project, after-school dance classes AFLY5 (Active for Life Year 5), lesson plans and training for classroom teachers, homework activities KISS (Kinder-Sportstudie), multicomponent intervention including 2 additional PE lessons per week, PA breaks, PA homework Home active video games
ARTICLE IN PRESS
Date
G Model
JSAMS-1272; No. of Pages 7
Author
E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
Table 1 Intervention characteristics of included studies.
3
G Model
JSAMS-1272; No. of Pages 7
4
Date
Setting
location
Age (years)
Total n
Accelerometer criteria
PA outcome
Duration of intervention
Intervention
Magnusson et al.20
2011
School
Iceland
6 to 7
320 (6 schools)
Primary
2 years
Maloney et al.21
2008
Individual
US
7 to 8
60
Primary
28 weeks
Multi-component intervention including teacher workshops, equipment, additional PE Active video games (Dance Dance Revolution) at home
Meinhardt et al.22
2013
Individual
Switzerland
10 to 14
102
2 weekdays, 10 hrs/day and ≥85% of 6 h school day Valid data for >80% of waking hours or missing weekend day Not reported
Primary
1 school year
Patrick et al.23
2006
Individual
US
11 to 15
878
3 days, 10 h/day
Secondary
12 months
Puder et al.24
2011
School
Switzerland
4 to 5
652 (40 classes)
3 days (1 weekend day), 6 h
Secondary
1 school year
Smith et al.25
2014
School
Australia
12 to 14
361 (14 schools)
3 weekdays (1 weekend day for weekend analysis), 10 h/day
Secondary
20 weeks
Toftager et al.26
2014
School
Denmark
11 to 13
1348 (14 schools)
3 days, 10 h/day
Primary
2 years
Trost et al.27
2014
Family
US
8 to 12
75 (11 sites)
3 days, 9 h/day
Primary
16 weeks
Verloigne et al.28
2012
School
Belgium
10 to 11
740 (10 schools)
Primary
6 weeks
Wafa et al.29
2011
Individual
Malaysia
7 to 11
107
2 weekdays, 10 hrs/day: 1 weekend day, 8 h/day 4 days, 10 h/day
Secondary
26 weeks
Strength training PE lessons compared to regular PE PACE+ (Patient-Centred Assessment and Counselling For Exercise + Nutrition), primary-cared based including manual, telephone calls, mail Multicomponent curriculum including 4 × 45 min PA sessions, two workshops for teachers, environmental changes ATLAS (Active Teen Leaders Avoiding Screen-time), multicomponent intervention including teacher training, parent newsletters, enhanced school sport SPACE, multicomponent intervention with 11 components to change physical and organizational environment JOIN for ME, pediatric weight management program, 60 minute weekly sessions in parent-child small groups UP4FUN, 1–2 lessons per week conducted by classroom teachers, newsletter MASCOT (Malaysian Childhood Obesity Treatment Trial), 8 1-h group sessions
ARTICLE IN PRESS
Author
E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
Table 1 (Continued)
G Model
ARTICLE IN PRESS
JSAMS-1272; No. of Pages 7
E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
5
Table 2 Attrition and non-compliance sorted by percentage of missingness. Study
n (Available at baseline)
n (Valid PA at baseline)
n (Available at follow-up)
n (Valid PA at follow-up)
Attrition (%)
Non-compliance (baseline) (%)
Non-compliance (follow-up) (%)
Missingness (%)
Maloney et al.21 Catenacci et al.8 Meinhardt et al.22 Baranowski et al.7 Jago et al.16 Magnusson et al.20 Puder et al.24 Toftager et al.26 Kriemler et al.18 Verloigne et al.28 Wafa et al.29 Trost et al.27 Maddison et al.19 Kipping et al.17 Grydeland et al.14 Cliff et al.9 Fairclough et al.12 De Craemer et al.10 Hughes et al.15 Patrick et al.23 Smith et al.25 Fitzgibbon et al.13 Dewar et al.11
60 98 102 157 210 269 652 1348 502 740 107 75 322 2221 1580 165 318 1150 134 819 361 146 357
59 92 96 150 203 196 542 1233 352 566 86 53 297 1289 1129 137 280 653* 117 330 240 70 221
60 96 102 144 197 254 632 1159 498 732* 80 60 259 2121 1464 114 230 1074 97 690 293 143 294
58 90 93 138 181 224 484 912 335 479 65 44 187 1252 892 87 170 599* 67 330 120 46 88
0 2 0 8 6 6 3 14 1 1 25 20 20 5 7 31 28 7 28 16 19 2 18
2 6 6 4 3 27 17 9 30 24 20 29 8 42 29 17 12 43 13 60 34 52 38
3 6 9 4 8 12 23 21 33 35 19 27 28 41 39 24 26 44 31 52 59 68 70
3 8 9 12 14 17 26 32 33 35 39 41 42 44 44 47 47 48 50 60 67 68 75
*
Additional unpublished information was supplied by authors.
Individual study missingness results can be seen in Table 2. The mean attrition was 11.5% (SD 10.1%, range 0–30.9%). The mean accelerometer non-compliance at baseline was 22.7% (SD 16.4%, range 1.7–67.8%) and 29.6% (SD 19.4%, range 3.3%, 70.1%) at followup. The mean missingness was 37.4% (SD 20.2%) and ranged from 3.3% to 75.4%. Eleven studies did not report any attrition analyses. Ten compared baseline demographics or baseline values of outcome variables between those who completed from those who dropped out or did not have complete data. One study compared drop-out rates between the intervention and control groups. One study used a statistical test to assess whether the data was missing at random. Of the statistical methods for addressing missing data, 8 of the studies used complete case analysis (for 5 this was not reported by authors but inferred from n used in analyses). Three studies used last observation carried forward. Four studies used mixed models and stated this method was used to account for missing data. Five studies used mixed models, which can account for missing data, but did not specify how missing data was accounted for (i.e. only complete cases used or all available data). One study conducted a sensitivity analysis with those with non-valid accelerometry data (the primary analysis was inferred to be complete cases). One used a method that included an indicator variable for individuals with missing data. One study did not report any techniques for addressing missing data and no inferences were able to be made. There were no statistically significant differences in missingness between any of the study characteristics. However, due to limited power from small sample sizes, the mean estimates are presented below. There was no correlation between the number of participants entering the study and the amount of missingness (Spearman’s rho = .35, p = .10). When study size was stratified by sample size, studies with 200–500 participants had the greatest percentage of missingness (43.5%, n = 6), followed by studies with greater than 500 participant (40.2%, n = 8), and studies with less than 200 participants (31.0%, n = 9). Studies with adolescents had the greatest percentage of missingness (52.9%, n = 3), followed by young children (47.4%, n = 3), and children (33.0%, n = 17).
Studies conducted in a school setting had the highest level of missingness (42.2%, n = 13) followed by individual settings (32.8%, n = 8) and family settings (24.7%, n = 2). There was no correlation between the duration of follow-up and the amount of missingness (r = .20, p = .37). When the duration was stratified, interventions longer than 52 weeks had the highest missingness (42.2%, n = 7) followed by interventions less than 24 weeks (37.8% missingness, n = 8) and 24 to 52 weeks (33.0%, n = 8). Missingness was lower in studies where accelerometer measured PA was a primary outcome (33.2%, n = 16) compared to those where it was not (41.7%, n = 7). There was no correlation between valid wear criteria and amount of missingness (rho = .06, p = .81), compliance at baseline (.34, p = .14)), or compliance at follow-up (−.005, p = .982). Only four studies reported providing incentives for decreasing missingness as either cash incentives or regular catch up visits. Three studies used rewards as part of the intervention to decrease attrition. One study explicitly stated no financial incentives were used while the remaining did not report on the use of incentives. 4. Discussion Overall, there was a high amount of missingness in RCTs measuring physical activity using accelerometers in children. There are several implications of this missingness that researchers need to plan for and address when conducting research in order to improve the quality of intervention studies. Before intervention, researchers must use valid estimates of missingness in order to have sufficient participants at follow-up with complete data for adequate power. This is critical to maximise effectiveness of grant funding. Traditionally, missingness has been estimated using an assumption of 20% attrition.30 A brief review of sample size calculations from protocols published in the past year for trials using accelerometers with children have shown that researchers estimated between 10% 31 and 30% 32 attrition, with many estimating 20 to 25%,33,34 or no account for attrition.35 Given the current review found an average missingness of 37%, this would be an appropriate figure to use in future research planning. However, several other factors such as accelerometer type, protocol, population and setting may need to be considered.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
G Model JSAMS-1272; No. of Pages 7
ARTICLE IN PRESS E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
6
Underestimating missingness may result in inadequate power due to a smaller than anticipated analysis sample size and thus true effects of successful interventions may be missed. During an intervention, methods are needed to reduce attrition and non-compliance. Few studies included in this review reported providing incentives to reduce missingness, with monetary compensation, reminder phone calls, and visits with research staff being the strategies reported. Other strategies such as reminder phone calls or logs may help to improve compliance.36 The method for distributing and collecting the accelerometers may also influence compliance, but this was unable to be examined in the current study due to homogeneity of the distribution in-person by research staff. Regular reminder SMS messages may be a low-cost strategy, though the effectiveness of these needs to be tested. Recent studies have suggested that 24-h wear protocols or wrist-worn devices may increase weartime,37,38 which may decrease the number of days needed for reliable estimates. After the trial is completed, missingness must be addressed through attrition and outcome analyses. The randomness of missingness should be assessed by, at a minimum, comparing baseline demographic or outcome variables between those available at follow-up and those missing. Reporting these comparisons will help to describe any potential for bias from unique characteristics of those with missing data. Additionally, attrition rates between sub-populations within a sample should be compared to identify potentially underrepresented sub-populations. High missingness in certain sub-populations may require over sampling of these underrepresented populations in future studies. As there is likely to be high missingness, appropriate statistical methods should be used for handling missingness in outcome analyses, such as multiple imputation for complete data or partial data (i.e. missing epochs) or maximum likelihood estimation, which may help to reduce bias from missing data.39 However, these methods rely on the assumption that the data is missing at random, which may be unlikely. Solely using complete case analysis may bias the findings by excluding underrepresented populations with higher attrition rates.40 Missingness, including non-compliance with accelerometers, should be reported in accordance with CONSORT guidelines. This was a brief technical review and was limited to RCTs with the purpose of illustrating the existence of non-compliance, attrition, and missingness, though many other study designs are likely to have issues of missingness and thus should follow similar steps to plan for and address missingness. RCT studies have a higher probability of reporting the necessary information for extraction, as they should report attrition and non-compliance in accordance CONSORT recommendations. The included RCT’s represent multiple settings and a large range of number of participants which illustrates the presence of missingness across differing study characteristics. Missingness appears to be common among RCTs using accelerometry in children. A higher level of missingness alone should not be considered a major methodological flaw limiting its contribution to the evidence base, as this would result in bias and may further reduce available evidence in difficult to access populations. However, how the researchers anticipate, transparently report, and account for missingness should all be included when assessing the quality of a study and the confidence in its findings.
Practical implications • Missingness needs to be adequately anticipated in the powering of intervention studies using accelerometers, using a figure close to 37%. • As missingness is highly prevalent, attempts to minimize missingness should be included in study procedures.
• When analysing and reporting results, missingness should be accounted for and reported. Appendix A. Search strategy (1) (Randomised or controlled or trial or RCT or intervention).mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier]. (2) (Child* or youth or adolescen* or juvenile or teen* or infant* or boy*).tw. (3) (Physical activity or activ* or exercis* or training or sedentary or obesity or overweight or BMI).tw. (4) (Objective* or acceleromet* or activity monitor or actigraph or MTI or CSA or actical or actiheart or tritrac or unidimensional or triaxial or MVPA).tw. (5) 1 and 2 and 3 and 4. References 1. van Sluijs EM, McMinn AM, Griffin SJ. Effectiveness of interventions to promote physical activity in children and adolescents: systematic review of controlled trials. Br J Sports Med 2008; 42(8):653–657. 2. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research ed) 2010; 340:c332. 3. Lee PH, Macfarlane DJ, Lam TH. Factors associated with participant compliance in studies using accelerometers. Gait Posture 2013; 38(4):912–917. 4. Belton S, O’Brien W, Wickel EE et al. Patterns of noncompliance in adolescent field-based accelerometer research. J Phys Act Health 2013; 10(8):1181–1185. 5. Cain KL, Sallis JF, Conway TL et al. Using accelerometers in youth physical activity studies: a review of methods. J Phys Act Health 2013; 10(3):437–450. 6. Metcalf B, Henley W, Wilkin T. Effectiveness of intervention on physical activity of children: systematic review and meta-analysis of controlled trials with objectively measured outcomes (EarlyBird 54). BMJ (Clinical Research ed) 2012; 345:e5888. 7. Baranowski T, Baranowski J, Thompson D et al. Video game play, child diet, and physical activity behavior change a randomized clinical trial. Am J Prev Med 2011; 40(1):33–38. 8. Catenacci VA, Barrett C, Odgen L et al. Changes in physical activity and sedentary behavior in a randomized trial of an internet-based versus workbook-based family intervention study. J Phys Act Health 2014; 11(2):348–358. 9. Cliff DP, Okely AD, Morgan PJ et al. Movement skills and physical activity in obese children: randomized controlled trial. Med Sci Sports Exerc 2011; 43(1):90–100. 10. De Craemer M, De Decker E, Verloigne M et al. The effect of a kindergartenbased, family-involved intervention on objectively measured physical activity in Belgian preschool boys and girls of high and low SES: the ToyBox-study. Int J Behav Nutr Phys Act 2014; 11(1):38. 11. Dewar DL, Morgan PJ, Plotnikoff RC et al. Exploring changes in physical activity, sedentary behaviors and hypothesized mediators in the NEAT girls group randomized controlled trial. J Sci Med Sport 2014; 17(1):39–46. 12. Fairclough SJ, Hackett AF, Davies IG et al. Promoting healthy weight in primary school children through physical activity and nutrition education: a pragmatic evaluation of the CHANGE! randomised intervention study. BMC Public Health 2013; 13:626. 13. Fitzgibbon ML, Stolley MR, Schiffer L et al. Family-based hip-hop to health: outcome results. Obesity (Silver Spring) 2013; 21(2):274–283. 14. Grydeland M, Bergh IH, Bjelland M et al. Intervention effects on physical activity: the HEIA study—a cluster randomized controlled trial. Int J Behav Nutr Phys Act 2013; 10:17. 15. Hughes AR, Stewart L, Chapple J et al. Randomized, controlled trial of a best-practice individualized behavioral program for treatment of childhood overweight: Scottish Childhood Overweight Treatment Trial (SCOTT). Pediatrics 2008; 121(3):e539–e546. 16. Jago R, Sebire SJ, Cooper AR et al. Bristol girls dance project feasibility trial: outcome and process evaluation results. Int J Behav Nutr Phys Act 2012; 9:83. 17. Kipping RR, Howe LD, Jago R et al. Effect of intervention aimed at increasing physical activity, reducing sedentary behaviour, and increasing fruit and vegetable consumption in children: active for Life Year 5 (AFLY5) school based cluster randomised controlled trial. BMJ (Clinical Research ed) 2014; 348:g3256. 18. Kriemler S, Zahner L, Schindler C et al. Effect of school based physical activity programme (KISS) on fitness and adiposity in primary schoolchildren: cluster randomised controlled trial. BMJ (Clinical Research ed) 2010; 340:c785. 19. Maddison R, Foley L, Ni Mhurchu C et al. Effects of active video games on body composition: a randomized controlled trial. Am J Clin Nutr 2011; 94(1):156–163. 20. Magnusson KT, Sigurgeirsson I, Sveinsson T et al. Assessment of a two-year school-based physical activity intervention among 7–9-year-old children. Int J Behav Nutr Phys Act 2011; 8:138.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520
G Model JSAMS-1272; No. of Pages 7
ARTICLE IN PRESS E.K. Howie, L.M. Straker / Journal of Science and Medicine in Sport xxx (2016) xxx–xxx
21. Maloney AE, Bethea TC, Kelsey KS et al. A pilot of a video game (DDR) to promote physical activity and decrease sedentary screen time. Obesity (Silver Spring) 2008; 16(9):2074–2080. 22. Meinhardt U, Witassek F, Petro R et al. Strength training and physical activity in boys: a randomized trial. Pediatrics 2013; 132(6):1105–1111. 23. Patrick K, Calfas KJ, Norman GJ et al. Randomized controlled trial of a primary care and home-based intervention for physical activity and nutrition behaviors: PACE+ for adolescents. Arch Pediatr Adolesc Med 2006; 160(2):128–136. 24. Puder JJ, Marques-Vidal P, Schindler C et al. Effect of multidimensional lifestyle intervention on fitness and adiposity in predominantly migrant preschool children (Ballabeina): cluster randomised controlled trial. BMJ (Clinical Research ed) 2011; 343:d6195. 25. Smith JJ, Morgan PJ, Plotnikoff RC et al. Smart-phone obesity prevention trial for adolescent boys in low-income communities: the ATLAS RCT. Pediatrics 2014; 134(3):e723–e731. 26. Toftager M, Christiansen LB, Ersboll AK et al. Intervention effects on adolescent physical activity in the multicomponent SPACE study: a cluster randomized controlled trial. PLoS ONE 2014; 9(6):e99369. 27. Trost SG, Sundal D, Foster GD et al. Effects of a pediatric weight management program with and without active video games a randomized trial. JAMA Pediatr 2014; 168(5):407–413. 28. Verloigne M, Bere E, Van Lippevelde W et al. The effect of the UP4FUN pilot intervention on objectively measured sedentary time and physical activity in 10–12 year old children in Belgium: the ENERGY-project. BMC Public Health 2012; 12:805. 29. Wafa SW, Talib RA, Hamzaid NH et al. Randomized controlled trial of a good practice approach to treatment of childhood obesity in Malaysia: Malaysian Childhood Obesity Treatment Trial (MASCOT). Int J Pediatr Obes 2011; 6(2–2):e62–e69. 30. Karlson CW, Rapoff MA. Attrition in randomized controlled trials for pediatric chronic conditions. J Pediatr Psychol 2009; 34(7):782–793.
7
31. Guagliano JM, Lonsdale C, Kolt GS et al. Increasing girls’ physical activity during an organised youth sport basketball program: a randomised controlled trial protocol. BMC Public Health 2014; 14:383. 32. Larsen KT, Huang T, Moller NC et al. Effectiveness of a one-year multi-component day-camp intervention for overweight children: study protocol of the Odense overweight intervention study (OOIS). BMC Public Health 2014; 14:313. 33. O’Malley G, Clarke M, Burls A et al. A smartphone intervention for adolescent obesity: study protocol for a randomised controlled non-inferiority trial. Trials 2014; 15:43. 34. Wells NM, Myers BM, Henderson Jr CR. Study protocol: effects of school gardens on children’s physical activity. Arch Public Health 2014; 72(1):43. 35. Riley N, Lubans DR, Holmes K et al. Rationale and study protocol of the EASY Minds (Encouraging Activity to Stimulate Young Minds) program: cluster randomized controlled trial of a primary school-based physical activity integration program for mathematics. BMC Public Health 2014; 14:816. 36. Ward DS, Evenson KR, Vaughn A et al. Accelerometer use in physical activity: best practices and research recommendations. Med Sci Sports Exerc 2005; 37(11 Suppl):S582–S588. 37. Fairclough SJ, Noonan R, Rowlands AV et al. Wear compliance and activity in children wearing wrist and hip-mounted accelerometers. Med Sci Sports Exerc 2015; 48:245–253. 38. Tudor-Locke C, Barreira TV, Schuna Jr JM et al. Improving wear time compliance with a 24-hour waist-worn accelerometer protocol in the International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE). Int J Behav Nutr Phys Act 2015; 12:11. 39. Little RJ, Rubin DB. Statistical analysis with missing data, 2nd edition Hoboken, New Jersey, John Wiley & Sons, 2002. 40. Amico KR. Percent total attrition: a poor metric for study rigor in hosted intervention designs. Am J Public Health 2009; 99(9):1567–1575.
Please cite this article in press as: Howie EK, Straker LM. Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport (2016), http://dx.doi.org/10.1016/j.jsams.2015.12.520