- Email: [email protected]
Evaluation of a screening instrument for developmental coordination disorder
JOURNAL OF ADOLESCENT HEALTH 2004;34:308 –313
ORIGINAL ARTICLE
Evaluation of a Screening Instrument for Developmental Coordination Disorder JOHN A. HAY, Ph.D., ROBERT HAWES, M.Sc., AND BRENT E. FAUGHT, Ph.D.
Purpose: To validate the Children’s Self-Perceptions of Adequacy in, and Predilection for Physical Activity (CSAPPA) scale as a proxy for the BOTMP test in diagnosing DCD. Methods: A sample of 209 children (M ⴝ 121; F ⴝ 87) consented to the BOTMP test, CSAPPA scale, Participation Questionnaire, Le´ger 20-meter Shuttle Run, and body fat using bioelectric impedance. Receiver Operating Characteristic (ROC) curve analysis and Kappa statistic were used to validate the CSAPPA scale as a predictor for significant clumsiness on the BOTMP test. Results: Prevalence of DCD was .09 ⴞ .03 of both males and females, all previously undiagnosed. A positive cutoff of < 47 and < 53 for DCD on the CSAPPA scale was identified in male and females, respectively. Both gender cut-offs demonstrated significant agreement (p < .01) with a positive BOTMP test. Males’ results indicated a sensitivity and specificity values of .90 (CI ⴝ .18) and .89 (CI ⴝ .22). Likewise, the female subject cutoff demonstrated high sensitivity [.88 (CI ⴝ .05)] and specificity [75 (CI ⴝ .09)]. Gender specific analysis of variance (ANOVA) indicated that students identified as clumsy were not significantly different in age or height from their peers, but demonstrated significantly (p < .01) lower self-efficacy, aerobic fitness, and had significantly (p < .01) higher relative body fat. These results held true for both genders. Conclusions: These findings are consistent with the characteristics of children with DCD. These results suggest that the CSAPPA scale is a promising instrument for
From the Department of Community Health Sciences, Brock University, St. Catharines, Ontario, Canada (J.A.H., B.E.F.); and the Department of Epidemiology, Queen’s University, Kingston, Ontario, Canada (R.H.). Address correspondence to: John A. Hay, Ph.D., Professor, Department of Community Health Sciences, Brock University, St. Catharines, ON L2S 3A1, Canada. E-mail: [email protected] Manuscript accepted July 17, 2003. 1054-139X/04/$–see front matter doi:10.1016/j.jadohealth.2003.07.004
use in screening children for developmental coordination disorder. © Society for Adolescent Medicine, 2004 KEY WORDS:
Clumsy children CSAPPA scale Developmental coordination disorder Screening instrument Canada
Physically awkward children and adolescents face a host of difficulties and yet rarely receive attention. Over the past decade, an awareness of the challenges faced by these individuals has been growing, and terms such as “clumsy child syndrome” [1] or “specific developmental disorder” have been replaced by “developmental coordination disorder” (DCD) based on the recommendations from the International Consensus Meeting on Children and Clumsiness in 1994 [2]. Initially described in the Diagnostic and Statistical Manual III, the definition of DCD was revised in the DSM-IV to: “1) a marked impairment in the development of motor coordination; 2) the impairment interferes with academic achievement or activities of daily living; and 3) the coordination difficulties are not owing to a general medical condition or Pervasive Developmental Disorder” [3]. Individuals with DCD have slower movement rates [4,5], and more difficulty in visuoproprioceptual tasks [6] and reflexive and volitional processing [7] than their non-DCD peers. The prevalence of DCD in the North American school-age population is estimated from 5%–10% [3,8], however, at present only a fraction of these cases are identified. DCD has been shown to be consistently higher in males than females [4,9,10], with male–female ratios © Society for Adolescent Medicine, 2004 Published by Elsevier Inc., 360 Park Avenue South, New York, NY 10010
April 2004
ranging between 4:1 and 7:1 [8]. However, this disparity may be owing, in part, to gender biases in activity expectations [11,12] or identification via predominantly male comorbidities [13]. Children are unlikely to grow out of DCD [14,15]. During adolescence, DCD is strongly associated with learning difficulties, school failures, avoidance of physical activity, and psychological problems [16]. Clumsy children are less well-liked by their peers, more introverted, and have less self-confidence with respect to physical activities and social skills [8]. Difficulties with motor coordination are worsened when the perception of low physical competency or social exclusion results in avoidance or withdrawal from physical activity or hobbies, leading to an increase in obesity [14]. Adolescents with DCD have increased anxiety, lower self-worth, and perceive themselves as even less competent as to younger children with DCD [17]. Lack of confidence in physical aptitude influences decisions regarding physical activity in children as young as age six years [10]. Academically, children who display DCD perform significantly poorer on tests of scholastic skills and tests of writing speed compared with their peers [14]. It is apparent that early diagnosis is necessary to allow the development of interventions that may prevent subsequent negative outcomes during adolescence. There is currently no gold standard test for the diagnosis of DCD and no effective screening device for early detection [14]. DCD is diagnosed with motor competence tests such as the Bruininks-Oseretsky test (BOTMP) or the Movement-Assessment Battery for Children (M-ABC). The BOTMP is the most commonly used standardized test to diagnose DCD in North America [18]. A short form of the BOTMP has been validated for elementary school age children [19,20]. However, regardless of the version used, the major limitation of the BOTMP is the expense and time (approximately 30 –120 minutes) for administration. Trained personnel are required and only one child can be tested at a time. Consequently, the BOTMP is not a practical screening instrument and is only used for verification of a diagnosis. Hay and Missiuna reported that the Children’s Self-perceptions of Adequacy in and Predilection for Physical Activity (CSAPPA) scale had significant potential as a screening device for DCD [21]. This scale demonstrated strong sensitivity (91%) and specificity (87%) for clumsiness when compared with the BOTMP short form. The CSAPPA scale has been established previously as a feasible and reliable measure for children and adolescents. However, the
SCREENING TOOL FOR DC DISORDER
309
sensitivity and specificity calculations reported included only children scoring below the 5th and above the 95th percentile on the CSAPPA. As a result, these findings only hinted at the possible utility of the CSAPPA scale as a potential screening tool for DCD. Therefore, the purpose of this study was to determine if the CSAPPA scale was a valid screening tool for the identification of DCD in a full spectrum cohort of children. To further examine the validity, an entire school population was investigated to examine the relationship among CSAPPA scores and motor performance and overall physical activity as well as aerobic capacity and obesity, constructs strongly related to actual physical activity. Children presenting with DCD are not only motorically compromised, but are typically less physically active than their peers, resulting in poorer fitness levels and increased obesity.
Methods Design A cross-sectional design was used with a convenience sample of 206 children from a single public elementary school. After obtaining permission from the District School Board of Niagara and the principal of the elementary school, letters requesting informed consent were sent home with the children. Completion of the informed consent form was required for the child to participate.
Subjects All 258 students in grades four through eight at an elementary school in Southern Ontario, Canada were invited to participate in this study. Fourteen students with known learning disorders were able to take part in the study, but were excluded from all analyses. Two children with pre-existing physical limitations and excused from physical education class for medical reasons were excluded from the study. The students were primarily Caucasian and of middle class socioeconomic status, but came from rural and urban backgrounds. A total of 206 eligible participants (120 males, 86 females; aged 11.5 years) of a potential 242 children (85%) provided informed consent and during late spring completed all motor and fitness testing in the school gymnasium. Questionnaires were administered in the student’s classroom during regular class times before all other testing. Participants completed the CSAPPA scale, the Participation Questionnaire [22], the BOTMP short form,
310
HAY ET AL
an aerobic fitness test, and were measured for body fat, height and weight. Measures The CSAPPA scale is a 20-item scale designed to measure children’s self-perceptions of their adequacy in performing, and their desire to participate in, physical activities [22]. This self-report scale requires approximately 20 minutes to complete, and uses a structured alternative choice format to present descriptions of physical activities. For example, a child is asked to choose what most describes them between pairs of sentences such as “some kids are among the last to be chosen for active games” but “other kids are usually picked to play first” and then to indicate whether the selected sentence was either “sort of true for me” or “really true for me”. Hay designed the CSAPPA scale for children aged 9 –16 years and it has demonstrated a high test-retest reliability (r ⫽ .84 –.90) as well as strong predictive and construct validity [22,23]. The CSAPPA scale has three imbedded factors: Adequacy (confidence), Predilection (enjoyment), and Enjoyment of Physical Education class. The scale in total measures generalized self-efficacy toward physical activity. Body fat was measured using bioelectrical impedance analysis (BIA) with a portable body composition system (RJL Systems, Clinton Township, MI). BIA is an inexpensive and accurate clinical test for evaluating body fat and has been previously validated for use in children [24]. Two trained research assistants were responsible for body fat measurement. Height and weight were measured using a calibrated hospital scale and stadiometer. Aerobic capacity was evaluated using the 20-meter shuttle run test [25]. Testing was completed in the school gymnasium during the regularly scheduled physical education classes following standard protocols. Aerobic capacity expressed as the maximum volume of oxygen used during physical activity (VO2max) was predicted using the regression equation ⫺24.4 ⫹ (6.0 ⫻ maximum running speed) [25]. Motor proficiency was evaluated using the short form Bruininks-Oseretsky Test of Motor Proficiency (BOTMP-SF). This test examines the full scope of motor proficiency (static and dynamic balance, reaction time, bilateral coordination, etc.) using selected items from the full scale, The short form has been validated against the full scale with inter-correlations between .90 and .91 for children aged 8 to 14 [19]. It was designed for use when large numbers of children are to be surveyed as it takes 30 minutes to
JOURNAL OF ADOLESCENT HEALTH Vol. 34, No. 4
complete as opposed to two hours for the full version. Although this does not provide an in-depth analysis of each aspect of motor proficiency, it does provide an excellent assessment of general motor functioning. The BOTMP-SF was individually administered to each consenting child in the school’s gymnasium behind a curtained barrier to ensure confidentiality. Furthermore, the BOTMP-SF examiner was blinded to the CSAPPA scale results. The Participation Questionnaire is a 61-item questionnaire that asks children to report their actual participation levels in the areas of free-time play, seasonal recreational pursuits, school sports, community sports teams and clubs, and sports dances and lessons. Participation in organized activities encompasses a one-year period, and free play is recalled from typical pastime choices. Subtotals are available for unorganized activity (free play), organized activity (sports teams, lessons), and seasonal activity. The scale is calculated using “activity units” with each unit corresponding to a physical activity choice, sports team, sport, or dance lesson. Consistency of the Participation Questionnaire among elementary school children has been previously established with a test-retest reliability of .81 [22]. As reduced levels of physical activity are a typical characteristic of children with DCD, this provides additional diagnostic criteria.
Results Receiver-operator characteristic (ROC) curve techniques were adopted to establish positivity criterion for the CSAPPA scale. ROC curves provide a graphic means of assessing the ability of a test to discriminate between those with, and without, a condition of interest. The larger the area under the ROC curve, the more potent the diagnostic ability of the test as this represents accurately identified cases. A test providing no validity would have a straight diagonal line from the bottom left corner to top right corner. Therefore, the screening test is considered more useful the further the ROC curve is from this reference line. The ROC curves resulting from this investigation are characteristic of a very strong diagnostic tool. The ROC curves were developed using BOTMP standard scores ⬍37 as an indication of significant clumsiness. According to Bruininks [19], this score should capture 7% of the population and was used as this represents a consensus of DCD prevalence. Selection of the optimal cutoff was based on maintaining a low frequency of false-positives and false-
April 2004
Figure 1. Male CSAPPA Scale ROC Curve
negatives. Separate ROC curves were computed for male and female subjects because significant gender differences are consistently present in CSAPPA scores [22]. ROC curves are based on the proportion of subjects that are accurately identified as either positive or negative for a given condition. Specifically, the term sensitivity refers to the probability of a positive test in those who have the target condition [26]. Similarly, specificity is the term used to describe the probability of a negative test in those who do not have the target condition [26]. In this instance where the damage of a false positive is minimized by the need for further confirmatory testing and the need to avoid false negatives is substantial, cutoffs favoring high sensitivity are preferable. Male subject results indicated a cutoff score of ⬍47 on the CSAPPA scale. This proved to be highly predictive of a positive BOTMP test with sensitivity and specificity values of .90 (CI ⫽ .18) and .89 (CI ⫽ .22), respectively. Similarly, the female subject cutoff ⬍53 on the CSAPPA scale demonstrated strong sensitivity (.88 [CI ⫽ .05]) and specificity (.75 [CI ⫽ .09]). The ROC curves for the CSAPPA scale, which illustrates the trade-off between sensitivity and specificity in determining positivity criterion for male and female subjects are presented in Figures 1 and 2, respectively. Cohen’s Kappa statistic was used to evaluate the statistical accuracy of the CSAPPA scale with the BOTMP test. The CSAPPA scale demonstrated significant Kappa agreement (p ⬍ .01) with the confirmed diagnosis for developmental coordination disorder for both males and females. ANOVA was used to determine differences between male and female subjects on selected biophysical and psychosocial factors (Table 1). The results
SCREENING TOOL FOR DC DISORDER
311
Figure 2. Female CSAPPA Scale ROC Curve
indicated no significant gender differences (p ⬎ .05) with respect to age, body mass index (BMI), total activity, CSAPPA score, and BOTMP rank percentile. However, male subjects demonstrated significantly (p ⬍ .001) lower body fat and higher aerobic fitness level. Finally, ANOVA was used to determine gender differences with respect to a DCD diagnosis (Table 2). Positive DCD males demonstrated significantly (p ⬍ .01) lower CSAPPA scores and aerobic fitness compared with their non-DCD counterparts. Similarly, positive DCD females demonstrated significantly higher BMI, lower CSAPPA scores, and inferior aerobic fitness. ANOVA techniques were also used to determine if significant differences existed between adolescents (87) and preadolescents (119). These ANOVAs revealed no significant differences among age groupings for BOTMP, CSAPPA, fitness level, or body fat scores, results that are consistent with other findings. Adolescents did report significantly higher involvement in organized sports (6.2 vs. 5.2) and inactive pursuits (3.8 vs. 3.1), a result that is again consistent with their increased opportunity for both types of activity. Table 1. Subject Characteristics Age (yrs) Body fat (%) BMI (kg/m2) Total Activity CSAPPA VO2max (ml/kg/min) BOTMP (rank %)
Males (120)
Females (86)
11.5 ⫾ 1.5 12.8 ⫾ 7.1 19.6 ⫾ 3.4 17.1 ⫾ 10.7 61.1 ⫾ 12.8 36.5 ⫾ 2.0 63.6 ⫾ 30.1
11.6 ⫾ 1.5 22.6 ⫾ 8.8* 20.4 ⫾ 4.4 19.2 ⫾ 10.5 59.1 ⫾ 12.8 33.2 ⫾ 3.5* 63.8 ⫾ 29.4
* ANOVA significance (p ⬍ .001).
312
HAY ET AL
JOURNAL OF ADOLESCENT HEALTH Vol. 34, No. 4
Table 2. Differences Between DCD and Non-DCD Children by Gender Male ⫹
DCD (10) Age (yrs) Body fat (%) BMI (kg/m2) Total Activity CSAPPA VO2max (ml/ kg/min) BOTMP (Rank %)
Female ⫺
DCD (110)
⫹
DCD (7)
DCD⫺ (79)
11.4 ⫾ 1.0 15.5 ⫾ 8.1 19.6 ⫾ 5.2 12.6 ⫾ 8.8 41.9 ⫾ 11.4 31.6 ⫾ 2.1
11.5 ⫾ 1.5 12.6 ⫾ 7.0 19.5 ⫾ 3.3 17.5 ⫾ 10.8 62.9 ⫾ 11.3* 36.9 ⫾ 6.2*
11.4 ⫾ 1.4 26.9 ⫾ 14.7 23.7 ⫾ 8.8 17.8 ⫾ 9.1 40.1 ⫾ 14.4 30.1 ⫾ 1.2
11.6 ⫾ 1.5 22.2 ⫾ 8.1 20.1 ⫾ 3.7* 19.3 ⫾ 10.7 61.1 ⫾ 11.0* 33.4 ⫾ 3.5*
5.6 ⫾ 4.0
68.3 ⫾ 26.1*
3.5 ⫾ 3.7
68.4 ⫾ 25.1*
* ANOVA significance (p ⬍ .01).
Discussion This study concluded that the CSAPPA scale appears to be an accurate and feasible screening tool in identifying children with DCD when using the BOTMP as the reference standard. Building on previous research [21], the current study adopted ROC curve techniques to establish a positivity criterion score of ⬍ 47 and ⬍ 53 on the CSAPPA scale for males and females, respectively. The optimal cutoff in establishing a positivity criterion was determined by selecting the point that indicated the lowest simultaneous frequency of false-negatives and falsepositives. This conservative approach is essential during a screening from the child’s perspective as it limits both needless academic modifications as a form of intervention (false-positive) and unnecessary suffering from poor self-efficacy (false-negative) in a child that would have otherwise benefited from an effective intervention. The ROC analysis indicated that CSAPPA score ⬍ 47 and ⬍ 53 in male and female children, respectively, are considered a positive diagnosis for developmental coordination disorder. Therefore, CSAPPA scores that exceed these gender specific cut-offs should be considered negative for DCD. The results in the current study provides an expanded interpretation and confirm the findings of Hay and Missiuna [21], which examined children at the 5th and 95th percentile of CSAPPA scores. Children with low CSAPPA scores were significantly clumsier, less active, less efficacious, and less physically fit than their peers. The differences were more pronounced among boys. These findings are all consistent with a diagnosis of DCD. Although it captures some children with low selfperceptions for reasons other than DCD, the scale appears to almost completely identify those with
DCD. The sensitivity and specificity reported here indicate that should the CSAPPA scale be used to screen for DCD in a school population with 1000 boys and 1000 girls, 63 of 70 boys and 62 of 70 girls with DCD would be identified, anticipating a 7% prevalence. One-hundred boys and 123 girls would be misdiagnosed if the CSAPPA was used alone. Therefore further confirmatory testing with the BOTMP would be required with a total of 348 (17.4%) of the 2000 children being tested. This would reduce the cost of diagnosis for a school population by as much as 80% from the present situation, which requires screening of all students using the BOTMP and as such is prohibitively expensive.
Limitations This investigation was limited to a single school with 85% participation. Although this is the second investigation to report similar encouraging results, the relatively small sample size, given the prevalence of DCD, does not allow full confidence in the of the screening capabilities to be established. It is recommended that the CSAPPA scale be further validated in a large-scale multischool study of both urban and rural settings. Employing additional diagnostic criteria beyond the BOMTP will further establish the utility of the CSAPPA scale as a valid screening tool. Finally, the therapeutic ramifications of identifying DCD children via the CSAPPA scale must be further explored, because variations in treatment outcomes are highly dependent on the method of identification [27]. Until significant numbers of children with DCD can be identified at an early point, investigations of useful therapeutic interventions are severely compromised [28].
Conclusions The CSAPPA scale appears to hold significant promise for early identification and thus provides an avenue of hope for the development of therapies for this under-recognized and underserved population. The need for early identification during childhood is clear. Children do not grow out of DCD and the academic, psychosocial, and physiological costs associated with the disorder become fully realized during adolescence. The consequences of inaction become most evident during adolescence, at which point it is increasingly difficult to recover from years of unrecognized or misattributed difficulties. Without early identification intervention, programs can-
April 2004
not be implemented or evaluated, and these children will remain subject to haphazard diagnosis with the majority left to face this challenge alone. The authors wish to acknowledge the assistance of Wayne de Ruiter and Sandra Matias for data collection as well as Kellie Murphy and Glenys Jenkyns for literature contributions to this manuscript. The cooperation of the District School Board of Niagara is gratefully acknowledged. Finally, this research was supported by the Social Science and Humanities Research Council of Canada.
References 1. Adams M. Management of attention deficit disorders. J Natl Med Assoc 1983;75:187–9. 2. Polatajko H, Fox M, Missiuna C. An international consensus on children with developmental coordination disorder. Can J Occup Ther 1995;62:3–6. 3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (4th Edition). Washington, DC: American Psychiatric Association, 1994. 4. Henderson L, Rose P, Henderson S. Reaction time and movement time in children with developmental coordination disorder. J Child Psychol Psychiatry 1992;33:895–905. 5. Missiuna C. Motor skill acquisition in children with developmental coordination disorder. Adapt Phys Act Q 1994;11:214 – 35. 6. Mon-Williams MA, Wann JP, Pascal E. Visual-proprioceptive mapping in children with developmental coordination disorder. Dev Med Child Neurol 1999;41:247–54. 7. Raynor AJ. Fractioned reflex and reaction time in children with developmental coordination disorder. Motor Control 1998;2:114 –24. 8. Kadesjo B, Gillberg C. Developmental coordination disorder in Swedish 7-year-old children. J Am Acad Child Adolesc Psychiatry 1999;38:820 –8. 9. Miller LT, Missiuna CA, Macnab JJ, et al. Clinical description of children with developmental coordination disorder. Can J Occup Ther 2001;68:5–15. 10. Shoemaker MM, Kalverboer AF. Social and affective problems of children who are clumsy: How early do they begin? Adapt Phys Act Q 1994;11:130 –40. 11. Hay J, Donnelly P. Sorting out the boys from the girls: Teacher and student perceptions of student physical ability. Avante 1996;2:36 –52. 12. Henderson SE, Hall D. Concomitants of clumsiness in young school children. Dev Med Child Neurol 1982;24:448 –60.
SCREENING TOOL FOR DC DISORDER
313
13. Rappaport L, Levine MD, Aufseeser C, Incerto RA. Children’s descriptors of their developmental dysfunctions: Field testing of a self-administered student profile. Am J Dis Child 1983; 137:369 –74. 14. Dewey D, Wilson BN. Developmental coordination disorder: What is it? Phys Occup Ther Pediatr 2001;20:5–27. 15. Henderson SE, Henderson L. Toward an understanding of developmental coordination disorder. Adapt Phys Act Q 2002;19:12–31. 16. Silva PA, Ross B. Gross motor development and delays in development in early childhood: Assessment and significance. J Hum Mov Stud 1980;6:211–26. 17. Skinner RA, Piek JP. Psychosocial implications of poor motor coordination in children and adolescents. Hum Mov Sci 2001; 20:73–94. 18. Crawford SG, Wilson BN, Dewey D. Identifying developmental coordination disorder: Consistency between tests. Phys Occup Ther Pediatr 2001;20:29 –50. 19. Bruininks RH. Bruininks-Oseretsky Test of Motor Proficiency: Examiner’s Manual. Circle Pines, MN: American Guidance Service, 1978. 20. Bruininks V, Bruininks R. Motor proficiency of learning disabled and nondisabled students. Percept Mot Skills 1977;44: 1131–7. 21. Hay J, Missiuna C. Motor proficiency in children reporting low levels of participation in physical activity. Can J Occup Ther 1998;65:64 –71. 22. Hay J. Adequacy in and predilection for physical activity in children. Clin J Sports Med 1992;2:92–201. 23. Hay J. Predicting the selection of physical education class in grade ten from self-perceptions reported in grades seven eight and nine. Brock Educ 1996;6:59 –69. 24. Houtkooper LB, Lohman TG, Going SB, Howell WH. Why bioelectrical impedance analysis should be used for estimating adiposity. Am J Clin Nutr 1996;64:436S–448S. 25. Le´ ger LA, Mercier D, Gadoury C, Lambert J. The multistage 20 metre shuttle run test for aerobic fitness. J Sports Sci 1988;6: 93–101. 26. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice (2nd Edition). Toronto: Prentice-Hall Incorporated, 2000. 27. Sugden DA, Chambers ME. Intervention approaches and children with developmental coordination disorder. Pediatr Rehabil 1998;2:139 –47. 28. Holsti L, Grunau RVA, Whitfield MF. Developmental coordination disorder in extremely low birth weight children at nine years. J Dev Behav Pediatr 2002;23:9 –15.
ORIGINAL ARTICLE
Evaluation of a Screening Instrument for Developmental Coordination Disorder JOHN A. HAY, Ph.D., ROBERT HAWES, M.Sc., AND BRENT E. FAUGHT, Ph.D.
Purpose: To validate the Children’s Self-Perceptions of Adequacy in, and Predilection for Physical Activity (CSAPPA) scale as a proxy for the BOTMP test in diagnosing DCD. Methods: A sample of 209 children (M ⴝ 121; F ⴝ 87) consented to the BOTMP test, CSAPPA scale, Participation Questionnaire, Le´ger 20-meter Shuttle Run, and body fat using bioelectric impedance. Receiver Operating Characteristic (ROC) curve analysis and Kappa statistic were used to validate the CSAPPA scale as a predictor for significant clumsiness on the BOTMP test. Results: Prevalence of DCD was .09 ⴞ .03 of both males and females, all previously undiagnosed. A positive cutoff of < 47 and < 53 for DCD on the CSAPPA scale was identified in male and females, respectively. Both gender cut-offs demonstrated significant agreement (p < .01) with a positive BOTMP test. Males’ results indicated a sensitivity and specificity values of .90 (CI ⴝ .18) and .89 (CI ⴝ .22). Likewise, the female subject cutoff demonstrated high sensitivity [.88 (CI ⴝ .05)] and specificity [75 (CI ⴝ .09)]. Gender specific analysis of variance (ANOVA) indicated that students identified as clumsy were not significantly different in age or height from their peers, but demonstrated significantly (p < .01) lower self-efficacy, aerobic fitness, and had significantly (p < .01) higher relative body fat. These results held true for both genders. Conclusions: These findings are consistent with the characteristics of children with DCD. These results suggest that the CSAPPA scale is a promising instrument for
From the Department of Community Health Sciences, Brock University, St. Catharines, Ontario, Canada (J.A.H., B.E.F.); and the Department of Epidemiology, Queen’s University, Kingston, Ontario, Canada (R.H.). Address correspondence to: John A. Hay, Ph.D., Professor, Department of Community Health Sciences, Brock University, St. Catharines, ON L2S 3A1, Canada. E-mail: [email protected] Manuscript accepted July 17, 2003. 1054-139X/04/$–see front matter doi:10.1016/j.jadohealth.2003.07.004
use in screening children for developmental coordination disorder. © Society for Adolescent Medicine, 2004 KEY WORDS:
Clumsy children CSAPPA scale Developmental coordination disorder Screening instrument Canada
Physically awkward children and adolescents face a host of difficulties and yet rarely receive attention. Over the past decade, an awareness of the challenges faced by these individuals has been growing, and terms such as “clumsy child syndrome” [1] or “specific developmental disorder” have been replaced by “developmental coordination disorder” (DCD) based on the recommendations from the International Consensus Meeting on Children and Clumsiness in 1994 [2]. Initially described in the Diagnostic and Statistical Manual III, the definition of DCD was revised in the DSM-IV to: “1) a marked impairment in the development of motor coordination; 2) the impairment interferes with academic achievement or activities of daily living; and 3) the coordination difficulties are not owing to a general medical condition or Pervasive Developmental Disorder” [3]. Individuals with DCD have slower movement rates [4,5], and more difficulty in visuoproprioceptual tasks [6] and reflexive and volitional processing [7] than their non-DCD peers. The prevalence of DCD in the North American school-age population is estimated from 5%–10% [3,8], however, at present only a fraction of these cases are identified. DCD has been shown to be consistently higher in males than females [4,9,10], with male–female ratios © Society for Adolescent Medicine, 2004 Published by Elsevier Inc., 360 Park Avenue South, New York, NY 10010
April 2004
ranging between 4:1 and 7:1 [8]. However, this disparity may be owing, in part, to gender biases in activity expectations [11,12] or identification via predominantly male comorbidities [13]. Children are unlikely to grow out of DCD [14,15]. During adolescence, DCD is strongly associated with learning difficulties, school failures, avoidance of physical activity, and psychological problems [16]. Clumsy children are less well-liked by their peers, more introverted, and have less self-confidence with respect to physical activities and social skills [8]. Difficulties with motor coordination are worsened when the perception of low physical competency or social exclusion results in avoidance or withdrawal from physical activity or hobbies, leading to an increase in obesity [14]. Adolescents with DCD have increased anxiety, lower self-worth, and perceive themselves as even less competent as to younger children with DCD [17]. Lack of confidence in physical aptitude influences decisions regarding physical activity in children as young as age six years [10]. Academically, children who display DCD perform significantly poorer on tests of scholastic skills and tests of writing speed compared with their peers [14]. It is apparent that early diagnosis is necessary to allow the development of interventions that may prevent subsequent negative outcomes during adolescence. There is currently no gold standard test for the diagnosis of DCD and no effective screening device for early detection [14]. DCD is diagnosed with motor competence tests such as the Bruininks-Oseretsky test (BOTMP) or the Movement-Assessment Battery for Children (M-ABC). The BOTMP is the most commonly used standardized test to diagnose DCD in North America [18]. A short form of the BOTMP has been validated for elementary school age children [19,20]. However, regardless of the version used, the major limitation of the BOTMP is the expense and time (approximately 30 –120 minutes) for administration. Trained personnel are required and only one child can be tested at a time. Consequently, the BOTMP is not a practical screening instrument and is only used for verification of a diagnosis. Hay and Missiuna reported that the Children’s Self-perceptions of Adequacy in and Predilection for Physical Activity (CSAPPA) scale had significant potential as a screening device for DCD [21]. This scale demonstrated strong sensitivity (91%) and specificity (87%) for clumsiness when compared with the BOTMP short form. The CSAPPA scale has been established previously as a feasible and reliable measure for children and adolescents. However, the
SCREENING TOOL FOR DC DISORDER
309
sensitivity and specificity calculations reported included only children scoring below the 5th and above the 95th percentile on the CSAPPA. As a result, these findings only hinted at the possible utility of the CSAPPA scale as a potential screening tool for DCD. Therefore, the purpose of this study was to determine if the CSAPPA scale was a valid screening tool for the identification of DCD in a full spectrum cohort of children. To further examine the validity, an entire school population was investigated to examine the relationship among CSAPPA scores and motor performance and overall physical activity as well as aerobic capacity and obesity, constructs strongly related to actual physical activity. Children presenting with DCD are not only motorically compromised, but are typically less physically active than their peers, resulting in poorer fitness levels and increased obesity.
Methods Design A cross-sectional design was used with a convenience sample of 206 children from a single public elementary school. After obtaining permission from the District School Board of Niagara and the principal of the elementary school, letters requesting informed consent were sent home with the children. Completion of the informed consent form was required for the child to participate.
Subjects All 258 students in grades four through eight at an elementary school in Southern Ontario, Canada were invited to participate in this study. Fourteen students with known learning disorders were able to take part in the study, but were excluded from all analyses. Two children with pre-existing physical limitations and excused from physical education class for medical reasons were excluded from the study. The students were primarily Caucasian and of middle class socioeconomic status, but came from rural and urban backgrounds. A total of 206 eligible participants (120 males, 86 females; aged 11.5 years) of a potential 242 children (85%) provided informed consent and during late spring completed all motor and fitness testing in the school gymnasium. Questionnaires were administered in the student’s classroom during regular class times before all other testing. Participants completed the CSAPPA scale, the Participation Questionnaire [22], the BOTMP short form,
310
HAY ET AL
an aerobic fitness test, and were measured for body fat, height and weight. Measures The CSAPPA scale is a 20-item scale designed to measure children’s self-perceptions of their adequacy in performing, and their desire to participate in, physical activities [22]. This self-report scale requires approximately 20 minutes to complete, and uses a structured alternative choice format to present descriptions of physical activities. For example, a child is asked to choose what most describes them between pairs of sentences such as “some kids are among the last to be chosen for active games” but “other kids are usually picked to play first” and then to indicate whether the selected sentence was either “sort of true for me” or “really true for me”. Hay designed the CSAPPA scale for children aged 9 –16 years and it has demonstrated a high test-retest reliability (r ⫽ .84 –.90) as well as strong predictive and construct validity [22,23]. The CSAPPA scale has three imbedded factors: Adequacy (confidence), Predilection (enjoyment), and Enjoyment of Physical Education class. The scale in total measures generalized self-efficacy toward physical activity. Body fat was measured using bioelectrical impedance analysis (BIA) with a portable body composition system (RJL Systems, Clinton Township, MI). BIA is an inexpensive and accurate clinical test for evaluating body fat and has been previously validated for use in children [24]. Two trained research assistants were responsible for body fat measurement. Height and weight were measured using a calibrated hospital scale and stadiometer. Aerobic capacity was evaluated using the 20-meter shuttle run test [25]. Testing was completed in the school gymnasium during the regularly scheduled physical education classes following standard protocols. Aerobic capacity expressed as the maximum volume of oxygen used during physical activity (VO2max) was predicted using the regression equation ⫺24.4 ⫹ (6.0 ⫻ maximum running speed) [25]. Motor proficiency was evaluated using the short form Bruininks-Oseretsky Test of Motor Proficiency (BOTMP-SF). This test examines the full scope of motor proficiency (static and dynamic balance, reaction time, bilateral coordination, etc.) using selected items from the full scale, The short form has been validated against the full scale with inter-correlations between .90 and .91 for children aged 8 to 14 [19]. It was designed for use when large numbers of children are to be surveyed as it takes 30 minutes to
JOURNAL OF ADOLESCENT HEALTH Vol. 34, No. 4
complete as opposed to two hours for the full version. Although this does not provide an in-depth analysis of each aspect of motor proficiency, it does provide an excellent assessment of general motor functioning. The BOTMP-SF was individually administered to each consenting child in the school’s gymnasium behind a curtained barrier to ensure confidentiality. Furthermore, the BOTMP-SF examiner was blinded to the CSAPPA scale results. The Participation Questionnaire is a 61-item questionnaire that asks children to report their actual participation levels in the areas of free-time play, seasonal recreational pursuits, school sports, community sports teams and clubs, and sports dances and lessons. Participation in organized activities encompasses a one-year period, and free play is recalled from typical pastime choices. Subtotals are available for unorganized activity (free play), organized activity (sports teams, lessons), and seasonal activity. The scale is calculated using “activity units” with each unit corresponding to a physical activity choice, sports team, sport, or dance lesson. Consistency of the Participation Questionnaire among elementary school children has been previously established with a test-retest reliability of .81 [22]. As reduced levels of physical activity are a typical characteristic of children with DCD, this provides additional diagnostic criteria.
Results Receiver-operator characteristic (ROC) curve techniques were adopted to establish positivity criterion for the CSAPPA scale. ROC curves provide a graphic means of assessing the ability of a test to discriminate between those with, and without, a condition of interest. The larger the area under the ROC curve, the more potent the diagnostic ability of the test as this represents accurately identified cases. A test providing no validity would have a straight diagonal line from the bottom left corner to top right corner. Therefore, the screening test is considered more useful the further the ROC curve is from this reference line. The ROC curves resulting from this investigation are characteristic of a very strong diagnostic tool. The ROC curves were developed using BOTMP standard scores ⬍37 as an indication of significant clumsiness. According to Bruininks [19], this score should capture 7% of the population and was used as this represents a consensus of DCD prevalence. Selection of the optimal cutoff was based on maintaining a low frequency of false-positives and false-
April 2004
Figure 1. Male CSAPPA Scale ROC Curve
negatives. Separate ROC curves were computed for male and female subjects because significant gender differences are consistently present in CSAPPA scores [22]. ROC curves are based on the proportion of subjects that are accurately identified as either positive or negative for a given condition. Specifically, the term sensitivity refers to the probability of a positive test in those who have the target condition [26]. Similarly, specificity is the term used to describe the probability of a negative test in those who do not have the target condition [26]. In this instance where the damage of a false positive is minimized by the need for further confirmatory testing and the need to avoid false negatives is substantial, cutoffs favoring high sensitivity are preferable. Male subject results indicated a cutoff score of ⬍47 on the CSAPPA scale. This proved to be highly predictive of a positive BOTMP test with sensitivity and specificity values of .90 (CI ⫽ .18) and .89 (CI ⫽ .22), respectively. Similarly, the female subject cutoff ⬍53 on the CSAPPA scale demonstrated strong sensitivity (.88 [CI ⫽ .05]) and specificity (.75 [CI ⫽ .09]). The ROC curves for the CSAPPA scale, which illustrates the trade-off between sensitivity and specificity in determining positivity criterion for male and female subjects are presented in Figures 1 and 2, respectively. Cohen’s Kappa statistic was used to evaluate the statistical accuracy of the CSAPPA scale with the BOTMP test. The CSAPPA scale demonstrated significant Kappa agreement (p ⬍ .01) with the confirmed diagnosis for developmental coordination disorder for both males and females. ANOVA was used to determine differences between male and female subjects on selected biophysical and psychosocial factors (Table 1). The results
SCREENING TOOL FOR DC DISORDER
311
Figure 2. Female CSAPPA Scale ROC Curve
indicated no significant gender differences (p ⬎ .05) with respect to age, body mass index (BMI), total activity, CSAPPA score, and BOTMP rank percentile. However, male subjects demonstrated significantly (p ⬍ .001) lower body fat and higher aerobic fitness level. Finally, ANOVA was used to determine gender differences with respect to a DCD diagnosis (Table 2). Positive DCD males demonstrated significantly (p ⬍ .01) lower CSAPPA scores and aerobic fitness compared with their non-DCD counterparts. Similarly, positive DCD females demonstrated significantly higher BMI, lower CSAPPA scores, and inferior aerobic fitness. ANOVA techniques were also used to determine if significant differences existed between adolescents (87) and preadolescents (119). These ANOVAs revealed no significant differences among age groupings for BOTMP, CSAPPA, fitness level, or body fat scores, results that are consistent with other findings. Adolescents did report significantly higher involvement in organized sports (6.2 vs. 5.2) and inactive pursuits (3.8 vs. 3.1), a result that is again consistent with their increased opportunity for both types of activity. Table 1. Subject Characteristics Age (yrs) Body fat (%) BMI (kg/m2) Total Activity CSAPPA VO2max (ml/kg/min) BOTMP (rank %)
Males (120)
Females (86)
11.5 ⫾ 1.5 12.8 ⫾ 7.1 19.6 ⫾ 3.4 17.1 ⫾ 10.7 61.1 ⫾ 12.8 36.5 ⫾ 2.0 63.6 ⫾ 30.1
11.6 ⫾ 1.5 22.6 ⫾ 8.8* 20.4 ⫾ 4.4 19.2 ⫾ 10.5 59.1 ⫾ 12.8 33.2 ⫾ 3.5* 63.8 ⫾ 29.4
* ANOVA significance (p ⬍ .001).
312
HAY ET AL
JOURNAL OF ADOLESCENT HEALTH Vol. 34, No. 4
Table 2. Differences Between DCD and Non-DCD Children by Gender Male ⫹
DCD (10) Age (yrs) Body fat (%) BMI (kg/m2) Total Activity CSAPPA VO2max (ml/ kg/min) BOTMP (Rank %)
Female ⫺
DCD (110)
⫹
DCD (7)
DCD⫺ (79)
11.4 ⫾ 1.0 15.5 ⫾ 8.1 19.6 ⫾ 5.2 12.6 ⫾ 8.8 41.9 ⫾ 11.4 31.6 ⫾ 2.1
11.5 ⫾ 1.5 12.6 ⫾ 7.0 19.5 ⫾ 3.3 17.5 ⫾ 10.8 62.9 ⫾ 11.3* 36.9 ⫾ 6.2*
11.4 ⫾ 1.4 26.9 ⫾ 14.7 23.7 ⫾ 8.8 17.8 ⫾ 9.1 40.1 ⫾ 14.4 30.1 ⫾ 1.2
11.6 ⫾ 1.5 22.2 ⫾ 8.1 20.1 ⫾ 3.7* 19.3 ⫾ 10.7 61.1 ⫾ 11.0* 33.4 ⫾ 3.5*
5.6 ⫾ 4.0
68.3 ⫾ 26.1*
3.5 ⫾ 3.7
68.4 ⫾ 25.1*
* ANOVA significance (p ⬍ .01).
Discussion This study concluded that the CSAPPA scale appears to be an accurate and feasible screening tool in identifying children with DCD when using the BOTMP as the reference standard. Building on previous research [21], the current study adopted ROC curve techniques to establish a positivity criterion score of ⬍ 47 and ⬍ 53 on the CSAPPA scale for males and females, respectively. The optimal cutoff in establishing a positivity criterion was determined by selecting the point that indicated the lowest simultaneous frequency of false-negatives and falsepositives. This conservative approach is essential during a screening from the child’s perspective as it limits both needless academic modifications as a form of intervention (false-positive) and unnecessary suffering from poor self-efficacy (false-negative) in a child that would have otherwise benefited from an effective intervention. The ROC analysis indicated that CSAPPA score ⬍ 47 and ⬍ 53 in male and female children, respectively, are considered a positive diagnosis for developmental coordination disorder. Therefore, CSAPPA scores that exceed these gender specific cut-offs should be considered negative for DCD. The results in the current study provides an expanded interpretation and confirm the findings of Hay and Missiuna [21], which examined children at the 5th and 95th percentile of CSAPPA scores. Children with low CSAPPA scores were significantly clumsier, less active, less efficacious, and less physically fit than their peers. The differences were more pronounced among boys. These findings are all consistent with a diagnosis of DCD. Although it captures some children with low selfperceptions for reasons other than DCD, the scale appears to almost completely identify those with
DCD. The sensitivity and specificity reported here indicate that should the CSAPPA scale be used to screen for DCD in a school population with 1000 boys and 1000 girls, 63 of 70 boys and 62 of 70 girls with DCD would be identified, anticipating a 7% prevalence. One-hundred boys and 123 girls would be misdiagnosed if the CSAPPA was used alone. Therefore further confirmatory testing with the BOTMP would be required with a total of 348 (17.4%) of the 2000 children being tested. This would reduce the cost of diagnosis for a school population by as much as 80% from the present situation, which requires screening of all students using the BOTMP and as such is prohibitively expensive.
Limitations This investigation was limited to a single school with 85% participation. Although this is the second investigation to report similar encouraging results, the relatively small sample size, given the prevalence of DCD, does not allow full confidence in the of the screening capabilities to be established. It is recommended that the CSAPPA scale be further validated in a large-scale multischool study of both urban and rural settings. Employing additional diagnostic criteria beyond the BOMTP will further establish the utility of the CSAPPA scale as a valid screening tool. Finally, the therapeutic ramifications of identifying DCD children via the CSAPPA scale must be further explored, because variations in treatment outcomes are highly dependent on the method of identification [27]. Until significant numbers of children with DCD can be identified at an early point, investigations of useful therapeutic interventions are severely compromised [28].
Conclusions The CSAPPA scale appears to hold significant promise for early identification and thus provides an avenue of hope for the development of therapies for this under-recognized and underserved population. The need for early identification during childhood is clear. Children do not grow out of DCD and the academic, psychosocial, and physiological costs associated with the disorder become fully realized during adolescence. The consequences of inaction become most evident during adolescence, at which point it is increasingly difficult to recover from years of unrecognized or misattributed difficulties. Without early identification intervention, programs can-
April 2004
not be implemented or evaluated, and these children will remain subject to haphazard diagnosis with the majority left to face this challenge alone. The authors wish to acknowledge the assistance of Wayne de Ruiter and Sandra Matias for data collection as well as Kellie Murphy and Glenys Jenkyns for literature contributions to this manuscript. The cooperation of the District School Board of Niagara is gratefully acknowledged. Finally, this research was supported by the Social Science and Humanities Research Council of Canada.
References 1. Adams M. Management of attention deficit disorders. J Natl Med Assoc 1983;75:187–9. 2. Polatajko H, Fox M, Missiuna C. An international consensus on children with developmental coordination disorder. Can J Occup Ther 1995;62:3–6. 3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (4th Edition). Washington, DC: American Psychiatric Association, 1994. 4. Henderson L, Rose P, Henderson S. Reaction time and movement time in children with developmental coordination disorder. J Child Psychol Psychiatry 1992;33:895–905. 5. Missiuna C. Motor skill acquisition in children with developmental coordination disorder. Adapt Phys Act Q 1994;11:214 – 35. 6. Mon-Williams MA, Wann JP, Pascal E. Visual-proprioceptive mapping in children with developmental coordination disorder. Dev Med Child Neurol 1999;41:247–54. 7. Raynor AJ. Fractioned reflex and reaction time in children with developmental coordination disorder. Motor Control 1998;2:114 –24. 8. Kadesjo B, Gillberg C. Developmental coordination disorder in Swedish 7-year-old children. J Am Acad Child Adolesc Psychiatry 1999;38:820 –8. 9. Miller LT, Missiuna CA, Macnab JJ, et al. Clinical description of children with developmental coordination disorder. Can J Occup Ther 2001;68:5–15. 10. Shoemaker MM, Kalverboer AF. Social and affective problems of children who are clumsy: How early do they begin? Adapt Phys Act Q 1994;11:130 –40. 11. Hay J, Donnelly P. Sorting out the boys from the girls: Teacher and student perceptions of student physical ability. Avante 1996;2:36 –52. 12. Henderson SE, Hall D. Concomitants of clumsiness in young school children. Dev Med Child Neurol 1982;24:448 –60.
SCREENING TOOL FOR DC DISORDER
313
13. Rappaport L, Levine MD, Aufseeser C, Incerto RA. Children’s descriptors of their developmental dysfunctions: Field testing of a self-administered student profile. Am J Dis Child 1983; 137:369 –74. 14. Dewey D, Wilson BN. Developmental coordination disorder: What is it? Phys Occup Ther Pediatr 2001;20:5–27. 15. Henderson SE, Henderson L. Toward an understanding of developmental coordination disorder. Adapt Phys Act Q 2002;19:12–31. 16. Silva PA, Ross B. Gross motor development and delays in development in early childhood: Assessment and significance. J Hum Mov Stud 1980;6:211–26. 17. Skinner RA, Piek JP. Psychosocial implications of poor motor coordination in children and adolescents. Hum Mov Sci 2001; 20:73–94. 18. Crawford SG, Wilson BN, Dewey D. Identifying developmental coordination disorder: Consistency between tests. Phys Occup Ther Pediatr 2001;20:29 –50. 19. Bruininks RH. Bruininks-Oseretsky Test of Motor Proficiency: Examiner’s Manual. Circle Pines, MN: American Guidance Service, 1978. 20. Bruininks V, Bruininks R. Motor proficiency of learning disabled and nondisabled students. Percept Mot Skills 1977;44: 1131–7. 21. Hay J, Missiuna C. Motor proficiency in children reporting low levels of participation in physical activity. Can J Occup Ther 1998;65:64 –71. 22. Hay J. Adequacy in and predilection for physical activity in children. Clin J Sports Med 1992;2:92–201. 23. Hay J. Predicting the selection of physical education class in grade ten from self-perceptions reported in grades seven eight and nine. Brock Educ 1996;6:59 –69. 24. Houtkooper LB, Lohman TG, Going SB, Howell WH. Why bioelectrical impedance analysis should be used for estimating adiposity. Am J Clin Nutr 1996;64:436S–448S. 25. Le´ ger LA, Mercier D, Gadoury C, Lambert J. The multistage 20 metre shuttle run test for aerobic fitness. J Sports Sci 1988;6: 93–101. 26. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice (2nd Edition). Toronto: Prentice-Hall Incorporated, 2000. 27. Sugden DA, Chambers ME. Intervention approaches and children with developmental coordination disorder. Pediatr Rehabil 1998;2:139 –47. 28. Holsti L, Grunau RVA, Whitfield MF. Developmental coordination disorder in extremely low birth weight children at nine years. J Dev Behav Pediatr 2002;23:9 –15.