- Email: [email protected]
Development of a preschool developmental assessment scale for assessment of developmental disabilities
Research in Developmental Disabilities 31 (2010) 1358–1365
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Development of a preschool developmental assessment scale for assessment of developmental disabilities Cynthia Leung a,*, Rose Mak b, Vanessa Lau b, Jasmine Cheung b, Catherine Lam b a b
Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong Child Assessment Service, Department of Health, Hong Kong SAR Government, 2/F., 147L Argyle Street, Kowloon, Hong Kong
A R T I C L E I N F O
A B S T R A C T
Article history: Received 15 June 2010 Received in revised form 2 July 2010 Accepted 5 July 2010
The aim of this paper was to describe the development of the cognitive domain of the Preschool Developmental Assessment Scale (PDAS) for assessment of preschool children with developmental disabilities. The initial version of the cognitive domain consisted of 87 items. They were administered to 324 preschool children, including 240 children from preschools and 84 children with developmental disabilities. Initial Rasch analysis results indicated that the fit statistics of 42 of the items were outside the acceptable range. Based on the fit statistics and considering the overall structure of the scale, the revised version consisted of 40 items and this version conformed to the Rasch expectations. The revised 40-item scale could differentiate between children with typical development and children with developmental disabilities. It could also differentiate between children from different age groups. The internal consistency estimate (KR-20) was .93. The cognitive domain of the PDAS is considered a promising developmental assessment tool for assessment of developmental disabilities. ß 2010 Elsevier Ltd. All rights reserved.
Keywords: Developmental disability Assessment Cognitive
1. Introduction It is generally accepted that early identification and intervention are important strategies in promoting the development of children with developmental delay or disabilities. Developmental assessment is often conducted by professionals and subsequent intervention services are determined based on the assessment results (Long, Blackman, Farrell, Smolkin, & Conaway, 2005). Developmental assessment consists of ‘‘a linear progression of skills acquisition’’ (Long et al., 2005, p. 157) derived theoretically or based on research evidence. It is generally expected that developmental assessment should provide information on a child’s developmental status, and be able to contrast normal and abnormal development (Petermann, 2008). In other words, developmental assessment should demonstrate an ordering of how children progress through a certain domain (Wilson, 2008). Furthermore, children are compared against their same age peers in terms of functioning. Developmental assessment should cover a range of domains including cognitive abilities, receptive and expressive language, fine and gross motor skills, visual perception, and social-emotional skills (Petermann, 2008). There are many commonly used developmental assessment scales for different domains such as Griffiths Mental Development Scales – Revised (Griffiths, 1996), Bayley Scales of Infant and Toddler Development – Third Edition (Bayley, 2006), Reynell Developmental Language Scales (Reynell & Huntley, 1985). These scales can only be administered by trained professionals such as psychologists, speech therapists or paediatricians.
* Corresponding author. Tel.: +852 2766 4670; fax: +852 2773 6558. E-mail address: [email protected] (C. Leung). 0891-4222/$ – see front matter ß 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ridd.2010.07.004
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1359
To provide a comprehensive assessment for preschool children, it is often necessary to have combined administration of tests on different domains by respective disciplines. This is often resource intensive and sometimes not logistically viable. The aim of the present project is to develop a Preschool Developmental Assessment Scale (PDAS) which covers areas on cognitive, social, language, literacy, numeracy, visual perception, fine and gross motor skills, which can be administered by different professionals such as psychologists, speech therapists, paediatricians, physiotherapists and occupational therapists. Children being identified to have significant delays in particular domains can then be referred to professionals in the specific areas for more detailed assessment. This paper describes the development of the cognitive domain of the PDAS. Based on the literature, there are some criteria governing the development of a developmental assessment tool. In terms of the psychometric characteristics, Johnson and Marlow (2006) point out that the normative sample should be representative of the population for whom the test is designed for and the sample should be large enough so accurate comparisons can be made. Furthermore, the test should be reliable and valid, meaning that the test should yield the same results when repeatedly administered and it should measure what it is supposed to be measured. In terms of the content, Long et al. (2005) also specify that the skills measured should be based on sound theory and research evidence. 1.1. Content Sattler (2008) describes four theoretical perspectives for assessment. The developmental perspective focuses on the interplay between genetic disposition and environmental influences, and development progresses towards specific goals. Growth is conceptualized to be both qualitative (new processes or structures) and quantitative (change in degree of magnitude). The normative-developmental perspective is an extension of the developmental perspective and focuses on changes in cognition, affect and behaviour with reference to same age or sex peers. A cognitive-behavioural perspective emphasizes the importance of cognition and environment in influencing behaviour and emotion. A family-systems perspective emphasizes the importance of the family system in influencing the child’s behaviour. Sattler (2008) argues for an eclectic approach in assessment. In the development of the items for the cognitive domain of the PDAS, this study drew from several sources. The first was The Early Learning and Development Benchmarks (Kagan, Britto, Kauerz, & Tarrant, 2005) which outlines the goals for young children’s development. It integrates different theoretical perspectives with established research, which is in line with Sattler (2008) and Long et al. (2005). It is grounded in a multi-dimensional view of child learning, including physical health and motor development, social and emotional development, approaches to learning, general knowledge and cognitive development, and, language and literacy (Kagan, Britto, Kauerz, & Tarrant, 1995). The second was the Early Development Instrument (EDI) (Janus & Offord, 2007) which measures children’s readiness to begin learning at school and is completed by kindergarten teachers. It is a population level measure which cannot be used as a measure of children’s academic achievement or as a tool for identifying children with special education needs. In Canada, it is used for children between 4 and 6 years of age. The EDI has been adapted for use in Australia as a general measure of child development for children aged 4–5 (Andrich & Styles, 2004; Brinkman et al., 2007). Finally, the local preschool curriculum guide (Curriculum Development Council, Hong Kong Education and Manpower Bureau, 2006) was also used to guide the development of items as the assessment tool would need to be grounded within the local context. To map out the progress of the items and skills, the Wright map would be used. The Wright map charts the ordering of the items from the easiest to the most difficult, based on empirical data. In the Wright map, both child’s ability and item difficulty will be located on the same scale (Wilson, 2008). This would also enable us to examine the adequacy of the items in targeting the ability range of the children. 1.2. Psychometric properties For reliability, the internal consistency approach assumes that the items of an instrument should be measuring the same construct, or in other words, the items should be homogeneous (Pedhazur & Pedhazur Schmelkin, 1991). The most often used is the alpha coefficient which has been shown to be identical to Kuder-Richardson 20 (KR-20) (Pedhazur & Pedhazur Schmelkin, 1991). In terms of validity, Wilson (2008) argues that there should be a single underlying characteristic that an instrument is designed to measure, and this instrument should function as planned. This is related to the concept of unidimensionality which is a key concept in Rasch analysis. The idea of unidimensionality is also related to the concept of additivity. It is only meaningful to sum up items measuring the same construct to form a total score. Rasch analysis provides infit and outfit statistics to indicate whether the requirement of unidimensionality can be met. In relation to the functioning of the developmental assessment instrument, as stated by Petermann (2008), the instrument should be able to provide information on a child’s developmental status, and be able to contrast normal and abnormal development. In this study, Rasch analysis would be used to examine the unidimensionality of the instrument and items would be removed or modified based on Rasch analysis results and theoretical considerations. The Wright map would be used to examine the targeting of the items and the ordering of the items. Furthermore, the instrument would be used with children from different age groups to test whether it could differentiate between the age groups. In addition, children in preschools and children known to have developmental disabilities would be assessed to investigate whether this instrument could differentiate between these two groups.
1360
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
2. Method 2.1. Participants The participants included 324 children, with 108 children in each of three age groups (i.e. 3 years 4 months to 4 years 3 months, 4 years 4 months to 5 years 3 months, and 5 years 4 months to 6 years 3 months). There were 162 boys and 162 girls. Among these 324 children, 240 were recruited through eight preschools (preschool group) from four districts in Hong Kong with different socioeconomic status as measured by median monthly household income based on the 2008 Census information (Census and Statistics Department, 2009). There were 18 districts in Hong Kong and the four districts chosen included the districts with highest and lowest median monthly household income, a district that ranked 6th and another that ranked 12th in terms of median monthly household income. There were 40 boys and 40 girls from each age group. In addition, there were 48 children who attended integrated programmes (for children with special education needs) in mainstream preschools (IP group) in the four chosen districts as described above, with 8 boys and 8 girls from each age group. Furthermore, 36 children were randomly selected among clients attending Child Assessment Service (CAS) (CAS group), including 6 boys and 6 girls from each age group. The CAS provides comprehensive assessment, rehabilitation prescriptions and management services to children and families in Hong Kong (Department of Health, 2007). Children were included if they and at least one of their parents were residents of Hong Kong and they should be normally resident in Hong Kong. They and their parents should be Cantonese-speaking (Cantonese being the main dialect in Hong Kong) and the children should be currently attending preschools. Children with physical impairment or severe hearing or visual impairment were excluded. The overall sample size calculation was based on Bond and Fox (2007) where 300 was regarded as adequate for Rasch analysis. For IP children and CAS children, based on the assumption that children in IP programme or clients from CAS would be at least one standard deviation below that of preschool children with typical development in measures of development, a sample size of 21 per age group would be adequate (alpha = .05, power = .90). The current calculation (84 children with 28 in each age group) allowed for 30% wastage. 2.2. Measures The children were assessed on the Preschool Developmental Assessment Scale (PDAS) which consisted of five domains, namely, cognitive (87 items), social (29 items), language (158 items), literacy and numeracy (58 items for literacy and 25 items for numeracy), and gross and fine motor skills (18 items for gross motor, 13 items for fine motor, 77 items for visualperceptual). Each item was scored as ‘‘1’’ for correct answer/pass item and ‘‘0’’ for incorrect answer/fail item. As the focus of this paper was on the cognitive domain, the items under the cognitive domain were described in details as follows. 2.2.1. Colour concept (six items) Children were presented with different colour plates in a booklet and were required to name the colours. 2.2.2. Shape concept (10 items) Children were presented with different shapes in a booklet and were required to name them. If they could not name a shape, they would be requested to point to the correct shape later in another page. 2.2.3. Quantity (six items) Children were requested to point to the correct answer (out of three pictures) in terms of length (longest–shortest), sequence (first–last) and quantity (more–least). 2.2.4. Similarity (five items) Children were presented with three pictures and were requested to point out the two pictures which were similar. 2.2.5. Categorization (12 items) This was a matrix type question where children were presented with three pictures in a row with the 4th box empty and were asked to select a picture out of four choices that could best fit in the box. They were then requested to explain their choice. 2.2.6. Sequence (12 items) This was a matrix type question where children were presented with three pictures in a row with the 4th box empty and were asked to select a picture out of four that could best fit in the box. They were then requested to explain their choice. 2.2.7. Part–whole relationship (three items) This was a jigsaw puzzle type item where children were presented a stimulus figure with one part missing and they had to choose a picture out of four to complete the stimulus picture.
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1361
2.2.8. Inductive/deductive reasoning (three items) This was a matrix type question where children were presented with three pictures in a two-by-two matrix with the bottom right box empty and were asked to select a picture out of four that could best fit in the box. 2.2.9. Difference (three items) Children were presented with three pictures and they had to indicate and explain which one was different from the other two. 2.2.10. Body parts (10 items) Children were asked to name body parts out of a picture or to answer questions on functions of body parts. If they could not give the answer, they would be requested to point to the relevant body parts on a picture. 2.2.11. Picture recognition (five items) Children were shown five stimulus pictures and they were asked to point to the one named by the tester. 2.2.12. Comprehension (three items) Children were to answer three verbal items related to everyday events. 2.2.13. Picture story (nine items) Children were presented with a picture and they were asked various questions related to daily life activities and function of common objects. They had to answer verbally. 2.3. Procedures A multi-stage sampling method was used for the selection of children in preschools. In each district, the Education Bureau (EDB) preschool list was used as the sampling frame and two preschools were selected randomly, using a random number generator. In each selected preschool, 30 children were randomly selected, including 5 boys and 5 girls from each age group (i.e. 3 years 4 months to 4 years 3 months, 4 years 4 months to 5 years 3 months, and 5 years 4 months to 6 years 3 months). In each selected preschool, the class list was used as the sampling frame and children were selected based on random numbers generated by a random number generator. For the IP group, a similar procedure was used. In each of the four chosen districts, one preschool with IP programme was randomly selected from the EDB school list. In each selected IP school, 12 students (2 boys and 2 girls from each age group) were randomly chosen, using the class list as the sampling frame. The CAS group was randomly selected among clients attending Child Assessment Centres (CACs) during the research period. The children were assessed on the pilot PDAS by research assistants in preschools or CACs. This study was approved by the Ethics Committee of the Department of Health. 2.4. Calculation The measurement properties of the pilot version was examined using Rasch analysis using Winsteps 3.69.1 and modifications were made where appropriate, based on theory-driven consideration of the analysis results. Developmental pattern and comparison between preschool and IP/CAS sample were conducted using independent t test and analysis of variance, using Statistical Package for Social Sciences (SPSS) version 16. 3. Results Univariate analysis of variance (ANOVA) and Chi-squared test were used to examine possible differences in demographic background among the preschool group, IP group and CAS group. There was no significant difference in child sex, child age, language spoken at home, child length of residence in Hong Kong, premature birth, parent education level, parent length of residence in Hong Kong, marital status, family status and family income. The demographic characteristics of the participants are shown in Table 1. 3.1. Unidimensionality The infit and outfit mean squares of 42 items were outside the 0.70–1.30 range recommended by Bond and Fox (2007). The mean infit mean square was 1.00 (SD = 0.20) and the mean outfit mean square was 0.95 (SD = 0.51). The person separation was 4.50 and the person reliability was .95. The item separation was 9.97 and the item reliability was .99. The KR20 raw score reliability was .97. Principal component analysis (PCA) of the residuals indicated that the variance explained by measures was 47.9%. The variance explained by items was 26.5% which was 9.14 times the variance of the first contrast (2.9%). The eigen value of the first contrast was 4.8. As a general guideline, variance explained by items should be at least four times that explained by the first contrast. It is regarded as excellent if the variance explained by the first contrast is less than 5%.
1362
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
Table 1 Demographic characteristics of participants.
Sex – male Birth – full-term Birth – pre-term Birth – others Language at home – Cantonese Family status – nuclear Family status – extended Family status – others Marital status – married Mother education – 9 years or less Father education – 9 years or less Family income – HK$19,000 or below Child length of residence in Hong Kong Mother length of residence in Hong Kong Father length of residence in Hong Kong
Preschool (n = 240)
IP (n = 48)
CAS (n = 36)
Significance
120 (50%) 209 (91.7%) 11 (4.8%) 8 (3.5%) 207 (97.2%) 157 (70.1%) 29 (28.3%) 9 (3.9%) 224 (94.5%) 82 (35.3%) 78 (33.5%) 118 (53.2%) 4.16 [3.99, 4.32] 20.85 [18.68, 23.01] 32.10 [30.38, 33.82]
24 (50%) 38 (82.6%) 7 (15.2%) 1 (2.2%) 39 (97.5%) 66 (61.7%) 16 (34.0%) 1 (2.1%) 44 (95.7%) 13 (30.2%) 10 (23.3%) 23 (52.3%) 4.38 [4.07, 4.70] 20.71 [15.82, 25.60] 33.24 [29.19, 37.29]
18 (50%) 28 (87.5%) 4 (12.5%) 0 (0%) 28 (96.6%) 20 (64.5%) 8 (25.8%) 1 (3.2%) 31 (96.9%) 8 (25.0%) 8 (26.7%) 19 (63.3%) 4.50 [4.19, 4.81] 22.33 [16.34, 28.32] 32.69 [27.91, 37.47]
x2(2) = 0, p = 1.00 x2(4) = 8.8, p = .06 x2(4) = 0.45, p = 0.978 x2(4) = 8.74, p = .07 x2(10) = 4.54, p = .92 x2(2) = 1.60, p = .45 x2(2) = 2.10, p = .45 x2(2) = 1.17, p = .56 F(2,263) = 1.61, p = .35 F(2,265) = 0.12, p = .89 F(2,258) = 0.15, p = .86
3.2. Targeting and ordering of items The Wright map (item–person map) indicated that the items targeted the ability range of the participants well, except those at the lowest end, with some small gaps at the upper end. The easiest items were those related to basic concepts such as colour, shape and quantity, and the more difficult items were those related to abstract concepts such as differences, categorization and inductive/deductive reasoning. 3.3. Differential item functioning Differential item functioning (DIF) was conducted to investigate the interaction between items and groups (preschool versus IP/CAS). With the large number of items, Bonferroni adjustment was used to adjust for inflated alpha. After Bonferroni adjustment, there was no item with statistically significant DIF. The results suggested that there was no systematic item bias effect. 3.4. Revision As the infit and outfit mean square of 42 items were outside the 0.70–1.30 range, revision was indicated. The majority of these items were removed. One item (item 44) was retained so as to keep the overall structure of the other items in that subdomain. Another item (item 26) was kept because it was a difficult item which was put in the beginning of the series. It was felt that children’s performance might be less erratic should the item be placed at the end of the series. Some other items (14, 29, 37, 53, 39 and 75) within the 0.70–1.30 range were removed again to keep the overall structure of the other items in the sub-domain. Another item (item 77) was removed because it was too commonly asked. These resulted in 40 items. There were four items each for colour, shape and quantity, five items for similarity, five items for categorization, six items for sequence, three items for difference, four items for body parts and five items for comprehension/picture story. The infit mean squares of all these 40 items were within 0.70–1.30, while there were six items with outfit mean squares outside the 0.70–1.30 range. According to Bond and Fox (2007), Rasch model usually places more emphasis on infit values than do outfit values as outfit values are more sensitive to outlying scores. The mean infit mean square was 1.00 (SD = 0.24) and the mean outfit mean square was 0.94 (SD = 0.60). The person separation was 3.17 and the person reliability was .91. The item separation was 10.55 and the item reliability was .99. The KR-20 raw score reliability was .93 (95% CI: .92–.94), with the CI calculation based on Fan and Thompson (2001). PCA of the residuals indicated that the variance explained by measures was 47.7%, which was very similar to the 87-item version. The variance explained by items was 26.2% which was 7.94 times the unexplained variance of the first contrast (3.3%). The eigen value of the first contrast was 2.5. Eigen value of the first contrast being less than 3 is regarded as good. The PCA results provided further support for unidimensionality. Exploratory factor analysis (principal component with varimax rotation) was conducted to examine the factor structure of the revised scale. Using eigen value above one as the criteria, there were 10 factors accounting for 58.87% of the variance. The first factor accounted for 27.74% of the variance (eigen value = 11.10), whereas the second factor accounted for 5.56% of the variance (eigen value = 2.23). These results indicated that the scale was likely to be unifactorial (Bond & Fox, 2007). In terms of targeting, the revised version could still target the range of abilities of the participants, except a few at the lower end. There were, however, fewer items at the upper end. The item–person map is shown in Fig. 1. The correlation between the logit score and raw score of the revised version was .99. The correlation between logit score of the revised version and the raw score of the original version was .96.
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
Fig. 1. Item map for the revised 40-item version.
1363
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1364
Table 2 Mean and confidence intervals of PDAS cognitive domain items. Age
Preschool (n = 240)
IP/CAS (n = 84)
Total
Raw score
Logit score
Raw score
Logit score
Raw score
Logit score
3:4 to 4:3 4:4 to 5:3 5:4 to 6:3
16.03 [14.4, 17.64] 26.29 [24.96, 27.61] 30.09 [28.87, 31.31]
0.84 [ 1.15, 0.53] 1.05 [0.80, 1.30] 1.89 [1.60, 2.16]
14.07 [11.10, 17.05] 24.36 [21.35, 27.36] 27.21 [24.33, 30.10]
1.32 [ 2.00, 0.63] 0.77 [0.18, 1.36] 1.17 [0.48, 1.87]
15.52 [14.11, 16.92] 25.79 [24.55, 27.02] 29.34 [28.17, 30.52]
0.96 [ 1.25, 0.68] 0.98 [0.74, 1.21] 1.70 [1.42, 1.97]
Total
24.13 [23.04, 25.23]
0.70 [0.48, 0.91]
21.88 [19.83, 23.93]
0.21 [ 0.23, 0.64]
23.55 [22.58, 24.52]
0.57 [0.37, 0.77]
3.5. Developmental status ANOVA results indicated that the revised PDAS cognitive domain items could significantly differentiate between the three age groups, F(2,321) = 124.54, p = .0005 (partial h2 = 0.44). Posthoc test (scheffe) indicated that the three groups differed significantly from one another, with older children attaining higher scores. The results were similar when using logit scores. The details are shown in Table 2. To examine the differences between preschool group and the IP/CAS group, independent t test was used. The results was significant, t(322) = 2.01, p = .045 (d = 0.25). When the children were split into three age groups, there was a significant difference between the preschool group and the IP/CAS group for the 5–6-year-old group, t(106) = 2.17, p = .033 (d = 0.48). The differences between the preschool group and the IP/CAS group were not significant for the 3–4-year-old group, t(106) = 1.21, p = .229 (d = 0.27), or the 4–5-year-old group, t(106) = 1.36, p = .176 (d = 0.30). The results were similar when using logit scores. The details are shown in Table 2.
4. Discussion The results indicated that the revised PDAS cognitive domain items were consistent with the requirement for unidimensionality, and a total score could be computed by summing up the individual item scores. In terms of targeting, it could target the ability range of 3–6-year-old children. The ordering of the items indicated that the basic concepts (colour, shape and quantity) were the easier ones whereas the abstract concepts (e.g. categorization and differences) were the more difficult ones. The reliability estimate of the scale was satisfactory (above .90). In terms of validity, it could differentiate between children of different age groups, as well as children with developmental disabilities from the preschool sample (children with typical development). In the present sample, to allow for adequate sample size in each age group, about 25% were children with developmental disabilities (IP/CAS group). One may argue that there is a higher proportion of children with developmental disabilities in the present sample. Assuming normal distribution of cognitive functioning, one would expect about 16% of the sample to be one standard deviation below the mean. In our case, the sample is slightly biased with more children in the lower end. There is the possibility that the items might be targeted towards children at the lower end of the ability range. However, in clinical practice, the PDAS is likely to be used to identify children with developmental disabilities. Children who perform relatively poorly on this scale are expected to need further support and assessment. Though the cognitive domain items could differentiate IP/CAS children as a whole group from the preschool group, the differences were not significant for the 3–4-year-old group and 4–5-year-old group. However, in both cases, the IP/CAS children attained lower scores than the preschool children. The difference was significant for the 5–6-year-old group. One possible explanation is that children with mild intellectual disability may be less distinguishable from children without disability until a later age, as most of them would have developed some social and communication skills, with minimal impairment in sensory-motor areas during the preschool years (American Psychiatric Association, 1994). The results supported a unidimensional and unifactorial scale. For the Wechsler Preschool and Primary Scale of Intelligence – Third Edition (WPPSI-III), an intelligence scale for preschool children, their results supported a two-factor solution for the 2 years 6 months to 3 years 11 months age group. The results did not provide clear support for a threefactor solution for the 4–7-year-old group (Sattler, 2008). Unlike the WPPSI-III where there are verbal and non-verbal (no verbal response needed) items, the PDAS cognitive domain items require a verbal response for most of the items. In the exploratory factor analysis of the revised version, the first factor accounted for 27.74% of the variance. Based on the Spearman tradition, a large first principal component is often regarded as g or general intelligence (Ceci, Baker-Sennett, & Bronfenbrenner, 1994; Gardner & Clark, 1992). Sattler (2008) maintains that the g factor is best understood as a summary measure of the positive correlations among various ability measures, rather than an underlying cognitive factor. Though IQ is often regarded as a measure of g, it should best be regarded as a summary measure of many abilities. In terms of the ordering of the items, items requiring colour or shape naming were the easier ones whereas items requiring comparison and categorization were the more difficult ones. This is consistent with the generally accepted view that abstract-conceptual reasoning expands and increases with age (Sattler, 2008).
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1365
The present study provided some promising initial evidence on the reliability and validity of the PDAS cognitive domain. The administration time required for the original form (87 items) was about 20 min. It is expected that the revised version could be administered within about 10 min and the administration and scoring procedures are simple and straight forward. It can be administered by health professionals, psychologists and other personnel with training in child development and psychometrics. This provides a quick assessment tool for initial assessment of developmental disabilities and research on the cognitive development of preschool children. However, there are some limitations in the present study. First, test–retest reliability has not been measured. Second, the PDAS scores have not been compared with intelligence test scores such as WPPSI scores. Third, norms have not been established and there is not enough data for the establishment of cut-off points. Future studies would need to address these areas. Finally, most of the PDAS items require some verbal response and the tool might not be suitable for children with no expressive language. 5. Conclusions The PDAS is a promising developmental assessment tool for preschool children. It is a unidimensional measure with satisfactory reliability estimates. It can also differentiate between children of different age groups, as well as children with developmental disabilities. References American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. Andrich, D., & Styles, I. (2004). Report on the psychometric analysis of the Early Development Instrument (EDI) using the Rasch model Retrieved from http:// www.rch.org.au/emplibrary/australianedi/SecondRaschReport.pdf. Bayley, N. (2006). Bayley scales of infant and toddler development (3rd ed.). San Antonio, TX: Pearson Education Inc. Bond, T., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Brinkman, S. A., Silburn, S., Lawrence, D., Foldfeld, S., Sayers, M., & Oberklaid, F. (2007). Investigating the validity of the Australian Early Development Index. Early Education and Development, 18, 427–451. Ceci, S. J., Baker-Sennett, J. G., & Bronfenbrenner, U. (1994). Psychometric and everyday intelligence: Synonyms, antonyms and anonyms. In M. Rutter & D. Hay (Eds.), Development through life: A handbook for clinicians. Oxford: Blackwell Science. Census and Statistics Department. (2009). Population and household statistics analysed by district council district. Hong Kong SAR Government: Census and Statistics Department. Curriculum Development Council. (2006). Local preschool curriculum guide. Hong Kong Education and Manpower Bureau. Department of Health. (2007). Child Assessment Service Retrieved from http://www.dhcas.gov.hk/eindex.html. Fan, X., & Thompson, B. (2001). Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517–531. Gardner, M. K., & Clark, E. (1992). The psychometric perspective on intellectual development in childhood and adolescence. In R. J. Sternberg & C. A. Berg (Eds.), Intellectual development. Cambridge: Cambridge University Press. Griffiths, R. (1996). The Griffiths Mental Development Scales from birth to 2 years, manual, the 1996 revision. Henley: Association for Research in Infant and Child Development, Test Agency. Janus, M., & Offord, D. R. (2007). Development and psychometric properties of the early development instrument (EDI): A measure of children’s school readiness. Canadian Journal of Behavioural Science, 39, 1–22. Johnson, S., & Marlow, N. (2006). Developmental screen or developmental testing? Early Human Development, 82, 173–183. Kagan, S. L., Britto, P. R., Kauerz, K., & Tarrant, K. (1995). Washington State Early learning and development benchmarks, a guide to young children’s learning and development: From birth to kindergarten entry. The State of Washington. Kagan, S. L., Britto, P. R., Kauerz, K., & Tarrant, K. (2005). Early learning and developmental benchmarks: A guide to young children’s learning and development: From birth to kindergarten entry. The State of Washington. Long, C. E., Blackman, J. A., Farrell, W. J., Smolkin, M. E., & Conaway, M. R. (2005). A comparison of developmental versus functional assessment in the rehabilitation of young children. Pediatric Rehabilitation, 8, 156–161. Pedhazur, E. J., & Pedhazur Schmelkin, L. (1991). Measurement, design and analysis: An integrated approach. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Petermann, F. (2008). Developmental assessment. Zeitschrift fu¨r Psychologie/Journal of Psychology, 216, 125–126. Reynell, J., & Huntley, M. (1985). Reynell Developmental Language Scales: Revised. Windsor: NFER-Nelson. Sattler, J. M. (2008). Assessment of children: Cognitive foundations (5th ed.). San Diego: Jermoe M Sattler Publisher Inc. Wilson, M. (2008). Cognitive diagnosis using item response models. Zeitschrift fu¨r Psychologie/Journal of Psychology, 216, 74–88.
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Development of a preschool developmental assessment scale for assessment of developmental disabilities Cynthia Leung a,*, Rose Mak b, Vanessa Lau b, Jasmine Cheung b, Catherine Lam b a b
Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong Child Assessment Service, Department of Health, Hong Kong SAR Government, 2/F., 147L Argyle Street, Kowloon, Hong Kong
A R T I C L E I N F O
A B S T R A C T
Article history: Received 15 June 2010 Received in revised form 2 July 2010 Accepted 5 July 2010
The aim of this paper was to describe the development of the cognitive domain of the Preschool Developmental Assessment Scale (PDAS) for assessment of preschool children with developmental disabilities. The initial version of the cognitive domain consisted of 87 items. They were administered to 324 preschool children, including 240 children from preschools and 84 children with developmental disabilities. Initial Rasch analysis results indicated that the fit statistics of 42 of the items were outside the acceptable range. Based on the fit statistics and considering the overall structure of the scale, the revised version consisted of 40 items and this version conformed to the Rasch expectations. The revised 40-item scale could differentiate between children with typical development and children with developmental disabilities. It could also differentiate between children from different age groups. The internal consistency estimate (KR-20) was .93. The cognitive domain of the PDAS is considered a promising developmental assessment tool for assessment of developmental disabilities. ß 2010 Elsevier Ltd. All rights reserved.
Keywords: Developmental disability Assessment Cognitive
1. Introduction It is generally accepted that early identification and intervention are important strategies in promoting the development of children with developmental delay or disabilities. Developmental assessment is often conducted by professionals and subsequent intervention services are determined based on the assessment results (Long, Blackman, Farrell, Smolkin, & Conaway, 2005). Developmental assessment consists of ‘‘a linear progression of skills acquisition’’ (Long et al., 2005, p. 157) derived theoretically or based on research evidence. It is generally expected that developmental assessment should provide information on a child’s developmental status, and be able to contrast normal and abnormal development (Petermann, 2008). In other words, developmental assessment should demonstrate an ordering of how children progress through a certain domain (Wilson, 2008). Furthermore, children are compared against their same age peers in terms of functioning. Developmental assessment should cover a range of domains including cognitive abilities, receptive and expressive language, fine and gross motor skills, visual perception, and social-emotional skills (Petermann, 2008). There are many commonly used developmental assessment scales for different domains such as Griffiths Mental Development Scales – Revised (Griffiths, 1996), Bayley Scales of Infant and Toddler Development – Third Edition (Bayley, 2006), Reynell Developmental Language Scales (Reynell & Huntley, 1985). These scales can only be administered by trained professionals such as psychologists, speech therapists or paediatricians.
* Corresponding author. Tel.: +852 2766 4670; fax: +852 2773 6558. E-mail address: [email protected] (C. Leung). 0891-4222/$ – see front matter ß 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ridd.2010.07.004
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1359
To provide a comprehensive assessment for preschool children, it is often necessary to have combined administration of tests on different domains by respective disciplines. This is often resource intensive and sometimes not logistically viable. The aim of the present project is to develop a Preschool Developmental Assessment Scale (PDAS) which covers areas on cognitive, social, language, literacy, numeracy, visual perception, fine and gross motor skills, which can be administered by different professionals such as psychologists, speech therapists, paediatricians, physiotherapists and occupational therapists. Children being identified to have significant delays in particular domains can then be referred to professionals in the specific areas for more detailed assessment. This paper describes the development of the cognitive domain of the PDAS. Based on the literature, there are some criteria governing the development of a developmental assessment tool. In terms of the psychometric characteristics, Johnson and Marlow (2006) point out that the normative sample should be representative of the population for whom the test is designed for and the sample should be large enough so accurate comparisons can be made. Furthermore, the test should be reliable and valid, meaning that the test should yield the same results when repeatedly administered and it should measure what it is supposed to be measured. In terms of the content, Long et al. (2005) also specify that the skills measured should be based on sound theory and research evidence. 1.1. Content Sattler (2008) describes four theoretical perspectives for assessment. The developmental perspective focuses on the interplay between genetic disposition and environmental influences, and development progresses towards specific goals. Growth is conceptualized to be both qualitative (new processes or structures) and quantitative (change in degree of magnitude). The normative-developmental perspective is an extension of the developmental perspective and focuses on changes in cognition, affect and behaviour with reference to same age or sex peers. A cognitive-behavioural perspective emphasizes the importance of cognition and environment in influencing behaviour and emotion. A family-systems perspective emphasizes the importance of the family system in influencing the child’s behaviour. Sattler (2008) argues for an eclectic approach in assessment. In the development of the items for the cognitive domain of the PDAS, this study drew from several sources. The first was The Early Learning and Development Benchmarks (Kagan, Britto, Kauerz, & Tarrant, 2005) which outlines the goals for young children’s development. It integrates different theoretical perspectives with established research, which is in line with Sattler (2008) and Long et al. (2005). It is grounded in a multi-dimensional view of child learning, including physical health and motor development, social and emotional development, approaches to learning, general knowledge and cognitive development, and, language and literacy (Kagan, Britto, Kauerz, & Tarrant, 1995). The second was the Early Development Instrument (EDI) (Janus & Offord, 2007) which measures children’s readiness to begin learning at school and is completed by kindergarten teachers. It is a population level measure which cannot be used as a measure of children’s academic achievement or as a tool for identifying children with special education needs. In Canada, it is used for children between 4 and 6 years of age. The EDI has been adapted for use in Australia as a general measure of child development for children aged 4–5 (Andrich & Styles, 2004; Brinkman et al., 2007). Finally, the local preschool curriculum guide (Curriculum Development Council, Hong Kong Education and Manpower Bureau, 2006) was also used to guide the development of items as the assessment tool would need to be grounded within the local context. To map out the progress of the items and skills, the Wright map would be used. The Wright map charts the ordering of the items from the easiest to the most difficult, based on empirical data. In the Wright map, both child’s ability and item difficulty will be located on the same scale (Wilson, 2008). This would also enable us to examine the adequacy of the items in targeting the ability range of the children. 1.2. Psychometric properties For reliability, the internal consistency approach assumes that the items of an instrument should be measuring the same construct, or in other words, the items should be homogeneous (Pedhazur & Pedhazur Schmelkin, 1991). The most often used is the alpha coefficient which has been shown to be identical to Kuder-Richardson 20 (KR-20) (Pedhazur & Pedhazur Schmelkin, 1991). In terms of validity, Wilson (2008) argues that there should be a single underlying characteristic that an instrument is designed to measure, and this instrument should function as planned. This is related to the concept of unidimensionality which is a key concept in Rasch analysis. The idea of unidimensionality is also related to the concept of additivity. It is only meaningful to sum up items measuring the same construct to form a total score. Rasch analysis provides infit and outfit statistics to indicate whether the requirement of unidimensionality can be met. In relation to the functioning of the developmental assessment instrument, as stated by Petermann (2008), the instrument should be able to provide information on a child’s developmental status, and be able to contrast normal and abnormal development. In this study, Rasch analysis would be used to examine the unidimensionality of the instrument and items would be removed or modified based on Rasch analysis results and theoretical considerations. The Wright map would be used to examine the targeting of the items and the ordering of the items. Furthermore, the instrument would be used with children from different age groups to test whether it could differentiate between the age groups. In addition, children in preschools and children known to have developmental disabilities would be assessed to investigate whether this instrument could differentiate between these two groups.
1360
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
2. Method 2.1. Participants The participants included 324 children, with 108 children in each of three age groups (i.e. 3 years 4 months to 4 years 3 months, 4 years 4 months to 5 years 3 months, and 5 years 4 months to 6 years 3 months). There were 162 boys and 162 girls. Among these 324 children, 240 were recruited through eight preschools (preschool group) from four districts in Hong Kong with different socioeconomic status as measured by median monthly household income based on the 2008 Census information (Census and Statistics Department, 2009). There were 18 districts in Hong Kong and the four districts chosen included the districts with highest and lowest median monthly household income, a district that ranked 6th and another that ranked 12th in terms of median monthly household income. There were 40 boys and 40 girls from each age group. In addition, there were 48 children who attended integrated programmes (for children with special education needs) in mainstream preschools (IP group) in the four chosen districts as described above, with 8 boys and 8 girls from each age group. Furthermore, 36 children were randomly selected among clients attending Child Assessment Service (CAS) (CAS group), including 6 boys and 6 girls from each age group. The CAS provides comprehensive assessment, rehabilitation prescriptions and management services to children and families in Hong Kong (Department of Health, 2007). Children were included if they and at least one of their parents were residents of Hong Kong and they should be normally resident in Hong Kong. They and their parents should be Cantonese-speaking (Cantonese being the main dialect in Hong Kong) and the children should be currently attending preschools. Children with physical impairment or severe hearing or visual impairment were excluded. The overall sample size calculation was based on Bond and Fox (2007) where 300 was regarded as adequate for Rasch analysis. For IP children and CAS children, based on the assumption that children in IP programme or clients from CAS would be at least one standard deviation below that of preschool children with typical development in measures of development, a sample size of 21 per age group would be adequate (alpha = .05, power = .90). The current calculation (84 children with 28 in each age group) allowed for 30% wastage. 2.2. Measures The children were assessed on the Preschool Developmental Assessment Scale (PDAS) which consisted of five domains, namely, cognitive (87 items), social (29 items), language (158 items), literacy and numeracy (58 items for literacy and 25 items for numeracy), and gross and fine motor skills (18 items for gross motor, 13 items for fine motor, 77 items for visualperceptual). Each item was scored as ‘‘1’’ for correct answer/pass item and ‘‘0’’ for incorrect answer/fail item. As the focus of this paper was on the cognitive domain, the items under the cognitive domain were described in details as follows. 2.2.1. Colour concept (six items) Children were presented with different colour plates in a booklet and were required to name the colours. 2.2.2. Shape concept (10 items) Children were presented with different shapes in a booklet and were required to name them. If they could not name a shape, they would be requested to point to the correct shape later in another page. 2.2.3. Quantity (six items) Children were requested to point to the correct answer (out of three pictures) in terms of length (longest–shortest), sequence (first–last) and quantity (more–least). 2.2.4. Similarity (five items) Children were presented with three pictures and were requested to point out the two pictures which were similar. 2.2.5. Categorization (12 items) This was a matrix type question where children were presented with three pictures in a row with the 4th box empty and were asked to select a picture out of four choices that could best fit in the box. They were then requested to explain their choice. 2.2.6. Sequence (12 items) This was a matrix type question where children were presented with three pictures in a row with the 4th box empty and were asked to select a picture out of four that could best fit in the box. They were then requested to explain their choice. 2.2.7. Part–whole relationship (three items) This was a jigsaw puzzle type item where children were presented a stimulus figure with one part missing and they had to choose a picture out of four to complete the stimulus picture.
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1361
2.2.8. Inductive/deductive reasoning (three items) This was a matrix type question where children were presented with three pictures in a two-by-two matrix with the bottom right box empty and were asked to select a picture out of four that could best fit in the box. 2.2.9. Difference (three items) Children were presented with three pictures and they had to indicate and explain which one was different from the other two. 2.2.10. Body parts (10 items) Children were asked to name body parts out of a picture or to answer questions on functions of body parts. If they could not give the answer, they would be requested to point to the relevant body parts on a picture. 2.2.11. Picture recognition (five items) Children were shown five stimulus pictures and they were asked to point to the one named by the tester. 2.2.12. Comprehension (three items) Children were to answer three verbal items related to everyday events. 2.2.13. Picture story (nine items) Children were presented with a picture and they were asked various questions related to daily life activities and function of common objects. They had to answer verbally. 2.3. Procedures A multi-stage sampling method was used for the selection of children in preschools. In each district, the Education Bureau (EDB) preschool list was used as the sampling frame and two preschools were selected randomly, using a random number generator. In each selected preschool, 30 children were randomly selected, including 5 boys and 5 girls from each age group (i.e. 3 years 4 months to 4 years 3 months, 4 years 4 months to 5 years 3 months, and 5 years 4 months to 6 years 3 months). In each selected preschool, the class list was used as the sampling frame and children were selected based on random numbers generated by a random number generator. For the IP group, a similar procedure was used. In each of the four chosen districts, one preschool with IP programme was randomly selected from the EDB school list. In each selected IP school, 12 students (2 boys and 2 girls from each age group) were randomly chosen, using the class list as the sampling frame. The CAS group was randomly selected among clients attending Child Assessment Centres (CACs) during the research period. The children were assessed on the pilot PDAS by research assistants in preschools or CACs. This study was approved by the Ethics Committee of the Department of Health. 2.4. Calculation The measurement properties of the pilot version was examined using Rasch analysis using Winsteps 3.69.1 and modifications were made where appropriate, based on theory-driven consideration of the analysis results. Developmental pattern and comparison between preschool and IP/CAS sample were conducted using independent t test and analysis of variance, using Statistical Package for Social Sciences (SPSS) version 16. 3. Results Univariate analysis of variance (ANOVA) and Chi-squared test were used to examine possible differences in demographic background among the preschool group, IP group and CAS group. There was no significant difference in child sex, child age, language spoken at home, child length of residence in Hong Kong, premature birth, parent education level, parent length of residence in Hong Kong, marital status, family status and family income. The demographic characteristics of the participants are shown in Table 1. 3.1. Unidimensionality The infit and outfit mean squares of 42 items were outside the 0.70–1.30 range recommended by Bond and Fox (2007). The mean infit mean square was 1.00 (SD = 0.20) and the mean outfit mean square was 0.95 (SD = 0.51). The person separation was 4.50 and the person reliability was .95. The item separation was 9.97 and the item reliability was .99. The KR20 raw score reliability was .97. Principal component analysis (PCA) of the residuals indicated that the variance explained by measures was 47.9%. The variance explained by items was 26.5% which was 9.14 times the variance of the first contrast (2.9%). The eigen value of the first contrast was 4.8. As a general guideline, variance explained by items should be at least four times that explained by the first contrast. It is regarded as excellent if the variance explained by the first contrast is less than 5%.
1362
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
Table 1 Demographic characteristics of participants.
Sex – male Birth – full-term Birth – pre-term Birth – others Language at home – Cantonese Family status – nuclear Family status – extended Family status – others Marital status – married Mother education – 9 years or less Father education – 9 years or less Family income – HK$19,000 or below Child length of residence in Hong Kong Mother length of residence in Hong Kong Father length of residence in Hong Kong
Preschool (n = 240)
IP (n = 48)
CAS (n = 36)
Significance
120 (50%) 209 (91.7%) 11 (4.8%) 8 (3.5%) 207 (97.2%) 157 (70.1%) 29 (28.3%) 9 (3.9%) 224 (94.5%) 82 (35.3%) 78 (33.5%) 118 (53.2%) 4.16 [3.99, 4.32] 20.85 [18.68, 23.01] 32.10 [30.38, 33.82]
24 (50%) 38 (82.6%) 7 (15.2%) 1 (2.2%) 39 (97.5%) 66 (61.7%) 16 (34.0%) 1 (2.1%) 44 (95.7%) 13 (30.2%) 10 (23.3%) 23 (52.3%) 4.38 [4.07, 4.70] 20.71 [15.82, 25.60] 33.24 [29.19, 37.29]
18 (50%) 28 (87.5%) 4 (12.5%) 0 (0%) 28 (96.6%) 20 (64.5%) 8 (25.8%) 1 (3.2%) 31 (96.9%) 8 (25.0%) 8 (26.7%) 19 (63.3%) 4.50 [4.19, 4.81] 22.33 [16.34, 28.32] 32.69 [27.91, 37.47]
x2(2) = 0, p = 1.00 x2(4) = 8.8, p = .06 x2(4) = 0.45, p = 0.978 x2(4) = 8.74, p = .07 x2(10) = 4.54, p = .92 x2(2) = 1.60, p = .45 x2(2) = 2.10, p = .45 x2(2) = 1.17, p = .56 F(2,263) = 1.61, p = .35 F(2,265) = 0.12, p = .89 F(2,258) = 0.15, p = .86
3.2. Targeting and ordering of items The Wright map (item–person map) indicated that the items targeted the ability range of the participants well, except those at the lowest end, with some small gaps at the upper end. The easiest items were those related to basic concepts such as colour, shape and quantity, and the more difficult items were those related to abstract concepts such as differences, categorization and inductive/deductive reasoning. 3.3. Differential item functioning Differential item functioning (DIF) was conducted to investigate the interaction between items and groups (preschool versus IP/CAS). With the large number of items, Bonferroni adjustment was used to adjust for inflated alpha. After Bonferroni adjustment, there was no item with statistically significant DIF. The results suggested that there was no systematic item bias effect. 3.4. Revision As the infit and outfit mean square of 42 items were outside the 0.70–1.30 range, revision was indicated. The majority of these items were removed. One item (item 44) was retained so as to keep the overall structure of the other items in that subdomain. Another item (item 26) was kept because it was a difficult item which was put in the beginning of the series. It was felt that children’s performance might be less erratic should the item be placed at the end of the series. Some other items (14, 29, 37, 53, 39 and 75) within the 0.70–1.30 range were removed again to keep the overall structure of the other items in the sub-domain. Another item (item 77) was removed because it was too commonly asked. These resulted in 40 items. There were four items each for colour, shape and quantity, five items for similarity, five items for categorization, six items for sequence, three items for difference, four items for body parts and five items for comprehension/picture story. The infit mean squares of all these 40 items were within 0.70–1.30, while there were six items with outfit mean squares outside the 0.70–1.30 range. According to Bond and Fox (2007), Rasch model usually places more emphasis on infit values than do outfit values as outfit values are more sensitive to outlying scores. The mean infit mean square was 1.00 (SD = 0.24) and the mean outfit mean square was 0.94 (SD = 0.60). The person separation was 3.17 and the person reliability was .91. The item separation was 10.55 and the item reliability was .99. The KR-20 raw score reliability was .93 (95% CI: .92–.94), with the CI calculation based on Fan and Thompson (2001). PCA of the residuals indicated that the variance explained by measures was 47.7%, which was very similar to the 87-item version. The variance explained by items was 26.2% which was 7.94 times the unexplained variance of the first contrast (3.3%). The eigen value of the first contrast was 2.5. Eigen value of the first contrast being less than 3 is regarded as good. The PCA results provided further support for unidimensionality. Exploratory factor analysis (principal component with varimax rotation) was conducted to examine the factor structure of the revised scale. Using eigen value above one as the criteria, there were 10 factors accounting for 58.87% of the variance. The first factor accounted for 27.74% of the variance (eigen value = 11.10), whereas the second factor accounted for 5.56% of the variance (eigen value = 2.23). These results indicated that the scale was likely to be unifactorial (Bond & Fox, 2007). In terms of targeting, the revised version could still target the range of abilities of the participants, except a few at the lower end. There were, however, fewer items at the upper end. The item–person map is shown in Fig. 1. The correlation between the logit score and raw score of the revised version was .99. The correlation between logit score of the revised version and the raw score of the original version was .96.
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
Fig. 1. Item map for the revised 40-item version.
1363
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1364
Table 2 Mean and confidence intervals of PDAS cognitive domain items. Age
Preschool (n = 240)
IP/CAS (n = 84)
Total
Raw score
Logit score
Raw score
Logit score
Raw score
Logit score
3:4 to 4:3 4:4 to 5:3 5:4 to 6:3
16.03 [14.4, 17.64] 26.29 [24.96, 27.61] 30.09 [28.87, 31.31]
0.84 [ 1.15, 0.53] 1.05 [0.80, 1.30] 1.89 [1.60, 2.16]
14.07 [11.10, 17.05] 24.36 [21.35, 27.36] 27.21 [24.33, 30.10]
1.32 [ 2.00, 0.63] 0.77 [0.18, 1.36] 1.17 [0.48, 1.87]
15.52 [14.11, 16.92] 25.79 [24.55, 27.02] 29.34 [28.17, 30.52]
0.96 [ 1.25, 0.68] 0.98 [0.74, 1.21] 1.70 [1.42, 1.97]
Total
24.13 [23.04, 25.23]
0.70 [0.48, 0.91]
21.88 [19.83, 23.93]
0.21 [ 0.23, 0.64]
23.55 [22.58, 24.52]
0.57 [0.37, 0.77]
3.5. Developmental status ANOVA results indicated that the revised PDAS cognitive domain items could significantly differentiate between the three age groups, F(2,321) = 124.54, p = .0005 (partial h2 = 0.44). Posthoc test (scheffe) indicated that the three groups differed significantly from one another, with older children attaining higher scores. The results were similar when using logit scores. The details are shown in Table 2. To examine the differences between preschool group and the IP/CAS group, independent t test was used. The results was significant, t(322) = 2.01, p = .045 (d = 0.25). When the children were split into three age groups, there was a significant difference between the preschool group and the IP/CAS group for the 5–6-year-old group, t(106) = 2.17, p = .033 (d = 0.48). The differences between the preschool group and the IP/CAS group were not significant for the 3–4-year-old group, t(106) = 1.21, p = .229 (d = 0.27), or the 4–5-year-old group, t(106) = 1.36, p = .176 (d = 0.30). The results were similar when using logit scores. The details are shown in Table 2.
4. Discussion The results indicated that the revised PDAS cognitive domain items were consistent with the requirement for unidimensionality, and a total score could be computed by summing up the individual item scores. In terms of targeting, it could target the ability range of 3–6-year-old children. The ordering of the items indicated that the basic concepts (colour, shape and quantity) were the easier ones whereas the abstract concepts (e.g. categorization and differences) were the more difficult ones. The reliability estimate of the scale was satisfactory (above .90). In terms of validity, it could differentiate between children of different age groups, as well as children with developmental disabilities from the preschool sample (children with typical development). In the present sample, to allow for adequate sample size in each age group, about 25% were children with developmental disabilities (IP/CAS group). One may argue that there is a higher proportion of children with developmental disabilities in the present sample. Assuming normal distribution of cognitive functioning, one would expect about 16% of the sample to be one standard deviation below the mean. In our case, the sample is slightly biased with more children in the lower end. There is the possibility that the items might be targeted towards children at the lower end of the ability range. However, in clinical practice, the PDAS is likely to be used to identify children with developmental disabilities. Children who perform relatively poorly on this scale are expected to need further support and assessment. Though the cognitive domain items could differentiate IP/CAS children as a whole group from the preschool group, the differences were not significant for the 3–4-year-old group and 4–5-year-old group. However, in both cases, the IP/CAS children attained lower scores than the preschool children. The difference was significant for the 5–6-year-old group. One possible explanation is that children with mild intellectual disability may be less distinguishable from children without disability until a later age, as most of them would have developed some social and communication skills, with minimal impairment in sensory-motor areas during the preschool years (American Psychiatric Association, 1994). The results supported a unidimensional and unifactorial scale. For the Wechsler Preschool and Primary Scale of Intelligence – Third Edition (WPPSI-III), an intelligence scale for preschool children, their results supported a two-factor solution for the 2 years 6 months to 3 years 11 months age group. The results did not provide clear support for a threefactor solution for the 4–7-year-old group (Sattler, 2008). Unlike the WPPSI-III where there are verbal and non-verbal (no verbal response needed) items, the PDAS cognitive domain items require a verbal response for most of the items. In the exploratory factor analysis of the revised version, the first factor accounted for 27.74% of the variance. Based on the Spearman tradition, a large first principal component is often regarded as g or general intelligence (Ceci, Baker-Sennett, & Bronfenbrenner, 1994; Gardner & Clark, 1992). Sattler (2008) maintains that the g factor is best understood as a summary measure of the positive correlations among various ability measures, rather than an underlying cognitive factor. Though IQ is often regarded as a measure of g, it should best be regarded as a summary measure of many abilities. In terms of the ordering of the items, items requiring colour or shape naming were the easier ones whereas items requiring comparison and categorization were the more difficult ones. This is consistent with the generally accepted view that abstract-conceptual reasoning expands and increases with age (Sattler, 2008).
C. Leung et al. / Research in Developmental Disabilities 31 (2010) 1358–1365
1365
The present study provided some promising initial evidence on the reliability and validity of the PDAS cognitive domain. The administration time required for the original form (87 items) was about 20 min. It is expected that the revised version could be administered within about 10 min and the administration and scoring procedures are simple and straight forward. It can be administered by health professionals, psychologists and other personnel with training in child development and psychometrics. This provides a quick assessment tool for initial assessment of developmental disabilities and research on the cognitive development of preschool children. However, there are some limitations in the present study. First, test–retest reliability has not been measured. Second, the PDAS scores have not been compared with intelligence test scores such as WPPSI scores. Third, norms have not been established and there is not enough data for the establishment of cut-off points. Future studies would need to address these areas. Finally, most of the PDAS items require some verbal response and the tool might not be suitable for children with no expressive language. 5. Conclusions The PDAS is a promising developmental assessment tool for preschool children. It is a unidimensional measure with satisfactory reliability estimates. It can also differentiate between children of different age groups, as well as children with developmental disabilities. References American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. Andrich, D., & Styles, I. (2004). Report on the psychometric analysis of the Early Development Instrument (EDI) using the Rasch model Retrieved from http:// www.rch.org.au/emplibrary/australianedi/SecondRaschReport.pdf. Bayley, N. (2006). Bayley scales of infant and toddler development (3rd ed.). San Antonio, TX: Pearson Education Inc. Bond, T., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Brinkman, S. A., Silburn, S., Lawrence, D., Foldfeld, S., Sayers, M., & Oberklaid, F. (2007). Investigating the validity of the Australian Early Development Index. Early Education and Development, 18, 427–451. Ceci, S. J., Baker-Sennett, J. G., & Bronfenbrenner, U. (1994). Psychometric and everyday intelligence: Synonyms, antonyms and anonyms. In M. Rutter & D. Hay (Eds.), Development through life: A handbook for clinicians. Oxford: Blackwell Science. Census and Statistics Department. (2009). Population and household statistics analysed by district council district. Hong Kong SAR Government: Census and Statistics Department. Curriculum Development Council. (2006). Local preschool curriculum guide. Hong Kong Education and Manpower Bureau. Department of Health. (2007). Child Assessment Service Retrieved from http://www.dhcas.gov.hk/eindex.html. Fan, X., & Thompson, B. (2001). Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517–531. Gardner, M. K., & Clark, E. (1992). The psychometric perspective on intellectual development in childhood and adolescence. In R. J. Sternberg & C. A. Berg (Eds.), Intellectual development. Cambridge: Cambridge University Press. Griffiths, R. (1996). The Griffiths Mental Development Scales from birth to 2 years, manual, the 1996 revision. Henley: Association for Research in Infant and Child Development, Test Agency. Janus, M., & Offord, D. R. (2007). Development and psychometric properties of the early development instrument (EDI): A measure of children’s school readiness. Canadian Journal of Behavioural Science, 39, 1–22. Johnson, S., & Marlow, N. (2006). Developmental screen or developmental testing? Early Human Development, 82, 173–183. Kagan, S. L., Britto, P. R., Kauerz, K., & Tarrant, K. (1995). Washington State Early learning and development benchmarks, a guide to young children’s learning and development: From birth to kindergarten entry. The State of Washington. Kagan, S. L., Britto, P. R., Kauerz, K., & Tarrant, K. (2005). Early learning and developmental benchmarks: A guide to young children’s learning and development: From birth to kindergarten entry. The State of Washington. Long, C. E., Blackman, J. A., Farrell, W. J., Smolkin, M. E., & Conaway, M. R. (2005). A comparison of developmental versus functional assessment in the rehabilitation of young children. Pediatric Rehabilitation, 8, 156–161. Pedhazur, E. J., & Pedhazur Schmelkin, L. (1991). Measurement, design and analysis: An integrated approach. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Petermann, F. (2008). Developmental assessment. Zeitschrift fu¨r Psychologie/Journal of Psychology, 216, 125–126. Reynell, J., & Huntley, M. (1985). Reynell Developmental Language Scales: Revised. Windsor: NFER-Nelson. Sattler, J. M. (2008). Assessment of children: Cognitive foundations (5th ed.). San Diego: Jermoe M Sattler Publisher Inc. Wilson, M. (2008). Cognitive diagnosis using item response models. Zeitschrift fu¨r Psychologie/Journal of Psychology, 216, 74–88.