- Email: [email protected]
Psychometric properties and validation of Portuguese version of Ages & Stages Questionnaires (3rd edition): 9, 18 and 30 Questionnaires
Early Human Development 91 (2015) 527–533
Contents lists available at ScienceDirect
Early Human Development journal homepage: www.elsevier.com/locate/earlhumdev
Psychometric properties and validation of Portuguese version of Ages & Stages Questionnaires (3rd edition): 9, 18 and 30 Questionnaires Sónia Lopes a,⁎, Patrícia Graça b, Salete Teixeira b, Ana Maria Serrano c, Jane Squires d a
Escola Superior de Saúde Jean Piaget V. N. Gaia, Instituto Piaget, Portugal Centro Integrado de Saúde Desenvolvimento Educação e Cultura, Rua Padre António Caldas, n° 1281, Oliveira do Castelo, 4810-246 Guimarães, Portugal Instituto de Educação (IE), Universidade do Minho, Campus de Gualtar, 4710–057 Braga, Portugal d Center on Human Development, University of Oregon, 901 E 18th Ave., Suite 139, Eugene, OR 97403-5253, United States b c
a r t i c l e
i n f o
Article history: Received 1 December 2014 Received in revised form 8 June 2015 Accepted 23 June 2015 Keywords: Early intervention Screening Ages and Stages Questionnaires Psychometry Validation
a b s t r a c t Background: The essential underlying foundations of Early Intervention (EI), in which parents/family play a critical role in their child's development, leads us to conclude that their contribution assessing early detection of problems is fundamental. The Ages & Stages Questionnaires (ASQ) is a standardized screening instrument that has been successfully studied in different countries and cultures. Aims: Translate and study the psychometrics proprieties of the Portuguese version of the 9, 18 and 30 month questionnaires of the Ages and Stages Questionnaires, 3rd edition (ASQ-3). Study design: Cross-sectional study. Subjects: Validity and reliability were studied in a sample of 234 parents of children within 9, 18 and 30 months. Results: The results indicated that the questionnaires had good internal consistency, strong agreement between observers and between observations with two weeks interval, and strong Pearson product–moment correlation coefficients between the overall and the total for each domain. The cutoff points (i.e. 2 standard deviations below the mean domain score), that identifies children who should receive further referral for more comprehensive assessment, were close to those determined in the original ASQ-3 psychometric studies. Cronbach's alpha ranging from .42 to .70 and Pearson's r values varies from .22 to .60. Conclusions: Although some weaknesses were noted in psychometric qualities analysis, it can be concluded that the ASQ-PT of 9, 18 and 30 months of age fulfills the requirements of a screening tool validated for the Portuguese population. Practice implications: To allow the early identification of children with developmental problems. © 2015 Elsevier Ireland Ltd. All rights reserved.
1. Introduction The ability of the brain to rearrange or adjust decreases as skills become more and more complex. Over time, the brain eliminates circuits that are less used strengthening those used often. Since the brain is particularly responsive to experiences in early years, early intervention is a way to assure a positive development and good health throughout life [1,2]. Developmental delays can be silent and occur in infants that apparently are developing without problems. Heckman [3] states that competence generates competence, and that, sooner the skills are acquired by the person the greater are the chances of achieving new skills. The investment return of policies
⁎ Corresponding author at: Campus Académico de Vila Nova de Gaia, Escola Superior de Saúde Jean Piaget em V. N. de Gaia, Alameda Jean Piaget, 4405-678 Gulpilhares, Vila Nova de Gaia, Portugal. Tel.: +351 227 536 620. E-mail addresses: [email protected] (S. Lopes), [email protected] (P. Graça), [email protected] (S. Teixeira), [email protected] (A.M. Serrano), [email protected] (J. Squires).
http://dx.doi.org/10.1016/j.earlhumdev.2015.06.006 0378-3782/© 2015 Elsevier Ireland Ltd. All rights reserved.
based on the implementation of early intervention can be reflected in the future, at the personal level of higher education, higher rates of physical, mental well-being and higher wage gains, and also to society with lower rates of crime and delinquency, reducing public expenditure and higher tax revenues. Currently a large percentage, 50% to 80%, of children with developmental delays is only detected after school entry, thereby missing the opportunity to achieve maximum outcomes in the preschool years [4–6]. This late referral may be due to the type of symptoms or to the cost and time-consuming nature of evaluations by professionals. Questionnaires completed by parents could be a viable alternative to professionals' assessments, thus decreasing the money spent and making them more economically efficient [7–12]. The American Academy of Pediatrics recommends using a valid and standardized developmental screening tool at 9, 18 and 30 months of age [12,13]. Developmental screening is mainly a preventive measure, that by using a brief, reliable and valid instrument, professionals can identify children who are at risk for developmental delay and need further evaluation [14–17]. It is important to implement formal and valid
528
S. Lopes et al. / Early Human Development 91 (2015) 527–533
screening tools, not only for children with suspicions of developmental disabilities, as well as for children who have normal development, as a way to increase families' awareness about aspects of child development and behavior expected (1) [9,11,18,19]. In Portugal, legal changes led us to the implementation of the Decree-Law 281/2009 [20], which created the National Early Childhood Intervention System. The 4th Article sets the system goals, one of them is “detect and signalize all children at risk for changes or changes in functions and structures of the body or serious risk of developmental delay.” The Ages and Stages Questionnaires, Third Edition (ASQ-3) [21] is a developmental screening instrument composed of 21 questionnaires for children from the first month until five and half years of age. The parents or caregivers of infants (2, 4, 6 and 8 months), toddlers (9, 10, 12, 14, 16, 18, 20, 22, 24, 27, 30 and 33 months) and preschoolers (36, 42, 48, 54 and 60 months) observe their child's development in five domains, communication, gross motor, fine motor, problem solving and personal-social. They observe their child's skills and respond “yes”, “sometimes” or “not yet” to six questions in each of the 5 domains. Studies on the ASQ have been carried out in numerous countries, with overall positive results [22]. Some countries in which ASQ-3 studies were completed include: Norway [23], Brazil [24], United States of America [25], Chile [26], Peru [27], Netherlands [28], Australia [29], China [30] and India [31]. The aim of this study was to determine a reliable and valid instrument for the Portuguese population to enable the screening of children's development. This way, by ensuring the psychometric qualities of the 9, 18 and 30 questionnaires of the Portuguese version of the ASQ (ASQPT), we intended to contribute to the design and implementation of a screening system to answer the needs of the Portuguese population. 2. Methods 2.1. Participants The present study was undertaken as a national validation study using a sample of the Portuguese population to find appropriate psychometric qualities of the ASQ-PT, namely reliability and validity. Data were collected from families with children from 9 + 1 months, 18 ± 1 months and 30 ± 1.5 months, enrolled in daycare and health care centers. Portuguese literacy parents, that completed at least elementary school, and who assented to take part, were invited to participate in the study. Portuguese speaking parents and caregivers of children with 9, 18 and 30 months were recruited from health care and daycare centers. In each five regions, designated by the Nomenclature of Territorial Units for Statistics System II, a set of organizations, selected by convenience, were invited to participate in this study. It was guaranteed that these organizations properly represent the five regions, considering both rural and urban context and the different children ages. Organizations were initially contacted by phone, email or in person to verify if they were interested in taking part of the research. If they agreed, all parents/caregivers of 9, 18 and 30 month old children were invited to complete the questionnaires.
compared the original version and the Portuguese version, and identified differences that justified a meeting between the three judges. Changes were made in syntactic structure, which was characterized by removing particles or elements with redundant meaning, and culturally adapting some terms since they were not commonly used in Portuguese. Finally, a pretest of each age questionnaire was administered to verify the clarity, understanding, cultural relevance and adjustment of the words used. 2.3. Procedures Data collection took place between May 2011 and June 2012. Study goals, including procedures were explained before the distribution of the questionnaires. An Excel database was established with the current date and the birthday of all children from the organizations that agreed to participate to automatically calculate the questionnaire delivered to each child. The questionnaires were sent by mail or hand-delivered and subsequently handed over to parents by the teachers of each preschool classroom or by nurses from the health centers. All parents and caregivers who participated in this study signed a consent form before completing the questionnaire. A personalized communication explaining that information would be kept confidential, the questionnaires were anonymous, and explaining the importance of early identification of developmental delays was used to increase the participant response rate. Twenty-five incomplete questionnaires were eliminated. 2.4. Statistical analysis Data were stored in a database built in the Statistical Package for Social Science (SPSS®), conducting an analysis based on descriptive and inferential statistics. During this study, we analyzed the psychometric qualities of three questionnaires in the Portuguese version. Reliability analyses were used to measure the consistency or stability of the ASQ3-PT, with internal consistency measured by Cronbach's alpha. Test– retest reliability was determined by a second administration of the same questionnaire to the same parents at a two-week time interval. Inter-observer of ASQ-PT was calculated by the comparing the questionnaires completed by a parent and by a professional on the same child. Both reliability tests used a convenience sample. Item means and standard deviations were calculated, as well as the item-total correlation (ITC). Validity was measured, using the Pearson product moment test and factor analysis (FA). The FA was calculated considering the principal components' analysis and varimax rotation limited to five factors, since this is the number of domains or areas observed in the ASQ. For the 30 items correlation coefficient equal or greater than .40 was accepted. Communalities, eigenvalues, and the total variance explained were also analyzed. Pearson product–moment correlation coefficients were calculated between each domain and domain score and the total score for each questionnaire. Mean scores and cutoff points for Portuguese population were generated for each domain and compared to the US normative sample. Finally, clinical validity was measured using comparative analyses between mean domain scores for children with and without risk for developmental delays.
2.2. The ASQ-PT 3. Results The translation of 9, 18 and 30 month questionnaires of ASQ-3 to Portuguese was completed using the method of back translation after initial translation, considered the most appropriate with less likelihood of bias [32]. Each questionnaire was translated from English into Portuguese by two professionals (one Portuguese with fluent knowledge of English and another of English and Portuguese nationality, English teacher in Portugal). The Portuguese judge completed the translation and English judge participated in this stage as a consultant to clarify slight differences in the English language. Thereafter, a third judge, bilingual, proceeded to back-translation. In the end, the first two judges
3.1. Descriptive analysis Data from 234 ASQ-PT at 9, 18 and 30 months questionnaires were included in this study. The response rate was about 60%. The sample represents 1% of the Portuguese population corresponding to a margin of error of 6.5%. Mothers corresponded to 80.3% of persons who completed the questionnaires. Table 1 summarizes demographic characteristics; only 15.0% of children were not enrolled from daycare. A high percentage of the
S. Lopes et al. / Early Human Development 91 (2015) 527–533
Total
the Communication domain, .43 to .78 for the Gross Motor domain, .68 to .75 for the Fine Motor domain, .60 to .70 for Problem Solving domain and .42 to .60 for the Personal Social domain. (5)
57 (50.9) 55 (49.1)
111 (47.4) 123 (52.6)
3.2.2. Item total correlation Concerning item total correlation (ITC), 10 items (11.1%) from a total of 90 presented values below .20. The communication was the domain with the highest number of items below .20 (i.e., 5 of 12 items).
52 (82.5) 11 (17.5)
102 (91.1) 10 (8.9)
199 (85.0) 35 (15.0)
51 (86.4) 8 (13.6)
59 (93.7) 4 (6.3)
107 (95.5) 5 (4.5)
217 (92.7) 17 (7.3)
32.7 ± 6.6
31.6 ± 4.6
33.1 ± 5.6
32.6 ± 5.6
8 (13.6) 27 (45.8) 24 (40.7)
6 (9.5) 26 (41.3) 31 (49.2)
9 (8.0) 57 (50.9) 46 (41.1)
23 (9.8) 110 (47.0) 101 (43.2)
23 (39.0) 14 (23.7)
16 (25.4) 26 (41.3)
26 (23.2) 35 (31.2)
65 (27.8) 75 (32.1)
22 (37.3)
21 (33.3)
51 (45.5)
94 (40.2)
Table 1 Sociodemographic characteristics sample. Characteristics
Questionnaire ASQ-PT 9 months (n = 59) (% = 25.2)
18 months (n = 63) (% = 26.9)
30 months (n = 112) (% = 47.9)
Gender Female Male
20 (33.9) 39 (66.1)
34 (54.0) 29 (46.0)
Kindergarten Yes No
45 (76.3) 14 (23.7)
Gestational age Term (N37 weeks) Moderate to late preterm (32 to b37 weeks) Caregiver age (mean, ±SD) Socioeconomic status High Middle Low Mother level of education Higher education Secondary education (≤12 years) Basic education (≤9 years)
mothers (40.2%) noted an education level less or equal to the 9th grade or basic education. ASQ-3 parent responses were distributed mainly in the “yes” and “sometimes” categories. Means were in the lower half of the scale (below 5) for 4.4% of 90 items. These lower scores were in the 18 month interval (i.e., 2 items: if the child is able to make a sentence with two or three words and if the child is able to copy a line in any direction) and on the 30 month questionnaire (2 items: if the child can string small items onto a sting and if the child put on a coat, jacket, or shirt by himself). Three of the four items refer to competencies that presume the use of the hands and arms on drawing activities and putting on a coat. 3.2. Internal consistency 3.2.1. Cronbach's alpha Cronbach's alpha values for the five domains of the three questionnaires from ASQ-PT are summarized in Table 2. Across the 9, 18 and 30 month questionnaires, Cronbach's alpha ranged from .52 to .64 for
Table 2 Internal consistency for the 9, 18 and 30 month ASQ-PT questionnaires (6). Questionnaire
Dimension
n
α
α if item deleted
9 months
Communication Gross motor Fine motor Problem solving Personal–social Communication Gross motor Fine motor Problem solving Personal–social Communication Gross motor Fine motor Problem solving Personal–social
59
.52 .78 .70 .70 .60 .64 .62 .68 .64 .54 .52 .43 .75 .60 .42
.59(1) .62(3)
18 months
30 months
529
63
112
.61(1) .67(1) .69(1) .57(1) .53(1 e 3) .49 (1) .77(6) .59(1) .46(6)
3.2.3. Test–retest and interobserver reliability Test–retest reliability was analyzed by comparing ASQ-PT questionnaires completed by the same parent (n = 36), within a two weeks interval. On the 9 month questionnaire (n = 6), Pearson product– moment correlation coefficient varied from .82 in the gross motor dimension to .99 in the problem solving. On the 18 month questionnaire (n = 12), the Pearson's r correlation ranged from .92 in the communication and problem solving dimensions to .97 in the fine motor dimension. On the 30 month questionnaire (n = 18), correlations varied from .84 in gross motor domain to .97 in fine motor domain. Concerning inter-observer reliability, agreement between the answers from parents and the child's educator (n = 39), varied from moderate (r = .55, p b .05) in communication dimension on the 30 month questionnaire to strong (r = .93, p b .001) in fine motor dimension also on the 30 month questionnaire. The only non-significant correlation was found in the problem solving dimension on 9 the month questionnaire (r = .53, n.s.). 3.3. Validity analysis 3.3.1. Factor analysis Factor analysis was explored using the varimax rotation procedure, limited to five factors. The total explained variance varies between 54.8% on the 9 month questionnaire, through 52.7% in the 18 months to 43.5% on the 30 month questionnaire. 3.3.2. Pearson product–moment correlation coefficient Pearson product–moment correlation coefficients were calculated between developmental area scores for each questionnaire and between developmental area score and the total score for each questionnaire. Table 3 shows the correlations between developmental area and overall score that ranged from .45 (gross motor, 30 months) to .82 (fine motor, 30 months). All correlations were significant at p b .01. On the 30 month questionnaire, three correlations were below .60. Correlations between developmental areas (Table 4), results were consistent for 9 and 18 months and ranged from .26 (gross-motor/ communication, 18 months) to .60 (problem-solving/fine motor, 18 months). The only exception was on the 9 month questionnaire, between fine motor and communication (.10). The 30 month questionnaire presented lower Pearson product–moment correlation coefficients. 3.3.3. Normative study As shown in Table 5, comparing the cutoff points from the original version of ASQ-3 and the Portuguese version (9, 18 and 30 months), the communication domain scores from Portuguese version were higher than the original US version. The personal–social domain demonstrates this same trend, the exception is an insignificant decrease of the first standard deviation on the 30 month questionnaire. Comparing the three remaining domains, the cutoff values were lower in the Portuguese version. Only on the 30 month questionnaire the cutoff points were higher (i.e., gross motor and problem solving domains) the fine motor domain was the only one that values consistently decrease in all three questionnaires.
530
S. Lopes et al. / Early Human Development 91 (2015) 527–533
Table 3 Correlations between developmental area and overall on the 9, 18 and 30 month ASQ-PT questionnaires (6). Questionnaire
n
Communication
Gross motor
Fine motor
Problem solving
Personal–social
9 months 18 months 30 months
59 63 112
.67⁎⁎ .71⁎⁎ .59⁎⁎
.79⁎⁎ .67⁎⁎ .45⁎⁎
.66⁎⁎ .79⁎⁎ .82⁎⁎
.72⁎⁎ .79⁎⁎ .74⁎⁎
.66⁎⁎ .78⁎⁎ .52⁎⁎
⁎⁎ p b .01.
3.3.4. Comparative study between risk and non-risk samples The criteria for the risk sample were the same as for the original ASQ-3 [21] normative sample. Children from families who met one or more of the following criteria were included: 1) extreme poverty; 2) mother's age less than 19 years at the date of the child's birth; 3) mother's educational level of less than 12th grade; 4) parents who are involved with child protection associations for abuse and/or neglect of their children; 5) medical risk including preterm birth; and 6) children with low birth weight. For this study four of the risk factors were considered, namely, baby's weight at birth (below 2500 g exclusive), gestational age (below 37 weeks of gestation), mother's educational level (less than 9th grade, inclusive) and the socioeconomic level of the family (low). Regarding the risk factors, 9% of the sample was born weighing less than 2500 g, 7.3% of the infants are preterm, 40.2% of the mothers had an education level below the 9th grade and 43.2% presented low socioeconomic level. Merging data results showed that 58.6% of the children had at least one risk factor (24.4% with one risk factor, 28.2% with two and 6% with three to four risk factors). Figs. 1, 2 and 3 show the graphic comparison between the means of the risk and no risk samples, for each developmental domains in 9, 18 and 30 month questionnaires. In general, means were lower in the risk sample except for the personal–social dimension at 9 and 18 months. At 9 months the gross motor mean presented the large difference, with a value significantly higher in the risk group.
3.3.5. Clinical validity Means, standard deviations and minimum and maximum scores from a small sample of children with special needs are summarized in Table 6. A total of 16 questionnaires (i.e., 8 for 18 months and 8 for 30 months) were collected, and results indicated that means were in the lower half of the scale, below 30. In addition, almost all parents (67%) answered “yes” to the question “Does anything about your child worry you?”.
4. Discussion Related to internal consistency results, communication and personal–social were domains that presented lower Cronbach's alpha. According to Pomés [25], the communication dimension, was the one that reflected the largest differences of functioning between the original version of ASQ-3 and the Spanish version studied in the US. In other international studies, personal–social dimension presented the lowest alphas, as studied in Norway [23], India [31] and Brazil [24]. Different possibilities were discussed when trying to understand these facts, like cultural differences, and Kvestad [31] and Filgueiras et al. [24] argued that personal–social domain has several constructs, such as child independence and social behavior that are culturally laden. In addition, these items also include motor skills. Hornman et al. [28] stated that the lowest values of internal consistency for the various domains could be due to the small number of items when compared to the total items of each questionnaire. Communication also presented a large number of items with item-total correlations below 0.2. Correlation coefficients were computed between the total domain score and the overall score for each questionnaire and the domain scores of the three questionnaires. Strong correlation values between the overall scores indicated concurrency, consistent with ASQ-3 original [21] study and Indian study [31]. Moderate to weak correlation
Table 5 Comparing the two standards deviation, between ASQ-3 and ASQ-PT on 9, 18 and 30 questionnaires (6). Questionnaire
Dimension
9 months
Communication Gross motor Fine motor Problem solving Personal–social
18 months Table 4 Correlations between each developmental area score for the 9, 18 and 30 month ASQ-PT questionnaires (6). Questionnaire
Dimension
Communication
9 months
Gross motor Fine motor Problem solving Personal–social Gross motor Fine motor Problem solving Personal–social Gross motor Fine motor Problem solving Personal–social
.55⁎⁎⁎ .10 .28⁎ .48⁎⁎⁎ .26⁎ .34⁎⁎ .48⁎⁎⁎ .49⁎⁎⁎
18 months
30 months
⁎ p b .05. ⁎⁎ p b .01. ⁎⁎⁎ p b .001.
.15 .42⁎⁎⁎ .32⁎⁎ .10
Gross motor
Fine motor
.40⁎⁎ .41⁎⁎ .30⁎
.49⁎⁎⁎ .31⁎
.50⁎⁎⁎ .34⁎⁎ .56⁎⁎⁎
.60⁎⁎⁎ .50⁎⁎⁎
.08 .09 .14
.38⁎⁎⁎ .22⁎
Communication Gross motor Fine motor
Problem solving
Problem solving Personal–Social
.33⁎
30 months
Communication Gross motor
.47⁎⁎⁎
Fine motor Problem solving Personal–social
.16 a b
One standard deviation below the mean. Two standard deviation below the mean.
ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT
1st SDa
2nd SDb
26.26 35.66 32.27 30.03 41.82 36.35 39.11 34.62 30.69 36.08 27.68 28.55 46.42 43.37 43.38 34.29 35.86 26.30 37.55 38.76 43.56 48.39 44.84 45.08 33.02 23.69 38.63 41.65 41.94 41.40
13.97 24.51 17.82 14.63 31.32 23.82 28.72 21.74 18.91 24.75 13.06 15.12 37.38 33.88 34.32 21.91 25.74 13.71 27.19 28.87 33.30 41.47 36.14 37.26 19.25 8.41 27.08 31.74 32.01 34.01
S. Lopes et al. / Early Human Development 91 (2015) 527–533
531
Fig. 1. Graphic comparison between the means of the risk and no risk samples.
Fig. 2. Graphic comparison between the means of the risk and no risk samples.
coefficients between the five domains of each questionnaire indicated concurrence, but demonstrated that each domain measures different developmental skills. To analyze inter-observer agreement and test–retest reliability for 9, 18 and 30 months questionnaires, Pearson's r test was tested. The results are similar to previous studies, including the original ASQ-3 [21] and the Indian version [31] where satisfactory agreement was noted in both tests. Exploratory factor analysis does not reproduce the original organization of the five dimensions. These results may be due to the artificial child development fracturing into “domains” that are theoretical and may not be reflected in each child [33–35]. It is possible that tools that intend to track child's development look at a large variety of domains and capacities that are closely related and bi-directional influence [36]. Pomés [25] claimed that it's difficult to embrace different developmental domains in only one questionnaire with a reduced number of items since they track general skills while measuring a child's functional capacity. The factor analysis wasn't performed in the study of ASQ-3 [21], and Filgueiras [24] in the Brazilian study used a different technique, item response modeling. Unlike the studies in the Brazilian [24] and Norwegian [23] populations, where the cut off points consistently decreased, and in the Dutch study [28] in which all cut off points increased, we did not see a tendency to fall or rise. However, when comparing referral rates (i.e., children with scores above the cut off points), a larger percentage
of children were identified with the Portuguese questionnaires than with the Brazilian and Norwegian studies. The World Report on Disability [37,38], estimated that 15.3% of the world's population have severe or moderate disabilities and 2.9% have severe disabilities, specifying for the Portuguese population, in 2002–2004, they calculated a prevalence of 11.2%. ASQ-3 [21] reference values varies between 12% to 16% children identified for further evaluation in one developmental domain, and 2% to 7% in two or more domains. Comparing the identification rate of ASQ-PT with the prevalence rate of disability that the WHO estimates for Portugal, we can say that the values are very close. In a small sample of children with special needs, mean scores were lower in all domains on the three questionnaires studied. This analysis allows us to deduce that if children with special needs were screened with ASQ-PT, they would be identified and refer to a professional assessment. Even though the ASQ was not designed to evaluate children with special needs, in this study the ASQ-PT was sensitive to developmental changes. In general, parents' concerns with child development were congruent with their child's problems, so parents are the most effective vehicle for identifying problems and acting on them [5,10,39]. When parent's comments seem to be incongruent with their child's development, the ASQ can help expectations adjustment, as well as parental information about development [5,7,10,40]. Concurrent validity between the ASQ-PT wasn't performed because there is no “gold standard” instruments validated for the Portuguese
Fig. 3. Graphic comparison between the means of the risk and no risk samples.
532
S. Lopes et al. / Early Human Development 91 (2015) 527–533
Table 6 Mean, standard deviation, minimum and maximum scores for each dimension in ASQ-PT for children with special needs on the 18 and 30 months questionnaires (6). ASQ-PT
18 30
n
8 8
Communication
Gross motor
Fine motor
Problem solving
Personal–social
M
SD.
Min– max
M
SD
Min– max
M
SD
Min– max
M
SD
Min– max
M
SD
Min– max
14.4 14.4
16.13 14.99
0–45 0–40
15.0 16.3
20.87 18.85
0–50 0–60
12.5 13.1
10.35 15.10
0–30 0–35
13.1 8.75
12.23 11.26
0–30 0–30
19.4 14.4
15.91 9.43
0–40 0–25
population, to make these comparisons. In future studies as a way to minimize this constraint would be to complement the use of the ASQPT with the evaluation of a professional, to compare and validate the results. Other limitations of the present study result from a small and non-probabilistic sample that should be accounted in future studies. In addition, to meet robust psychometric properties, the ASQ-PT will meet the recommended guidelines for Early Intervention related to parental involvement in their child's education and health [5,17,21]. The parent-completed ASQ system provides the opportunity to extend parents' knowledge of child development as they observe and recognize the skills of their children, contributing to their training and coresponsibility from the first moment in the child's life. Conflict of interest statement The authors Sonia Lopes, Patrícia Graça, Salete Teixeira and Ana Maria Serrano declare that there are no conflicts of interest. Jane Squires declares a conflict of interest as an author of the ASQ who receives publications royalties. Acknowledgments This paper has been funded with the support of national funds through FCT-Foundation for Science and Technology within the project PEst-OE/CED/UI1661/2014 do CIEd-UM. References [1] National Scientific Council on the Developing Child. Early experiences can alter gene expression and affect long-term development: working paper no. 10 [Internet]. Gene expression. Cambridge: Center on the Developing Child at Harvard University; 2010[Available from: www.developingchild.harvard.edu]. [2] Shonkoff JP, Levitt P. Neuroscience and the future of early childhood policy: moving from why to what and how. Neuron Sep 9 2010;67(5):689–91 [[Internet]. [cited 2014 May 4]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20826301]. [3] Heckman JJ. Skill formation and the economics of investing in disadvantaged children. Science Jun 30 2006;312(5782):1900–2 [[Internet]. [cited 2014 Apr 30]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16809525]. [4] Rosenberg SA, Zhang D, Robinson CC. Prevalence of developmental delays and participation in early intervention services for young children. Pediatrics Jun 2008; 121(6):e1503–9 [[Internet]. [cited 2014 May 3]. Available from: http://www.ncbi. nlm.nih.gov/pubmed/18504295]. [5] Glascoe FP, Shapiro HL. Introduction to developmental and behavioral screening. Dev Behav Pediatr 2004 [online.[cited 2011 Jan 10]. Available from: http://www. dbpeds.org/articles/detail.cfm?id = 5]. [6] Halfon N, Regalado M, Sareen H, Inkelas M, Reuland C, Glascoe FP, et al. Assessing development in the pediatric office. Pediatrics 2004;113(6 Suppl.):1926–33 [[Internet]. Available from: bGo to ISIN://WOS:000221782100005]. [7] Dixon G, Badawi N, French D, Kurinczuk JJ. Can parents accurately screen children at risk of developmental delay? J Paediatr Child Health May 2009;45(5):268–73 [[Internet]. [cited 2014 May 1]. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/19493118]. [8] Dobrez D, Sasso AL, Holl J, Shalowitz M, Leon S, Budetti P. Estimating the cost of developmental and behavioral screening of preschool children in general pediatric practice. Pediatrics Oct 1 2001;108(4):913–22 [[Internet]. [cited 2014 May 1]. Available from: http://pediatrics.aappublications.org/content/108/4/913.abstract]. [9] Earls MF, Hay SS. Setting the stage for success: implementation of developmental and behavioral screening and surveillance in primary care practice—the North Carolina Assuring Better Child Health and Development (ABCD) Project. Pediatrics Jul 2006;118(1):e183–8 [[Internet]. [cited 2014 May 2]. Available from: bGo to ISIN://000238726100086]. [10] Pinto-Martin JA, Dunkle M, Earls M, Fliedner D, Landes C. Developmental stages of developmental screening: steps to implementation of a successful program. Am J Public Health Nov 2005;95(11):1928–32 [[Internet]. [cited 2014 May 3]. Available from: bGo to ISIN://000233050800017].
[11] Rydz D, Shevell MI, Majnemer A, Oskoui M. Topical review: developmental screening. J Child Neurol Jan 1 2005;20(1):4–21 [[Internet]. [cited 2014 May 3]. Available from: http://jcn.sagepub.com/cgi/doi/10.1177/08830738050200010201]. [12] Committee on Children With Disabilities. Developmental Surveillance and Screening of Infants and Young Children. Pediatrics Jul 1 2001;108(1):192–5 [[Internet]. [cited 2014 May 1]. Available from: http://pediatrics.aappublications.org/content/108/1/ 192.full]. [13] American Academy of Pediatrics. Developmental Screening in Early Childhood Systems[Internet] ; 2009[Elk Grove Village. Available from: http://www. healthychildcare.org/pdf/DSECSreport.pdf]. [14] Gilliam WS, Meisels SJ, Mayes LC. Screening and surveillance in early intervention systems. In: Guralnick MJ, editor. The developmental systems approach to early intervention. Baltimore: Brookes Publishing Company; 2005. p. 73–98. [15] LaRosa A. Developmental and behavioral screening tests in primary care. [Internet]. UpToDate. 2011 [cited 2012 Jan 10]. Available from: http://www.uptodate.com/ contents/developmental-and-behavioral-screening-tests-in-primary-care#H10. [16] Marks KP, Glascoe FP, Macias MM. Enhancing the algorithm for developmentalbehavioral surveillance and screening in children 0 to 5 years. Clin Pediatr (Phila) Sep 2011;50(9):853–68 [[Internet]. [cited 2014 May 3]. Available from: bGo to ISIN://WOS:000294504500009]. [17] Smith LE, Akai CE, Klerman LV, Keltner BR. What mothers don't know and doctors don't say: detecting early developmental delays. Infant Ment Health J Jul 8 2010; 31(4):455–66 [[Internet]. [cited 2014 May 4]. Available from: http://doi.wiley. com/10.1002/imhj.20266]. [18] Brassard MR, Boehm AE. Preschool assessment: principles and practices. New York: Guilford Press; 2007. [19] Guralnick MJ, Conlon C. Early intervention. In: Batshaw M, Pelligrino L, Roizen N, editors. Children with disabilities. 6th ed. Baltimore: Paul H. Brookes; 2007. p. 511–21. [20] Decreto-Lei no 281/2009 de 6 de Outubro do Ministério da Saúde. [Internet] Diário da República: I Série, no 131; 2009[Available from: http://dre.pt/pdf1sdip/2009/10/ 19300/0729807301.pdf]. [21] Squires J, Twombly E, Bricker D, Potter L. ASQ-3 user's guide. Baltimore, Maryland: Brookes Publishing Co; 2009. [22] Macy M. The evidence behind developmental screening instruments. Infants Young Child 2012;25(1):19–61 [[Internet]. [cited 2014 Apr 30]. Available from: http:// content.wkhealth.com/linkback/openurl?sid = WKPTLP:landingpage&an = 00001163-201201000-00003]. [23] Alvik A, Grøholt B. Examination of the cut-off scores determined by the Ages and Stages Questionnaire in a population-based sample of 6 month-old Norwegian infants. BMC Pediatr Jan 2011;11(1):117 [[Internet]. [cited 2014 Apr 30]. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid = 3293730&tool = pmcentrez&rendertype = abstract]. [24] Filgueiras A, Pires P, Maissonette S, Landeira-Fernandez J. Psychometric properties of the Brazilian-adapted version of the Ages and Stages Questionnaire in public child daycare centers. Early Hum Dev 2013;89(8):561–76. [25] Pomés MP. Examination of the Spanish Translation of a Developmental Screening Instrument [Internet]. Unpublished doctoral dissertation, University of Oregon, Oregon, EUA.; 2012. Available from: http://hdl.handle.net/1794/12332. [26] Schonhaut L, Armijo I, Schönstedt M, Alvarez J, Cordero M. Validity of the ages and stages questionnaires in term and preterm infants. Pediatrics May 1 2013;131(5): e1468–74 [[Internet]. [cited 2014 May 1]. Available from: http://www.ncbi.nlm. nih.gov/pubmed/23629619]. [27] Kyerematen V, Hamb A, Oberhelman RA, Cabrera L, Bernabe-Ortiz A, Berry SJ. Exploratory application of the Ages and Stages (ASQ) child development screening test in a low-income Peruvian shantytown population. BMJ Open Jan 10 2014; 4(1):e004132 [[Internet]. [cited 2014 May 4]. Available from: http://bmjopen.bmj. com/content/4/1/e004132.abstract]. [28] Hornman J, Kerstjens JM, de Winter AF, Bos AF, Reijneveld SA. Validity and internal consistency of the Ages and Stages Questionnaire 60-month version and the effect of three scoring methods. Early Hum Dev Dec 2013;89(12):1011–5 [[Internet]. [cited 2014 May 4]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24041814]. [29] D'Aprano AL, Carapetis JR, Andrews R. Trial of a developmental screening tool in remote Australian Aboriginal communities: a cautionary tale. J Paediatr Child Health 2011:12–7. [30] Bian X, Yao G, Squires J, Hoselton R, Chen C-I, Murphy K, et al. Translation and use of parent-completed developmental screening test in Shanghai. J Early Child Res Mar 27 2012;10(2):162–75 [[Internet]. Available from: http://ecr.sagepub.com/cgi/doi/ 10.1177/1476718X11430071]. [31] Kvestad I, Taneja S, Kumar T, Bhandari N, Strand TA, Hysing M. The assessment of developmental status using the Ages and Stages questionnaire-3 in nutritional research in north Indian young children. Nutr J Jan 2013;12:50 [[Internet]. [cited 2014 May 4]. Available from: http://www.pubmedcentral.nih.gov/articlerender. fcgi?artid = 3637815&tool = pmcentrez&rendertype = abstract]. [32] Hill MM, Hill A. Investigação por Questionário. 2nd ed. Lisboa: Edições Sílabo; 2009.
S. Lopes et al. / Early Human Development 91 (2015) 527–533 [33] Aly Z, Taj F, Ibrahim S. Missed opportunities in surveillance and screening systems to detect developmental delay: a developing country perspective. Brain Dev 2010;32: 90–7 [[Internet]. Available from: bGo to ISIN://WOS:000274710000003]. [34] Levine LE, Munsch J. Child development: an active learning approach. Los Angeles: Sage; 2011. [35] Papalia DE, Olds SW, Feldman RD. Desenvolvimento humano. 8th ed. São Paulo: Artmed; 2006. [36] Copple C, Bredekamp S. Developmentally appropriate practice in early childhood programs serving children from birth through age 8. In: Copple C, Bredekamp S, editors. 3rd ed. Washington, DC: National Association for the Education of Young Children; 2009. [37] World Health Organization. World report on disability [Internet]. Geneva. Available from: http://www.who.int/disabilities/world_report/2011/report.pdf; 2011.
533
[38] World Health Organization. Early childhood development and disability: discussion paper [Internet]. Geneva: World Health Organization and UNICEF; 2012[Available from: http://apps.who.int/iris/bitstream/10665/75355/1/9789241504065_eng.pdf? ua=1]. [39] Bricker D, Squires J. The effectiveness of parental screening of at-risk infants: the infant monitoring questionnaires. Top Early Child Spec Educ 1989:67–85. [40] Nelson BB, Chung PJ, DuPlessis HM, Flores L, Ryan GW, Kataoka SH. Strengthening families of children with developmental concerns: parent perceptions of developmental screening and services in Head Start. Ethn Dis 2011;21(3 Suppl. 1):89–93 [[Internet]. Available from: bGo to ISIN://MEDLINE:22352085].
Contents lists available at ScienceDirect
Early Human Development journal homepage: www.elsevier.com/locate/earlhumdev
Psychometric properties and validation of Portuguese version of Ages & Stages Questionnaires (3rd edition): 9, 18 and 30 Questionnaires Sónia Lopes a,⁎, Patrícia Graça b, Salete Teixeira b, Ana Maria Serrano c, Jane Squires d a
Escola Superior de Saúde Jean Piaget V. N. Gaia, Instituto Piaget, Portugal Centro Integrado de Saúde Desenvolvimento Educação e Cultura, Rua Padre António Caldas, n° 1281, Oliveira do Castelo, 4810-246 Guimarães, Portugal Instituto de Educação (IE), Universidade do Minho, Campus de Gualtar, 4710–057 Braga, Portugal d Center on Human Development, University of Oregon, 901 E 18th Ave., Suite 139, Eugene, OR 97403-5253, United States b c
a r t i c l e
i n f o
Article history: Received 1 December 2014 Received in revised form 8 June 2015 Accepted 23 June 2015 Keywords: Early intervention Screening Ages and Stages Questionnaires Psychometry Validation
a b s t r a c t Background: The essential underlying foundations of Early Intervention (EI), in which parents/family play a critical role in their child's development, leads us to conclude that their contribution assessing early detection of problems is fundamental. The Ages & Stages Questionnaires (ASQ) is a standardized screening instrument that has been successfully studied in different countries and cultures. Aims: Translate and study the psychometrics proprieties of the Portuguese version of the 9, 18 and 30 month questionnaires of the Ages and Stages Questionnaires, 3rd edition (ASQ-3). Study design: Cross-sectional study. Subjects: Validity and reliability were studied in a sample of 234 parents of children within 9, 18 and 30 months. Results: The results indicated that the questionnaires had good internal consistency, strong agreement between observers and between observations with two weeks interval, and strong Pearson product–moment correlation coefficients between the overall and the total for each domain. The cutoff points (i.e. 2 standard deviations below the mean domain score), that identifies children who should receive further referral for more comprehensive assessment, were close to those determined in the original ASQ-3 psychometric studies. Cronbach's alpha ranging from .42 to .70 and Pearson's r values varies from .22 to .60. Conclusions: Although some weaknesses were noted in psychometric qualities analysis, it can be concluded that the ASQ-PT of 9, 18 and 30 months of age fulfills the requirements of a screening tool validated for the Portuguese population. Practice implications: To allow the early identification of children with developmental problems. © 2015 Elsevier Ireland Ltd. All rights reserved.
1. Introduction The ability of the brain to rearrange or adjust decreases as skills become more and more complex. Over time, the brain eliminates circuits that are less used strengthening those used often. Since the brain is particularly responsive to experiences in early years, early intervention is a way to assure a positive development and good health throughout life [1,2]. Developmental delays can be silent and occur in infants that apparently are developing without problems. Heckman [3] states that competence generates competence, and that, sooner the skills are acquired by the person the greater are the chances of achieving new skills. The investment return of policies
⁎ Corresponding author at: Campus Académico de Vila Nova de Gaia, Escola Superior de Saúde Jean Piaget em V. N. de Gaia, Alameda Jean Piaget, 4405-678 Gulpilhares, Vila Nova de Gaia, Portugal. Tel.: +351 227 536 620. E-mail addresses: [email protected] (S. Lopes), [email protected] (P. Graça), [email protected] (S. Teixeira), [email protected] (A.M. Serrano), [email protected] (J. Squires).
http://dx.doi.org/10.1016/j.earlhumdev.2015.06.006 0378-3782/© 2015 Elsevier Ireland Ltd. All rights reserved.
based on the implementation of early intervention can be reflected in the future, at the personal level of higher education, higher rates of physical, mental well-being and higher wage gains, and also to society with lower rates of crime and delinquency, reducing public expenditure and higher tax revenues. Currently a large percentage, 50% to 80%, of children with developmental delays is only detected after school entry, thereby missing the opportunity to achieve maximum outcomes in the preschool years [4–6]. This late referral may be due to the type of symptoms or to the cost and time-consuming nature of evaluations by professionals. Questionnaires completed by parents could be a viable alternative to professionals' assessments, thus decreasing the money spent and making them more economically efficient [7–12]. The American Academy of Pediatrics recommends using a valid and standardized developmental screening tool at 9, 18 and 30 months of age [12,13]. Developmental screening is mainly a preventive measure, that by using a brief, reliable and valid instrument, professionals can identify children who are at risk for developmental delay and need further evaluation [14–17]. It is important to implement formal and valid
528
S. Lopes et al. / Early Human Development 91 (2015) 527–533
screening tools, not only for children with suspicions of developmental disabilities, as well as for children who have normal development, as a way to increase families' awareness about aspects of child development and behavior expected (1) [9,11,18,19]. In Portugal, legal changes led us to the implementation of the Decree-Law 281/2009 [20], which created the National Early Childhood Intervention System. The 4th Article sets the system goals, one of them is “detect and signalize all children at risk for changes or changes in functions and structures of the body or serious risk of developmental delay.” The Ages and Stages Questionnaires, Third Edition (ASQ-3) [21] is a developmental screening instrument composed of 21 questionnaires for children from the first month until five and half years of age. The parents or caregivers of infants (2, 4, 6 and 8 months), toddlers (9, 10, 12, 14, 16, 18, 20, 22, 24, 27, 30 and 33 months) and preschoolers (36, 42, 48, 54 and 60 months) observe their child's development in five domains, communication, gross motor, fine motor, problem solving and personal-social. They observe their child's skills and respond “yes”, “sometimes” or “not yet” to six questions in each of the 5 domains. Studies on the ASQ have been carried out in numerous countries, with overall positive results [22]. Some countries in which ASQ-3 studies were completed include: Norway [23], Brazil [24], United States of America [25], Chile [26], Peru [27], Netherlands [28], Australia [29], China [30] and India [31]. The aim of this study was to determine a reliable and valid instrument for the Portuguese population to enable the screening of children's development. This way, by ensuring the psychometric qualities of the 9, 18 and 30 questionnaires of the Portuguese version of the ASQ (ASQPT), we intended to contribute to the design and implementation of a screening system to answer the needs of the Portuguese population. 2. Methods 2.1. Participants The present study was undertaken as a national validation study using a sample of the Portuguese population to find appropriate psychometric qualities of the ASQ-PT, namely reliability and validity. Data were collected from families with children from 9 + 1 months, 18 ± 1 months and 30 ± 1.5 months, enrolled in daycare and health care centers. Portuguese literacy parents, that completed at least elementary school, and who assented to take part, were invited to participate in the study. Portuguese speaking parents and caregivers of children with 9, 18 and 30 months were recruited from health care and daycare centers. In each five regions, designated by the Nomenclature of Territorial Units for Statistics System II, a set of organizations, selected by convenience, were invited to participate in this study. It was guaranteed that these organizations properly represent the five regions, considering both rural and urban context and the different children ages. Organizations were initially contacted by phone, email or in person to verify if they were interested in taking part of the research. If they agreed, all parents/caregivers of 9, 18 and 30 month old children were invited to complete the questionnaires.
compared the original version and the Portuguese version, and identified differences that justified a meeting between the three judges. Changes were made in syntactic structure, which was characterized by removing particles or elements with redundant meaning, and culturally adapting some terms since they were not commonly used in Portuguese. Finally, a pretest of each age questionnaire was administered to verify the clarity, understanding, cultural relevance and adjustment of the words used. 2.3. Procedures Data collection took place between May 2011 and June 2012. Study goals, including procedures were explained before the distribution of the questionnaires. An Excel database was established with the current date and the birthday of all children from the organizations that agreed to participate to automatically calculate the questionnaire delivered to each child. The questionnaires were sent by mail or hand-delivered and subsequently handed over to parents by the teachers of each preschool classroom or by nurses from the health centers. All parents and caregivers who participated in this study signed a consent form before completing the questionnaire. A personalized communication explaining that information would be kept confidential, the questionnaires were anonymous, and explaining the importance of early identification of developmental delays was used to increase the participant response rate. Twenty-five incomplete questionnaires were eliminated. 2.4. Statistical analysis Data were stored in a database built in the Statistical Package for Social Science (SPSS®), conducting an analysis based on descriptive and inferential statistics. During this study, we analyzed the psychometric qualities of three questionnaires in the Portuguese version. Reliability analyses were used to measure the consistency or stability of the ASQ3-PT, with internal consistency measured by Cronbach's alpha. Test– retest reliability was determined by a second administration of the same questionnaire to the same parents at a two-week time interval. Inter-observer of ASQ-PT was calculated by the comparing the questionnaires completed by a parent and by a professional on the same child. Both reliability tests used a convenience sample. Item means and standard deviations were calculated, as well as the item-total correlation (ITC). Validity was measured, using the Pearson product moment test and factor analysis (FA). The FA was calculated considering the principal components' analysis and varimax rotation limited to five factors, since this is the number of domains or areas observed in the ASQ. For the 30 items correlation coefficient equal or greater than .40 was accepted. Communalities, eigenvalues, and the total variance explained were also analyzed. Pearson product–moment correlation coefficients were calculated between each domain and domain score and the total score for each questionnaire. Mean scores and cutoff points for Portuguese population were generated for each domain and compared to the US normative sample. Finally, clinical validity was measured using comparative analyses between mean domain scores for children with and without risk for developmental delays.
2.2. The ASQ-PT 3. Results The translation of 9, 18 and 30 month questionnaires of ASQ-3 to Portuguese was completed using the method of back translation after initial translation, considered the most appropriate with less likelihood of bias [32]. Each questionnaire was translated from English into Portuguese by two professionals (one Portuguese with fluent knowledge of English and another of English and Portuguese nationality, English teacher in Portugal). The Portuguese judge completed the translation and English judge participated in this stage as a consultant to clarify slight differences in the English language. Thereafter, a third judge, bilingual, proceeded to back-translation. In the end, the first two judges
3.1. Descriptive analysis Data from 234 ASQ-PT at 9, 18 and 30 months questionnaires were included in this study. The response rate was about 60%. The sample represents 1% of the Portuguese population corresponding to a margin of error of 6.5%. Mothers corresponded to 80.3% of persons who completed the questionnaires. Table 1 summarizes demographic characteristics; only 15.0% of children were not enrolled from daycare. A high percentage of the
S. Lopes et al. / Early Human Development 91 (2015) 527–533
Total
the Communication domain, .43 to .78 for the Gross Motor domain, .68 to .75 for the Fine Motor domain, .60 to .70 for Problem Solving domain and .42 to .60 for the Personal Social domain. (5)
57 (50.9) 55 (49.1)
111 (47.4) 123 (52.6)
3.2.2. Item total correlation Concerning item total correlation (ITC), 10 items (11.1%) from a total of 90 presented values below .20. The communication was the domain with the highest number of items below .20 (i.e., 5 of 12 items).
52 (82.5) 11 (17.5)
102 (91.1) 10 (8.9)
199 (85.0) 35 (15.0)
51 (86.4) 8 (13.6)
59 (93.7) 4 (6.3)
107 (95.5) 5 (4.5)
217 (92.7) 17 (7.3)
32.7 ± 6.6
31.6 ± 4.6
33.1 ± 5.6
32.6 ± 5.6
8 (13.6) 27 (45.8) 24 (40.7)
6 (9.5) 26 (41.3) 31 (49.2)
9 (8.0) 57 (50.9) 46 (41.1)
23 (9.8) 110 (47.0) 101 (43.2)
23 (39.0) 14 (23.7)
16 (25.4) 26 (41.3)
26 (23.2) 35 (31.2)
65 (27.8) 75 (32.1)
22 (37.3)
21 (33.3)
51 (45.5)
94 (40.2)
Table 1 Sociodemographic characteristics sample. Characteristics
Questionnaire ASQ-PT 9 months (n = 59) (% = 25.2)
18 months (n = 63) (% = 26.9)
30 months (n = 112) (% = 47.9)
Gender Female Male
20 (33.9) 39 (66.1)
34 (54.0) 29 (46.0)
Kindergarten Yes No
45 (76.3) 14 (23.7)
Gestational age Term (N37 weeks) Moderate to late preterm (32 to b37 weeks) Caregiver age (mean, ±SD) Socioeconomic status High Middle Low Mother level of education Higher education Secondary education (≤12 years) Basic education (≤9 years)
mothers (40.2%) noted an education level less or equal to the 9th grade or basic education. ASQ-3 parent responses were distributed mainly in the “yes” and “sometimes” categories. Means were in the lower half of the scale (below 5) for 4.4% of 90 items. These lower scores were in the 18 month interval (i.e., 2 items: if the child is able to make a sentence with two or three words and if the child is able to copy a line in any direction) and on the 30 month questionnaire (2 items: if the child can string small items onto a sting and if the child put on a coat, jacket, or shirt by himself). Three of the four items refer to competencies that presume the use of the hands and arms on drawing activities and putting on a coat. 3.2. Internal consistency 3.2.1. Cronbach's alpha Cronbach's alpha values for the five domains of the three questionnaires from ASQ-PT are summarized in Table 2. Across the 9, 18 and 30 month questionnaires, Cronbach's alpha ranged from .52 to .64 for
Table 2 Internal consistency for the 9, 18 and 30 month ASQ-PT questionnaires (6). Questionnaire
Dimension
n
α
α if item deleted
9 months
Communication Gross motor Fine motor Problem solving Personal–social Communication Gross motor Fine motor Problem solving Personal–social Communication Gross motor Fine motor Problem solving Personal–social
59
.52 .78 .70 .70 .60 .64 .62 .68 .64 .54 .52 .43 .75 .60 .42
.59(1) .62(3)
18 months
30 months
529
63
112
.61(1) .67(1) .69(1) .57(1) .53(1 e 3) .49 (1) .77(6) .59(1) .46(6)
3.2.3. Test–retest and interobserver reliability Test–retest reliability was analyzed by comparing ASQ-PT questionnaires completed by the same parent (n = 36), within a two weeks interval. On the 9 month questionnaire (n = 6), Pearson product– moment correlation coefficient varied from .82 in the gross motor dimension to .99 in the problem solving. On the 18 month questionnaire (n = 12), the Pearson's r correlation ranged from .92 in the communication and problem solving dimensions to .97 in the fine motor dimension. On the 30 month questionnaire (n = 18), correlations varied from .84 in gross motor domain to .97 in fine motor domain. Concerning inter-observer reliability, agreement between the answers from parents and the child's educator (n = 39), varied from moderate (r = .55, p b .05) in communication dimension on the 30 month questionnaire to strong (r = .93, p b .001) in fine motor dimension also on the 30 month questionnaire. The only non-significant correlation was found in the problem solving dimension on 9 the month questionnaire (r = .53, n.s.). 3.3. Validity analysis 3.3.1. Factor analysis Factor analysis was explored using the varimax rotation procedure, limited to five factors. The total explained variance varies between 54.8% on the 9 month questionnaire, through 52.7% in the 18 months to 43.5% on the 30 month questionnaire. 3.3.2. Pearson product–moment correlation coefficient Pearson product–moment correlation coefficients were calculated between developmental area scores for each questionnaire and between developmental area score and the total score for each questionnaire. Table 3 shows the correlations between developmental area and overall score that ranged from .45 (gross motor, 30 months) to .82 (fine motor, 30 months). All correlations were significant at p b .01. On the 30 month questionnaire, three correlations were below .60. Correlations between developmental areas (Table 4), results were consistent for 9 and 18 months and ranged from .26 (gross-motor/ communication, 18 months) to .60 (problem-solving/fine motor, 18 months). The only exception was on the 9 month questionnaire, between fine motor and communication (.10). The 30 month questionnaire presented lower Pearson product–moment correlation coefficients. 3.3.3. Normative study As shown in Table 5, comparing the cutoff points from the original version of ASQ-3 and the Portuguese version (9, 18 and 30 months), the communication domain scores from Portuguese version were higher than the original US version. The personal–social domain demonstrates this same trend, the exception is an insignificant decrease of the first standard deviation on the 30 month questionnaire. Comparing the three remaining domains, the cutoff values were lower in the Portuguese version. Only on the 30 month questionnaire the cutoff points were higher (i.e., gross motor and problem solving domains) the fine motor domain was the only one that values consistently decrease in all three questionnaires.
530
S. Lopes et al. / Early Human Development 91 (2015) 527–533
Table 3 Correlations between developmental area and overall on the 9, 18 and 30 month ASQ-PT questionnaires (6). Questionnaire
n
Communication
Gross motor
Fine motor
Problem solving
Personal–social
9 months 18 months 30 months
59 63 112
.67⁎⁎ .71⁎⁎ .59⁎⁎
.79⁎⁎ .67⁎⁎ .45⁎⁎
.66⁎⁎ .79⁎⁎ .82⁎⁎
.72⁎⁎ .79⁎⁎ .74⁎⁎
.66⁎⁎ .78⁎⁎ .52⁎⁎
⁎⁎ p b .01.
3.3.4. Comparative study between risk and non-risk samples The criteria for the risk sample were the same as for the original ASQ-3 [21] normative sample. Children from families who met one or more of the following criteria were included: 1) extreme poverty; 2) mother's age less than 19 years at the date of the child's birth; 3) mother's educational level of less than 12th grade; 4) parents who are involved with child protection associations for abuse and/or neglect of their children; 5) medical risk including preterm birth; and 6) children with low birth weight. For this study four of the risk factors were considered, namely, baby's weight at birth (below 2500 g exclusive), gestational age (below 37 weeks of gestation), mother's educational level (less than 9th grade, inclusive) and the socioeconomic level of the family (low). Regarding the risk factors, 9% of the sample was born weighing less than 2500 g, 7.3% of the infants are preterm, 40.2% of the mothers had an education level below the 9th grade and 43.2% presented low socioeconomic level. Merging data results showed that 58.6% of the children had at least one risk factor (24.4% with one risk factor, 28.2% with two and 6% with three to four risk factors). Figs. 1, 2 and 3 show the graphic comparison between the means of the risk and no risk samples, for each developmental domains in 9, 18 and 30 month questionnaires. In general, means were lower in the risk sample except for the personal–social dimension at 9 and 18 months. At 9 months the gross motor mean presented the large difference, with a value significantly higher in the risk group.
3.3.5. Clinical validity Means, standard deviations and minimum and maximum scores from a small sample of children with special needs are summarized in Table 6. A total of 16 questionnaires (i.e., 8 for 18 months and 8 for 30 months) were collected, and results indicated that means were in the lower half of the scale, below 30. In addition, almost all parents (67%) answered “yes” to the question “Does anything about your child worry you?”.
4. Discussion Related to internal consistency results, communication and personal–social were domains that presented lower Cronbach's alpha. According to Pomés [25], the communication dimension, was the one that reflected the largest differences of functioning between the original version of ASQ-3 and the Spanish version studied in the US. In other international studies, personal–social dimension presented the lowest alphas, as studied in Norway [23], India [31] and Brazil [24]. Different possibilities were discussed when trying to understand these facts, like cultural differences, and Kvestad [31] and Filgueiras et al. [24] argued that personal–social domain has several constructs, such as child independence and social behavior that are culturally laden. In addition, these items also include motor skills. Hornman et al. [28] stated that the lowest values of internal consistency for the various domains could be due to the small number of items when compared to the total items of each questionnaire. Communication also presented a large number of items with item-total correlations below 0.2. Correlation coefficients were computed between the total domain score and the overall score for each questionnaire and the domain scores of the three questionnaires. Strong correlation values between the overall scores indicated concurrency, consistent with ASQ-3 original [21] study and Indian study [31]. Moderate to weak correlation
Table 5 Comparing the two standards deviation, between ASQ-3 and ASQ-PT on 9, 18 and 30 questionnaires (6). Questionnaire
Dimension
9 months
Communication Gross motor Fine motor Problem solving Personal–social
18 months Table 4 Correlations between each developmental area score for the 9, 18 and 30 month ASQ-PT questionnaires (6). Questionnaire
Dimension
Communication
9 months
Gross motor Fine motor Problem solving Personal–social Gross motor Fine motor Problem solving Personal–social Gross motor Fine motor Problem solving Personal–social
.55⁎⁎⁎ .10 .28⁎ .48⁎⁎⁎ .26⁎ .34⁎⁎ .48⁎⁎⁎ .49⁎⁎⁎
18 months
30 months
⁎ p b .05. ⁎⁎ p b .01. ⁎⁎⁎ p b .001.
.15 .42⁎⁎⁎ .32⁎⁎ .10
Gross motor
Fine motor
.40⁎⁎ .41⁎⁎ .30⁎
.49⁎⁎⁎ .31⁎
.50⁎⁎⁎ .34⁎⁎ .56⁎⁎⁎
.60⁎⁎⁎ .50⁎⁎⁎
.08 .09 .14
.38⁎⁎⁎ .22⁎
Communication Gross motor Fine motor
Problem solving
Problem solving Personal–Social
.33⁎
30 months
Communication Gross motor
.47⁎⁎⁎
Fine motor Problem solving Personal–social
.16 a b
One standard deviation below the mean. Two standard deviation below the mean.
ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT ASQ-3 ASQ-PT
1st SDa
2nd SDb
26.26 35.66 32.27 30.03 41.82 36.35 39.11 34.62 30.69 36.08 27.68 28.55 46.42 43.37 43.38 34.29 35.86 26.30 37.55 38.76 43.56 48.39 44.84 45.08 33.02 23.69 38.63 41.65 41.94 41.40
13.97 24.51 17.82 14.63 31.32 23.82 28.72 21.74 18.91 24.75 13.06 15.12 37.38 33.88 34.32 21.91 25.74 13.71 27.19 28.87 33.30 41.47 36.14 37.26 19.25 8.41 27.08 31.74 32.01 34.01
S. Lopes et al. / Early Human Development 91 (2015) 527–533
531
Fig. 1. Graphic comparison between the means of the risk and no risk samples.
Fig. 2. Graphic comparison between the means of the risk and no risk samples.
coefficients between the five domains of each questionnaire indicated concurrence, but demonstrated that each domain measures different developmental skills. To analyze inter-observer agreement and test–retest reliability for 9, 18 and 30 months questionnaires, Pearson's r test was tested. The results are similar to previous studies, including the original ASQ-3 [21] and the Indian version [31] where satisfactory agreement was noted in both tests. Exploratory factor analysis does not reproduce the original organization of the five dimensions. These results may be due to the artificial child development fracturing into “domains” that are theoretical and may not be reflected in each child [33–35]. It is possible that tools that intend to track child's development look at a large variety of domains and capacities that are closely related and bi-directional influence [36]. Pomés [25] claimed that it's difficult to embrace different developmental domains in only one questionnaire with a reduced number of items since they track general skills while measuring a child's functional capacity. The factor analysis wasn't performed in the study of ASQ-3 [21], and Filgueiras [24] in the Brazilian study used a different technique, item response modeling. Unlike the studies in the Brazilian [24] and Norwegian [23] populations, where the cut off points consistently decreased, and in the Dutch study [28] in which all cut off points increased, we did not see a tendency to fall or rise. However, when comparing referral rates (i.e., children with scores above the cut off points), a larger percentage
of children were identified with the Portuguese questionnaires than with the Brazilian and Norwegian studies. The World Report on Disability [37,38], estimated that 15.3% of the world's population have severe or moderate disabilities and 2.9% have severe disabilities, specifying for the Portuguese population, in 2002–2004, they calculated a prevalence of 11.2%. ASQ-3 [21] reference values varies between 12% to 16% children identified for further evaluation in one developmental domain, and 2% to 7% in two or more domains. Comparing the identification rate of ASQ-PT with the prevalence rate of disability that the WHO estimates for Portugal, we can say that the values are very close. In a small sample of children with special needs, mean scores were lower in all domains on the three questionnaires studied. This analysis allows us to deduce that if children with special needs were screened with ASQ-PT, they would be identified and refer to a professional assessment. Even though the ASQ was not designed to evaluate children with special needs, in this study the ASQ-PT was sensitive to developmental changes. In general, parents' concerns with child development were congruent with their child's problems, so parents are the most effective vehicle for identifying problems and acting on them [5,10,39]. When parent's comments seem to be incongruent with their child's development, the ASQ can help expectations adjustment, as well as parental information about development [5,7,10,40]. Concurrent validity between the ASQ-PT wasn't performed because there is no “gold standard” instruments validated for the Portuguese
Fig. 3. Graphic comparison between the means of the risk and no risk samples.
532
S. Lopes et al. / Early Human Development 91 (2015) 527–533
Table 6 Mean, standard deviation, minimum and maximum scores for each dimension in ASQ-PT for children with special needs on the 18 and 30 months questionnaires (6). ASQ-PT
18 30
n
8 8
Communication
Gross motor
Fine motor
Problem solving
Personal–social
M
SD.
Min– max
M
SD
Min– max
M
SD
Min– max
M
SD
Min– max
M
SD
Min– max
14.4 14.4
16.13 14.99
0–45 0–40
15.0 16.3
20.87 18.85
0–50 0–60
12.5 13.1
10.35 15.10
0–30 0–35
13.1 8.75
12.23 11.26
0–30 0–30
19.4 14.4
15.91 9.43
0–40 0–25
population, to make these comparisons. In future studies as a way to minimize this constraint would be to complement the use of the ASQPT with the evaluation of a professional, to compare and validate the results. Other limitations of the present study result from a small and non-probabilistic sample that should be accounted in future studies. In addition, to meet robust psychometric properties, the ASQ-PT will meet the recommended guidelines for Early Intervention related to parental involvement in their child's education and health [5,17,21]. The parent-completed ASQ system provides the opportunity to extend parents' knowledge of child development as they observe and recognize the skills of their children, contributing to their training and coresponsibility from the first moment in the child's life. Conflict of interest statement The authors Sonia Lopes, Patrícia Graça, Salete Teixeira and Ana Maria Serrano declare that there are no conflicts of interest. Jane Squires declares a conflict of interest as an author of the ASQ who receives publications royalties. Acknowledgments This paper has been funded with the support of national funds through FCT-Foundation for Science and Technology within the project PEst-OE/CED/UI1661/2014 do CIEd-UM. References [1] National Scientific Council on the Developing Child. Early experiences can alter gene expression and affect long-term development: working paper no. 10 [Internet]. Gene expression. Cambridge: Center on the Developing Child at Harvard University; 2010[Available from: www.developingchild.harvard.edu]. [2] Shonkoff JP, Levitt P. Neuroscience and the future of early childhood policy: moving from why to what and how. Neuron Sep 9 2010;67(5):689–91 [[Internet]. [cited 2014 May 4]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20826301]. [3] Heckman JJ. Skill formation and the economics of investing in disadvantaged children. Science Jun 30 2006;312(5782):1900–2 [[Internet]. [cited 2014 Apr 30]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16809525]. [4] Rosenberg SA, Zhang D, Robinson CC. Prevalence of developmental delays and participation in early intervention services for young children. Pediatrics Jun 2008; 121(6):e1503–9 [[Internet]. [cited 2014 May 3]. Available from: http://www.ncbi. nlm.nih.gov/pubmed/18504295]. [5] Glascoe FP, Shapiro HL. Introduction to developmental and behavioral screening. Dev Behav Pediatr 2004 [online.[cited 2011 Jan 10]. Available from: http://www. dbpeds.org/articles/detail.cfm?id = 5]. [6] Halfon N, Regalado M, Sareen H, Inkelas M, Reuland C, Glascoe FP, et al. Assessing development in the pediatric office. Pediatrics 2004;113(6 Suppl.):1926–33 [[Internet]. Available from: bGo to ISIN://WOS:000221782100005]. [7] Dixon G, Badawi N, French D, Kurinczuk JJ. Can parents accurately screen children at risk of developmental delay? J Paediatr Child Health May 2009;45(5):268–73 [[Internet]. [cited 2014 May 1]. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/19493118]. [8] Dobrez D, Sasso AL, Holl J, Shalowitz M, Leon S, Budetti P. Estimating the cost of developmental and behavioral screening of preschool children in general pediatric practice. Pediatrics Oct 1 2001;108(4):913–22 [[Internet]. [cited 2014 May 1]. Available from: http://pediatrics.aappublications.org/content/108/4/913.abstract]. [9] Earls MF, Hay SS. Setting the stage for success: implementation of developmental and behavioral screening and surveillance in primary care practice—the North Carolina Assuring Better Child Health and Development (ABCD) Project. Pediatrics Jul 2006;118(1):e183–8 [[Internet]. [cited 2014 May 2]. Available from: bGo to ISIN://000238726100086]. [10] Pinto-Martin JA, Dunkle M, Earls M, Fliedner D, Landes C. Developmental stages of developmental screening: steps to implementation of a successful program. Am J Public Health Nov 2005;95(11):1928–32 [[Internet]. [cited 2014 May 3]. Available from: bGo to ISIN://000233050800017].
[11] Rydz D, Shevell MI, Majnemer A, Oskoui M. Topical review: developmental screening. J Child Neurol Jan 1 2005;20(1):4–21 [[Internet]. [cited 2014 May 3]. Available from: http://jcn.sagepub.com/cgi/doi/10.1177/08830738050200010201]. [12] Committee on Children With Disabilities. Developmental Surveillance and Screening of Infants and Young Children. Pediatrics Jul 1 2001;108(1):192–5 [[Internet]. [cited 2014 May 1]. Available from: http://pediatrics.aappublications.org/content/108/1/ 192.full]. [13] American Academy of Pediatrics. Developmental Screening in Early Childhood Systems[Internet] ; 2009[Elk Grove Village. Available from: http://www. healthychildcare.org/pdf/DSECSreport.pdf]. [14] Gilliam WS, Meisels SJ, Mayes LC. Screening and surveillance in early intervention systems. In: Guralnick MJ, editor. The developmental systems approach to early intervention. Baltimore: Brookes Publishing Company; 2005. p. 73–98. [15] LaRosa A. Developmental and behavioral screening tests in primary care. [Internet]. UpToDate. 2011 [cited 2012 Jan 10]. Available from: http://www.uptodate.com/ contents/developmental-and-behavioral-screening-tests-in-primary-care#H10. [16] Marks KP, Glascoe FP, Macias MM. Enhancing the algorithm for developmentalbehavioral surveillance and screening in children 0 to 5 years. Clin Pediatr (Phila) Sep 2011;50(9):853–68 [[Internet]. [cited 2014 May 3]. Available from: bGo to ISIN://WOS:000294504500009]. [17] Smith LE, Akai CE, Klerman LV, Keltner BR. What mothers don't know and doctors don't say: detecting early developmental delays. Infant Ment Health J Jul 8 2010; 31(4):455–66 [[Internet]. [cited 2014 May 4]. Available from: http://doi.wiley. com/10.1002/imhj.20266]. [18] Brassard MR, Boehm AE. Preschool assessment: principles and practices. New York: Guilford Press; 2007. [19] Guralnick MJ, Conlon C. Early intervention. In: Batshaw M, Pelligrino L, Roizen N, editors. Children with disabilities. 6th ed. Baltimore: Paul H. Brookes; 2007. p. 511–21. [20] Decreto-Lei no 281/2009 de 6 de Outubro do Ministério da Saúde. [Internet] Diário da República: I Série, no 131; 2009[Available from: http://dre.pt/pdf1sdip/2009/10/ 19300/0729807301.pdf]. [21] Squires J, Twombly E, Bricker D, Potter L. ASQ-3 user's guide. Baltimore, Maryland: Brookes Publishing Co; 2009. [22] Macy M. The evidence behind developmental screening instruments. Infants Young Child 2012;25(1):19–61 [[Internet]. [cited 2014 Apr 30]. Available from: http:// content.wkhealth.com/linkback/openurl?sid = WKPTLP:landingpage&an = 00001163-201201000-00003]. [23] Alvik A, Grøholt B. Examination of the cut-off scores determined by the Ages and Stages Questionnaire in a population-based sample of 6 month-old Norwegian infants. BMC Pediatr Jan 2011;11(1):117 [[Internet]. [cited 2014 Apr 30]. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid = 3293730&tool = pmcentrez&rendertype = abstract]. [24] Filgueiras A, Pires P, Maissonette S, Landeira-Fernandez J. Psychometric properties of the Brazilian-adapted version of the Ages and Stages Questionnaire in public child daycare centers. Early Hum Dev 2013;89(8):561–76. [25] Pomés MP. Examination of the Spanish Translation of a Developmental Screening Instrument [Internet]. Unpublished doctoral dissertation, University of Oregon, Oregon, EUA.; 2012. Available from: http://hdl.handle.net/1794/12332. [26] Schonhaut L, Armijo I, Schönstedt M, Alvarez J, Cordero M. Validity of the ages and stages questionnaires in term and preterm infants. Pediatrics May 1 2013;131(5): e1468–74 [[Internet]. [cited 2014 May 1]. Available from: http://www.ncbi.nlm. nih.gov/pubmed/23629619]. [27] Kyerematen V, Hamb A, Oberhelman RA, Cabrera L, Bernabe-Ortiz A, Berry SJ. Exploratory application of the Ages and Stages (ASQ) child development screening test in a low-income Peruvian shantytown population. BMJ Open Jan 10 2014; 4(1):e004132 [[Internet]. [cited 2014 May 4]. Available from: http://bmjopen.bmj. com/content/4/1/e004132.abstract]. [28] Hornman J, Kerstjens JM, de Winter AF, Bos AF, Reijneveld SA. Validity and internal consistency of the Ages and Stages Questionnaire 60-month version and the effect of three scoring methods. Early Hum Dev Dec 2013;89(12):1011–5 [[Internet]. [cited 2014 May 4]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24041814]. [29] D'Aprano AL, Carapetis JR, Andrews R. Trial of a developmental screening tool in remote Australian Aboriginal communities: a cautionary tale. J Paediatr Child Health 2011:12–7. [30] Bian X, Yao G, Squires J, Hoselton R, Chen C-I, Murphy K, et al. Translation and use of parent-completed developmental screening test in Shanghai. J Early Child Res Mar 27 2012;10(2):162–75 [[Internet]. Available from: http://ecr.sagepub.com/cgi/doi/ 10.1177/1476718X11430071]. [31] Kvestad I, Taneja S, Kumar T, Bhandari N, Strand TA, Hysing M. The assessment of developmental status using the Ages and Stages questionnaire-3 in nutritional research in north Indian young children. Nutr J Jan 2013;12:50 [[Internet]. [cited 2014 May 4]. Available from: http://www.pubmedcentral.nih.gov/articlerender. fcgi?artid = 3637815&tool = pmcentrez&rendertype = abstract]. [32] Hill MM, Hill A. Investigação por Questionário. 2nd ed. Lisboa: Edições Sílabo; 2009.
S. Lopes et al. / Early Human Development 91 (2015) 527–533 [33] Aly Z, Taj F, Ibrahim S. Missed opportunities in surveillance and screening systems to detect developmental delay: a developing country perspective. Brain Dev 2010;32: 90–7 [[Internet]. Available from: bGo to ISIN://WOS:000274710000003]. [34] Levine LE, Munsch J. Child development: an active learning approach. Los Angeles: Sage; 2011. [35] Papalia DE, Olds SW, Feldman RD. Desenvolvimento humano. 8th ed. São Paulo: Artmed; 2006. [36] Copple C, Bredekamp S. Developmentally appropriate practice in early childhood programs serving children from birth through age 8. In: Copple C, Bredekamp S, editors. 3rd ed. Washington, DC: National Association for the Education of Young Children; 2009. [37] World Health Organization. World report on disability [Internet]. Geneva. Available from: http://www.who.int/disabilities/world_report/2011/report.pdf; 2011.
533
[38] World Health Organization. Early childhood development and disability: discussion paper [Internet]. Geneva: World Health Organization and UNICEF; 2012[Available from: http://apps.who.int/iris/bitstream/10665/75355/1/9789241504065_eng.pdf? ua=1]. [39] Bricker D, Squires J. The effectiveness of parental screening of at-risk infants: the infant monitoring questionnaires. Top Early Child Spec Educ 1989:67–85. [40] Nelson BB, Chung PJ, DuPlessis HM, Flores L, Ryan GW, Kataoka SH. Strengthening families of children with developmental concerns: parent perceptions of developmental screening and services in Head Start. Ethn Dis 2011;21(3 Suppl. 1):89–93 [[Internet]. Available from: bGo to ISIN://MEDLINE:22352085].