- Email: [email protected]
Psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire
Brain & Development 37 (2015) 501–507 www.elsevier.com/locate/braindev
Original article
Psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire Yuki Shibata a,⇑, Kaori Okada b, Rie Fukumoto b, Kenji Nomura b b
a Research Center for Psychological Science, Doshisha University, Japan Center for Developmental Clinical Psychology and Psychiatry, Nagoya University, Japan
Received 25 February 2014; received in revised form 16 July 2014; accepted 4 August 2014
Abstract Objective: This study examined the psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ). Method: Parents and teachers of 1487 elementary school children (759 boys and 728 girls aged 6–12 years) participated in this study. Results: The results of confirmatory factor analyses of the parent and teacher versions of the SDQ supported the five-factor structure reported in previous studies. However, factor invariance across sex was not observed. The alpha coefficients for the subscales of the SDQs varied between 0.55 and 0.86, the same reliability measures that were also reported in previous studies. Moreover, analyses of variance showed significant differences on all of the subscales according to sex and teacher–parent ratings. Conclusion: The factor structure of the SDQ was generally supported, but more gender-segregated investigations of the factor structures are needed. Parents tended to give higher ratings on the difficulties and strengths of children compared to the teachers. Boys were rated higher than girls were on difficulties, while girls were rated higher than boys were on strengths. Ó 2014 The Japanese Society of Child Neurology. Published by Elsevier B.V. All rights reserved.
Keywords: Elementary school; Factor structure; Factor invariance; Sex differences; Teacher-rated; Parent-rated; Reliability
1. Introduction The Strengths and Difficulties Questionnaire (SDQ) is an easy-to-use, 25-item instrument designed to assess children and youth for the presence of maladaptive behaviors [1]. Previous studies have revealed a relationship between the SDQ and more severe problems, such as attention deficit hyperactivity disorder, emotional ⇑ Corresponding author. Address: Research Center for Psychological Science, Doshisha University, 1-3 Tatara Miyakodani, Kyotanabe-shi, Kyoto-fu 610-0394, Japan. Tel.: +81 75 251 3969; fax: +81 75 251 3750. E-mail address: [email protected] (Y. Shibata).
disorder, conduct disorder [2,3], and psychiatric disorders [4]. The SDQ is widely used as a screening tool for the early detection of behavioral problems in children and youth. It consists of four subscales (emotional symptoms, conduct problems, hyperactivity–inattention, and peer problems), which have a total score for evaluating behavioral difficulties (total difficulties score), and one subscale (prosocial behavior) for evaluating behavioral adaptation. Moderate to high reliability coefficients have been reported for the parent (a = 0.57–0.82), teacher (a = 0.70–0.87), and youth versions (a = 0.41–0.80) of the SDQ [5]. An advantage of the SDQ is that the parents or teachers of 4–16-year-olds can evaluate the children for various problems, and the
http://dx.doi.org/10.1016/j.braindev.2014.08.001 0387-7604/Ó 2014 The Japanese Society of Child Neurology. Published by Elsevier B.V. All rights reserved.
502
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
children (11–16-year-olds) can evaluate themselves for the same problems. The present study however, focuses on the parent and teacher versions of the SDQ as a screening instrument for behavioral problems in elementary school children. Parents’ and teachers’ observations of children in their everyday activities differ, since parents observe children primarily at home, while the teachers observe them primarily at school. It is not surprising that the correlations between the teacher and parent ratings are consistently moderate to high [1,5]. In addition, the cutoff points of the SDQ scores differ between the parent and teacher versions, as parents tend to give higher ratings of difficulties in their children compared to the teachers’ ratings [1]. With respect to sex differences in the behavioral adaptation of children, teachers report almost twice as many behavioral problems in elementary and middle school boys compared to girls of the same age [6]. The teacher-rated reports on the SDQ reflect a tendency of scores on the difficulties subscales to be higher for boys than girls, who receive higher scores on the strengths subscales [7,8]. The same trend is observed in the parent-rated SDQ as well, with boys scoring higher on the difficulties subscales both at school and at home. The SDQ is widely used as a five-factor scale comprising emotional symptoms, conduct problems, hyperactivity–inattention, peer problems, and prosocial behavior. Although a study by Goodman [5] found a robust and simple structure in the parent-rated SDQ, it did not find a robust five-factor structure in the teacher-rated SDQ, which had a few items that either had a loading on multiple factors, or did not load on the general factors. Some previous studies that conducted confirmatory factor analyses on the five-factor model reported that the goodness of fit of the five-factor model is inadequate. For example, it has been reported that the goodness of fit of the teacher version is lower than the parent version [5,9], and that the loading is inadequate for some items, such as item 22, “steals from home, school, or elsewhere” [10]. Debates on the appropriateness of the factor structure of the SDQ continue [11]. Given the sex differences in the scores on the difficulties and strengths subscales, it may be presumed that there are different factor structures for boys and for girls. However, few studies have investigated the factor invariance across sex. One of those studies, d’Acremont and Vander Linden [10], reported a strong factor invariance across sex in a slightly modified five-factor model of the teacher-rated SDQ, with a sample of middle-school children. However, the age group of their study’s participants [10] was restricted to the degree that the results did not shed light on the factor invariance across sex in the total SDQ target age group. In Japan, problems related to the mental health of children are considered extremely important; therefore,
clarification of the psychometric properties of the SDQ is imperative. Using a large-scale dataset from a sample of 4–12-year-olds, Matsuishi et al. [12] found that the Japanese version of the SDQ for parents has a fivefactor structure, similar to that of Goodman’s [5] SDQ version for parents. This study also found that although the alpha coefficients for the peer problems and conduct problems subscales were low (0.52 each), the total difficulties scores, and the scores for conduct problems, hyperactivity/inattention, and peer problems were significantly higher for boys than girls. On the other hand, the scores on the emotional symptoms and prosocial behavior subscales were significantly higher for girls than boys. The results of the Matsuishi et al. [12] study were obtained from an exploratory factor analysis, but Mellor and Stokes [9], who investigated the factor structure of the SDQ, insisted that confirmatory factor analysis, and not exploratory factor analysis, should be used to verify the appropriateness of the factor structure. In response to this argument, Nishimura and Koizumi [13] conducted a confirmatory factor analysis of the teacher-rated SDQ with a sample of 4–6-year-old kindergarten children. The goodness of fit of the five-factor structure reported by Nishimura and Koizumi [13] was adequate, but as reported by d’Acremont and Vander Linden [10], the factor loading of item 22 was extremely low, and it was difficult to confirm the factor structure for conduct problems. The knowledge gained by Nishimura and Koizumi [13] is extremely important, but since their sample was restricted to 4–6-year-olds, the data are extremely limited and the instrument requires further research to extend our knowledge of its psychometric properties. Therefore, the present study was conducted to clarify the psychometric properties of the SDQ with a sample of elementary school children, using the Japanese versions of SDQ for parents and teachers. The specific aim was to investigate the reproducibility of the five-factor structure advocated by Goodman [5], by conducting confirmatory factor analysis. We also investigated the association between the parent and teacher versions, as well as sex differences, with the objective of learning about the assessment of difficulties and strengths in elementary school children. 2. Methods 2.1. Participants The study sample consisted of children (from 6 to 12 years old) in grades 1–6 of all public elementary schools in a city in Aichi prefecture. Consent to participate in the study was obtained in writing from the parents of 1578 children (consent rate = 82.1%) after an explanation of the study procedure was provided. The parents and teachers of the children also participated
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
in the study. After excluding questionnaires with missing data, we analyzed the questionnaires from the parents and teachers of 1487 children (759 boys; 728 girls). 2.2. Measures We used the Japanese version of the SDQ published by the Ministry of Health, Labour and Welfare [14]. The parents were asked to complete the SDQ for their own children, and the teachers were asked to complete it for the children in their classrooms. The SDQ is a 25-item scale consisting of five subscales (five items in each subscale). Parents and teachers were asked to answer the 25 items of the SDQ on a 3-point scale, assigning (0) for “not true,” (1) for “somewhat true,” and (2) for “certainly true.” As per the original version of the SDQ [5], the scores were inverted and calculated so that higher scores indicated higher difficulty for “emotional symptoms,” “conduct problems,” “hyperactivity–inattention,” and “peer problems” (scores for all subscales ranged from 0–10 points). For “prosocial behavior,” the scores ranged from 1 to 10 points, with higher scores assigned to higher ratings of sociability. For the total difficulties score (i.e., the four difficulty subscales), scores ranged from 0 to 40 points, with higher scores reflecting higher difficulty.
503
SDQ for the parents and the SDQ for the teachers permitted the examination of the factor invariance across sex. Four models with different levels of restriction on the equivalence between sexes were created and simultaneous analyses of several groups were conducted. In model 1, the equivalence between sex was restricted for only the layout of factors and items; in model 2, it was restricted for the layout and factor patterns; in model 3, it was restricted for error variance in the observed variables in addition to model 2; and in model 4, it was restricted for variance/covariance of factors in addition to model 3. The goodness of fit of the models was evaluated using v2, the comparative fit index (CFI), the adjusted goodness of fit index (AGFI), and the root mean square error of approximation (RMSEA), and the models were compared using the Akaike information criteria (AIC). In order to investigate the difference between the parents’ and teachers’ evaluations of the children, a two-way repeated measures analysis of variance (evaluator (parent and teacher) sex (boys and girls)) was performed on the subscales of the SDQ for the parents and the teachers. All statistical analyses were conducted using SPSS PASW Statistics 18 and IBM SPSS Amos 21. 3. Results
2.3. Procedures First, the consent of the parents allowing their children to participate in the study was solicited in conjunction with the distribution and collection of the questionnaires. The consent forms and the questionnaires describing the SDQ to the parents were distributed to them by the schools. Next, the consent forms and the completed questionnaires were collected at the same time by the schools from the parents. Therefore, the reply rate was the same as the consent rate (82.1%) reported in Section 2.1. The SDQ questionnaires for the teachers of the children for whom consent was obtained were also distributed to and collected from the teachers by the schools. The study of the parents was conducted in July 2013 and the study of the teachers, in September of the same year. The Ethics Committee of the Graduate School of Education and Human Development, Nagoya University (approval number 298), approved the study. 2.4. Statistical analyses In order to investigate whether the parent-rated and teacher-rated SDQ versions had the same five-factor structure as the original version, models were created based on the association between the observed variables and the factors reported by Goodman [5]. Confirmatory factor analyses were conducted using the maximum-likelihood procedure. Using the five-factor model of the
3.1. Factor structure of the parent-rated and teacher-rated SDQs Confirmatory factor analysis was conducted for the parent-rated and teacher-rated SDQs (Table 1). The goodness of fit for the parent-rated SDQ was not excellent, but satisfactory. For the total sample, the indices were v2 (265) = 1708.575, p = 0.000, AGFI = 0.882, CFI = 0.816, RMSEA = 0.081. For the boys, the indices were v2 (265) = 1045.993, p = 0.000, AGFI = 0.864, CFI = 0.793, RMSEA = 0.062. For the girls, the indices were v2 (265) = 1043.453, p = 0.000, AGFI = 0.860, CFI = 0.809, RMSEA = 0.064. The goodness of fit for the teacher-rated SDQ was not excellent, but satisfactory. For the total sample, the indices were v2 (265) = 2951.173, p = 0.000, AGFI = 0.810, CFI = 0.824, RMSEA = 0.081. For the boys, they were v2 (265) = 1654.701, p = 0.000, AGFI = 0.802, CFI = 0.822, RMSEA = 0.081. For the girls, they were v2 (265) = 1755.936, p = 0.000, AGFI = 0.779, CFI = 0.775, RMSEA = 0.087. Compared to the SDQ for the parents, the goodness of fit in the SDQ for the teachers was somewhat low, particularly for the girls. In all of the models, the standardized path coefficients from the factors to the items were significant, but the values in the path from “conduct problems” to item 22 (“steals”) were low. The covariances among the factors were significant for all of the subscales except for the “prosocial behavior” and
504
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
Table 1 Standardized coefficients for the five-factor models of the SDQs and alpha coefficients (Total sample N = 1487, Boys: n = 759, Girls: n = 728). Parent-rated SDQ
Teacher-rated SDQ
Total
Boys
Girls
Total
Boys
Girls
I Hyperactivity/inattention 2 Restless 10 Fidgety 15 Distractible 21 Reflectivea 25 Persistenta
(a = 0.768) 0.762 0.465*** 0.797*** 0.565*** 0.572***
(a = 0.768) 0.766 0.495*** 0.775*** 0.584*** 0.532***
(a = 0.750) 0.746 0.428*** 0.817*** 0.504*** 0.587***
(a = 0.858) 0.835 0.622*** 0.883*** 0.669*** 0.706***
(a = 0.859) 0.835 0.594*** 0.900*** 0.679*** 0.700***
(a = 0.807) 0.755 0.620*** 0.806*** 0.612*** 0.686***
II Prosocial behavior 1 Considerate 4 Shares 9 Caring 17 Kind to Kids 20 Helps out
(a = 0.730) 0.660 0.524*** 0.693*** 0.539*** 0.565***
(a = 0.720) 0.679 0.502*** 0.708*** 0.529*** 0.525***
(a = 0.717) 0.624 0.530*** 0.666*** 0.513*** 0.576***
(a = 0.846) 0.797 0.624*** 0.809*** 0.700*** 0.707***
(a = 0.834) 0.778 0.630*** 0.804*** 0.660*** 0.682***
(a = 0.838) 0.783 0.592*** 0.792*** 0.712*** 0.710***
III Emotional symptoms 3 Somatic complaints 8 Worries 13 Unhappy 16 Clingy 24 Fears
(a = 0.656) 0.332 0.668*** 0.520*** 0.608*** 0.563***
(a = 0.610) 0.289 0.649*** 0.499*** 0.582*** 0.493***
(a = 0.693) 0.361 0.676*** 0.530*** 0.648*** 0.635***
(a = 0.757) 0.465 0.708*** 0.535*** 0.739*** 0.662***
(a = 0.743) 0.440 0.664*** 0.490*** 0.754*** 0.641***
(a = 0.771) 0.500 0.746*** 0.590*** 0.725*** 0.682***
IV Conduct problems 5 Tempers 7 Obedienta 12 Fights 18 Lies, cheats 22 Steals
(a = 0.552) 0.542 0.455*** 0.494*** 0.570*** 0.198***
(a = 0.529) 0.538 0.466*** 0.473*** 0.543*** 0.129***
(a = 0.521) 0.541 0.431*** 0.526*** 0.597*** 0.305***
(a = 0.727) 0.714 0.628*** 0.759*** 0.647*** 0.174***
(a = 0.750) 0.744 0.653*** 0.795*** 0.648*** 0.145***
(a = 0.612) 0.520 0.567*** 0.567*** 0.634*** 0.259***
V Peer problems 6 Solitary 11 Good friendsa 14 Populara 19 Picked on, bullied 23 Best with adults
(a = 0.559) 0.446 0.340*** 0.379*** 0.556*** 0.537***
(a = 0.551) 0.517 0.320*** 0.381*** 0.509*** 0.495***
(a = 0.570) 0.370 0.341*** 0.366*** 0.607*** 0.598***
(a = 0.656) 0.383 0.612*** 0.724*** 0.429*** 0.460***
(a = 0.690) 0.440 0.617*** 0.735*** 0.438*** 0.483***
(a = 0.600) 0.250 0.613*** 0.773*** 0.378*** 0.361***
Covariances IMI I M III I M IV IMV II M III II M IV II M V III M IV III M V IV M V
0.351*** 0.356*** 0.734*** 0.424*** 0.031 0.494*** 0.310*** 0.506*** 0.664*** 0.546***
0.277*** 0.358*** 0.685*** 0.403*** 0.022 0.496*** 0.334*** 0.487*** 0.656*** 0.430***
0.358*** 0.405*** 0.762*** 0.458*** 0.074 0.452*** 0.280*** 0.534*** 0.683*** 0.697***
0.504*** 0.357*** 0.642*** 0.525*** 0.057 0.466*** 0.591*** 0.302*** 0.464*** 0.598***
0.490*** 0.399*** 0.594*** 0.513*** 0.099* 0.455*** 0.575*** 0.314*** 0.556*** 0.605***
0.411*** 0.414*** 0.662*** 0.564*** 0.032 0.441*** 0.639*** 0.373*** 0.322*** 0.656***
a Coefficients of the total Difficulties scores: parent total a = 0.690, parent boys a = 0.675, parent girls a = 0.713, teacher total a = 0.695, teacher boys a = 0.722, and teacher girls a = 0.672. *** p < 0.001, *p < 0.05. a Scores on these items were reverse coded before they were entered into the analysis.
“emotional symptoms” subscales, and they were the highest between the “hyperactivity/inattention” and “conduct problems” subscales (the range of standardized coefficients was 0.594–0.762). All of the other subscales showed moderate associations. The covariance between the “prosocial behavior” and “emotional symptoms” subscales was insignificant in almost all of the models. Even for the boys in the teacher-rated SDQ, in which the covariance was significant, there were very few standardized coefficients.
3.2. Factor invariance of the parent-rated and teacherrated SDQs across sex Next, the factor invariance across sex was tested in the five-factor models of the parent-rated and teacherrated SDQs. The results showed the goodness of fit being at the same level in both the parent-rated and teacher-rated SDQs. The AIC values in the models, starting with model 1, were 2348.018, 2354.727, 2474.768, and 2503.768 in parent-rated SDQ; and
***
***
n.s.
***
*
***
**
***
* ***b *a
(2.08) (1.80) (1.57) (1.56) 6.57 1.54 1.85 1.62
6.15 1.43 2.00 1.65
(2.11) (1.69) (1.59) (1.59)
7.00 1.65 1.70 1.60
(1.96) (1.91) (1.53) (1.54)
6.05 0.92 1.01 1.37
(2.72) (1.68) (1.63) (1.66)
5.36 0.87 1.38 1.51
(2.70) (1.62) (1.91) (1.79)
6.78 0.98 0.62 1.23
(2.55) (1.73) (1.16) (1.50)
*** *** ***
***
Girls
*** * *** ***
(4.74) (2.08) 4.57 1.74 (6.14) (2.90) 7.23 3.48 (5.66) (2.68) 5.93 2.63 (5.15) (2.06) 7.74 2.79 (5.05) (2.32) 8.75 3.66 (5.13) (2.24) 8.25 3.24
Mean (SD) Mean
Girls
(SD)
*** *** ***
SDQs
Boys
Sex
Teacher
(SD) Mean
Boys
Mean
(SD) Total Boys
(SD)
Main effects were recorded for emotional symptoms. (a) Shows the main effects by gender, while (b) shows the parent-teacher main effects. p < 0.001. ** p < 0.01. * p < 0.05.
Girls 0.390*** 0.441*** 0.218*** 0.273*** 0.236*** 0.322***
***
p < 0.001.
Boys 0.447*** 0.450*** 0.191*** 0.246*** 0.330*** 0.456***
a,b
***
Total 0.428*** 0.476*** 0.245*** 0.262*** 0.302*** 0.396***
Total difficulties score Hyperactivity/ inattention Prosocial behavior Emotional symptom Conduct problems Peer problems
Total difficulties score Hyperactivity/inattention Prosocial behavior Emotional symptom Conduct problems Peer problems
Mean
Table 2 Pearson’s correlations between parent-rated and teacher-rated SDQs (Total N = 1487, Boys: n = 759, Girls: n = 728).
Total
The average of the total difficulties score and each subscale of the parent-rated SDQ and teacher-rated SDQ were calculated by sex (Table 3). A two-way repeated measures analysis of variance was performed. The main effects of evaluator and sex were observed in the emotional symptoms subscale. Compared to the teachers, the parents gave significantly higher ratings to the boys than to the girls (evaluator: F (1, 1485) = 126.62,
Teacher-rated SDQ
3.5. Two-way repeated measures analysis of variance for parent-rated and teacher-rated SDQs
Parent-rated SDQ
The Pearson’s correlation coefficients between the parents’ and teachers’ ratings were calculated for the total sample, boys and girls (Table 2). The correlation coefficients were 0.245–0.476 for the total sample, 0.191–0.456 for the boys and 0.218–0.441 for the girls, showing a moderate correlation. Among all of the groups, the subscale showing the lowest correlation coefficient was prosocial behavior. The subscales showing the highest correlation coefficients were hyperactivity/inattention in the total sample and the girls, and peer problems in the boys.
Table 3 Means and standard deviations for parent-rated and teacher-rated SDQs (Total N = 1487, Boys: n = 759, Girls: n = 728).
3.4. Parent-teacher correlations
Mean
Girls
(SD)
Cronbach’s a coefficient was calculated for each subscale of the parent-rated and teacher-rated SDQs. The alpha level was between 0.521 and 0.768 in the parent version and between 0.600 and 0.859 in the teacher version, with the teacher version showing a somewhat higher reliability. At less than 0.60, the reliability of the conduct problems and peer problems subscales was extremely low in the parent version.
Parent
Simple main effect
3.3. Internal consistency
505
Sex
3801.405, 3856.800, 4719.748, and 5080.935 in the teacher-rated SDQ. In both the parent-rated and teacherrated SDQs, model 1 with the lowest AIC was adopted (parent-rated SDQ: v2 (532) = 2112.018, p = 0.000, AGFI = 0.861, CFI = 0.798, RMSEA = 0.045; teacher-rated SDQ: v2 (532) = 3565.405, p = 0.000, AGFI = 0.772, CFI = 0.782, RMSEA = 0.062).
SDQs
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
506
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
p < 0.001; sex: F (1, 1485) = 5.23, p < 0.05). Moreover, a significant interaction was observed in the total difficulties, conduct problems, hyperactivity/inattention, peer problems, and prosocial behavior scores (total difficulties score: F (1, 1485) = 31.13, p < 0.001; conduct problems: F (1, 1485) = 22.61, p < 0.001; hyperactivity/inattention: F (1, 1485) = 44.58, p < 0.001; peer problems: F (1, 1485) = 6.051, p < 0.05; and prosocial behaviors: F (1, 1485) = 13.54, p < 0.001). Accordingly, the simple main effects were tested using the Bonferroni test, which showed that the average scores of the parents were significantly higher than those of the teachers in all of the subscale scores in both the boys and girls. Moreover, in the parent version, the average scores on the total difficulties, hyperactivity/inattention, and conduct behavior subscales were significantly higher in the boys than the girls. The average scores on the prosocial behavior and emotional symptoms subscales were significantly higher in the girls than the boys. In the teacher version, the average scores of the total difficulties, hyperactivity/inattention, conduct behavior, and peer problems scores were significantly higher in the boys compared to the girls, and the averages on the prosocial behavior subscales were significantly higher in the girls than in the boys. 4. Discussion The aim of this study was to investigate the psychometric properties of the parent and teacher forms of the Japanese version of the SDQ. To meet this objective, the following analyses were conducted: factor structure, factor invariance across sex, internal consistency, correlations between the parent and teacher versions, and differences in averages between sexes and forms of the SDQ. Although the five-factor structure of the Japanese version of the parent-rated and teacher-rated SDQs was supported, it was evident that it was not a robust structure. Moreover, it was observed that difficulties were assessed to be higher in boys than girls, and that the parents gave higher ratings on childrens’ difficulties and strengths than did the teachers. Here, we shall consider the factor structure, internal consistency, the assessments of boys and girls, and the evaluations made by parents and teachers. The results of the confirmatory factor analyses in this study for the boys and girls on the parent-rated and teacher-rated SDQs generally supported the five-factor structure advocated by Goodman [5]. Even though the goodness of fit was slightly better in the parent version compared to the teacher version, it was not judged excellent in either of the versions. As in the previous studies [10,13], item 22 (“steals from home, school or elsewhere”) had an extremely low factor loading on all of the models in the present study, and there was not a high enough factor loading to constitute a factor of conduct problem. Consistent with the results of a previous
research [12], the present study found that peer problems and conduct problems in the parents’ version of the scale had low reliabilities. Of course, subscales with fewer items generally provide lower reliability; however, other subscales with the same number of items had moderate levels of reliability. As seen in Table 1, these two subscales had many items with low factor loadings, and therefore it is presumed that the instability of the factor structure lowered the reliabilities of the subscales. Thus, future research is necessary to examine whether the factor instability of the scale stems from a problem of the equivalence of item representation or from cultural differences in the parental assessments of children. In this study, factor invariance across sex was observed in neither the parent nor the teacher version, suggesting that factor structures were different for boys and girls. In the recent years, the SDQ factor structure has been debated and new factor structures have been proposed for the SDQ [11]. Thus, in order to contribute to the development of a scale structure that accurately evaluates adaptive and maladaptive behaviors among boys and girls, future studies are needed to explore the possibilities of factor structures across sex are required. As in the previous studies [7,8,12], difficulties in the boys were evaluated to be higher in comparison to the girls, while the strengths in the girls were evaluated higher in comparison to the boys. The results of the present study of the parent version were consistent with those of Matsuishi et al. [12], with difficulty in emotional symptoms rated higher in the girls than the boys. On the other hand, sex differences in peer problems were observed in only the teacher version, showing that parents and teachers judged the level of difficulty in boys and girls differently. In the present study, the parents assessed higher levels of difficulty in their children as compared to the teachers, as predicted by Goodman [1]. As suggested by the moderate correlations between the teachers and the parents in the present and previous studies, these results indicate that this could be because parents and teachers observe children in different contexts. Therefore, to be able to use the SDQ to assess the specific characteristics of children, it will be necessary to extract the details of the characteristics’ differences that need to be evaluated by parents and teachers, including sex differences. The present study reports the psychometric properties, such as the factor structure and average values, by gender in the Japanese versions of parent-rated and teacher-rated SDQs. The study also provides vital information about the use of the SDQ, which is projected to be a very efficient instrument for detecting behavioral problems in elementary school children. However, this study has several limitations. First, the applicable age group for the parent version and teacher version of the SDQ is 4–16 years, while the participants of this study were aged 6–12 years. Second, the findings of this study
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
are based on a comprehensive survey of elementary schools of a particular town, and are limited in terms of the generalizability of the results. Additionally, a careful interpretation of the study’s results is necessary because the reliability levels of the peer problems and conduct problems subscales of the parents’ version were low. Furthermore, the parents’ and teachers’ ratings were scheduled two months apart, raising concerns about the possibility that the behavior of the children changed during these two months, thereby influencing the results of the study. It will be necessary to accumulate more reliable information in the future by designing studies that include sophisticated research methods and a broader range of childrens’ ages and larger study samples from more diverse settings.
[5]
[6]
[7]
[8]
[9]
Acknowledgment The present study was conducted as a part of the Therapeutic Educational Support Project for Developmental Disorders, funded by a special education research grant from the Ministry of Education, Culture, Sports, Science, and Technology.
[10]
[11]
[12]
References [1] Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry 1997;38:581–6. [2] Becker A, Woerner W, Hasselhorn M, Banaschewski T, Rothenberger A. Validation of the parent and teacher SDQ in a clinical sample. Eur Child Adolesc Psychiatry 2004;13(Suppl. ii):11–6. [3] Ullebø AK, Posserud MB, Heiervang E, Gillberg C, Obel C. Screening for the attention deficit hyperactivity disorder phenotype using the Strengths and Difficulties Questionnaire. Eur Child Adolesc Psychiatry 2011;20:451–8. [4] Goodman R, Ford T, Simmons H, Gatward R, Meltzer H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for
[13]
[14]
507
child psychiatric disorders in a community sample. Br J Psychiatry 2000;177:534–9. Goodman R. Psychometric properties of the Strengths and Difficulties Questionnaire. J Am Acad Child Adolesc Psychiatry 2001;40:1337–45. Ministry of Education, Culture, Sports, Science and Technology Japan. Research on children attending regular classes having possible developmental disorders needing special educational support 2012 (http://www.mext.go.jp/a_menu/shotou/tokubetu/ material/__icsFiles/afieldfile/2012/12/10/1328729_01.pdf) The date of publication: 2012/12/5; the date of access: 2014/2/10 (in Japanese). van Widenfelt BM, Goedhart AW, Treffers PDA, Goodman R. Dutch version of the Strengths and Difficulties Questionnaire (SDQ). Eur Child Adolesc Psychiatry 2003;12:281–9. Woerner W, Becker A, Rothenberger A. Normative data and scale properties of the German parent SDQ. Eur Child Adolesc Psychiatry 2004;13(Suppl. 2):II3–II10. Mellor D, Stokes M. The factor structure of the Strengths and Difficulties Questionnaire. Eur J Psychol Assess 2007;23:105–12. d’Acremont M, Van der Linden M. Confirmatory factor analysis of the Strengths and Difficulties Questionnaire in a community sample of French-speaking adolescents. Eur J Psychol Assess 2008;24:1–8. Goodman A, Lamping DL, Ploubidis GB. When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children. J Abnorm Child Psychol 2010;38:1179–91. Matsuishi T, Nagano M, Araki Y, Tanaka Y, Iwasaki M, Yamashita Y, et al. Scale properties of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ): a study of infant and school children in community samples. Brain Dev 2008;30:410–5. Nishimura T, Koizumi R. Nursery staff assessment of children using the Japanese version of the Strengths and Difficulties Questionnaire (SDQ) (in Japanese). Bulletin of Fukuoka University of Education. Part 4 Education and Psychology 2010;59:103– 9. Ministry of Health, Labour and Welfare. Strengths and Difficulties Questionnaire. 2007 (http://www.mhlw.go.jp/bunya/kodomo/ boshi-hoken07/h7_04d.html) (in Japanese). The date of publication: 2007/1; the date of access: 2014/2/10.
Original article
Psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire Yuki Shibata a,⇑, Kaori Okada b, Rie Fukumoto b, Kenji Nomura b b
a Research Center for Psychological Science, Doshisha University, Japan Center for Developmental Clinical Psychology and Psychiatry, Nagoya University, Japan
Received 25 February 2014; received in revised form 16 July 2014; accepted 4 August 2014
Abstract Objective: This study examined the psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ). Method: Parents and teachers of 1487 elementary school children (759 boys and 728 girls aged 6–12 years) participated in this study. Results: The results of confirmatory factor analyses of the parent and teacher versions of the SDQ supported the five-factor structure reported in previous studies. However, factor invariance across sex was not observed. The alpha coefficients for the subscales of the SDQs varied between 0.55 and 0.86, the same reliability measures that were also reported in previous studies. Moreover, analyses of variance showed significant differences on all of the subscales according to sex and teacher–parent ratings. Conclusion: The factor structure of the SDQ was generally supported, but more gender-segregated investigations of the factor structures are needed. Parents tended to give higher ratings on the difficulties and strengths of children compared to the teachers. Boys were rated higher than girls were on difficulties, while girls were rated higher than boys were on strengths. Ó 2014 The Japanese Society of Child Neurology. Published by Elsevier B.V. All rights reserved.
Keywords: Elementary school; Factor structure; Factor invariance; Sex differences; Teacher-rated; Parent-rated; Reliability
1. Introduction The Strengths and Difficulties Questionnaire (SDQ) is an easy-to-use, 25-item instrument designed to assess children and youth for the presence of maladaptive behaviors [1]. Previous studies have revealed a relationship between the SDQ and more severe problems, such as attention deficit hyperactivity disorder, emotional ⇑ Corresponding author. Address: Research Center for Psychological Science, Doshisha University, 1-3 Tatara Miyakodani, Kyotanabe-shi, Kyoto-fu 610-0394, Japan. Tel.: +81 75 251 3969; fax: +81 75 251 3750. E-mail address: [email protected] (Y. Shibata).
disorder, conduct disorder [2,3], and psychiatric disorders [4]. The SDQ is widely used as a screening tool for the early detection of behavioral problems in children and youth. It consists of four subscales (emotional symptoms, conduct problems, hyperactivity–inattention, and peer problems), which have a total score for evaluating behavioral difficulties (total difficulties score), and one subscale (prosocial behavior) for evaluating behavioral adaptation. Moderate to high reliability coefficients have been reported for the parent (a = 0.57–0.82), teacher (a = 0.70–0.87), and youth versions (a = 0.41–0.80) of the SDQ [5]. An advantage of the SDQ is that the parents or teachers of 4–16-year-olds can evaluate the children for various problems, and the
http://dx.doi.org/10.1016/j.braindev.2014.08.001 0387-7604/Ó 2014 The Japanese Society of Child Neurology. Published by Elsevier B.V. All rights reserved.
502
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
children (11–16-year-olds) can evaluate themselves for the same problems. The present study however, focuses on the parent and teacher versions of the SDQ as a screening instrument for behavioral problems in elementary school children. Parents’ and teachers’ observations of children in their everyday activities differ, since parents observe children primarily at home, while the teachers observe them primarily at school. It is not surprising that the correlations between the teacher and parent ratings are consistently moderate to high [1,5]. In addition, the cutoff points of the SDQ scores differ between the parent and teacher versions, as parents tend to give higher ratings of difficulties in their children compared to the teachers’ ratings [1]. With respect to sex differences in the behavioral adaptation of children, teachers report almost twice as many behavioral problems in elementary and middle school boys compared to girls of the same age [6]. The teacher-rated reports on the SDQ reflect a tendency of scores on the difficulties subscales to be higher for boys than girls, who receive higher scores on the strengths subscales [7,8]. The same trend is observed in the parent-rated SDQ as well, with boys scoring higher on the difficulties subscales both at school and at home. The SDQ is widely used as a five-factor scale comprising emotional symptoms, conduct problems, hyperactivity–inattention, peer problems, and prosocial behavior. Although a study by Goodman [5] found a robust and simple structure in the parent-rated SDQ, it did not find a robust five-factor structure in the teacher-rated SDQ, which had a few items that either had a loading on multiple factors, or did not load on the general factors. Some previous studies that conducted confirmatory factor analyses on the five-factor model reported that the goodness of fit of the five-factor model is inadequate. For example, it has been reported that the goodness of fit of the teacher version is lower than the parent version [5,9], and that the loading is inadequate for some items, such as item 22, “steals from home, school, or elsewhere” [10]. Debates on the appropriateness of the factor structure of the SDQ continue [11]. Given the sex differences in the scores on the difficulties and strengths subscales, it may be presumed that there are different factor structures for boys and for girls. However, few studies have investigated the factor invariance across sex. One of those studies, d’Acremont and Vander Linden [10], reported a strong factor invariance across sex in a slightly modified five-factor model of the teacher-rated SDQ, with a sample of middle-school children. However, the age group of their study’s participants [10] was restricted to the degree that the results did not shed light on the factor invariance across sex in the total SDQ target age group. In Japan, problems related to the mental health of children are considered extremely important; therefore,
clarification of the psychometric properties of the SDQ is imperative. Using a large-scale dataset from a sample of 4–12-year-olds, Matsuishi et al. [12] found that the Japanese version of the SDQ for parents has a fivefactor structure, similar to that of Goodman’s [5] SDQ version for parents. This study also found that although the alpha coefficients for the peer problems and conduct problems subscales were low (0.52 each), the total difficulties scores, and the scores for conduct problems, hyperactivity/inattention, and peer problems were significantly higher for boys than girls. On the other hand, the scores on the emotional symptoms and prosocial behavior subscales were significantly higher for girls than boys. The results of the Matsuishi et al. [12] study were obtained from an exploratory factor analysis, but Mellor and Stokes [9], who investigated the factor structure of the SDQ, insisted that confirmatory factor analysis, and not exploratory factor analysis, should be used to verify the appropriateness of the factor structure. In response to this argument, Nishimura and Koizumi [13] conducted a confirmatory factor analysis of the teacher-rated SDQ with a sample of 4–6-year-old kindergarten children. The goodness of fit of the five-factor structure reported by Nishimura and Koizumi [13] was adequate, but as reported by d’Acremont and Vander Linden [10], the factor loading of item 22 was extremely low, and it was difficult to confirm the factor structure for conduct problems. The knowledge gained by Nishimura and Koizumi [13] is extremely important, but since their sample was restricted to 4–6-year-olds, the data are extremely limited and the instrument requires further research to extend our knowledge of its psychometric properties. Therefore, the present study was conducted to clarify the psychometric properties of the SDQ with a sample of elementary school children, using the Japanese versions of SDQ for parents and teachers. The specific aim was to investigate the reproducibility of the five-factor structure advocated by Goodman [5], by conducting confirmatory factor analysis. We also investigated the association between the parent and teacher versions, as well as sex differences, with the objective of learning about the assessment of difficulties and strengths in elementary school children. 2. Methods 2.1. Participants The study sample consisted of children (from 6 to 12 years old) in grades 1–6 of all public elementary schools in a city in Aichi prefecture. Consent to participate in the study was obtained in writing from the parents of 1578 children (consent rate = 82.1%) after an explanation of the study procedure was provided. The parents and teachers of the children also participated
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
in the study. After excluding questionnaires with missing data, we analyzed the questionnaires from the parents and teachers of 1487 children (759 boys; 728 girls). 2.2. Measures We used the Japanese version of the SDQ published by the Ministry of Health, Labour and Welfare [14]. The parents were asked to complete the SDQ for their own children, and the teachers were asked to complete it for the children in their classrooms. The SDQ is a 25-item scale consisting of five subscales (five items in each subscale). Parents and teachers were asked to answer the 25 items of the SDQ on a 3-point scale, assigning (0) for “not true,” (1) for “somewhat true,” and (2) for “certainly true.” As per the original version of the SDQ [5], the scores were inverted and calculated so that higher scores indicated higher difficulty for “emotional symptoms,” “conduct problems,” “hyperactivity–inattention,” and “peer problems” (scores for all subscales ranged from 0–10 points). For “prosocial behavior,” the scores ranged from 1 to 10 points, with higher scores assigned to higher ratings of sociability. For the total difficulties score (i.e., the four difficulty subscales), scores ranged from 0 to 40 points, with higher scores reflecting higher difficulty.
503
SDQ for the parents and the SDQ for the teachers permitted the examination of the factor invariance across sex. Four models with different levels of restriction on the equivalence between sexes were created and simultaneous analyses of several groups were conducted. In model 1, the equivalence between sex was restricted for only the layout of factors and items; in model 2, it was restricted for the layout and factor patterns; in model 3, it was restricted for error variance in the observed variables in addition to model 2; and in model 4, it was restricted for variance/covariance of factors in addition to model 3. The goodness of fit of the models was evaluated using v2, the comparative fit index (CFI), the adjusted goodness of fit index (AGFI), and the root mean square error of approximation (RMSEA), and the models were compared using the Akaike information criteria (AIC). In order to investigate the difference between the parents’ and teachers’ evaluations of the children, a two-way repeated measures analysis of variance (evaluator (parent and teacher) sex (boys and girls)) was performed on the subscales of the SDQ for the parents and the teachers. All statistical analyses were conducted using SPSS PASW Statistics 18 and IBM SPSS Amos 21. 3. Results
2.3. Procedures First, the consent of the parents allowing their children to participate in the study was solicited in conjunction with the distribution and collection of the questionnaires. The consent forms and the questionnaires describing the SDQ to the parents were distributed to them by the schools. Next, the consent forms and the completed questionnaires were collected at the same time by the schools from the parents. Therefore, the reply rate was the same as the consent rate (82.1%) reported in Section 2.1. The SDQ questionnaires for the teachers of the children for whom consent was obtained were also distributed to and collected from the teachers by the schools. The study of the parents was conducted in July 2013 and the study of the teachers, in September of the same year. The Ethics Committee of the Graduate School of Education and Human Development, Nagoya University (approval number 298), approved the study. 2.4. Statistical analyses In order to investigate whether the parent-rated and teacher-rated SDQ versions had the same five-factor structure as the original version, models were created based on the association between the observed variables and the factors reported by Goodman [5]. Confirmatory factor analyses were conducted using the maximum-likelihood procedure. Using the five-factor model of the
3.1. Factor structure of the parent-rated and teacher-rated SDQs Confirmatory factor analysis was conducted for the parent-rated and teacher-rated SDQs (Table 1). The goodness of fit for the parent-rated SDQ was not excellent, but satisfactory. For the total sample, the indices were v2 (265) = 1708.575, p = 0.000, AGFI = 0.882, CFI = 0.816, RMSEA = 0.081. For the boys, the indices were v2 (265) = 1045.993, p = 0.000, AGFI = 0.864, CFI = 0.793, RMSEA = 0.062. For the girls, the indices were v2 (265) = 1043.453, p = 0.000, AGFI = 0.860, CFI = 0.809, RMSEA = 0.064. The goodness of fit for the teacher-rated SDQ was not excellent, but satisfactory. For the total sample, the indices were v2 (265) = 2951.173, p = 0.000, AGFI = 0.810, CFI = 0.824, RMSEA = 0.081. For the boys, they were v2 (265) = 1654.701, p = 0.000, AGFI = 0.802, CFI = 0.822, RMSEA = 0.081. For the girls, they were v2 (265) = 1755.936, p = 0.000, AGFI = 0.779, CFI = 0.775, RMSEA = 0.087. Compared to the SDQ for the parents, the goodness of fit in the SDQ for the teachers was somewhat low, particularly for the girls. In all of the models, the standardized path coefficients from the factors to the items were significant, but the values in the path from “conduct problems” to item 22 (“steals”) were low. The covariances among the factors were significant for all of the subscales except for the “prosocial behavior” and
504
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
Table 1 Standardized coefficients for the five-factor models of the SDQs and alpha coefficients (Total sample N = 1487, Boys: n = 759, Girls: n = 728). Parent-rated SDQ
Teacher-rated SDQ
Total
Boys
Girls
Total
Boys
Girls
I Hyperactivity/inattention 2 Restless 10 Fidgety 15 Distractible 21 Reflectivea 25 Persistenta
(a = 0.768) 0.762 0.465*** 0.797*** 0.565*** 0.572***
(a = 0.768) 0.766 0.495*** 0.775*** 0.584*** 0.532***
(a = 0.750) 0.746 0.428*** 0.817*** 0.504*** 0.587***
(a = 0.858) 0.835 0.622*** 0.883*** 0.669*** 0.706***
(a = 0.859) 0.835 0.594*** 0.900*** 0.679*** 0.700***
(a = 0.807) 0.755 0.620*** 0.806*** 0.612*** 0.686***
II Prosocial behavior 1 Considerate 4 Shares 9 Caring 17 Kind to Kids 20 Helps out
(a = 0.730) 0.660 0.524*** 0.693*** 0.539*** 0.565***
(a = 0.720) 0.679 0.502*** 0.708*** 0.529*** 0.525***
(a = 0.717) 0.624 0.530*** 0.666*** 0.513*** 0.576***
(a = 0.846) 0.797 0.624*** 0.809*** 0.700*** 0.707***
(a = 0.834) 0.778 0.630*** 0.804*** 0.660*** 0.682***
(a = 0.838) 0.783 0.592*** 0.792*** 0.712*** 0.710***
III Emotional symptoms 3 Somatic complaints 8 Worries 13 Unhappy 16 Clingy 24 Fears
(a = 0.656) 0.332 0.668*** 0.520*** 0.608*** 0.563***
(a = 0.610) 0.289 0.649*** 0.499*** 0.582*** 0.493***
(a = 0.693) 0.361 0.676*** 0.530*** 0.648*** 0.635***
(a = 0.757) 0.465 0.708*** 0.535*** 0.739*** 0.662***
(a = 0.743) 0.440 0.664*** 0.490*** 0.754*** 0.641***
(a = 0.771) 0.500 0.746*** 0.590*** 0.725*** 0.682***
IV Conduct problems 5 Tempers 7 Obedienta 12 Fights 18 Lies, cheats 22 Steals
(a = 0.552) 0.542 0.455*** 0.494*** 0.570*** 0.198***
(a = 0.529) 0.538 0.466*** 0.473*** 0.543*** 0.129***
(a = 0.521) 0.541 0.431*** 0.526*** 0.597*** 0.305***
(a = 0.727) 0.714 0.628*** 0.759*** 0.647*** 0.174***
(a = 0.750) 0.744 0.653*** 0.795*** 0.648*** 0.145***
(a = 0.612) 0.520 0.567*** 0.567*** 0.634*** 0.259***
V Peer problems 6 Solitary 11 Good friendsa 14 Populara 19 Picked on, bullied 23 Best with adults
(a = 0.559) 0.446 0.340*** 0.379*** 0.556*** 0.537***
(a = 0.551) 0.517 0.320*** 0.381*** 0.509*** 0.495***
(a = 0.570) 0.370 0.341*** 0.366*** 0.607*** 0.598***
(a = 0.656) 0.383 0.612*** 0.724*** 0.429*** 0.460***
(a = 0.690) 0.440 0.617*** 0.735*** 0.438*** 0.483***
(a = 0.600) 0.250 0.613*** 0.773*** 0.378*** 0.361***
Covariances IMI I M III I M IV IMV II M III II M IV II M V III M IV III M V IV M V
0.351*** 0.356*** 0.734*** 0.424*** 0.031 0.494*** 0.310*** 0.506*** 0.664*** 0.546***
0.277*** 0.358*** 0.685*** 0.403*** 0.022 0.496*** 0.334*** 0.487*** 0.656*** 0.430***
0.358*** 0.405*** 0.762*** 0.458*** 0.074 0.452*** 0.280*** 0.534*** 0.683*** 0.697***
0.504*** 0.357*** 0.642*** 0.525*** 0.057 0.466*** 0.591*** 0.302*** 0.464*** 0.598***
0.490*** 0.399*** 0.594*** 0.513*** 0.099* 0.455*** 0.575*** 0.314*** 0.556*** 0.605***
0.411*** 0.414*** 0.662*** 0.564*** 0.032 0.441*** 0.639*** 0.373*** 0.322*** 0.656***
a Coefficients of the total Difficulties scores: parent total a = 0.690, parent boys a = 0.675, parent girls a = 0.713, teacher total a = 0.695, teacher boys a = 0.722, and teacher girls a = 0.672. *** p < 0.001, *p < 0.05. a Scores on these items were reverse coded before they were entered into the analysis.
“emotional symptoms” subscales, and they were the highest between the “hyperactivity/inattention” and “conduct problems” subscales (the range of standardized coefficients was 0.594–0.762). All of the other subscales showed moderate associations. The covariance between the “prosocial behavior” and “emotional symptoms” subscales was insignificant in almost all of the models. Even for the boys in the teacher-rated SDQ, in which the covariance was significant, there were very few standardized coefficients.
3.2. Factor invariance of the parent-rated and teacherrated SDQs across sex Next, the factor invariance across sex was tested in the five-factor models of the parent-rated and teacherrated SDQs. The results showed the goodness of fit being at the same level in both the parent-rated and teacher-rated SDQs. The AIC values in the models, starting with model 1, were 2348.018, 2354.727, 2474.768, and 2503.768 in parent-rated SDQ; and
***
***
n.s.
***
*
***
**
***
* ***b *a
(2.08) (1.80) (1.57) (1.56) 6.57 1.54 1.85 1.62
6.15 1.43 2.00 1.65
(2.11) (1.69) (1.59) (1.59)
7.00 1.65 1.70 1.60
(1.96) (1.91) (1.53) (1.54)
6.05 0.92 1.01 1.37
(2.72) (1.68) (1.63) (1.66)
5.36 0.87 1.38 1.51
(2.70) (1.62) (1.91) (1.79)
6.78 0.98 0.62 1.23
(2.55) (1.73) (1.16) (1.50)
*** *** ***
***
Girls
*** * *** ***
(4.74) (2.08) 4.57 1.74 (6.14) (2.90) 7.23 3.48 (5.66) (2.68) 5.93 2.63 (5.15) (2.06) 7.74 2.79 (5.05) (2.32) 8.75 3.66 (5.13) (2.24) 8.25 3.24
Mean (SD) Mean
Girls
(SD)
*** *** ***
SDQs
Boys
Sex
Teacher
(SD) Mean
Boys
Mean
(SD) Total Boys
(SD)
Main effects were recorded for emotional symptoms. (a) Shows the main effects by gender, while (b) shows the parent-teacher main effects. p < 0.001. ** p < 0.01. * p < 0.05.
Girls 0.390*** 0.441*** 0.218*** 0.273*** 0.236*** 0.322***
***
p < 0.001.
Boys 0.447*** 0.450*** 0.191*** 0.246*** 0.330*** 0.456***
a,b
***
Total 0.428*** 0.476*** 0.245*** 0.262*** 0.302*** 0.396***
Total difficulties score Hyperactivity/ inattention Prosocial behavior Emotional symptom Conduct problems Peer problems
Total difficulties score Hyperactivity/inattention Prosocial behavior Emotional symptom Conduct problems Peer problems
Mean
Table 2 Pearson’s correlations between parent-rated and teacher-rated SDQs (Total N = 1487, Boys: n = 759, Girls: n = 728).
Total
The average of the total difficulties score and each subscale of the parent-rated SDQ and teacher-rated SDQ were calculated by sex (Table 3). A two-way repeated measures analysis of variance was performed. The main effects of evaluator and sex were observed in the emotional symptoms subscale. Compared to the teachers, the parents gave significantly higher ratings to the boys than to the girls (evaluator: F (1, 1485) = 126.62,
Teacher-rated SDQ
3.5. Two-way repeated measures analysis of variance for parent-rated and teacher-rated SDQs
Parent-rated SDQ
The Pearson’s correlation coefficients between the parents’ and teachers’ ratings were calculated for the total sample, boys and girls (Table 2). The correlation coefficients were 0.245–0.476 for the total sample, 0.191–0.456 for the boys and 0.218–0.441 for the girls, showing a moderate correlation. Among all of the groups, the subscale showing the lowest correlation coefficient was prosocial behavior. The subscales showing the highest correlation coefficients were hyperactivity/inattention in the total sample and the girls, and peer problems in the boys.
Table 3 Means and standard deviations for parent-rated and teacher-rated SDQs (Total N = 1487, Boys: n = 759, Girls: n = 728).
3.4. Parent-teacher correlations
Mean
Girls
(SD)
Cronbach’s a coefficient was calculated for each subscale of the parent-rated and teacher-rated SDQs. The alpha level was between 0.521 and 0.768 in the parent version and between 0.600 and 0.859 in the teacher version, with the teacher version showing a somewhat higher reliability. At less than 0.60, the reliability of the conduct problems and peer problems subscales was extremely low in the parent version.
Parent
Simple main effect
3.3. Internal consistency
505
Sex
3801.405, 3856.800, 4719.748, and 5080.935 in the teacher-rated SDQ. In both the parent-rated and teacherrated SDQs, model 1 with the lowest AIC was adopted (parent-rated SDQ: v2 (532) = 2112.018, p = 0.000, AGFI = 0.861, CFI = 0.798, RMSEA = 0.045; teacher-rated SDQ: v2 (532) = 3565.405, p = 0.000, AGFI = 0.772, CFI = 0.782, RMSEA = 0.062).
SDQs
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
506
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
p < 0.001; sex: F (1, 1485) = 5.23, p < 0.05). Moreover, a significant interaction was observed in the total difficulties, conduct problems, hyperactivity/inattention, peer problems, and prosocial behavior scores (total difficulties score: F (1, 1485) = 31.13, p < 0.001; conduct problems: F (1, 1485) = 22.61, p < 0.001; hyperactivity/inattention: F (1, 1485) = 44.58, p < 0.001; peer problems: F (1, 1485) = 6.051, p < 0.05; and prosocial behaviors: F (1, 1485) = 13.54, p < 0.001). Accordingly, the simple main effects were tested using the Bonferroni test, which showed that the average scores of the parents were significantly higher than those of the teachers in all of the subscale scores in both the boys and girls. Moreover, in the parent version, the average scores on the total difficulties, hyperactivity/inattention, and conduct behavior subscales were significantly higher in the boys than the girls. The average scores on the prosocial behavior and emotional symptoms subscales were significantly higher in the girls than the boys. In the teacher version, the average scores of the total difficulties, hyperactivity/inattention, conduct behavior, and peer problems scores were significantly higher in the boys compared to the girls, and the averages on the prosocial behavior subscales were significantly higher in the girls than in the boys. 4. Discussion The aim of this study was to investigate the psychometric properties of the parent and teacher forms of the Japanese version of the SDQ. To meet this objective, the following analyses were conducted: factor structure, factor invariance across sex, internal consistency, correlations between the parent and teacher versions, and differences in averages between sexes and forms of the SDQ. Although the five-factor structure of the Japanese version of the parent-rated and teacher-rated SDQs was supported, it was evident that it was not a robust structure. Moreover, it was observed that difficulties were assessed to be higher in boys than girls, and that the parents gave higher ratings on childrens’ difficulties and strengths than did the teachers. Here, we shall consider the factor structure, internal consistency, the assessments of boys and girls, and the evaluations made by parents and teachers. The results of the confirmatory factor analyses in this study for the boys and girls on the parent-rated and teacher-rated SDQs generally supported the five-factor structure advocated by Goodman [5]. Even though the goodness of fit was slightly better in the parent version compared to the teacher version, it was not judged excellent in either of the versions. As in the previous studies [10,13], item 22 (“steals from home, school or elsewhere”) had an extremely low factor loading on all of the models in the present study, and there was not a high enough factor loading to constitute a factor of conduct problem. Consistent with the results of a previous
research [12], the present study found that peer problems and conduct problems in the parents’ version of the scale had low reliabilities. Of course, subscales with fewer items generally provide lower reliability; however, other subscales with the same number of items had moderate levels of reliability. As seen in Table 1, these two subscales had many items with low factor loadings, and therefore it is presumed that the instability of the factor structure lowered the reliabilities of the subscales. Thus, future research is necessary to examine whether the factor instability of the scale stems from a problem of the equivalence of item representation or from cultural differences in the parental assessments of children. In this study, factor invariance across sex was observed in neither the parent nor the teacher version, suggesting that factor structures were different for boys and girls. In the recent years, the SDQ factor structure has been debated and new factor structures have been proposed for the SDQ [11]. Thus, in order to contribute to the development of a scale structure that accurately evaluates adaptive and maladaptive behaviors among boys and girls, future studies are needed to explore the possibilities of factor structures across sex are required. As in the previous studies [7,8,12], difficulties in the boys were evaluated to be higher in comparison to the girls, while the strengths in the girls were evaluated higher in comparison to the boys. The results of the present study of the parent version were consistent with those of Matsuishi et al. [12], with difficulty in emotional symptoms rated higher in the girls than the boys. On the other hand, sex differences in peer problems were observed in only the teacher version, showing that parents and teachers judged the level of difficulty in boys and girls differently. In the present study, the parents assessed higher levels of difficulty in their children as compared to the teachers, as predicted by Goodman [1]. As suggested by the moderate correlations between the teachers and the parents in the present and previous studies, these results indicate that this could be because parents and teachers observe children in different contexts. Therefore, to be able to use the SDQ to assess the specific characteristics of children, it will be necessary to extract the details of the characteristics’ differences that need to be evaluated by parents and teachers, including sex differences. The present study reports the psychometric properties, such as the factor structure and average values, by gender in the Japanese versions of parent-rated and teacher-rated SDQs. The study also provides vital information about the use of the SDQ, which is projected to be a very efficient instrument for detecting behavioral problems in elementary school children. However, this study has several limitations. First, the applicable age group for the parent version and teacher version of the SDQ is 4–16 years, while the participants of this study were aged 6–12 years. Second, the findings of this study
Y. Shibata et al. / Brain & Development 37 (2015) 501–507
are based on a comprehensive survey of elementary schools of a particular town, and are limited in terms of the generalizability of the results. Additionally, a careful interpretation of the study’s results is necessary because the reliability levels of the peer problems and conduct problems subscales of the parents’ version were low. Furthermore, the parents’ and teachers’ ratings were scheduled two months apart, raising concerns about the possibility that the behavior of the children changed during these two months, thereby influencing the results of the study. It will be necessary to accumulate more reliable information in the future by designing studies that include sophisticated research methods and a broader range of childrens’ ages and larger study samples from more diverse settings.
[5]
[6]
[7]
[8]
[9]
Acknowledgment The present study was conducted as a part of the Therapeutic Educational Support Project for Developmental Disorders, funded by a special education research grant from the Ministry of Education, Culture, Sports, Science, and Technology.
[10]
[11]
[12]
References [1] Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry 1997;38:581–6. [2] Becker A, Woerner W, Hasselhorn M, Banaschewski T, Rothenberger A. Validation of the parent and teacher SDQ in a clinical sample. Eur Child Adolesc Psychiatry 2004;13(Suppl. ii):11–6. [3] Ullebø AK, Posserud MB, Heiervang E, Gillberg C, Obel C. Screening for the attention deficit hyperactivity disorder phenotype using the Strengths and Difficulties Questionnaire. Eur Child Adolesc Psychiatry 2011;20:451–8. [4] Goodman R, Ford T, Simmons H, Gatward R, Meltzer H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for
[13]
[14]
507
child psychiatric disorders in a community sample. Br J Psychiatry 2000;177:534–9. Goodman R. Psychometric properties of the Strengths and Difficulties Questionnaire. J Am Acad Child Adolesc Psychiatry 2001;40:1337–45. Ministry of Education, Culture, Sports, Science and Technology Japan. Research on children attending regular classes having possible developmental disorders needing special educational support 2012 (http://www.mext.go.jp/a_menu/shotou/tokubetu/ material/__icsFiles/afieldfile/2012/12/10/1328729_01.pdf) The date of publication: 2012/12/5; the date of access: 2014/2/10 (in Japanese). van Widenfelt BM, Goedhart AW, Treffers PDA, Goodman R. Dutch version of the Strengths and Difficulties Questionnaire (SDQ). Eur Child Adolesc Psychiatry 2003;12:281–9. Woerner W, Becker A, Rothenberger A. Normative data and scale properties of the German parent SDQ. Eur Child Adolesc Psychiatry 2004;13(Suppl. 2):II3–II10. Mellor D, Stokes M. The factor structure of the Strengths and Difficulties Questionnaire. Eur J Psychol Assess 2007;23:105–12. d’Acremont M, Van der Linden M. Confirmatory factor analysis of the Strengths and Difficulties Questionnaire in a community sample of French-speaking adolescents. Eur J Psychol Assess 2008;24:1–8. Goodman A, Lamping DL, Ploubidis GB. When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children. J Abnorm Child Psychol 2010;38:1179–91. Matsuishi T, Nagano M, Araki Y, Tanaka Y, Iwasaki M, Yamashita Y, et al. Scale properties of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ): a study of infant and school children in community samples. Brain Dev 2008;30:410–5. Nishimura T, Koizumi R. Nursery staff assessment of children using the Japanese version of the Strengths and Difficulties Questionnaire (SDQ) (in Japanese). Bulletin of Fukuoka University of Education. Part 4 Education and Psychology 2010;59:103– 9. Ministry of Health, Labour and Welfare. Strengths and Difficulties Questionnaire. 2007 (http://www.mhlw.go.jp/bunya/kodomo/ boshi-hoken07/h7_04d.html) (in Japanese). The date of publication: 2007/1; the date of access: 2014/2/10.