- Email: [email protected]
Psychological distress and parent reporting on child health: The case of developmental delay
Research in Developmental Disabilities 63 (2017) 11–17
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Research paper
Psychological distress and parent reporting on child health: The case of developmental delay Scott Veldhuizen a,∗ , Chloe Bedard b , Christine Rodriguez a , John Cairney a a INfant and Child Health (INCH) Lab, Department of Family Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada b INfant and Child Health (INCH) Lab, Department of Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada
a r t i c l e
i n f o
Article history: Received 20 September 2016 Received in revised form 14 February 2017 Accepted 15 February 2017 Number of reviews completed is 2 Keywords: Developmental screening ASQ Validation Distress
a b s t r a c t Background: Caregiver-completed screening questionnaires are a common first step in the identification of developmental delay. A caregiver’s mood and anxiety level, however, may affect how he or she perceives and reports possible problems. Aims: In this article, we consider the association between caregiver distress and the accuracy of the Ages and Stages Questionnaire (ASQ), a widely-used screen. Methods and procedures: Our sample includes 857 parent-child dyads drawn from the Psychometric Assessment of the NDDS Study (PANS) and the NDDS Alternate Responses Study (NARS). Parents completed the ASQ and the K6, a brief measure of generalized distress. Children were assessed using the Bayley Scales of Infant and Child Development (BSID). We divided children on BSID result and used logistic regression to examine how distress influenced the ASQ’s accuracy in each group. Results: Of our 857 children, 9% had at least one domain below −2 standard deviations on the BSID, and 17.3% had positive ASQ results. Caregiver distress predicted a positive ASQ substantially and significantly more strongly among BSID-positive children than among others. This translates into slightly reduced ASQ specificity but greatly improved sensitivity among caregivers with higher distress. Conclusions: At low to moderate levels of distress, greater distress is associated with greater ASQ accuracy. © 2017 Elsevier Ltd. All rights reserved.
What this paper adds Mood and anxiety levels have been shown to affect reporting of symptoms in some medical contexts, and there is some evidence that it also influences caregivers’ perceptions of their children’s health. This may affect the performance of screening instruments, which commonly ask caregivers to express concerns or to identify problems. In this study, we find that, at the low and moderate levels of distress that predominate in the general population, more distress is actually associated with greater screen accuracy: More-distressed caregivers seem to slightly over-report child problems, but they are also far more
∗ Corresponding author at: Infant and Child Health Lab, Department of Family Medicine, David Braley Health Sciences Centre, McMaster University, 100 Main Street West, 5th Floor, Hamilton, ON L8P 1H6, Canada. E-mail addresses: [email protected] (S. Veldhuizen), [email protected] (C. Bedard), [email protected] (C. Rodriguez), [email protected] (J. Cairney). http://dx.doi.org/10.1016/j.ridd.2017.02.006 0891-4222/© 2017 Elsevier Ltd. All rights reserved.
12
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
likely to identify genuine issues. This suggests that moderate anxiety and depression are not barriers to accurate reporting, and may also support the incorporation of parent characteristics into predictive models or clinical judgements.
1. Introduction Identifying developmental delay in young children often requires the reporting of caregivers or other third parties (Glascoe & Dworkin, 1995). Delay usually lacks obvious physical signs that can be directly observed by physicians during clinical appointment; and, although clinical standardized instruments for its assessment exist, they are not often used systematically in the course of standard clinical practice (Guttmann, Klein-Geltink, Kopp, & Cairney, 2011; Limbos, Joyce, & Nguyen, 2012). Instead, identification of developmental delay often depends in part on the information primary caregivers communicate to their primary care provider (Raspa et al., 2015). Caregivers may raise concerns, or may disclose them when asked about a child’s development (Glascoe & Dworkin, 1995). There are also a number of brief parent-completed screening questionnaires. Some of these ask caregivers about their concerns (e.g., the Parents’ Evaluation of Developmental Status; PEDS) while others ask them to complete checklists of age-specific developmental milestones (e.g., Ages and Stages Questionnaires; ASQ; Brothers, Glascoe, & Robertshaw, 2008; Squires, Twombly, Bricker, & LaWanda, 2009). Both approaches rely on caregivers’ perception and judgement. For measures that elicit concerns, such as the PEDS, responses will necessarily reflect caregivers’ beliefs about what constitutes normal development and functioning. Measures that ask about particular milestones, meanwhile, require that parents understand what is being asked of them, and accurately judge whether or not the milestone has been reached. In both cases, there may be factors that affect the accuracy of reporting. Some caregiver characteristics, such as education and parenting experience, have been evaluated in previous work and have been found not to be strongly associated with accuracy of reports on developmental progress (Glascoe & Dworkin, 1995; Waters et al., 2000). Another possibility, however, is that reporting may be affected by the caregiver’s level of psychological distress. Distress is a state of psychological discomfort, usually measured in terms of tense, worried, anxious, sad, stressed, and tired mental states. Although distress is highest among people with psychiatric disorders, it occurs in most people at lower levels (Slade, Grove, & Burgess, 2011). Studies in various contexts suggest that mood or anxiety problems can lead to symptom amplification or over-reporting (Barsky, Goodson, Lane, & Cleary, 1988; Henningsen, Zimmermann, & Sattel, 2003; Watson & Pennebaker, 1989). In pediatrics, one particular source of concern is the ‘anxious parent’ (Kroes, Veerman, & De Bruyn, 2003): Anxiety has been shown to affect mothers’ reports of their children’s psychopathology (Briggs-Gowan, Carter, & Schwab-Stone, 1996), physical health (Waters et al., 2000), and quality of life (Davis, Davies, Waters, & Priest, 2008). Little research considered the effect of distress on screen accuracy, and we know of none that has tested possible effects on screens for developmental delay specifically. Dulcan et al. noted, however, that parents with psychiatric disorder were more likely than others to consult pediatricians about their children’s mental health, and that this resulted in a higher detection rate (Dulcan et al., 1990). Glascoe and Dworkin, while noting the possibility of over-identification, have also suggested that parents with “obvious” distress might offer more, not less, accurate clinical information (Glascoe & Dworkin, 1995). A relationship between caregiver distress and screen performance is therefore plausible. Such an association could take several forms. More-distressed caregivers might 1) simply over-report children’s difficulties; 2) be poorer at ruling problems out when they are absent; and/or 3) be more alert to problems when they are present. Variation of this kind might help to explain the moderate agreement between parent-report tools and clinical assessments of child development, and may also provide an area for improvement in development of new versions of screening instruments (Limbos & Joyce, 2011; Sices, Stancin, Kirchner, & Bauchner, 2009; Veldhuizen, Clinton, Rodriguez, Wade, & Cairney, 2015). Evidence on effects of distress on reporting might also help clinicians to better interpret information from caregivers. Errors related to caregiver distress are also problematic, in that already-distressed parents may see a positive screen result as more reason to be anxious; until a final diagnosis can rule out delay, a distressed caregiver must live with unnecessary stress over their child’s development. Scarce resources may also be unnecessarily directed to a costly in-depth clinical developmental assessment of children who will not go on to receive diagnoses (Foster & Wolraich, 1997). In this report, we consider the association between caregiver distress and the accuracy of the ASQ, a widely-used screen for developmental delay. We also consider some mechanisms that might underlie such a relationship. One possibility is that it is mediated by other variables. For example, if higher education were associated with both low distress and high screen accuracy, then distress and screen accuracy would appear to be inversely related, but only because of the effect of education. Although research to date does not suggest that variables such as education or parenting experience strongly affect the accuracy of parent report (Glascoe & Dworkin, 1995; Glascoe, 1994, 2003; Pulsifer, Hoon, Palmer, Gopalan, & Capute, 1994), it seems prudent to evaluate available covariates in order to provide an accurate estimate of the independent effect of distress on screening accuracy. It is also possible that children’s difficulties lead to caregiver distress, in which case distress would be in part simply an indicator of caregivers’ awareness of a child’s delay. For this reason, we aimed to examine the impact of caregiver distress on the validity of a developmental screening test while controlling for independently-measured child functioning.
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
13
Table 1 Sample characteristics by study. PANS (n = 590)
NARS (n = 272)
Total (n = 862)
Child Gender Male Female Child age (months) (SD)
303 (51%) 287 (49%) 27.9 (21.5)
132 (49%) 140 (51%) 27.7 (22.1)
435 (50%) 427 (50%) 27.9 (21.5)
ASQ result Positive Negative
105 (18%) 485 (82%)
44 (16%) 228 (84%)
149 (17%) 713 (83%)
BSID-III result Positive Negative
43 (7%) 547 (93%)
25 (10%) 237 (90%)
68 (8%) 784 (92%)
Perinatal concern No Yes
466 (79%) 124 (21%)
234 (86%) 38 (14%)
700 (81%) 162 (19%)
University degree No Yes
244 (41%) 346 (59%)
92 (34%) 180 (66%)
336 (39%) 526 (61%)
Older siblings 0 1 2 3+
297 (51%) 212 (36%) 52 (9%) 23 (4%)
101 (37%) 107 (39%) 49 (18%) 15 (6%)
398 (46%) 319 (37%) 101 (12%) 38 (4%)
Abbreviations: PANS: Psychometric Assessment of the NDDS Study; NARS: NDDS Alternate Responses Study SD: standard deviation; ASQ: Ages and Stages Questionnaire; BSID-III: 3rd edition of the Bayley Scales of Infant and Child Development.
2. Methods and measures Our sample was drawn from two validation studies conducted between 2010 and 2015: The Psychometric Assessment of the NDDS Study (PANS) and the NDDS Alternate Responses Study (NARS). These projects were conducted in Hamilton, Ontario, Canada using convenience samples of families recruited from community agencies (e.g., Ontario Early Years Centres). Caregivers in both studies were eligible if they could speak and read English and were the child’s primary caregiver and legal guardian, while children were excluded only if they had known health conditions that would have prevented assessment. Caregiver and child characteristics are shown in Table 1. Parents received study materials in the mail, and were requested to complete the ASQ shortly before the appointment at which the BSID-III was administered. Both PANS and NARS received ethical approval from the McMaster University Research Ethics Board, and all parents provided informed, written consent. PANS included 594 children who received the 3rd edition of the Bayley Scales of Infant and Child Development (BSID-III) and NARS 279 (total n = 873). Of these participants, we exclude 16 (2%) because of missing data. Our final pooled sample includes 434 boys and 423 girls (total n = 857) aged 8–42 months (mean = 17.2, SD = 11.8). Ninety-seven percent of caregivers were birth mothers, with the remaining 3% either adoptive mothers or biological fathers. Participant characteristics are given in Table 1. Overall validity results for the PANS study have been previously reported (Veldhuizen et al., 2015). The parent-completed measure used in this study is the third edition of the ASQ (Squires et al., 2009). The ASQ is a set of 21 questionnaires available for age ranges from 1 month to 5.5 years. Each interval of the ASQ assesses child functioning on 5 domains: Communication, gross motor, fine motor, problem solving, and personal-social. Each ASQ domain consists of 6 behaviors or milestones. Parents are asked to observe whether the child exhibits each behaviour (score of 10), does so only “sometimes” (score of 5), or does not do so yet (score of 0). Responses to each item are summed and can range from 0 to 60, with higher scores indicating more milestones reached. Published norms permit the identification of children falling various distances below the expected mean in each area. Following publisher recommendations, we identify a positive screen as any test with at least one domain scoring below the −2 standard deviation (SD) cut-point. As our sample is aged 8–42 months, our data include the 15 ASQ intervals covering this age range. Our independent measure of functioning is the BSID-III (Bayley, 2006). The BSID-III contains 325 items divided into 5 domains (cognitive, receptive communication, expressive communication, fine motor, and gross motor). Items are approximately ordered by difficulty, and a child’s age determines the first item administered in each domain. The test continues until 5 consecutive items are missed. The number of items skipped or passed in each domain comprises the raw score, which is ordinarily converted to a standard score using a set of published tables. The BSID-III was administered by trained research assistants. All assistants held undergraduate or master’s degrees and received a minimum of 8 h of training and 10 h of supervised test administration. It has been argued that BSID-III norms are problematic (Anderson, De Luca, Hutchinson, Roberts, & Doyle, 2010), and we have reported previously on issues with their use in the PANS sample (Veldhuizen, Rodriguez, Wade, & Cairney, 2014). Briefly,
14
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
they identified a much smaller proportion of children than other measures administered at the same time, and the prevalence was strongly related to age. As a result, we do not use these norms in this analysis. Instead, we use a regression-based method (described below) to estimate age-adjusted z-scores. We measure distress with the K6, a six-item measure developed for use in the general population (Kessler et al., 2002). Questions ask how frequently in the previous month the respondent felt “sad”, “nervous”, “restless or fidgety”, “hopeless”, “worthless”, and “that everything was an effort”. Possible responses are “all of the time,” “most of the time,” “some of the time,” “a little of the time,” and “none of the time.” The total score has a range of 24, with higher scores indicating greater distress. Research suggests that a K6 score of approximately 12 or higher is consistent with serious mental illness (Kessler et al., 2010), while a score of 5 has been proposed as the threshold of “moderate” distress (Prochaska, Sung, Max, Shi, & Ong, 2012). The K6 has been widely used as both a distress scale and as a screen for mental disorder, and has demonstrated excellent validity (Furukawa, Kessler, Slade, & Andrews, 2003; Kessler et al., 2010; Prochaska, Sung, Max, Shi, & Ong, 2012). The internal consistency of the K6 was acceptable in our data (alpha = 0.83). Covariates include caregiver age, caregiver education, a count of the child’s older siblings, child age, and an indicator of whether there were any perinatal concerns related to the child’s birth. Perinatal concerns were ascertained from a questionnaire administered to the caregiver and included prematurity (21 days or more) and the receipt of any form of special medical care after birth. Due to the small number of respondents in the less-common outcome category and the difficulty in ordering categories within caregiver education, we dichotomized education at the level of university completion for the logistic regression models. 3. Analysis To identify children with possible developmental delay, we used sample data to derive −2 SD cut-points on each BSID-III domain. We have used and described this approach with these data previously (Veldhuizen et al., 2014). Briefly, we used linear regression to convert BSID-III raw scores to age-specific z-scores. We modeled each of the 5 raw BSID-III domain scores as a function of age, using fractional polynomials to fit appropriate curves. We then fit a second model regressing the absolute residuals of the first model on age. The resulting equations provide a simple way to estimate the mean and variance of the raw scores as functions of age. By subtracting each child’s raw score from the predicted mean and dividing by the estimated standard deviation, we obtained an approximate age-adjusted z-score. More robust methods for generating growth curves exist (Cole, 1990), but this simpler approach can be expected to produce reasonable results as long as the age curve fits well and resulting z-scores are unskewed and normal. We identified as BSID ‘positives’ (i.e., as individuals with possible developmental delay) those children who had a minimum domain z-score below −2. We next explored bivariate associations among screen result, BSID status, and caregiver distress using contingency tables, t-tests, and chi-square tests. To evaluate the effects of distress on screen accuracy, we then divided the sample into children positive and negative on the BSID. This makes it possible to examine effects on sensitivity and specificity separately (an analysis based on simple agreement would give much greater weight to specificity, due to the relatively low prevalence of delay). Within each of these groups, we used logistic regression to estimate the association between caregiver distress and screen result. In these models, we first regressed screen result on K6 score, controlling for overall child functioning (the lowest z-score on any 5 BSID-III domains). We then added our covariates to test whether these associations were independent of parent and child characteristics. For our specificity model, we included all variables simultaneously. For the sensitivity model, however, the number of BSID-positive individuals was too small to do this (Hosmer & Lemeshow, 2000). We therefore entered each variable into the model separately, alongside the minimum BSID-III z-score, to test whether any had a substantial impact on the association with distress. To test whether the effect of distress differed significantly by BSID result, we then fit a full-sample model including the BSID-III result, minimum BSID-III z-score, K6, and an interaction between the K6 and the BSID-III result. Finally, we used the equations from the initial logistic regression models (i.e., those including only K6 and BSID minimum score) to produce predicted probabilities as functions of distress, with the BSID-III minimum score held constant at its mean. For children with positive and negative BSID results, these probabilities correspond to estimates of sensitivity and specificity, respectively. We then combined these numbers with the study prevalence to estimate positive and negative predictive values. We used an alpha level of 0.05 for all statistical tests and used Stata 14 in all analyses (StataCorp, 2015). 4. Results Sample characteristics are shown in Table 1. Seventy-eight (9.1%, 95% CI = 7.3%–11.2%) children in our pooled sample were positive on the BSID, and 149 (17.3%, 95% CI = 14.9%-20.0%) were positive on the ASQ. The overall sensitivity of the ASQ (i.e., the proportion of BSID-III positive results that were also positive on the ASQ) was 41% and its specificity (i.e., the proportion of BSID-III negatives that were also negative on the ASQ) was 85%. These results are very similar to those previously reported for the PANS study (which provided most of the participants included in this analysis; Veldhuizen et al., 2015). The mean K6 score among caregivers was 3.3 (SD = 3.4), with 75% scoring 4 or lower and only 10% scoring 8 or higher. Overall, levels of distress were higher among caregivers whose ASQ results indicated concern (K6 mean = 4.2, SD = 4.1) than among those with negative screens (K6 mean = 3.1, SD = 3.2) (t(855) = 3.71, p < 0.01). The association between distress
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
15
Table 2 Results of logistic models regressing ASQ status on distress and BSID-III result. Negative BSID-III (n = 779)
K6 BSID-III min. z-score Intercept
Positive BSID-III (n = 78)
OR (95% CI)
z
p
OR (95% CI)
z
p
1.05 (1.00−1.11) 0.45 (0.32−0.64) 0.07 (0.05−0.11)
1.93 −4.51 −11.72
0.05 <0.001 <0.001
1.28 (1.08–1.53) 0.52 (0.22–1.21) 0.05 (0.005–0.49)
2.83 −1.52 −2.56
0.01 0.13 0.01
Abbreviations: OR: Odds Ratio; CI: Confidence Interval; BSID-III: 3rd edition of the Bayley Scales of Infant and Child Development. Table 3 Predicted sensitivities, specificities, positive predictive values, and negative predictive values by K6 score (at prevalence of 9.1%). K6 score
Sensitivity
Specificity
PPV
NPV
0 1 2 3 4 5 6 7 8
0.23 0.27 0.32 0.37 0.43 0.49 0.55 0.61 0.66
0.87 0.87 0.86 0.85 0.85 0.84 0.83 0.83 0.82
0.15 0.17 0.19 0.2 0.22 0.24 0.25 0.26 0.27
0.92 0.92 0.93 0.93 0.94 0.94 0.95 0.95 0.96
Abbreviations: PPV: positive prediction value; NPV: negative predictive value.
and BSID-III result was not significant (for caregivers of children with a positive BSID, K6 mean = 3.7, SD = 3.8; for others, K6 mean = 3.2, SD = 3.4) (t(855) = 1.28, p = 0.20). Regressing K6 score on the result of both measures and then testing the equality of the coefficients, however, showed that the independent associations did not differ significantly (F(1854) = 2.38, p = 0.12). The regression-based transformation of BSID-III raw scores into z-scores produced results that were normally distributed and uncorrelated with age. In the logistic regression models including only the K6 and the BSID-III minimum domain zscore, higher K6 scores were associated with ASQ result among children with a positive BSID, independent of BSID-III minimum score (Table 2). Among others, this relationship was relatively weak and was marginally non-significant (OR = 1.05, 95%CI = 1.00–1.11, p = 0.054; model chi-square = 26.50, p < 0.001); among participants with a positive BSID, it was considerably stronger (OR = 1.28, 95% CI = 1.08–1.53, p = 0.005; model chi-square = 16.43, p < 0.001). This difference in effects was significant in a full-sample logistic regression model with an interaction between K6 and BSID-III status (interaction OR = 1.22, 95% CI = 1.01–1.46, p = 0.03). These results also held in simpler analyses. We divided caregivers at the approximate K6 median by creating one group scoring <3 and another scoring 3 or higher and calculated sensitivity and specificity within each group. Among the lowerdistress caregivers, sensitivity was 26% and specificity 86%; in the higher-distress group, these numbers were 53% and 84%, respectively. All caregivers, then, tended to produce more positive screen results when their own distress was higher, but this was more pronounced for parents of children with positive BSID results. Model results are given per unit of the K6; for a 1 SD (3.3 unit) change in distress, ORs were 1.19 for BSID-negative and 2.26 for BSID-positive individuals. The minimum BSID-III domain z-score was highly significant in the model including BSID-negative participants; its effect was similar among BSIDpositive individuals, but was non-significant, in keeping with the smaller sample size. Among the other covariates, child age was significant in the model for BSID-negative individuals, with parents of older children less likely to produce false positive results (OR per year of age = 0.66, 95% CI = 0.52-0.82, p < 0.01). All other variables in this model were non-significant, both individually and jointly. No variables approached significance in the model predicting ASQ result among BSID-positive children. The effect of distress was not substantially affected by the presence of other variables in any models. Using model results to obtain predicted sensitivities and specificities at different levels of distress, independent of the BSID-III minimum z-score, produces the results in Table 3. These are shown for K6 scores up to 8, which is the 90th percentile in our sample. As model results imply, the net effect of distress is to increase overall screen accuracy. Youden’s J (the sum of sensitivity and specificity) increases steadily with distress, as do both positive and negative predictive values (shown for the study prevalence of 9.1%). Results therefore indicate that, at low to moderate levels of distress, self-reported psychological distress is associated with greater accuracy of developmental screening. 5. Discussion Our results imply that, at low to moderate levels, higher self-reported psychological distress is associated with better accuracy on parent-completed screening instruments for developmental delay. As anticipated, parents with higher distress reported more missed milestones, and were more likely to produce false positive results. This was outweighed, however, by their greater ability to correctly identify problems when they were present. The net effect of distress in this sample was therefore to markedly improve sensitivity while only modestly depressing specificity.
16
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
These results imply that parent emotional state affects the accuracy with which they report on their children’s health. This suggests that a general awareness of parental distress may be useful to clinicians when incorporating screen results into clinical decision-making. This variability in screen functioning may also be of interest to instrument developers seeking to make measures more robust. It is important to emphasize that our sample had relatively low levels of distress: Half scored 2 or lower on the K6 and 75% were below 5 (a proposed threshold for ‘moderate’ distress). Results therefore do not reveal how screens might perform if completed by caregivers with severe anxiety or depression, or with other psychiatric conditions. Variation in our sample instead illustrates variation across low and moderate levels of distress. The effect of distress on the false positive rate was small, but in the expected direction. We can speculate that greater anxiety or lower mood might lead parents to amplify any difficulties they observed, or perhaps to decide that behaviors were not performed quite consistently enough or well enough to justify “yes” responses. This is consistent with previouslydescribed work on the effects of distress on symptom reporting for other health-related outcomes such as psychopathology, physical health, and quality of life (Briggs-Gowan et al., 1996; Davis et al., 2008; Waters et al., 2000). However, it should be noted a false positive within the context of developmental screening may point towards other developmental concerns, such as poor language skills, that still warrant investigation and possible intervention (Glascoe, 2001). Despite the suggestion of Glascoe & Gworkin that people with “obvious distress” might provide accurate reports (Glascoe & Dworkin, 1995), the striking improvement in the true positive rate at higher levels of distress was somewhat unexpected. This finding seems unlikely to be due to distress that arises from child difficulties: the association between distress and screen result persisted when we controlled for child functioning, and distress and child functioning were not strongly related to begin with. Only post hoc speculation is possible, but several explanations seem worth mentioning. First, distress may produce greater alertness to possible problems through heightened vigilance. Second, parents who experience no distress may perhaps have a positive view of their child that is not easily altered. Third, psychological research has shown that some degree of stress is necessary for performance in general (Cohen, 2011); it is possible that a general lack of arousal among low-distress caregivers may have led to, for example, poor concentration when completing the ASQ. Fourth, it is possible that K6 scores and ASQ scores could sometimes be related through a shared process of ‘defensive denial’ (Shedler, Mayman, & Manis, 1993). Finally, a simple reluctance to make the effort necessary to complete study instruments accurately might lead to low scores on both the ASQ and the K6, which would be reflected in poor sensitivity for people with low distress. This study has limitations that restrict broad generalizations of the results. First, our sample was drawn from a small geographical area, and the moderate number of positive BSID results restricts the power of our regression analyses. Second, the vast majority of the participating caregivers were mothers. We cannot conclude that the associations we report would be true for fathers, or other caregivers. Third, as already mentioned, the overall level of distress in our sample was low to moderate, and therefore does not represent the full spectrum of caregivers seen in clinical practice. It does, however, include levels of distress that are most common in the general population. Fourth, our sample included children aged 8–42 months and 15 age-specific forms of the screening instrument; results may vary with child age, with relevant BSID-III items, and with ASQ age interval. Finally, all efforts to validate screens for developmental delay among young children are complicated by the absence of a reference measure of known accuracy. Our results describe the effect of distress on the agreement of the ASQ with the BSID-III, but its role on the validity of the ASQ with respect to true developmental status is unknown. That we found variation related to distress is of some interest in part because the ASQ does not invite expressions of concern; rather it asks about specific behaviours and milestones, and it does so in a structured way. Our results show that distress affects reporting even of this kind, independent of actual child functioning. It would therefore be interesting to consider the same question with an instrument (e.g., the Parent Evaluation of Developmental Status) that instead elicits general concerns about functioning, and to investigate the effect of higher levels of distress. Conflicts of interest None. Funding support The studies from which data are taken were supported by a grant from the Ministry of Children and Youth Services of Ontario (037-370203-A518-A16061-577010); The MCYS did not have any involvement in the design, analysis, or interpretation of the current data. References Anderson, P. J., De Luca, C. R., Hutchinson, E., Roberts, G., & Doyle, L. W. (2010). Underestimation of developmental delay by the new Bayley-III Scale. Archives of Pediatrics & Adolescent Medicine, 164(4), 352–356. Barsky, A. J., Goodson, J. D., Lane, R. S., & Cleary, P. D. (1988). The amplification of somatic symptoms. Psychosomatic Medicine, 50(5), 510–519. Bayley, N. (2006). Bayley scales of infant and toddler development. Briggs-Gowan, M. J., Carter, A. S., & Schwab-Stone, M. (1996). Discrepancies among mother, child, and teacher reports: examining the contributions of maternal depression and anxiety. Journal of Abnormal Child Psychology, 24(6), 749–765. http://dx.doi.org/10.1007/bf01664738
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
17
Brothers, K. B., Glascoe, F. P., & Robertshaw, N. S. (2008). PEDS: developmental milestones—an accurate brief tool for surveillance and screening. Clinical Pediatrics, 47(3), 271–279. Cohen, R. A. (2011). Yerkes–dodson law. In Encyclopedia of clinical neuropsychology. pp. 2737–2738. Springer. Cole, T. J. (1990). The LMS method for constructing normalized growth standards. European Journal of Clinical Nutrition, 44(1), 45–60. Davis, E., Davies, B., Waters, E., & Priest, N. (2008). The relationship between proxy reported health-related quality of life and parental distress: gender differences. Child: Care, Health and Development, 34(6), 830–837. http://dx.doi.org/10.1111/j.1365-2214.2008.00866.x Dulcan, M. K., Costello, E. J., ANTHON’Y, J. C., Edelbrock, C., Brent, D., & Janiszewski, S. (1990). The pediatrician as gatekeeper to mental health care for children: do parents’ concerns open the gate? Journal of the American Academy of Child & Adolescent Psychiatry, 29(3), 453–458. Foster, F. M., & Wolraich, M. L. (1997). An economic evaluation of four methods for detecting developmental problems. Pediatrics, 99, 830–837. Furukawa, T. A., Kessler, R. C., Slade, T., & Andrews, G. (2003). The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychological Medicine, 33(02), 357–362. Glascoe, & Dworkin, P. H. (1995). The role of parents in the detection of developmental and behavioral problems. Pediatrics, 95(6), 829–836. Glascoe, F. (1994). It’s not what it seems. Clinical Pediatrics, 33(5), 292–296. Glascoe. (2001). Are overreferrals on developmental screening tests really a problem? Archives of Pediatrics & Adolescent Medicine, 155(1), 54–59. Glascoe, F. (2003). Parents’ evaluation of developmental status: how well do parents’ concerns identify children with behavioral and emotional problems? Clinical Pediatrics, 42(2), 133–138. Guttmann, A., Klein-Geltink, J., Kopp, A., & Cairney, J. (2011). Uptake of the new fee code for Ontario’s enhanced 18-Month well baby visit: a preliminary evaluation. Toronto: Institute for Clinical Evaluative Sciences. Henningsen, P., Zimmermann, T., & Sattel, H. (2003). Medically unexplained physical symptoms, anxiety, and depression: a meta-Analytic review. Psychosomatic Medicine, 65(4), 528–533. Hosmer, D., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley. Kessler, R. C., Andrews, G., Colpe, L. J., Hiripi, E., Mroczek, D. K., Normand, S.-L., Walters, E. E., & Zaslavsky, A. M. (2002). Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine, 32(06), 959–976. Kessler, R. C., Green, J. G., Gruber, M. J., Sampson, N. A., Bromet, E., Cuitan, M., Furukawa, T. A., Gureje, O., Hinkov, H., Hu, C. Y., Lara, C., Lee, S., Mneimneh, Z., Myer, L., Oakley-Browne, M., Posada-Villa, J., Sagar, R., Viana, M. C., & Zaslavsky, A. M. (2010). Screening for serious mental illness in the general population with the K6 screening scale: results from the WHO World Mental Health (WMH) survey initiative. International Journal of Methods in Psychiatric Research, 4–22. http://dx.doi.org/10.1002/mpr.310 Kroes, G., Veerman, J. W., & De Bruyn, E. E. (2003). Bias in parental reports? Maternal psychopathology and the reporting of problem behavior in clinic-referred children. European Journal of Psychological Assessment, 19(3), 195. Limbos, M. M., & Joyce, D. P. (2011). Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. Journal of Developmental & Behavioral Pediatrics, 32(7), 499–511. Limbos, M. M., Joyce, D. P., & Nguyen, T. K. (2012). Developmental screening practices in Canada: a survey of primary care providers. Journal of Intellectual Disability Research, 56, 765 [Hoboken, New Jersey: Wiley-Blackwell]. Prochaska, J. J., Sung, H.-Y., Max, W., Shi, Y., & Ong, M. (2012). Validity study of the K6 scale as a measure of moderate mental distress based on mental health treatment need and utilization. International Journal of Methods in Psychiatric Research, 21(2), 88–97. Pulsifer, M. B., Hoon, A. H., Palmer, F. B., Gopalan, R., & Capute, A. J. (1994). Maternal estimates of developmental age in preschool children. The Journal of Pediatrics, 125(1), S18–S24. Raspa, M., Levis, D. M., Kish-Doto, J., Wallace, I., Rice, C., Barger, B., . . . & Wolf, R. B. (2015). Examining parents’ experiences and information needs regarding early identification of developmental delays: qualitative research to inform a public health campaign. Journal of Developmental & Behavioral Pediatrics, 36(8), 575–585. Shedler, J., Mayman, M., & Manis, M. (1993). The illusion of mental health. American Psychologist, 48(11), 1117. Sices, L., Stancin, T., Kirchner, H. L., & Bauchner, H. (2009). PEDS and ASQ developmental screening tests may not identify the same children. Pediatrics, 124(4), e640–e647. http://dx.doi.org/10.1542/peds.2008-2628 Slade, T., Grove, R., & Burgess, P. (2011). Kessler psychological distress scale: normative data from the 2007 Australian National Survey of Mental Health and Wellbeing. Australian and New Zealand Journal of Psychiatry, 45(4), 308–316. Squires, J., Twombly, E., Bricker, L., & Potter, P. (2009). ASQ-3 Ages and Stages Questionnaires user’s guide. Lane County, OR: Brookes Publishing. StataCorp. (2015). Stata statistical software: release 14. College Station, TX: StataCorp LP. Veldhuizen, S., Rodriguez, C., Wade, T. J., & Cairney, J. (2014). Misclassification due to age grouping in measures of child development. Archives of Disease in Childhood, 100(3), 220–224. http://dx.doi.org/10.1136/archdischild-2014-306548 Veldhuizen, S., Clinton, J., Rodriguez, C., Wade, T. J., & Cairney, J. (2015). Concurrent validity of the ages and stages questionnaires and bayley developmental scales in a general population sample. Academic Pediatrics, 15(2), 231–237. http://dx.doi.org/10.1016/j.acap.2014.08.002 Waters, E., Doyle, J., Wolfe, R., Wright, M., Wake, M., & Salmon, L. (2000). Influence of parental gender and self-reported health and illness on parent-reported child health. Pediatrics, 106(6), 1422–1428. Watson, D., & Pennebaker, J. W. (1989). Health complaints, stress, and distress: exploring the central role of negative affectivity. Psychological Review, 96(2), 234.
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Research paper
Psychological distress and parent reporting on child health: The case of developmental delay Scott Veldhuizen a,∗ , Chloe Bedard b , Christine Rodriguez a , John Cairney a a INfant and Child Health (INCH) Lab, Department of Family Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada b INfant and Child Health (INCH) Lab, Department of Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada
a r t i c l e
i n f o
Article history: Received 20 September 2016 Received in revised form 14 February 2017 Accepted 15 February 2017 Number of reviews completed is 2 Keywords: Developmental screening ASQ Validation Distress
a b s t r a c t Background: Caregiver-completed screening questionnaires are a common first step in the identification of developmental delay. A caregiver’s mood and anxiety level, however, may affect how he or she perceives and reports possible problems. Aims: In this article, we consider the association between caregiver distress and the accuracy of the Ages and Stages Questionnaire (ASQ), a widely-used screen. Methods and procedures: Our sample includes 857 parent-child dyads drawn from the Psychometric Assessment of the NDDS Study (PANS) and the NDDS Alternate Responses Study (NARS). Parents completed the ASQ and the K6, a brief measure of generalized distress. Children were assessed using the Bayley Scales of Infant and Child Development (BSID). We divided children on BSID result and used logistic regression to examine how distress influenced the ASQ’s accuracy in each group. Results: Of our 857 children, 9% had at least one domain below −2 standard deviations on the BSID, and 17.3% had positive ASQ results. Caregiver distress predicted a positive ASQ substantially and significantly more strongly among BSID-positive children than among others. This translates into slightly reduced ASQ specificity but greatly improved sensitivity among caregivers with higher distress. Conclusions: At low to moderate levels of distress, greater distress is associated with greater ASQ accuracy. © 2017 Elsevier Ltd. All rights reserved.
What this paper adds Mood and anxiety levels have been shown to affect reporting of symptoms in some medical contexts, and there is some evidence that it also influences caregivers’ perceptions of their children’s health. This may affect the performance of screening instruments, which commonly ask caregivers to express concerns or to identify problems. In this study, we find that, at the low and moderate levels of distress that predominate in the general population, more distress is actually associated with greater screen accuracy: More-distressed caregivers seem to slightly over-report child problems, but they are also far more
∗ Corresponding author at: Infant and Child Health Lab, Department of Family Medicine, David Braley Health Sciences Centre, McMaster University, 100 Main Street West, 5th Floor, Hamilton, ON L8P 1H6, Canada. E-mail addresses: [email protected] (S. Veldhuizen), [email protected] (C. Bedard), [email protected] (C. Rodriguez), [email protected] (J. Cairney). http://dx.doi.org/10.1016/j.ridd.2017.02.006 0891-4222/© 2017 Elsevier Ltd. All rights reserved.
12
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
likely to identify genuine issues. This suggests that moderate anxiety and depression are not barriers to accurate reporting, and may also support the incorporation of parent characteristics into predictive models or clinical judgements.
1. Introduction Identifying developmental delay in young children often requires the reporting of caregivers or other third parties (Glascoe & Dworkin, 1995). Delay usually lacks obvious physical signs that can be directly observed by physicians during clinical appointment; and, although clinical standardized instruments for its assessment exist, they are not often used systematically in the course of standard clinical practice (Guttmann, Klein-Geltink, Kopp, & Cairney, 2011; Limbos, Joyce, & Nguyen, 2012). Instead, identification of developmental delay often depends in part on the information primary caregivers communicate to their primary care provider (Raspa et al., 2015). Caregivers may raise concerns, or may disclose them when asked about a child’s development (Glascoe & Dworkin, 1995). There are also a number of brief parent-completed screening questionnaires. Some of these ask caregivers about their concerns (e.g., the Parents’ Evaluation of Developmental Status; PEDS) while others ask them to complete checklists of age-specific developmental milestones (e.g., Ages and Stages Questionnaires; ASQ; Brothers, Glascoe, & Robertshaw, 2008; Squires, Twombly, Bricker, & LaWanda, 2009). Both approaches rely on caregivers’ perception and judgement. For measures that elicit concerns, such as the PEDS, responses will necessarily reflect caregivers’ beliefs about what constitutes normal development and functioning. Measures that ask about particular milestones, meanwhile, require that parents understand what is being asked of them, and accurately judge whether or not the milestone has been reached. In both cases, there may be factors that affect the accuracy of reporting. Some caregiver characteristics, such as education and parenting experience, have been evaluated in previous work and have been found not to be strongly associated with accuracy of reports on developmental progress (Glascoe & Dworkin, 1995; Waters et al., 2000). Another possibility, however, is that reporting may be affected by the caregiver’s level of psychological distress. Distress is a state of psychological discomfort, usually measured in terms of tense, worried, anxious, sad, stressed, and tired mental states. Although distress is highest among people with psychiatric disorders, it occurs in most people at lower levels (Slade, Grove, & Burgess, 2011). Studies in various contexts suggest that mood or anxiety problems can lead to symptom amplification or over-reporting (Barsky, Goodson, Lane, & Cleary, 1988; Henningsen, Zimmermann, & Sattel, 2003; Watson & Pennebaker, 1989). In pediatrics, one particular source of concern is the ‘anxious parent’ (Kroes, Veerman, & De Bruyn, 2003): Anxiety has been shown to affect mothers’ reports of their children’s psychopathology (Briggs-Gowan, Carter, & Schwab-Stone, 1996), physical health (Waters et al., 2000), and quality of life (Davis, Davies, Waters, & Priest, 2008). Little research considered the effect of distress on screen accuracy, and we know of none that has tested possible effects on screens for developmental delay specifically. Dulcan et al. noted, however, that parents with psychiatric disorder were more likely than others to consult pediatricians about their children’s mental health, and that this resulted in a higher detection rate (Dulcan et al., 1990). Glascoe and Dworkin, while noting the possibility of over-identification, have also suggested that parents with “obvious” distress might offer more, not less, accurate clinical information (Glascoe & Dworkin, 1995). A relationship between caregiver distress and screen performance is therefore plausible. Such an association could take several forms. More-distressed caregivers might 1) simply over-report children’s difficulties; 2) be poorer at ruling problems out when they are absent; and/or 3) be more alert to problems when they are present. Variation of this kind might help to explain the moderate agreement between parent-report tools and clinical assessments of child development, and may also provide an area for improvement in development of new versions of screening instruments (Limbos & Joyce, 2011; Sices, Stancin, Kirchner, & Bauchner, 2009; Veldhuizen, Clinton, Rodriguez, Wade, & Cairney, 2015). Evidence on effects of distress on reporting might also help clinicians to better interpret information from caregivers. Errors related to caregiver distress are also problematic, in that already-distressed parents may see a positive screen result as more reason to be anxious; until a final diagnosis can rule out delay, a distressed caregiver must live with unnecessary stress over their child’s development. Scarce resources may also be unnecessarily directed to a costly in-depth clinical developmental assessment of children who will not go on to receive diagnoses (Foster & Wolraich, 1997). In this report, we consider the association between caregiver distress and the accuracy of the ASQ, a widely-used screen for developmental delay. We also consider some mechanisms that might underlie such a relationship. One possibility is that it is mediated by other variables. For example, if higher education were associated with both low distress and high screen accuracy, then distress and screen accuracy would appear to be inversely related, but only because of the effect of education. Although research to date does not suggest that variables such as education or parenting experience strongly affect the accuracy of parent report (Glascoe & Dworkin, 1995; Glascoe, 1994, 2003; Pulsifer, Hoon, Palmer, Gopalan, & Capute, 1994), it seems prudent to evaluate available covariates in order to provide an accurate estimate of the independent effect of distress on screening accuracy. It is also possible that children’s difficulties lead to caregiver distress, in which case distress would be in part simply an indicator of caregivers’ awareness of a child’s delay. For this reason, we aimed to examine the impact of caregiver distress on the validity of a developmental screening test while controlling for independently-measured child functioning.
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
13
Table 1 Sample characteristics by study. PANS (n = 590)
NARS (n = 272)
Total (n = 862)
Child Gender Male Female Child age (months) (SD)
303 (51%) 287 (49%) 27.9 (21.5)
132 (49%) 140 (51%) 27.7 (22.1)
435 (50%) 427 (50%) 27.9 (21.5)
ASQ result Positive Negative
105 (18%) 485 (82%)
44 (16%) 228 (84%)
149 (17%) 713 (83%)
BSID-III result Positive Negative
43 (7%) 547 (93%)
25 (10%) 237 (90%)
68 (8%) 784 (92%)
Perinatal concern No Yes
466 (79%) 124 (21%)
234 (86%) 38 (14%)
700 (81%) 162 (19%)
University degree No Yes
244 (41%) 346 (59%)
92 (34%) 180 (66%)
336 (39%) 526 (61%)
Older siblings 0 1 2 3+
297 (51%) 212 (36%) 52 (9%) 23 (4%)
101 (37%) 107 (39%) 49 (18%) 15 (6%)
398 (46%) 319 (37%) 101 (12%) 38 (4%)
Abbreviations: PANS: Psychometric Assessment of the NDDS Study; NARS: NDDS Alternate Responses Study SD: standard deviation; ASQ: Ages and Stages Questionnaire; BSID-III: 3rd edition of the Bayley Scales of Infant and Child Development.
2. Methods and measures Our sample was drawn from two validation studies conducted between 2010 and 2015: The Psychometric Assessment of the NDDS Study (PANS) and the NDDS Alternate Responses Study (NARS). These projects were conducted in Hamilton, Ontario, Canada using convenience samples of families recruited from community agencies (e.g., Ontario Early Years Centres). Caregivers in both studies were eligible if they could speak and read English and were the child’s primary caregiver and legal guardian, while children were excluded only if they had known health conditions that would have prevented assessment. Caregiver and child characteristics are shown in Table 1. Parents received study materials in the mail, and were requested to complete the ASQ shortly before the appointment at which the BSID-III was administered. Both PANS and NARS received ethical approval from the McMaster University Research Ethics Board, and all parents provided informed, written consent. PANS included 594 children who received the 3rd edition of the Bayley Scales of Infant and Child Development (BSID-III) and NARS 279 (total n = 873). Of these participants, we exclude 16 (2%) because of missing data. Our final pooled sample includes 434 boys and 423 girls (total n = 857) aged 8–42 months (mean = 17.2, SD = 11.8). Ninety-seven percent of caregivers were birth mothers, with the remaining 3% either adoptive mothers or biological fathers. Participant characteristics are given in Table 1. Overall validity results for the PANS study have been previously reported (Veldhuizen et al., 2015). The parent-completed measure used in this study is the third edition of the ASQ (Squires et al., 2009). The ASQ is a set of 21 questionnaires available for age ranges from 1 month to 5.5 years. Each interval of the ASQ assesses child functioning on 5 domains: Communication, gross motor, fine motor, problem solving, and personal-social. Each ASQ domain consists of 6 behaviors or milestones. Parents are asked to observe whether the child exhibits each behaviour (score of 10), does so only “sometimes” (score of 5), or does not do so yet (score of 0). Responses to each item are summed and can range from 0 to 60, with higher scores indicating more milestones reached. Published norms permit the identification of children falling various distances below the expected mean in each area. Following publisher recommendations, we identify a positive screen as any test with at least one domain scoring below the −2 standard deviation (SD) cut-point. As our sample is aged 8–42 months, our data include the 15 ASQ intervals covering this age range. Our independent measure of functioning is the BSID-III (Bayley, 2006). The BSID-III contains 325 items divided into 5 domains (cognitive, receptive communication, expressive communication, fine motor, and gross motor). Items are approximately ordered by difficulty, and a child’s age determines the first item administered in each domain. The test continues until 5 consecutive items are missed. The number of items skipped or passed in each domain comprises the raw score, which is ordinarily converted to a standard score using a set of published tables. The BSID-III was administered by trained research assistants. All assistants held undergraduate or master’s degrees and received a minimum of 8 h of training and 10 h of supervised test administration. It has been argued that BSID-III norms are problematic (Anderson, De Luca, Hutchinson, Roberts, & Doyle, 2010), and we have reported previously on issues with their use in the PANS sample (Veldhuizen, Rodriguez, Wade, & Cairney, 2014). Briefly,
14
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
they identified a much smaller proportion of children than other measures administered at the same time, and the prevalence was strongly related to age. As a result, we do not use these norms in this analysis. Instead, we use a regression-based method (described below) to estimate age-adjusted z-scores. We measure distress with the K6, a six-item measure developed for use in the general population (Kessler et al., 2002). Questions ask how frequently in the previous month the respondent felt “sad”, “nervous”, “restless or fidgety”, “hopeless”, “worthless”, and “that everything was an effort”. Possible responses are “all of the time,” “most of the time,” “some of the time,” “a little of the time,” and “none of the time.” The total score has a range of 24, with higher scores indicating greater distress. Research suggests that a K6 score of approximately 12 or higher is consistent with serious mental illness (Kessler et al., 2010), while a score of 5 has been proposed as the threshold of “moderate” distress (Prochaska, Sung, Max, Shi, & Ong, 2012). The K6 has been widely used as both a distress scale and as a screen for mental disorder, and has demonstrated excellent validity (Furukawa, Kessler, Slade, & Andrews, 2003; Kessler et al., 2010; Prochaska, Sung, Max, Shi, & Ong, 2012). The internal consistency of the K6 was acceptable in our data (alpha = 0.83). Covariates include caregiver age, caregiver education, a count of the child’s older siblings, child age, and an indicator of whether there were any perinatal concerns related to the child’s birth. Perinatal concerns were ascertained from a questionnaire administered to the caregiver and included prematurity (21 days or more) and the receipt of any form of special medical care after birth. Due to the small number of respondents in the less-common outcome category and the difficulty in ordering categories within caregiver education, we dichotomized education at the level of university completion for the logistic regression models. 3. Analysis To identify children with possible developmental delay, we used sample data to derive −2 SD cut-points on each BSID-III domain. We have used and described this approach with these data previously (Veldhuizen et al., 2014). Briefly, we used linear regression to convert BSID-III raw scores to age-specific z-scores. We modeled each of the 5 raw BSID-III domain scores as a function of age, using fractional polynomials to fit appropriate curves. We then fit a second model regressing the absolute residuals of the first model on age. The resulting equations provide a simple way to estimate the mean and variance of the raw scores as functions of age. By subtracting each child’s raw score from the predicted mean and dividing by the estimated standard deviation, we obtained an approximate age-adjusted z-score. More robust methods for generating growth curves exist (Cole, 1990), but this simpler approach can be expected to produce reasonable results as long as the age curve fits well and resulting z-scores are unskewed and normal. We identified as BSID ‘positives’ (i.e., as individuals with possible developmental delay) those children who had a minimum domain z-score below −2. We next explored bivariate associations among screen result, BSID status, and caregiver distress using contingency tables, t-tests, and chi-square tests. To evaluate the effects of distress on screen accuracy, we then divided the sample into children positive and negative on the BSID. This makes it possible to examine effects on sensitivity and specificity separately (an analysis based on simple agreement would give much greater weight to specificity, due to the relatively low prevalence of delay). Within each of these groups, we used logistic regression to estimate the association between caregiver distress and screen result. In these models, we first regressed screen result on K6 score, controlling for overall child functioning (the lowest z-score on any 5 BSID-III domains). We then added our covariates to test whether these associations were independent of parent and child characteristics. For our specificity model, we included all variables simultaneously. For the sensitivity model, however, the number of BSID-positive individuals was too small to do this (Hosmer & Lemeshow, 2000). We therefore entered each variable into the model separately, alongside the minimum BSID-III z-score, to test whether any had a substantial impact on the association with distress. To test whether the effect of distress differed significantly by BSID result, we then fit a full-sample model including the BSID-III result, minimum BSID-III z-score, K6, and an interaction between the K6 and the BSID-III result. Finally, we used the equations from the initial logistic regression models (i.e., those including only K6 and BSID minimum score) to produce predicted probabilities as functions of distress, with the BSID-III minimum score held constant at its mean. For children with positive and negative BSID results, these probabilities correspond to estimates of sensitivity and specificity, respectively. We then combined these numbers with the study prevalence to estimate positive and negative predictive values. We used an alpha level of 0.05 for all statistical tests and used Stata 14 in all analyses (StataCorp, 2015). 4. Results Sample characteristics are shown in Table 1. Seventy-eight (9.1%, 95% CI = 7.3%–11.2%) children in our pooled sample were positive on the BSID, and 149 (17.3%, 95% CI = 14.9%-20.0%) were positive on the ASQ. The overall sensitivity of the ASQ (i.e., the proportion of BSID-III positive results that were also positive on the ASQ) was 41% and its specificity (i.e., the proportion of BSID-III negatives that were also negative on the ASQ) was 85%. These results are very similar to those previously reported for the PANS study (which provided most of the participants included in this analysis; Veldhuizen et al., 2015). The mean K6 score among caregivers was 3.3 (SD = 3.4), with 75% scoring 4 or lower and only 10% scoring 8 or higher. Overall, levels of distress were higher among caregivers whose ASQ results indicated concern (K6 mean = 4.2, SD = 4.1) than among those with negative screens (K6 mean = 3.1, SD = 3.2) (t(855) = 3.71, p < 0.01). The association between distress
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
15
Table 2 Results of logistic models regressing ASQ status on distress and BSID-III result. Negative BSID-III (n = 779)
K6 BSID-III min. z-score Intercept
Positive BSID-III (n = 78)
OR (95% CI)
z
p
OR (95% CI)
z
p
1.05 (1.00−1.11) 0.45 (0.32−0.64) 0.07 (0.05−0.11)
1.93 −4.51 −11.72
0.05 <0.001 <0.001
1.28 (1.08–1.53) 0.52 (0.22–1.21) 0.05 (0.005–0.49)
2.83 −1.52 −2.56
0.01 0.13 0.01
Abbreviations: OR: Odds Ratio; CI: Confidence Interval; BSID-III: 3rd edition of the Bayley Scales of Infant and Child Development. Table 3 Predicted sensitivities, specificities, positive predictive values, and negative predictive values by K6 score (at prevalence of 9.1%). K6 score
Sensitivity
Specificity
PPV
NPV
0 1 2 3 4 5 6 7 8
0.23 0.27 0.32 0.37 0.43 0.49 0.55 0.61 0.66
0.87 0.87 0.86 0.85 0.85 0.84 0.83 0.83 0.82
0.15 0.17 0.19 0.2 0.22 0.24 0.25 0.26 0.27
0.92 0.92 0.93 0.93 0.94 0.94 0.95 0.95 0.96
Abbreviations: PPV: positive prediction value; NPV: negative predictive value.
and BSID-III result was not significant (for caregivers of children with a positive BSID, K6 mean = 3.7, SD = 3.8; for others, K6 mean = 3.2, SD = 3.4) (t(855) = 1.28, p = 0.20). Regressing K6 score on the result of both measures and then testing the equality of the coefficients, however, showed that the independent associations did not differ significantly (F(1854) = 2.38, p = 0.12). The regression-based transformation of BSID-III raw scores into z-scores produced results that were normally distributed and uncorrelated with age. In the logistic regression models including only the K6 and the BSID-III minimum domain zscore, higher K6 scores were associated with ASQ result among children with a positive BSID, independent of BSID-III minimum score (Table 2). Among others, this relationship was relatively weak and was marginally non-significant (OR = 1.05, 95%CI = 1.00–1.11, p = 0.054; model chi-square = 26.50, p < 0.001); among participants with a positive BSID, it was considerably stronger (OR = 1.28, 95% CI = 1.08–1.53, p = 0.005; model chi-square = 16.43, p < 0.001). This difference in effects was significant in a full-sample logistic regression model with an interaction between K6 and BSID-III status (interaction OR = 1.22, 95% CI = 1.01–1.46, p = 0.03). These results also held in simpler analyses. We divided caregivers at the approximate K6 median by creating one group scoring <3 and another scoring 3 or higher and calculated sensitivity and specificity within each group. Among the lowerdistress caregivers, sensitivity was 26% and specificity 86%; in the higher-distress group, these numbers were 53% and 84%, respectively. All caregivers, then, tended to produce more positive screen results when their own distress was higher, but this was more pronounced for parents of children with positive BSID results. Model results are given per unit of the K6; for a 1 SD (3.3 unit) change in distress, ORs were 1.19 for BSID-negative and 2.26 for BSID-positive individuals. The minimum BSID-III domain z-score was highly significant in the model including BSID-negative participants; its effect was similar among BSIDpositive individuals, but was non-significant, in keeping with the smaller sample size. Among the other covariates, child age was significant in the model for BSID-negative individuals, with parents of older children less likely to produce false positive results (OR per year of age = 0.66, 95% CI = 0.52-0.82, p < 0.01). All other variables in this model were non-significant, both individually and jointly. No variables approached significance in the model predicting ASQ result among BSID-positive children. The effect of distress was not substantially affected by the presence of other variables in any models. Using model results to obtain predicted sensitivities and specificities at different levels of distress, independent of the BSID-III minimum z-score, produces the results in Table 3. These are shown for K6 scores up to 8, which is the 90th percentile in our sample. As model results imply, the net effect of distress is to increase overall screen accuracy. Youden’s J (the sum of sensitivity and specificity) increases steadily with distress, as do both positive and negative predictive values (shown for the study prevalence of 9.1%). Results therefore indicate that, at low to moderate levels of distress, self-reported psychological distress is associated with greater accuracy of developmental screening. 5. Discussion Our results imply that, at low to moderate levels, higher self-reported psychological distress is associated with better accuracy on parent-completed screening instruments for developmental delay. As anticipated, parents with higher distress reported more missed milestones, and were more likely to produce false positive results. This was outweighed, however, by their greater ability to correctly identify problems when they were present. The net effect of distress in this sample was therefore to markedly improve sensitivity while only modestly depressing specificity.
16
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
These results imply that parent emotional state affects the accuracy with which they report on their children’s health. This suggests that a general awareness of parental distress may be useful to clinicians when incorporating screen results into clinical decision-making. This variability in screen functioning may also be of interest to instrument developers seeking to make measures more robust. It is important to emphasize that our sample had relatively low levels of distress: Half scored 2 or lower on the K6 and 75% were below 5 (a proposed threshold for ‘moderate’ distress). Results therefore do not reveal how screens might perform if completed by caregivers with severe anxiety or depression, or with other psychiatric conditions. Variation in our sample instead illustrates variation across low and moderate levels of distress. The effect of distress on the false positive rate was small, but in the expected direction. We can speculate that greater anxiety or lower mood might lead parents to amplify any difficulties they observed, or perhaps to decide that behaviors were not performed quite consistently enough or well enough to justify “yes” responses. This is consistent with previouslydescribed work on the effects of distress on symptom reporting for other health-related outcomes such as psychopathology, physical health, and quality of life (Briggs-Gowan et al., 1996; Davis et al., 2008; Waters et al., 2000). However, it should be noted a false positive within the context of developmental screening may point towards other developmental concerns, such as poor language skills, that still warrant investigation and possible intervention (Glascoe, 2001). Despite the suggestion of Glascoe & Gworkin that people with “obvious distress” might provide accurate reports (Glascoe & Dworkin, 1995), the striking improvement in the true positive rate at higher levels of distress was somewhat unexpected. This finding seems unlikely to be due to distress that arises from child difficulties: the association between distress and screen result persisted when we controlled for child functioning, and distress and child functioning were not strongly related to begin with. Only post hoc speculation is possible, but several explanations seem worth mentioning. First, distress may produce greater alertness to possible problems through heightened vigilance. Second, parents who experience no distress may perhaps have a positive view of their child that is not easily altered. Third, psychological research has shown that some degree of stress is necessary for performance in general (Cohen, 2011); it is possible that a general lack of arousal among low-distress caregivers may have led to, for example, poor concentration when completing the ASQ. Fourth, it is possible that K6 scores and ASQ scores could sometimes be related through a shared process of ‘defensive denial’ (Shedler, Mayman, & Manis, 1993). Finally, a simple reluctance to make the effort necessary to complete study instruments accurately might lead to low scores on both the ASQ and the K6, which would be reflected in poor sensitivity for people with low distress. This study has limitations that restrict broad generalizations of the results. First, our sample was drawn from a small geographical area, and the moderate number of positive BSID results restricts the power of our regression analyses. Second, the vast majority of the participating caregivers were mothers. We cannot conclude that the associations we report would be true for fathers, or other caregivers. Third, as already mentioned, the overall level of distress in our sample was low to moderate, and therefore does not represent the full spectrum of caregivers seen in clinical practice. It does, however, include levels of distress that are most common in the general population. Fourth, our sample included children aged 8–42 months and 15 age-specific forms of the screening instrument; results may vary with child age, with relevant BSID-III items, and with ASQ age interval. Finally, all efforts to validate screens for developmental delay among young children are complicated by the absence of a reference measure of known accuracy. Our results describe the effect of distress on the agreement of the ASQ with the BSID-III, but its role on the validity of the ASQ with respect to true developmental status is unknown. That we found variation related to distress is of some interest in part because the ASQ does not invite expressions of concern; rather it asks about specific behaviours and milestones, and it does so in a structured way. Our results show that distress affects reporting even of this kind, independent of actual child functioning. It would therefore be interesting to consider the same question with an instrument (e.g., the Parent Evaluation of Developmental Status) that instead elicits general concerns about functioning, and to investigate the effect of higher levels of distress. Conflicts of interest None. Funding support The studies from which data are taken were supported by a grant from the Ministry of Children and Youth Services of Ontario (037-370203-A518-A16061-577010); The MCYS did not have any involvement in the design, analysis, or interpretation of the current data. References Anderson, P. J., De Luca, C. R., Hutchinson, E., Roberts, G., & Doyle, L. W. (2010). Underestimation of developmental delay by the new Bayley-III Scale. Archives of Pediatrics & Adolescent Medicine, 164(4), 352–356. Barsky, A. J., Goodson, J. D., Lane, R. S., & Cleary, P. D. (1988). The amplification of somatic symptoms. Psychosomatic Medicine, 50(5), 510–519. Bayley, N. (2006). Bayley scales of infant and toddler development. Briggs-Gowan, M. J., Carter, A. S., & Schwab-Stone, M. (1996). Discrepancies among mother, child, and teacher reports: examining the contributions of maternal depression and anxiety. Journal of Abnormal Child Psychology, 24(6), 749–765. http://dx.doi.org/10.1007/bf01664738
S. Veldhuizen et al. / Research in Developmental Disabilities 63 (2017) 11–17
17
Brothers, K. B., Glascoe, F. P., & Robertshaw, N. S. (2008). PEDS: developmental milestones—an accurate brief tool for surveillance and screening. Clinical Pediatrics, 47(3), 271–279. Cohen, R. A. (2011). Yerkes–dodson law. In Encyclopedia of clinical neuropsychology. pp. 2737–2738. Springer. Cole, T. J. (1990). The LMS method for constructing normalized growth standards. European Journal of Clinical Nutrition, 44(1), 45–60. Davis, E., Davies, B., Waters, E., & Priest, N. (2008). The relationship between proxy reported health-related quality of life and parental distress: gender differences. Child: Care, Health and Development, 34(6), 830–837. http://dx.doi.org/10.1111/j.1365-2214.2008.00866.x Dulcan, M. K., Costello, E. J., ANTHON’Y, J. C., Edelbrock, C., Brent, D., & Janiszewski, S. (1990). The pediatrician as gatekeeper to mental health care for children: do parents’ concerns open the gate? Journal of the American Academy of Child & Adolescent Psychiatry, 29(3), 453–458. Foster, F. M., & Wolraich, M. L. (1997). An economic evaluation of four methods for detecting developmental problems. Pediatrics, 99, 830–837. Furukawa, T. A., Kessler, R. C., Slade, T., & Andrews, G. (2003). The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychological Medicine, 33(02), 357–362. Glascoe, & Dworkin, P. H. (1995). The role of parents in the detection of developmental and behavioral problems. Pediatrics, 95(6), 829–836. Glascoe, F. (1994). It’s not what it seems. Clinical Pediatrics, 33(5), 292–296. Glascoe. (2001). Are overreferrals on developmental screening tests really a problem? Archives of Pediatrics & Adolescent Medicine, 155(1), 54–59. Glascoe, F. (2003). Parents’ evaluation of developmental status: how well do parents’ concerns identify children with behavioral and emotional problems? Clinical Pediatrics, 42(2), 133–138. Guttmann, A., Klein-Geltink, J., Kopp, A., & Cairney, J. (2011). Uptake of the new fee code for Ontario’s enhanced 18-Month well baby visit: a preliminary evaluation. Toronto: Institute for Clinical Evaluative Sciences. Henningsen, P., Zimmermann, T., & Sattel, H. (2003). Medically unexplained physical symptoms, anxiety, and depression: a meta-Analytic review. Psychosomatic Medicine, 65(4), 528–533. Hosmer, D., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley. Kessler, R. C., Andrews, G., Colpe, L. J., Hiripi, E., Mroczek, D. K., Normand, S.-L., Walters, E. E., & Zaslavsky, A. M. (2002). Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine, 32(06), 959–976. Kessler, R. C., Green, J. G., Gruber, M. J., Sampson, N. A., Bromet, E., Cuitan, M., Furukawa, T. A., Gureje, O., Hinkov, H., Hu, C. Y., Lara, C., Lee, S., Mneimneh, Z., Myer, L., Oakley-Browne, M., Posada-Villa, J., Sagar, R., Viana, M. C., & Zaslavsky, A. M. (2010). Screening for serious mental illness in the general population with the K6 screening scale: results from the WHO World Mental Health (WMH) survey initiative. International Journal of Methods in Psychiatric Research, 4–22. http://dx.doi.org/10.1002/mpr.310 Kroes, G., Veerman, J. W., & De Bruyn, E. E. (2003). Bias in parental reports? Maternal psychopathology and the reporting of problem behavior in clinic-referred children. European Journal of Psychological Assessment, 19(3), 195. Limbos, M. M., & Joyce, D. P. (2011). Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. Journal of Developmental & Behavioral Pediatrics, 32(7), 499–511. Limbos, M. M., Joyce, D. P., & Nguyen, T. K. (2012). Developmental screening practices in Canada: a survey of primary care providers. Journal of Intellectual Disability Research, 56, 765 [Hoboken, New Jersey: Wiley-Blackwell]. Prochaska, J. J., Sung, H.-Y., Max, W., Shi, Y., & Ong, M. (2012). Validity study of the K6 scale as a measure of moderate mental distress based on mental health treatment need and utilization. International Journal of Methods in Psychiatric Research, 21(2), 88–97. Pulsifer, M. B., Hoon, A. H., Palmer, F. B., Gopalan, R., & Capute, A. J. (1994). Maternal estimates of developmental age in preschool children. The Journal of Pediatrics, 125(1), S18–S24. Raspa, M., Levis, D. M., Kish-Doto, J., Wallace, I., Rice, C., Barger, B., . . . & Wolf, R. B. (2015). Examining parents’ experiences and information needs regarding early identification of developmental delays: qualitative research to inform a public health campaign. Journal of Developmental & Behavioral Pediatrics, 36(8), 575–585. Shedler, J., Mayman, M., & Manis, M. (1993). The illusion of mental health. American Psychologist, 48(11), 1117. Sices, L., Stancin, T., Kirchner, H. L., & Bauchner, H. (2009). PEDS and ASQ developmental screening tests may not identify the same children. Pediatrics, 124(4), e640–e647. http://dx.doi.org/10.1542/peds.2008-2628 Slade, T., Grove, R., & Burgess, P. (2011). Kessler psychological distress scale: normative data from the 2007 Australian National Survey of Mental Health and Wellbeing. Australian and New Zealand Journal of Psychiatry, 45(4), 308–316. Squires, J., Twombly, E., Bricker, L., & Potter, P. (2009). ASQ-3 Ages and Stages Questionnaires user’s guide. Lane County, OR: Brookes Publishing. StataCorp. (2015). Stata statistical software: release 14. College Station, TX: StataCorp LP. Veldhuizen, S., Rodriguez, C., Wade, T. J., & Cairney, J. (2014). Misclassification due to age grouping in measures of child development. Archives of Disease in Childhood, 100(3), 220–224. http://dx.doi.org/10.1136/archdischild-2014-306548 Veldhuizen, S., Clinton, J., Rodriguez, C., Wade, T. J., & Cairney, J. (2015). Concurrent validity of the ages and stages questionnaires and bayley developmental scales in a general population sample. Academic Pediatrics, 15(2), 231–237. http://dx.doi.org/10.1016/j.acap.2014.08.002 Waters, E., Doyle, J., Wolfe, R., Wright, M., Wake, M., & Salmon, L. (2000). Influence of parental gender and self-reported health and illness on parent-reported child health. Pediatrics, 106(6), 1422–1428. Watson, D., & Pennebaker, J. W. (1989). Health complaints, stress, and distress: exploring the central role of negative affectivity. Psychological Review, 96(2), 234.