P

Psychiatry Research 245 (2016) 303–310

Contents lists available at ScienceDirect

Psychiatry Research journal homepage: www.elsevier.com/locate/psychres

Predictors of parent-child agreement on child anxiety diagnoses on the ADIS-IV-C/P Rebecca J. Hamblin a,b,n, Alison Salloum c, Ross Andel d, Joshua M. Nadeau a,b, Nicole M. McBride a, Adam B. Lewin a,f,g, Eric A. Storch a,b,e,h a

Department of Pediatrics, University of South Florida, Tampa, FL, USA Rogers Behavioral Health – Tampa Bay, Tampa, FL, USA c School of Social Work, University of South Florida, Tampa, FL, USA d School of Aging Studies, University of South Florida, Tampa, FL, USA e Department of Health Policy and Management, University of South Florida, Tampa, FL, USA f Department of Psychiatry & Behavioral Neurosciences, University of South Florida, Tampa, FL, USA g Department of Psychology, University of South Florida, Tampa, FL, USA h All Children's Hospital – Johns Hopkins Medicine, St. Petersburg, FL, USA b

art ic l e i nf o

a b s t r a c t

Article history: Received 16 February 2016 Received in revised form 22 June 2016 Accepted 22 July 2016 Available online 25 July 2016

Diagnostic agreement between parents’ and children's reports on children's anxiety problems is notoriously poor; however, very few investigations have examined specific predictors of inter-rater agreement on child anxiety diagnoses. This study examined predictors of categories of parent and child diagnostic endorsement on the Anxiety Disorders Interview Schedule for Children–IV. One hundred eight children (ages 7–13) and their parents completed structured diagnostic interviews for non-OCD/PTSD anxiety diagnoses and paper and pencil measures of functioning and impairment in a variety of domains. Parentchild agreement was statistically significant for social phobia and separation anxiety disorder, but was overall poor for all anxiety diagnoses. Externalizing disorder status, family accommodation frequency, and child rated impairment in various domains differentially predicted informant discrepancies for different anxiety disorders. These data are among the first to suggest variables that may explain parentchild concordance. & Published by Elsevier Ireland Ltd.

Keywords: Specific phobia Separation anxiety Generalized anxiety disorder Inter-rater agreement Anxiety disorders interview schedule Pediatric anxiety Parent-child agreement Diagnostic validity Diagnostic reliability

1. Introduction The multiple informant approach to assessment of child psychopathology is emphasized for obtaining comprehensive information regarding the child's emotional and behavioral functioning across contexts (Achenbach, 2011; Achenbach et al., 1987). Information obtained from children and parents is often contradictory, however, which creates a challenge for clinicians when interpreting the data reported. Inter-informant agreement between children and parents on measures of psychopathology is often poor, with Pearson correlations and kappa agreement ratings reported in the range of 0.2–.3 on average (Achenbach et al., 1987; Foley et al., 2004, 2005; Grills and Ollendick, 2003; Jensen et al., 1999; Kraemer et al., 2003; Lewin et al., 2014; Safford et al., 2005). n Correspondence to: University of South Florida, Department of Pediatrics, Rothman Center for Neuropsychiatry, Child Development & Rehabilitation Center, 880 6th Street South Suite 460 Box 7523, St. Petersburg, FL 33701, USA. E-mail address: [email protected] (R.J. Hamblin).

http://dx.doi.org/10.1016/j.psychres.2016.07.041 0165-1781/& Published by Elsevier Ireland Ltd.

When children and parents give conflicting accounts regarding symptom presence and interference, diagnostic considerations rely upon subjective clinical judgment to determine which informant has provided the most valid information (Achenbach et al., 1987; De Los Reyes and Kazdin, 2005; Grills and Ollendick, 2003; Jensen et al., 1999; Lewin et al., 2012). Informant discrepancies play a major role in the assessment and subsequent treatment of children's mental health problems (De Los Reyes and Kazdin, 2005). Identifying the factors related to parent-child disagreement on measures of psychopathology is crucial for improving the reliability and validity of clinical assessment and guiding treatment decisions. Parent-child agreement on measures of anxiety tends to be poor for both semi-structured interviews and rating-scale questionnaires within clinical and community samples (Choudhury et al., 2003; Foley et al., 2005; Miller et al., 2014; Nauta et al., 2004; Safford et al., 2005; Storch et al., 2012). There are limited available data to guide interpretation of inter-informant discrepancies. Clinicians often consider parent report as more valid

304

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

than child report, particularly for younger children (DiBartolo et al., 1998; Safford et al., 2005; Smith, 2007; Storch et al., 2012); however, there is no systematic evidence base supporting this. In fact, in the only study to date that has compared predictive validity of informant reports, children's self-reported social anxiety better predicted in-vivo anxiety ratings and observer-rated inhibition in a public speaking task than did parent or teacher reports, suggesting the need for further evaluation of factors that predict informant ratings (DiBartolo and Grills, 2006). Examinations of moderators of parent-child informant discrepancies have generally focused on characteristics of the child, parent, and/or type of problem measured. Very few child characteristics have been identified as consistent moderators; older age is inconsistently related to better (but still poor) agreement for anxiety disorders (Choudhury et al., 2003; Grills and Ollendick, 2003) though there have been contrasting findings reported (Safford et al., 2005). When present, this relationship is typically attributed to older children being better able than younger children to verbalize symptoms, both to their parents and in the context of assessment (Canavera et al., 2009; Choudhury et al., 2003; Duhig et al., 2000; Grills and Ollendick, 2003; Storch et al., 2012; van de Looij-Jansen et al., 2011). Examinations of child gender, ethnicity, and social desirability as moderators have yielded inconsistent or null results (Canavera et al., 2009; Duhig et al., 2000; Storch et al., 2012; van de Looij-Jansen et al., 2011). In contrast, higher levels of parental psychopathology (i.e., depression and anxiety) is associated with greater parent-child disagreement in most studies (Becker et al., 2016; Berg-Nielsen et al., 2003; Briggs-Gowan et al., 1996; Foley et al., 2005; Hughes and Gullone, 2010). Informant agreement tends to be higher for observable and objective behavior than for private events (e.g., thoughts and feelings) and subjective ratings (Comer and Kendall, 2004; Duhig et al., 2000). Children may not have the lexicon to verbalize experiences of private events, leaving parents to infer distress based on external signs, which, in children, may not appear to be closely related to internal distress (DiBartolo and Grills, 2006). It has been suggested that parents may under-report when they do not recognize unseen symptoms of anxiety in their children (DiBartolo and Grills, 2006; Safford et al., 2005); on the other hand, children may under-report if they do not have adequate perspective taking ability or insight to accurately report on a pattern of private events and subjective distress over time (Grills and Ollendick, 2003). Although existing research provides descriptive information pertaining to the magnitude of informant discrepancies, there is little evidence to date that contributes to understanding why there are informant discrepancy or providing guidance for differentially determining informant validity in the context of assessment (De Los Reyes and Kazdin, 2005; Foley et al., 2005). Additionally, studies have focused on moderators of overall concordance, but have not examined whether specific informants are more likely to endorse or deny diagnostic criteria under given conditions. The purpose of this study was to examine predictors of child and parent endorsement or denial of major anxiety diagnoses and the convergence or divergence of their reports. It was hypothesized that parents and children would endorse diagnoses only when symptoms presented a significant personal impact as measured by the following indices. 1.1. Child distress and impairment across context It has been suggested that informant discrepancies are a result of differing interactions of the child across different settings (Achenbach, 2011; Achenbach et al., 1987). It has also been suggested that differences in perceived distress or impairment may account for differences in informant reports (De Los Reyes and

Kazdin, 2005). If children do not perceive their symptoms or behavior as problematic, they are unlikely to report them as such even if their behaviors pose a difficulty for others. On the other hand, if anxiety poses problems that are relevant to the child and create distress in a context that is valuable to them (i.e., social) they may be more likely to perceive and report the anxiety as interfering. For example, if anxiety is primarily problematic at home and does not manifest in the school setting, the parents may be more likely than the child to describe the problem as rising to the level of diagnostic concern. Alternatively, if the anxiety causes distress in a context unseen by the parent but relevant to the child (e.g., social reticence with peers at school), the child may be more likely than the parent to endorse a diagnosis (De Los Reyes et al., 2015). This investigation examined the relationship between children's perceived anxiety related functional impairment across contexts (school, social, and family) and parent and child endorsement of anxiety disorders. 1.2. Externalizing behavior Externalizing behavior is directly disruptive to family functioning and is easily observed by parents. Attempts to escape or avoid aversive stimuli (e.g., schoolwork, social situations) may present as tantrums, defiance, or aggression on the part of the child. These behaviors are likely to disrupt a parent's routine but may not create distress for the child, especially if the externalizing behavior functions to successfully avoid aversive stimuli. Therefore, parents may be more disturbed by a child's symptoms than the child when externalizing behaviors are prominent. Children with externalizing behaviors such as opposition or defiance may also be unwilling to cooperate with interviewers and may intentionally provide inaccurate information (Foley et al., 2004; Storch et al., 2012). 1.3. Family accommodation Family accommodation, or behaviors on the part of family members to reduce distress or functional impairment associated with anxiety, is highly prevalent within families of anxious youth (Lebowitz et al., 2016; Storch et al., 2013; Thompson-Hollands et al., 2014). Accommodation, such as providing verbal reassurance or changing family routines to allow avoidance, serves to temporarily relieve anxiety and increases over time vis a vis negative reinforcement (Benito and Freeman, 2011; Lebowitz et al., 2014). To the extent that accommodations are distressing or burdensome to the parents, the parents may be more likely to perceive a problem and be motivated to seek treatment for the child and therefore more likely to endorse symptoms at the diagnostic level (Benito and Freeman, 2011; Foley et al., 2004; Lebowitz et al., 2016). In contrast, greater frequency and intensity of family accommodation is likely to produce the opposite effect in children; the more they are successful in avoiding feared situations and relieving fear and doubt, the less they have to endure long periods of subjective internal discomfort (Thompson-Hollands et al., 2015). Indeed, family accommodation is associated with reduced insight in children and adolescents (Bipeta et al., 2013), which supports the idea that accommodation serves to reduce their subjective discomfort and may translate to reduced reporting. The purpose of this study was to examine the overall level of agreement between parents, children, and clinicians across common child anxiety disorders and to examine predictors of parent and child endorsement and non-endorsement of the same disorders. We expected that older age and absence of externalizing disorders would predict higher overall agreement for each of the diagnoses examined. We also expected that the relative impact of the child's symptoms on parent vs. child reports and across

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

settings would predict different categories of disagreement and agreement (i.e., parent and child endorsement and denial). Specifically, we hypothesized that family accommodation would be related to greater disagreement between parents and youth with parents more likely to endorse anxiety diagnoses in the absence of child endorsement. We expected that child-rated impairment from anxiety in the school and social contexts would predict child endorsement in the absence of parent endorsement while impairment in the home setting would predict mutual endorsement of the presence of disorders. Finally, we expected that clinicianrated global illness severity would predict mutual parent and child endorsement of diagnoses.

2. Method 2.1. Participants Participants included 108 children and adolescents (60 male) and their primary caregiver who consented for inclusion in a study examining the efficacy of computer-assisted cognitive-behavioral therapy to address anxiety disorders. Youth ranged in age from 7 to 13 years (M ¼9.90 years). Most participants were Caucasian (71%), with the remainder identified by parents as Hispanic (11%), Black (13%), Asian/Pacific Islander (2%), or Other (2%). More than half (55.9%) of the families reported an annual income of less than $40,000; 24.7% reported annual incomes between $40,000 and $90,000, and 19.4% of the families reported earning over $90,000. All participants met criteria for a diagnosis of an anxiety disorder. These diagnoses were based upon structured clinical interview via the Anxiety Disorders Interview Schedule for DSM-IV-Child and Parent Versions (Silverman and Albano, 1996), as administered by a trained independent evaluator in consensus with a review team. Sample characteristics by diagnostic category are presented in Table 1. Most participants (75.9%) met criteria for more than one diagnosis. 2.2. Procedures Procedures specific to the study of computer-assisted cognitive behavioral therapy are described in Storch et al. (2015). Upon scheduling the baseline assessment for entry into the therapybased study, written informed consent from parents and assent from children were obtained. Next, an independent evaluator at a university-based research center completed the assessment interview with child and parents separately via a secure web camera-based communication program. Independent evaluators were trained in a day-long initial workshop with didactics and live practice, were observed and given feedback during subsequent practice confederate interviews, then received weekly supervision during the course of the study. Diagnoses were determined by a consensus review team including the independent evaluator and were based upon all available clinical information. A diagnosis was Table 1 Sample diagnostic characteristics.

Generalized anxiety Social phobia Separation anxiety Specific phobia Disruptive behavior disorder Panic disorder Depressive disorder Attention-deficit disorders

Primary diagnosis

Diagnosis present

41 (38.0%) 27 (25.0%) 26 (24.1%) 9 (8.3%) 2 (1.9%) 1 (0.9%) 1 (0.9%) 0 (0.0%)

70 (64.8%) 40 (37.0%) 43 (39.8%) 22 (20.4%) 11 (10.2%) 3 (2.8%) 10 (9.3%) 36 (33.3%)

305

given if the Clinical Severity Rating was Z4 for that diagnosis based upon degree of distress/impairment. Participants were located in one of three community mental health agencies located in Florida. All study procedures were approved by the local Institutional Review Board. 2.3. Measures 2.3.1. Anxiety dsorders interview schedule – child and parent The ADIS-C/P (Silverman and Albano, 1996) are clinician-administered semi-structured interviews, based upon the Diagnostic and Statistical Manual for Mental Disorders- IV-TR (American Psychiatric Association, 2000) diagnostic system designed to assess anxiety, mood and externalizing childhood disorders. Although there is considerable overlap between parent and child versions of the ADIS, the parent version includes coverage of additional disorders (e.g., conduct disorder, oppositional-defiant disorder, enuresis), as well as greater detail regarding etiology of specific problems. The interviews utilize obtained ratings of interference in the child's daily life on a scale from 0 (symptom-free) to 8 (serious symptoms, limited quality of life) for each psychiatric disorder. When combined, the separate ratings from child and parent interview administrations aid the clinician in deciding whether or not the child meets diagnostic criteria. A cut-off score of 4 (moderate impairment) on the 0–8 scale is commonly used to suggest severity consistent with diagnostic level. Test-retest (Silverman et al., 2001) and concurrent validity (Wood et al., 2002) are reported as excellent for the ADIS-C/P for DSM-IV. All sections of the ADIS-C/P were administered for this study; the sections for panic disorder, generalized anxiety disorder, separation anxiety disorder, and specific phobia were analyzed for parent-child agreement. 2.3.2. Pediatric Accommodation Scale-Parent Report (PAS-PR) The PAS-PR (Benito et al., 2015) is a 10-item parent-rated measure assessing the degree to which family members have accommodated the child's anxiety symptoms during the previous month (9 items), as well as the level of distress or impairment that family members and patient experience as a result of the family accommodating or not accommodating the child (4 items). Items are combined to create a total score. Each item is scored on a 5-point Likert-type scale, with specific anchors varying by item. Psychometric properties, including internal consistency, interrater reliability, and convergent and discriminant validity, are reported as good for the PAS-PR. (Benito et al., 2015). 2.3.3. Child Anxiety Impact Scale – Child (CAIS-C) The CAIS-C (Langley et al., 2014) is a 33-item self-report scale assessing impairment caused by anxiety in several domains of functioning; school functioning, social, and home. Items are rated on a 4-point scale ranging from 0 (no problem) to 3 (very much) reflecting how much anxiety has gotten in the way of performing daily activities in each domain. The CAIS-C has demonstrated adequate psychometric properties (Langley et al., 2014). Internal consistency reliabilities for the present sample were good for the School, social, and home scales (Cronbach's α ¼ .82, 0.80, and 0.80, respectively).

3. Analytic plan The conventional alpha level of po 0.05 was set for all analyses. Cohen's κ (Cohen, 1960) was used to evaluate inter-rater agreement on the ADIS-C/P. According to criteria established by Mannuzza et al. (1989), values of κ o0.40 are considered to indicate poor inter-rater agreement, values in the range of 0.40–0.60 are

306

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

considered fair, and values above 0.60 and 0.74 and considered good to excellent, respectively. We considered the agreement between raters on diagnoses of separation anxiety disorder (SAD), social phobia, specific phobia, and generalized anxiety disorder (GAD). Panic disorder was not included in the analyses due to very few participants meeting diagnostic criteria. Raters were considered to have endorsed a diagnosis if they endorsed symptoms meeting DSM-IV threshold criteria and gave a CSR impairment rating of greater than or equal to 4 on the ADIS-IV-C/P. If both raters gave CSR ratings Z4 they were considered to be in agreement. The moderating influences of age and externalizing disorder were examined by evaluating Cohen's κ after splitting the sample into groups for each moderating variable. Specifically, agreement by age was examined in younger (ages 7–10) and older children (ages 11–13), and children were grouped as having an externalizing disorder if they met criteria for a disruptive behavior disorder (i.e., oppositional defiant or conduct disorder) according to clinician rating on the ADIS-C/P. A series of multinomial logistic regression analyses was conducted to examine the predictive value of family accommodation and child-rated impairment in the school, social and home contexts. For each analysis, agreement between child and parent was categorized as 1¼ both parties agree to presence of diagnosis, 2 ¼child endorses and parent denies diagnosis, 3¼ child denies and parent endorses, and 4 ¼parent and child deny diagnosis. Category 4 was set as the reference category for all analyses.

4. Results

Table 3 Multiple informant agreement on the ADIS-C/P by age as measured by Cohen's κ. Diagnosis

Ages 7–10 Separation an0.26* xiety disorder Social phobia 0.10 Specific phobia 0.05 Generalized an-  0.10 xiety disorder * **

Child-clinician agreement by diagnosis

Ages 11–13

Ages 7–10

Ages 11–13

Ages 7–10

Ages 11–13

0.37**

0.80**

0.58**

0.25*

0.32*

0.29**  0.04 0.15

0.22* 0.32** 0.31*

0.42** 0.19* 0.50**

0.18 0.18 0.13

0.22  0.03 0.24*

p o.05. p o .01.

Table 4 Agreement between parent and child on the ADIS-C/P for children by externalizing status as measured by Cohen's κ. Diagnosis

Non-externalizing (n ¼97)

Externalizing (n¼ 11)

Separation anxiety disorder Social phobia Specific phobia Generalized anxiety disorder

0.40**

0.12

* **

Results of inter-rater agreement analyses between child, parent, and clinician are presented in Table 2. Parent-child agreement was generally poor, though it was statistically significant for SAD and social phobia. Parent and clinician agreement across diagnoses was uniformly significant, and SAD and GAD demonstrated excellent and fair agreement, respectively. Results of the moderation analysis for age are presented in Table 3. Age was a significant moderator for parent-child agreement for social phobia only, where older children and parents demonstrated higher overall agreement. Externalizing disorder status was a significant moderator only for social phobia diagnosis, with non-externalizing children showing higher agreement with parents. Results of the moderation analysis by externalizing status are presented in Table 4. The results of the multinomial logistic regression analysis predicting agreement category by family accommodation scores are presented in Table 5. Frequency of family accommodation increased the likelihood of being in both the first and third

Parent-clinician agreement by diagnosis

Child-parent agreement by diagnosis

*

0.15 0.44 0.11

0.19  0.05 0.00

p o0.05. p o 0.01.

categories for SAD and in the third category for social phobia. Frequency of accommodation failed to predict any of the agreement categories for specific phobia diagnosis or GAD diagnosis. Results of the subsequent regression analyses predicting agreement category by child-rated impact in the school, social, and home environments are summarized in Tables 6–8. For the second regression analysis, child-rated impact of anxiety on well-being at school predicted membership in category 2 (child endorsed parent denied) for both SAD and GAD. Impact of anxiety at school failed to predict any of the agreement categories for social phobia or specific phobia. Child-rated impact of anxiety in the social environment predicted all three agreement categories for SAD and category 1 (parent and child both endorsed) for social phobia but none of the categories for specific phobia or GAD. Child-rated impact of anxiety at home predicted membership in categories 1 (parent and child both endorsed) and 2 (child endorsed parent denied) for

Table 2 Multiple informant agreement on the ADIS-C/P as measured by Cohen's κ. Diagnosis

Separation anxiety disorder Social phobia Specific phobia Generalized anxiety disorder

Child-parent agreement

p o 0.05. po 0.01. *** p o 0.001. **

Child-clinician agreement

κ

þþ

þ

þ



κ

þþ

þ

þ



κ

þþ

þ

þ



0.38*** 0.20** 0.00 0.01

27 31 30 24

8 4 20 7

25 46 35 58

48 27 23 19

0.76*** 0.32** 0.26*** 0.39**

41 38 21 62

11 39 44 20

2 2 1 8

54 29 42 18

0.36*** 0.21* 0.11 0.19**

23 18 13 26

12 17 37 5

20 22 9 44

53 51 49 33

þ þ ¼Both parties endorsed. þ  ¼First party endorsed, second denied.  þ ¼First party denied, second endorsed.   ¼Both deny. *

Parent-clinician agreement

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

Table 5 Nominal logistic regression predicting agreement category by family accommodation frequency.

307

Table 7 Nominal logistic regression predicting agreement category by child reported impact of anxiety in the social environment.

Agreement category

Separation anxiety

B

Sig.

Exp (B)

Agreement category

Separation anxiety

B

Sig.

Exp (B)

1 2 3

Family Accom Frequency Family Accom Frequency Family Accom Frequency

0.21 0.06 0.27

0.003 0.586 0.000

1.23 1.06 1.30

1 2 3

CAISC Social CAISC Social CAISC Social

0.13 0.14 0.09

0.003 0.021 0.050

1.14 1.15 1.09

1 2 3

Social phobia Family Accom Frequency Family Accom Frequency Family Accom Frequency

0.07 0.10 0.17

0.332 0.450 0.010

1.07 1.11 1.18

1 2 3

Social phobia CAISC Social CAISC Social CAISC Social

0.13 0.01 0.04

0.011 0.905 0.390

1.13 1.01 1.04

1 2 3

Specific phobia Family Accom Frequency Family Accom Frequency Family Accom Frequency

0.02  0.04 0.03

0.791 0.566 0.712

1.02 0.96 1.03

1 2 3

Specific phobia CAISC Social CAISC Social CAISC Social

0.02 0.02  0.01

0.630 0.636 0.808

1.02 1.02 1.00

1 2 3

GAD Family Accom Frequency Family Accom Frequency Family Accom Frequency

0.13 0.11 0.11

0.109 0.339 0.106

1.14 1.11 1.12

1 2 3

GAD CAISC Social CAISC Social CAISC Social

0.00 0.12  0.02

0.977 0.065 0.649

1.00 1.12 0.980

Category 1¼ Child Endorse/Parent Endorse. Category 2¼ Child Endorse/Parent Deny. Category 3¼ Child Deny/Parent Endorse.

Category 1¼ Child Endorse/Parent Endorse, Category 2¼ Child Endorse/Parent Deny, Category 3¼ Child Deny/Parent Endorse.

Table 6 Nominal logistic regression predicting agreement category by child reported impact of anxiety in the school environment. Agreement category

Separation anxiety

B

Sig.

Exp (B)

1 2 3

CAISC School CAISC School CAISC School

0.03 0.13 0.06

0.439 0.015 0.113

1.03 1.14 1.06

1 2 3

Social phobia CAISC School CAISC School CAISC School

0.08 0.05  0.02

0.05 0.564 0.659

1.08 1.05 0.98

1 2 3

Specific phobia CAISC School CAISC School CAISC School

0.00 0.08 0.00

0.938 0.064 0.956

1.00 1.09 1.00

1 2 3

GAD CAISC School CAISC School CAISC School

0.06 0.14 0.06

0.221 0.030 0.228

1.06 1.15 1.06

Category 1 ¼Child Endorse/Parent Endorse, Category 2 ¼Child Endorse/Parent Deny, Category 3¼ Child Deny/Parent Endorse.

SAD, category 1 (parent and child both endorsed) for social phobia, and category 2 (child endorsed parent denied) for GAD.

5. Discussion This study investigated agreement between parents, children, and clinicians across common child anxiety disorders and predictors of parent and child endorsement and non-endorsement of child anxiety disorders. As expected, parent-child agreement was poor across all diagnostic categories, although separation anxiety and social phobia agreement reached statistical significance at small effect sizes. Parent-clinician agreement was markedly higher than parent-child agreement and child-clinician agreement. This is consistent with other clinical samples (Grills and Ollendick, 2003;

Table 8 Nominal logistic regression predicting agreement category by child reported impact of anxiety in the home environment. Agreement category

Separation anxiety

B

Sig.

Exp (B)

1 2 3

CAISC Home CAISC Home CAISC Home

0.17 0.22 0.12

0.012 0.016 0.085

1.18 1.24 1.13

1 2 3

Social phobia CAISC Home CAISC Home CAISC Home

0.17 0.15  0.02

0.018 0.225 0.784

1.19 1.16 1.00

1 2 3

Specific phobia CAISC Home CAISC Home CAISC Home

0.02 0.04  0.03

0.788 0.561 0.624

1.02 1.04 0.97

1 2 3

GAD CAISC Home CAISC Home CAISC Home

0.15 0.23 0.10

0.091 0.035 0.238

1.16 1.26 1.10

Category 1¼ Child Endorse/Parent Endorse, Category 2¼ Child Endorse/Parent Deny, Category 3¼ Child Deny/Parent Endorse.

Storch et al., 2012; Ung et al., 2014) and may reflect the population of treatment-seeking families; children are referred by parents for treatment and thus, parents have identified anxiety in at least one area to present for treatment. The fact that clinicians tend to base their own assessments on parent report may be an artifact of parents reporting more symptoms and/or diagnoses than children. That is, clinicians may be more likely to accept an endorsement over a denial of symptoms. This pattern has been seen in other clinical samples as well (Choudhury et al., 2003; Grills and Ollendick, 2003; Lewin et al., 2014; Lewin et al., 2012). In the case of GAD, however, child-clinician agreement was superior to childparent agreement. Further, although most of the parents endorsed the diagnosis, the clinicians did not accept all of the parent endorsements, and did accept some child endorsements where the parent did not endorse, indicating that the clinicians may favor the

308

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

child's report some of the time. Previous studies have reported inconsistent findings with regard to agreement level as a function of age and externalizing status (Grills and Ollendick, 2003; Storch et al., 2012). Contrary to expectations, we found little difference in overall agreement levels of older versus younger children, or for externalizing versus nonexternalizing children, with the exception that agreement was better for the diagnosis of social phobia for older children and those without comorbid externalizing disorders. As noted in previous studies, older children may be better able than younger children to verbalize distress symptoms both to parents and in the context of an interview (Adelman and Lebowitz, 2012; Brown-Jacobsen et al., 2011; Canavera et al., 2009; Choudhury et al., 2003). It is also possible that parents and children converge on the level of shyness or social anxiety deemed developmentally appropriate as children age, and that social anxiety becomes increasingly impairing and distressing as children get older. With regard to externalizing diagnosis, it is possible that parents may be more likely to miss anxiety symptoms if they interpret behavior as oppositional or poor conduct. It is also possible that children with externalizing disorders may lack insight and perspective taking necessary to report on symptoms; however, this does not explain why the difference was found only for social phobia. Another possibility is that the parent-child relationship may be stronger in the absence of disruptive behavior problems, thus children and parents are more likely to agree due to better overall communication and understanding; however, again, this does not explain why the difference was only found for social phobia. Children with social anxiety may be uncomfortable in the clinical setting with a novel authority figure, and those with externalizing behavior problems may be more likely to deny symptoms to escape the interview than children who do not tend to manifest oppositional behavior. We expected increased frequency of family accommodation to predict parent endorsement and child non-endorsement of symptoms for all diagnoses. Our hypothesis was partially supported by the finding that accommodation predicted this agreement category for SAD and social phobia. Family accommodation may be most relevant for SAD, which by definition involves the relationship between the parent and child. Further, SAD frequently involves school reluctance, and children may refuse to leave parents to attend school, or call multiple times per day for reassurance or to be picked up (Lebowitz et al., 2014). To the extent that the parents cooperate with these accommodation requests, increased demand is placed on the parent and they may be particularly motivated to seek treatment (Benito and Freeman, 2011; Brown-Jacobsen et al., 2011). The mutual communication regarding the anxiety symptoms involved in SAD also explains the prediction of mutual agreement on the presence of symptoms. For some children, accommodation of symptoms may release the distress burden, decreasing the perceived significance of the problem and their motivation to seek treatment. Other children, however, may be bothered by having to leave school, or by their distress in spite of frequent opportunities to escape. With regard to social phobia, family accommodation may come in the form of the parents changing schedules to allow the child to sit out of activities, parents speaking for the child in novel social situations, or providing frequent reassurance about social behavior (Lebowitz et al., 2016; Thompson-Hollands et al., 2014). Again, these activities place demand on the parents and relieve distress for the child, which may influence motivation for treatment on the part of both parties. Frequency of accommodation failed to predict any of the agreement categories for specific phobia diagnosis or GAD diagnosis, suggesting that family accommodation was not a major factor in determining agreement for these disorders. While

children with GAD may frequently seek reassurance and engage in checking behavior (Lebowitz et al., 2016; Thompson-Hollands et al., 2014), it is possible that these behaviors are less bothersome to parents than the types of accommodation common in SAD and social anxiety, and therefore bear less relevance in predicting parent endorsement of symptoms. Additionally, checking and reassurance seeking may provide little relief for children plagued with chronic worry and therefore not be predictive of motivation to report symptoms or seek treatment. Child-rated impact of anxiety on well-being at school predicted child endorsement and parent non-endorsement for both SAD and GAD. We hypothesized that anxiety in the school environment would be especially bothersome for children whereas the symptoms would not necessarily be seen by parents. Among community samples, evidence suggests that teacher and child reports tend to converge more on behavior in the classroom than do parent and child report (Miller et al., 2014). Surprisingly, impact of anxiety at school failed to predict any of the agreement categories for social phobia. It was expected that children would encounter the widest array of social and performance situations while at school and therefore might be most affected by their anxiety in a context away from their parents; however, impairment in the social environment predicted parent-child agreement for presence of social phobia, suggesting that parents are as aware of impairment in this domain as children. Child-rated impairment in the social environment also predicted all three agreement categories for SAD, suggesting that social impairment was predictive of both endorsement and nonendorsement of SAD on the part of parents and children. In the case of child endorsement parent non-endorsement of SAD, it is possible that children with separation fears are subjected to teasing or ridicule outside the knowledge of their parents. On the other hand, parents may become aware of their children's fears if the child refuses to sleepover at the houses of friends or relatives. In spite of significant distress, children who experience embarrassment or shame in relation to their anxiety may be less likely to endorse symptoms even within the clinical setting for fear of judgment. This may be especially relevant in the case of SAD, the symptoms of which may be viewed as developmentally inappropriate even by the children. We expected that child-rated impact of anxiety at home would predict parent-child agreement on endorsement, which was partially supported by findings for SAD and social phobia. The home environment is shared by parents and children, and thus is the setting in which both informants have the most similar information on the child's behavior and functioning. It was therefore surprising that home impairment predicted child endorsement parent non-endorsement of GAD symptoms. One explanation is that children with GAD may present as irritable and restless (Stoddard et al., 2014), which may be attributed to misbehavior or other sources by parents. The observation that agreement did not vary by externalizing status partially diminishes this possibility, though it is likely that the irritability associated with GAD would be sub-threshold for a disruptive behavior disorder in most cases. It is also possible that GAD symptoms may not manifest in overt attempts at escape and avoidance of concrete stimuli, and are therefore less noticeable to parents. None of the variables examined predicted agreement or disagreement on the diagnosis of specific phobia. Though they did not often agree, both parents and children generally tended to endorse specific phobia with greater frequency than other disorders. In contrast, clinicians tended to disagree with parent and child endorsements of specific phobia more frequently than for other disorders. One possibility is that parents and children reported on subclinical levels of anxiety or developmentally normative fears, while clinicians did not necessarily assess the fears as

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

causing clinically significant impairment. If it is indeed the case that parents and children tended to over-report on specific phobia diagnoses, it follows that family accommodation and impact of the phobia would be low and less relevant to whether or not parents and children agreed. In fact, it may be that due to more limited contact with specific phobia stimuli, specific phobia may present impairment with less frequency or in fewer domains, again limiting the relevance of family accommodation and impairment ratings. There was an overall trend for child-rated impairment to predict parent and child agreement on endorsement or child endorsement parent non-endorsement of symptoms. Previous researchers have suggested that although parent reports are beneficial for obtaining the most information in relation to child anxiety, the child's self-rated distress and impairment are crucial in gaining understanding of the functional impact of symptoms and motivation for treatment (De Los Reyes and Kazdin, 2005). Additionally, as anxiety disorders are subjective phenomena, it is important to respect the child's experiences, as they may be aware of problems that they are not able or not asked to verbalize to parents (DiBartolo and Grills, 2006). The current findings generally supported our hypothesis that impairment in different contexts would differentially predict who endorsed which diagnoses. These findings call into question the current practice of emphasizing parent over child report of symptoms. Additionally, our findings reinforce the need to obtain written reports in addition to verbal reports. It may be easier for some children, especially those with social anxiety, shame about symptoms, or disruptive behavior to give quantitative ratings privately and at their own pace than to provide verbal descriptions in an interview. This suggestion is underscored by findings of much higher agreement between parents and children and higher ratings of anxiety overall when assessed via self -report measures (Miller et al., 2014; Stevanovic et al., 2012). The present study has several limitations. First, as noted, we examined endorsement of diagnoses including the reporting of significant impairment rather than the report of symptoms. It is possible that agreement would have been higher if we were to examine individual symptom endorsement. Stevanovic et al. (2012) found good agreement on a self and parent report measure for child anxiety disorders when using raw scores, but agreement was only moderate when using criterion referenced scores indicating clinical levels of anxiety. Second, we used general measures of family accommodation and impairment that did not specify ratings of each by individual diagnoses. Given the high rate of comorbidity among anxiety disorders, it is likely that accommodation and impairment ratings were applied to multiple anxiety domains and were less relevant to some domains than others. Future studies could address this issue by gathering impairment and accommodation ratings for each individual diagnosis. Relatedly, this study depended on self and parent-report measures of impairment and accommodation. To the extent that some children and or/parents tended to give high ratings for study questions, it would be expected that ratings on paper reports and verbal endorsement of diagnoses would be related due to the tendency to over-report; however, the findings that family accommodation and child-rated impairment predicted endorsement and non-endorsement patterns generally in the manner hypothesized and that results were significant for some diagnoses and not others undermines the notion that findings were an artifact of overreporting. Conclusions of causation are precluded by the cross-sectional design of this study; prospective studies examining whether reduction in family accommodation and child rated impairment mediate improvements in agreement ratings would be informative. Future studies should also examine other potential

309

variables that could influence parent-child agreement. Examples include parents’ and children's knowledge and perceptions of mental health issues, children's anxiety about treatment, and parent and child treatment expectations. Despite its limitations, the present study advances the current body of knowledge regarding parent-child agreement on anxiety disorders by providing information on when parents and children tend to endorse/deny symptoms in relation to one another. In other words, this study builds upon previous studies by examining which party is endorsing a disorder when the other does not. This study is also the first to examine predictors of agreement with variables that might explain motivation to report or not report. Taken collectively, the findings demonstrated a pattern of endorsement by the party bearing the greatest burden from the anxiety symptoms and by the individual(s) who would observe the symptoms in the context in which they manifest. These findings reinforce the need to consider child report as well as parent report and to consider the context of the impairment when deciding which report to weigh most heavily in the presence of disagreement on diagnosis.

References Achenbach, T.M., 2011. Commentary: definitely more than measurement error–but how should we understand and deal with informant discrepancies? J. Clin. Child Adolesc. Psychol. 40 (1), 80–86. Achenbach, T.M., McConaughy, S.H., Howell, C.T., 1987. Child-adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol. Bull., 213. Adelman, C.B., Lebowitz, E.R., 2012. Poor insight in pediatric obsessive compulsive disorder: developmental considerations, treatment implications, and potential strategies for improving insight. J. Obsessive-Compuls. Relat. Disord. 1, 119–124. American Psychiatric Association, 2000. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV-TR. Author, Washington, DC. Becker, E.M., Jensen-Doss, A., Kendall, P.C., Birmaher, B., Ginsburg, G.S., 2016. All anxiety is not created equal: correlates of parent/youth agreement vary across subtypes of anxiety. J. Psychopathol. Behav. Assess., 1–10. Benito, K.G., Freeman, J., 2011. Pediatric Anxiety: How Family Accommodation May Hinder Treatment. Wiley Periodicals, Inc., p. 1. Benito, K.G., Caporino, N.E., Frank, H.E., Ramanujam, K., Garcia, A., Freeman, J., Kendall, P.C., Geffken, G., Storch, E.A., 2015. Development of the pediatric accommodation scale: reliability and validity of clinician- and parent-report measures. J. Anxiety Disord. 29, 14–24. Berg-Nielsen, T.S., Vika, A., Dahl, A.A., 2003. When adolescents disagree with their mothers: CBCL-YSR discrepancies related to maternal depression and adolescent self-esteem. Child: Care Health Dev. 29 (3), 207–213. Bipeta, R., Yerramilli, S.S.R.R., Pingali, S., Karredla, A.R., Ali, M.O., 2013. A crosssectional study of insight and family accommodation in pediatric obsessivecompulsive disorder. Child. Adolesc. Psychiatry Ment. Health, 7. Briggs-Gowan, M.J., Carter, A.S., Schwab-Stone, M., 1996. Discrepancies among mother, child, and teacher reports: examining the contributions of maternal depression and anxiety. J. Abnorm. Child Psychol. 24, 749–765. Brown-Jacobsen, A.M., Wallace, D.P., Whiteside, S.P.H., 2011. Multimethod, multiinformant agreement, and positive predictive value in the identification of Child Anxiety Disorders using the SCAS and ADIS-C. Assessment 18 (3), 382–392. Canavera, K.E., Wilkins, K.C., Pincus, D.B., Ehrenreich-May, J.T., 2009. Parent-child agreement in the assessment of obsessive-compulsive disorder. J. Clin. Child Adolesc. Psychol. 38 (6), 909–915. Choudhury, M.S., Pimentel, S.S., Kendall, P.C., 2003. Childhood anxiety disorders: parent-child (dis)agreement using a structured interview for the DSM-IV. J. Am. Acad. Child Adolesc. Psychiatry 42 (8), 957–964. Cohen, J., 1960. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20 (1), 37. Comer, J.S., Kendall, P.C., 2004. A symptom-level examination of parent-child agreement in the diagnosis of anxious youths. J. Am. Acad. Child Adolesc. Psychiatry 7, 878. De Los Reyes, A., Kazdin, A.E., 2005. Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychol. Bull. 131 (4), 483–509. De Los Reyes, A., Augenstein, T.M., Wang, M., Thomas, S.A., Drabick, D.A.G., Burgers, D.E., Rabinowitz, J., 2015. The validity of the multi-informant approach to assessing child and adolescent mental health. Psychol. Bull. 141 (4), 858–900. DiBartolo, P.M., Grills, A.E., 2006. Who is best at predicting children's anxiety in response to a social evaluative task?: A comparison of child, parent, and teacher reports. J. Anxiety Disord. 20 (5), 630–645. DiBartolo, P.M., Albano, A.M., Barlow, D.H., Heimberg, R.G., 1998. Cross-informant

310

R.J. Hamblin et al. / Psychiatry Research 245 (2016) 303–310

agreement in the assessment of social phobia in youth. J. Abnorm. Child Psychol. 3, 213. Duhig, A.M., Renk, K., Epstein, M.K., Phares, V., 2000. Interparental agreement on internalizing, externalizing, and total behavior problems: a meta-analysis. Clin. Psychol.: Sci. Pract. 7 (4), 435. Foley, D., Rutter, M., Pickles, A., Angold, A., Maes, H., Silberg, J., Eaves, L., 2004. Informant disagreement for separation anxiety disorder. J. Am. Acad. Child Adolesc. Psychiatry 4, 452. Foley, D.L., Rutter, M., Angold, A., Pickles, A., Maes, H.M., Silberg, J.L., Eaves, L.J., 2005. Making sense of informant disagreement for overanxious disorder. J. Anxiety Disord. 19, 193–210. Grills, A.E., Ollendick, T.H., 2003. Multiple informant agreement and the anxiety disorders interview schedule for parents and children. J. Am. Acad. Child Adolesc. Psychiatry 1, 30. Hughes, E.K., Gullone, E., 2010. Discrepancies between adolescent, mother, and father reports of adolescent internalizing symptom levels and their association with parent symptoms. J. Clin. Psychol. 66 (9), 978–995. Jensen, P.S., Rubio-Stipec, M., Canino, G., Bird, H.R., Dulcan, M.K., Schwab-Stone, M. E., Lahey, B.B., 1999. Articles: parent and child contributions to diagnosis of mental disorder: are both informants always necessary? J. Am. Acad. Child Adolesc. Psychiatry 38, 1569–1579. Kraemer, H.C., Measelle, J.R., Ablow, J.C., Essex, M.J., Boyce, W.T., Kupfer, D.J., 2003. A new approach to integrating data from multiple informants in psychiatric assessment and research: mixing and matching contexts and perspectives. Am. J. Psychiatry 160 (9), 1566. Langley, A.K., Falk, A., Peris, T., Wiley, J.F., Kendall, P.C., Ginsburg, G., Birmaher, B., March, J., Albano, A.M., Piacentini, J., 2014. The Child anxiety impact scale: examining parent- and child-reported impairment in child anxiety disorders. J. Clin. Child Adolesc. Psychol. 43 (4), 579–591. Lebowitz, E.R., Scharfstein, L.A., Jones, J., 2014. Comparing family accommodation in pediatric obsessive-compulsive disorder, anxiety disorders, and nonanxious children. Depression Anxiety 31 (12), 1018–1025 (1091–4269). Lebowitz, E.R., Panza, K.E., Bloch, M.H., 2016. Family accommodation in obsessivecompulsive and anxiety disorders: a five-year update. Exp. Rev. Neurother. 16 (1), 45. Lewin, A.B., Peris, T.S., De Nadai, A.S., McCracken, J.T., Piacentini, J., 2012. Agreement between therapists, parents, patients, and independent evaluators on clinical improvement in pediatric obsessive-compulsive disorder. J. Consult. Clin. Psychol. 6, 1103. Lewin, A.B., Hanks, C., Parker-Athill, E.C., Holbrook, J.R., Bitsko, R.H., Mink, J.W., Storch, E.A., 2014. Utility of the diagnostic interview schedule for children for assessing tourette syndrome in children. J. Child Adolesc. Psychopharmacol. 5, 275. Mannuzza, S., Fyer, A.J., Martin, L.Y., Gallops, M.S., Endicott, J., Gorman, J., Liebowitz, M.R., Klein, D.F., 1989. Reliability of anxiety assessment: diagnostic agreement. Arch. Gen. Psychiatry 46 (12), 1093–1101. Miller, L.D., Martinez, Y.J., Shumka, E., Baker, H., 2014. Multiple informant agreement of child, parent, and teacher ratings within community samples. Can. J. Psychiatry/La Rev. Can. De. Psychiatr. 59 (1), 34–39. Nauta, M.H., Scholing, A., Rapee, R.M., Abbott, M., Spence, S.H., Waters, A., 2004. A

parent-report measure of children's anxiety: Psychometric properties and comparison with child-report in a clinic and normal sample. Behav. Res. Ther. 42, 813–839. Safford, S.M., Kendall, P.C., Flannery-Schroeder, E., Webb, A., Sommer, H., 2005. A longitudinal look at parent-child diagnostic agreement in youth treated for anxiety disorders. J. Clin. Child Adolesc. Psychol. 34 (4), 747–757. Silverman, W.K., Albano, A.M., 1996. Anxiety Disorders Interview Schedule for DSM-IV: Parent Interview Schedule. Oxford University Press. Silverman, W.K., Saavedra, L.M., Pina, A.A., 2001. Articles: test-retest reliability of anxiety symptoms and diagnoses with the anxiety disorders interview schedule for DSM-IV: child and parent versions. J. Am. Acad. Child Adolesc. Psychiatry 40, 937–944. Smith, S.R., 2007. Making sense of multiple informants in child and adolescent psychopathology: a guide for clinicians. J. Psychoeduc. Assess. 25 (2), 139–149. Stevanovic, D., Jancic, J., Topalovic, M., Tadic, I., 2012. Agreement between children and parents when reporting anxiety and depressive symptoms in pediatric epilepsy. Epilepsy Behav. 25, 141–144. Stoddard, J., Stringaris, A., Brotman, M.A., Montville, D., Pine, D.S., Leibenluft, E., 2014. Irritability in child and adolescent anxiety disorders. Depression Anxiety 31 (7), 566–573. Storch, E.A., Salloum, A., King, M.A., Crawford, E.A., Andel, R., McBride, N.M., Lewin, A.B., 2015. A randomized controlled trial in community mental health centers of computer-assisted cognitive behavioral therapy versus treatment as usual for children with anxiety. Depression Anxiety 32 (11), 843 (1091–4269). Storch, E.A., Ehrenreich May, J., Wood, J.J., Jones, A.M., De Nadai, A.S., Lewin, A.B., Arnold, E.B., Murphy, T.K., 2012. Multiple informant agreement on the anxiety disorders interview schedule in youth with autism spectrum disorders. J. Child Adolesc. Psychopharmacol. 22 (4), 292–299. Storch, E.A., Wu, M.S., Small, B.J., Crawford, E.A., Lewin, A.B., Horng, B., Murphy, T.K., 2013. Mediators and moderators of functional impairment in adults with obsessive–compulsive disorder. Comprehensive Psychiatry. Thompson-Hollands, J., Kerns, C.E., Pincus, D.B., Comer, J.S., 2014. Parental accommodation of child anxiety and related symptoms: range, impact, and correlates. J. Anxiety Disord. 28, 765–773. Thompson-Hollands, J., Abramovitch, A., Tompson, M.C., Barlow, D.H., 2015. A randomized clinical trial of a brief family intervention to reduce accommodation in obsessive-compulsive disorder: a preliminary study. Behav. Ther. 46, 218–229. Ung, D., Arnold, E., De Nadai, A., Lewin, A., Phares, V., Murphy, T., Storch, E., 2014. Inter-rater reliability of the anxiety disorders interview schedule for dsm-iv in high-functioning youth with autism spectrum disorder. J. Dev. Phys. Disabil. 26 (1), 53–65. van de Looij-Jansen, P.M., Jansen, W., de Wilde, E.J., Donker, M.C.H., Verhulst, F.C., 2011. Discrepancies between parent-child reports of internalizing problems among preadolescent children: Relationships with gender, ethnic background, and future internalizing problems. J. Early Adolesc. 31 (3), 443–462. Wood, J.J., Piacentini, J.C., Bergman, R.L., McCracken, J., Barrios, V., 2002. Concurrent Validity of the Anxiety Disorders Section of the Anxiety Disorders Interview Schedule for DSM-IV: Child and Parent Versions. Routledge, p. 335.