- Email: [email protected]
Preliminary Validation of a Parent-Child Relational Framework for Teaching Developmental Assessment to Pediatric Residents
Preliminary Validation of a Parent-Child Relational Framework for Teaching Developmental Assessment to Pediatric Residents Michael Regalado, MD; Janet U. Schneiderman, RN, PhD; Lei Duan, PhD; Gisele Ragusa, PhD From the Division of Developmental Behavioral Pediatrics, Department of Pediatrics, Keck School of Medicine (Dr Regalado), Suzanne Dworak-Peck School of Social Work, Children, Youth and Families Department, Nursing Department (Dr Schneiderman), Hamovitch Center for Science in the Human Services, USC Suzanne Dworak-Peck School of Social Work (Dr Duan), and Division of Engineering Education, Viterbi School of Engineering (Dr Ragusa), University of Southern California, Los Angeles, Calif The authors have no conflicts of interest to disclose. Address correspondence to Michael Regalado, MD, LAC USC Medical Center, Department of Pediatrics, 1200 N State St, IRD 126, Los Angeles, CA 90033 (e-mail: [email protected]). Received for publication January 14, 2016; accepted September 12, 2016.
ABSTRACT OBJECTIVE: A parent–child relational framework was used as a method to train pediatric residents in basic knowledge and observation skills for the assessment of child development. Components of the training framework and its preliminary validation as an alternative to milestone-based approaches are described. METHODS: Pediatric residents were trained during a 4-week clinical rotation to use a semistructured interview and observe parent–child behavior during health visits using clinical criteria for historical information and observed behavior that reflect developmental change in the parent–child relationship. Clinical impressions of concern versus no concern for developmental delay were derived from parent–child relational criteria and the physical examination. A chart review yielded 330 preterm infants evaluated using this methodology at 4 and 15 months corrected age who also had standardized developmental testing at 6 and 18 months corrected age. Sensitivities and specificities were computed to examine the validity of the clinical assess-
ment compared with standardized testing. A subset of residents who completed 50 or more assessments during the rotation was timed at the end of 4 weeks. RESULTS: Parent–child behavioral markers elicited from the history and/or observed during the health visit correlated highly with standardized developmental assessment. Sensitivities and specificities were 0.72/0.98 and 0.87/0.96 at 4 to 6 and 15 to 18 months, respectively. Residents completed their assessments <1 minute on average if they had completed at least 50 supervised assessments. CONCLUSIONS: A parent–child relational framework is a potentially efficient and effective approach to training residents in the clinical knowledge and skills of child development assessment.
WHAT’S NEW
health visits and whenever surveillance raises a concern.4 Developmental surveillance includes addressing parental concerns, maintaining a developmental history, making accurate and informed observations of the child, identifying risk and protective factors, and documenting the process and findings.4 Clinical assessment of child developmental progress using history and observation is a central component of surveillance for which most clinicians ask parents about their concerns and use milestone-based checklists.5 However, evidence suggests that clinical impression using this approach is quite inaccurate.6–8 One approach to improving the accuracy of surveillance and screening has been to use validated tools for eliciting parental concerns, documenting developmental milestones, and examining psychosocial risk factors.9–11 Alternative approaches for training pediatricians to improve clinical knowledge and skills central to assessment of child development at health encounters are options to be explored. Parent–child observation is a recommended
KEYWORDS:
developmental assessment; developmental surveillance; medical education; parent–child relationship
ACADEMIC PEDIATRICS 2017;17:74–78
Markers of developmental change of the parent–child relationship reflect commensurate change in a child’s development. The result is an alternative approach to training pediatricians in the assessment of child development at health visits offering potential advantages of efficiency and accuracy.
CLINICAL KNOWLEDGE AND skills of child development are considered essential to health supervision activities.1 This assumption, however, stands in the face of well documented discontent with current pediatric training to prepare primary care practitioners to manage developmental and behavioral concerns.2,3 This is most evident for the detection of developmental delay for which current guidelines recommend a protocol for developmental surveillance at each health visit with developmental screening at specified ACADEMIC PEDIATRICS Copyright ª 2016 by Academic Pediatric Association
74
Volume 17, Number 1 January–February 2017
ACADEMIC PEDIATRICS
PARENT–CHILD FRAMEWORK FOR TEACHING CHILD DEVELOPMENT ASSESSMENT
activity for the health encounter and has been suggested as a possible approach to the developmental assessment component of surveillance despite lacking evidence, parameters, and clinical criteria to guide interpretation.1,4 Nevertheless, there is ample rationale to explore this approach. Parent–child relational assessment has been used in early intervention and infant mental health.12,13 From this work, stage-salient features have been elaborated for the developmental tasks of this relationship that include attachment, autonomy/self-awareness, and self-control/ peer relationships (Table 1).14 Theoretically, behavioral criteria for these developmental tasks can be defined and sampled as an alternative to milestone-based approaches. For example, infant proximity seeking at 9 to 12 months reflects a cognitive appraisal (eg, of a stranger), an emotional reaction (fear), social communication (looking for the parent’s reaction), and motor activity (movement toward parent). Because a primary function of the parent–child relationship is support for competence in self-regulation, its relevance as a clinical framework extends to the promotion of optimal physical health, behavioral health, and learning.14,15 We describe a parent–child relational framework (PCRF) and present the results of a preliminary evaluation for training and clinical care. We hypothesize that parent– child relational behavior correlates with child maturation and propose that a PCRF provides a viable alternative for training pediatricians in basic knowledge and skill in clinical assessment of child development and behavior. To explore the validity of this approach, we examine the accuracy and speed of PCRF-based assessment of preterm infants by pediatric residents compared with developmental standardized testing.
METHODS SAMPLE Subjects were low birth weight preterm infants and their caregivers seen consecutively at the Los Angeles County
75
and University of Southern California Medical Center’s Premature Infant Clinic between September 1, 2009, and December 31, 2014. Infants were seen routinely by pediatric residents at 4, 9, 15, and 24 months, and they received standardized developmental testing at 6 and 18 months— all ages corrected for prematurity. A review of 456 medical records yielded a sample of 330 infants with pediatric resident assessment and standardized testing results allowing for comparisons between the 2 at 4 to 6 months and 15 to 18 months. The university institutional review board approved the study. DEFINITIONS OF PARENT–CHILD RELATIONAL PARAMETERS Historical and observation criteria of developmental change in the parent–child relationship were defined for each developmental task and criteria for posture control were defined for the physical examination (Table 1).14,16 For pediatric assessments at 4 months, social turn-taking was defined as infant smiling and vocalizing in response to the mother’s or the examiner’s social bid to the infant. For pediatric assessments at 15 months, exploration and experimentation were defined by mobility away from, but anchored to the parent, and behavior whose intent was to discover where things are or how things work. Experimentation was also credited by observing the child’s indiscriminate (trial and error) approach to placement of a shape into the puzzle board. Social referencing (ie, looking to the parent when separated), was emphasized as primary evidence for intention to communicate at 15 months.14 Residents’ training included 10 hours of lecture based on the PCRF and direct supervision of their assessments. RESIDENT ASSESSMENT PROCEDURE Pediatric residents used a semistructured interview organized by the recommended activities of developmental surveillance to elicit parent concerns, describe the infant’s current developmental competencies, and discuss the child’s psychosocial context by discussing parental adaptation and family resources. Interviews were structured using
Table 1. Developmental Tasks, Behavior, and Physical Examination Criteria Developmental Task (Age, Months) Physiological regulation (0–3)
Social reciprocity (3–6) Initiative (6–9)
Attachment (9–12)
Exploration and experimentation (12–18) Autonomy and self-awareness (18–30)
Relational Behavior Criteria
Circadian patterning of sleep and wake Predictable feeding rhythm Crying behavior peaks then declines Baby smiles/coos when talked to Examiner engages infant likewise Grasps objects, explores with senses Parent scaffolds, elaborates experience
Seeks proximity with stress Fear/wariness with stranger or separation Social referencing: looks to parent when uncertain Explores larger surround Trial and error approach to problems Self-assertion: resists help; says “no,” “mine” Self-awareness: points to self and others Shows a positive self-evaluation with praise, pouts, saddens with limit-setting Reflects before acting (with puzzle board)
Physical Examination Criteria Primitive reflexes Head control
Increasing trunk control Primitive reflexes decline Independent sitting Protective reflexes Radial digital grasp Mobility Pulling to stand Pincer grasp Independent walking Spoon use Running, jumping
76
REGALADO ET AL
4 basic open-ended questions: What concerns or questions do you have? What is your baby learning to do? How is your baby learning to communicate with you? How are you doing as a parent? Follow-up questions clarified features of the developmental task expected for the period. For example, if the parent did not describe social turntaking behavior in her 4-month-old, the resident asked, “How does your baby respond when you smile and talk to him?” Residents observed parent–child interaction throughout the interview. Developmental progress was determined in postencounter debriefings by reviewing the history, observations of behavior during the interview, and physical examination using a structured form. When criteria were met by either historical data or observations during the visit, the dyad was credited with displaying the expected relational behavior for that phase (“no concern”). Absence of criteria for developmental tasks was coded as “concern.” Residents had access to the medical record diagnosis list before conducting their assessments. Residents were timed in administration of the assessment during their final week of service. STANDARDIZED TESTING Standardized developmental testing used the Bayley Scales of Infant and Toddler Development, Third Edition.17 A developmental delay was defined by a composite score in any domain <70 (<2 SD), which is the local eligibility criterion for early intervention. The pediatric resident, supervising faculty, and clinical psychologist performing the developmental testing were unaware of each other’s results at the time the clinical assessments were entered into the record. DATA COLLECTION AND ANALYSIS Historical and observational data reported in the medical record at 4 and 15 months were abstracted using structured debriefing forms. Sensitivity, specificity, positive and negative predictive validity, and accuracy were calculated to compare clinical assessment at 4 months with developmental testing data at 6 months, and clinical assessment at 15 months with developmental testing data at 18 months, separately. Cohen k was calculated to determine the level of agreement between the clinical assessment and developmental testing with k agreement level of 0.41 to 0.60 considered as moderate, 0.61 to 0.80 as substantial, and >0.81 as almost perfect agreement.18 The exact 95% confidence intervals for those indexes were calculated using standard methods for proportions.19 Stratified analyses were conducted to determine if clinical assessment varied as a function of birth weight by comparing infants with birth weight <1000 g versus >1000 g.
RESULTS SAMPLE CHARACTERISTICS Infants were from low-income families (100% Medicaid-eligible) and mostly of Hispanic ethnicity
ACADEMIC PEDIATRICS Table 2. Sample Characteristics (N ¼ 330) Characteristic
Value
Child race/ethnicity, n (%) African American Asian Caucasian Hispanic Child gender, male, n (%) Mean maternal age at time of delivery (SD), y Mean birth weight (SD), g Mean Bayley-III composite score (SD) 6 Mo Cognitive Language Motor Socioemotional 18 Mo Cognitive Language Motor Socioemotional
24 (9.1) 18 (6.8) 8 (3.0) 215 (81.1) 140 (52.8) 28.4 (6.7) 10.7 (3.5)
96.1 (10.0) 90.5 (7.5) 95.1 (17.5) 97.0 (15.5) 90.0 (10.7) 82.0 (9.2) 94.0 (10.1) 96.2 (18.1)
Bayley-III indicates Bayley Scales of Infant and Toddler Development, Third Edition.
(83.9%) with a slight majority of male infants (52.4%). Infant gestational ages ranged from 23 to 37 weeks and birth weights ranged from 270 to 1660 grams. The average level of maternal education was <11 years (Table 2). DEVELOPMENTAL TESTING AND RESIDENT ASSESSMENTS The average composite scores for Bayley Scales of Infant and Toddler Development, Third Edition testing at 6 and 18 months of age are summarized in Table 2. Assessment and standardized testing results were available for 257 infants at 4 and 6 months and for 205 infants at 15 and 18 months. The data were analyzed as independent samples because concerns and symptoms change over time. Seven percent of the sample met criteria for developmental delay at both assessment times. Comparison of resident assessment with standardized testing yielded sensitivity, specificity, and accuracy of 0.72/0.98/0.96 at 4 to 6 months and 0.87/0.96/0.95 at 15 to 18 months (Table 3). Positive predictive values were 0.72 and 0.62
Table 3. Two by Two Table for Surveillance and Bayley-III Testing Bayley-III Testing DD
DD
Surveillance
Absent
Present
4 Mo No concern Concern
234 5
15 Mo No concern Concern
182 8
95% CI
6 Mo 5 13
Sensitivity Specificity Accuracy k
0.72 (0.47–0.90) 0.98 (0.95–0.99) 0.96 (0.93–0.98) 0.70 (0.53–0.88)
2 13
Sensitivity Specificity Accuracy k
0.87 (0.60–0.98) 0.96 (0.92–0.98) 0.95 (0.91–0.98) 0.70 (0.52–0.87)
18 Mo
Bayley-III indicates Bayley Scales of Infant and Toddler Development, Third Edition; CI, confidence interval; and DD, developmental delay.
ACADEMIC PEDIATRICS
PARENT–CHILD FRAMEWORK FOR TEACHING CHILD DEVELOPMENT ASSESSMENT
and negative predictive values were 0.98 and 0.99 at 4 to 6 months and 15 to 18 months, respectively. The k statistics showed substantial agreement between the 2 tests at both times (k ¼ 0.7).18 Sensitivity scores for infants with birth weights <1000 g were higher than those with birth weights $1000 g, at 4 to 6 months (0.91 vs 0.43) and 15 to 18 months (0.91 vs 0.75), whereas specificity scores were approximately the same (0.96 vs 0.99 and 0.94 vs 0.97; data not shown). Assessments were completed by 63 residents. Twelve residents (19%) who performed at least 50 assessments completed them in <1 minute on average during their final week on service.
DISCUSSION We explored the validity of a PCRF as an alternative approach to teaching pediatric residents the knowledge and clinical skills of child development by comparing their assessments of the parent–child relationship with a criterion of standardized developmental testing. Our findings suggest that clinical assessment of the parent–child relationship correlates highly with standardized measures of child development achieving sensitivities of 72% and 87% and specificities between 96% and 98%. These results are much higher than those from previous reports for clinical impression (32%–33% and 82%–95%, respectively) and fall within the range of accuracy recommended for developmental screening tests.4,6–8 The results also compare favorably with studies that evaluated the accuracy of validated parent-completed questionnaires with low-risk, and high-risk samples, as well as an expert-generated milestone checklist.10,20,21 Sensitivity varied as a function of birth weight and age similar to a previous study of a parent-completed questionnaire with extremely preterm infants.21 The study also shows that training of pediatric residents in developmental assessment to a competency level relevant to the demands of clinical practice is achievable. There were substantial time costs of direct faculty supervision; however, 10 hours of didactic training is comparable with other efforts for training residents in the administration of screening tests.22 The training has been well received and highly rated with comments appreciating a deeper understanding of children and families than can be conveyed by a milestone-based approach. The use of a convenience sample limits the generalizability of this study. The sample also represents an atypical group because expectations were for a higher percentage of developmental delays with low birth weight infants.23 An effect of birth weight on sensitivity was evident, which might be related to other limitations including expectation bias because of limited access to the patients’ medical records, or to the interval between resident assessment and standardized testing that could account for the discrepancy in sensitivities between 4 and 6 months and 15 to 18 months. This discrepancy might point to the effect of highly variable motor development in preterm infants or to an inherent lack of stability within individuals and across domains of development over time.24,25 Other
77
methodological limitations to address in future work include examinations of reliability and serial assessment on the accuracy of surveillance. In summary, this preliminary evaluation of a PCRF shows its potential for effective training of pediatric residents in basic knowledge of child development and observation skills of parent–child relational behavior. These clinical skills can be used in developmental surveillance to document the developmental history and observe behavior in combination with assessments of parents’ concerns and psychosocial risk and protective factors. The accuracy and speed of assessment observed in this study support its potential as a clinical approach and provides a foundation for a more rigorous validation of the methodology. Future studies of the PCRF should explore teaching strategies that are time-efficient and mindful of faculty constraints, as well as examine the use of the PCRF in the general population and within the complete protocol for developmental surveillance and screening.
ACKNOWLEDGMENTS Authorship Statement: M.R. conceptualized the original idea for this report, drafted the initial manuscript, and approved the final manuscript as submitted. J.U.S. contributed to data collection and analysis, and edited the manuscript. L.D. performed the statistical analyses, drafted the results and tables, and edited the manuscript. G.R. assisted with the initial conceptual idea for the study, data analyses, and editing of the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
REFERENCES 1. Hagan JF, Shaw JS, Duncan P, eds. Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents. 3rd ed. Elk Grove Village, Ill: American Academy of Pediatrics; 2008. 2. Boreman CD, Thomasgard MC, Fernandez SA, et al. Residency training in developmental behavioral pediatrics: where do we stand? Clin Pediatr. 2007;46:135–145. 3. Rosenberg AA, Kamin C, Glicken AD, et al. Training gaps for pediatric residents planning a career in primary care: a qualitative and quantitative study. J Grad Med Educ. 2011;3:309–314. 4. American Academy of Pediatrics, Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening. Pediatrics. 2006;118:405–420. 5. Sices L, Feudtner C, McLaughlin J, et al. How do primary care physicians identify young children with developmental delays? A National Survey. J Dev Behav Pediatr. 2003;24:409–417. 6. Hix-Small H, Marks K, Squires J, et al. Impact of developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120:381–389. 7. King TM, Rosenberg LA, Fuddy L, et al. Prevalence and early identification of language delays among at risk three year olds. J Dev Behav Pediatr. 2005;26:293–303. 8. Sheldrick RC, Merchant S, Perrin EC. Identification of developmental-behavioral problems in primary care: a systematic review. Pediatrics. 2011;128:356–363. 9. Glascoe FP. Can clinical judgment detect children with speechlanguage problems? Pediatrics. 1991;87:317–322. 10. Sheldrick RC, Perrin EC. Evidence-based milestones for surveillance of cognitive, language, and motor development. Acad Pediatr. 2013; 13:577–586.
78
REGALADO ET AL
11. Kemper KJ, Osborn LM, Hansen DF, et al. Family psychosocial screening: should we focus on high-risk settings? J Dev Behav Pediatr. 1994;15:336–341. 12. Kelly JF, Barnard KE. Assessment of parent-child interaction. Implications for early intervention. In: Shonkoff JP, Meisels SJ, eds. Handbook of Early Childhood Intervention. 2nd ed. Cambridge, UK: Cambridge University Press; 2000:258–289. 13. Zeanah CH, ed. Handbook of Infant Mental Health. 3rd ed. New York, NY: The Guilford Press; 2009. 14. Sroufe LA. Emotional Development: The Organization of Emotional Life in the Early Years. Cambridge: Cambridge University Press; 1995. 15. Tinsley BJ, Markey CN, Ericksen AJ, et al. Health promotion for parents. In: Bornstein MH, ed. Handbook of Parenting: Practical Issues in Parenting. Vol 5. Mahwah, NJ: Lawrence Erlbaum Assoc; 2002: 311–328. 16. Amiel-Tison C. Neurological Assessment During the First Year of Life. Oxford, UK: Oxford University Press; 1986. 17. Bayley N. Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III). San Antonio, TX: Psychological Corporation; 2005. 18. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.
ACADEMIC PEDIATRICS 19. Gardner MJ, Altman DG. Calculating confidence intervals for proportions and their differences. In: Gardner MJ, Altman DG, eds. Statistics With Confidence. London: BMJ Publishing Group; 1989:28–33. 20. Limbos MM, Joyce DP. Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. J Dev Behav Pediatr. 2011;32:499–511. 21. Schonhaut L, Armijo I, Schonstedt M, et al. Validity of the Ages and Stages Questionnaires in term and preterm infants. Pediatrics. 2013; 131:e1468–e1474. 22. Bauer SC, Smith PJ, Chien AT, et al. Educating pediatric residents about developmental and social-emotional health. Infant Young Child. 2009;22:309–320. 23. Vohr BR, Wright LL, Dusick AM, et al. Neurodevelopmental and functional outcomes of extremely low birth weight infants in the National Institute of Child Health and Human Development Neonatal Research Network, 1993-1994. Pediatrics. 2000;105:1216–1226. 24. Darrah J, Hodge M, Magill-Evans J, et al. Stability of serial assessments of motor and communication abilities in typically developing infants – implications for screening. Early Hum Dev. 2003;72: 97–110. 25. Rydz D, Srour M, Oksoui M, et al. Screening for developmental delay in the setting of a community pediatric clinic: a prospective assessment of parent-report questionnaires. Pediatrics. 2006;118: e1178–e1186.
ABSTRACT OBJECTIVE: A parent–child relational framework was used as a method to train pediatric residents in basic knowledge and observation skills for the assessment of child development. Components of the training framework and its preliminary validation as an alternative to milestone-based approaches are described. METHODS: Pediatric residents were trained during a 4-week clinical rotation to use a semistructured interview and observe parent–child behavior during health visits using clinical criteria for historical information and observed behavior that reflect developmental change in the parent–child relationship. Clinical impressions of concern versus no concern for developmental delay were derived from parent–child relational criteria and the physical examination. A chart review yielded 330 preterm infants evaluated using this methodology at 4 and 15 months corrected age who also had standardized developmental testing at 6 and 18 months corrected age. Sensitivities and specificities were computed to examine the validity of the clinical assess-
ment compared with standardized testing. A subset of residents who completed 50 or more assessments during the rotation was timed at the end of 4 weeks. RESULTS: Parent–child behavioral markers elicited from the history and/or observed during the health visit correlated highly with standardized developmental assessment. Sensitivities and specificities were 0.72/0.98 and 0.87/0.96 at 4 to 6 and 15 to 18 months, respectively. Residents completed their assessments <1 minute on average if they had completed at least 50 supervised assessments. CONCLUSIONS: A parent–child relational framework is a potentially efficient and effective approach to training residents in the clinical knowledge and skills of child development assessment.
WHAT’S NEW
health visits and whenever surveillance raises a concern.4 Developmental surveillance includes addressing parental concerns, maintaining a developmental history, making accurate and informed observations of the child, identifying risk and protective factors, and documenting the process and findings.4 Clinical assessment of child developmental progress using history and observation is a central component of surveillance for which most clinicians ask parents about their concerns and use milestone-based checklists.5 However, evidence suggests that clinical impression using this approach is quite inaccurate.6–8 One approach to improving the accuracy of surveillance and screening has been to use validated tools for eliciting parental concerns, documenting developmental milestones, and examining psychosocial risk factors.9–11 Alternative approaches for training pediatricians to improve clinical knowledge and skills central to assessment of child development at health encounters are options to be explored. Parent–child observation is a recommended
KEYWORDS:
developmental assessment; developmental surveillance; medical education; parent–child relationship
ACADEMIC PEDIATRICS 2017;17:74–78
Markers of developmental change of the parent–child relationship reflect commensurate change in a child’s development. The result is an alternative approach to training pediatricians in the assessment of child development at health visits offering potential advantages of efficiency and accuracy.
CLINICAL KNOWLEDGE AND skills of child development are considered essential to health supervision activities.1 This assumption, however, stands in the face of well documented discontent with current pediatric training to prepare primary care practitioners to manage developmental and behavioral concerns.2,3 This is most evident for the detection of developmental delay for which current guidelines recommend a protocol for developmental surveillance at each health visit with developmental screening at specified ACADEMIC PEDIATRICS Copyright ª 2016 by Academic Pediatric Association
74
Volume 17, Number 1 January–February 2017
ACADEMIC PEDIATRICS
PARENT–CHILD FRAMEWORK FOR TEACHING CHILD DEVELOPMENT ASSESSMENT
activity for the health encounter and has been suggested as a possible approach to the developmental assessment component of surveillance despite lacking evidence, parameters, and clinical criteria to guide interpretation.1,4 Nevertheless, there is ample rationale to explore this approach. Parent–child relational assessment has been used in early intervention and infant mental health.12,13 From this work, stage-salient features have been elaborated for the developmental tasks of this relationship that include attachment, autonomy/self-awareness, and self-control/ peer relationships (Table 1).14 Theoretically, behavioral criteria for these developmental tasks can be defined and sampled as an alternative to milestone-based approaches. For example, infant proximity seeking at 9 to 12 months reflects a cognitive appraisal (eg, of a stranger), an emotional reaction (fear), social communication (looking for the parent’s reaction), and motor activity (movement toward parent). Because a primary function of the parent–child relationship is support for competence in self-regulation, its relevance as a clinical framework extends to the promotion of optimal physical health, behavioral health, and learning.14,15 We describe a parent–child relational framework (PCRF) and present the results of a preliminary evaluation for training and clinical care. We hypothesize that parent– child relational behavior correlates with child maturation and propose that a PCRF provides a viable alternative for training pediatricians in basic knowledge and skill in clinical assessment of child development and behavior. To explore the validity of this approach, we examine the accuracy and speed of PCRF-based assessment of preterm infants by pediatric residents compared with developmental standardized testing.
METHODS SAMPLE Subjects were low birth weight preterm infants and their caregivers seen consecutively at the Los Angeles County
75
and University of Southern California Medical Center’s Premature Infant Clinic between September 1, 2009, and December 31, 2014. Infants were seen routinely by pediatric residents at 4, 9, 15, and 24 months, and they received standardized developmental testing at 6 and 18 months— all ages corrected for prematurity. A review of 456 medical records yielded a sample of 330 infants with pediatric resident assessment and standardized testing results allowing for comparisons between the 2 at 4 to 6 months and 15 to 18 months. The university institutional review board approved the study. DEFINITIONS OF PARENT–CHILD RELATIONAL PARAMETERS Historical and observation criteria of developmental change in the parent–child relationship were defined for each developmental task and criteria for posture control were defined for the physical examination (Table 1).14,16 For pediatric assessments at 4 months, social turn-taking was defined as infant smiling and vocalizing in response to the mother’s or the examiner’s social bid to the infant. For pediatric assessments at 15 months, exploration and experimentation were defined by mobility away from, but anchored to the parent, and behavior whose intent was to discover where things are or how things work. Experimentation was also credited by observing the child’s indiscriminate (trial and error) approach to placement of a shape into the puzzle board. Social referencing (ie, looking to the parent when separated), was emphasized as primary evidence for intention to communicate at 15 months.14 Residents’ training included 10 hours of lecture based on the PCRF and direct supervision of their assessments. RESIDENT ASSESSMENT PROCEDURE Pediatric residents used a semistructured interview organized by the recommended activities of developmental surveillance to elicit parent concerns, describe the infant’s current developmental competencies, and discuss the child’s psychosocial context by discussing parental adaptation and family resources. Interviews were structured using
Table 1. Developmental Tasks, Behavior, and Physical Examination Criteria Developmental Task (Age, Months) Physiological regulation (0–3)
Social reciprocity (3–6) Initiative (6–9)
Attachment (9–12)
Exploration and experimentation (12–18) Autonomy and self-awareness (18–30)
Relational Behavior Criteria
Circadian patterning of sleep and wake Predictable feeding rhythm Crying behavior peaks then declines Baby smiles/coos when talked to Examiner engages infant likewise Grasps objects, explores with senses Parent scaffolds, elaborates experience
Seeks proximity with stress Fear/wariness with stranger or separation Social referencing: looks to parent when uncertain Explores larger surround Trial and error approach to problems Self-assertion: resists help; says “no,” “mine” Self-awareness: points to self and others Shows a positive self-evaluation with praise, pouts, saddens with limit-setting Reflects before acting (with puzzle board)
Physical Examination Criteria Primitive reflexes Head control
Increasing trunk control Primitive reflexes decline Independent sitting Protective reflexes Radial digital grasp Mobility Pulling to stand Pincer grasp Independent walking Spoon use Running, jumping
76
REGALADO ET AL
4 basic open-ended questions: What concerns or questions do you have? What is your baby learning to do? How is your baby learning to communicate with you? How are you doing as a parent? Follow-up questions clarified features of the developmental task expected for the period. For example, if the parent did not describe social turntaking behavior in her 4-month-old, the resident asked, “How does your baby respond when you smile and talk to him?” Residents observed parent–child interaction throughout the interview. Developmental progress was determined in postencounter debriefings by reviewing the history, observations of behavior during the interview, and physical examination using a structured form. When criteria were met by either historical data or observations during the visit, the dyad was credited with displaying the expected relational behavior for that phase (“no concern”). Absence of criteria for developmental tasks was coded as “concern.” Residents had access to the medical record diagnosis list before conducting their assessments. Residents were timed in administration of the assessment during their final week of service. STANDARDIZED TESTING Standardized developmental testing used the Bayley Scales of Infant and Toddler Development, Third Edition.17 A developmental delay was defined by a composite score in any domain <70 (<2 SD), which is the local eligibility criterion for early intervention. The pediatric resident, supervising faculty, and clinical psychologist performing the developmental testing were unaware of each other’s results at the time the clinical assessments were entered into the record. DATA COLLECTION AND ANALYSIS Historical and observational data reported in the medical record at 4 and 15 months were abstracted using structured debriefing forms. Sensitivity, specificity, positive and negative predictive validity, and accuracy were calculated to compare clinical assessment at 4 months with developmental testing data at 6 months, and clinical assessment at 15 months with developmental testing data at 18 months, separately. Cohen k was calculated to determine the level of agreement between the clinical assessment and developmental testing with k agreement level of 0.41 to 0.60 considered as moderate, 0.61 to 0.80 as substantial, and >0.81 as almost perfect agreement.18 The exact 95% confidence intervals for those indexes were calculated using standard methods for proportions.19 Stratified analyses were conducted to determine if clinical assessment varied as a function of birth weight by comparing infants with birth weight <1000 g versus >1000 g.
RESULTS SAMPLE CHARACTERISTICS Infants were from low-income families (100% Medicaid-eligible) and mostly of Hispanic ethnicity
ACADEMIC PEDIATRICS Table 2. Sample Characteristics (N ¼ 330) Characteristic
Value
Child race/ethnicity, n (%) African American Asian Caucasian Hispanic Child gender, male, n (%) Mean maternal age at time of delivery (SD), y Mean birth weight (SD), g Mean Bayley-III composite score (SD) 6 Mo Cognitive Language Motor Socioemotional 18 Mo Cognitive Language Motor Socioemotional
24 (9.1) 18 (6.8) 8 (3.0) 215 (81.1) 140 (52.8) 28.4 (6.7) 10.7 (3.5)
96.1 (10.0) 90.5 (7.5) 95.1 (17.5) 97.0 (15.5) 90.0 (10.7) 82.0 (9.2) 94.0 (10.1) 96.2 (18.1)
Bayley-III indicates Bayley Scales of Infant and Toddler Development, Third Edition.
(83.9%) with a slight majority of male infants (52.4%). Infant gestational ages ranged from 23 to 37 weeks and birth weights ranged from 270 to 1660 grams. The average level of maternal education was <11 years (Table 2). DEVELOPMENTAL TESTING AND RESIDENT ASSESSMENTS The average composite scores for Bayley Scales of Infant and Toddler Development, Third Edition testing at 6 and 18 months of age are summarized in Table 2. Assessment and standardized testing results were available for 257 infants at 4 and 6 months and for 205 infants at 15 and 18 months. The data were analyzed as independent samples because concerns and symptoms change over time. Seven percent of the sample met criteria for developmental delay at both assessment times. Comparison of resident assessment with standardized testing yielded sensitivity, specificity, and accuracy of 0.72/0.98/0.96 at 4 to 6 months and 0.87/0.96/0.95 at 15 to 18 months (Table 3). Positive predictive values were 0.72 and 0.62
Table 3. Two by Two Table for Surveillance and Bayley-III Testing Bayley-III Testing DD
DD
Surveillance
Absent
Present
4 Mo No concern Concern
234 5
15 Mo No concern Concern
182 8
95% CI
6 Mo 5 13
Sensitivity Specificity Accuracy k
0.72 (0.47–0.90) 0.98 (0.95–0.99) 0.96 (0.93–0.98) 0.70 (0.53–0.88)
2 13
Sensitivity Specificity Accuracy k
0.87 (0.60–0.98) 0.96 (0.92–0.98) 0.95 (0.91–0.98) 0.70 (0.52–0.87)
18 Mo
Bayley-III indicates Bayley Scales of Infant and Toddler Development, Third Edition; CI, confidence interval; and DD, developmental delay.
ACADEMIC PEDIATRICS
PARENT–CHILD FRAMEWORK FOR TEACHING CHILD DEVELOPMENT ASSESSMENT
and negative predictive values were 0.98 and 0.99 at 4 to 6 months and 15 to 18 months, respectively. The k statistics showed substantial agreement between the 2 tests at both times (k ¼ 0.7).18 Sensitivity scores for infants with birth weights <1000 g were higher than those with birth weights $1000 g, at 4 to 6 months (0.91 vs 0.43) and 15 to 18 months (0.91 vs 0.75), whereas specificity scores were approximately the same (0.96 vs 0.99 and 0.94 vs 0.97; data not shown). Assessments were completed by 63 residents. Twelve residents (19%) who performed at least 50 assessments completed them in <1 minute on average during their final week on service.
DISCUSSION We explored the validity of a PCRF as an alternative approach to teaching pediatric residents the knowledge and clinical skills of child development by comparing their assessments of the parent–child relationship with a criterion of standardized developmental testing. Our findings suggest that clinical assessment of the parent–child relationship correlates highly with standardized measures of child development achieving sensitivities of 72% and 87% and specificities between 96% and 98%. These results are much higher than those from previous reports for clinical impression (32%–33% and 82%–95%, respectively) and fall within the range of accuracy recommended for developmental screening tests.4,6–8 The results also compare favorably with studies that evaluated the accuracy of validated parent-completed questionnaires with low-risk, and high-risk samples, as well as an expert-generated milestone checklist.10,20,21 Sensitivity varied as a function of birth weight and age similar to a previous study of a parent-completed questionnaire with extremely preterm infants.21 The study also shows that training of pediatric residents in developmental assessment to a competency level relevant to the demands of clinical practice is achievable. There were substantial time costs of direct faculty supervision; however, 10 hours of didactic training is comparable with other efforts for training residents in the administration of screening tests.22 The training has been well received and highly rated with comments appreciating a deeper understanding of children and families than can be conveyed by a milestone-based approach. The use of a convenience sample limits the generalizability of this study. The sample also represents an atypical group because expectations were for a higher percentage of developmental delays with low birth weight infants.23 An effect of birth weight on sensitivity was evident, which might be related to other limitations including expectation bias because of limited access to the patients’ medical records, or to the interval between resident assessment and standardized testing that could account for the discrepancy in sensitivities between 4 and 6 months and 15 to 18 months. This discrepancy might point to the effect of highly variable motor development in preterm infants or to an inherent lack of stability within individuals and across domains of development over time.24,25 Other
77
methodological limitations to address in future work include examinations of reliability and serial assessment on the accuracy of surveillance. In summary, this preliminary evaluation of a PCRF shows its potential for effective training of pediatric residents in basic knowledge of child development and observation skills of parent–child relational behavior. These clinical skills can be used in developmental surveillance to document the developmental history and observe behavior in combination with assessments of parents’ concerns and psychosocial risk and protective factors. The accuracy and speed of assessment observed in this study support its potential as a clinical approach and provides a foundation for a more rigorous validation of the methodology. Future studies of the PCRF should explore teaching strategies that are time-efficient and mindful of faculty constraints, as well as examine the use of the PCRF in the general population and within the complete protocol for developmental surveillance and screening.
ACKNOWLEDGMENTS Authorship Statement: M.R. conceptualized the original idea for this report, drafted the initial manuscript, and approved the final manuscript as submitted. J.U.S. contributed to data collection and analysis, and edited the manuscript. L.D. performed the statistical analyses, drafted the results and tables, and edited the manuscript. G.R. assisted with the initial conceptual idea for the study, data analyses, and editing of the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
REFERENCES 1. Hagan JF, Shaw JS, Duncan P, eds. Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents. 3rd ed. Elk Grove Village, Ill: American Academy of Pediatrics; 2008. 2. Boreman CD, Thomasgard MC, Fernandez SA, et al. Residency training in developmental behavioral pediatrics: where do we stand? Clin Pediatr. 2007;46:135–145. 3. Rosenberg AA, Kamin C, Glicken AD, et al. Training gaps for pediatric residents planning a career in primary care: a qualitative and quantitative study. J Grad Med Educ. 2011;3:309–314. 4. American Academy of Pediatrics, Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening. Pediatrics. 2006;118:405–420. 5. Sices L, Feudtner C, McLaughlin J, et al. How do primary care physicians identify young children with developmental delays? A National Survey. J Dev Behav Pediatr. 2003;24:409–417. 6. Hix-Small H, Marks K, Squires J, et al. Impact of developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120:381–389. 7. King TM, Rosenberg LA, Fuddy L, et al. Prevalence and early identification of language delays among at risk three year olds. J Dev Behav Pediatr. 2005;26:293–303. 8. Sheldrick RC, Merchant S, Perrin EC. Identification of developmental-behavioral problems in primary care: a systematic review. Pediatrics. 2011;128:356–363. 9. Glascoe FP. Can clinical judgment detect children with speechlanguage problems? Pediatrics. 1991;87:317–322. 10. Sheldrick RC, Perrin EC. Evidence-based milestones for surveillance of cognitive, language, and motor development. Acad Pediatr. 2013; 13:577–586.
78
REGALADO ET AL
11. Kemper KJ, Osborn LM, Hansen DF, et al. Family psychosocial screening: should we focus on high-risk settings? J Dev Behav Pediatr. 1994;15:336–341. 12. Kelly JF, Barnard KE. Assessment of parent-child interaction. Implications for early intervention. In: Shonkoff JP, Meisels SJ, eds. Handbook of Early Childhood Intervention. 2nd ed. Cambridge, UK: Cambridge University Press; 2000:258–289. 13. Zeanah CH, ed. Handbook of Infant Mental Health. 3rd ed. New York, NY: The Guilford Press; 2009. 14. Sroufe LA. Emotional Development: The Organization of Emotional Life in the Early Years. Cambridge: Cambridge University Press; 1995. 15. Tinsley BJ, Markey CN, Ericksen AJ, et al. Health promotion for parents. In: Bornstein MH, ed. Handbook of Parenting: Practical Issues in Parenting. Vol 5. Mahwah, NJ: Lawrence Erlbaum Assoc; 2002: 311–328. 16. Amiel-Tison C. Neurological Assessment During the First Year of Life. Oxford, UK: Oxford University Press; 1986. 17. Bayley N. Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III). San Antonio, TX: Psychological Corporation; 2005. 18. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.
ACADEMIC PEDIATRICS 19. Gardner MJ, Altman DG. Calculating confidence intervals for proportions and their differences. In: Gardner MJ, Altman DG, eds. Statistics With Confidence. London: BMJ Publishing Group; 1989:28–33. 20. Limbos MM, Joyce DP. Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. J Dev Behav Pediatr. 2011;32:499–511. 21. Schonhaut L, Armijo I, Schonstedt M, et al. Validity of the Ages and Stages Questionnaires in term and preterm infants. Pediatrics. 2013; 131:e1468–e1474. 22. Bauer SC, Smith PJ, Chien AT, et al. Educating pediatric residents about developmental and social-emotional health. Infant Young Child. 2009;22:309–320. 23. Vohr BR, Wright LL, Dusick AM, et al. Neurodevelopmental and functional outcomes of extremely low birth weight infants in the National Institute of Child Health and Human Development Neonatal Research Network, 1993-1994. Pediatrics. 2000;105:1216–1226. 24. Darrah J, Hodge M, Magill-Evans J, et al. Stability of serial assessments of motor and communication abilities in typically developing infants – implications for screening. Early Hum Dev. 2003;72: 97–110. 25. Rydz D, Srour M, Oksoui M, et al. Screening for developmental delay in the setting of a community pediatric clinic: a prospective assessment of parent-report questionnaires. Pediatrics. 2006;118: e1178–e1186.