Social perspective-taking performance: Construct, measurement, and relations with academic performance and engagement

Journal of Applied Developmental Psychology 57 (2018) 24–41

Contents lists available at ScienceDirect

Journal of Applied Developmental Psychology journal homepage: www.elsevier.com/locate/jappdp

Social perspective-taking performance: Construct, measurement, and relations with academic performance and engagement

T

⁎

Ha Yeon Kima, , Maria D. LaRussoa,b, Lisa B. Hsina, Allen G. Harbaughc, Robert L. Selmana, Catherine E. Snowa,b a

Harvard University, USA Strategic Education Research Partnership (SERP), USA c Boston University, USA b

A R T I C LE I N FO

A B S T R A C T

Keywords: Social perspective-taking Social emotional competence Academic engagement Academic performance Categorical confirmatory factor analysis

Social perspective-taking (SPT), the social-cognitive skill of inferring, considering, and evaluating others' perspectives, is critical in allowing students to engage not only with others in social interactions, but also with many academic materials and activities. However, due to complex conceptualizations of SPT, as well as challenges to its measurement, little research has examined the role of SPT in student learning. Using a new measure, the Assessment of Social Perspective-taking Performance (ASPP), this article presents evidence that SPT can be conceptualized as a set of measurable developmental performative skills relevant to classroom learning in early adolescence: articulation of actors' beliefs/intentions and positioning of actors in their social contexts. Examination of its psychometric properties provides evidence that ASPP can precisely assess students' SPT performance across varying levels of those skills. Lastly, we find positive associations of SPT with academic engagement and standardized test scores.

Introduction Theoretical and empirical evidence from both experimental and correlational studies consistently suggest that social-emotional competence can promote students' school success by improving the attitudes and behaviors that foster commitment to academics and school performance (Birch & Ladd, 1997; Denham et al., 2012; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Ladd, Birch, & Buhs, 1999; Pianta, Steinberg, & Rollins, 1995; Rhoades, Warren, Domitrovich, & Greenberg, 2011; Zins, Bloodworth, Weissberg, & Walberg, 2007). One aspect of social-emotional competence, social perspective-taking skill (SPT: Gehlbach, 2004a, 2004b; Selman, 1980, 2007), may be especially relevant to academic performance, as it encompasses the social-cognitive ability to infer, consider, and critically evaluate the points of view of authors of academic texts or of characters in literature, and is therefore a plausible bridge to engagement with those texts (Duhaylongsod, Snow, Selman, & Donovan, 2015). However, highly variable conceptualizations scattered across different strands of research have limited researchers' abilities both to measure SPT performance as it matures throughout the school years, and consequently, to develop assessment tools that could justifiably be used to assess efforts

to foster SPT in school through curricular interventions. This article presents one way of moving beyond these limitations, by developing a conceptualization of SPT that is especially relevant to educational contexts. Building on Austin's (1962) notion of speech acts, the current paper conceptualizes SPT acts (Diazgranados, Selman, & Dionne, 2016) as having two facets: articulation of the content of social actors' communicative acts, and positioning those acts as products of the social actors' experiences. In this article we explore how a measure based on this conceptualization can capture children's SPT performance across social contexts and situations in schools, and examine the relationship between SPT acts and academic engagement as well as performance in standardized tests. Conceptualizing SPT While different versions of SPT have been explored across research fields pertaining to child development, there is disagreement within the literature on how best to characterize this construct or to measure it in academic contexts (Diazgranados et al., 2016). Current understandings of SPT during middle childhood and adolescence emerge from research on social-emotional development (e.g., Webster-Stratton & Reid, 2004),

⁎ Corresponding author at: Global TIES for Children, Steinhardt School of Culture, Education, and Human Development, New York University, 246 Greene Street, New York 10003, USA. E-mail address: [email protected] (H.Y. Kim).

https://doi.org/10.1016/j.appdev.2018.05.005 Received 12 March 2017; Received in revised form 26 April 2018; Accepted 1 May 2018 0193-3973/ © 2018 Elsevier Inc. All rights reserved.

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

academic content and instructional activities and their comprehension of academic texts (Gehlbach, 2011; LaRusso et al., 2016). By being able (and, importantly, willing and ready) to consider others' perspectives, students can better relate their own experiences to those of actors mentioned in academic texts. For example, in literary and historical narratives, different actors may have distinct experiences of the same events (e.g., emancipation from a slave's point of view is property theft from a plantation owner's). Beyond the classroom, students may encounter news and op-ed articles with very different accounts of the same social, civic, or political issues (e.g., the minimum wage, amnesty for immigrants) that implicitly or explicitly reflect their authors' ideological positions. Because individuals hold and express their own distinctive points of view on contentious issues, managing these conflicting characterizations of events and information depends in part on SPT skills: a reader must draw inferences both about and from the capabilities, attributes, expectations, feelings, and potential reactions (Light, 1979) of the actors in the texts in light of the text and its context (implicit as well as explicit) if she is to critically analyze and synthesize the information, and ultimately integrate it with what she already knows. This may be an increasingly important skill in middle and high school classrooms, where the complexity of the academic texts rises, often taking detached, general, and authoritative stances instead of the personal, concrete, and specific perspectives that students are accustomed to in literary texts and informal communicative settings (Goldman & Snow, 2015; Snow & Uccelli, 2009).

cognitive development (e.g., Ryskin, Benjamin, Tullis, & BrownSchmidt, 2015), and atypical development (e.g., Brown, Oram-Cardy, & Johnson, 2012; Ricketts, Jones, Happé, & Charman, 2013). In addition, SPT is often identified with complementary but distinct concepts such as theory of mind (Pelletier, 2006), narrative empathy (Keen, 2007), social cognition (Shantz, 1982), or social information processing (Dodge, 1986; Donahue, 2001). These lines of research often highlight the various cognitive processes that are prerequisites for SPT. For example, theory of mind, the ability to take others as agents with their own thoughts, knowledge and intentions (e.g., Carlson & Moses, 2001), is a necessary precursor to understanding and drawing inferences about the perceived intentions of another person. Gehlbach (2004a) conceptualization of SPT, comprising the skill of reading others' beliefs and intentions as well as the motivation to enact that skill, implicates motivational as well as cognitive processes. Conceptualizations of SPT as a multi-dimensional construct are particularly useful for increasing the current understanding of how SPT is applied and can best be fostered in academic contexts. The notion of SPT acts that we adopt and explore here arises out of social-relational models of perspective-taking, rather than cognitiverepresentational models (Diazgranados et al., 2016), precisely because of their applicability to academic contexts. While many experiences and contextual cues can facilitate the enactment of SPT skills (e.g., drawing inferences from another's tone of voice, reading facial expressions: Gehlbach, 2017), such cues are typically unavailable during academic activities such as comprehending academic texts. For example, when reading a historical text about an oppressed ancient people or writing an essay recommending a course of action to address a current social or political issue, students must consider and analyze the various underlying perspectives, based only on the text and their own accumulated knowledge and experiences. They must bear in mind not only the perspective of one or more other individuals but also the provenance of that perspective. That is, independent of a student's motivation to take another person's perspective (Gehlbach, Brinkworth, & Wang, 2012), we can explore that student's propensity for articulating others' perspectives and, more crucially, for expressing her grasp of the social basis of those perspectives—how the social role of a person whose perspective is taken and expressed fundamentally shapes that perspective. By foregrounding the ability to articulate and position others' perspectives in a social context, the conceptualization of SPT as SPT acts highlights a distinctly academically relevant variant of perspective-taking, and one that could potentially be intervened upon to enhance students' capability to apply their social-cognitive functions in their understanding of curricular content and their engagement with their classroom communities. Although ‘accurate’ and ‘ethical’ SPT is of great value in human discourse and interaction, in educational contexts simply displaying the skill to generate perspective-taking must be present, and more importantly, enacted.

Measuring SPT: advances and challenges Given the variety of SPT conceptualizations that have been proposed in the literature, efforts to measure this construct have also varied in their methods, each with their own strengths and limitations (Diazgranados et al., 2016; Gehlbach, 2004a). Self-reports are a particularly popular approach for assessing many aspects of SPT, both in children and adults. However, for some key components of SPT, such as SPT self-efficacy, scores on these self-report measures suffer from subjectivity bias and have been found to be poor indicators of SPT competence in actual social contexts (Ames & Kammrath, 2004; Hall, Andrzejewski, & Yopchick, 2009; Realo et al., 2003). Scenario- or situation-based measures (e.g., Gehlbach, 2004b; Schultz, Selman, & LaRusso, 2003; Selman & Feigenberg, 2010) offer another approach that has some advantages for assessing SPT competencies over self-report. By probing SPT across multiple scenarios or hypothetical situations, these measures take into account the variability of SPT across contexts (e.g., a peer versus a parent-child conflict, a familiar situation versus an unfamiliar one, a violent setting versus a safe setting) (Gehlbach, 2004a; LaRusso & Selman, 2011). With existing situationbased assessments, children typically select answers to SPT questions from multiple choice options or lists (Gehlbach, 2004b; Schultz et al., 2003), which captures their capacity to recognize the best or most accurate answer. While accuracy is one aspect of children's SPT competence, these measures do not provide an opportunity to observe children taking and interpreting another's perspective on their own without the prompts offered by the multiple choice options. The assessment of children's ability to perform SPT is critical; however, methods to assess SPT performance are rare. Situation-based interviews are one method to examine SPT performance. Open-ended questions provide access to the child's understanding so that it is also possible to assess more complex forms of SPT, for example, third-person perspective-taking and the integration of multiple points of view. Unfortunately, interviews are typically very costly and time-consuming, given the need to interview participants individually and transcribe responses for coding. In school settings, the logistics of scheduling interviews, interruptions to instructional time, and consent procedures for research purposes can also be prohibitive. In addition, SPT assessment conducted via verbal interviews is unlikely to provide a valid picture of students' ability to apply SPT skills and

SPT and success in school Being able to consider multiple perspectives and intentions is a critical developmental skill for learning in 21st-century classrooms, facilitating the processing of information from a variety of sources as well as effective communication with others (Schlitz et al., 2011; Takacs, 2003). SPT is associated with cooperative behaviors (e.g., Johnson, 1975), which teachers have identified as essential to students' success in the classroom (Lane, Givner, & Pierson, 2004), and it is predictive of long-term academic achievement (Caprara, Barbaranelli, Pastorelli, Bandura, & Zimbardo, 2000). SPT is also important for building and fostering meaningful social relationships in school (Dodge, 1986; Dodge et al., 2003). Positive social relationships serve as the basis for students' sense of relatedness to peers and teachers alike, which promotes academic motivation and engagement (Gehlbach, 2011; Ryan & Deci, 2000). In addition, SPT can facilitate both students' engagement with 25

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

multiple perspectives (Chapman, 1991; Martin, Sokol, & Elfers, 2008; Piaget, 1951; Selman, 1980).

perform in academic contexts, which often requires comprehension and interpretation of text and production of written argument, for example, understanding and writing essays on a historical event from different perspectives. These limitations in the measurement of SPT performance have hampered the ability to accurately assess and explicitly examine students' skills in understanding and interpreting others' perspectives across interpersonal situations and settings portrayed or embedded in academic texts. To address many of these limitations, this study demonstrates how the construct of SPT acts can be captured as a contextdependent performative skill with simple and complex elements, using a newly-developed direct assessment of SPT: the Assessment of Social Perspective-taking Performance (ASPP). The ASPP is a situation-based measure that presents multiple scenarios and contexts for analysis and interpretation in textual media, assesses SPT performance through written responses to open-ended questions, and provides scores for both relatively simple SPT actions (i.e., articulating how one or several actors think, feel, or are inclined to behave) and relatively complex SPT actions (i.e., contextualizing the position one or more actors take, with consideration of their roles, circumstances, experiences, and motivations).

Assessment of social perspective-taking performance The Assessment of Social Perspective-taking Performance (ASPP) is a revised and extended version of the Social Perspective-Taking Acts Measure (SPTAM: Diazgranados et al., 2016), a scenario-based assessment of students' ability to perform SPT acts in response to written texts about specific social situations. As operationalized in the SPTAM and ASPP, SPT acts refer to the demonstration or application of skills utilized to “read” the social world, e.g., through print text and social discourse, and in the navigation of complex social relationships and civic participation. The SPTAM and ASPP capture young adolescents' skill at inferring and articulating how the different actors in a given social situation would think and feel in light of their roles, experiences, circumstances, contexts, cultural background, and motivations in their own social world, as well as at integrating their perspectives into the solutions they offer to hypothetical social problems. The instrument analyzes a participant's short written responses to standard questions across a set of hypothetical social dilemmas that commonly occur in school settings. Specifically, the SPTAM and ASPP separately count the number of points of view that the examinee articulates and situates contextually (i.e., positions) within each of the supplied scenarios, without regard to their moral value, topical content, or valence, generating a frequency count of both of these SPT acts without assessing their accuracy or appropriateness. The initial SPTAM included two scenarios of social situations that were both related to bullying and teasing in school: Jariah's Weird Style (SS) and Lee's Journal (SJ: see Fig. 1 for details). Subsequently the measure was expanded to a wider array of social settings. The revised measure (ASPP) includes two additional scenarios that present social situations involving violation of institutional school rules: discovery of a friend having cheated on an exam (Math Exam: SM) and finding graffiti on a school building (Graffiti on the Wall: SG. See Fig. 1 for details). A coding manual (Selman, Diazgranados, & Dionne, n.d.) is used to assess the expression of both performative social skills (acts): articulation and positioning.1 The original SPTAM asked students to report their own point of view (POV2), in addition to taking the POV of two individuals in each scenario. While the items on their own POV ask student examinees to put themselves in a hypothetical social situation and give them the opportunity to express their own opinion, they do not directly provide any information about a student's ability to actually perform the act of taking another person's POV and make inferences about influences on that position. The revised measure, ASPP, focuses on students' abilities to perform SPT acts by measuring their functional ability to take the perspective of two individuals as consultants, each with a POV on the social situation that might be related to their own past experiences in situations, similar to the social situation presented. Specifically, the measure purposefully focuses on the possible advice to be given to the witness to the transgression, by the two consulting individuals who have distinctive life experiences related to the current situation (e.g., as a perpetrator or as a victim of teasing).

Distinctive SPT acts: articulation and positioning Few accounts of SPT would argue that the construct is unidimensional. Indeed previous work, such as Selman's (1980, 2003) developmental-structural approach, has described increasingly complex and conceptually distinct (i.e., differentiated or hierarchically integrated) skills that develop as children mature. In Selman's earlier model, the earliest levels of SPT development are characterized by a shift from social egocentrism, where children do not yet clearly distinguish others' points of view from their own, to more differentiated and integrated SPT that includes being able to perceive the thoughts and feelings of others and to understand that people can perceive each other's thoughts and feelings (Selman, 1980). It is this latter level of SPT development that is captured in the performance of social perspective articulation, which we define as the act of making an explicit statement that refers to the thoughts, feelings, preferences, and orientations to the actions of another person. For example, a student may observe or read about another student's expression after receiving a test report and say, “Lou felt discouraged by her exam score.” The presence or absence of an act of articulation underscores a student's ability or inability to bring another's perspective to bear—and also paves the way for importing a deeper level of social awareness into her reasoning and writing. Beyond the articulation of another's thought or feeling are the more sophisticated SPT acts we refer to as acts of social perspective positioning. Positioning in the context of SPT acts involves explicitly situating, and usually interpreting, the point of view attributed to another person in the contextualization of knowledge about that person (e.g., previous social experiences, cultural background, roles or relations to others, personal history, personality attributes). For example, “I'd say Lou felt discouraged by her exam score because she was accustomed to getting all A's.” Here, the articulated perspective of ‘feeling discouraged’ is further explained and justified by explicit reference (i.e., an act of SPT) to Lou's past experiences and her role as someone who typically earns high marks. Positioning a point of view, whether one's own or another's, is a complex act that requires holding in mind the multiplicity of factors, both personal and situational, that can drive individuals to distinct thoughts, feelings, or actions. While articulation is akin to the relatively earlier cognitive-developmental levels of SPT development, positioning is a skill that is invoked in higher levels of SPT development, as when adolescents consider the personal, situational, and cultural influences on another's perspectives (Selman, 1980). In this revised conceptual SPT framework, positioning, we posit, is both more complex than and qualitatively distinct from articulating a perspective, and an important prerequisite for even more advanced acts such as the integration of

1 The coding manual also includes a still more basic skill, acknowledgement. Here we do not analyze acknowledgment, coded as a simple count of number of actors mentioned in the essay and without any indication of a student's intention to take or use the actor's perspective. While it is a precondition of any perspective-taking act—in order to articulate a person's point of view the person herself must be acknowledged, as we discussed above—simply listing a relatively large number of actors present in the scenario does not index a student's ability to take the listed actors' perspectives and is therefore excluded from the scoring of ASPP. 2 We use an acronym for “point of view”, POV, to reference the specific questions or items of the ASPP. We do not abbreviate or capitalize the term when we reference the conceptual meaning of the “point of view.”

26

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

SS: Jariah’s Weird Style

SM: Math Exam

SG: Graffiti on the wall

Some students are teasing a new student in class, Jariah, because they think Jariah is weird. Casey is a student who has many friends and gets along with most classmates. Casey is asking different people for advice.

Rene is the star of the soccer team at school, but struggles with math. The principal has warned that students must not fail any class to play in the upcoming sports tournaments. Rene asks Amari, who is good at math, to let him copy the answers on the exam. Amari is asking different students for advice. SMP1: Azren SMP2: Perry Studying hard Soccer team for the exam member

Morgan saw Jordan and other students writing graffiti on the gym wall bragging about how great their basketball team was. The principal won’t let anybody use the gym until he knows who has done the damage. Morgan is asking different people for advice.

SSP1: Ali Recently teased

SSP2: Sam Teasers’ friend

Forms for Each Scenario Form A, Form B

SGP1: Jun Baseball team member

Form A, Form B

SJ: Lee’s Journal

Lee often gets excluded from games at recess. One day, Terry found Lee’s journal. Terry and other students were reading it out loud, laughing, and talking about how they were going to tell everybody about Lee’s secrets. Rene, the classroom president, doesn’t know what to do and looking for advice. SGP2: Alex SJP1: Carson SJP2: Andy Student Often teased by Laughed about discipline code others Lee’s journal writer

Form A only

Form B only

Fig. 1. ASPP scenario summary and points of view (POV) included in Form A and B.

examine the variation in early adolescents' social perspective articulation and positioning performance across varying social situations and points of view. Lastly, Aim 3 is to test hypothesized positive associations between SPT acts and academic engagement and performance.

The ASPP was administered in a large-scale randomized controlled trial of a program designed to improve students' deep reading comprehension skills (Strategic Education Research Partnership, n.d.). Although the original version of the SPTAM had been validated with a similar population, the ASPP includes significant modifications (i.e., the addition of new social scenarios and changes in elicited perspectives of the two consultant roles), requiring a reevaluation of its psychometric properties prior to its use for evaluation of the intervention program. In addition, in order to compare the scores of students who took different forms of the measure, potential differences among the scenarios and items presented needed to be examined. The categorical confirmatory factor analysis (CCFA) approach that we employ here helps to ensure that ASPP scores are used appropriately as a function of specific item response patterns accounting for differences in scenarios and forms. The CCFA approach can be viewed as an extension of multidimensional Item Response Theory (IRT) using a latent variable framework, that particularly allows flexible and multidimensional modeling of the factor analysis with ordered categorical responses (Wirth & Edwards, 2007). Like IRT, the advantage of the CCFA approach is that it evaluates the level of measurement precision of each item in capturing the overarching latent constructs, social perspective articulation and positioning, and then adjusts the degree to which the responses to each item contribute to the scaled score, depending on the measurement precision of the item. Thus, the CCFA approach provides robust scores for the articulation and positioning scales that are not dependent on the particular set of items used in the assessment, and they represent the underlying SPT skill level of each student across the varying performances in different ASPP items.

Method Data and participants This study utilized a subset of the data from an Institute of Education Sciences-funded evaluation of Word Generation, a researchbased academic language and literacy program for upper elementary and middle school students. The Word Generation evaluation included 25 schools from four school districts across two states, randomized to treatment and control conditions and followed through the 2012–13 and 2013–14 school years. The current study utilized data from the spring 2014 ASPP administration, when the academic outcome data used in this study were collected. Students who participated in the larger study but were not assessed on the ASPP in 2014 (n = 150) were excluded from the analysis as they are missing all information necessary for the analysis for Aim 1 and 2. The participants in the current study were 1299 fourth to seventh grade students from 66 classrooms in four control group schools in one state. Because the academic performance outcomes included state-specific standardized tests that are not directly comparable, we streamlined analyses by focusing on one state, selecting the state with the more nuanced test. The students were from diverse backgrounds reflecting the demographics of the urban and semi-urban communities of the schools. The current participants included 52% female students, 14% Black, 39% White, 4% Asian, 39% Latino, 3% Mixed Race/Other, 79% eligible for free/reduced price lunch, 12% English language learners, and 14% with special education classification.

Research aims This article focuses on describing and testing the theoretical conceptualization of SPT acts, as measured by the ASPP, as a multi-dimensional, context-dependent, and transferrable skill in academic engagement and performance. Aim 1 of this study is to test and confirm that the conceptual distinction between articulation and positioning can be detected in empirical data. Building on the findings from Aim 1, Aim 2 is to examine the psychometric properties of the ASPP and

Measures SPT acts The ASPP was administered in two forms to reduce the student assessment burden: Form A and B. Each form consists of three 27

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

RMSEA = .00, 90% CI [.00, .044], CFI = 1.00, TLI = 1.00. Classroom engagement was measured using the following four items from the scale developed by Wellborn and Connell (1987): “When we start something new in class, this student participates in discussions”; “In my class, this student works as hard as he/she can”; “When I explain new material, this student listens carefully”; “In my class, this student does more than required” (Cronbach's α = .86). Confirmatory factor analysis models showed that a one-factor model for the Classroom Engagement items fit the data very well when the residual variances of items 3 and 4 were allowed to covary, χ2(1) = 1.37, p = .24, RMSEA = .02, 90% CI [.00, .08], CFI = 1.00, TLI = 1.00, with high factor loadings (.70–.91).

scenarios, two of which were common across forms and one of which was unique. Specifically, Form A contained scenarios SS (Jariah's weird style), SM (Math exam), and SG (Graffiti on the wall), and Form B has SS (Jariah's weird style), SM (Math exam), and SJ (Journal hacking). For each scenario, students—framed in the role of a witness—were asked about what advice they might be given by each of two other individuals (framed as observers or consultants) located within the given scenario, each of whom was portrayed as having a distinct relationship to the transgression/problem. For instance, POV1 of Scenario SS presents a point of view of a student who was recently teased by classmates; and POV2 is of a friend of the teasers. Students wrote short essay-like responses to the three standard open-ended questions within each scenario: “What do you think [scenario consultant] would recommend to [Main Actor as Witness]?”; “Why do you think [scenario consultant] would make that recommendation?”; “What might go wrong with [scenario consultant]’s recommendation?” The responses to each of the three questions were considered together as one POV essay; and each POV essay was coded separately for social perspective articulation and positioning. As described earlier, ASPP item scores of Articulation and Positioning are generated via low-inference coding of these students' POV essays (i.e., coding for the presence or absence of articulation or positioning followed straightforward criteria and required no substantive evaluation of the merit of any identified SPT act). The coding manual was developed specifically for the ASPP, based on the measure used in the previous study (SPTAM: Diazgranados et al., 2016) and revised to accommodate the addition of the new scenarios (coding manual available from the authors upon request). Using this coding manual, trained coders identified each actor mentioned by the examinees in each POV essay whose thoughts, feelings, and actions are inferred and articulated and/or whose social positions are elaborated, given the information presented in the scenario—but they did not evaluate the accuracy or plausibility of the expressed perspective. The coded Articulation and Positioning scores in each of these POV essays were summed separately by counting the number of distinct actors whose perspectives were articulated and positioned. This resulted in a total of 6 items per form (Form A: SSP1 SSP2, SMP1 SMP2, SGP1 SGP 2; Form B: SSP1 SSP2, SMP1 SMP2, SJP1 SJP2) with 8 items available for scoring across the two forms for the Articulation and Positioning subscales. A subsample of the ASPP essays in the larger study (representing 5% of the full sample, n = 244) were double coded by the master coder and trained coders. Average interrater agreement was 98%. The raw scores of each item ranged 0–6 for Articulation items and 0–4 for Positioning items. Due to the very low frequency of observation in Articulation items 0, 5, and 6, and in Positioning items 3, 4 (< 1% each), Articulation items were recoded to range from 1 (0 or 1) to 4 (4 or higher); and Positioning items were recoded to range from 0 to 2 (2 or higher).

Academic performance We used students' state test scores for ELA and Mathematics from both the study year and the year prior, as measures of students' academic performance. For both years, the tests were administered to all 3rd to 8th grade public school students, including students with disabilities and English language learners, and they were designed to measure performance based on the state curriculum standards. The test scores were normed for each grade level for each subject and can range from 200 to 280. Student characteristics Students' gender, race/ethnicity, free/reduced-price lunch eligibility, English language learner status, and special education eligibility, were collected from the school districts and coded as binary variables. In addition, students' grade level (grade four to seven) were included and centered at grade four for analyses. Procedure Data for the present study were collected as part of the Word Generation evaluation study, which involved standard educational practices for assessing the effectiveness of school curricula. Thus, the Institutional Review Board (IRB) at Harvard University certified this study as exempt from IRB review, and no documentation of parent consent or student assent was required. Three of the schools included in this study were assigned to take the ASPP Form A (student n = 940); and one school took Form B (student n = 359). Trained research assistants read the instructions to classes of students and walked the groups through the questions in the measure, providing them with 30 min to respond in writing. Students who completed a section early were allowed to move ahead at their own pace. Participants' responses were coded by nine graduate research assistants and a master coder who participated in the measure development. Prior to coding, all coders participated in a minimum of ten hours of training in the coding manual by the master coder. The training included independent practice coding of assessments (3 or more) and master coders' individualized feedback. After completing the training, each coder and the master coder concurrently coded a new assessment, and compared the codes. This process was repeated up to three times until coders achieved minimum 90% agreement with the master coders and were deemed reliable for independent coding. If any coder did not achieve 90% agreement with the master coders after three rounds of reliability testing, this coder was ineligible to continue the training process. All active coders (including the master coder) participated in reliability checks, which consisted of coding one assessment and comparing codes every three months to prevent coder drift, followed by any necessary retraining based on differences among coders and differences from the master coder. If a coder's performance fell under 90% agreement with the master coder during any of these reliability checks, they stopped coding and were retrained.

Academic engagement We collected students' reading engagement and classroom engagement data from each student's English Language Arts (ELA) teacher or, when the ELA teacher did not fill out the survey, from the teacher who knew each student best. On average, students' engagement data were provided by the teachers who had known them for approximately 9 months, with 91% of responses provided by teachers who had known the students for 9 or 10 months. Both engagement surveys used a fourpoint Likert scale (1 = Not at all true to 4 = Very true). All items were entered as categorical variables in both the preliminary CFA models and the final model for hypothesis testing. Reading engagement was measured using a shortened (4-item) version of the Reading Engagement Index (Wigfield et al., 2008): “This student is easily distracted in selfselected reading”; “This student works hard in reading”; “This student is a confident reader”; “This student uses comprehension strategies well” (Cronbach's α = .85). Confirmatory factor analysis models testing the single factor structure for the Reading Engagement items show high factor loadings (.67–.96) and excellent model fit, χ2(2) = .91, p = .63,

Analysis plan A series of structural equation models was used to examine the 28

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

math test scores, Reading Engagement, Classroom Engagement), as well as predictors of interest (i.e., ASPP), suggest that the students with missing prior-year test scores perform significantly worse in the current-year test scores, engagement, and ASPP scores. In all models, missing data were pair-wise deleted (i.e., all available information is used from all cases) to preserve the full sample (N = 1299) for the final model estimated for Aim 3 (Asparouhov & Muthén, 2010). For the models estimating psychometric properties of ASPP, all students with ASPP data (n = 1279) are included for analysis.

research aims. First, two multi-group CCFA models were tested to compare unidimensional versus multidimensional factor structure of ASPP (Aim 1) and examined the item psychometric characteristics (Aim 2). CCFA models are appropriate for full-credit ordered responses with no guessing, the type of data typically encountered in psychological research and the type used in this study (Wirth & Edwards, 2007). In addition, a structural equation model was fit to examine the correlational evidence for construct validity of ASPP (Aim 3). Prior to analysis, multicollinearity among independent variables was explored using bivariate and partial correlations; no noteworthy issues were detected. All models were estimated using weighted least squares means and variance adjusted (WLSMV) estimation in Mplus 7 software version 1.4 (Muthén & Muthén, 1998-2012) to address the ordinal scale of the ASPP items. Standard errors and model fit were computed taking into account complex sampling structure where the students were nested within classrooms. In order to link two forms with unique scenarios and estimate the common item parameters for common items, a multigroup analysis approach was implemented. This modeling approach allows concurrent calibration of the common scales across the common and unique items of the two forms of the ASPP. Accordingly, a set of new common variables were created to replace the unique variables in each form. For example, the Articulation scores from the Graffiti on the Wall Scenario POV 1 (item SGP1ART) and from the Journal Scenario POV 1 (item SJP1ART) both were coded as a common variable SXP1ART, representing Scenario X, POV 1 Articulation score (see Table 1 for the full coding scheme). The same recoding of the variables was done for the Positioning items. Then, eight true common items (i.e., four common items each for Articulation and Positioning scales) and these newly created four common items (i.e., two items each for Articulation and Positioning) are modeled as 12 common items in the model. Participants who took Form A were modeled as Group A; and those who took Form B are modeled as Group B. In all models, estimates for factor loadings and thresholds of the eight true common items, as well as all covariances, were constrained to be equal across groups; factor loadings and thresholds for the four newly created common items were freely estimated for each group. Overall fit of the models was evaluated using multiple model fit indices, including the model chi-square, the root mean square error of approximation (RMSEA), and comparative fit index (CFI) and Tucker Lewis Index (TLI). Following the suggestions of Hu and Bentler (1999), we used a cut-off criterion for RMSEA (≤.06) and CFI and TLI (≥.95).

Results Descriptive statistics Table 2 presents means, standard deviations, and inter-item polychoric correlations of social perspective articulation and positioning items. The overall comparison between the raw scores of each item revealed that Positioning items have a smaller mean and standard deviation than Articulation items. The polychoric inter-item correlations within the Articulation scale were moderate to high, at r = .29–.60. Positioning scores were correlated across items at r = .17–.47, generally lower inter-item correlations than for Articulation scales. Table 3 presents the number of observations, means, standard deviations, and ranges for the academic outcomes, including reading and classroom engagement scores of each item and previous and current year ELA and Mathematics state test scores. Aim 1: ASPP factor structure Two multi-group CCFA models were used to examine the factor structure while appropriately addressing the categorical item response functions (Mislevy, 1985; Wirth & Edwards, 2007): Model 1, a unidimensional model where all of the ASPP Articulation and Positioning items loaded on a single latent variable representing overall SPT act performance, and Model 2, a two-factor model where all Articulation items loaded on the Articulation latent variable and all Positioning items loaded on the Positioning latent variable, where the two latent variables were modeled to be correlated. In both models, variances were fixed at 1 and means and intercepts were fixed at 0 for all the latent variables and observed items for both groups.3 Item factor loadings and thresholds were freely estimated but constrained to be equal across the groups. The Mplus theta parametrization was used to estimate thresholds of categorical items.

Missing data Of 1299 students who were assessed with ASPP in the participating schools, 20 assessments (1.5%) were returned with very little information and therefore were not coded. Of those assessments that were coded, around 1% of each item of the ASPP was missing in each form (see Table 2 for the sample size of each item). In addition, 2–22% of students had missing information either in current-year or prior-year state test scores or teacher reports of engagement (Table 3). Descriptive comparisons of students who are missing prior year ELA and math test scores but have non-missing outcome data (i.e., current-year ELA and

Unidimensional model A multi-group item factor model (Model 1) for all items from Form A and Form B is presented in Fig. 2. The standardized factor loadings of each indicator to the overarching SPT Act factor ranged .54 to .74 for Articulation items and .34 to .45 for Positioning items. The overall model fit suggested that the unidimensional model did not represent the data very well, χ2(136) = 535.05 (Form A = 341.20, Form B = 193.86), p < .001, RMSEA = .07, 90% CI [.06, .07], CFI = .88 and TLI = .89. Multidimensional model Another multi-group item factor model (Model 2) tested the twofactor structure of the ASPP, Articulation and Positioning (Fig. 3). The standardized factor loadings of the articulation items on the Articulation factor ranged from .55 to .77, and factor loadings of the positioning items on the Positioning factor ranged from .49 to .68. This model with two dimensions of SPT, Articulation and Positioning, had a good fit,

Table 1 Coding scheme of the Form A- and Form B-unique items into common items for the multigroup analysis. Form A

Form B

Modeled as

Articulation scores SGP1ART SGP2ART

SJP1ART SJP2ART

SXP1ART SXP2ART

Positioning scores SGP1POS SGP2POS

SJP1POS SJP2POS

SXP1POS SXP2POS

3 These final models were not significantly different from the model where the latent means and covariance were freely estimated for Form B group, Model 1: χ2(1) = .22, p = .64, Model 2:, χ2(3) = 1.23, p = .75.

29

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Table 2 Descriptives of each ASPP item across two forms and inter-item polychoric correlations. Form A items

n

M

SD

1

2

3

4

5

6

7

8

9

10

11

1 2 3 4 5 6 7 8 9 10 11 12

919 915 919 919 913 910 926 922 926 926 920 917

2.35 2.18 2.23 2.31 2.17 2.03 .89 .93 .83 1.09 .68 .63

.83 .77 .72 .82 .77 .73 .64 .56 .60 .70 .64 .60

.46⁎⁎⁎ .54⁎⁎⁎ .40⁎⁎⁎ .43⁎⁎⁎ .43⁎⁎⁎ .15⁎⁎⁎ .13⁎⁎ .17⁎⁎⁎ .22⁎⁎⁎ .12⁎⁎ .18⁎⁎⁎

.53⁎⁎⁎ .47⁎⁎⁎ .49⁎⁎⁎ .47⁎⁎⁎ .17⁎⁎⁎ .11⁎⁎ .17⁎⁎⁎ .24⁎⁎⁎ .13⁎⁎ .14⁎⁎⁎

.60⁎⁎⁎ .57⁎⁎⁎ .59⁎⁎⁎ .18⁎⁎⁎ .16⁎⁎⁎ .26⁎⁎⁎ .26⁎⁎⁎ .18⁎⁎⁎ .22⁎⁎⁎

.55⁎⁎⁎ .57⁎⁎⁎ .23⁎⁎⁎ .17⁎⁎⁎ .22⁎⁎⁎ .29⁎⁎⁎ .17⁎⁎⁎ .21⁎⁎⁎

.58⁎⁎⁎ .21⁎⁎⁎ .16⁎⁎⁎ .20⁎⁎⁎ .27⁎⁎⁎ .10⁎⁎ .18⁎⁎⁎

.11⁎⁎⁎ .07 .15⁎⁎⁎ .20⁎⁎⁎ .11⁎⁎ .17⁎⁎⁎

.29⁎⁎⁎ .31⁎⁎⁎ .29⁎⁎⁎ .30⁎⁎⁎ .30⁎⁎⁎

.32⁎⁎⁎ .27⁎⁎⁎ .29⁎⁎⁎ .32⁎⁎⁎

.37⁎⁎⁎ .35⁎⁎⁎ .42⁎⁎⁎

.39⁎⁎⁎ .37⁎⁎⁎

.45⁎⁎⁎

Form B items

n

M

SD

1

2

3

4

5

6

7

8

9

10

11

1 2 3 4 5 6 7 8 9 10 11 12

354 354 355 354 354 352 919 915 919 919 913 910

2.42 2.18 2.18 2.28 2.26 2.08 .89 .93 .83 1.09 .68 .64

.88 .76 .74 .83 .83 .81 .64 .56 .60 .70 .64 .60

.51⁎⁎⁎ .39⁎⁎⁎ .43⁎⁎⁎ .42⁎⁎⁎ .29⁎⁎⁎ .16⁎⁎ .23⁎⁎⁎ .24⁎⁎⁎ .17⁎⁎ .30⁎⁎⁎ .29⁎⁎⁎

.37⁎⁎⁎ .51⁎⁎⁎ .43⁎⁎⁎ .41⁎⁎⁎ .03⁎⁎⁎ .16⁎ .21⁎⁎⁎ .18⁎⁎ .17⁎⁎ .15⁎⁎

.53⁎⁎⁎ .52⁎⁎⁎ .37⁎⁎⁎ .16⁎⁎ .18⁎⁎ .06 .16⁎⁎ .20⁎⁎ .19⁎⁎

.49⁎⁎⁎ .41⁎⁎⁎ .20⁎⁎⁎ .22⁎⁎⁎ .25⁎⁎⁎ .07 .18⁎⁎ .24⁎⁎⁎

.47⁎⁎⁎ .26⁎⁎⁎ .23⁎⁎⁎ .18⁎⁎ .16⁎⁎ .14⁎⁎ .23⁎⁎⁎

.11 .13 .10 .23⁎⁎⁎ .13⁎⁎ .14⁎⁎

.35⁎⁎⁎ .27⁎⁎⁎ .22⁎⁎⁎ .25⁎⁎⁎ .27⁎⁎⁎

.17⁎⁎⁎ .25⁎⁎⁎ .45⁎⁎⁎ .39⁎⁎⁎

.39⁎⁎⁎ .43⁎⁎⁎ .23⁎⁎⁎

.33⁎⁎⁎ .36⁎⁎⁎

.47⁎⁎⁎

⁎ ⁎⁎

SSP1 Articulation SSP2 Articulation SMP1 Articulation SMP2 Articulation SGP1 Articulation SGP2 Articulation SSP1 Positioning SSP2 Positioning SMP1 Positioning SMP2 Positioning SGP1 Positioning SGP2 Positioning

SSP1 Articulation SSP2 Articulation SMP1 Articulation SMP2 Articulation SJP1 Articulation SJP2 Articulation SSP1 Positioning SSP2 Positioning SMP1 Positioning SMP2 Positioning SJP1 Positioning SJP2 Positioning

12

12

p < .05. p < .01. p < .001.

⁎⁎⁎

Table 3 Descriptives of student demographic characteristics and academic outcomes. Variables

n

M

Reading engagement This student is easily distracted in self-selected reading This student works hard in reading This student is a confident reader This student uses comprehension strategies well

1025 1016 1014 1026

2.77 3.12 3.17 3.16

Classroom engagement When we start something new in class, this student participates in discussions In my class, this student works as hard as he/she can When I explain new material, this student listens carefully In my class, this student does more than required

1141 1125 1124 1124

State test scores 2013 ELA State Test 2014 ELA State Test 2013 Math State Test 2014 Math State Test

1171 1269 1140 1274

SD

Min

Max

1.01 .90 .89 .91

1 1 1 1

4 4 4 4

2.94 3.07 3.19 2.50

1.00 .92 .91 1.01

1 1 1 1

4 4 4 4

238.67 239.81 239.93 235.60

14.35 14.20 16.87 17.30

206 206 204 204

280 280 280 280

parameters were freely estimated for each group, Δχ2(17) = 23.36, p = .13.

χ2(135) = 174.72 (Form A = 67.44, Form B = 107.28), p = .01, RMSEA = .02, 90% CI [.01, .03], CFI = .99 and TLI = .99. These multidimensional models fit the data significantly better than the unidimensional models, Δχ2(1) = 231.91, p < .001, providing empirical evidence supporting the conceptual distinction between Articulation and Positioning as measured by ASPP. A set of post-hoc models testing the measurement invariance across two groups, Form A and Form B, confirmed the common items achieved scaler invariance, i.e., the model where common items were constrained to have the same factor loadings, response thresholds, and covariance between factors fit as well as the model where these

Aim 2: item functioning of the ASPP Aim 2 of this study was to assess the psychometric properties of the ASPP as a measure and to explore if different social situations or POVs give rise to variation in adolescents' SPT articulation and positioning performance. This step was critical to establish the viability of the ASPP as a reliable measure of SPT skills, generalizing across multiple social situations. For ease of interpretation of response patterns, the CCFA

30

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Fig. 2. Model 1: Unidimensional CFA model of social perspective-taking acts for (a) ASPP form A and (b) ASPP Form B. ART represents Articulation and POS represents Positioning; Each indicator represents Articulation or Positioning item score. Variance (fixed at 1 for identification) and mean (fixed at 0 to be equal across groups) of latent variables and measurement errors in observed variables are omitted from the figure. All factor loadings presented are standardized estimates and significant at p < .001. Standard errors are presented in parentheses.

Fig. 3. Model 2: Multidimensional CFA model of social perspective-taking acts for (a) ASPP form A and (b) ASPP Form B. ART represents Articulation and POS represents Positioning; Each indicator represents Articulation or Positioning score for each item. Variance (fixed at 1 for identification) and mean (fixed at 0 to be equal across groups) of latent variables (fixed at 1 for identification) and measurement errors in observed variables are omitted from the figure. All factor loadings presented are standardized estimates and significant at p < .001. Standard errors are presented in parentheses.

within the item, in the Articulation and Positioning scales. The discrimination (α) and difficulty (β) parameters are converted from the factor loading (b) and threshold (τ) parameters for a latent factor i of an item j from the CCFA model using the following equations4:

model estimates of each social scenario and POVs across the two forms were converted into IRT parameters (i.e., discriminant function and difficulty levels). In light of the Aim 1 findings, for this step we used the multidimensional model (Model 2) where Articulation and Positioning scales were evaluated separately as two different constructs within a single model. A CCFA model is analytically equivalent to a 2PL or graded-response model with a probit link function, and the CCFA parameters can be easily translated as IRT parameters (Lord, 1952; Lord & Novick, 2008; Takane & de Leeuw, 1987; Wirth & Edwards, 2007). While CCFA estimates are not directly equivalent to IRT parameters, the converted IRT estimates can be interpreted using conventional IRT criteria (e.g., Baker, 2001). Table 4 shows the discrimination parameter of each item, as well as the threshold parameter of each score category

4Forero, Maydeu-Olivares, and Gallardo-Pujol (2009, p. 278) present a multidimensional IRT version of these formulas in which their model freely estimates factor loadings for all items (thus, each item's factor loading is a vector). Consequently, the correlation structure among the multidimensional IRT factors must be included in these conversion formulas. However, for our CCFA model, the factor loading matrix was constrained to have zero factor loadings for items associated with different factors. Thus, the correlation adjustment disappears, and these formulas are equivalent to those of Forero et al. (2009) under our model conditions. 31

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Table 4 Articulation and positioning scale item discrimination (α) and difficulty level (β1- β6) estimates. Item factor analysis estimates Articulation items SSP1Articulation SSP2 Articulation SMP1 Articulation SMP2 Articulation SGP1 Articulation SGP2 Articulation SJP1 Articulation SJP1 Articulation

IRT parameters (transformed) τ2

τ1

b

−1.08 −.93 −1.12 −1.07 −.96 −.76 −.93 −.74

.63 .66 .77 .74 .74 .74 .70 .55

τ3

.21 .50 .53 .38 .55 .74 .36 .68

1.27 1.69 1.70 1.36 1.63 1.92 1.43 1.53

Item factor analysis estimates Positioning items

b

τ1

SSP1 Positioning SSP2 Positioning SMP1 Positioning SMP2 Positioning SGP1 Positioning SGP2 Positioning SJP1 Positioning SJP2 Positioning

.49 .51 .59 .61 .58 .65 .68 .62

−.55 −.79 −.56 −.84 −.22 −.19 −.36 −.33

⎛ bij α ij = ⎜ 1 − bij2 ⎝

⎞ ⎟D ⎠

and βij =

α

β1

β2

−1.71 −1.41 −1.44 −1.44 −1.29 −1.03 −1.34 −1.34

1.38 1.50 2.07 1.86 1.88 1.85 1.65 1.13

.34 .76 .68 .51 .74 1.00 .52 1.24

β3 2.02 2.55 2.19 1.83 2.20 2.61 2.05 2.77

IRT parameters (transformed) τ2 1.07 1.13 1.22 .54 1.31 1.53 1.37 1.49

α

β1

β2

.96 1.00 1.26 1.30 1.21 1.44 1.58 1.35

−1.11 −1.55 −.94 −1.38 −.38 −.29 −.53 −.54

2.18 2.23 2.06 .88 2.26 2.37 2.02 2.40

Positioning scale Table 4 also presents the discrimination (α) and threshold (β) parameters of each item in the Positioning scale. All of the items had at least moderate (> .65) to high (> 1.35) α values. Similar to the Articulation scale, all items related to violations of school rules (Math exam (SMP1, SMP2), Graffiti on the wall (SGP1, SGP2) had a higher range of α values, compared to one of the teasing and bullying scenarios (Jariah's weird style, SSP1, SSP2), suggesting these scenarios were more sensitive to individual variation in positioning skills. However, the Form B-unique teasing and bullying scenario items (Journal hacking, SJP1, SJP2) had similar or higher discriminating values than the Math Exam scenario items (SMP1, SMP2), the rule violation scenario in the same form (B). The β parameters of the Positioning scale were generally much higher than for Articulation items. Students with positioning skills 1 probit lower than average (θ < −1) have < 50% probability of scoring 1 or higher in most items (SMP1, SGP1, SGP2, SJP1, SJP2). The probability of being coded for 2 perspectives in Positioning items was much lower for the majority of students, with β for ≥2 level for all items except SMP2. This suggests that positioning was a difficult exercise for most students in the sample, at least as measured by the ASPP. As with the Articulation items, the Positioning item characteristic curve plots, presented in Fig. 6, indicated that each response category was the modal response for some range of values along the unidimensional spectrum. However, most of the item category response curves peaked > 0 except the curve representing the probability of scoring 0 or 1, suggesting that most students in the sample positioned with only one perspective, or did not position at all, on ASPP items. Fig. 7 illustrates the overall function of the Positioning scale. The values of item test function (Fig. 7a) were the highest around θ > −1 range. This confirmed that the Positioning scale did a better job in capturing variation among students with higher Positioning scores (at least > −1.5 probit of the sample) but measured students' Positioning scales in the very low range (−2 probit or lower) less accurately. As with the Articulation subscale, the Positioning items are well targeted to the individual ability estimates (see person-item plots in Fig. 7b).

τij bij

where D is scaling constant 1.7, used to transform probit CCFA model estimates to logit IRT model estimates.

Articulation scale All of the items had high discrimination values (α > 1.35: Baker, 2001), with an exception of the SJP1 item, suggesting that Articulation items were highly precise in differentiating students' ability on the construct of social perspective articulation. All items related to violations of school rules (SM, SG) had very high discrimination values (> 1.75: Baker, 2001). This suggested the scenarios related to school rule violations may more effectively detect variation in students' articulation skills than the scenarios related to bullying and teasing. The difficulty parameters (β) of articulation shown in Table 4 indicate that the β values for scoring 1 or 0 (β1) in all Articulation items were low on the latent articulation skills captured by the item (all < −1 probit), suggesting that the majority of students, including students who had low articulation skill, were still able to articulate at least one perspective when providing advice in social dilemma situations. A visual examination of the characteristic curve plots (Fig. 4) revealed the desired “rolling hills” image, clearly indicating each response category was the modal response for some range of values along the unidimensional spectrum. Furthermore, each line representing the probability distribution of the lower scores (< 2) peaked at articulation skill θ < 0, suggesting that most students in the sample were able to articulate one or two perspectives. Fig. 5a presents the overall test function of the Articulation scale and 4b presents the matched person-estimates for the two versions of the instrument. Specifically, the person-item distribution plot shows the distribution of individual values of test information across θ values of −3 probit to +3 probit. After collapsing the response scales as indicated above, it was clear that the positions of the item thresholds were well-targeted for this sample. The test information function values (see Fig. 5a) were relatively high across most of the Articulation θ range, except for the students who were in θ < −2 probit below the average, suggesting good coverage of test information across a wide range of articulation skill levels.

Reliability of the scales While internal reliability estimates were challenging to obtain for model-based scoring protocols, the resulting model-produced omega reliabilities (obtained from the standardized factor loadings: McDonald, 32

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Fig. 4. Item characteristic curve plots for Articulation scale.

1999; Raykov, 1997) are high, ΩA = .86 and ΩB = .83 for the Articulation scale and ΩA = .74 and ΩB = .76 for the Positioning scale. These estimates were higher than conventional Cronbach alpha coefficients, αA = .82 and αB = .78 for Articulation, and αA = .67 and αB = .66 for Positioning. Overall, these findings suggest that the model-based Articulation and Positioning scales provided reliable estimates of students' SPT acts, performing better than conventional aggregated scores.

measurement models for ASPP Form A and Form B in different groups as tested for Aim 1.5 Reading Engagement and Classroom Engagement were included as latent variables with item-level measurement models. The observed scores of ELA and Math State Test scores were rescaled (divided by 100) to be on similar scales with other variables for easier parameter interpretation and model convergence. The residual errors of the four academic variables were allowed to covary. All path and covariance estimates were fixed to be the same across the Form A and B

Aim 3: relations of ASPP constructs with academic outcomes 5 Model 2, fitted with the means, variances, and covariances constrained to be equal across the group, shows that the final model (Model 3) could be run as one group analysis for concurrent calibration and maximum likelihood estimation, and that such an approach nicely handles the planned missingness created by using two forms. While this study continues to use a multi-group approach for consistency, future studies may consider a planned missingness approach for simplicity once the group equivalence is established.

Aim 3 was to examine the hypothesized relations of social perspective articulation and positioning to teacher-reported classroom and reading engagement as well as students' performance on standardized assessments of ELA and Mathematics. We estimated the predictive relations between SPT skills and academic performance and engagement in a multi-group SEM model (Fig. 6), where we included the full 33

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Fig. 5. Test information function and person-item plot for Articulation scale.

prior year in ELA and Mathematics state tests. On the other hand, English language learners and students eligible for special education were likely to score lower in both SPT acts. Latino/a students scored lower on the Articulation scale than white peers, but not on the Positioning scale. Fig. 8 presents an abbreviated Model 3 diagram and unstandardized estimates for hypothesized associations; Table 6 presents standardized estimates of the same parameters as well as residual variances and covariances. The findings suggest that Articulation was significantly associated with Reading Engagement and Classroom Engagement, but

groups. A set of exogenous variables, including grade level, gender, race/ethnicity, free/reduced price lunch eligibility, English language learner status, special education eligibility, previous year ELA and math state test scores, were entered as covariates when predicting ASPP Articulation and Positioning scores, as well as academic engagement and performance outcomes. The associations between these exogenous variables and SPT acts are presented in Table 5. The results revealed that students who scored higher on SPT Articulation and Positioning scales were more likely to be in higher grades, female, and to have earned higher test scores in the 34

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Fig. 6. Item characteristic curve plots for Positioning scale.

Discussion

not with ELA and Mathematics state test scores. In contrast, Positioning significantly predicted ELA and Mathematics state test scores, but not Reading Engagement or Classroom Engagement. Table 7 shows residual variances and the model-explained variances (R2) for Articulation and Positioning, as well as the academic outcome variables. The overall model explained 52–54% of the variance in Reading Engagement, 33–34% of the variance in Classroom Engagement, 62–67% of the variance in ELA, and 62–64% of the variance in the Mathematics test scores in Form A and B. The model had adequate goodness of fit, χ2(913) = 1138.10 (Form A = 501.37, Form B = 636.73), p < .001, RMSEA = .02, 90% CI [.02, .02], CFI = .97 and TLI = .96.

In this paper, we empirically explored the theoretical conceptualization of two types of SPT acts and examined the psychometric properties of the ASPP. Informed by a broad range of theories across multiple disciplines and research strands, the results of analyses pursuing Aim 1 confirmed the theorized conceptual distinction between articulation and positioning using confirmatory factor analysis with empirical data. In Aim 2 we found strong evidence that the ASPP can reliably and validly capture early adolescents' SPT articulation and positioning performance across the perspectives of different actors in different social situations presented in the ASPP scenarios. In Aim 3 we

35

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Fig. 7. Test information function and person-item plot for Positioning scale.

articulating another's perspective without explicit contextualization or positioning (Selman, 1980). The consistently lower raw item scores on Positioning than Articulation may be explained in a number of ways. Perhaps positioning is intrinsically more challenging than articulation, as it requires more world knowledge and social experience, as well as more linguistic sophistication (e.g., use of subordinate clauses). On the other hand, the scenarios and prompts may have been more explicit in eliciting articulation than positioning; if so, then the students who provided positioning statements may have been particularly motivated rather than more skilled. It seems likely that both articulation and positioning skills continue to develop during middle childhood and adolescence, being refined

found that SPT acts predict academic outcomes. Below, we interpret these findings in relation to the existing literature and discuss implications for application in practice. Articulating versus positioning social perspectives The current study empirically tested and confirmed in a relatively large sample of young adolescents that the acts of articulating and positioning others' perspectives in social situations are related yet distinct skills. This finding is aligned with the hypothesis that being able to contextualize others' perspectives based on a person's history and other characteristics is distinguishable from understanding and 36

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Psychometric properties of the ASPP

Table 5 Predictors of social perspective articulation and social perspective positioning. Unstandardized coefficients

SE

Aim 2 represents one of the few rigorous psychometric investigations to use CCFA modeling in the field of social-emotional development. The results suggest that the ASPP has good psychometric properties with moderate to high discrimination values among students with varying levels of social perspective articulation and positioning skills. Because it is easy to administer and code, the ASPP could provide a valuable tool for researchers and educators to precisely assess students' SPT performance skills at a relatively low cost and with less time investment than other available alternatives such as situation-based interviews. Though there is evidence to suggest the Positioning scale is not as reliable and precise in assessing students at the very low end of the positioning skill range (as seen with the threshold parameters for scoring at least 1 or higher for each item being mostly > −1 probit or higher), the person-item plots for both scales indicate the instrument is well-suited to target this population. Taken together with good internal model-based reliability of the Articulation and Positioning scales, these findings show strong evidence that the ASPP Articulation and Positioning scales can reliably and validly capture students' SPT acts for the target population of the current study. This is consistent with the conceptualization that social perspective positioning is a more mature SPT skill that builds and expands upon the simpler skill of identifying the thoughts, feelings, and orientation of an actor without regard to the actor's social position or history (Selman, 2007). As a performance-oriented measure, the ASPP requires both the use of social-cognitive ability to take different actors' perspectives and the ability to express and situate those perspectives in the actors' backgrounds and orientations. One can easily imagine, then, that even if students have emerging positioning skills, it could be more challenging to demonstrate this skill in a consistent manner across various social situations than to simply articulate how a person in the scenario would think, feel, and act. While it may be difficult to observe positioning skills for low-SPT skill students with the ASPP, it is encouraging that we can relatively precisely assess a broad range of skill levels of

Standardized coefficients Form A

Form B

Social perspective articulation on Grade levels Female Black Asian Latino/a Other Race/Ethnicity Free Lunch Eligibility English Language Learners Special Education Prior Year ELA State Test Prior Year Math State Test

.10⁎⁎⁎ .41⁎⁎⁎ −.07 −.04 −.10⁎ −.22 −.11 −.43⁎⁎⁎ −.34⁎⁎⁎ 1.34⁎⁎⁎ .47⁎

.03 .05 .07 .14 .05 .13 .06 .07 .06 .27 .24

.12 .52 −.09 −.04 −.13 −.28 −.14 −.54 −.43 .21 .09

.13 .54 −.09 −.05 −.14 −.29 −.14 −.57 −.45 .20 .09

Social perspective positioning on Grade Levels Female Black Asian Latino/a Other Race/Ethnicity Free Lunch Eligibility English Language Learners Special Education Prior Year ELA State Test Prior Year Math State Test

.08⁎⁎⁎ .22⁎⁎⁎ −.08 −.03 .02 −.16 .03 −.30⁎⁎⁎ −.23⁎⁎⁎ .91⁎⁎⁎ .43⁎

.02 .04 .06 .07 .04 .09 .05 .06 .05 .21 .18

.16 .42 −.16 −.05 .03 −.31 .07 −.58 −.45 .22 .12

.17 .47 −.17 −.06 .03 −.34 .07 −.65 −.50 .23 .13

**p < .01. ⁎ p < .05. ⁎⁎⁎ p < .001.

beyond the emergent and cognition-centered theory of mind posited to develop in early childhood and the early elementary years (Apperly & Robinson, 2003; Hulme, Mitchell, & Wood, 2003; Wellman, Cross, & Watson, 2001).

Fig. 8. Model 3: Multi-group structural equation model for ASPP Form A and B, testing associations of social perspective articulation and positioning with academic engagement (reading, classroom) and performance (ELA and Math test scores). Unstandardized estimates are presented with standard errors in parentheses. Standardized coefficients for path and covariance estimates are available in Table 6. Paths from exogenous variables to Social Perspective Articulation and Social Perspective Positioning are presented in Table 5, paths from exogenous variables to Reading Engagement, Classroom Engagement, ELA State Test, and Math State Test are omitted and available upon request. All estimates presented are unstandardized coefficients that are fixed to be equal across the Form A and Form B groups. ⁎p < .05, ⁎⁎p < .01, ⁎⁎⁎p < .001.

37

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

Table 6 Standardized and unstandardized parameter estimates for Model 3: associations and covariances of the social perspective articulation and positioning with academic engagement and performance. Standardized coefficients (Form A)

Standardized coefficients (Form B)

Path parameters Reading engagement on Social Perspective Articulation Social Perspective Positioning

.11 .04

.11 .04

Classroom engagement on Social Perspective Articulation Social Perspective Positioning

.12 .05

.12 .05

ELA state test on Social Perspective Articulation Social Perspective Positioning

.04 .14

.04 .13

Math state test on Social Perspective Articulation Social Perspective Positioning

.03 .09

.03 .08

Residual covariances Social Perspective Positioning

.28

.32

Reading engagement with Classroom Engagement ELA State Test Math State Test

.91 .30 .15

1.02 .32 .17

Classroom engagement with ELA State Test Math State Test

.24 .21

.26 .24

ELA state test scores with Math State Test

.30

.32

Table 7 Residual variances and model-explained variances (R2) of the Model 3. R2

Residual variance Unstandardized

Social Perspective Articulation Social Perspective Positioning Reading Engagement Classroom Engagement ELA State Test Math State Test

Standardized

Form A

Form B

Form A

Form B

Form A

Form B

.51 .23 .64 .64 .01 .01

.41 .17 .59 .58 .01 .01

.75 .76 .46 .66 .34 .38

.78 .73 .67 .48 .38 .36

.25 .24 .34 .54 .67 .62

.22 .27 .33 .52 .62 .64

and situations, as these are likely to differentially influence students' performance depending on their aptitudes, motivation, and past experience (Gehlbach, 2004a). Analysis and scoring of SPT performance in scenario-based measures using the IRT or CCFA approach, as in this study, makes it possible to not only assess students' SPT performance in a more precise way, but also to examine differences in students' abilities to perform SPT in varying social scenarios. Understanding the context-dependency of SPT performance is an important area of focus for future research. In the current study, we have tested students' SPT in two types of social situations in school settings (teasing/bullying, and school-rule violations) and for two types of actors (those sympathetic to aggressors or rule-violators and those sympathetic to victims or rule enforcers). These social situations were designed to maximize students' performance in the SPT tasks by providing familiar and relevant scenarios for upper elementary and middle school age children in the sample, minimizing the constraints of situational features (e.g., unfamiliarity) that may impair students' performance. While we did find varying levels of sensitivity to differences in students' SPT skill level, we did not find consistent patterns by types

social perspective articulation, on which positioning skills can and should later be built. Variability in SPT performance across different social situations The findings of Aim 2 also speak to the variability of SPT performance across different actors' perspectives in different social situations. Performance of SPT is context-dependent, as it can be constrained by one's experiences, feelings, and motivations in a particular social situation (Gehlbach, 2004a, 2011). When a given social situation is simple or familiar, students are better able to comprehend, and hence express, the perspectives of others in the scenario. Similarly, if the social situation is relevant and meaningful to students, they are more motivated to actively understand and take the positions of the people involved in the situation. The variation in SPT performance across multiple scenarios and perspectives reported in this study suggests that it is important to measure SPT acts in varied social conditions in order to capture more generalized SPT skills. It also suggests that measures of SPT should explicitly consider and weigh different features of SPT tasks 38

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

skills into daily interactions and instructional practices within academic curricula (Jones & Bouffard, 2012). For example, teachers can ask questions and lead discussions that are designed to help students to articulate and explicitly position the points of view that are presented in academic texts. Curriculum materials and activities that are designed to be relevant to students' lives and that explicitly make connections to students' experiences are more effective in engaging students' participation. Such efforts are emerging, both by virtue of integrating explicit attention to perspective-taking in literacy instruction (e.g., Word Generation: Jones et al., in press; Pathway Project: Olson & Land, 2007) and by embedding a social-emotional learning focus into literacy-rich domains like history (e.g., Facing History and Ourselves: Barr et al., 2015)—and, encouragingly, both types of programs report significant impacts on students' academic outcomes. Nonetheless, a focus on SPT within mainstream curricula remains rare, perhaps because the SPT construct is only beginning to be conceptualized through an academic lens and feasibly operationalized in ways that facilitate educators' understanding of how to foster it. In particular, the conceptualization and measurement of the performative aspects of SPT can support educators and interventionists in developing practices to promote SPT as part of learning about literature, history, and other content areas.

of social situation or actors' perspectives. It is possible that certain commonalities in the scenarios (peer-focused, school-based dilemmas) resulted in greater consistency than we would have found with a broader range of scenarios (e.g., parent-child dilemmas, political or cultural divisions). We believe the ASPP provides a flexible platform for measuring SPT acts with easily adaptable and expandable social scenarios, when accompanied by adequate analysis considering contextspecific response patterns as shown in this study. The findings from this study suggest that further research using the ASPP may benefit by including, and explicitly examining and modeling, students' SPT performance across a wider variety of social situations (e.g., familiar and unfamiliar, in school settings and in out-of-school settings, centered around peer or child-adult interactions) not only to more accurately assess students' SPT performance with higher precision and generalizability, but also to help identify the best ways to assess either generalizable or context-specific SPT skills that are relevant to academic and social-emotional development. SPT and classroom learning The findings from Aim 3 suggest that both social perspective articulation and positioning skills are relevant to students' learning, potentially through different pathways. First, social perspective articulation may support academic learning by facilitating students' engagement with academic materials and activities. We report that social perspective articulation is uniquely associated with both reading and classroom engagement, over and above student demographic characteristics and previous-year academic performance on state tests. There is ample evidence that motivation and engagement promote reading performance in general (Guthrie & Wigfield, 2000; Guthrie, Wigfield, Metsala, & Cox, 1999; Guthrie, Wigfield, & You, 2012); if SPT skills increase students' likelihood of seeing the relevance of classroom topics and texts to their own lives, this could well explain the positive relation between SPT and test scores. The finding that social perspective articulation significantly predicts reading and classroom engagement whereas positioning does not deserves further study. Perhaps being able to recognize and articulate perspectives of others (teachers, classmates) in the classroom, as well as of characters in academic texts, is critical for students to be motivated to engage in academic activities and reading. Fostering students' recognition of characters' perspectives is an explicit goal for ELA teachers, but understanding why characters in literature act as they do is not so widely emphasized in instruction for middle school students. Second, we find that both ELA and math test scores are uniquely associated with social perspective positioning, but not with articulation, over and above the prior-year test scores and student demographic characteristics. This association between positioning and academic competence may be mediated by the demand for SPT in the texts students encounter in late elementary and middle grade classrooms – texts that require sophisticated understanding of positional differences as a basis for inferring and explaining the intentions of different agents, characters, and/or authors. It is well established that strong SPT performance is critical to successful academic performance for linguistically-intensive subjects, such as ELA and history (Donahue, 2014; Gehlbach, 2011; Grant, 2001). For example, some social studies instruction may emphasize the development of historical empathy (Endacott & Brooks, 2013), a skill closely related to positioning. Even mathematics, which is typically thought of as a less linguistically intensive subject, is often assessed using word problems, test items that can be socially contextualized and linguistically complex. Think-aloud studies of standardized achievement tests suggest that difficulties understanding the language of such texts can depress performance for students who may actually possess skills to perform the required mathematical problem-solving (e.g., Martiniello, 2008). The associations between SPT and classroom learning found in this study point to possible benefits of integrating the teaching and reinforcement of SPT

Limitations and future research The findings from this research should be interpreted with consideration of the limitations inherent in the exploratory nature of the study and related to several features of the ASPP. First, it is important to note that the ASPP requires performance of SPT skills in short-answer written form, and therefore is likely to underestimate the actual socialcognitive SPT abilities of students who write poorly or are minimally engaged with the assessment. Second, the ASPP does not assess the accuracy of the social inference about others' perspectives, and instead assesses students' performance in demonstrating their own interpretation of perspectives (i.e., by articulating them) with or without providing a rationale and explanation. We contend that this approach is best suited for assessing students' SPT in real social situations that often involve ambiguous moral standings and contradictory social cues. Similar to classroom conversations about academic texts and materials where multiple points of view have valid claims and often enhance learning, the accuracy of SPT is less relevant in such situations, whereas the ability to consider and make inferences about multiple perspectives and to provide justifications and evidence is critical in both negotiating complex social situations and interpreting academic content. In addition, the ASPP focuses on only two types of SPT skills that are salient and easily measurable in upper elementary and middle school classrooms: articulation and positioning. More rudimentary skills that emerge earlier in development (e.g., theory of mind type skills) and more complex skills that require higher-order cognitive ability (e.g., interpretation, critical theorizing) that may mature in late adolescence and beyond are beyond the scope of what the ASPP can measure (Martin et al., 2008). Thirdly, a couple of analytic limitations should be noted. Specifically, because of the sampling design that assigned different forms to different schools, school effects and form effects are not distinguishable in the current study. In addition, the current analysis did not examine potential alternative models beyond the hypothesized models, such as models addressing the possible residual local dependency among items due to the scenario effect, or other competing models that have hierarchical factor structure, such as a bi-factor model or a model with higher-order factors. Lastly, the restricted range of social situations presented in the ASPP scenarios limits the generalizability of this study and conclusions about the SPT skills measured by ASPP. However, a recent study investigating SPT in essays across a wider array of topics found relationships between performance on the ASPP and SPT use in those longer essays (Hsin & Snow, 2017), indicating that performance on the ASPP can be 39

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

generalizable to the display of social perspectives in academic writing where there has been less explicit instruction to engage in SPT acts, and where the topics are less personally inflected. Despite these limitations, the findings from this study show ASPP as a promising measure of SPT performance with robust evidence for its reliability and validity. Future research should consider diverse samples, with purposeful adaptation and expansion of scenarios of ASPP for a contextually, culturally, and developmentally appropriate tool to capture SPT performance for the target population. These adaptations may include use of multiple-form data collection with a planned missingness approach, with common and unique scenarios across different forms, to capture SPT acts in a wider range of social situations, with a careful sampling design. In this case, research should first examine and confirm psychometric properties and confirm cross-form equivalence through multi-group analysis. Once cross-form equivalence is established, concurrent calibration across forms with a planned missing approach to produce factor scores or a single-group measurement model, could be used to answer other empirical questions about SPT acts.

Birch, S. H., & Ladd, G. W. (1997). The teacher-child relationship and children's early school adjustment. Journal of School Psychology, 35, 61–79. https://doi.org/10.1016/ S0022-4405(96)00029-5. Brown, H. M., Oram-Cardy, J., & Johnson, A. (2012). A meta-analysis of the reading comprehension skills of individuals on the autism spectrum. Journal of Autism and Developmental Disorders, 43, 932–955. https://doi.org/10.1007/s10803-012-1638-1. Caprara, G. V., Barbaranelli, C., Pastorelli, C., Bandura, A., & Zimbardo, P. G. (2000). Prosocial foundations of children's academic achievement. Psychological Science, 11, 302–306. https://doi.org/10.1111/1467-9280.00260. Carlson, S. M., & Moses, L. J. (2001). Individual differences in inhibitory control and children's theory of mind. Child Development, 72, 1032–1053. https://doi.org/10. 1111/1467-8624.00333. Chapman, J. G. (1991). The impact of socially projected group composition on behavior in a commons dilemma: A self-attention perspective. Current Psychology, 10, 183–198. https://doi.org/10.1007/BF02686774. Denham, S. A., Bassett, H., Mincic, M., Kalb, S., Way, E., Wyatt, T., & Segal, Y. (2012). Social–emotional learning profiles of preschoolers' early school success: A personcentered approach. Learning and Individual Differences, 22, 178–189. https://doi.org/ 10.1016/j.lindif.2011.05.001. Diazgranados, S., Selman, R. L., & Dionne, M. (2016). Acts of social perspective taking: A functional construct and the validation of a performance measure for early adolescents. Social Development, 25, 572–601. https://doi.org/10.1111/sode.12157. Dodge, K. A. (1986). A social information processing model of social competence in children. In M. Perlmutter (Vol. Ed.), Minnesota symposium on child psychology. Vol. 18. Minnesota symposium on child psychology (pp. 77–125). Mahwah, NJ: Lawrence Erlbaum. Dodge, K. A., Lansford, J. E., Burks, V. S., Bates, J. E., Pettit, G. S., Fontaine, R., & Price, J. M. (2003). Peer rejection and social information-processing factors in the development of aggressive behavior problems in children. Child Development, 74, 374–393. https://doi.org/10.1111/1467-8624.7402004. Donahue, M. L. (2001). Hanging with friends: Making sense of research on peer discourse in children with language and learning disabilities. In K. G. Butler, & E. R. Silliman (Eds.). Speaking, reading, and writing in children with language learning disabilities (pp. 239–258). Mahwah, NJ: Lawrence Erlbaum. Donahue, M. L. (2014). Perspective-taking and reading comprehension of narratives: Lessons learned from “The Bean.”. In C. A. Stone, E. Silliman, B. Ehren, & G. Wallach (Eds.). Handbook of language and literacy: Development and disorders (pp. 323–338). (2nd Ed). New York, NY: Guilford Press. Duhaylongsod, L., Snow, C. E., Selman, R. L., & Donovan, M. S. (2015). Toward disciplinary literacy: Dilemmas and challenges in designing history curriculum to support middle school students. Harvard Educational Review, 85, 587–608. https://doi. org/10.17763/0017-8055.85.4.587. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. https://doi. org/10.1007/s10464-008-9165-0. Endacott, J., & Brooks, S. (2013). An updated theoretical and practical model for promoting historical empathy. Social Studies Research and Practice, 8, 41–57. Retrieved from http://www.socstrpr.org/wp-content/uploads/2013/04/MS_06482_no3.pdf. Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling: A Multidisciplinary Journal, 16, 625–641. https://doi. org/10.1080/10705510903203573. Gehlbach, H. (2004a). A New perspective on perspective taking: A multidimensional approach to conceptualizing an aptitude. Educational Psychology Review, 16, 207–234. https://doi.org/10.1023/B:EDPR.0000034021.12899.11. Gehlbach, H. (2004b). Social perspective taking: A facilitating aptitude for conflict resolution, historical empathy, and social studies achievement. Theory & Research in Social Education, 32, 39–55. https://doi.org/10.1080/00933104.2004.10473242. Gehlbach, H. (2011). Making social studies social: Engaging students through different forms of social perspective taking. Theory Into Practice, 50, 311–318. https://doi.org/ 10.1080/00405841.2011.607394. Gehlbach, H. (2017). Learning to walk in another's shoes. Phi Delta Kappan, 98, 8–12. https://doi.org/10.1177/0031721717696471. Gehlbach, H., Brinkworth, M. E., & Wang, M.-T. (2012). The social perspective taking process: What motivates individuals to take another's perspective? Teachers College Record, 114, 197–225. Retrieved from http://nrs.harvard.edu/urn-3:HUL. InstRepos:11393841. Goldman, S., & Snow, C. E. (2015). Adolescent literacy: Development and instruction. In A. Pollatsek, & R. Treiman (Eds.). Handbook on reading (pp. 463–478). New York, NY: Oxford University Press. Grant, S. G. (2001). It's just the facts, or is it? The relationship between teachers' practices and students' understandings of history. Theory & Research in Social Education, 29, 65–108. https://doi.org/10.1080/00933104.2001.10505930. Guthrie, J., Wigfield, A., Metsala, J., & Cox, K. (1999). Motivational and cognitive predictors of text comprehension and reading amount. Scientific Studies of Reading, 3, 231–256. https://doi.org/10.1207/s1532799xssr0303_3. Guthrie, J. T., & Wigfield, A. (2000). Engagement and motivation in reading. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Vol. Eds.), Handbook of reading research. Vol. 3. Handbook of reading research (pp. 403–422). New York, NY: Erlbaum. Guthrie, J. T., Wigfield, A., & You, W. (2012). Instructional contexts for engagement and achievement in reading. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.). Handbook of research on student engagement (pp. 601–634). New York, NY: Springer. Hall, J. A., Andrzejewski, S. A., & Yopchick, J. E. (2009). Psychosocial correlates of interpersonal sensitivity: A meta-analysis. Journal of Nonverbal Behavior, 33, 149–180. https://doi.org/10.1007/s10919-009-0070-5.

Conclusion This article focused on conceptualizing, measuring, and examining SPT acts as a construct relevant to academic competence and engagement with a diverse sample of early adolescents. Students in middle and secondary grades are often called upon to infer the perspectives of characters in novels, to consider the perspectives of historical actors, to understand the epistemological positioning of science reporting, and to engage in classroom discussion that may require juggling peers' and teachers' views about issues and integrating them with one's own. In light of the new demands for literacy and argumentation posed by contemporary learning standards, it is encouraging that we find relationships between academic performance and SPT acts, the latter measurable by the accessible ASPP assessment and, with further development of SPT-focused curricula, transferable to teaching practice. Acknowledgments The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305F100026 to the Strategic Educational Research Partnership Institute as part of the Reading for Understanding Research Initiative. We would like to thank the administrators, teachers, and students who made this important work possible. We would also like to thank Dr. Andrew Ho for his comments on this paper, Silvia Diazgranados and Michelle Dionne for their work conceptualizing perspective-taking performance, and our dedicated and skilled research staff who worked tirelessly, over several years, on this project. References Ames, D. R., & Kammrath, L. K. (2004). Mind-reading and metacognition: Narcissism, not actual competence, predicts self-estimated ability. Journal of Nonverbal Behavior, 28, 187–209. https://doi.org/10.1023/B:JONB.0000039649.20015.0e. Apperly, I. A., & Robinson, E. J. (2003). When can children handle referential opacity? Evidence for systematic variation in 5- and 6-year-old children's reasoning about beliefs and belief reports. Journal of Experimental Child Psychology, 85, 297–311. https://doi.org/10.1016/S0022-0965(03)00099-7. Asparouhov, T., & Muthén, B. (2010). Weighted least squares estimation with missing data. Mplus technical appendix. Retrieved from https://www.statmodel.com/ download/GstrucMissingRevision.pdf. Austin, J. L. (1962). How to do things with words. Cambridge, MA: Harvard Univesity Press. Baker, F. B. (2001). The basics of item response theory (2nd ed.). College Park, MD: ERIC Clearinghouse on Assessment and Evaluation. Retrieved from http://files.eric.ed. gov/fulltext/ED458219. Barr, D., Boulay, B., Selman, R. L., McCormick, R., Lowenstein, E., Gamse, B., ... Brielle Leonard, M. (2015). A randomized controlled trial of professional development for interdisciplinary civic education: Impacts on humanities teachers and their students. Teachers College Record, 117, 1–52. Retrieved from http://www.tcrecord.org/ Content.asp?ContentId=17470.

40

Journal of Applied Developmental Psychology 57 (2018) 24–41

H.Y. Kim et al.

between preschool social–emotional competence and first grade academic achievement: The role of attention skills. Early Childhood Research Quarterly, 26, 182–191. https://doi.org/10.1016/j.ecresq.2010.07.003. Ricketts, J., Jones, C., Happé, F., & Charman, T. (2013). Reading comprehension in autism spectrum disorders: The role of oral language and social functioning. Journal of Autism and Developmental Disorders, 43, 807–816. https://doi.org/10.1007/ s10803-012-1619-4. Ryan, R., & Deci, E. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68–78. Ryskin, R. A., Benjamin, A. S., Tullis, J., & Brown-Schmidt, S. (2015). Perspective-taking in comprehension, production, and memory: An individual differences approach. Journal of Experimental Psychology: General, 144, 898–915. https://doi.org/10.1037/ xge0000093. Schlitz, M. M., Vieten, C., Miller, E., Homer, K., Petersen, K., & Erickson-Freeman, K. (2011). The worldview literacy project: Exploring new capacities for the 21st century student. New Horizons for Learning, 9. Retrieved from http://jhepp.library.jhu.edu/ ojs/index.php/newhorizons/article/view/37. Schultz, L. H., Selman, R. L., & Larusso, M. D. (2003). The assessment of psychosocial maturity in children and adolescents: Implications for the evaluation of school-based character education programs. Journal of Research in Character Education, 1, 67–87. Retrieved from https://search.proquest.com/docview/231749985?accountid= 12768. Selman, R. L. (1980). The growth of interpersonal understanding. New York, NY: Academic Press. Selman, R. L. (2007). The promotion of social awareness: powerful lessons from the partnership of developmental theory and classroom practice. New York, NY: Russell Sage Foundation. Selman, R.L., Diazgranados, S., Dionne, M. (n.d.). [Assessment coding manual]. Codebook for the Assessment of Acts of Social Perspective Taking in the students as advisors: Social situations in schools. Retrieved from https://www.researchgate.net/project/ The-Promotion-of-Social-Awareness-across-Cultures. Selman, R. L., & Feigenberg, L. F. (2010). Between neurons and neighborhoods: Innovative methods to assess the development and depth of adolescent social awareness. In P. Zelazo, E. Crone, & M. Chandler (Eds.). Developmental social cognitive neuroscience (pp. 227–247). New York, NY: Routledge. Shantz, C. U. (1982). Children's understanding of social rules and the social context. In F. C. Serafica (Ed.). Social-cognitive development in context (pp. 167–198). New York, NY: Psychology Press. Snow, C. E., & Uccelli, P. (2009). The challenge of academic language. In D. R. Olson, & N. Torrance (Eds.). The Cambridge handbook of literacy (pp. 112–133). New York, NY: Cambridge University Press. Takacs, D. (2003). How does your positionality bias your epistemology? Thought & Action, 19, 27–38. Retrieved from http://repository.uchastings.edu/faculty_scholarship/ 1264. Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408. https://doi.org/ 10.1007/BF02294363. Webster-Stratton, C., & Reid, M. J. (2004). Strengthening social and emotional competence in young children-the foundation for early school readiness and success: Incredible Years classroom social skills and problem-solving curriculum. Infants & Young Children, 17, 96–113. Retrieved from http://journals.lww.com/iycjournal/ Abstract/2004/04000/Strengthening_Social_and_Emotional_Competence_in.2.aspx. Wellborn, J. G., & Connell, J. P. (1987). Manual for the Rochester Assessment Package for Schools. Rochester, NY: University of Rochester. Wellman, H. M., Cross, D., & Watson, J. (2001). Meta-analysis of theory-of-mind development: The truth about false belief. Child Development, 72, 655–684. https://doi. org/10.1111/1467-8624.00304. Wigfield, A., Guthrie, J. T., Perencevich, K. C., Taboada, A., Klauda, S. L., McRae, A., & Barbosa, P. (2008). Role of reading engagement in mediating the effects of reading comprehension instruction on reading outcomes. Psychology in the Schools, 45, 432–445. https://doi.org/10.1002/pits.20307. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79. https://doi.org/10.1037/1082-989X.12. 1.58. Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2007). The scientific base linking social and emotional learning to school success. Journal of Educational and Psychological Consultation, 17, 191–210. https://doi.org/10.1080/ 10474410701413145.

Hsin, L., & Snow, C. (2017). Social perspective taking: a benefit of bilingualism in academic writing. Reading and Writing, 30, 1193–1214. https://doi.org/10.1007/ s11145-016-9718-9. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. https://doi.org/10.1080/10705519909540118. Hulme, S., Mitchell, P., & Wood, D. (2003). Six-year-olds' difficulties handling intensional contexts. Cognition, 87, 73–99. https://doi.org/10.1016/s0010-0277(02)00199-3. Johnson, D. W. (1975). Cooperativeness and social perspective taking. Journal of Personality and Social Psychology, 31, 241–244. Jones, S. M., & Bouffard, S. M. (2012). Social and emotional learning in schools: From programs to strategies. Social Policy Report, 26, 1–22. Retrieved from https://eric.ed. gov/?id=ED540203. Jones, S. M., Kim, J., LaRusso, M. D., Kim, H. Y., Selman, R. L., Uccelli, P., … Snow, C. E. (in press). Experimental effects of Word Generation on vocabulary, academic language, perspective taking, and reading comprehension in high poverty middle schools. Journal of Research in Educational Effectiveness. Keen, S. (2007). Empathy and the novel. Oxford, United Kingdom: Oxford University Press. Ladd, G. W., Birch, S. H., & Buhs, E. S. (1999). Children's social and scholastic lives in kindergarten: Related spheres of influence? Child Development, 70, 1373–1400. https://doi.org/10.1111/1467-8624.00101. Lane, K. L., Givner, C. C., & Pierson, M. R. (2004). Teacher expectations of student behavior social skills necessary for success in elementary school classrooms. The Journal of Special Education, 38, 104–110. https://doi.org/10.1177/ 00224669040380020401. LaRusso, M., Kim, H. Y., Selman, R., Uccelli, P., Dawson, T., Jones, S., & Snow, C. (2016). Contributions of academic language, perspective taking, and complex reasoning to deep reading comprehension. Journal of Research on Educational Effectiveness, 9, 201–222. https://doi.org/10.1080/19345747.2015.1116035. LaRusso, M., & Selman, R. (2011). Early adolescent health risk behaviors, conflict resolution strategies, and school climate. Journal of Applied Developmental Psychology, 32, 354–362. https://doi.org/10.1016/j.appdev.2011.05.003. Light, P. (1979). The development of social sensitivity: A study of social aspects of role-taking in young children. Cambridge, United Kingdom: Cambridge University Press. Lord, F. (1952). A theory of test scores. New York, NY: Psychometric Society. Lord, F. M., & Novick, M. R. (2008). Statistical theories of mental test scores. Reading, MA: Addison-Wesley. Martin, J., Sokol, B. W., & Elfers, T. (2008). Taking and coordinating perspectives: From prereflective interactivity, through reflective intersubjectivity, to metareflective sociality. Human Development, 51, 294–317. https://doi.org/10.1159/000170892. Martiniello, M. (2008). Language and the performance of English-language learners in math word problems. Harvard Educational Review, 78, 333–368. https://doi.org/10. 17763/haer.78.2.70783570r1111t32. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum. Mislevy, R. J. (1985). Recent developments in the factor analysis of categorical variables. ETS Research Report Series, 67. https://doi.org/10.1002/j.2330-8516.1985. tb00109.x. Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus user's guide (7th Edition). Los Angeles, CA: Muthén & Muthén. Olson, C. B., & Land, R. (2007). A Cognitive strategies approach to reading and writing instruction for English language learners in secondary school. Research in the Teaching of English, 41, 269–303. Retrieved from https://eric.ed.gov/?id=EJ776476. Strategic Education Research Partnership. (n.d.). Catalyzing comprehension through discussion and debate. Retrieved from http://ccdd.serpmedia.org Pelletier, J. (2006). Relations among theory of mind, metacognitive language, reading skills and story comprehension In L1 and L2 learners. In A. Antonietti, O. LivertaSempio, & A. Marchetti (Eds.). Theory of mind and language in developmental contexts (pp. 77–92). New York, NY: Springer. Piaget, J. (1951). The child's conception of the world. Lanham, MD: Rowman & Littlefield. Pianta, R., Steinberg, M., & Rollins, K. (1995). The first two years of school: Teacher-child relationships and deflections in children's classroom adjustment. Development and Psychopathology, 7, 295–312. https://doi.org/10.1017/S0954579400006519. Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21, 173–184. https://doi.org/10.1177/ 01466216970212006. Realo, A., Allik, J., Nõlvak, A., Valk, R., Ruus, T., Schmidt, M., & Eilola, T. (2003). Mindreading ability: Beliefs and performance. Journal of Research in Personality, 37, 420–445. https://doi.org/10.1016/S0092-6566(03)00021-7. Rhoades, B., Warren, H., Domitrovich, C., & Greenberg, M. (2011). Examining the link

41