Executive functioning difficulties as predictors of academic performance: Examining the role of grade goals

Learning and Individual Differences 36 (2014) 19–26

Contents lists available at ScienceDirect

Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif

Executive functioning difficulties as predictors of academic performance: Examining the role of grade goals Laura E. Knouse a,⁎, Greg Feldman b, Emily J. Blevins a,1 a b

University of Richmond, 28 Westhampton Way, Richmond, VA 23173, USA Simmons College, 300 Fenway, Boston, MA 02115, USA

a r t i c l e

i n f o

Article history: Received 5 November 2013 Received in revised form 19 May 2014 Accepted 18 July 2014 Available online xxxx Keywords: Executive functioning Grade point average Goal setting

a b s t r a c t Concepts related to self-regulation have emerged repeatedly in research on college academic achievement. We hypothesized that self-reported executive functioning (EF) deficits would predict academic performance and investigated whether grade goals could account for this relationship. In Study 1 we obtained data on the Barkley Deficits in Executive Functioning Scale (BDEFS) and self-reported GPA from co-educational university students (N = 250). In Study 2, we collected BDEFS and GPA goals from students at a women’s college (N = 229) and obtained grades from the registrar. EF deficits predicted GPA concurrently and prospectively even when controlling for prior grades. Self-motivation problems were most consistently related to grades and mediation analysis revealed a significant indirect effect via lower grade goals. However, goals did not fully account for this relationship. While our results suggest potential value for goal-setting interventions, additional measures to improve self-regulation are likely needed to help struggling students with self-motivation problems. © 2014 Elsevier Inc. All rights reserved.

1. Introduction 1.1. Background For decades, educational and social science researchers have been interested in identifying factors that predict success in college, as measured by academic performance or grade point average (GPA), in order to improve students' academic outcomes (Crede & Kuncel, 2008). While high school GPA and scores on standardized tests like the SAT and ACT are the most frequently used predictors of college success, they only account for about 25% of the variance in college GPA, motivating investigations of so-called “third factors” as predictors of academic performance (Robbins et al., 2004). Several such “third factors” have received support in the literature including achievement motivation and academic self-efficacy (Robbins et al., 2004), study habits, skills, and attitudes (Crede & Kuncel, 2008) and personality traits—in particular, conscientiousness and self-control (Conard, 2005; Noftle & Robins, 2007; O'Connor & Paunonen, 2007; Poropat, 2009; Tangney,

⁎ Corresponding author at: University of Richmond Department of Psychology, 28 Westhampton Way, University of Richmond, VA 23221, USA. Tel.: +1 804 287 6347; fax: +1 804 287 1905. E-mail addresses: [email protected] (L.E. Knouse), [email protected] (G. Feldman), [email protected] (E.J. Blevins). 1 Present address: University of Maryland, 2103 Cole Fieldhouse, College Park, MD 20742, USA.

http://dx.doi.org/10.1016/j.lindif.2014.07.001 1041-6080/© 2014 Elsevier Inc. All rights reserved.

Baumeister, & Boone, 2004). Most recently, Richardson, Abraham, and Bond (2012) conducted a comprehensive meta-analysis and identified students' self-selected goals for their own grades, academic selfefficacy, and effort regulation as the most consistent predictors of GPA out of 42 psychological factors. Because “third factors” may be useful in identifying which recently-admitted students may be in need of additional services to improve academic outcomes and retention (Richardson et al., 2012), there is a need for additional research on predictors of low GPA and changes in GPA over time that can be the target of interventions. Executive function (EF)—defined as self-regulation to achieve future goals—has far-ranging impact on daily functioning and quality of life and encompasses a broad array of self-directed cognitions and actions including problem-solving, working memory, impulse control, selfmotivation, and emotion regulation (Barkley, 2012a, 2012b). Executive functioning in daily life is an excellent candidate for a “third factor” that would meaningfully predict college GPA. First, successful functioning in college requires extensive self-regulation above and beyond what is required in high school. From class attendance to self-structured studying to balancing multiple academic and non-academic pursuits, college students must self-regulate their own learning in the face of distraction. Second, educational and developmental research has emphasized the importance of EF to successful academic functioning at earlier points in the developmental trajectory, from preschool through adolescence (Biederman et al., 2004; Duckworth, Tsukayama, & May, 2010; St. Clair-Thompson & Gathercole, 2006; Valiente, Lemery-Chalfant, & Swanson, 2010). Third, a few studies have begun to identify a link

20

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

between academic functioning in young adults and laboratory tasks tapping key aspects of executive cognition including working memory and delay discounting or the willingness to wait for larger long-term rewards (Cowan et al., 2005; Gropper & Tannock, 2009; Kirby, Winston, & Santiesteban, 2005). The present study contributes to this research by examining EF deficits evident in daily life with a brief multi-dimensional selfreport measure called the Barkley Deficits in Executive Functioning Scale—Short Form (BDEFS, Barkley, 2012a). Although EF is often assessed via laboratory tasks, the ecological validity of these tasks is often poor (Barkley & Murphy, 2011) and rating scales of EF have been shown to out-perform lab tasks in predicting outcomes including occupational functioning (Barkley & Fischer, 2011; Barkley & Murphy, 2010), depression symptoms and diagnoses (Knouse, Barkley, & Murphy, 2013), and an array of outcomes in college students (Wingo, Kalkut, Tuminello, Asconape, & Han, 2013). Importantly, the BDEFS allows an examination of the unique contribution of distinct aspects of EF to academic performance. It consists of five subscales measuring deficits in various domains of executive functioning: self-management to time (procrastination), self-organization (information-processing inefficiency), self-restraint (impulsivity), self-motivation (difficulty with sustained effort), and self-regulation of emotions (delayed emotional recovery following stressor). These facets of EF assessed by the BDEFS conceptually align with constructs that have been previously linked to GPA. In particular, the role of self-motivation with respect to academic tasks has emerged in several studies. Richardson et al.'s (2012) metaanalysis identified effort regulation, defined as the ability to maintain effort in the face of academic challenge, as one of the most strongly correlated with academic performance (r = .32) even when taking into account high school GPA and standardized test scores (β = .22). Personality studies show that achievement striving and self-discipline, akin to self-motivation, are the facets of conscientiousness with the strongest and most consistent relationships to GPA (O'Connor & Paunonen, 2007). Other constructs measured by the BDEFS may also be expected to predict GPA based on prior research. The relevance of the selfmanagement to time construct is supported by prior studies on the role of procrastination in GPA (Richardson et al., 2012). The relevance of self-organization and self-restraint is conceptually supported by research on working memory and delay discounting (Cowan et al., 2005; Gropper & Tannock, 2009; Kirby et al., 2005). In contrast, self-regulation of emotions might not be expected to correlate as strongly with GPA. Emotion dysregulation is a facet of EF has been consistently linked to psychopathology in college student samples (Aldao, Nolen-Hoeksema, & Schweizer, 2010; Feldman, Knouse, & Robinson, 2013); however, the effects of emotional distress per se on GPA are less clear. For instance, neuroticism and symptoms of depression tend to exhibit weak and inconsistent associations with GPA (Richardson et al., 2012). Taken together, the present study can help to clarify which facets of EF suggested by prior research are most strongly linked to GPA. 1.2. Study aims The first aim of our study was to evaluate whether the BDEFS rating scale measure of EF deficits in daily life would predict cumulative college GPA concurrently. In Study 1, we examined the association of BDEFS with self-reported GPA in a large sample of students at a coeducational institution to establish whether EF deficits were associated with academic performance. In Study 2 we examined the association to grades obtained from the registrar in a sample of students at a women's college. Validation research of the BDEFS in a nationally representative sample of adults (Barkley, 2012a) found that the scale was not strongly correlated with intelligence or

academic ability but was moderately associated with educational attainment and self-reported history of impairment in academic functioning. Similarly, a separate study of 77 female college students (Wingo et al., 2013) found that a different self-report measure of EF was associated with self-rated academic impairment. However, we are not aware of any study that has yet assessed the association of self-ratings of EF deficits with actual grades. Furthermore, we tested the replicability of cross-sectional findings across samples of students at two colleges with distinct academic profiles. Our second aim, tested in Study 2, was to test whether concurrent associations would be replicated in prospective analyses. Research to date on the BDEFS and earlier versions of the measure have focused on its association with outcomes assessed concurrently and retrospectively (Barkley, 2012a). Prospective analyses are an important step in the validation of any individual difference measure, but particularly this one given the centrality of self-regulation in the service of future goals and desired states emphasized in the conceptualization of EF that informed the development of this measure (Barkley, 2012a). Relatedly, our third aim, tested in both Studies 1 and 2, was to examine unique contributions of each of the five BDEFS subscales to the prediction of current and future GPA. Although the subscales are highly correlated with one another (Barkley, 2012a), as described above, particular subscales might be expected to better predict college academic performance (e.g., effort regulation) and so we wished to examine whether any subscales appeared to uniquely predict GPA. Our fourth and fifth aims, addressed in Study 2, involved further examining the relationship between EF deficits and grades using goalsetting theory. Because the EF construct emphasizes self-regulation in the service of future goals, we reasoned that the goals that students set for themselves might be an important factor in the relationship between EF deficits and GPA. The recent comprehensive meta-analysis by Richardson et al. (2012) identified “grade goal” as one of the three “non-intellective” constructs with the strongest correlation to college grades. Grade goal (r = .35) included students' self-reports of their goal grades or expected grades on an assignment or in a course. Grade goal continued to predict college grades above and beyond high school GPA and standardized test scores (β = .17). Notably, the studies cited in Richardson et al. (2012) and others published previously (Locke & Bryan, 1968; Wood & Locke, 1987) show an association between grade goal and later grade in a single course but none provide data on whether grade goal predicts semester GPA. Thus, our fourth aim was to test whether students' self-selected grade goals correlated with later semester GPA. For our fifth aim, we were interested in examining whether lower grade goals account for (mediate) the relationship between EF deficits and academic performance. This hypothesis emerges from Locke and Latham's (1990) goal-setting theory. Goal-setting theory, a theory of motivation, states that goals regulate behavior such that difficult goals lead to greater persistence and effort than easy goals or vague encouragement to “do your best,” and thus lead to better task performance. Goals have been found to mediate or partially mediate the effects of other variables, including personality traits, on performance (Locke & Latham, 2006). Students' grade goals, for example, have been shown to partially mediate the impact of goal orientation and personality characteristics derived from self-determination theory on academic performance (Lee, Sheldon, & Turban, 2003; VandeWalle, Cron, & Slocum, 2001). We are not aware of any studies that have tested goal-setting as a mediator between EF deficits and academic performance. Thus, based on prior research and the importance of goal-directed action in the EF construct, we hypothesize that students with EF deficits set lower goals and that this contributes to poorer academic performance. Importantly, when testing the possible mediating role of self-selected goals on later semester GPA, we took into account students' prior college performance (cumulative GPA from the previous semester). Students' grade goals are strongly related to their past performance

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

(Wood & Locke, 1987) and thus taking prior semester performance into account provides a rigorous test of the impact of grade goal and enhances the applicability of our results to possible interventions. In sum, we tested the following hypotheses: 1) EF deficits as measured by the BDEFS will predict lower GPA in cross-sectional analyses in two different college samples (Studies 1 and 2). 2) EF deficits will predict lower GPA in prospective analyses, above and beyond prior academic performance (Study 2). 3) Self-management to time and self-motivation deficits will be the most strongly and uniquely related to GPA across the analyses (Studies 1 and 2). 4) Students' self-selected grade goals will correlate with their subsequent academic performance (semester GPA; Study 2). 5) Grade goals will mediate the relationship between EF deficits and subsequent GPA when controlling for past academic performance (cumulative GPA; Study 2). 2. Study 1 2.1. Method 2.1.1. Participants and procedure A sample of 314 undergraduates was recruited over two semesters at a small, private co-educational university in the Southeastern U.S. that is classified as a National Liberal Arts College and rated as “more selective” (U.S. News and World Report Best Colleges Ranking, 2013). The sample analyzed for this study (N = 250) included those who had a self-reported cumulative college GPA and complete data for the BDEFS. 56 participants were first semester students who were excluded from analyses because they had not yet received grades in college. An additional 6 participants did not answer this question. The participants who did not answer the GPA question did not differ from those retained for analyses on any of the BDEFS scales. Two additional participants with GPA data did not complete the BDEFS. Of the 250 undergraduate students who had complete data (Age: M = 19.76, SD = 1.13), 71.6% identified as White, 11.2% as Asian or Pacific Islander, 15.2% as Black or African-American, 4% as Native American, and 9.6% self-identified as “Other.” (Note that participants could indicate more than one race.) 8.8% of participants self-identified as Hispanic. Data on gender were not recorded for 22% participants due to a software programming error. Of those who were administered the question assessing gender, 31.3% were male and 68.7% were female. Participants were recruited through a variety of means including introductory psychology students who received course credit for participation (28%) as well as students who were recruited from campus (72%) who received $5 in exchange for participation. All data collection procedures were approved by the relevant Institutional Review Board and participants completed informed consent procedures before participating. 2.1.2. Measures 2.1.2.1. Barkley Deficits in Executive Functioning Scale—Short Form (BDEFS; Barkley, 2012a). This 20-item self-report scale measures executive functioning problems in daily life. The scale is designed to tap perceived problems with self-regulatory behavior or “executive action,” which is related to but distinct from “executive cognition” as tapped by laboratory tasks of executive functioning (Barkley, 2012b). Participants rate the frequency with which they have experienced each problem over the past six months on a four-point scale from “Never or Rarely” to “Very Often.” This short form of the scale comprises five subscales of four items each tapping deficits in self-management to time

21

(procrastination/poor planning), self-organization/problem solving (information processing difficulties/cognitive inflexibility), self-restraint (impulsivity), self-motivation (low/inconsistent effort and work quality), and self-regulation of emotions (delayed recovery from negative emotions). Subscale items are those that loaded most strongly on each factor from the BDEFS Long Form. The BDEFS was normed on a large nationally representative sample of adults (n = 1240) and has demonstrated reliability and validity (Barkley, 2012a). Internal consistency of the overall BDEFS scale and subscales in this sample was acceptable: Total score (α = .87), self-management to time (α = .78), self-organization and problem-solving (α = .73), self-restraint (α = .73), self-motivation (α = .78), and self-regulation of emotions (α = .90). 2.1.2.2. Grade point average (GPA). Cumulative college GPA was self-reported by participants at the time of study participation. 2.2. Results The association between difficulties in executive functioning total score and subscale scores and cumulative college GPA was first assessed with zero-order correlations (Table 1, first column). Deficits in self-motivation, self-management to time, self-organization and the BDEFS total score were significantly negatively correlated with cumulative GPA. Self-restraint was also negatively associated with GPA, although less strongly so. Unexpectedly, deficits in emotion regulation showed a small but significant positive association with cumulative GPA. To evaluate unique contributions of each BDEFS subscale, we conducted a multiple regression analysis with all subscales entered simultaneously as predictors (Table 2, first column). Self-motivation problems emerged as the strongest negative predictor of GPA, while self-regulation of emotions continued to show a positive relationship in the context of the other subscales. Weaker, but still significant, was the negative association between self-organization and problemsolving deficits and GPA. 2.3. Discussion Study 1 initially established that EF deficits are associated with academic performance of college students. In this sample, executive functioning deficits as measured by the BDEFS total score was negatively correlated with self-reported GPA (r = −.18). Our analyses, however, support using BDEFS subscales rather than the total score, as individual subscales were more strongly related to GPA. Of note, one subscale (emotion regulation) positively predicted GPA, which may have weakened the negative total score association. The subscale measuring self-motivation problems showed the strongest and most consistent negative association with GPA across zero-order correlations (r = − .32) and the regression analysis controlling for the other subscales (β = − .25). As mentioned previously, the construct of effort regulation emerged as one of the strongest predictors of college grades in the meta-analysis by Richardson et al. (2012) and thus the results of Study 1 conceptually replicate this finding and confirm that students' ability to maintain consistent effort towards goals is an important predictor of academic achievement in college. Self-organization and problem-solving deficits were also negatively associated with GPA in both sets of analyses. The items in this scale tap difficulties with efficient information processing and cognitive flexibility. Managing diverse demands across a variety of courses may be particularly difficult for students with these types of difficulties. In contrast to the other scales, emotion regulation problems, tapping the difficulty in recovering from negative emotions when upset, showed a positive association with self-reported GPA. This finding was puzzling given that prior studies do not support a consistent association between GPA and the related construct of neuroticism. However, a recent study

22

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

Table 1 Association of BDEFS subscales and total scores to GPA in Samples 1 and 2.

BDEFS Time Organization Self-restraint Motivation Emotion Total

Sample 1 N = 250, GPA self-reported (cumulative) R

Sample 2 N = 229, GPA data obtained from the registrar T1 GPA (cumulative) R

T2 GPA (semester) r

T2 GPA (controlling for T1) pr

−0.23⁎⁎⁎ −0.22⁎⁎⁎ −0.13⁎ −0.32⁎⁎⁎ 0.14⁎ −0.18⁎⁎

−0.26⁎⁎⁎ −0.10 −0.14⁎ −0.27⁎⁎⁎

−0.34⁎⁎⁎ −0.12 −0.16⁎ −0.39⁎⁎⁎

−0.22⁎⁎ 0.06 −0.08 −0.29⁎⁎⁎

0.12 −0.18⁎⁎

0.00 −0.29⁎⁎⁎

−0.12 −0.23⁎⁎

BDEFS = Barkley Deficits in Executive Functioning Scale, GPA = grade point average. ⁎ p b .05. ⁎⁎ p b .01. ⁎⁎⁎ p b .001.

by De Feyter, Caers, Vigna, and Berings (2012) suggests that neuroticism may positively predict academic performance at higher levels of conscientiousness and self-efficacy. If this describes the college sample in Study 1, then the positive relationship between emotion regulation problems and GPA may not generalize to other samples. We examined whether this finding replicated by conducting the same analyses in Study 2. Study 1 provided preliminary evidence for links between academic performance in college and executive functioning deficits but this study had a number of limitations. One limitation is the use of self-reported GPA. Self-reported GPA is commonly used in studies of correlates of college academic achievement (Kuncel, Credé, & Thomas, 2005); furthermore, there is evidence that selfreport GPA is highly correlated with grades obtained directly from a college registrar (Noftle & Robins, 2007). Nonetheless, there is some concern that self-reported GPA may be systematically biased. Specifically, students with lower prior achievement (as measured by college entrance exams) are often less accurate in grade recall than those with higher prior achievement (Kuncel et al., 2005). A second limitation of Study 1 was the use of a cross-sectional design which in effect tests the retrospective association between academic performance in the prior semesters with EF rated in the subsequent semester. As such, the goals of Study 2 included both replicating and extending the results of Study 1 using GPA data obtained from a registrar and a prospective design. 3. Study 2 The first goal of Study 2 was to determine whether key findings on the association EF deficits with self-reported GPA from Study 1 would

replicate in a new sample of undergraduate students using grade data obtained from the registrar. The second goal was to evaluate the association of the BDEFS and its subscales with GPA prospectively, controlling for prior college performance (Time 1 cumulative GPA). The third goal of Study 2 was to examine goal-setting as a possible mediator of the effects of self-motivation problems—the most robust EF predictor of GPA emerging from both studies. 3.1. Method 3.1.1. Participants and procedure Data in Study 2 were collected over two semesters from undergraduate students attending a small, private women's college in the Northeastern U.S. classified as a Regional University and rated “selective” (U.S. News and World Report Best Colleges Ranking, 2013). Students participated in this study in exchange for credit applied towards a psychology course in which they were enrolled. After completing an in-person survey in a laboratory session, participants were asked to provide consent for the college Office of the Registrar to release to investigators the participant's cumulative GPA (as of the prior semester) and semester GPA at the end of the semester once final grades had been submitted. Questionnaire data was collected five-to-eight-weeks before the completion of final exams from 302 students. Permission was denied by 46 participants and granted by 256. Of those granting permission, 27 were first-semester students who did not have a prior semester cumulative GPA and were excluded from analyses, resulting in a final sample of 229 participants (Age: M = 19.98, SD = 2.75). In terms of ethnicity, 77.7% identified as White, 9.2% as Asian or Pacific Islander, 3.1% as Black or African-American, 9.6% circled two or more ethnicities or circled “Other,” and .4% left this item blank. 94.8% identified as

Table 2 Linear regression predicting GPA in Samples 1 and 2. Sample 1 N = 250, GPA self-reported (cumulative) β

BDEFS Time Organization Self-restraint Motivation Emotion

−.12 −.18⁎ .03 −.25⁎⁎ .26⁎⁎⁎

Sample 2 N = 229, GPA data obtained from the registrar T1 GPA (cumulative) R2 = .13 β

T2 GPA (semester) R2 = .17 β

T2 GPA (controlling for T1 cumulative GPA) R2 change = .05 β

−.16 −.04 −.08 −.17⁎ .21⁎⁎

−.16⁎ .02 −.03 −.30⁎⁎⁎ .08

−.06 .04 .02 −.19⁎⁎ −.05

BDEFS = Barkley Deficits in Executive Functioning Scale, GPA = grade point average. ⁎ p b .05. ⁎⁎ p b .01. ⁎⁎⁎ p b .001.

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

non-Hispanic, 4.4% identified as Hispanic, and .9% left this item blank. All participants were female. Students completed this survey in exchange for credit applied towards a psychology course in which they were enrolled. 3.1.2. Measures Participants completed the 20-item BDEFS (Barkley, 2012a) as described in Study 1. Internal consistency of the BDEFS total and subscale scores was acceptable: Total score (α = .86), Self-management to time (α = .80), self-organization and problem-solving (α = .74), selfrestraint (α = .68), self-motivation (α = .72), and self-regulation of emotions (α = .90).

23

emotion regulation problems were again positively associated with GPA. Unique associations between emotion regulation deficits and grades disappeared in the prospective analyses. Problems with time management were negatively associated with Time 2 semester GPA but, again, self-motivation showed the strongest unique association with grades. Notably, the negative association between self-motivation problems and Time 2 grades continued to be significant when Time 1 cumulative GPA was first entered into the model. In sum, across all analyses, self-motivation problems showed the most consistent and robust relationship with past and future GPA.

3.2.2. Goal-setting, self-motivation, and GPA 3.1.2.1. GPA. As described above, cumulative GPA (as of the prior semester) and semester GPA data were obtained from the college Office of the Registrar for participants who provided consent. The group that denied permission to access GPA data did not differ from the 229 participants retained for analyses in terms of any of the BDEFS scales. 3.1.2.2. Grade goals. Students responded to the statement, “This semester, my GOAL for my GPA is:” by selecting one of eight categories of grade ranges from A (4.0 to 3.86) to C− or lower (1.67 or less). We decided to include both letter grades and GPA ranges in each response option given that students may differ in terms of whether they think about their personal grade goals as letter grades (which they would receive on assignments, exams, and course semester grades) or semester grade point averages (which students would typically only see when reviewing their transcript). 3.1.3. Plan of analysis We first repeated the analyses of Study 1 including correlations between cumulative GPA and BDEFS total score and subscale scores and a multiple regression analysis with all subscales entered simultaneously. We also calculated zero-order correlations with Time 2 semester GPA and partial correlations with this variable controlling for Time 1 cumulative GPA (Table 1) and multiple regression analyses predicting these same outcomes with all BDEFS subscales entered simultaneously (Table 2). In considering students' self-selected GPA goals, we first examined correlations between goals and Time 1 cumulative and Time 2 semester GPA to replicate prior work on grade goals. Next, we tested a simple mediation model with GPA goal as the mediator between self-motivation problems and Time 2 semester GPA using Model 4 in PROCESS for SPSS by Hayes (2013). Indirect effects were evaluated by examining whether 95% bias corrected confidence intervals around the estimate, derived from 10,000 bootstrap resamples, included zero. Next, we repeated this analysis controlling for Time 1 cumulative GPA to see whether effects would hold. 3.2. Results 3.2.1. EF deficits predicting GPA Deficits in self-management to time and self-motivation, as well as the BDEFS total score, were significantly associated with cumulative Time 1 GPA and prospectively with Time 2 semester GPA in zero-order correlations and in partial correlations controlling for Time 1 cumulative GPA (Table 1). Self-restraint was also associated with GPA concurrently and prospectively, but not when controlling for prior cumulative GPA. Although not significant, the positive association between deficits in emotion regulation and cumulative GPA observed in Study 1 was replicated in Study 2, yet the prospective association was not significant and actually became a negative, non-significant relationship after controlling for prior cumulative GPA. Follow-up multiple regression analysis evaluating unique contributions of BDEFS subscales (Table 2) showed that self-motivation problems were negatively associated with Time 1 cumulative GPA and

3.2.2.1. Goals and grades. We first examined the relationship between students' GPA goals and grades in order to replicate Richardson et al.'s (2012) findings on grade goal and academic performance. As expected, zero-order correlations of students' GPA goals with both past and subsequent performance were significant and strong (Time 1 cumulative GPA r = .69; Time 2 semester GPA r = .60; both p b .001). Self-motivation deficits were negatively correlated with GPA goal, r = −.28; p b .001. Thus, students with better past performance set higher goals and students with more self-motivation problems set lower goals.

3.2.2.2. GPA goal as a mediator between self-motivation problems and GPA. Illustrated in Fig. 1, mediation analysis using PROCESS Model 4 showed that self-motivation problems had a direct negative impact on GPA (−.07, p b .001). In addition, the indirect effect of self-motivation problems on GPA via lower goals was also significant and negative (− .04; confidence interval of indirect effect = −.07 to −.02; 10,000 bootstrap resamples used). Thus, the negative impact of self-motivation problems on later grades was both direct and indirect via an association with lower grade goals. In terms of total effects, every one-unit increase in self-motivation problems (12-point scale) was associated with a .11 unit decrease in Time 2 GPA. We next conducted this analysis controlling for students' Time 1 cumulative GPA. Cumulative GPA was strongly related to Time 2 semester GPA (.58, p b .001). Although the magnitude of the relationships in the mediation model was reduced, the pattern of results was similar. Selfmotivation problems continued to have a direct negative impact on GPA (− .06, p b .001). The indirect effect of self-motivation problems on GPA via GPA goal was reduced but remained significant and negative (−.0056; confidence interval of indirect effect = −.0174 to −.0001).2 In light of the finding that students with greater self-motivation deficits both set lower goals and achieve lower GPAs, one question left unanswered by the mediation analysis is whether students with lower self-motivation actually attain the goals that they set for themselves. We performed an exploratory analysis to examine this question. We first converted semester Time 2 GPA into the eight category response scale used in collecting students' grade goals [A (4.0 to 3.86) to C− or lower (1.67 or less)]. We then created a difference score to capture any discrepancy between goal and achieved GPA (Time 2 GPA—goal) such that positive scores on this variable would reflect exceeding one's stated goal. In this analysis, self-motivation problems were negatively correlated with this difference score (r = − .24, p b .001), suggesting that individuals with poor self-motivation fall further short of achieving their goals.

2 We decided to focus on the self-motivation subscale rather than the BDEFS total score in these analyses because of the heterogeneity of associations observed in Study 1 and Study 2 between GPA and the separate EF subscales that make up the EF total score (in particular, emotion regulation). Nonetheless, we repeated the mediator analyses in this section using the EF total score as the predictor. The pattern of results was similar although the magnitude of the effects was more modest and the indirect effect of BDEFS on Time 2 GPA via grade goal did not reach significance.

24

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

Fig. 1. Self-motivation problems predict subsequent GPA directly and indirectly via grade goals.

3.3. Discussion Results from Study 2 replicate and confirm the particular importance of self-motivation deficits among the facets of EF problems measured by the BDEFS in predicting academic performance. This factor was the strongest and most unique predictor of GPA prospectively and when controlling for prior cumulative GPA. In further understanding the relationship between self-motivation deficits and subsequent grades, setting lower goals partially explained why students with self-motivation problems attain lower grades. Importantly, this result held when controlling for prior cumulative GPA suggesting that the association between goals and later performance was not simply an artifact of students anchoring their goals to their prior performance. However, our mediation analysis also demonstrated an enduring direct effect of self-motivation problems on subsequent grades, suggesting that additional processes beyond setting low goals contribute to this association. In addition, we found that self-motivation problems were associated not only with lower grades compared to other students but also with a greater likelihood of failing to meet one's self-selected goal. Thus, students with self-motivation problems tend to both set lower goals than their peers and be less likely to meet those already-lower goals. In comparing the results of Study 1 and Study 2, self-management to time also emerged in Study 2 as a predictor of past and future GPA—however, this facet did not consistently predict grades when self-motivation problems were included in the model. As in Study 1, emotion regulation deficits showed some evidence of a positive relationship with concurrent cumulative GPA—however, it did not predict subsequent GPA in any of the Study 2 analyses suggesting that this facet may not be a useful predictor of academic performance over time. The prospective design of Study 2 is a significant strength, allowing us to make stronger inferences about the effects of executive functioning on later academic performance. However, as noted previously, BDEFS assessments took place mid-semester—five-to-eight-weeks before the completion of final exams—and so it is possible that students' knowledge of grades from the first half of the semester informed their self-assessment of EF deficits mid-semester. Furthermore, the midsemester assessment of goals may reflect a goal that has already been calibrated in response to grades on earlier assignments. Future research would benefit from assessment of EF deficits prior to the start of the semester to provide a more pure test of the predictive utility of this measure of EF functioning. Central to theories of self-regulation (e.g., Carver & Scheier, 1998) is the idea that behavior is regulated by feedback control processes. For example, a student sets a semester grade goal of earning an A − the first week of classes; a month later she earns a B on the first exam. This feedback (in the form of the exam grade) on her progress towards the semester grade goal would trigger anxiety and signal a need to

modify behavior (e.g., study more or in a different way), modify goals (e.g., aim for a B +), or both. However, it is likely that such normative processes in self-regulation may go awry in individuals with deficits in executive functioning. This is suggested by our finding that students with self-motivation deficits show greater discrepancies between their self-identified goals and their actual performance. Unfortunately, any potential calibration (or lack thereof) in grade goals resulting from EF deficits cannot be determined in the current design. Thus, a promising area for future research would be to examine the degree to which students with varying executive functioning abilities modify their grade goals throughout the semester as feedback on individual assignments become available. Overall, Study 2 confirmed the unique importance of self-motivation deficits as a facet of EF that concurrently and prospectively predicted academic performance above and beyond past college grades. Study 2 also identified a partial mediator of the effects of self-motivation problems—grade goals—and supports the idea that students with selfmotivation problems set lower goals and then are less likely to achieve those goals. Study 2 also indicated that additional processes other than goal-setting must operate to connect self-motivation deficits to grade outcomes. 4. Summary and conclusions The present study examined the concurrent and prospective associations of a brief self-report measure of deficits in executive functioning (EF) in daily life with grade point averages in college students. Overall, findings support deficits in EF as an important individual difference that may help to identify emerging adults at risk of academic difficulties. Correlation analyses revealed that overall EF deficits (as indexed by the BDEFS total score) was associated with cumulative GPA in both samples as well as prospectively associated with subsequent GPA, even after accounting for cumulative GPA. As such, this result indicates that deficits in EF in daily life are a predictor of impaired future academic performance. Deficits in self-motivation in particular showed the strongest and most consistent relationships with GPA across samples and methods. Self-motivation deficits predicted future GPA above and beyond cumulative GPA. Importantly, this finding is consistent with Richardson et al.'s (2012) findings with respect to the construct of effort regulation. This construct was one of the most strongly correlated with college academic performance when taking into account high school GPA and standardized test scores. It included the Learning and Study Strategies Inventory (LASSI; Weinstein & Palmer, 2002) motivation subscale (e.g., “Even if I do not like an assignment a course, I am able to get myself to work on it.”) and the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich, Smith, Garcia, & Mckeachie, 1993) effort regulation subscale

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

(e.g., “I work hard to do well in this class even if I don't like what we are doing.”), thus tapping the ability of students to put forth effort in contexts that provide little extrinsic or even intrinsic motivation. Our results confirm the importance of students' ability to generate consistent effort and suggest that more general problems in self-motivation (not just those directly related to academic tasks) may predict academic performance as well. Beyond measures of ability and past performance, success in college appears to rely on the capacity and willingness to apply one's self consistently to tasks that are not inherently reinforcing. To better understand the relationship between self-motivation problems and GPA, we examined it through the lens of goal-setting, as prior work has identified students' grade goals as a robust predictor of academic performance (Locke & Bryan, 1968; Richardson et al., 2012; Wood & Locke, 1987) and a mediator between other more distal variables such as personality and academic performance (Lee et al., 2003; VandeWalle et al., 2001). First, we were able to replicate the strong correlation between students' goals and their past and current academic performance (r = .60 and .69) using cross-course GPAs. In contrast, past studies have only reported associations between goals and grades in single courses. Next, through mediation analysis, we found that self-motivation problems predicted setting lower GPA goals and that this had a significant indirect effect on later grades. However, a direct effect of self-motivation problems on grades remained, which suggests that—in addition to setting lower goals—other processes operate to translate general self-motivation problems into poorer academic performance. The fact that students with more self-motivation problems not only set lower goals but were less likely to achieve those goals also hints at the operation of processes beyond goal-setting. Future studies should measure other processes that may connect selfmotivation deficits to academic outcomes—for example, failure to use specific self-motivational strategies or proportion of time spent on academic vs. non-academic pursuits. These findings have implications for the use of goal-setting interventions to improve academic performance for students with self-motivation problems as emphasized by Richardson et al. (2012). First, our results suggest that encouraging students to set more ambitious goals may be an important element of interventions to improve academic performance and increase motivation. However, the results of our mediation analysis also suggest that goal-setting cannot fully account for the relationship between self-motivation difficulties and academic problems, suggesting that interventions must also focus on improving other self-regulation skills rather than simply encouraging students to set higher goals and providing encouragement to achieve them. Indeed, our results suggest that the goals of students with low self-motivation may overshoot their subsequent accomplishments. One-dimensional interventions are unlikely to ameliorate EF-related academic problems and students in need are likely to require additional self-regulatory tools to fully translate higher goals into actual achievement. In other words, more ambitious goal-setting may be a necessary—but not sufficient—element in improving academic functioning for students with self-regulation problems. Given the prominent role of self-motivation and effort regulation in predicting GPA, students may benefit from training in using selfregulation strategies such as self-reinforcement to help them complete less intrinsically rewarding academic tasks. For example, students could learn to implement the Premack principle (Danaher, 1974) for avoided tasks whereby they allow themselves to engage in a desired activity (e.g., watching an episode of Arrested Development on Netflix) only after meeting a specific target for an undesired activity (e.g., reading 20 pages of an uninteresting textbook chapter). Of course, students would need to habitually rely on strategies like these if they are to have a long-term impact on their ability to self-regulate. Fortunately, there is growing evidence from the literature on skills-based treatments for adult ADHD—a disorder of self-regulation—that people can learn to consistently use skills to ameliorate their self-regulation deficits (Safren et al., 2010; Solanto et al., 2010).

25

Strengths of the present study include the use of two independent samples to examine the association of EF deficits to academic achievement at two colleges with distinct academic profiles. Specifically, Sample 1 was drawn from a national liberal arts college rated as “more selective” whereas Sample 2 was drawn from a “selective” regional university (U.S. News and World Report Best Colleges Ranking, 2013). Our study demonstrated a relationship with actual grades obtained from the registrar, not just with students' self-reported GPA. Another strength of this study is the use of prospective design to extend the cross-sectional results of Study 1. The demonstration that EF deficits exert an effect upon subsequent GPA suggests that measuring EF deficits can help to identify individuals at-risk for future poor academic performance. A third strength of the study was our attempt to further examine the relationship between self-motivation problems and grades via students' goals, a mechanism that is suggested by theory on EF (Barkley, 2012b) as well as prior research on college academic performance (Richardson et al., 2012) and motivation more broadly (Locke & Latham, 1990). The limitations of our study must also be considered in interpreting our findings and considering directions for future studies. First, the use of a college student sample may limit generalizability in that the full spectrum of EF deficits may not be represented given that EF deficits have been shown to be associated with lower levels of academic attainment in prior research (Barkley, 2012a). Furthermore, both samples were drawn from private institutions and it would be important to assess whether results replicate to students in a range of higher education settings including public 2- and 4-year colleges and universities. Second, Sample 2 consisted exclusively of female students attending a women's college. While this allowed us to generalize the results of Study 1, which took place at a co-educational institution, to a women's college, additional research on grade goals as a mediator of the association of self-motivation deficits and GPA will be needed in male college students as well as female students attending co-educational institutions. Third, the present analyses were limited to students who had already completed one semester of college to allow for statistical control of prior academic performance. The role of EF deficits in predicting performance and retention of first-semester students (above and beyond high school GPA and entrance exam scores) would be a valuable area for further research. Fourth, although we found relationships between self-reported EF deficits and academic achievement in the current study, readers should interpret our findings in light of the limitations of self report and future research should assess the extent to which scores on the BDEFS relate to other measures of self-regulation problems such as other-report (peers, instructors), self-ratings obtained through ecological momentary assessment, and specific self-regulation behaviors relevant to an academic setting. Finally, in this study, we did not manipulate goal setting and so we cannot make causal inferences either from our own results or from the prior correlational studies on goalsetting and grades. These ideas need to be directly evaluated using randomized-controlled intervention studies comparing the efficacy of interventions teaching use of compensatory self-regulatory skills such as goal setting and motivation enhancement. The relationship between EF deficits and other outcomes critical to college adjustment deserves further study. EF deficits assessed by the BDEFS have been shown to be associated with depression symptoms in college students (Feldman et al., 2013). In a separate study, EF assessed with a different rating scale was also found to be associated with depression as well as other markers of impairment in college students including academic, interpersonal, and substance use problems (Wingo et al., 2013). EF deficits may also be involved in other health behaviors of college students (eating pathology, risky sexual practices, irregular sleep habits). If the BDEFS is found to predict a variety of outcomes, it could be a relatively cost-effective tool for identifying students at-risk for a range of negative outcomes early in their college careers. Our findings support the value of assessment of EF deficits in research on academic achievement in college students and support the utility of the BDEFS as a brief, non-invasive measure of this construct.

26

L.E. Knouse et al. / Learning and Individual Differences 36 (2014) 19–26

In addition, our efforts to unpack the relationship of self-motivation problems to future academic performance with respect to goal-setting theory suggest that, while setting higher goals may be valuable to improving the academic performance of less motivated students, additional tools may be necessary to help them fully translate more ambitious goals into academic reality.

References Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010). Emotion-regulation strategies across psychopathology: A meta-analytic review. Clinical Psychology Review, 30(2), 217–237, http://dx.doi.org/10.1016/j.cpr.2009.11.004. Barkley, R. A. (2012a). The Barkley deficits in executive functioning scale. New York: Guilford Press. Barkley, R. A. (2012b). Executive functions: What they are, how they work, and why they evolved. New York: Guilford Press. Barkley, R. A., & Fischer, M. (2011). Predicting impairment in major life activities and occupational functioning in hyperactive children as adults: Self-reported executive function (EF) deficits versus EF tests. Developmental Neuropsychology, 36(2), 137–161, http://dx.doi.org/10.1080/87565641.2010.549877. Barkley, R. A., & Murphy, K. R. (2010). Impairment in occupational functioning and adult ADHD: The predictive utility of executive function (EF) ratings versus EF tests. Archives of Clinical Neuropsychology, 25(3), 157–173, http://dx.doi.org/10.1093/ arclin/acq014. Barkley, R. A., & Murphy, K. (2011). The nature of executive function (EF) deficits in daily life activities in adults with ADHD and their relationship to performance on EF tests. Journal of Psychopathology and Behavioral Assessment, 33(2), 137–158, http://dx.doi. org/10.1007/s10862-011-9217-x. Biederman, J., Monuteaux, M. C., Doyle, A. E., Seidman, L. J., Wilens, T. E., Ferrero, R., et al. (2004). Impact of executive functioning deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children. Journal of Consulting and Clinical Psychology, 72(5), 757–766, http://dx.doi.org/10.1037/0022-006X.72.5.757. Carver, C. S., & Scheier, M. F. (1998). On the self-regulation of behavior. New York: Cambridge University Press. Conard, M.A. (2005). Aptitude is not enough: How personality and behavior predict academic performance. Journal of Research in Personality, 40, 339–346. Cowan, N., Elliott, E. M., Saults, J. S., Morey, C. C., Mattox, S., Hismjatullina, A., et al. (2005). On the capacity of attention: Its estimation and its role in working memory and cognitive aptitudes. Cognitive Psychology, 51(1), 42–100, http://dx.doi.org/10.1016/j. cogpsych.2004.12.001. Crede, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–453, http://dx.doi.org/10.1111/j.1745-6924.2008.00089.x. Danaher, B. G. (1974). Theoretical foundations and clinical applications of the Premack principle: Review and critique. Behavior Therapy, 5, 307–324, http://dx.doi.org/10. 1016/S0005-7894(74)80001-8. De Feyter, T., Caers, R., Vigna, C., & Berings, D. (2012). Unraveling the impact of the Big Five personality traits on academic performance: The moderating and mediating effects of self-efficacy and academic motivation. Learning and Individual Differences, 22(4), 439–448, http://dx.doi.org/10.1016/j.lindif.2012.03.013. Duckworth, A. L., Tsukayama, E., & May, H. (2010). Establishing causality using longitudinal hierarchical linear modeling: An illustration predicting achievement from selfcontrol. Social Psychological and Personality Science, 1, http://dx.doi.org/10.1177/ 1948550609359707. Feldman, G., Knouse, L. E., & Robinson, A. (2013). Executive functioning difficulties and depression symptoms: Incremental validity and prospective associations. Journal of Cognitive and Behavioral Psychotherapies, 13, 259–274. Gropper, R. J., & Tannock, R. (2009). A pilot study of working memory and academic achievement in college students with ADHD. Journal of Attention Disorders, 12(6), 574–581, http://dx.doi.org/10.1177/1087054708320390. Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis. New York: Guilford Press. Kirby, K. N., Winston, G. C., & Santiesteban, M. (2005). Impatience and grades: Delaydiscount rates correlate with college GPA. Learning and Individual Differences, 15(3), 213–222, http://dx.doi.org/10.1016/j.lindif.2005.01.003.

Knouse, L. E., Barkley, R. A., & Murphy, K. R. (2013). Does executive functioning (EF) predict depression in clinic-referred adults?: EF tests vs. rating scales. Journal of Affective Disorders, 145, 270–275, http://dx.doi.org/10.1016/j.jad.2012.05.064. Kuncel, N. R., Credé, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of Educational Research, 75(1), 63–82, http://dx.doi.org/10.3102/ 00346543075001063. Lee, F. K., Sheldon, K. M., & Turban, D. B. (2003). Personality and the goal-striving process: The influence of achievement goal patterns, goal level, and mental focus on performance and enjoyment. Journal of Applied Psychology, 88(2), 256–265, http://dx.doi. org/10.1037/0021-9010.88.2.256. Locke, E. A., & Bryan, J. F. (1968). Grade goals as determinants of academic achievement. Journal of General Psychology, 79(2), 217–228, http://dx.doi.org/10.1080/00221309. 1968.9710469. Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice-Hall, Inc. Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions in Psychological Science, 15(5), 265–268, http://dx.doi.org/10.1111/j.14678721.2006.00449.x. Noftle, E. E., & Robins, R. W. (2007). Personality predictors of academic outcomes: Big five correlates of GPA and SAT scores. Journal of Personality and Social Psychology, 93(1), 116–130, http://dx.doi.org/10.1037/0022-3514.93.1.116. O'Connor, M. C., & Paunonen, S. V. (2007). Big five personality predictors of postsecondary academic performance. Personality and Individual Differences, 43(5), 971–990, http://dx.doi.org/10.1016/j.paid.2007.03.017. Pintrich, P. R., Smith, D. A., Garcia, T., & Mckeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813, http://dx.doi.org/10.1177/ 0013164493053003024. Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322–338, http://dx.doi.org/10.1037/ a0014996. Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students' academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387, http://dx.doi.org/10.1037/a0026838. Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes?: A meta-analysis. Psychological Bulletin, 130(2), 261–288, http://dx.doi.org/10.1037/0033-2909.130.2.261. Safren, S. A., Safren, S. A., Sprich, S., Mimiaga, M. J., Surman, C., Knouse, L. E., et al. (2010). Cognitive behavioral therapy vs. relaxation with educational support for medicationtreated adults with ADHD and persistent symptoms. JAMA, 304(8), 857–880, http:// dx.doi.org/10.1001/jama.2010.1192. Solanto, M. V., Marks, D. J., Wasserstein, J., Mitchell, K., Abikoff, H., Alvir, J., et al. (2010). Efficacy of metacognitive therapy for adult ADHD. American Journal of Psychiatry, 167, 958–968, http://dx.doi.org/10.1176/appi.ajp.2009.09081123. St. Clair-Thompson, H. L., & Gathercole, S. E. (2006). Executive functions and achievements in school: Shifting, updating, inhibition, and working memory. The Quarterly Journal of Experimental Psychology, 59(4), 745–759, http://dx.doi.org/10.1080/ 17470210500162854. Tangney, J. P., Baumeister, R. F., & Boone, A. (2004). High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. Journal of Personality, 72(2), 271–322, http://dx.doi.org/10.1111/j.0022-3506.2004.00263.x. U.S. News and World Report Best Colleges Ranking (2013). U.S. News and World Report. Retrieved from. http://colleges.usnews.rankingsandreviews.com/best-colleges Valiente, C., Lemery-Chalfant, K., & Swanson, J. (2010). Prediction of kindergarteners' academic achievement from their effortful control and emotionality: Evidence for direct and moderated relationships. Journal of Educational Psychology, 102(3), 550–560, http://dx.doi.org/10.1037/a001899. VandeWalle, D., Cron, W. L., & Slocum, J. W. (2001). The role of goal orientation following performance feedback. Journal of Applied Psychology, 86(4), 629–640, http://dx.doi. org/10.1037/0021-9010.86.4.629. Weinstein, C. E., & Palmer, D. R. (2002). Learning and Study Strategies Inventory (LASSI): User's manual (2nd ed.). Clearwater, FL: H & H Publishing. Wingo, J., Kalkut, E., Tuminello, E., Asconape, J., & Han, S. D. (2013). Executive functions, depressive symptoms, and college adjustment in women. Applied Neuropsychology: Adult, 20(2), 136–144, http://dx.doi.org/10.1080/09084282.2012.670154. Wood, R. E., & Locke, E. A. (1987). The relation of self-efficacy and grade goals to academic performance. Educational and Psychological Measurement, 47(4), 1013–1024, http://dx.doi.org/10.1177/0013164487474017.