A Neurophysiological examination of quality of learning in a feedback-based learning task

Neuropsychologia 93 (2016) 13–20

Contents lists available at ScienceDirect

Neuropsychologia journal homepage: www.elsevier.com/locate/neuropsychologia

crossmark

A Neurophysiological examination of quality of learning in a feedback-based learning task ⁎

Yael Arbela, , Hao Wub a b

Department of Communication Sciences and Disorders, Massachusetts General Hospital Institute of Health Professions, Boston, MA, USA Psychology Department, Boston College, Chestnut Hill, MA, USA

A R T I C L E I N F O

A BS T RAC T

Keywords: Feedback related negativity Declarative learning Feedback processing Event related potentials

The efficiency with which one processes external feedback contributes to the speed and quality of one’s learning. Previous findings that the feedback related negativity (FRN) event related potential (ERP) is modulated by learning outcomes suggested that this ERP reflects the extent to which feedback is used by the learner to improve performance. To further test this suggestion, we measured whether the FRN and the fronto-central positivity (FCP) that follows it are modulated by learning slopes, and as a function of individual differences in learning outcomes. Participants were tasked with learning names (non-words) of 42 novel objects in a twochoice feedback-based visual learning task. The items were divided into three sets of 14 items, each presented in five learning blocks and a sixth test block. Individual learning slopes based on performance on the task, as well as FRN and FCP slopes based on positive and negative feedback related activation in each block were created for 53 participants. Our data pointed to an interaction between slopes of the FRN elicited by negative feedback and learning slopes, such that a sharper decrease in the amplitude of the FRN to negative feedback was associated with sharper learning slopes. We further examined the predictive power of the FRN and FCP elicited in the training blocks on the learning outcomes as measured by performance on the test blocks. We found that small FRN to negative feedback, large FRN to positive feedback, and large FCP to negative feedback in the first training block predicted better learning outcomes. These results add to the growing evidence that the processes giving rise to the FRN and FCP are sensitive to individual differences in the extent to which feedback is used for learning.

1. Introduction 1.1. Feedback processing and learning Feedback processing is a component of Self-regulated learning, a construct that refers to the process by which individuals actively engage in their learning experience by planning, setting goals, self-monitoring, and using external feedback to adjust goals, performance or strategies (Butler and Winne, 1995; De la Fuente and Martínez, 2007; Efklides, 2011; Greene and Azevedo, 2010; Winne and Nesbit, 2010; Zimmerman and Schunk, 2008). A link between self-regulated learning and academic success has been reported in several studies (Abar and Loken, 2010; De Corte et al., 2000; Pressley and McCormick, 1995; Schunk and Zimmerman, 1994, 1998; Zimmerman, 1994,1998; Zimmerman and Schunk, 2008), where “high self-regulators” were found to be more effective learners than “low self-regulators”. As feedback processing a critical component of self-regulated learning, the efficient use of external feedback is crucial for becoming a successful

⁎

self-regulated learner, and in turn, for achieving academic success (Blair, 2002; Zimmerman and Chunk, 1989). The study of the relationship between feedback processing and learning has been enhanced by the discovery some twenty years ago of an event related potential associated with performance monitoring and feedback processing. The feedback related negativity, known as fERN or FRN is an event related potential (ERP) elicited by feedback in various tasks in which feedback guides response choice, and learning. This ERP component peaks at about 250–300 ms following the presentation of a feedback stimulus (Miltner et al., 1997), when information about the accuracy of a choice or action is unknown to the learner until it is communicated by an external source. Converging evidence points to the anterior cingulate cortex (ACC) as the generator of the FRN (Carter et al., 1998; Critchley et al., 2005; Dehaene et al., 1994; Holroyd et al., 1998; Kiehl et al., 2000; Ladouceur et al., 2007; Mathalon et al., 2003; Mars et al., 2005; Menon et al., 2001; van Veen and Carter, 2002).

Corresponding author. E-mail address: [email protected] (Y. Arbel).

http://dx.doi.org/10.1016/j.neuropsychologia.2016.10.001 Received 29 March 2016; Received in revised form 30 September 2016; Accepted 2 October 2016 Available online 04 October 2016 0028-3932/ © 2016 Elsevier Ltd. All rights reserved.

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

feedback). Results pertaining to learning related differences in the FRN elicited by negative feedback were reported by Bellebaum and Daum (2008), who examined the FRN as participants performed a reward probabilistic task that was governed by a rule. Participants were informed that by finding and applying the rule, they were likely to increase their gains. Trials were divided into pre-learning and postlearning trials for each of the learners, and into two parts for nonlearners based on the learners’ data. Findings suggested that the amplitude of the FRN to negative feedback increased from the prelearning trials to the post-learning trials only among participants who learned the rule. Different findings were reported in a study by Sailer, Florian, Fischmeister, and Bauer (2010), who tasked participants with learning the correct sequence of response outcomes, with the goal of maximizing their gains and reducing their losses. Participants were classified as “high learners” and “low-learners” based on their performance on the task, and trials were split into half to create “early phase” and “late phase” conditions. FRN was found to be overall smaller among “high learners”, and to show a reduction in amplitude from early trials to later trials in the two groups. In a study by Luft et al. (2013), participants were divided into two groups (“high” and “low” learners) based on performance on a time estimation task. Their data suggested that the FRN was not different between the two groups. It is worth noting that in Luft et al. (2013) the FRN was examined across all trials, while in other reports (e.g., Bellebaum and Daum, 2008; Sailer et al., 2010; Santesso et al., 2008) FRN was examined in two different periods in the leaning process. Given that some reports detected a difference between groups of learners only at a later portion of the task, it is possible that an existing difference was missed in Luft et al. (2013), because a distinction between early and late trials was not made. The inconsistency among the reported studies can stem from various factors, ranging from those related to task and participants, to factors related to the manner by which the FRN was measured and analyzed. It is important to consider the possible differences in the processing of feedback in a probabilistic learning task (e.g., Bellebaum and Daum, 2008; Sailer et al., 2010), in which participants learn to map a specific response to its probable outcome (feedback), and in a declarative word leaning task (e.g., Arbel et al., 2013, 2014), in which participants learn correct associations through a trial-and-error process guided by feedback. We argue that the “late phase” of each of the learning tasks captures a different type of feedback processing inherent in the nature of the learning task, with the late phase in a probabilistic learning task capturing the processing of an expected negative feedback that is no longer beneficial for learning, and with the same phase in a declarative learning task capturing the processing of a still very informative negative feedback. This difference stems from the fact that in most probabilistic learning tasks, outcomes are not 100% mapped to a particular response, and learners come to expect occasional negative feedback even after they have optimized their responses. In that sense, when early learning phases are compared with late phases, the comparison is between negative feedback that is informative for task performance and an expected negative feedback that is no longer contributing to learning. Another possible contributor to the varying findings is the classification of participants into learning groups, with some studies reporting the differences between “learners” and “non-learners” (e.g., Bellebaum and Daum, 2008; Santesso et al., 2008), and others comparing “high learners” with “low learners” (e.g., Sailer et al., 2010; Luft et al., 2013). In the comparison of “learners” with “nonlearners”, information is obtained about the processing of feedback by all participants who achieved a learning criterion, regardless of how fast or how well they have learned, with those who did not achieve a learning criterion. One may claim that whereas “low learners” extract information from feedback but at a slower rate, or with greater effort, when compared with “high learners”, “non-learners” fail to extract relevant information from feedback. Comparison between reports that

1.2. The FRN in feedback-based learning A growing number of studies examine the FRN with the goal of elucidating the link between feedback processing and learning. Such studies employ feedback-based learning tasks in which the processing of positive and negative feedback is assessed in connection with learning outcomes (Arbel et al., 2013, 2014; Eppinger et al., 2009; Krigolson et al., 2009; Luft, 2014; Pietschmann et al., 2008; Sailer et al., 2010; van der Helden et al., 2010; Van den Bos et al., 2009), and with within-task changes in decision making (Chase et al., 2011; Frank et al., 2005). Most studies that have reported a connection between the FRN and learning (see review by Luft (2014)) demonstrated this connection by showing a relationship between the FRN amplitude and response adjustment by the learner. For example, van der Helden et al. (2010), who examined the FRN using a motor sequence learning task reported that negative feedback to incorrect responses which were later modified was associated with larger FRN, than negative feedback provided to incorrect responses which were later repeated. These results were interpreted to suggest that the FRN is associated with effective adjustments of performance. Similarly, Cohen et al. (2007) found that the FRN was associated with a change of response after a loss, such that its amplitude was larger on “loss” trials after which participants changed their response, in comparison with “loss” trials after which participants repeated the action that previously resulted in unfavorable outcomes. Others have examined the extent to which the FRN was sensitive to learning outcomes in declarative learning tasks where associations between stimuli had to be learned and retained. Arbel et al. (2013) reported that in a feedback-based four-choice word learning task, FRN associated with positive feedback was sensitive to subsequent learning, such that words that were subsequently recalled elicited larger FRN to positive feedback during the learning process. Ernst and Steinhauser (2012), who studied the FRN in a multiplechoice declarative learning task, reported that the FRN amplitude associated with negative feedback was modulated by learning outcomes, such that smaller FRN amplitude was elicited in relation to successful learning. 1.3. The FRN as an index of individual differences in feedback-based learning Findings of a relationship between the FRN and learning can be interpreted to suggest that the FRN is an index of the extent to which feedback is used by the learner to improve performance and learning. Arbel et al. (2014) proposed the utility account of the FRN, positing that the FRN is a marker of the degree of utilization of the feedback by the learner. According to this account, the feedback-receiver extracts information from the feedback to improve learning and performance, and uses it to evaluate progress toward the goal. One prediction derived from this account is that if the FRN is indeed a marker of the use of feedback, it should show differentiation between individuals who are efficient at extracting information from feedback to facilitate learning and those who are not. Findings pertaining to this prediction are limited and inconsistent. In a study by Santesso et al. (2008), who employed a probabilistic reward learning task, FRN was examined in two groups of individuals who were classified as “learners” and “nonlearners” based on their performance on the task. In their study, learning was defined as a growing bias to select stimuli with high reward probability measured by change in response bias from the first block and the combination of the second and third blocks. Their results suggested that the FRN to reward (positive feedback associated with the selection of a stimulus with high reward probability) was more positive among “learners” when compared with “non-learners”, and showed a positive correlation with a positive change in response bias (i.e., learning), indicating that a greater reward related positivity was associated with better learning. It is important to note that in Santesso et al. (2008) FRN was only examined for reward feedback (positive 14

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

and analyze EEG data. EEG was continuously recorded at a 1000 Hz sampling rate, and the amplifier was set to a band pass of 0.1–100 Hz. The electrode impedance was kept below 50 kΩ. E-Prime 2.0 Professional by Psychology Software Tools, Inc. was used for task design and stimuli presentation.

treat learning as a categorical variable is therefore challenging. The choice of method for FRN measurement is another important candidate for inconsistent reports. While some studies measure the FRN as the most negative peak in a specified time window (e.g., Santesso et al., 2008), others used base-to-peak-to-peak measure by subtracting the average of preceding and proceeding positivity from the largest negativity (e.g., Sailer et al., 2010), calculated area measurement (e.g., Bellebaum and Daum, 2008), or conducted principal component analysis (e.g., Arbel et al., 2013, 2014). A particular concern related to the variation in FRN measurement is the extent to which the FRN is measured independently, or in combination with another feedback related ERP component that follows it in time, namely the fronto-central positivity (FCP).

2.3. Procedure Each participant visited the lab once for a two- hour long session. The session included the application of a 32-channel hydroCel net on the participant’s scalp, and the recording of EEG while participants completed a two-choice paired associate pseudo-word learning task. Participants sat in front of a 15 in. computer monitor and used a keyboard to record their responses.

1.3.1. The FCP and its relation to learning The FCP, first described by Butterfield and Mangels (2003) peaks about 350 ms following the presentation of the feedback, and is maximal at the same fronto-central electrode site (FCz) as the FRN. It is larger for negative feedback, and is assumed to reflect an attentional orienting process that proceeds the initial processing of feedback. Although the FCP can be detected in numerous FRN related reports, it has not been studied extensively. Butterfield and Mangels (2003), who examined the FCP in feedback-based learning tasks, reported that it was modulated by learning, such that a larger FCP amplitude was found to be associated with successful learning.

2.3.1. Feedback-based declarative word learning task Participants were engaged in a two-choice declarative learning task in which they viewed a picture of a novel object on a computer screen coupled with two possible names (non-words). Participants were instructed to choose one of the two possible names by pressing one of two buttons on a keyboard, and were provided with feedback (visual “xxx” for incorrect response and “√√√” for correct response) immediately following their response. Participants were tasked with learning the names of novel objects by using the provided feedback. The task included three sets of 14 new names of novel items to be learned, for a total of 42 items. Each set was presented in a block design, with five feedback-based training blocks, and a sixth feedback free testing block. Each of the 14 items was presented once in each block for a total of five presentations per item across the training blocks. This design allowed the separate examination of the first, second, third, fourth and fifth presentations of the items, and the feedback that followed the corresponding learners’ responses. The task was designed so that, unbeknown to the participants, in the first block of trials that contained each item once, positive and negative feedback were equally probable (0.5 positive feedback; 0.5 negative feedback). The associations between items and names were created for each participant based on their responses on this first block, and feedback was valid throughout the learning task. We will refer to each block of 14 items as a Round (Round 1, Round 2, Round 3, Round 4, Round 5). Round 6 of each set served as a testing phase, in which participants were not provided with feedback. EEG was time-locked to the presentation of the feedback stimuli in each training Round, and segmented based on accuracy (i.e, Positive Feedback vs. Negative Feedback). In each trial (see Fig. 1 for an illustration), a fixation point appeared on a computer screen for 500 msec, after which a picture of a novel object, accompanied by two possible names displayed underneath it, was presented. Participants were allotted 3000 ms to indicate their choice between the two names by pressing one of two buttons. Participants’ responses were immediately followed by a visual feedback presentation of “√√√” for correct responses, and “xxx” for incorrect responses. Feedback was presented on the screen for 800 ms.

1.4. The current study To further examine the FRN and FCP in relation to learning, the present study offers an evaluation of the relationship between these components and learning, by treating the feedback related ERPs and learning as continuous variables, and participants as a single cohort. The FRN and FCP to positive and negative feedback were examined in a feedback-based two-choice declarative learning task presented in a block-design. Each stimulus in this design was presented only once within a block, allowing for an examination of the learning process from the first presentation of a stimulus to its sixth presentation. In addition, unbeknown to the learners, in the first block of trials, positive and negative feedback stimuli were presented at an equal probability. This unique design allowed for the examination of the predictive value of the FRN and FCP elicited by equally probable positive and negative feedback in the first block of trials on learning outcomes. Temporal principal component analysis (TPCA) was employed to allow the separation of components which may overlap in time, and to avoid a selection of a time window that may either capture only part of the activity of interest or a combination of several activities. 2. Material and methods 2.1. Participants Sixty healthy young adults (50 females) from the Boston area participated in the study after signing a consent form. Participants were right handed individuals, between the ages of 18 and 35 (M=24.6, SD=3.1), with normal or corrected vision, who reported that they had no history of head injury or other neurological deficits, and that English was their predominant language. Participants were paid for their participation. ERP data of seven participants were excluded from the ERP analysis due to excessive artifacts, resulting in 53 (43 females) participants (mean age=24.4, SD=2.8) who were included in the analysis. 2.2. Apparatus The GES 400 system by Electrical Geodesics, Inc. (EGI) with 32channel HydroCel Geodesic Sensor Net from EGI was used to acquire

Fig. 1. An illustration of the sequence of fixation, stimulus, response, and feedback in each of the task trials.

15

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

2.4.3. ERP data: FRN and FCP slopes To obtain individual FRN and FCP slopes, a simple regression model for each participant was run separately for each of these components as the dependent variables, and Rounds (1−4) as a continuous independent variable. Separate slopes were created for positive and negative feedback.

2.3.1. .1. Stimuli. The novel objects were borrowed from Kroll and Potter (1984). Non-words were produced from the ARC nonword database (Rastle et al., 2002). The non-words were in three-letter consonant–vowel–consonant (CVC) format (e.g., joz), and were phonologically legal in English. The number of orthographic neighbors (i.e., real words in English that are written similarly) was between 0 and 10 (M=5.9); the number of phonological neighbors (i.e., real words in English whose pronunciation is similar) was between 5 and 20 (M=14).

2.4.4. Statistical analysis To evaluate the possible relationship between feedback related ERP (FRN and FCP) and individual differences in learning slopes, a general linear model with FRN amplitude as independent variable, Rounds (Round 1, Round 2, Round 3, & Round 4) as a within subject continuous variable, and Learning Slope as a dependent variable was fitted to the data. A separate analysis was conducted for the FRN slope to positive and negative feedback and their relationship to the learning slopes. The same analysis was conducted for the slope of the FCP. For the next step of the analysis our goal was to test the extent to which the FRN and FCP elicited in Round 1 are predictors of learning outcomes. The focus on Round 1 was motivated by the fact that positive and negative feedback were equally probable (0.5 positive; 0.5 negative) in this round, making the activation associated with positive and negative feedback unrelated to probability, and not likely to be influenced by expectancy. To evaluate the predictive value of the FRN and FCP in Round 1 on Learning Outcome as measured by performance on the testing round (Round 6), a logistic regression model with Learning Outcome as the dependent variable and FRN and FCP amplitudes in the four rounds and two feedback conditions as sixteen independent variables was fitted to the data. The predictive power of the FRN to positive and negative feedback in Round 1 on learning outcomes was examined after controlling for the contribution of the FCP and FRNs in other rounds. The predictive power of the FCP to positive and negative feedback in Round 1 on learning outcomes was examined after controlling for the contribution of the FRN and the FCP in other rounds.

2.4. Analysis 2.4.1. EEG recording and analysis parameters EEG data of 53 participants entered the analysis. Data from Round 5 were excluded from the analysis due to insufficient artifact free error trials for 40% of the participants. The continuous EEG data were filtered offline using a 30 Hz low-pass filter, and then segmented into 1000 ms-long epochs starting 200 ms before the onset of the feedback stimuli and ending 800 ms following the feedback. Baseline correction was performed on the first 100 ms of the epoch. An algorithm developed by Gratton et al. (1983) for offline removal of ocular artifacts was used to correct for eye movements and blinks. Averages of the artifact free baseline corrected epochs were re-referenced using average referencing. ERPs from the fronto central recording site, FCz, were subjected to a temporal principal component analysis (TPCA) to reduce the temporal dimensionality of the dataset by computing covariance among time points across participants, feedback (positive and negative feedback), and rounds. The temporal factors that were required to account for 95% of the variance in the input data set were retained for Varimax rotation (Spencer et al., 2001). The factor scores of the FRN were used for statistical analysis. 2.4.2. Behavioral data: learning outcomes and learning slopes Transformed Logistic Regression Model: For each participant, accuracy on the testing block served as the Learning Outcome measure. In addition, accuracy data were collected from each of the task rounds to create individual learning slopes. To create these learning slopes, logistic model with zero intercept was fitted to each individual’s learning data in the six task rounds (five training rounds and one testing round). The zero intercept ensured that the chance of correctness at trial 1 was 0.5. The slope parameter β in the logistic regression was estimated for each individual. Because the standard error of this slope parameter differs for different slope values, a variance-stabilizing transformation β′=tan−1( 2 β ) was employed so that the standard error of β’ was the same for all individuals. The transformed slope β’for each participant was considered the measure of Learning Slope. To evaluate the extent to which the logistic model captures not only the speed of learning, but also the maximal accuracy achieved by the learner, another method was adopted to examine learning. In this method, learning was defined as a combination of speed and accuracy, such that the strongest learning was expected to be characterized by the steepest rise in accuracy from Round 1 (0.5 accuracy for all participants) to Round 2, and by the highest accuracy level achieved throughout the learning process. For each participant, a standardized score (z score) of the speed of learning as measured by the slope of Round 2 as well as a standardized score of the maximal accuracy level achieved were computed. The two standardized scores were combined for each participant to create our second measure of Learning. A correlation analysis between the two measures of Learning across participants revealed a strong correlation between the two variables, r (51)=0.85, p < 0.001. All analyses were done with each of the two measures of learning. Since the results of these analyses were almost identical, we will only report the results of the Learning Slope measure.

3. Results 3.1. Behavioral data Accuracy level on Round 1 was 0.5 across all participants due to the design of the task. Mean accuracy rate was found to be 0.6 (SD=0.08) on Round 2, 0.7 (SD=0.1) on Round 3, 0.77 (SD=0.1) on Round 4, 0.83 (SD=0.11) on Round 5, and 0.84 (SD=0.1) on Round 6. A generalized estimating equation analysis of accuracy data of all participants with an unstructured working correlation matrix revealed a Round effect, χ2 (5)=620.36, p < 0.001, confirming that there was a significant increase in accuracy level across rounds. Pairwise comparison indicated that accuracy levels increased from round to round (ps < 0.001), except for Round 6 (test round) that was not found different from Round 5, χ2 (1) =1.08, p=0.28. It is important to note that Round 6 was different from the other rounds in two aspects: it was presented to the participants as a “test”, and it was free of feedback. Performance in this round could have been affected by these factors. Individual learning slopes were created for each participant.1 3.2. ERP data Consistent with previous reports, The FRN was observed to be 1 A participant with a flat learning slope has been identified as a possible outlier. However, an examination of this participant’s responses revealed that, although learning was considered poor in this participant, some learning had occurred. More specifically, when learning of a single item was defined as at least three out of five consecutive correct responses associated with an item, this participant learned the names of eight novel objects. Because no indications of lack of attentiveness, effort, or learning have been determined for this participant, his/her data were not excluded from the analysis.

16

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

Round 1

Amplitude (µV)

−5

Round 2

Round 3

−5

FRN

Round 4

−5

−5

0

0

0

0

5

5

5

5

10

10

10

10

15

15

15

15

Positive Fb Negative Fb

FCP 20 −200

0

200

400

600

800

20 −200

0

Time (ms)

200

400

600

800

20 −200

0

200

Time (ms)

400

600

20 −200

800

0

Time (ms)

Temporal Factor 5 (FRN)

200

400

600

800

Time (ms)

Temporal Factor 3 (FCP)

0.8

1.2 1

0.6

Loadings

Loadings

0.8

0.4 0.2

0.6 0.4 0.2

0 0

−0.2 −200

−100

0

100

200

300

400

500

600

700

800

−0.2 −200

−100

0

100

200

300

400

500

600

700

800

Time (ms)

Time (ms)

Fig. 2. Top: Grand average ERPs elicited to positive and negative feedback in each of the four rounds; Bottom: Virtual epochs created from the loadings of temporal factor 5 (FRN component, peaking at 250 ms) on the left, and temporal factor 3 (FCP component, peaking at 350 ms) on the right.

FRN and Learning Slopes

FRN Slope

maximal at electrode FCz. A fronto-central positivity (FCP), which typically follows the FRN, but is underreported, was also observed to have a maximal amplitude at FCz. Grand average ERP data from this electrode obtained from the 53 participants are presented in Fig. 2. A visual inspection of the waveforms suggests differences in the FRN and FCP between positive and negative feedback that change over the course of the four rounds. To measure the magnitude of the FRN separately from that of the FCP, temporal PCA was conducted, resulting in seven temporal factors (TFs) accounting for 95% of the variance in the data. Temporal factor 5 (TF5) which peaked at approximately 250 ms captured the FRN activity, while temporal factor 3 (TF3) with a peak of 350 ms measured the activation of the frontocentral positivity that followed the FRN.

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 -0.2 0 0.8 1

y = 0.4402x - 0.2264 R² = 0.0858

0.2

0.4

0.6

0.8

1

Learning Slope 3.2.1. Learning and FRN slopes When FRN to positive feedback was analyzed as a function of Rounds and Learning Slope, the interaction between the linear trend of Rounds and Learning Slope was not significant, F(1,51)=0.29, p=0.59, suggesting no evidence of moderation effect of the slope of FRN to positive feedback on the Learning Slope. When FRN to negative feedback was analyzed as a function of Rounds and Learning Slope, an interaction between the linear trend of Rounds and Learning Slope was found, F(1,51)=4.79, p=0.03, partial η2=0.086, suggesting a moderation effect of the slope of FRN to negative feedback on the Learning Slope. This interaction suggests that individuals’ learning slopes were moderated by their FRN slopes to negative feedback. A plot (see Fig. 3) of the individual FRN slopes against individuals’ Learning Slopes reveals a negative relationship with a correlation of 0.28, suggesting that every SD decrease in the amplitude of the FRN is associated with a 0.28 SD increase in learning.

Fig. 3. Interaction between the slopes of the FRN to negative feedback (y axis) and learning slopes (x axis). For the FRN slopes, negative values are plotted up to reflect the nature of this ERP component, whose size is determined by how relatively negative its amplitude is (i.e., a large FRN has more negative values than a small FRN). A positive FRN slope means a reduction in FRN amplitude over time. A negative FRN slope means an increase in FRN amplitude over time. The observed negative relationship between FRN slope and learning slope suggests that a sharper reduction in FRN amplitude is associated with faster learning.

on the Learning Slope. We followed this analysis with a repeated measure ANOVA with Rounds (1−4) and Feedback (positive and negative) as within subject factors to examine the extent to which the FCP changed as a function of rounds, without considering individual differences in learning slopes. This analysis resulted in a Feedback effect, F (1, 52)=21.60, p < 0.001, partial η2=0.29, indicating that negative feedback elicited a larger positivity than positive feedback. A Round effect was found, F (3, 50) =16.11, p < 0.001, partial η2=0.49, as well as an interaction between Round and Feedback F (3, 50)=4.65, p=0.006, partial η2=0.22. A post hoc pairwise comparison showed that whereas the FCP to negative feedback became larger from the first round to the second round, t (52) =5.60, p < 0.001, partial η2=0.22, and remained stable thereafter, the FCP to positive feedback became larger from the first round to the second, t (52)=4.55, p < 0.001, partial η2=0.16 , but then showed a decrease in amplitude from Round 2 to Round 3, t (52)=−4.81, p <

3.2.2. Learning and FCP slopes When a separate analysis was conducted of FCP to positive and negative feedback as a function of Rounds and Learning Slope, the interaction between the linear trend of Rounds and Learning Slope was not significant for both positive and negative feedback, F (1, 51)=2.43, p=0.12, F (1, 51)=1.24, p=0.27 respectively, suggesting no evidence of moderation effect of the slope of FCP to positive or negative feedback 17

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

processing of corrective feedback that leads to faster learning can take different forms, one of which will be discussed in relation to the FRN findings of this study. We propose that fast learners are more efficient at extracting relevant information from negative feedback, and are becoming more efficient at extracting task specific relevant information as the task progresses. In other words, these learners decipher faster how to best use negative feedback to enhance their learning in a given task. In this context, FRN can be viewed as reflecting the amount of attentional or processing resources allocated to the extraction of information from the feedback, with faster, more efficient learners using less processing resources as learning progresses. The steep reduction in FRN amplitude to negative feedback among learners with steep learning slopes in our sample is in line with this suggestion. The findings that small FRN amplitude to negative feedback in Round 1 was associated with better learning outcomes can be explained within the utility account of the FRN (Arbel et al., 2014). Since better learning outcomes are achieved by faster learners, and faster learners are more efficient (use less resources) at extracting information from negative feedback, it is possible that the small FRN to negative feedback in Round 1 that predicted good learning outcomes was the product of an overall smaller FRN to negative feedback among faster learners. In addition to the association between the initial negative feedback and learning outcomes, we found that large FRN amplitude to positive feedback in Round 1 was a predictor of better learning outcomes. Taken together, our data suggest that better learning was associated with reduced differences between the activation related to positive and negative feedback. If the amplitude of the FRN reflects the amount of resources allocated to the processing of the feedback, it is possible that better learning is associated with smaller differentiation in the processing of positive and negative feedback. The findings can also be interpreted to suggest that allocating more resources to the processing of positive feedback during the initial stage of learning leads to better learning. The connection between the initial processing of positive feedback and learning outcomes has been previously demonstrated by Arbel et al. (2013) who found that learned items in a similar declarative learning task were associated with larger FRN to the initial positive feedback when compared with non-learned items. Other reports have emphasized the role of positive feedback processing in learning (Kreussel et al., 2012; Baker and Holroyd, 2011; Foti et al., 2011; Holroyd et al., 2008; Potts et al., 2006). It is important to note that no interaction between the slope of the FRN to positive feedback and learning slopes have been found in the current study, suggesting that individuals were not different in the modulation of the FRN to positive feedback across the learning rounds. If the FRN reflects the amount of resources dedicated to the processing of the feedback, lack of difference across different learning speeds implies that faster learners were not different from slow learners in the way they processed positive feedback during the learning process. The proposed account does not exclude the possibility that efficient processing of negative feedback is also associated with the ability to extract relevant information from the feedback while ignoring the negative valence of the feedback, and that the FRN is a manifestation of the combined activation related to the processing of the feedback valence and the processing associated with the extraction of information from the feedback. This suggestion is supported by the view of the reinforcement learning system as both a reward-seeking and information seeking system (e.g., Bromberg-Martin and Hikosaka, 2011; Niv and Chan, 2011). Larger FRN amplitude to negative feedback when positive and negative feedback are equally probable can provide support to the contention that the FRN is a reflection of the proposed combined processing, if one assumes that extracting information from equally probable negative and positive feedback in a two-choice learning task require the same level of processing. However, it is reasonable to suggest that extracting information from negative feedback requires more processing resources as the learner needs to reject one hypothesis and accept the alternative.

Table 1 Correlations between the amplitudes of FRN and FCP to positive and negative feedback in Round 1 (R1). FRN-R1PosFb 1. FRN-R1PosFb 2. FRN-R1NegFb 3. FCP-R1PosFb 4. FCP-R1NegFb ** *

FRN-R1NegFb

FCP-R1PosFb

FCP-R1NegFb

1 0.753**

1

−0.423

−0.328*

1

.

−0.392**

0.805**

1

**

−0.443**

Correlation is significant at the 0.01 level. Correlation is significant at the 0.05 level.

0.001, partial η2=0.17, before stabling off. 3.2.3. FRN and FCP in Round 1 and learning outcomes The FRN and FCP to positive and negative feedback in Rounds 1–4 were entered into a logistic regression analysis as predictors of learning outcomes. Correlation data between the FRN and FCP to positive and negative feedback in the first round are presented in Table 1. The analysis revealed a significant total contribution of the FRN and FCP in Round 1 to Learning Outcomes, after controlling for these ERPs in the subsequent rounds, χ2 (4)=31.229, p < 0.001, partial η2=0.211. In particular, the analysis revealed a significant effect of the FRN to positive and negative feedback in the first round on Learning Outcomes, with χ2 (1)=12.27, p < 0.001, partial η2=0.095 and a coefficient of −0.522 for FRN elicited by positive feedback, and χ2 (1) =18.18, p < 0.001, partial η2=0.135 with a coefficient of 0.830, for FRN to negative feedback. As FRN amplitude is measured by how negative it is (i.e., larger FRN is associated with greater negativity), the discussion of the size of the FRN based on these coefficients should be adjusted. Negative coefficients reflect increase in FRN size, whereas positive coefficients reflect decrease in FRN size. These results indicate that larger FRN amplitude to positive feedback, and smaller FRN amplitude to negative feedback in Round1 predicted better performance on the testing round after the FCP and FRNs at the subsequent rounds were held constant. The analysis also revealed a significant effect of FCP elicited by negative feedback in Round 1, χ2 (1)=16.77, p < 0.001, partial η2=0.126, with a coefficient of 0.906. No such effect was found for FCP elicited to positive feedback (coefficient=−0.096, χ2 (1)=0.302, p=0.58). These results indicate that larger FCP amplitude to negative feedback in Round1 predicted better performance on the testing round, after the FRN and FCPs at the subsequent rounds were held constant. 4. Discussion In the present study, the slope of the FRN to negative feedback was found related to individual differences in the slope of learning, such that a faster decrease in FRN amplitude to negative feedback during the learning process was associated with a sharper learning slope (faster learning). Moreover, activation associated with positive and negative feedback during the first round of the learning task was found to be a predictor of learning outcomes, such that smaller FRN to negative feedback, larger FRN to positive feedback, and larger FCP to negative feedback in Round 1 were related to better learning outcomes as measured by accuracy level on the test round. These results add to the growing evidence of the relationship between feedback processing and learning. 4.1. The utility account of the FRN In the context of feedback-based declarative learning, an efficient 18

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

4.2. The expectancy account of the FRN

feedback has minimal to no evaluative value, and that expectations have yet to be built by performance, it is impossible to discuss the FRN elicited in this round without making assumptions about individual differences in feedback expectancy that are not driven by performance on the examined task. It is indeed possible to assume that strong learners expect to err and don’t expect to be correct in the first round, and that this attitude is more beneficial for learning than expecting to be correct and being surprised by negative feedback. Our data do not allow the examination of such assumption.

Within the framework of the reinforcement learning theory of the FRN, the amplitude of the FRN is suggested to be modulated by expectancy, such that large FRN is elicited when events are worse than expected. In the context of learning, the FRN amplitude is expected to decrease when negative feedback becomes more predictable. Probabilistic learning tasks are ideal for testing the expectancy account, because when outcomes are not 100% mapped to specific outcomes (due to the probabilistic nature of such tasks), negative feedback continues to be presented even after participants learn to map responses to optimal outcomes. In such tasks, negative feedback becomes increasingly predictable during the learning process, and is more predictable by those who gain a stronger grasp of the mapping. Examinations of previous findings of the behavior of the FRN in a probabilistic learning task before and after response-outcome contingencies have been learned do not provide support to this account. For example, in Bellebaum and Daum, (2008), the amplitude of the FRN to negative feedback was found to be larger post-learning in comparison to pre-learning. These results are in contrast to the hypothesis derived from the expectancy account that FRN to negative feedback becomes smaller as negative feedback becomes more predicted. Sailer et al. (2010) found a general reduction in the FRN amplitude to negative feedback from pre-learning to post-learning, which is consistent with the expectancy account. However, no differences were found between “high” and “low” learners in their study, leading the authors to reject the expectancy account and to suggest that the FRN was modulated by the motivational significance of the outcome, with reduced motivation among low learners due to failure to achieve desired outcomes, and a similar reduction among high learners due to the realization that negative feedback is no longer beneficial for maximizing positive outcomes. Given that our data show a reduction in the amplitude of the FRN to negative feedback that is related to learning, the compatibility of the expectancy account of the FRN with our data should be examined. To accept the expectancy account as an explanation for the relationship between a sharp reduction in the FRN amplitude and fast learning, one has to propose that fast learning was associated with a growing predictability of negative feedback. The assumption of an increased predictability of negative feedback in a declarative learning task is not as intuitive or easily accepted as it is in a probabilistic learning task. Two competing suggestions can be made about the predictability of negative feedback during the learning process in a declarative task. The first considers the level of surprise, suggesting that learners come to expect positive feedback as their learning improves, and the delivery of negative feedback at a late stage in the learning process results in greater surprise in comparison to negative feedback provided at the beginning of the task. The adoption of this suggestion leads to the rejection of the expectancy account as a framework to explain our data. The second suggestion considers the level of self-knowledge about performance, proposing that after learners develop a general sense of what they know and what they have yet to learn, they come to expect negative feedback to items that they are less confident about. Within this context, stronger learners may have a clearer view of their progress toward the goal and are likely to become less surprised by negative feedback earlier in the learning process. The adoption of this suggestion permits the interpretation of our results within the framework of the expectancy account. The relationship between the amplitude of the FRN to negative and positive feedback in Round 1 and learning outcomes is even more difficult to interpret within this account. If the FRN is modulated by expectancy, and better learning outcomes are associated with a small FRN to negative feedback and a large FRN to positive feedback, the argument would be that strong learners are those who are less “surprised” by negative feedback and more “surprised” by positive feedback at the initial presentation of feedback. Given that in Round 1

4.3. Fronto-central positivity (FCP) and learning The FCP elicited to negative feedback in the first round of the task was found to be a predictor of learning outcomes, such that large FCP amplitude to negative feedback was associated with better learning outcomes. These results are in line with previous findings that the FCP is modulated by learning (Arbel et al., 2013; Butterfield and Mangels, 2003, Mangels et al., 2006). In light of Butterfield and Mangels’ (2003) proposal that the feedback related FCP is a manifestation of an attentional orienting process, our results suggest that greater attention given to the initial negative feedback led to better learning outcomes. Interestingly, the relationship between the FCP of the initial negative feedback and learning outcomes was in opposite direction to that found for the FRN. More specifically, better learning outcomes were found to be associated with large FCP activation and small FRN activation related to the initial negative feedback. This pattern can be interpreted to suggest that strong learning is associated with an effective early processing of negative feedback that triggers heightened attention to such feedback. Our analysis did not reveal a relationship between individual differences in the slope of the FCP and learning slopes. These results indicate that learners did not differ in the pattern of FCP change over the course of the learning task. Although no interaction was found between change in FCP across rounds and learning slopes, we found that the FCP changed across all learners as a function of feedback and rounds. While negative feedback showed a sharp increase in FCP amplitude from Round 1 to Round 2, after which it stabilized, positive feedback showed an initial increase in amplitude, followed by a sharp decrease. If the FCP reflects an attentional orienting process, its sharp increase in relation to negative feedback from Round 1 to Round 2, followed by an unchanged amplitude across the remaining rounds may suggest that negative feedback called for heightened attention that remained stable over the course of the learning task. The initial increase of FCP in relation to positive feedback followed by a sharp decrease may suggest that attention to positive feedback was enhanced in Round 2 and dropped sharply thereafter. Within this framework, the processing of negative feedback as depicted by the FCP activation remains important throughout the learning process, whereas the processing of positive feedback peaks when positive feedback first confirms the correctness of choices and becomes less important as learning progressed. 5. Conclusions The results of the study suggest that individual differences in feedback processing as reflected by the FRN and FCP are related to individual differences in learning. More specifically, individual slopes created by the change in FRN amplitude to negative feedback over the course of the learning task were found related to individual learning slopes. Additionally, individual differences in the processing of the initial negative and positive feedback were found associated with learning outcomes. More specifically, better learning outcomes were associated with small FRN amplitude and large FCP amplitude to the initial negative feedback, and large FRN amplitude to the initial positive feedback. 19

Neuropsychologia 93 (2016) 13–20

Y. Arbel, H. Wu

on the amplitude of the feedback negativity. Psychophysiology 49, 207–219. Krigolson, O.E., Pierce, L.J., Holroyd, C.B., Tanaka, J.W., 2009. Learning to become an expert: reinforcement learning and the acquisition of perceptual expertise. J. Cogn. Neurosci. 21, 1834–1841. Kroll, J.F., Potter, M.C., 1984. Recognizing words, pictures, and concepts: a comparison of lexical, object, and reality decisions. J. Verbal Learn. Verbal Behav. 23, 39–66. Ladouceur, C.D., Dahl, R.E., Carter, C.S., 2007. Development of action monitoring through adolescence into adulthood: erp and source localization. Dev. Sci. 10, 874–891. Luft, C.D., 2014. Learning from feedback: the neural mechanisms of feedback processing facilitating better performance. Behav. Brain Res. 261, 356–368. Luft, C.D., Nolte, G., Bhattacharya, J., 2013. High-learners present larger mid-frontal theta power and connectivity in response to incorrect performance feedback. J. Neurosci. 33 (5), 2029–2038. Mangels, J.A., Butterfield, B., Lamb, J., Good, C., Dweck, C.S., 2006. Why do beliefs about intelligence influence learning success? A social cognitive neuroscience model.. Soc. Cogn. Affect. Neurosci. 1 (2), 75–86. Mars, R.B., Coles, M.G.H., Grol, M.J., Holroyd, C.B., Nieuwenhuis, S., Hulstijn, W., 2005. Neural dynamics of error processing in medial frontal cortex. Neuroimage 28, 1007–1013. Mathalon, D.H., Whitfield, S.L., Ford, J.M., 2003. Anatomy of an error: erp and fMRI. Biol. Psychol. 64, 119–141. Menon, V., Adleman, N.E., White, C.D., Glover, G.H., Reiss, A.L., 2001. Error-related brain activation during a go/nogo response inhibition task. Hum. Brain Mapp. 12, 131–143. Miltner, W.H.R., Braun, C.H., Coles, M.G.H., 1997. Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection. J. Cogn. Neurosci. 9, 788–798. Niv, Y., Chan, S., 2011. On the value of information and other rewards. Nat. Neurosci. 14, 1095–1097. Pietschmann, M., Simon, K., Endrass, T., Kathmann, N., 2008. Changes of performance monitoring with learning in older and younger adults. Psychophysiology 45, 559–568. Potts, G.F., Martin, L.E., Burton, P., Montague, P.R., 2006. When things are better or worse than expected: the medial frontal cortex and the allocation of processing resources. J. Cogn. Neurosci. 18, 1112–1119. Pressley, M., McCormick, C.B., 1995. Advanced educational psychology: for educators, researchers, and policymakers. Harper Collins College Publishers, New York. Rastle, K., Harrington, J., Coltheart, M., 2002. 358,534 nonwords: the ARC nonword database. Q. J. Exp. Psychol. 55A, 1339–1362. Sailer, U., Fischmeister, F.P., Bauer, H., 2010. Effects of learned helplessness on performance and event-related potentials in a learning task. Brain Res. 1342, 85–93. Santesso, D.L., Dillon, D., G., Birk, J.L., Holmes, A.J., Goetz, E., Bogdan, R., Pizzagalli, D.A., 2008. Individual Differences in Reinforcement Learning: behavioral, Electrophysiological, and Neuroimaging Correlates. Neuroimage 42 (2), 87–816. Schunk, D.H., Zimmerman, B.J., 1994. Self- regulation of learning and performance: Issues and educational applications. Erlbaum, Hillsdale, NJ. Schunk, D.H., Zimmerman, B.J., 1998. Self- regulated learning: From teaching to selfreflective practice. Guilford Press, New York. Spencer, K., Dien, J., Donchin, E., 2001. Spatiotemporal analysis of the late ERP responses to deviant stimuli. Psychophysiology 38, 343–358. Van den Bos, W., Güroğlu, B., van der Bulk, B.G., Rombouts, S.A.R.B., Crone, E.A., 2009. Better than expected or as bad as you thought? The neurocognitive development of probabilistic feedback processing. Front. Neurosci. 3. van der Helden, J., Boksem, M.A.S., Blom, J.H.G., 2010. The importance of failure: feedback-related negativity predicts motor learning efficiency. Cereb. Cortex 20 (7), 1596–1603. van Veen, V., Carter, C.S., 2002. The timing of action-monitoring processes in the anterior cingulate cortex. J. Cogn. Neurosci. 14, 593–602. Winne, P.H., Nesbit, J.C., 2010. The Psychology of academic achievement. Annu. Rev. Psychol. 61, 653–678. Zimmerman, B.J., 1994. Dimensions of academic self-regulation: a conceptual framework for education. In: Schunk, D.H., Zimmerman, B.J. (Eds.), Self-regulation and Learning And Performance: Issues And Educational Applications. Erlbaum, Hillsdale, NJ, 3–21. Zimmerman, B.J., 1998. Developing self-fulfilling cycles of academic regulation: an analysis of exemplary instructional methods. In: Schunk, D.H., Zimmerman, B.J. (Eds.), Self-regulated Learning. From Teaching to Self-reflective Practice. The Guilford Press, New York, London, 1–19. Zimmerman, B.J., Chunk, D., 1989. Self-regulated Learning and Academic Achievement: Theory, Research, and Practice. Springer-Varlag, New York. Zimmerman, B.J., Schunk, D.H., 2008. Motivation: an essential dimension of selfregulated learning. In: Schunk, D.H., Zimmerman, B.J. (Eds.), Motivation and Selfregulated Learning: Theory, Research, and Applications. Laurence Erlbaum, New York.

Acknowledgement This study was funded by a Faculty Research Fellowship from the MGH Institute of Health Professions, Boston, MA, United States (Grant no. 300259) to Yael Arbel. References Abar, B., Loken, E., 2010. Self-regulated learning and self-directed study in a pre-college sample. Learn. Individ. Differ. 20 (1), 25–29. Arbel, Y., Goforth, K., Donchin, E., 2013. The Good, the Bad, or the Useful? The examination of the relationship between the Feedback Related Negativity (FRN) and long-term learning outcomes. J. Cogn. Neurosci. 25 (8), 1249–1260. Arbel, Y., Murphy, A., Donchin, E., 2014. On the utility of positive and negative feedback in a paired-associate learning task. J. Cogn. Neurosci. 26 (7), 1445–1453. Baker, T.E., Holroyd, C.B., 2011. Dissociated roles of the anterior cingulate cortex in reward and conflict processing as revealed by feedback error-related negativity and N200. Biol. Psychol. 87, 25–34. Bellebaum, C., Daum, I., 2008. Learning-related changes in reward expectancy are reflected in the feedback-related negativity. Eur. J. Neurosci. 27 (7), 1823–1835. Blair, C., 2002. School readiness: integrating cognition and emotion in a neurobiological conceptualization of children’s functioning at school entry. Am. Psychol. 57, 111–127. Bromberg-Martin, E., Hikosaka, O., 2011. Lateral habenula neurons signal errors in the prediction of reward information. Nat. Neurosci. 14, 1209–1216. Butterfield, B., Mangels, J.A., 2003. Neural correlates of error detection and correction in a semantic retrieval task. Cogn. Brain Res. 17, 793–817. Butler, D.L., Winne, P.H., 1995. Feedback and self-regulated learning: a theoretical synthesis. Rev. Educ. Res. 65 (3), 245–281. Carter, C.S., Braver, T.S., Barch, D.M., Botvinick, M.M., Noll, D., Cohen, J.D., 1998. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749. Chase, H.W., Swainson, R., Durham, L., Benham, L., Cools, R., 2011. Feedback-related negativity codes prediction error but not behavioral adjustment during probabbilistic reversal learning. J. Cogn. Neurosci. 23 (4), 936–946. Cohen, M.X., Elger, C.E., Ranganath, C., 2007. Reward expectation modulates feedbackrelated negativity and EEG spectra. Neuroimage 35, 968–978. Critchley, H.D., Tang, J., Glaser, D., Butterworth, B., Dolana, R.J., 2005. Anterior cingulate activity during error and autonomic response. Neuroimage 27, 885–895. De Corte, E., Verschaffel, L., Op’ t Eynde, P., 2000. Self-regulation, a characteristic and a goal of mathematics education. In: Boekaerts, M., Pnitrich, P., Zeider, M. (Eds.), Handbook of Self-regulation. Academic press, USA. De la Fuente, J., Martínez, J.M., 2007. Scales for interactive assessment of the teaching learning process, IATLP. Education and Psychology, I+D+i: e-publishing, Almeria. Dehaene, S., Posner, M.I., Tucker, D.M., 1994. Localization of a neural system for error detection and compensation. Psychol. Sci. 5, 303–305. Efklides, A., 2011. Interactions of Metacognition with motivation and affect in selfregulated learning: the MASRL Model. Educ. Psychol. 46 (1), 6–25. Eppinger, B., Mock, B., Kray, J., 2009. Developmental differences in learning and error processing: evidence from ERPs. Psychophysiology 46, 1043–1053. Ernst, B., Steinhauser, M., 2012. Feedback-related brain activity predicts learning from feedback in multiple-choice testing. Cogn. Affect. Behav. Neurosci. 12 (2), 323–336. Frank, M.J., Woroch, B.S., Curran, T., 2005. Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47, 495–501. Foti, D., Weinberg, A., Dien, J., Hajcak, G., 2011. Event-related potential activity in the basal ganglia differentiates reward from non-rewards: temporal spatial principal components analysis and source localization of the feedback negativity. Hum. Brain Mapp. 32, 2207–2216. Gratton, G., Coles, M.G.H., Donchin, E., 1983. A new method for off-line removal of ocular artifact. Electroencephalogr. Clin. Neurophysiol. 55, 468–484. Greene, J.A., Azevedo, R., 2010. Introduction: the measurement of learners' selfregulated cognitive and metacognitive processes while using computer-based learning environments. Educ. Psychol. 45 (4), 203–209. Holroyd, C.B., Dien, J., Coles, M.G.H., 1998. Error-related scalp potentials elicited by hand and foot movements: evidence for an output-independent error-processing system in humans. Neurosci. Lett. 242, 65–68. Holroyd, C.B., Pakzad-Vaezi, K.L., Krigolson, O.E., 2008. The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology 45, 688–697. Kiehl, K.A., Liddle, P.F., Hopfinger, J.B., 2000. Error processing and the rostral anterior cingulate: an event-related fMRI study. Psychophysiology 37, 216–223. Kreussel, L., Hewig, J., Kretschmer, N., Hecht, H., Coles, M.G.H., Miltner, W.H.R., 2012. The influence of the magnitude, probability, and valence of potential wins and losses

20