- Email: [email protected]
Using semantics to enhance new word learning: An ERP investigation
Author's Accepted Manuscript
Using Semantics to enhance new word learning: An ERP investigation Anthony J. Angwin, Bernadette Phua, David A. Copland
www.elsevier.com/locate/neuropsychologia
PII: DOI: Reference:
S0028-3932(14)00150-X http://dx.doi.org/10.1016/j.neuropsychologia.2014.05.002 NSY5160
To appear in:
Neuropsychologia
Received date: 9 July 2013 Revised date: 4 April 2014 Accepted date: 3 May 2014 Cite this article as: Anthony J. Angwin, Bernadette Phua, David A. Copland, Using Semantics to enhance new word learning: An ERP investigation, Neuropsychologia, http://dx.doi.org/10.1016/j.neuropsychologia.2014.05.002 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Using Semantics to Enhance New Word Learning: An ERP Investigation
Anthony J. Angwin1, Bernadette Phua2, David A. Copland1,2
1
University of Queensland, School of Health and Rehabilitation Sciences, St Lucia, Queensland,
Australia, 4072 2
University of Queensland, Language Neuroscience Laboratory, Centre for Clinical Research,
Brisbane, Queensland, Australia, 4072
Correspondence to:
Anthony Angwin University of Queensland, School of Health and Rehabilitation Sciences Brisbane, Queensland, 4072 Australia Phone: +617 3346 7460 Fax:
+617 3365 1877
Email: [email protected]
1
Abstract This study aimed to investigate whether the addition of meaning (semantics) would enhance new word learning for novel objects, and whether it would influence the neurophysiological response to new words. Twenty-five young healthy adults underwent 4 days of training to learn the names of 80 novel objects. Half of the items were learnt under a ‘semantic’ condition, whereby the name consisted of a legal non-word and two adjectives denoting semantic attributes. The remaining items were learnt under a ‘name’ condition, whereby the name consisted of a legal non-word and two proper names. Participants demonstrated superior recognition of names in the semantic condition compared to the name condition during training sessions 1-3. On the 5th day, following training, ERPs were recorded whilst participants performed a picture-word judgement task including familiar items. Analysis of the results revealed an N400 for incongruent items in the semantic condition, whilst no ERP component was observed for the name condition. These findings suggest that items learnt with semantic information form stronger associations than those trained without semantics. Keywords Event-related potentials; learning; N400; semantics
2
1. Introduction Humans display an incredible capacity for learning and retaining vast numbers of words (Gaskell & Ellis, 2009). Not surprisingly, much research has been conducted on exploring the nature of new word learning in healthy adults, including research that examines whether the provision of semantic information impacts learning. A number of studies have investigated the learning of new names for novel objects, utilising pictures of ancient Finnish farm tools. The impact of semantic information on learning in such studies has been assessed by pairing stimuli with a description of the novel object’s function during learning. In a review of studies using this paradigm, Laine and Salmelin (2010) concluded that the retrieval of newly learned names is subserved predominantly by cortical regions in the left hemisphere, but that the nature of the initial training/learning task may impact on which left cortical regions are involved. Specifically, left cortical activity may vary according to whether participants are asked to learn only the object names, leading to subsequent word retrieval based on visual-phonological associations, or whether participants are asked to learn both the names and definitions, which may subsequently induce word retrieval via semantics. Of particular interest, studies by Cornelissen et al. (2004) and Whiting, Chenery, Chalk, Darnell and Copland (2007), did not find improved learning with the addition of semantic information. Cornelissen et al. (2004) suggested that the type of semantic information provided, which in their study was short dictionary style definitions for each object based on ancient Finnish farming culture, was neither enriching nor relevant enough to participants to facilitate superior recall. In contrast, Whiting et al. (2007) suggested that too much semantic information may be counterproductive in enhancing new word recall under limited time constraints (such that less time is spent learning the word form when additional semantic information is also presented). In order to address these issues, the current study restricts the semantic information to be
3
learnt to just two adjectives, and uses semantic attributes that are assumed to be more engaging so as to adequately facilitate the formation of a strong mental representation. The findings of Whiting et al. (2007) and Cornelissen et al. (2004) also suggest that in order to effectively compare new word learning for items trained with versus without semantic information, these conditions both need to involve the learning of comparable amounts of information. James and Gauthier (2004) devised a learning paradigm addressing this issue in a functional magnetic resonance imaging (fMRI) study associating semantic features with novel objects. In their semantic condition, the novel object was associated with a proper name and 3 semantic features. This condition was then contrasted with a control condition whereby the novel object was associated with 3 proper names. This design ensured that objects in both conditions would have names associated with them, but only objects from the semantic condition would have significant associations. Whilst the authors failed to find a learning advantage for novel objects trained with semantic features, the authors did observe increased activation in the left inferior frontal cortex during subsequent processing of these items relative to non-trained novel objects and novel objects that had been associated with proper names. Such findings indicate that training novel objects with semantics can lead to altered patterns of neural activation. However, these findings do not provide an indication of any temporal changes in neurophysiological activity. The current study will employ the measurement of event-related potentials (ERPs), which are better suited for monitoring rapidly occurring cognitive processes such as word recognition. The N400 is a negatively deflecting ERP component that usually peaks approximately 400ms after a stimulus is presented and is often elicited in response to semantic anomalies (Kutas & Federmeier, 2000; Kutas & Hillyard, 1980). Whilst classical N400 paradigms examine incongruous sentence endings (Kutas & Hillyard, 1980) or effects of semantic priming using word pair stimuli (Franklin, Dien, Neely, Huber, & Waterson, 2007; Hill, Ott
4
& Weisbrod, 2005; Smith, Chenery, Angwin & Copland, 2009), the N400 can also be elicited by incongruous picture-word pairs. In an object/written word matching task, Hurley et al. (2009) observed an N400 effect in healthy adults for mismatching object/word pairs. Similarly, Hamm, Johnson and Kirk (2002) observed an N400 for semantically mismatched pairs in a written word/picture task. Such paradigms may also be useful to explore new word learning, with the N400 providing an index of the strength of association between a novel object and a newly learnt word. In an ERP study of new word learning, McLaughlin, Osterhout and Kim (2004) utilised the N400 component from a prime-target lexical decision task to investigate how brain activity reflects the acquisition of a second language. The researchers observed an N400 effect for word-nonword pairs after approximately 14 hours of instruction in the second language. After further instruction, participants also displayed reduced N400 effects for related prime-target pairs relative to unrelated pairs. These findings indicated that adults learning a second language are able to acquire information regarding word form and word meaning at a remarkably fast rate. Other studies have also shown the N400 to be a useful index of new word learning. Perfetti, Wlotko and Hart (2005) trained adults in the meanings of rare unknown words. During a subsequent semantic judgement task on pairs of words, an N400 was observed for unrelated word pairs involving the trained words, but was not observed for those involving untrained words. Mestres-Misse, Rodriguez-Fornells and Munte (2007) used ERPs to investigate context-based learning of new words. Specifically, the researchers embedded novel word forms in the sentence final position of meaningful sentence triplet contexts, which participants read during the training phase. In a subsequent relatedness judgement task on word pairs consisting of a trained novel word (prime) and a real word (target), the researchers observed a reduction in the N400 for targets that were related to the novel word prime
5
relative to unrelated targets. In a similar study, Batterink and Neville (2011) examined contextual learning by embedding novel words within meaningful short story contexts. Analysis of ERPs from a subsequent relatedness judgement task using novel word primes and real word targets again revealed a reduction in the N400 for targets related to the novel word prime. Researchers have also shown the N400 to be sensitive to new word learning after just a single exposure to a novel word within the context of a highly constraining sentence (Borovsky, Elman & Kutas, 2012; Borovsky, Kutas & Elman, 2010). The results from such studies provide neurophysiological evidence for the acquisition of new word meaning. Similar findings have also been observed in a magnetoencephalography (MEG) study of new word learning by Dobel et al. (2009). The researchers paired novel words with pictures of known objects during a training phase. Some words were frequently paired with one particular object (learned set), whilst others were paired with many different objects (unlearned set). Prior to training, an N400m mismatch effect, the MEG counterpart to the N400, was observed for all novel words in a cross-modal priming task of word-picture pairs. After training was completed, however, the N400m was substantially reduced for wordpicture pairs involving the learned novel words, whereas a pronounced N400m effect was still observed for the untrained novel words. Accordingly, there is much evidence to suggest the N400 is a reliable index of new word learning, but it is unclear from such research whether the provision of semantics during training facilitates new word learning more than the provision of other types of information. Providing some potential insight into this issue is a new word learning study by Balass, Nelson and Perfetti (2010). The researchers asked participants to perform a semantic judgement task on related/unrelated word pairs after completion of a learning phase. The researchers found that responses to the semantic judgement task were faster and more accurate for words that had been paired with meaning during the learning phase. Similarly,
6
the researchers observed a larger reduction in the N400 effect for related trials involving words for which participants had learned the meaning. This reduction in the N400 was also more evident for words that were paired with orthography and meaning during training, compared to those that were paired with phonology and meaning. Overall, the findings from the study indicate that the nature of the encoding task, including whether it involves semantics, influences the accuracy and underlying neurophysiological mechanisms of new word learning. In contrast to Balass et al. (2010), the present study will investigate new word learning associated with novel objects. In summary, to address the limitations of previous studies on new word learning, this study will utilise distinct novel objects to examine the learning of new words associated with semantic attributes, control for the amount of information learnt in semantic and nonsemantic conditions, and will directly examine the learning of semantic information and its neural correlates using both behavioural measures and ERPs. Specifically, the learning of novel objects associated with a nonword and two semantic attributes (semantic condition) will be compared to the learning of a nonword and two proper names (name condition). Learning will be assessed via behavioural measures of recall and recognition, and an ERP picture-word judgement task will be used to assess the neurophysiological correlates of learning. It was hypothesised that learning new words associated with semantic information would result in superior learning rates and accuracy, as well as an enhanced N400 effect in the picture-word judgement task, compared to words learnt without semantic information. 2. Methods 2.1 Participants Twenty-five healthy young adults (10 males) with a mean age of 22 years (range 20– 26) participated in the study. Two participants were left handed and 23 participants were right handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971). All participants
7
were either native English speakers or proficiently fluent in English and reported no history of neurological illness, head injury, mental illness and developmental delay or disorder (including learning, speech and language). All participants had normal or corrected-to-normal vision, and received monetary reimbursement for their participation. 2.2 Stimuli In the learning paradigm used in this study, coloured images of 80 space aliens devised for use in word learning studies by Gupta et al. (2004) were each paired with a name. These unique alien drawings were ideal for this experiment as participants had to learn new associations between non-word names and novel objects that they had never seen before and for which there are no pre-existing names. The semantic (SEM) condition used names comprised of a legal non-word name (consisting of letter combinations that adhere to the rules of English; e.g., Zealt) associated with two semantic attributes (two adjectives; e.g., Timid Proud). The name (NAM) condition used names comprised of a legal non-word name (e.g., Proach) associated with two proper names (e.g., Mansey Smeath). A list of 80 non-words was generated from the ARC Nonword Database (4-6 letters long, neighbourhood size of 1-3 and summed frequency of neighbours between 5 and 60) (Rastle, Harrington & Coltheart, 2002) to be used as first names for the aliens. For the SEM condition, a list of 80 adjectives 4-6 letters long that were non-visual in modality was generated from the Medical Research Council (MRC) Psycholinguistic Database (Wilson, 1988), and these items were randomly paired to create 40 semantic attribute pairs. In any instance where a pair was found to contain inconsistent adjectives (e.g., Loud Quiet), these items were re-paired with other adjectives. For the NAM condition, 80 proper names were gathered from a list of names taken from records of the British Census between the years 1538 – 2005. In order to ensure that these names were not common enough to have any semantic meaning for participants, a list of 435 of the less common names were
8
compiled. A final list of 80 names, which three Bachelor of Speech Pathology graduates indicated were all unfamiliar names, was compiled and these names were randomly paired to produce 40 proper name pairs. Two lists of stimuli were subsequently created, each containing 40 alien/name pairs in the SEM condition and 40 alien/name pairs in the NAM condition. Each list consisted of the same stimuli, but with different alien/name pairings such that nonword names assigned to the SEM or NAM condition in list 1 were assigned to the other condition (and a different alien picture) in list 2. The two sets of nonwords, which were used as the alien names in the SEM/NAM conditions for each list, did not differ in length (t(39) = 1.88, p = 0.435), neighbourhood size (t(39) = -1.27, p = 0.485) or summed frequency of neighbours (t(39) = 0.75, p = 0.245). The use of these two different lists was then counterbalanced across participants for each of the experimental tasks.
2.3 Learning Task Participants took part in four 1 hour computerised training sessions that were conducted over 4 consecutive days. During these training sessions, participants completed a learning task followed by criterion testing comprised of a recall task and a recognition task. E-prime 1.1 software (Psychology Software Tools Pittsburgh, PA, USA) was used to present the stimuli for all of these tasks as well as record responses to the recall and recognition tasks. The purpose of criterion testing was to ensure that the participants gradually learnt all the names. By the end of the fourth session, participants were required to have achieved a criterion of at least 90% accuracy on recognition tasks. In the learning task, participants were seated at a computer and presented with the 80 space aliens together with their 3 word written name (40 SEM and 40 NAM) one at a time. Each alien/name was presented 3 times per session, resulting in a total of 240 trials per
9
session. Each trial consisted of a fixation point ‘+’ for 1500ms, followed by a picture of an alien in the centre of the computer screen together with its 3 written names presented underneath the picture for 5000ms. The next trial would then begin automatically after a 500ms blank screen (Figure 1). Short rest breaks were provided after every 20 trials.
During the recall task, the 80 alien pictures were presented one at a time in a random order in the centre of the computer screen. For each picture, participants were instructed to use the computer keyboard to type only the first name (i.e., the nonword name) of that alien and then press ‘enter’ on the keyboard. The name that the participant typed would appear underneath the picture with each keystroke. Each picture stimulus remained on screen until the participant hit the ‘enter’ key, and then the next alien picture was displayed automatically. In the recognition task, the 80 alien pictures were again presented in a random order one at a time on the computer screen, together with a 3 word written name presented underneath the picture. The written name was either the correct 3 names for that picture or the 3 names of another alien learnt in the same session. The participants were required to indicate whether the 3 word name was correct or incorrect using the computer mouse, by pressing the left mouse button with their index finger if the name was correct and pressing the right mouse button with their middle finger if the name was incorrect. After a response had been provided by the participant, the next trial was then displayed automatically.
2.4 Cognitive Testing Immediately prior to EEG testing, participants completed three cognitive assessment tasks to obtain measures of short term memory, reading skill and an IQ estimate. These measures were taken in order to determine whether such aspects of cognition impacted learning success for the different conditions. The assessments used were the Repeatable
10
Battery for the Assessment of Neuropsychological Status (RBANS) Digit Span subtest (Randolf, 1998), Neuropsychological Assessment Battery (NAB) Digits Backwards subtest (Stern & White, 2003) and the National Adults Reading Test (NART; Nelson & Willison, 1991). The results from these tests are summarised in Table 1.
2.5 EEG Procedure On the fifth day, following the last training session, participants completed a computerised recognition task comprised of 200 stimulus items presented whilst continuous EEG data was acquired. The 200 stimulus items were comprised of the following: i) congruous semantic name and image pair (sem-con, n=20) ii) incongruous semantic name and image pair (sem-incon, n=20) iii) congruous proper name and image pair (nam-con, n=20) iv) incongruous proper name and image pair (nam-incon, n=20) v) congruous familiar object and name pair (fam-con, n=40) vi) incongruous familiar object and name pair (fam-incon, n=40) vii) novel name and learnt image pairs (nov, n = 40) For conditions (i) to (iv), the stimuli consisted of the pictures learned during the training sessions, paired with either the correct nonword name learned during training (congruent) or an incorrect nonword name that had been learned with a different picture during training (incongruent). Conditions (v) and (vi) included pictures of familiar (FAM) objects (Snodgrass & Vanderwart, 1980) paired with either the correct word for that picture (congruent) (e.g., hammer - hammer) or an incorrect word (incongruent) (e.g., candle lizard). These FAM conditions were used as a control to compare ERP differences between known objects (FAM) and novel objects (SEM, NAM). The novel name and learnt image
11
pairs (condition (vii)) consisted of alien pictures learnt during the training sessions paired together with a novel nonword name that had not been presented during training. This condition was employed to account for repeated exposure to nonword letter strings, compared to entirely novel nonwords, however due to a large number of participant errors made on this condition during task performance it could not subsequently be included in the analyses. The experiment was divided into four blocks with each block containing an equal number of stimuli randomly selected from each condition. The procedure for each trial was as follows: a fixation cross was presented for 1000ms, followed by a picture for 3000ms, then a blank screen for 500ms followed by a written nonword name. When the nonword name appeared, participants had to decide whether the name matched the picture they had just seen, and respond by clicking a button on the Serial Response Box (Psychology Software Tools, Pittsburgh, PA, USA). Participants were asked to click the left button with their index finger if the nonword name was correct, and to click the right button with their middle finger if the nonword name was incorrect.
2.6 Post Tests In order to assess how well participants had learnt all aspects of the alien names, participants were asked to complete two final computerised recognition tasks immediately after the EEG task. In the first task, each trial consisted of 2 aliens presented side by side on the computer screen with 1 nonword name (i.e., an alien’s first name) written centrally towards the bottom of the screen. Participants used the left and right computer mouse buttons to indicate which of the 2 aliens (i.e., left or right of screen) correctly matched the written nonword name. In the second task, each trial consisted of 2 aliens presented side by side on the computer screen with 2 second names (i.e., either two semantic attributes (SEM condition) or two proper names (NAM condition)) written centrally towards the bottom of the screen. Participants again used the computer mouse buttons to indicate which of the 2 aliens 12
correctly matched the written name. For both post-tests, each trial was preceded by a 1000ms fixation cross and was followed by a 1000ms blank screen.
2.7 Data Analysis A Shapiro-Wilk test was conducted on all data sets to test for normality. An arc sine transformation was applied to the recall and recognition accuracy data to normalise the distribution. Similarly, a log transformation was applied to the recognition RT data to normalise the distribution. The training session data was subsequently analysed with a series of repeated-measures analysis of variance (ANOVA) using condition (SEM, NAM) and session (1-4) as within-subjects factors, and accuracy (recall and recognition tasks) and reaction time (RT) (recognition task only) as the dependent variables. Spearman’s correlation was also used to determine whether performance on the Digit Span, Digits Backwards or the NART was associated with performance in training session 4. The ERP accuracy data were also transformed with an arc sine transformation to normalize the distribution of the data, while the RT data was normalised with a log transformation. The ERP data was then analysed separately for the SEM/NAM conditions and the FAM condition using repeated-measures ANOVAs. Post test behavioural data, using condition (SEM, NAM) as the independent variable and accuracy and RT as the dependent variables, were analysed with paired samples t-tests. For the second post test, data was not collected from 6 participants due to computer data logging errors.
2.8 ERP Recording and Analysis ERP data was obtained using a 128 channel high-density EEG system (Electrical Geodesics, Inc.) with a sampling rate of 500Hz. Electrode impedance was kept below 50k,
13
which is acceptable with the use of high impedance amplifiers (Ferree, Luu, Russell, & Tucker, 2001). Netstation 4.4.2 (Electrical Geodesics, Inc.) was used to process all ERP data offline. ERP data from 4 participants was excluded due to excessive EEG artifacts. For the remaining 21 participants, data was digitally filtered from 0.1-30 Hz and segmented into 1100 ms epochs commencing 100ms prior to the onset of the target name. Trials upon which participants made an incorrect response were excluded from analysis, resulting in the exclusion of 4.49% of the total data set (sem-con 5.95%; sem-incon 7.86%; nam-con 7.86%; nam-incon 10.48%; fam-con 1.07%; fam-incon 0.83%). An ocular artefact reduction procedure (Gratton, Coles & Donchin, 1983) was applied to data containing eye movements and blinks. Any subsequent trials consisting of remaining ocular artefacts (defined as vertical or horizontal electro-oculogram channel differences of greater than 70 μV) or that consisted of more than 20% bad channels (defined as reaching amplitudes greater than 200 μV) were subsequently excluded from analysis. The data was re-referenced to the average of all electrodes and baseline corrected to the 100ms pre-target interval. Twelve electrodes (6 within each hemisphere), which provided a well distributed sample of the recording electrode sites, were selected for analysis. Based upon visual inspection of this data, a 200-500ms time window was selected for analysis of the N400. Repeated-measures ANOVAs were used to analyse the mean amplitude of the data within this time region, using condition (FAM, SEM, NAM), congruency (congruent, incongruent), hemisphere of electrode site (Left, Right) and electrode (6 levels) as the independent variables. The Greenhouse-Geisser correction was utilised when the assumption of sphericity was violated (corrected p value reported).
14
3. Results 3.1 Behavioural Results 3.1.1 Training Accuracy For the recall task, the participants’ accuracy improved over the four training sessions (Figure 2) as evidenced by a main effect of session (F(3,72) = 111.91, p < 0.001), however there was no significant main effect of condition (p = 0.268) nor an interaction between session and condition (p = 0.314). For the recognition task (Figure 2), main effects of session and condition were evident (F(3,72) = 93.04, p < 0.001; and F(1,24) = 27.34, p < 0.001, respectively), indicating that overall accuracy improved over the 4 training sessions and that overall accuracy was higher for the SEM condition relative to the NAM condition. There was also a significant session by condition interaction (F(3,72) = 8.91, p < 0.001), indicating that names in the SEM condition were learnt faster, as demonstrated by t-tests showing significantly better recognition for SEM items in sessions 1-3 (t(24) = -6.82, p < 0.001, t(24) = -3.79, p = 0.001, and t(24) = -3.17, p = 0.004, respectively). By the fourth session, this difference was no longer significant (p = 0.173) as participants were able to accurately recognise most alien-name pairs for both conditions.
3.1.2 Correlations No significant correlations were observed between any of the cognitive tests (Digit Span, Digit Backwards and NART IQ) and recall accuracy for the SEM or NAM conditions in training session 4. Similarly, no significant correlations were evident between cognitive testing and recognition accuracy in training session 4.
15
3.1.3 Training Reaction Time For RT in the recognition task, there was again a significant improvement over the four training sessions (Table 2) as evidenced by a main effect of session (F(3,72) = 58.45, p < 0.001). There was no significant effect for condition (p = 0.989) or condition X session (p = 0.556).
3.1.4 ERP task behavioural performance For accuracy during the ERP task, comparing only the SEM and NAM conditions alone, accuracy was equivalent for all conditions (Table 3) as indicated by the absence of significant effects for condition (p = .216), congruency (p = 0.148) and condition X congruency (p = 0.903). For RT in the EEG task (Table 3), there was no main effect of condition (p = .845) or condition X congruency interaction (p = 0.946), however, a main effect of congruency (F(1,24) = 22.975, p < 0.001) indicated that RTs were faster for congruent items compared to incongruent items. Analysis of the FAM condition data revealed no difference in accuracy between congruent and incongruent items (p = .720), however RTs were faster in the congruent relative to the incongruent condition (F(1,24) = 13.36, p = .001).
3.1.5 Post Tests Analysis of the data from the first post test, where participants were tested on recognition of first names only (i.e., the nonword name), revealed no significant difference in accuracy (p = 0.986) or RT (p = 0.264) between names that had previously been paired with semantic attributes and names that had been paired with proper names (Table 4). In the second post test, where participants were tested on their recognition of the two last names
16
only (either two semantic attributes or two proper names), participants displayed higher accuracy for recognising semantic attributes than proper names (t(18) = -3.72, p = 0.002) and faster RTs for semantic attributes than proper names (t(18) = 5.06, p < 0.001) (Table 4).
3.2 ERP Results Mean amplitude of the N400 (200-500ms) was analysed with repeated measures ANOVA using condition (FAM, SEM, NAM), congruency (congruent, incongruent), hemisphere (Left, Right) and electrode (6 levels) as independent variables. This analysis revealed main effects of condition (F(2,40) = 3.78, p = 0.031), hemisphere (F(1,20) = 78.65, p < .001) and electrode (F(5,100) = 32.04, p < .001), as well as interaction effects of congruency X hemisphere (F(1,20) = 10.71, p = .004), congruency X hemisphere X electrode (F(5,100) = 4.75, p = .001) and condition X congruency X hemisphere (F(2,40) = 4.77, p = .014). In light of the interaction between condition, congruency and hemisphere, the data was subsequently explored further using additional repeated measures ANOVAs, which were conducted separately for each condition. Main effects of hemisphere and electrode are not reported. Analysis of the FAM condition data revealed a significant main effect of congruency (F(1,20) = 6.62, p = .018), and interaction effects of congruency X hemisphere (F(1,20) = 6.98, p = .016) and congruency X hemisphere X electrode (F(5,100) = 4.00, p = .002). Further analysis within each hemisphere with congruency and electrode as independent variables revealed an N400 effect in the right hemisphere as evidenced by a main effect of congruency in the right hemisphere (F(1,20) = 12.46, p = .002), and no main or interaction effects of congruency in the left hemisphere (Figure 3). Analysis of the SEM condition revealed a main effect of congruency (F(1,20) = 4.41, p = .049) and a significant interaction effect of congruency X hemisphere (F(1,20) = 12.80, p = .002). Further analysis within each
17
hemisphere indicated a significant N400 in the right hemisphere, as evidenced by a significant main effect of congruency in the right hemisphere (F(1,20) = 10.91, p = .004), but no significant main or interaction effects of congruency in the left hemisphere (Figure 3). Analysis of the NAM condition revealed no significant main or interaction effects involving congruency (Figure 3).
To determine whether a difference in the latency of the N400 was evident between the FAM and SEM conditions, difference waves were calculated (incongruent minus congruent) for each condition (figure 4) and the peak latency of the N400 was calculated within the 200500ms time window. This data was analysed using a repeated measures ANOVA, with condition (FAM, SEM), hemisphere (Right, Left) and electrode (6 levels) as independent variables. No significant effects were evident, indicating a similar latency of the N400 for both conditions.
In order to further explore the temporal and topographical distribution of the N400 effects evident for the SEM and FAM conditions, and to accommodate for the possibility of later effects beyond the 200-500ms time window, the mean amplitude for the SEM and FAM conditions was calculated within 100ms time segments from 200-700ms. This data was calculated across 30 different scalp regions, encompassing 126 electrodes (the nasion and inferior eye channels were excluded). The data within each time segment was analysed using repeated measures ANOVAs, with condition (FAM, SEM), congruency (congruent, incongruent) and region (30 levels) as independent variables. The analyses within each time segment revealed no significant interaction between condition and congruency, nor any significant 3 way interaction of condition x congruency x region, illustrating a similar topographical distribution of effects for the SEM and FAM conditions over time (Figure 5).
18
4. Discussion The aim of this study was to investigate whether the addition of semantic attributes would enhance new word learning for novel objects and whether it would influence neurophysiological activity. It was hypothesised that learning new names for novel objects that are associated with semantic attributes would be superior to learning new names presented without semantics. It was also hypothesised that N400 effects would be more prominent for names learnt with semantic information.
4.1 Behavioural results Participants demonstrated improved recognition of names learnt in the SEM condition compared to the NAM condition over training sessions 1-3, however this advantage was no longer evident at session 4. This pattern of results suggests that semantics may provide a learning advantage in the earlier stages of new word acquisition. It should be noted that whilst greater recognition accuracy was observed for the semantic names in the initial training sessions, the recall task did not show this same advantage. This could partially be attributed to differences in the level of task difficulty. Most participants achieved the criterion of 90% accuracy for recognition by the end of training. However, for the recall task in our study, only 9 out of the 25 participants reached criterion by the end of training. Given suggestions that recall is typically more difficult than recognition (Haist, Shimamura & Squire, 1992), it is possible that in the present experiment semantic information assisted in the processes involved in recognition, but not in the retrieval processes that are required for recall. Also worthy of note, however, is that participants only had to recall the nonword name for the recall task, but were presented with all three names during the recognition task. Hence, the results suggest that participants may have, to some
19
extent, learned a direct association between the picture and the semantic adjectives, which subsequently provided an advantage for the SEM condition in the recognition task, but not for the recall task. Turning to the results of the post-tests, analysis of the first post-test data revealed that participants were able to match the nonword names to the alien pictures with similar accuracy and RT for items from the SEM and NAM conditions. This finding is expected given that participants also had similar recognition accuracy for both conditions after the 4th training session. However in the second post-test, which assessed how well participants learnt the additional alien names in each condition (i.e., semantic attributes or proper names), the semantic attributes were recognised with greater accuracy and faster RTs than proper names. These results confirm that participants learnt the semantic information more effectively than the proper names.
4.2 ERP Results To investigate the neurophysiological effects of semantic information on new word learning, mean amplitudes in the 200-500ms region were examined in order to target the N400 component, an index of lexical-semantic processing. Similar to previous research involving paired picture-word or word-picture stimuli (Hamm et al., 2001; Hurley et al., 2009), an N400 was elicited to incongruent stimuli in the right hemisphere for the FAM condition. This N400 effect is indicative of the participant’s sensitivity to the semantically incongruent picture-word pairs in the FAM condition, providing a useful index of semantic processing. The absence of an N400 in the left hemisphere was not surprising, given that other ERP studies have observed a more prominent N400 in the right hemisphere (Hurley et al., 2009; Kutas & Federmeier, 2011).
20
A significant N400 effect was also observed in the 200-500ms region for incongruent items from the SEM condition, which was again restricted to the right hemisphere. This result adds to previous research that has shown the N400 to be a reliable index of word learning (Balass et al., 2010; Batterink and Neville, 2011; McLaughlin et al., 2004; Mestres-Misse et al., 2007; Perfetti et al., 2005). The similar topographical distribution of the N400 over time for both the FAM and SEM conditions also suggests that by the end of training, correctly recognised items from the SEM condition are processed similarly to already known/familiar items. Interestingly, previous studies of new word learning have differed with respect to the distribution of the N400 effect obtained in semantic judgement tasks after learning, with some research finding an N400 over frontal and parietal sites for both newly learned and familiar words (Frishkoff, Perfetti & Collins-Thompson, 2010), and others finding that the N400 for newly learned words has a more frontal distribution relative to known familiar words (Mestres-Misse et al., 2007). Further research is clearly needed to establish which stimulus factors impact the distribution of N400 effects for new versus familiar words. Whilst an N400 effect was observed for the SEM condition in the right hemisphere, no N400 effect was evident in either hemisphere for the NAM condition. Importantly, this finding is evident despite the fact that only nonword names were presented during the ERP task. Thus, the presence of an N400 for the SEM but not the NAM condition cannot be attributed to a learned direct association between the semantic adjectives and the picture. Instead, this finding suggests that nonword names from the SEM condition were more deeply encoded, resulting in a stronger lexical-semantic representation as indexed by the N400 effect for the SEM condition but not the NAM condition. The results demonstrate that although recognition accuracy was similar for both NAM and SEM conditions by the 4th training session, a processing advantage for the nonwords was still evident for the SEM condition at a neurophysiological level.
21
Worthy of note is that other studies have found increases in the N400 during speech segmentation of nonsense words, potentially reflecting processes relating to the learning and segmentation of the nonsense word stimuli (Cunillera, Toro, Sebastian-Galles & RodriguezFornells, 2006; Cunillera et al., 2009; Sanders, Newport & Neville, 2002). Thus, the N400 appears to have the capacity to index differential components of word learning, depending on the nature of the paradigm being used. Clearly, ongoing research is still required in order to elucidate our current understanding of the underlying cognitive processes that contribute to N400 learning effects. Overall, these novel findings provide further evidence that semantic information facilitates new word learning and add to the existing literature that suggests the inclusion of semantic information during word learning modifies neural activity (Balass et al., 2010; James & Gauthier, 2004). There are some limitations that could be addressed in future related studies. The present study only measured ERPs on the 5th day, after training had been completed. Future studies could implement ERP measurements on each day of training in order to more effectively track changes in learning and how such changes may differ for the NAM and SEM conditions. The existing ERP design was also limited by a small number of stimuli for the NAM and SEM conditions, and the ERP data may have been impacted by the motor responses that were required for the semantic judgement task. It would also be of interest to compare the effect of different forms of semantic information on new word learning, such as auditory semantics (e.g., honks, squeaks) relative to visual semantics (e.g., tall, fat). Previous fMRI and ERP studies have examined the learning of novel words from contextual information (Batterink et al., 2011; Mestres-Misse et al., 2007; Mestres-Misse, Camara, Rodriguez-Fornells, Rotte, & Munte, 2008), and it may be useful to apply similar paradigms to the learning of novel object names. Given that some ERP studies have found that reading skill can influence learning (Balass et al., 2010; Perfetti et al.,
22
2005), reading comprehension skill should also be factored into any ongoing research on new word learning for novel objects. Lastly, an MEG word learning study by Hulten, Laaksonen, Vihla, Laine and Salmelin (2010) showed that changes in neural activation from the end of the learning phase to one week after learning predicted the retention of the new vocabulary at 10 months post training. Future ERP investigations of new word learning should aim to identify any neural markers that predict learning success.
4.3 Conclusions This study explored the use of semantics in learning new names for novel objects and examined the neurophysiological impact of this approach. Recognition of items from the SEM condition was superior to that of the NAM condition during the first 3 training sessions. An N400 effect was also observed for incongruent items in the SEM condition, but not the NAM condition. These findings suggest that semantics provides a learning advantage in new word learning, which could be attributed to the development of more elaborate lexicalsemantic representations.
23
5. References Balass, M., Nelson, J.R., & Perfetti, C.A. (2010). Word learning: An ERP investigation of word experience effects on recognition and word processing. Contemporary Educational Psychology, 35, 126-140. Batterink, L., & Neville, H. (2011). Implicit and explicit mechanisms of word learning in a narrative context: an event-related potential study. Journal of Cognitive Neuroscience, 23(11), 3181-3196. Borovsky, A., Elman, J.L., & Kutas, M. (2012). Once is enough: N400 indexes semantic integration of novel word meanings from a single exposure in context. Language, Learning and Development, 8, 278-302. Borovsky, A., Kutas, M., & Elman, J. (2010). Learning to use words: Event-related potentials index single-shot contextual word learning. Cognition, 116, 289-296. Cornelissen, K., Laine, M., Renvall, K., Saarinen, T., Martin, N., & Salmelin, R. (2004). Learning new names for new objects: Cortical effects as measured by magnetoencephalography. Brain and Language, 89(3), 617-622. Cunillera, T., Camara, E., Toro, J.M., Marco-Pallares, J., Sebastian-Galles, N., Ortiz, H., Pujol, J., & Rodriguez-Fornells, A. (2009). Time course and functional neuroanatomy of speech segmentation in adults. NeuroImage, 48, 541-553. Cunillera, T., Toro, J.M., Sebastian-Galles, N., & Rodriguez-Fornells, A. (2006). The effects of stress and statistical cues on continuous speech segmentation: An event-related brain potential study. Brain Research, 1123, 168-178. Dobel, C., Junghofer, M., Breitenstein, C., Klauke, B., Knecht, S., Pantev, C., et al. (2009). New names for known things: On the association of novel word forms with existing semantic information. Journal of Cognitive Neuroscience, 22(6), 1251-1261.
24
Ferree, T.C., Luu, P., Russell, G.S., & Tucker, D.M. (2001). Scalp electrode impedance, infection risk, and EEG data quality. Clinical Neurophysiology, 112, 536-544. Franklin, M.S., Dien, J., Neely, J.H., Huber, E., & Waterson, L.D. (2007). Semantic priming modulates the N400, N300, and N400RP. Clinical Neurophysiology, 118(5), 10531068. Frishkoff, G.A., Perfetti, C.A., & Collins-Thompson, K. (2010). Lexical quality in the brain: ERP evidence for robust word learning from context. Developmental Neuropsychology, 35(4), 376-403. Gaskell, M.G., & Ellis, A.W. (2009). Word learning and lexical development across the lifespan. Philosophical Transactions of the Royal Society B-Biological Sciences, 364(1536), 3607-3615. Gratton, G., Coles, M.G.H., & Donchin, E. (1983). A new method for off-line removal of ocular artifacts. Electroencephalography and Clinical Neurophysiology, 55, 468-484. Gupta, P., Lipinski, J., Abbs, B., Lin, P.H., Aktunc, E., Ludden, D., et al. (2004). Space aliens and nonwords: Stimuli for investigating the learning of novel word-meaning pairs. Behavior Research Methods Instruments & Computers, 36(4), 599-603. Haist, F., Shimamura, A.P., & Squire, L.R. (1992). On the relationship between recall and recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(4), 691-702. Hamm, J.P., Johnson, B.W., & Kirk, I.J. (2002). Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts. Clinical Neurophysiology, 113(8), 1339-1350. Hill, H., Ott, F., & Weisbrod, M. (2005). SOA-dependent N400 and P300 semantic priming effects using pseudoword primes and a delayed lexical decision. International Journal of Psychophysiology, 56, 209-221.
25
Hulten, A., Laaksonen, H., Vihla, M., Laine, M., & Salmelin, R. (2010). Modulation of brain activity after learning predicts long-term memory for words. The Journal of Neuroscience, 30(45), 15160-15164. Hurley, R.S., Paller, K.A., Wieneke, C.A., Weintraub, S., Thompson, C.K., Federmeier, K.D., et al. (2009). Electrophysiology of object naming in primary progressive aphasia. Journal of Neuroscience, 29(50), 15762-15769. James, T.W., & Gauthier, I. (2004). Brain areas engaged during visual judgments by involuntary access to novel semantic information. Vision Research, 44(5), 429-439. Kutas, M., & Federmeier, K.D. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences, 4(12), 463-470. Kutas, M., & Federmeier, K.D. (2011). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62, 621–647. Kutas, M., & Hillyard, S.A. (1980). Reading senseless sentences: brain potentials reflect semantic incongruity. Science, 207(4427), 203-205. Laine, M., & Salmelin, R. (2010). Neurocognition of new word learning in the native tongue: lessons from the ancient farming equipment paradigm. Language Learning, 60, 25-44. McLaughlin, J., Osterhout, L., & Kim, A. (2004). Neural correlates of second-language word learning: minimal instruction produces rapid change. Nature Neuroscience, 7(7), 703704. Mestres-Misse, A., Camara, E., Rodriguez-Fornells, A., Rotte, M., & Munte, T.F. (2008). Functional neuroanatomy of meaning acquisition from context. Journal of Cognitive Neuroscience, 20(12), 2153-2166. Mestres-Misse, A., Rodriguez-Fornells, A., & Munte, T.F. (2007). Watching the brain during meaning acquisition. Cerebral Cortex, 17, 1858-1866.
26
Nelson, H. E., & Willison, J. R. (1991). The Revised National Adult Reading Test. Windsor, UK: NFER Nelson. Oldfield, R.C. (1971). Assessment and analysis of handedness - Edinburgh Inventory. Neuropsychologia, 9(1), 97-113. Perfetti, C.A., Wlotko, E.W., & Hart, L.A. (2005). Word learning and individual differences in word learning reflected in event-related potentials. Journal of Experimental Psychology: Learning, Memory and Cognition, 31(6), 1281-1292. Randolph, C. (1998). Repeatable Battery for the Assessment of Neuropsychological Status. San Antonio: Harcourt Assessment. Rastle, K., Harrington, J., & Coltheart, M. (2002). 358,534 nonwords: The ARC Nonword Database. Quarterly Journal of Experimental Psychology, 55A, 1339-1362. Sanders, L.D., Newport, E.L., & Neville, H.J. (2002). Segmenting nonsense: an event related potential index of perceived onsets in continuous speech. Nature Neuroscience, 5, 700-703. Smith, E.R., Chenery, H.J., Angwin, A.J., & Copland, D.A. (2009). Hemispheric contributions to semantic activation: A divided visual field and event-related potential investigation of time course. Brain Research, 1284, 125-144. Snodgrass, J.G., & Vanderwart, M. (1980). Standardized set of 260 pictures - Norms for name agreement, image agreement, familiarity and visual complexity. Journal of Experimental Psychology-Human Learning and Memory, 6(2), 174-215. Stern, R.A., & White, T. (2003). The Neuropsychological Assessment Battery. Lutz, FL: Psychological Assessment Resources, Inc. Whiting, E., Chenery, H., Chalk, J., Darnell, R., & Copland, D. (2007). Dexamphetamine enhances explicit new word learning for novel objects. International Journal of Neuropsychopharmacology, 10(6), 805-816.
27
Wilson, M. (1988). MRC psycholinguistic database: machine-usable dictionary, version 2.00. Behavior Research Methods, Instruments, and Computers, 20, 6–10.
28
Figure 1: An illustration of the procedure for the learning task. Figure 2: Mean proportion of correct responses for the semantic (SEM) and name (NAM) conditions at each training session for the recall (A) and recognition (B) tasks.Error bars represent standard error. Figure 3: Grand-averaged ERPs for congruent and incongruent stimuli in each condition. Figure 4: Difference waves (incongruent minus congruent) for the SEM and FAM conditions. Figure 5: Topographic maps (incongruent minus congruent) for the SEM and FAM conditions from 200-700ms. Maps oriented with nose facing up.
29
1500 ms
Figure 1
+
5000 ms
500 ms
Zealt Timid Proud
Fixation Cross
1500 ms
Stimulus
+
5000 ms
500 ms
Stimulus
Fixation Cross (beginning of next trial)
Proach Mansey Smeath
Blank screen
Blank screen
Figure 2
1
Proportion of Correct Responses
A.
0.8
0.6 SEM NAM
0.4
0.2
0 1
2
3
4
Training session
B. Proportion of Correct Responses
1
0.8
0.6 SEM NAM
0.4
0.2
0 1
2
3
Training session
4
Figure 3
NAM
SEM
FAM
+6 μV
Congruent -100
-6 μV
100
300
500
700
900
Incongruent
Figure 4
+6 μV
FAM -100
-6 μV
100
300
500
700
900
SEM
Figure 5
200-300ms
300-400ms
400-500ms
500-600ms
600-700ms
FAM
SEM
Table 1. Participant demographics and cognitive test scores (n=25) ________________________________________________________________________ Measure Mean S.D ________________________________________________________________________ Age (years) 22.04 1.62 Years of education
15.44
0.85
RBANs Digit Span (max. score: 16)
12.88
2.89
NAB Digits Backwards (max. score: 88)
40.72
21.00
NART (raw score, max. score: 50)
32.32
5.56
NART (estimated IQ equivalent) 116.72 5.78 _________________________________________________________________________ RBANS, Repeatable Battery for the Assessment of Neuropsychological Status, Digit Span subtest (Randolf, 1998); NAB, Neuropsychological Assessment Battery, Digits Backwards subtest (Stern & White, 2003); NART, National Adults Reading Test (Nelson & Willison, 1991).
30
Table 2: Mean RT on the recognition task at each training session for each condition (correct responses only). ________________________________________________ Training session SEM NAM ________________________________________________ 1 3208 (929) 3081 (1022) 2
2614 (1214)
2676 (1508)
3
1954 (732)
2000 (720)
4 1702 (530) 1751 (599) ________________________________________________ Standard deviations in brackets.
31
Table 3: Mean proportion of correct responses and mean RT for each condition during the ERP task. _____________________________________________________ Condition Accuracy RT _____________________________________________________ Sem-con
0.95 (0.08)
850 (378)
Sem-incon
0.93 (0.10)
1024 (489)
Nam-con
0.93 (0.09)
828 (311)
Nam-incon
0.91 (0.12)
997 (428)
Fam-con
0.99 (0.02)
719 (172)
Fam-incon 0.99 (0.01) 792 (215) _____________________________________________________ Standard deviations in brackets.
32
Table 4: Mean proportion accuracy and RT for each condition at each post-test. _____________________________________________________________________ Post-test 1 (n=25) Post-test 2 (n=19) Condition Acc RT Acc RT ______________________________________________________________________ SEM 0.95 (0.08) 1659 (469) 0.88 (0.12) 2578 (803) NAM 0.96 (0.06) 1723 (531) 0.79 (0.14) 3147 (873) ______________________________________________________________________ Standard deviations in brackets
33
Highlights x x x x
Theimpactofsemanticsonnewwordlearningfornovelobjectswasassessed. Aftertraining,ERPswererecordedduringapicturewordsemanticjudgementtask AnN400wasobservedforstimulithatwerelearnedwithsemanticinformation AnN400wasnotobservedforstimulilearnedwithoutsemanticinformation
34
Using Semantics to enhance new word learning: An ERP investigation Anthony J. Angwin, Bernadette Phua, David A. Copland
www.elsevier.com/locate/neuropsychologia
PII: DOI: Reference:
S0028-3932(14)00150-X http://dx.doi.org/10.1016/j.neuropsychologia.2014.05.002 NSY5160
To appear in:
Neuropsychologia
Received date: 9 July 2013 Revised date: 4 April 2014 Accepted date: 3 May 2014 Cite this article as: Anthony J. Angwin, Bernadette Phua, David A. Copland, Using Semantics to enhance new word learning: An ERP investigation, Neuropsychologia, http://dx.doi.org/10.1016/j.neuropsychologia.2014.05.002 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Using Semantics to Enhance New Word Learning: An ERP Investigation
Anthony J. Angwin1, Bernadette Phua2, David A. Copland1,2
1
University of Queensland, School of Health and Rehabilitation Sciences, St Lucia, Queensland,
Australia, 4072 2
University of Queensland, Language Neuroscience Laboratory, Centre for Clinical Research,
Brisbane, Queensland, Australia, 4072
Correspondence to:
Anthony Angwin University of Queensland, School of Health and Rehabilitation Sciences Brisbane, Queensland, 4072 Australia Phone: +617 3346 7460 Fax:
+617 3365 1877
Email: [email protected]
1
Abstract This study aimed to investigate whether the addition of meaning (semantics) would enhance new word learning for novel objects, and whether it would influence the neurophysiological response to new words. Twenty-five young healthy adults underwent 4 days of training to learn the names of 80 novel objects. Half of the items were learnt under a ‘semantic’ condition, whereby the name consisted of a legal non-word and two adjectives denoting semantic attributes. The remaining items were learnt under a ‘name’ condition, whereby the name consisted of a legal non-word and two proper names. Participants demonstrated superior recognition of names in the semantic condition compared to the name condition during training sessions 1-3. On the 5th day, following training, ERPs were recorded whilst participants performed a picture-word judgement task including familiar items. Analysis of the results revealed an N400 for incongruent items in the semantic condition, whilst no ERP component was observed for the name condition. These findings suggest that items learnt with semantic information form stronger associations than those trained without semantics. Keywords Event-related potentials; learning; N400; semantics
2
1. Introduction Humans display an incredible capacity for learning and retaining vast numbers of words (Gaskell & Ellis, 2009). Not surprisingly, much research has been conducted on exploring the nature of new word learning in healthy adults, including research that examines whether the provision of semantic information impacts learning. A number of studies have investigated the learning of new names for novel objects, utilising pictures of ancient Finnish farm tools. The impact of semantic information on learning in such studies has been assessed by pairing stimuli with a description of the novel object’s function during learning. In a review of studies using this paradigm, Laine and Salmelin (2010) concluded that the retrieval of newly learned names is subserved predominantly by cortical regions in the left hemisphere, but that the nature of the initial training/learning task may impact on which left cortical regions are involved. Specifically, left cortical activity may vary according to whether participants are asked to learn only the object names, leading to subsequent word retrieval based on visual-phonological associations, or whether participants are asked to learn both the names and definitions, which may subsequently induce word retrieval via semantics. Of particular interest, studies by Cornelissen et al. (2004) and Whiting, Chenery, Chalk, Darnell and Copland (2007), did not find improved learning with the addition of semantic information. Cornelissen et al. (2004) suggested that the type of semantic information provided, which in their study was short dictionary style definitions for each object based on ancient Finnish farming culture, was neither enriching nor relevant enough to participants to facilitate superior recall. In contrast, Whiting et al. (2007) suggested that too much semantic information may be counterproductive in enhancing new word recall under limited time constraints (such that less time is spent learning the word form when additional semantic information is also presented). In order to address these issues, the current study restricts the semantic information to be
3
learnt to just two adjectives, and uses semantic attributes that are assumed to be more engaging so as to adequately facilitate the formation of a strong mental representation. The findings of Whiting et al. (2007) and Cornelissen et al. (2004) also suggest that in order to effectively compare new word learning for items trained with versus without semantic information, these conditions both need to involve the learning of comparable amounts of information. James and Gauthier (2004) devised a learning paradigm addressing this issue in a functional magnetic resonance imaging (fMRI) study associating semantic features with novel objects. In their semantic condition, the novel object was associated with a proper name and 3 semantic features. This condition was then contrasted with a control condition whereby the novel object was associated with 3 proper names. This design ensured that objects in both conditions would have names associated with them, but only objects from the semantic condition would have significant associations. Whilst the authors failed to find a learning advantage for novel objects trained with semantic features, the authors did observe increased activation in the left inferior frontal cortex during subsequent processing of these items relative to non-trained novel objects and novel objects that had been associated with proper names. Such findings indicate that training novel objects with semantics can lead to altered patterns of neural activation. However, these findings do not provide an indication of any temporal changes in neurophysiological activity. The current study will employ the measurement of event-related potentials (ERPs), which are better suited for monitoring rapidly occurring cognitive processes such as word recognition. The N400 is a negatively deflecting ERP component that usually peaks approximately 400ms after a stimulus is presented and is often elicited in response to semantic anomalies (Kutas & Federmeier, 2000; Kutas & Hillyard, 1980). Whilst classical N400 paradigms examine incongruous sentence endings (Kutas & Hillyard, 1980) or effects of semantic priming using word pair stimuli (Franklin, Dien, Neely, Huber, & Waterson, 2007; Hill, Ott
4
& Weisbrod, 2005; Smith, Chenery, Angwin & Copland, 2009), the N400 can also be elicited by incongruous picture-word pairs. In an object/written word matching task, Hurley et al. (2009) observed an N400 effect in healthy adults for mismatching object/word pairs. Similarly, Hamm, Johnson and Kirk (2002) observed an N400 for semantically mismatched pairs in a written word/picture task. Such paradigms may also be useful to explore new word learning, with the N400 providing an index of the strength of association between a novel object and a newly learnt word. In an ERP study of new word learning, McLaughlin, Osterhout and Kim (2004) utilised the N400 component from a prime-target lexical decision task to investigate how brain activity reflects the acquisition of a second language. The researchers observed an N400 effect for word-nonword pairs after approximately 14 hours of instruction in the second language. After further instruction, participants also displayed reduced N400 effects for related prime-target pairs relative to unrelated pairs. These findings indicated that adults learning a second language are able to acquire information regarding word form and word meaning at a remarkably fast rate. Other studies have also shown the N400 to be a useful index of new word learning. Perfetti, Wlotko and Hart (2005) trained adults in the meanings of rare unknown words. During a subsequent semantic judgement task on pairs of words, an N400 was observed for unrelated word pairs involving the trained words, but was not observed for those involving untrained words. Mestres-Misse, Rodriguez-Fornells and Munte (2007) used ERPs to investigate context-based learning of new words. Specifically, the researchers embedded novel word forms in the sentence final position of meaningful sentence triplet contexts, which participants read during the training phase. In a subsequent relatedness judgement task on word pairs consisting of a trained novel word (prime) and a real word (target), the researchers observed a reduction in the N400 for targets that were related to the novel word prime
5
relative to unrelated targets. In a similar study, Batterink and Neville (2011) examined contextual learning by embedding novel words within meaningful short story contexts. Analysis of ERPs from a subsequent relatedness judgement task using novel word primes and real word targets again revealed a reduction in the N400 for targets related to the novel word prime. Researchers have also shown the N400 to be sensitive to new word learning after just a single exposure to a novel word within the context of a highly constraining sentence (Borovsky, Elman & Kutas, 2012; Borovsky, Kutas & Elman, 2010). The results from such studies provide neurophysiological evidence for the acquisition of new word meaning. Similar findings have also been observed in a magnetoencephalography (MEG) study of new word learning by Dobel et al. (2009). The researchers paired novel words with pictures of known objects during a training phase. Some words were frequently paired with one particular object (learned set), whilst others were paired with many different objects (unlearned set). Prior to training, an N400m mismatch effect, the MEG counterpart to the N400, was observed for all novel words in a cross-modal priming task of word-picture pairs. After training was completed, however, the N400m was substantially reduced for wordpicture pairs involving the learned novel words, whereas a pronounced N400m effect was still observed for the untrained novel words. Accordingly, there is much evidence to suggest the N400 is a reliable index of new word learning, but it is unclear from such research whether the provision of semantics during training facilitates new word learning more than the provision of other types of information. Providing some potential insight into this issue is a new word learning study by Balass, Nelson and Perfetti (2010). The researchers asked participants to perform a semantic judgement task on related/unrelated word pairs after completion of a learning phase. The researchers found that responses to the semantic judgement task were faster and more accurate for words that had been paired with meaning during the learning phase. Similarly,
6
the researchers observed a larger reduction in the N400 effect for related trials involving words for which participants had learned the meaning. This reduction in the N400 was also more evident for words that were paired with orthography and meaning during training, compared to those that were paired with phonology and meaning. Overall, the findings from the study indicate that the nature of the encoding task, including whether it involves semantics, influences the accuracy and underlying neurophysiological mechanisms of new word learning. In contrast to Balass et al. (2010), the present study will investigate new word learning associated with novel objects. In summary, to address the limitations of previous studies on new word learning, this study will utilise distinct novel objects to examine the learning of new words associated with semantic attributes, control for the amount of information learnt in semantic and nonsemantic conditions, and will directly examine the learning of semantic information and its neural correlates using both behavioural measures and ERPs. Specifically, the learning of novel objects associated with a nonword and two semantic attributes (semantic condition) will be compared to the learning of a nonword and two proper names (name condition). Learning will be assessed via behavioural measures of recall and recognition, and an ERP picture-word judgement task will be used to assess the neurophysiological correlates of learning. It was hypothesised that learning new words associated with semantic information would result in superior learning rates and accuracy, as well as an enhanced N400 effect in the picture-word judgement task, compared to words learnt without semantic information. 2. Methods 2.1 Participants Twenty-five healthy young adults (10 males) with a mean age of 22 years (range 20– 26) participated in the study. Two participants were left handed and 23 participants were right handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971). All participants
7
were either native English speakers or proficiently fluent in English and reported no history of neurological illness, head injury, mental illness and developmental delay or disorder (including learning, speech and language). All participants had normal or corrected-to-normal vision, and received monetary reimbursement for their participation. 2.2 Stimuli In the learning paradigm used in this study, coloured images of 80 space aliens devised for use in word learning studies by Gupta et al. (2004) were each paired with a name. These unique alien drawings were ideal for this experiment as participants had to learn new associations between non-word names and novel objects that they had never seen before and for which there are no pre-existing names. The semantic (SEM) condition used names comprised of a legal non-word name (consisting of letter combinations that adhere to the rules of English; e.g., Zealt) associated with two semantic attributes (two adjectives; e.g., Timid Proud). The name (NAM) condition used names comprised of a legal non-word name (e.g., Proach) associated with two proper names (e.g., Mansey Smeath). A list of 80 non-words was generated from the ARC Nonword Database (4-6 letters long, neighbourhood size of 1-3 and summed frequency of neighbours between 5 and 60) (Rastle, Harrington & Coltheart, 2002) to be used as first names for the aliens. For the SEM condition, a list of 80 adjectives 4-6 letters long that were non-visual in modality was generated from the Medical Research Council (MRC) Psycholinguistic Database (Wilson, 1988), and these items were randomly paired to create 40 semantic attribute pairs. In any instance where a pair was found to contain inconsistent adjectives (e.g., Loud Quiet), these items were re-paired with other adjectives. For the NAM condition, 80 proper names were gathered from a list of names taken from records of the British Census between the years 1538 – 2005. In order to ensure that these names were not common enough to have any semantic meaning for participants, a list of 435 of the less common names were
8
compiled. A final list of 80 names, which three Bachelor of Speech Pathology graduates indicated were all unfamiliar names, was compiled and these names were randomly paired to produce 40 proper name pairs. Two lists of stimuli were subsequently created, each containing 40 alien/name pairs in the SEM condition and 40 alien/name pairs in the NAM condition. Each list consisted of the same stimuli, but with different alien/name pairings such that nonword names assigned to the SEM or NAM condition in list 1 were assigned to the other condition (and a different alien picture) in list 2. The two sets of nonwords, which were used as the alien names in the SEM/NAM conditions for each list, did not differ in length (t(39) = 1.88, p = 0.435), neighbourhood size (t(39) = -1.27, p = 0.485) or summed frequency of neighbours (t(39) = 0.75, p = 0.245). The use of these two different lists was then counterbalanced across participants for each of the experimental tasks.
2.3 Learning Task Participants took part in four 1 hour computerised training sessions that were conducted over 4 consecutive days. During these training sessions, participants completed a learning task followed by criterion testing comprised of a recall task and a recognition task. E-prime 1.1 software (Psychology Software Tools Pittsburgh, PA, USA) was used to present the stimuli for all of these tasks as well as record responses to the recall and recognition tasks. The purpose of criterion testing was to ensure that the participants gradually learnt all the names. By the end of the fourth session, participants were required to have achieved a criterion of at least 90% accuracy on recognition tasks. In the learning task, participants were seated at a computer and presented with the 80 space aliens together with their 3 word written name (40 SEM and 40 NAM) one at a time. Each alien/name was presented 3 times per session, resulting in a total of 240 trials per
9
session. Each trial consisted of a fixation point ‘+’ for 1500ms, followed by a picture of an alien in the centre of the computer screen together with its 3 written names presented underneath the picture for 5000ms. The next trial would then begin automatically after a 500ms blank screen (Figure 1). Short rest breaks were provided after every 20 trials.
During the recall task, the 80 alien pictures were presented one at a time in a random order in the centre of the computer screen. For each picture, participants were instructed to use the computer keyboard to type only the first name (i.e., the nonword name) of that alien and then press ‘enter’ on the keyboard. The name that the participant typed would appear underneath the picture with each keystroke. Each picture stimulus remained on screen until the participant hit the ‘enter’ key, and then the next alien picture was displayed automatically. In the recognition task, the 80 alien pictures were again presented in a random order one at a time on the computer screen, together with a 3 word written name presented underneath the picture. The written name was either the correct 3 names for that picture or the 3 names of another alien learnt in the same session. The participants were required to indicate whether the 3 word name was correct or incorrect using the computer mouse, by pressing the left mouse button with their index finger if the name was correct and pressing the right mouse button with their middle finger if the name was incorrect. After a response had been provided by the participant, the next trial was then displayed automatically.
2.4 Cognitive Testing Immediately prior to EEG testing, participants completed three cognitive assessment tasks to obtain measures of short term memory, reading skill and an IQ estimate. These measures were taken in order to determine whether such aspects of cognition impacted learning success for the different conditions. The assessments used were the Repeatable
10
Battery for the Assessment of Neuropsychological Status (RBANS) Digit Span subtest (Randolf, 1998), Neuropsychological Assessment Battery (NAB) Digits Backwards subtest (Stern & White, 2003) and the National Adults Reading Test (NART; Nelson & Willison, 1991). The results from these tests are summarised in Table 1.
2.5 EEG Procedure On the fifth day, following the last training session, participants completed a computerised recognition task comprised of 200 stimulus items presented whilst continuous EEG data was acquired. The 200 stimulus items were comprised of the following: i) congruous semantic name and image pair (sem-con, n=20) ii) incongruous semantic name and image pair (sem-incon, n=20) iii) congruous proper name and image pair (nam-con, n=20) iv) incongruous proper name and image pair (nam-incon, n=20) v) congruous familiar object and name pair (fam-con, n=40) vi) incongruous familiar object and name pair (fam-incon, n=40) vii) novel name and learnt image pairs (nov, n = 40) For conditions (i) to (iv), the stimuli consisted of the pictures learned during the training sessions, paired with either the correct nonword name learned during training (congruent) or an incorrect nonword name that had been learned with a different picture during training (incongruent). Conditions (v) and (vi) included pictures of familiar (FAM) objects (Snodgrass & Vanderwart, 1980) paired with either the correct word for that picture (congruent) (e.g., hammer - hammer) or an incorrect word (incongruent) (e.g., candle lizard). These FAM conditions were used as a control to compare ERP differences between known objects (FAM) and novel objects (SEM, NAM). The novel name and learnt image
11
pairs (condition (vii)) consisted of alien pictures learnt during the training sessions paired together with a novel nonword name that had not been presented during training. This condition was employed to account for repeated exposure to nonword letter strings, compared to entirely novel nonwords, however due to a large number of participant errors made on this condition during task performance it could not subsequently be included in the analyses. The experiment was divided into four blocks with each block containing an equal number of stimuli randomly selected from each condition. The procedure for each trial was as follows: a fixation cross was presented for 1000ms, followed by a picture for 3000ms, then a blank screen for 500ms followed by a written nonword name. When the nonword name appeared, participants had to decide whether the name matched the picture they had just seen, and respond by clicking a button on the Serial Response Box (Psychology Software Tools, Pittsburgh, PA, USA). Participants were asked to click the left button with their index finger if the nonword name was correct, and to click the right button with their middle finger if the nonword name was incorrect.
2.6 Post Tests In order to assess how well participants had learnt all aspects of the alien names, participants were asked to complete two final computerised recognition tasks immediately after the EEG task. In the first task, each trial consisted of 2 aliens presented side by side on the computer screen with 1 nonword name (i.e., an alien’s first name) written centrally towards the bottom of the screen. Participants used the left and right computer mouse buttons to indicate which of the 2 aliens (i.e., left or right of screen) correctly matched the written nonword name. In the second task, each trial consisted of 2 aliens presented side by side on the computer screen with 2 second names (i.e., either two semantic attributes (SEM condition) or two proper names (NAM condition)) written centrally towards the bottom of the screen. Participants again used the computer mouse buttons to indicate which of the 2 aliens 12
correctly matched the written name. For both post-tests, each trial was preceded by a 1000ms fixation cross and was followed by a 1000ms blank screen.
2.7 Data Analysis A Shapiro-Wilk test was conducted on all data sets to test for normality. An arc sine transformation was applied to the recall and recognition accuracy data to normalise the distribution. Similarly, a log transformation was applied to the recognition RT data to normalise the distribution. The training session data was subsequently analysed with a series of repeated-measures analysis of variance (ANOVA) using condition (SEM, NAM) and session (1-4) as within-subjects factors, and accuracy (recall and recognition tasks) and reaction time (RT) (recognition task only) as the dependent variables. Spearman’s correlation was also used to determine whether performance on the Digit Span, Digits Backwards or the NART was associated with performance in training session 4. The ERP accuracy data were also transformed with an arc sine transformation to normalize the distribution of the data, while the RT data was normalised with a log transformation. The ERP data was then analysed separately for the SEM/NAM conditions and the FAM condition using repeated-measures ANOVAs. Post test behavioural data, using condition (SEM, NAM) as the independent variable and accuracy and RT as the dependent variables, were analysed with paired samples t-tests. For the second post test, data was not collected from 6 participants due to computer data logging errors.
2.8 ERP Recording and Analysis ERP data was obtained using a 128 channel high-density EEG system (Electrical Geodesics, Inc.) with a sampling rate of 500Hz. Electrode impedance was kept below 50k,
13
which is acceptable with the use of high impedance amplifiers (Ferree, Luu, Russell, & Tucker, 2001). Netstation 4.4.2 (Electrical Geodesics, Inc.) was used to process all ERP data offline. ERP data from 4 participants was excluded due to excessive EEG artifacts. For the remaining 21 participants, data was digitally filtered from 0.1-30 Hz and segmented into 1100 ms epochs commencing 100ms prior to the onset of the target name. Trials upon which participants made an incorrect response were excluded from analysis, resulting in the exclusion of 4.49% of the total data set (sem-con 5.95%; sem-incon 7.86%; nam-con 7.86%; nam-incon 10.48%; fam-con 1.07%; fam-incon 0.83%). An ocular artefact reduction procedure (Gratton, Coles & Donchin, 1983) was applied to data containing eye movements and blinks. Any subsequent trials consisting of remaining ocular artefacts (defined as vertical or horizontal electro-oculogram channel differences of greater than 70 μV) or that consisted of more than 20% bad channels (defined as reaching amplitudes greater than 200 μV) were subsequently excluded from analysis. The data was re-referenced to the average of all electrodes and baseline corrected to the 100ms pre-target interval. Twelve electrodes (6 within each hemisphere), which provided a well distributed sample of the recording electrode sites, were selected for analysis. Based upon visual inspection of this data, a 200-500ms time window was selected for analysis of the N400. Repeated-measures ANOVAs were used to analyse the mean amplitude of the data within this time region, using condition (FAM, SEM, NAM), congruency (congruent, incongruent), hemisphere of electrode site (Left, Right) and electrode (6 levels) as the independent variables. The Greenhouse-Geisser correction was utilised when the assumption of sphericity was violated (corrected p value reported).
14
3. Results 3.1 Behavioural Results 3.1.1 Training Accuracy For the recall task, the participants’ accuracy improved over the four training sessions (Figure 2) as evidenced by a main effect of session (F(3,72) = 111.91, p < 0.001), however there was no significant main effect of condition (p = 0.268) nor an interaction between session and condition (p = 0.314). For the recognition task (Figure 2), main effects of session and condition were evident (F(3,72) = 93.04, p < 0.001; and F(1,24) = 27.34, p < 0.001, respectively), indicating that overall accuracy improved over the 4 training sessions and that overall accuracy was higher for the SEM condition relative to the NAM condition. There was also a significant session by condition interaction (F(3,72) = 8.91, p < 0.001), indicating that names in the SEM condition were learnt faster, as demonstrated by t-tests showing significantly better recognition for SEM items in sessions 1-3 (t(24) = -6.82, p < 0.001, t(24) = -3.79, p = 0.001, and t(24) = -3.17, p = 0.004, respectively). By the fourth session, this difference was no longer significant (p = 0.173) as participants were able to accurately recognise most alien-name pairs for both conditions.
3.1.2 Correlations No significant correlations were observed between any of the cognitive tests (Digit Span, Digit Backwards and NART IQ) and recall accuracy for the SEM or NAM conditions in training session 4. Similarly, no significant correlations were evident between cognitive testing and recognition accuracy in training session 4.
15
3.1.3 Training Reaction Time For RT in the recognition task, there was again a significant improvement over the four training sessions (Table 2) as evidenced by a main effect of session (F(3,72) = 58.45, p < 0.001). There was no significant effect for condition (p = 0.989) or condition X session (p = 0.556).
3.1.4 ERP task behavioural performance For accuracy during the ERP task, comparing only the SEM and NAM conditions alone, accuracy was equivalent for all conditions (Table 3) as indicated by the absence of significant effects for condition (p = .216), congruency (p = 0.148) and condition X congruency (p = 0.903). For RT in the EEG task (Table 3), there was no main effect of condition (p = .845) or condition X congruency interaction (p = 0.946), however, a main effect of congruency (F(1,24) = 22.975, p < 0.001) indicated that RTs were faster for congruent items compared to incongruent items. Analysis of the FAM condition data revealed no difference in accuracy between congruent and incongruent items (p = .720), however RTs were faster in the congruent relative to the incongruent condition (F(1,24) = 13.36, p = .001).
3.1.5 Post Tests Analysis of the data from the first post test, where participants were tested on recognition of first names only (i.e., the nonword name), revealed no significant difference in accuracy (p = 0.986) or RT (p = 0.264) between names that had previously been paired with semantic attributes and names that had been paired with proper names (Table 4). In the second post test, where participants were tested on their recognition of the two last names
16
only (either two semantic attributes or two proper names), participants displayed higher accuracy for recognising semantic attributes than proper names (t(18) = -3.72, p = 0.002) and faster RTs for semantic attributes than proper names (t(18) = 5.06, p < 0.001) (Table 4).
3.2 ERP Results Mean amplitude of the N400 (200-500ms) was analysed with repeated measures ANOVA using condition (FAM, SEM, NAM), congruency (congruent, incongruent), hemisphere (Left, Right) and electrode (6 levels) as independent variables. This analysis revealed main effects of condition (F(2,40) = 3.78, p = 0.031), hemisphere (F(1,20) = 78.65, p < .001) and electrode (F(5,100) = 32.04, p < .001), as well as interaction effects of congruency X hemisphere (F(1,20) = 10.71, p = .004), congruency X hemisphere X electrode (F(5,100) = 4.75, p = .001) and condition X congruency X hemisphere (F(2,40) = 4.77, p = .014). In light of the interaction between condition, congruency and hemisphere, the data was subsequently explored further using additional repeated measures ANOVAs, which were conducted separately for each condition. Main effects of hemisphere and electrode are not reported. Analysis of the FAM condition data revealed a significant main effect of congruency (F(1,20) = 6.62, p = .018), and interaction effects of congruency X hemisphere (F(1,20) = 6.98, p = .016) and congruency X hemisphere X electrode (F(5,100) = 4.00, p = .002). Further analysis within each hemisphere with congruency and electrode as independent variables revealed an N400 effect in the right hemisphere as evidenced by a main effect of congruency in the right hemisphere (F(1,20) = 12.46, p = .002), and no main or interaction effects of congruency in the left hemisphere (Figure 3). Analysis of the SEM condition revealed a main effect of congruency (F(1,20) = 4.41, p = .049) and a significant interaction effect of congruency X hemisphere (F(1,20) = 12.80, p = .002). Further analysis within each
17
hemisphere indicated a significant N400 in the right hemisphere, as evidenced by a significant main effect of congruency in the right hemisphere (F(1,20) = 10.91, p = .004), but no significant main or interaction effects of congruency in the left hemisphere (Figure 3). Analysis of the NAM condition revealed no significant main or interaction effects involving congruency (Figure 3).
To determine whether a difference in the latency of the N400 was evident between the FAM and SEM conditions, difference waves were calculated (incongruent minus congruent) for each condition (figure 4) and the peak latency of the N400 was calculated within the 200500ms time window. This data was analysed using a repeated measures ANOVA, with condition (FAM, SEM), hemisphere (Right, Left) and electrode (6 levels) as independent variables. No significant effects were evident, indicating a similar latency of the N400 for both conditions.
In order to further explore the temporal and topographical distribution of the N400 effects evident for the SEM and FAM conditions, and to accommodate for the possibility of later effects beyond the 200-500ms time window, the mean amplitude for the SEM and FAM conditions was calculated within 100ms time segments from 200-700ms. This data was calculated across 30 different scalp regions, encompassing 126 electrodes (the nasion and inferior eye channels were excluded). The data within each time segment was analysed using repeated measures ANOVAs, with condition (FAM, SEM), congruency (congruent, incongruent) and region (30 levels) as independent variables. The analyses within each time segment revealed no significant interaction between condition and congruency, nor any significant 3 way interaction of condition x congruency x region, illustrating a similar topographical distribution of effects for the SEM and FAM conditions over time (Figure 5).
18
4. Discussion The aim of this study was to investigate whether the addition of semantic attributes would enhance new word learning for novel objects and whether it would influence neurophysiological activity. It was hypothesised that learning new names for novel objects that are associated with semantic attributes would be superior to learning new names presented without semantics. It was also hypothesised that N400 effects would be more prominent for names learnt with semantic information.
4.1 Behavioural results Participants demonstrated improved recognition of names learnt in the SEM condition compared to the NAM condition over training sessions 1-3, however this advantage was no longer evident at session 4. This pattern of results suggests that semantics may provide a learning advantage in the earlier stages of new word acquisition. It should be noted that whilst greater recognition accuracy was observed for the semantic names in the initial training sessions, the recall task did not show this same advantage. This could partially be attributed to differences in the level of task difficulty. Most participants achieved the criterion of 90% accuracy for recognition by the end of training. However, for the recall task in our study, only 9 out of the 25 participants reached criterion by the end of training. Given suggestions that recall is typically more difficult than recognition (Haist, Shimamura & Squire, 1992), it is possible that in the present experiment semantic information assisted in the processes involved in recognition, but not in the retrieval processes that are required for recall. Also worthy of note, however, is that participants only had to recall the nonword name for the recall task, but were presented with all three names during the recognition task. Hence, the results suggest that participants may have, to some
19
extent, learned a direct association between the picture and the semantic adjectives, which subsequently provided an advantage for the SEM condition in the recognition task, but not for the recall task. Turning to the results of the post-tests, analysis of the first post-test data revealed that participants were able to match the nonword names to the alien pictures with similar accuracy and RT for items from the SEM and NAM conditions. This finding is expected given that participants also had similar recognition accuracy for both conditions after the 4th training session. However in the second post-test, which assessed how well participants learnt the additional alien names in each condition (i.e., semantic attributes or proper names), the semantic attributes were recognised with greater accuracy and faster RTs than proper names. These results confirm that participants learnt the semantic information more effectively than the proper names.
4.2 ERP Results To investigate the neurophysiological effects of semantic information on new word learning, mean amplitudes in the 200-500ms region were examined in order to target the N400 component, an index of lexical-semantic processing. Similar to previous research involving paired picture-word or word-picture stimuli (Hamm et al., 2001; Hurley et al., 2009), an N400 was elicited to incongruent stimuli in the right hemisphere for the FAM condition. This N400 effect is indicative of the participant’s sensitivity to the semantically incongruent picture-word pairs in the FAM condition, providing a useful index of semantic processing. The absence of an N400 in the left hemisphere was not surprising, given that other ERP studies have observed a more prominent N400 in the right hemisphere (Hurley et al., 2009; Kutas & Federmeier, 2011).
20
A significant N400 effect was also observed in the 200-500ms region for incongruent items from the SEM condition, which was again restricted to the right hemisphere. This result adds to previous research that has shown the N400 to be a reliable index of word learning (Balass et al., 2010; Batterink and Neville, 2011; McLaughlin et al., 2004; Mestres-Misse et al., 2007; Perfetti et al., 2005). The similar topographical distribution of the N400 over time for both the FAM and SEM conditions also suggests that by the end of training, correctly recognised items from the SEM condition are processed similarly to already known/familiar items. Interestingly, previous studies of new word learning have differed with respect to the distribution of the N400 effect obtained in semantic judgement tasks after learning, with some research finding an N400 over frontal and parietal sites for both newly learned and familiar words (Frishkoff, Perfetti & Collins-Thompson, 2010), and others finding that the N400 for newly learned words has a more frontal distribution relative to known familiar words (Mestres-Misse et al., 2007). Further research is clearly needed to establish which stimulus factors impact the distribution of N400 effects for new versus familiar words. Whilst an N400 effect was observed for the SEM condition in the right hemisphere, no N400 effect was evident in either hemisphere for the NAM condition. Importantly, this finding is evident despite the fact that only nonword names were presented during the ERP task. Thus, the presence of an N400 for the SEM but not the NAM condition cannot be attributed to a learned direct association between the semantic adjectives and the picture. Instead, this finding suggests that nonword names from the SEM condition were more deeply encoded, resulting in a stronger lexical-semantic representation as indexed by the N400 effect for the SEM condition but not the NAM condition. The results demonstrate that although recognition accuracy was similar for both NAM and SEM conditions by the 4th training session, a processing advantage for the nonwords was still evident for the SEM condition at a neurophysiological level.
21
Worthy of note is that other studies have found increases in the N400 during speech segmentation of nonsense words, potentially reflecting processes relating to the learning and segmentation of the nonsense word stimuli (Cunillera, Toro, Sebastian-Galles & RodriguezFornells, 2006; Cunillera et al., 2009; Sanders, Newport & Neville, 2002). Thus, the N400 appears to have the capacity to index differential components of word learning, depending on the nature of the paradigm being used. Clearly, ongoing research is still required in order to elucidate our current understanding of the underlying cognitive processes that contribute to N400 learning effects. Overall, these novel findings provide further evidence that semantic information facilitates new word learning and add to the existing literature that suggests the inclusion of semantic information during word learning modifies neural activity (Balass et al., 2010; James & Gauthier, 2004). There are some limitations that could be addressed in future related studies. The present study only measured ERPs on the 5th day, after training had been completed. Future studies could implement ERP measurements on each day of training in order to more effectively track changes in learning and how such changes may differ for the NAM and SEM conditions. The existing ERP design was also limited by a small number of stimuli for the NAM and SEM conditions, and the ERP data may have been impacted by the motor responses that were required for the semantic judgement task. It would also be of interest to compare the effect of different forms of semantic information on new word learning, such as auditory semantics (e.g., honks, squeaks) relative to visual semantics (e.g., tall, fat). Previous fMRI and ERP studies have examined the learning of novel words from contextual information (Batterink et al., 2011; Mestres-Misse et al., 2007; Mestres-Misse, Camara, Rodriguez-Fornells, Rotte, & Munte, 2008), and it may be useful to apply similar paradigms to the learning of novel object names. Given that some ERP studies have found that reading skill can influence learning (Balass et al., 2010; Perfetti et al.,
22
2005), reading comprehension skill should also be factored into any ongoing research on new word learning for novel objects. Lastly, an MEG word learning study by Hulten, Laaksonen, Vihla, Laine and Salmelin (2010) showed that changes in neural activation from the end of the learning phase to one week after learning predicted the retention of the new vocabulary at 10 months post training. Future ERP investigations of new word learning should aim to identify any neural markers that predict learning success.
4.3 Conclusions This study explored the use of semantics in learning new names for novel objects and examined the neurophysiological impact of this approach. Recognition of items from the SEM condition was superior to that of the NAM condition during the first 3 training sessions. An N400 effect was also observed for incongruent items in the SEM condition, but not the NAM condition. These findings suggest that semantics provides a learning advantage in new word learning, which could be attributed to the development of more elaborate lexicalsemantic representations.
23
5. References Balass, M., Nelson, J.R., & Perfetti, C.A. (2010). Word learning: An ERP investigation of word experience effects on recognition and word processing. Contemporary Educational Psychology, 35, 126-140. Batterink, L., & Neville, H. (2011). Implicit and explicit mechanisms of word learning in a narrative context: an event-related potential study. Journal of Cognitive Neuroscience, 23(11), 3181-3196. Borovsky, A., Elman, J.L., & Kutas, M. (2012). Once is enough: N400 indexes semantic integration of novel word meanings from a single exposure in context. Language, Learning and Development, 8, 278-302. Borovsky, A., Kutas, M., & Elman, J. (2010). Learning to use words: Event-related potentials index single-shot contextual word learning. Cognition, 116, 289-296. Cornelissen, K., Laine, M., Renvall, K., Saarinen, T., Martin, N., & Salmelin, R. (2004). Learning new names for new objects: Cortical effects as measured by magnetoencephalography. Brain and Language, 89(3), 617-622. Cunillera, T., Camara, E., Toro, J.M., Marco-Pallares, J., Sebastian-Galles, N., Ortiz, H., Pujol, J., & Rodriguez-Fornells, A. (2009). Time course and functional neuroanatomy of speech segmentation in adults. NeuroImage, 48, 541-553. Cunillera, T., Toro, J.M., Sebastian-Galles, N., & Rodriguez-Fornells, A. (2006). The effects of stress and statistical cues on continuous speech segmentation: An event-related brain potential study. Brain Research, 1123, 168-178. Dobel, C., Junghofer, M., Breitenstein, C., Klauke, B., Knecht, S., Pantev, C., et al. (2009). New names for known things: On the association of novel word forms with existing semantic information. Journal of Cognitive Neuroscience, 22(6), 1251-1261.
24
Ferree, T.C., Luu, P., Russell, G.S., & Tucker, D.M. (2001). Scalp electrode impedance, infection risk, and EEG data quality. Clinical Neurophysiology, 112, 536-544. Franklin, M.S., Dien, J., Neely, J.H., Huber, E., & Waterson, L.D. (2007). Semantic priming modulates the N400, N300, and N400RP. Clinical Neurophysiology, 118(5), 10531068. Frishkoff, G.A., Perfetti, C.A., & Collins-Thompson, K. (2010). Lexical quality in the brain: ERP evidence for robust word learning from context. Developmental Neuropsychology, 35(4), 376-403. Gaskell, M.G., & Ellis, A.W. (2009). Word learning and lexical development across the lifespan. Philosophical Transactions of the Royal Society B-Biological Sciences, 364(1536), 3607-3615. Gratton, G., Coles, M.G.H., & Donchin, E. (1983). A new method for off-line removal of ocular artifacts. Electroencephalography and Clinical Neurophysiology, 55, 468-484. Gupta, P., Lipinski, J., Abbs, B., Lin, P.H., Aktunc, E., Ludden, D., et al. (2004). Space aliens and nonwords: Stimuli for investigating the learning of novel word-meaning pairs. Behavior Research Methods Instruments & Computers, 36(4), 599-603. Haist, F., Shimamura, A.P., & Squire, L.R. (1992). On the relationship between recall and recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(4), 691-702. Hamm, J.P., Johnson, B.W., & Kirk, I.J. (2002). Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts. Clinical Neurophysiology, 113(8), 1339-1350. Hill, H., Ott, F., & Weisbrod, M. (2005). SOA-dependent N400 and P300 semantic priming effects using pseudoword primes and a delayed lexical decision. International Journal of Psychophysiology, 56, 209-221.
25
Hulten, A., Laaksonen, H., Vihla, M., Laine, M., & Salmelin, R. (2010). Modulation of brain activity after learning predicts long-term memory for words. The Journal of Neuroscience, 30(45), 15160-15164. Hurley, R.S., Paller, K.A., Wieneke, C.A., Weintraub, S., Thompson, C.K., Federmeier, K.D., et al. (2009). Electrophysiology of object naming in primary progressive aphasia. Journal of Neuroscience, 29(50), 15762-15769. James, T.W., & Gauthier, I. (2004). Brain areas engaged during visual judgments by involuntary access to novel semantic information. Vision Research, 44(5), 429-439. Kutas, M., & Federmeier, K.D. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences, 4(12), 463-470. Kutas, M., & Federmeier, K.D. (2011). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62, 621–647. Kutas, M., & Hillyard, S.A. (1980). Reading senseless sentences: brain potentials reflect semantic incongruity. Science, 207(4427), 203-205. Laine, M., & Salmelin, R. (2010). Neurocognition of new word learning in the native tongue: lessons from the ancient farming equipment paradigm. Language Learning, 60, 25-44. McLaughlin, J., Osterhout, L., & Kim, A. (2004). Neural correlates of second-language word learning: minimal instruction produces rapid change. Nature Neuroscience, 7(7), 703704. Mestres-Misse, A., Camara, E., Rodriguez-Fornells, A., Rotte, M., & Munte, T.F. (2008). Functional neuroanatomy of meaning acquisition from context. Journal of Cognitive Neuroscience, 20(12), 2153-2166. Mestres-Misse, A., Rodriguez-Fornells, A., & Munte, T.F. (2007). Watching the brain during meaning acquisition. Cerebral Cortex, 17, 1858-1866.
26
Nelson, H. E., & Willison, J. R. (1991). The Revised National Adult Reading Test. Windsor, UK: NFER Nelson. Oldfield, R.C. (1971). Assessment and analysis of handedness - Edinburgh Inventory. Neuropsychologia, 9(1), 97-113. Perfetti, C.A., Wlotko, E.W., & Hart, L.A. (2005). Word learning and individual differences in word learning reflected in event-related potentials. Journal of Experimental Psychology: Learning, Memory and Cognition, 31(6), 1281-1292. Randolph, C. (1998). Repeatable Battery for the Assessment of Neuropsychological Status. San Antonio: Harcourt Assessment. Rastle, K., Harrington, J., & Coltheart, M. (2002). 358,534 nonwords: The ARC Nonword Database. Quarterly Journal of Experimental Psychology, 55A, 1339-1362. Sanders, L.D., Newport, E.L., & Neville, H.J. (2002). Segmenting nonsense: an event related potential index of perceived onsets in continuous speech. Nature Neuroscience, 5, 700-703. Smith, E.R., Chenery, H.J., Angwin, A.J., & Copland, D.A. (2009). Hemispheric contributions to semantic activation: A divided visual field and event-related potential investigation of time course. Brain Research, 1284, 125-144. Snodgrass, J.G., & Vanderwart, M. (1980). Standardized set of 260 pictures - Norms for name agreement, image agreement, familiarity and visual complexity. Journal of Experimental Psychology-Human Learning and Memory, 6(2), 174-215. Stern, R.A., & White, T. (2003). The Neuropsychological Assessment Battery. Lutz, FL: Psychological Assessment Resources, Inc. Whiting, E., Chenery, H., Chalk, J., Darnell, R., & Copland, D. (2007). Dexamphetamine enhances explicit new word learning for novel objects. International Journal of Neuropsychopharmacology, 10(6), 805-816.
27
Wilson, M. (1988). MRC psycholinguistic database: machine-usable dictionary, version 2.00. Behavior Research Methods, Instruments, and Computers, 20, 6–10.
28
Figure 1: An illustration of the procedure for the learning task. Figure 2: Mean proportion of correct responses for the semantic (SEM) and name (NAM) conditions at each training session for the recall (A) and recognition (B) tasks.Error bars represent standard error. Figure 3: Grand-averaged ERPs for congruent and incongruent stimuli in each condition. Figure 4: Difference waves (incongruent minus congruent) for the SEM and FAM conditions. Figure 5: Topographic maps (incongruent minus congruent) for the SEM and FAM conditions from 200-700ms. Maps oriented with nose facing up.
29
1500 ms
Figure 1
+
5000 ms
500 ms
Zealt Timid Proud
Fixation Cross
1500 ms
Stimulus
+
5000 ms
500 ms
Stimulus
Fixation Cross (beginning of next trial)
Proach Mansey Smeath
Blank screen
Blank screen
Figure 2
1
Proportion of Correct Responses
A.
0.8
0.6 SEM NAM
0.4
0.2
0 1
2
3
4
Training session
B. Proportion of Correct Responses
1
0.8
0.6 SEM NAM
0.4
0.2
0 1
2
3
Training session
4
Figure 3
NAM
SEM
FAM
+6 μV
Congruent -100
-6 μV
100
300
500
700
900
Incongruent
Figure 4
+6 μV
FAM -100
-6 μV
100
300
500
700
900
SEM
Figure 5
200-300ms
300-400ms
400-500ms
500-600ms
600-700ms
FAM
SEM
Table 1. Participant demographics and cognitive test scores (n=25) ________________________________________________________________________ Measure Mean S.D ________________________________________________________________________ Age (years) 22.04 1.62 Years of education
15.44
0.85
RBANs Digit Span (max. score: 16)
12.88
2.89
NAB Digits Backwards (max. score: 88)
40.72
21.00
NART (raw score, max. score: 50)
32.32
5.56
NART (estimated IQ equivalent) 116.72 5.78 _________________________________________________________________________ RBANS, Repeatable Battery for the Assessment of Neuropsychological Status, Digit Span subtest (Randolf, 1998); NAB, Neuropsychological Assessment Battery, Digits Backwards subtest (Stern & White, 2003); NART, National Adults Reading Test (Nelson & Willison, 1991).
30
Table 2: Mean RT on the recognition task at each training session for each condition (correct responses only). ________________________________________________ Training session SEM NAM ________________________________________________ 1 3208 (929) 3081 (1022) 2
2614 (1214)
2676 (1508)
3
1954 (732)
2000 (720)
4 1702 (530) 1751 (599) ________________________________________________ Standard deviations in brackets.
31
Table 3: Mean proportion of correct responses and mean RT for each condition during the ERP task. _____________________________________________________ Condition Accuracy RT _____________________________________________________ Sem-con
0.95 (0.08)
850 (378)
Sem-incon
0.93 (0.10)
1024 (489)
Nam-con
0.93 (0.09)
828 (311)
Nam-incon
0.91 (0.12)
997 (428)
Fam-con
0.99 (0.02)
719 (172)
Fam-incon 0.99 (0.01) 792 (215) _____________________________________________________ Standard deviations in brackets.
32
Table 4: Mean proportion accuracy and RT for each condition at each post-test. _____________________________________________________________________ Post-test 1 (n=25) Post-test 2 (n=19) Condition Acc RT Acc RT ______________________________________________________________________ SEM 0.95 (0.08) 1659 (469) 0.88 (0.12) 2578 (803) NAM 0.96 (0.06) 1723 (531) 0.79 (0.14) 3147 (873) ______________________________________________________________________ Standard deviations in brackets
33
Highlights x x x x
Theimpactofsemanticsonnewwordlearningfornovelobjectswasassessed. Aftertraining,ERPswererecordedduringapicturewordsemanticjudgementtask AnN400wasobservedforstimulithatwerelearnedwithsemanticinformation AnN400wasnotobservedforstimulilearnedwithoutsemanticinformation
34