- Email: [email protected]
Event-related potentials and the phonological matching of picture names
BRAIN
AND
38, 424-437 (IWO)
LANGUAGE
Event-Related Potentials and the Phonological Matching of Picture Names SARAH E. BARRETT Weikome
Brain
Research
Group,
AND MICHAEL
University
D. RUGG
of St. Andrews,
Fife,
United
Kingdom
Event-related potentials (ERPs) were recorded from one midline and three pairs of lateral electrodes while subjects determined whether a pair of sequentially presented pictures had rhyming or nonrhyming names. During the 1.56-set interval between the two pictures, the slow ERP wave recorded over the left hemisphere was more negative-going than that over the right, especially at frontal electrodes. The ERPs evoked by the second picture differed as a function of whether its name rhymed with its predecessor. This difference, taking the form of increased negativity in ERPs to nonrhyming items, had an earlier onset and a greater magnitude at right than at left hemisphere electrodes. This pattern of ERP asymmetries is qualitatively similar to that found when words are rhymematched. It is therefore concluded that such asymmetries do not depend on the employment of orthographic material and may reflect some aspect(s) of the phonological processing of visually presented material. 0 IWO Academic press. h.
INTRODUCTION
Neuropsychological evidence suggests that the phonological processing of visually presented material involves processes which are strongly lateralized to the left hemisphere in the majority of individuals (Coltheart, 1980, 1983; Patterson & Besner, 1984; Zaidel & Peters, 1981). A task such as rhyme-judgement, which requires subjects to determine whether two words share the same terminal phonology, would thus be expected to rely heavily on the left hemisphere. This task has therefore been used in several previous studies investigating the relationship of lateralized cognitive processes to lateral asymmetries in scalp-recorded event-related potentials (ERPs). The basic experimental paradigm employed in the studies reviewed below involves the sequential presentation of two items, separated by a This work was supported the U.K. We thank Karalyn reprint requests to M. D. chology, University of St.
by the Wellcome Trust and the Medical Research Council of Patterson for providing some of the stimulus materials. Address Rugg, Wellcome Brain Research Group, Department of PsyAndrews, Fife KY16 9JU, United Kingdom. 424
0093-934x/90 $3.00 Copyright All rights
0 1990 by Academic Press, Inc. of reproduction in any form reserved
ERPS AND PICTURE-MATCHING
425
short period of time (typically 1 or 1.56 set), with the requirement to determine whether the second item rhymes with the first. A series of experiments (Barrett & Rugg, 1989a; Rugg, 1984a,b, 1985a; Rugg & Barrett, 1987) has demonstrated that ERPs recorded during this task show a characteristic pattern of asymmetries, involving two different regions of the ERP waveform. The earlier of these asymmetries, which occurs during the interval between the two items, appears consistent with the neuropsychological evidence concerning the lateralization of processing during rhyme-matching. The direction of the later asymmetry, occurring in the ERPs evoked by the second item, seems at first sight to be at variance with this evidence. The first of these ERP asymmetries involves the slow negative wave (the “contingent negative variation” or CNV) that develops during the interval between the two words. This wave is more negative-going from left than from right hemisphere electrodes, especially at frontal and temporal sites. It has been suggested that this asymmetry reflects the preeminence of left hemisphere mechanisms in the encoding and short-term storage of the phonological information necessary to perform the rhymematching task (Rugg, 1984a). This interpretation is strengthened by the finding that the CNV that develops between sequentially presented faces in face-matching tasks shows an asymmetry in the opposite direction (Barrett, Rugg, & Perrett, 1988; Barrett & Rugg, 1989b). Because facematching would be expected to engage processes lateralized to the right hemisphere (Ellis, 1983), this gives support to the idea that CNV asymmetries signify the engagement of lateralized material- or processingspecific mechanisms. The other ERP asymmetry occuring during rhyme-matching follows the presentation of the second word. The ERP waveforms evoked by this word diverge as a consequence of whether or not the word rhymes with the initial item. This rhyme/nonrhyme difference takes the form of a more negative-going waveform in ERPs from nonrhyming words, and appears to involve the modulation of a negative-going wave peaking around 450 msec poststimulus (N450). This difference in the ERPs evoked by rhyming and nonrhyming words is larger from right than from left hemisphere electrodes, especially over frontal and temporal regions of the scalp. The negative wave modulated by the rhyme/nonrhyme manipulation, “N450,” may be allied to late negative ERP waves, such as the “N400,“’ I The terminology applied to late negative waves sensitive to linguistic manipulations is confusing. In the present paper, “N400” will be used to designate negativities modulated by semantic manipulations, and “N450” will refer to the negative wave modulated by phonological variables. These terms are descriptive only and should not be taken to imply that the waves to which they refer have different neural sources.
426
BARRETT
AND
RUGG
which have been shown to be sensitive to manipulations of semantic priming. Both in sentence (e.g., Kutas & Hillyard, 1980, 1984) and single word (Bentin, McCarthy, & Wood, 1985; Rugg, 1985b) priming experiments, N400 is larger in ERPs evoked by unprimed than by primed words, and it has been suggested that the amplitude of N400 is inversely proportional to the degree that a word has been semantically primed by its preceding context (Kutas & Hillyard, 1984). Unlike the modulation of N450 in rhyme-matching, however, the N400 effects observed in studies manipulating semantic relationships either do not differ significantly between the two hemispheres or are only slightly lateralized to right hemisphere electrodes (Barrett & Rugg, 1987; Kutas, Van Petten, & Besson, 1988). Extrapolating from the work investigating the influence of semantic priming on N400, Rugg (1984b) suggested that the modulation of N450 in the rhyme-judgement task may reflect the differing extents to which rhyming and nonrhyming words have been primed by the preceding item. Rugg and Barrett (1987) further suggested that N450 modulation in rhymematching is controlled not by the phonological relationship between the two members of a word pair but by the orthographic relationship. The “orthographic priming hypothesis” was inspired by the fact that while the number of rhyming word pairs in English is very large, the number of terminal orthographic segments that the members of such pairs can possess is relatively small. For example, almost all words that rhyme with PALE will end in -ALE, -AIL, or -EIL. Rugg and Barrett suggested that, in the rhyme-judgement task, the rhyme-sensitive N450 is modulated by whether the second word contains an orthographic segment primed by its predecessor. As virtually all rhyming words contain such a segment, but most nonrhyming words do not, N450 would be larger for nonrhyming than rhyming words. In their initial experiments, Rugg and Barrett (1987) obtained data consistent with the orthographic priming hypothesis. Nonrhyming words which shared the same terminal orthographic segment (e.g., COUGHBOUGH) did not evoke an asymmetric N450 wave, indicating that conflicting phonologies are not always sufficient to elicit the “rhyme-sensitive” N450. However, the results of the final experiment were not consistent with the orthographic priming hypothesis. This experiment employed nonrhyming word pairs in which the second item shared its terminal orthographic segment with another word which did rhyme with the first item (e.g., STAKE-FREAK [c.f. BREAK]). According to the orthographic priming hypothesis, such nonrhyming words contain a primed orthographic segment, and thus the ERPs which they evoke should not contain an asymmetrically enhanced N450 wave. This prediction was not borne out; “STAKE-FREAK” pairs gave rise to the same pattern of ERP effects in the N450 latency range as controls. Rugg
ERPS AND PICTURE-MATCHING
427
and Barrett therefore suggested that the mechanism responsible for modulating N450 is sensitive to the general level of “congruity” between word pairs, such that either a highly similar phonology or a highly similar orthography is sufficient to attenuate the N450 recorded from right hemisphere electrodes. Rugg and Barrett’s (1987) study casts doubt on whether the modulation of ERPs by rhyming and nonrhyming word-pairs is determined by the relationship between the words’ orthographic properties. The present experiment pursues this issue by employing pictorial stimuli in a rhymejudgement task. As such stimuli contain no explicit orthographic information, any effects on ERPs of the rhyme/nonrhyme manipulation can be attributed with some confidence to the phonological attributes of the items’ names. Therefore if these effects are similar to those observed when words are rhyme-matched, this would provide strong evidence in favor of the idea that asymmetrical rhyme/nonrhyme differences in ERPs reflect the outcome of phonological rather than orthographic processing. The use of pictorial stimuli also allows further investigation of the CNV asymmetries observed in previous studies of rhyme-matching. If these asymmetries do indeed reflect the lateralization of phonological processing, they should be as apparent when phonological information is derived from pictures as when it is derived from words. METHOD Subjects. Ten female and six male right-handed (as defined by writing hand) young adults participated in this experiment. Stimuli. One hundred and sixty black and white slides, each portraying a line drawing of a common object, were used to form a set of 80 “rhyming” stimulus pairs. A rhyming pair consisted of two pictures with rhyming names (e.g., spoon-moon, sock-clock).’ Two stimulus lists were constructed using these items. List I was constructed by randomly designating 40 of the pool of 80 pairs as rhyming pairs, and then rearranging the remainder so that these were nonrhyming. These 80 pairs of pictures, 40 rhyming and 40 nonrhyming. were then randomly ordered to form List I. List 2 was constructed by employing as rhyming stimuli those pairs of pictures that had been rearranged to provide the nonrhyming stimuli for List I, and rearranging the members of the rhyming pairs of List I so as to provide the 40 nonrhyming simuli. These 80 items were then randomly ordered to form List 2. Thus no picture was repeated within a list, but across the lists each appeared once in a nonrhyming and once in a rhyming pair. Half of the subjects were shown List I, and the remainder saw List 2, thereby eliminating any possible confound between items and the rhyme/nonrhyme manipulation. A further 5 rhyming and 5 nonrhyming picture pairs, none of which was employed in the experimental lists, were used to form a practice list. The stimulus slides were displayed on a TV monitor, and subtended a visual angle of approximately 2” at the viewing distance of 90 cm. Each slide was presented for 300 msec, with an interval of 1560 msec between the onset of the first (Sl) and second (S2) stimulus of a pair. The interval between consecutive Sl onsets was 8.5 sec. A small central fixation dot was continuously present on the monitor. A row of crosses, easily visible within ’ Copies of the stimuli are available from the authors upon request.
428
BARRETT
AND RUGG
subjects’ peripheral vision, was presented on a separate TV monitor immediately below the one on which the pictures were presented. The appearance of these crosses indicated when eye movements were permissible. They were displayed from 900 msec after the onset of S2 until 600 msec before the onset of SI on the subsequent trial. Procedure. Following electrode application, subjects were seated in front of the TV monitors, with the index fingers of each hand resting on laterally positioned microswitch push-buttons. They were instructed to respond quickly and accurately with one hand when the names of the two stimuli in a pair rhymed, and with the other hand when the stimuli did not rhyme. The hands used by a subject for rhyme and nonrhyme reponses remained the same throughout the experimental session, with the hand used for each response being counterbalanced across subjects. Subjects were also instructed to keep eye and body movements to a minimum throughout each trial, to maintain fixation on the dot in the center of the presentation window, and to blink only when the crosses were present on the lower monitor. A short break was given after 40 trials. ERP recording. EEG was recorded, with reference to linked mastoids, from a parietal midline electrode (the Pz site of the International ten twenty system; Jasper, 1958), and from lateral frontal, temporal, and parietal locations. Frontal electrodes were 75% of the distance from Fz to F7 on the left and F8 on the right, temporal electrodes were 75% of the distance from Cz to T3 on the left and T4 on the right, and parietal electrodes were 75% of the distance from Pz to TS on the left and T6 on the right. Cz served as ground and, in subjects 1 to 10, EOG was recorded between electrodes placed on the outer canthus of the left eye, and above the right eyebrow. As the experiment progressed, it became clear that the lateral ERP asymmetry emerging during the SI-S2 interval was typically frontally distributed, and that it would be necessary to take additional steps to eliminate the possibility that this asymmetry resulted from systematic horizontal eye-movement. Therefore, in subjects 11 to 14, an EOG montage more sensitive to horizontal movements was employed, by placing the right-sided EOG electrode more laterally and increasing the gain of EOG channel from 5K to 20K. In subjects I5 and I6 additional sensitivity to horizontal eye-movements was achieved by omitting the Pz channel and recording vertical and horizontal EOG on separate channels. In these two subjects, vertical EOG was recorded with electrodes above and below the left eye and horizontal EOG, with electrodes on the outer canthus of each eye. Interelectrode impedances were less than 5 kohm, and all channels were amplified with a bandwidth of 0.03-30 Hz (3dB points). The sampling rate was 1 point per IO msec, beginning 100 msec before the onset of Sl and continuing for 2460 msec thereafter. Separate ERPs were formed for each subject from the trials of each experimental condition. Only trials on which a correct behavioral response was made, and no eye movement artifact detected, were used to form the averages.
RESULTS
The ERPs were quantified by measuring the mean amplitude of selected latency regions of the waveforms with respect to a lOO-msec prestimulus baseline. These data were then subjected to repeated-measures ANOVA, employing the Greenhouse-Geisser correction for inhomogeneity of covariance. Post hoc tests were performed with the Tukey HSD test for pairwise comparisons between means and Scheffe’s procedure for decomposing interactions. The significance level for all ANOVAs and post hoc analyses was set at p < 0.05. Post-S1 analysis. Grand average post-S1 waveforms from lateral electrode sites are shown in Fig. 1, collapsed across rhyming and nonrhyming
ERPS AND PICTURE-MATCHING
I
’
Sl
429
I
1560msec
+ I 5UV
’
s2
-
L. HEM.
----
R. HEM.
FIG. 1. Grand average waveforms from lateral electrodes, aligned on a pre-Sl baseline and averaged over rhyming and nonrhyming trials. F, T, and P indicate frontal, temporal, and parietal sites, respectively.
conditions. After a series of transient deflections over approximately the first 400 msec following S 1, the waveforms are characterized by a prominent, frontally distributed negative-going deflection, followed by a slow wave lasting until S2. At frontal sites, this wave is positive-going, and substantially more negative over the left hemisphere from around 900 msec until the end of the post-S2 epoch. At the parietal sites there appears to be a tendency for the earlier region of the slow wave to show a reversal of this asymmetry. The waveforms during the S 1-S2 interval were quantified by measuring the mean amplitude of the regions 600-900 msec and 900-1560 msec post-S 1, to capture the early and late asymmetries apparent in Fig. I. These data are shown in Table 1. They were subjected to repeatedmeasures ANOVA with factors of hemisphere (left/right) and site (frontal/temporal/parietal). Analysis of the 600- to 900-msec region revealed no significant effects. ANOVA of the 900- to 1560-msec data gave rise to significant effects of hemisphere (F(1, 15) = 6.03, p < .05, M,, = 6.88) and site (F(2, 30) =
430
BARRETT
MEAN
AMPLITUDE
(MICROVOLTS)
TABLE
1
OF THE 600-
TO
POST-Sl WAVEFORMS, COLLAPSED
REGIONS OF THE CONDITIONS
500-900 900-1560
AND RUGG
900-
AND
900-
UNDER
TO ~~~O-MSEC
RHYMING
AND
LATENCY
NONRHYMING
LF
LT
LP
RF
RT
RP
1.15 2.18
1.82 0.26
1.08 -0.56
1.20 4.98
1.31 1.71
-0.43 -0.86
18.71, e = .67, p < .OOl, M,, = 7.97) and to a significant interaction between these factors (F(2, 30) = 13.02, e = .73, p = .OOl, MS, = 1.48). As can be seen from Table 1, the hemispheric asymmetries in this region of the waveform are maximal at frontal sites, and essentially nonexistent at the parietal electrodes. Tukey tests indicated that only the frontal asymmetry was significant (q(2, 15) = 4.27, p < .Ol). Post-S2 analysis. Figure 2 shows the grand average post-S2 waveforms, aligned around a lOO-msec pre-S2 baseline. Rhyme/nonrhyme differences take the form of more negative-going waveforms under the nonrhyming condition. These differences appear to onset earlier over the right than the left hemisphere and to be larger and more sustained at
-RHYME .------ NON-RHYME
w d LP
*,.....a.‘-..
+ I suv
RPJCI ii--+--
400msec
FIG. 2. Grand average post-S2 waveforms, aligned on a pre-S2 baseline. Pz signifies the parietal midline. LF, RF, LT, RT, LP, and RP indicate left and right frontal, temporal, and parietal sites, respectively. Note that the Pz waveform is based on only 14 subjects, this channel being used in 2 others for horizontal EOG (see text).
431
ERPS AND PICTURE-MATCHING
right frontal and temporal sites than at the corresponding left hemisphere electrodes. To assess the reliability of the apparent hemispheric asymmetry in the onset of the rhyme/nonrhyme differences, the mean amplitude of the 300- to 500-msec latency region of these waveforms was measured and analyzed. The remainder of the waveforms was quantified by measuring the 500- to 900-msec latency region. The data from these two latency regions are shown in Table 2 for the lateral electrode sites. ANOVAs on these data employed the factors of condition (rhyme/nonrhyme), hemisphere (left/right), and site (frontal/temporal/parietal). ANOVA of the 300- to 500-msec interval revealed a significant effect of site (F(2, 30) = 27.03, e = .57, p < .OOl, M,, = 26.24), and a significant interaction between condition and hemisphere (F(1, 15) = 10.33, p < .Ol, M,, = 1.49). As can be seen from Table 2, rhyme/nonrhyme differences in this region of the waveform are larger from right than from left hemisphere electrodes. A Tukey test revealed that the mean rhyme/nonrhyme difference from right hemisphere electrodes was significant (q(2, 15) = 3.12, p < .05), while that from left hemisphere sites was not (q(2, 15) = .64). Analysis of the 500- to 900-msec data gave rise to significant main effects of condition (F(1, 15) = 33.68, p < .OOl, M,, = 14.15) and site (F(2,30) = 15.21, e = .56, p = .OOl , M,, = 22.43). In addition, significant interactions were obtained between condition and hemisphere (F( 1, 15) = 8.3 1, p < .025, Mse = 1.84), condition and site (F(2, 30) = 1 I. 18, e = .54, p < .005, M,, = 4.42), and condition, hemisphere, and site (F(2, 30) = 3.97, e = .74, p < .05, M,, = 0.70). The condition by hemisphere interaction reflects the larger rhyme/nonrhyme differences over the right compared to the left hemisphere. The three-way interaction indicates that this asymmetry in rhyme/nonrhyme effects differs across electrodes. As shown in Table 2, these effects are asymmetric across frontal and temporal electrodes, but of almost equal magnitude at the
TABLE MEAN
AMPLITUDE
300-500 Rhyme Nonrhyme d
500-900 Rhyme Nonrhyme d
2
(MICROVOLTS) OF THE 300- TO 500- AND 500- TO 900-MSEC REGIONS OF THE POST-%? WAVEFORMS
LATENCY
LF
LT
LP
RF
RT
RP
-2.21 -2.46 0.25 0.81 0.14 0.67
- 1.46 - 1.55 0.09 3.86 1.72 2.14
3.84 3.32 0.52 7.89 2.94 4.95
- 1.20 -2.42 1.22 1.64 -0.87 2.51
-0.32 -1.71 1.39 4.06 0.58 3.48
5.36 3.71 1.65 7.24 2.08 5.16
432
BARRETT
AND
RUGG
parietal electrodes. This impression was confirmed by post hoc Scheffe tests, which revealed a significant condition by hemisphere interaction for the data from frontal and temporal electrodes (F(1, 15) = 10.99, p < .Ol), but not for the data from the parietal sites (F(1, 15) = < 1). Analysis of horizontal EOG. The strongly frontal distribution of the asymmetries observed in the Sl-S2 interval raises the question of whether these asymmetries are of electrooculographic rather than cerebral origin. The direction of the ERP asymmetries is consistent with what might be expected if subjects were systematically to scan from leftto-right during the post-S1 period, and it is possible that lateral eye movements in this direction are triggered by demanding verbal tasks (Springer & Deutsch, 1985). Inspection of the EOG channels from the initial 14 subjects (especially subjects 11-14, in whom EOG was recorded with a montage intended to maximize the detection of horizontal movements) suggested that post-S1 ERP asymmetries were not the result of systematic eye-movements. This impression was confirmed in subjects 15 and 16, in whom a separate horizontal EOG channel was recorded. The data from these two subjects are shown in Fig. 3, where it can be seen that substantial frontally distributed ERP asymmetries exist in the absence of any detectable systematic horizontal EOG deflection. These data, along with the lack of evidence for systematic eye-movement during the Sl-S2 interval in the remainder of the subjects, strongly suggest that the Sl-S2 asymmetries observed in this task are of cerebral origin. Behavioral data. RT and accuracy data are shown in Table 3. Analysis of the RTs revealed a significant effect of condition (F(1, 15) = 73.31, p < 0.001, M,, = 5801.00), reflecting faster responses on rhyming trials. No significant effect emerged in the analysis of the error data. DISCUSSION
ERPs evoked by sequentially presented pictures in a rhyme-judgement task exhibit reliable interhemispheric asymmetries in two regions of the waveform: during the interval between the presentation of the two items, when the ERPs are more negative-going over the left hemisphere, and following the second item, when rhyme/nonrhyme ERP differences emerge earlier, and are larger, over the right hemisphere. The asymmetries in both regions are qualitatively similar to those occurring when visually presented words are rhyme-matched (e.g., Barrett & Rugg, 1989a). Asymmetries
during the SI-S2
Interval
Two major differences exist between the post-S1 asymmetry observed in the present experiment and the asymmetries found in those studies of rhyme-matching which employed words. First, the onset latency of this asymmetry is substantially later than in the previous experiments.
433
ERPS AND PICTURE-MATCHING
A
B Ir
EOG -+,
I Sl
160msec Sl --z---
,I --
’ 560msec
I s2
-L. HEM ‘-----. R. HEM
1;“”
FIG. 3. ERP and horizontal EOG waveforms from two subjects (A and B), averaged over rhyming and nonrhyming trials, illustrating frontally distributed interhemispheric asymmetries in the absence of systematic horizontal eye-movement.
For example, in the experiment of Rugg (1985a), the left hemisphere was more negative-going than the right within 400 msec of the presentation of Sl. By contrast, in the present study, ERPs from the left hemisphere did not begin to go more negative than the right until around 900 msec after Sl onset. Assuming that this asymmetry reflects the onset of some aspect of phonological processing (see below), its relatively late onset in the present study suggests that it takes longer to derive phonological information from pictures than from words. This is consistent with the finding that RTs in this experiment were some 200 to 300 msec longer than those typically found when word-pairs are rhyme-matched. It is TABLE MEAN
(MSEC) RHVMING
RT % correct
AND AND
3
PERCENTAGE NONRHYMING
CORRECT
FOR
TRIALS
Rhyme
Nonrhyme
923 84.5
1153 89.5
434
BARRETT
AND
RUGG
also consistent with the fact that naming latencies are invariably longer for pictures than for words (Potter & Faulconer, 1975; Young, McWeeny, Ellis, & Hay, 1986). The second difference between the present post-S1 asymmetry and that found during word-matching concerns the anterior/posterior distribution of the effect. When rhyme-matching is performed on words, the post-S1 asymmetry is of approximately equal magnitude at frontal and temporal sites. In contrast, the asymmetry found in the present experiment has a strongly frontal distribution. Two possible explanations could account for these differences in scalp distribution between word- and picture-matching. They could reflect a difference in the neural substrate of the phonological processing of pictures and words, such that pictures engage more anterior brain regions. This might reflect a greater contribution from the language production system to the derivation and maintainance of picture as opposed to word names. Alternatively, the differing distributions of the post-S1 asymmetries in word- and picture-matching may reflect the presence in the picture task of a posteriorly distributed slow-wave asymmetry that overlaps and partially cancels the frontotemporal asymmetry. This possibility gains support from the finding that when faces are employed in sequential matching tasks, a posteriorly distributed, right-more-negative-than-left ERP waveform develops during the interstimulus interval (Barrett & Rugg, 1989b; Barrett et al., 1988). It may be that this asymmetry is evoked by nonverbal stimuli other than faces (e.g., pictures), and that it reflects the activation by such stimuli of processes lateralized to the right hemisphere. The nonsignificant trend in the present experiment towards greater negativity over right than over left parietal regions early in the interstimulus interval (see Fig. 1) is consistent with this possibility. Despite the differences between the present and previous experiments, it is clear that more negative-going ERPs from the left hemisphere during rhyme-matching do not depend upon the employment of words. Common to the rhyme-matching of pictures and words is the need to derive and maintain phonological information. It is therefore arguable that the ERP asymmetries developing during the interstimulus interval in these tasks reflect the activation of regions of the left hemisphere that subserve some aspect(s) of phonological processing common to picture and word names. Post-S2
Effects
As can be seen in Fig. 2, the rhyme/nonrhyme manipulation resulted in a marked modulation of the Pz-maximum late positive component (WOO), which is some 5 PV larger when evoked by rhyming than nonrhyming stimuli. The modulation of WOO clearly contributes significantly to the rhyme/nonrhyme effects observed in this study, particularly at parietal sites for the second of the two latency regions analyzed. How-
ERPS
AND
PICTURE-MATCHING
435
ever, it is not the only basis for the rhyme/nonrhyme effects observed in the post-S2 waveforms. These effects initially have a scalp distribution quite different from the Pz-maximum P700, in that they are of almost equal magnitude at frontal, temporal, and parietal sites. And, unlike later regions of the waveforms, the asymmetry of the early effects is equally extreme at all three electrode pairs (see Table 2). These effects most likely reflect the essentially unilateral modulation of an N450-like comby the subsequent bilateral, posterior-maxponent , “uncontaminated” imum, P700 wave. The post-S2 data indicate that asymmetries in rhyme/nonrhyme ERP differences do not depend upon the employment of orthographic material. These data are thus consistent with the view that asymmetrical rhyme/nonrhyme ERP differences reflect the outcome of phonological rather than orthographic processing. However, the influence of orthography cannot be ruled out entirely, as it is possible that activation of the phonological form of a picture name always entails the coactivation of its orthographic representation. Whether such coactivation occurs could perhaps be determined by testing whether, in a rhyme-judgement task, pictures that share orthographically similar nonrhyming names (e.g., COMB-BOMB) are harder to process than other nonrhyming items, as is the case for visually (Rugg & Barrett, 1987) and auditorily presented (Seidenberg & Tanenhaus, 1979; Donnenwerth-Nolan, Tanenhaus, & Seidenberg, 1981) words. If orthography does influence rhyme-judgements on picture names, future experiments attempting to determine the relative importance of orthographic and phonological variables with respect to rhyme/nonrhyme ERP effects may have to employ illiterate subjects! As discussed elsewhere (Rugg, 1984a; Rugg & Barrett, 1987; Rugg, Kok, Barrett, & Fischler, 1986), the lateralization of an ERP effect on the scalp does not necessarily entail that the generators of the effect are lateralized to the underlying hemisphere. However, in the absence of evidence to the contrary, this is a reasonable assumption. In the present case, the implication of this assumption is that the right hemisphere is more sensitive to variations in the phonological properties of words and picture names than is the left. This might indicate a more important role for the right hemisphere in phonological processing than suggested by present neuropsychological evidence. Alternatively, and much more likely, rhyme/nonrhyme ERP effects may reflect differences in the activity of brain systems responsive to the outcome of phonological processing, rather than in systems responsible for the processing per se. For example, as noted in the Introduction, Rugg and Barrett (1987) have suggested that N450 is sensitive to the degree of congruity between word pairs. Rugg and Barrett also speculated on why ERPs were modulated more asymmetrically by variations in phonological than semantic rela-
436
BARRETT
AND RUGG
tionships. They proposed that this was because the generators of N400/450 in the right hemisphere are responsive to congruity information from both semantic and nonsemantic processing domains, while the homologous generators in the left hemisphere are modulated almost exclusively by variations in semantic congruity. The data from the present experiment are consistent with this suggestion, as is the fact that when pictures are employed in a semantic matching task, match/nonmatch ERP effects are distributed bilaterally (Barrett & Rugg, 1989~). In conclusion, this experiment demonstrates that ERPs evoked in a pictorial phonological matching task exhibit qualitative similarities to those evoked by words in an analogous task. This suggests that some of the processes activated when picture and word pairs are rhymematched have common or closely related neural substrates. REFERENCES Barrett, S. E., & Rugg, M. D. 1987. Event-related potentials in semantic and phonological matching tasks. Psychophysiology, 24, 577-578. Barrett. S. E., & Rugg, M. D. 1989a. Asymmetries in event-related brain potentials during rhyme-matching: Confirmation of the null effects of handedness. Neuropsychologiu, 17, 539-548.
Barrett, S. E., & Rugg, M. D. 1989b. Event-related potentials and the semantic matching of faces. Neuropsychologia, 27, 913-922. Barrett, S. E., & Rugg, M. D. 1989~. Event-related potentials and the semantic matching of pictures, submitted for publication. Barrett, S. E., Rugg, M. D., & Perrett, D. 1. 1988. Event-related potentials and the matching of familiar and unfamiliar faces. Neuropsychologia, 26, 105-I 17. Bentin, S., McCarthy, G., & Wood, C. C. 1985. Event-related potentials, lexical decision and semantic priming. Electroencephalography and clinical Neurophysiology, 60, 343355. Coltheart, M. 1980. Deep dyslexia: A right hemisphere hypothesis. In M. Coltheart, K. Patterson, & J. C. Marshall (Eds.), Deep dyslexia. London: Routledge & Kegan Paul. Coltheart, M. 1983. The right hemisphere and disorders of reading. In A. W. Young (Ed.), Functions of the right cerebral hemisphere. London, Academic Press. Donnenwerth-Nolan, S., Tanenhaus, M. K., & Seidenberg, M. S. 1981. Multiple code activation in word recognition: Evidence from rhyme-monitoring. Journal of Experimental Psychology: Human Learning and Memory, 7, 170-180. Ellis, H. 1983. The role of the right hemisphere in face perception. In A. W. Young (Ed.), Functions of the right cerebral hemisphere. London, Academic Press. Jasper, H. H. 1958. The ten twenty system electrode system of the International Federation. Electroencephalography
and
Clinical
Neurophysiolology,
10, 371-375.
Kutas, M., & Hillyard, S. A. 1980. Reading senseless sentences: Brain potentials reflect semantic incongruity. Scienae, 207, 203-205. Kutas, M., & Hillyard, S. A. 1984. Brain potentials during reading reflect word expectancy and semantic association. Nature(London), 307, 161-163. Kutas, M., Van Petten, C., & Besson, M. 1988. Event-related potential asymmetries during the reading of sentences. Electroencephalography and Clinical Neurophysiology, 69, 218-233. Patterson, K., & Besner, D. 1984. Is the right hemisphere literate? Cognitive Neuropsychology,
1, 315-341.
ERPS AND PICTURE-MATCHING Potter, M. C., & Faulconer, ture(London),
B. A. 1975. Time to understand pictures and words. Na-
253, 437-438.
Rugg, M. D. 1984a. Event-related Language,
431
potentials in phonological
matching tasks. Brain
and
23, 225-240.
Rugg, M. D. 1984b. Event-related potentials and the phonological processing of words and non-words. Neuropsychologia, 22, 435-443. Rugg, M. D. 1985a. The effects of handedness on event-related potentials in a rhymematching task. Neuropsychologiu, 23, 765-775. Rugg, M. D. 1985b. The effects of semantic priming and word repetition on event-related potentials. Psychophysiology, 22, 642-647. Rugg, M. D., & Barrett, S. E. 1987. Event-related potentials and the interaction between orthographic and phonological information in a rhyme-judgment task. Brain and Language, 32, 336-361. Rugg, M. D., Kok, A., Barrett, G., & Fischler, I. 1986. ERPs associated with language and hemispheric specialization. In W. C. McCallum, R. Zappoli, and F. Denoth (Eds.), Cerebral psychophysiology: Studies in event-related potentials. Amsterdam: Elsevier, 1986. Seidenberg, M. S., & Tanenhaus, M. K. 1979. Orthographic effects on rhyme monitoring. Journal of Experimental Psychology: Human Learning and Memory, 5, 546-554. Springer, S. P., & Deutsch, G. 1985. Left brain, right brain. New York: W.H. Freeman. 2nd ed. Young, A. W., McWeeny, K. H., Ellis, A. W., & Hay, D. C. 1986. Naming and categorising faces and written names. Quarterly Journal of Experimental Psychology A, 38, 297318. Zaidel, E., & Peters, A. M. 1981. Phonological encoding and ideographic reading by the disconnected right hemisphere: Two case studies. Brain and Language, 14, 205-234.
AND
38, 424-437 (IWO)
LANGUAGE
Event-Related Potentials and the Phonological Matching of Picture Names SARAH E. BARRETT Weikome
Brain
Research
Group,
AND MICHAEL
University
D. RUGG
of St. Andrews,
Fife,
United
Kingdom
Event-related potentials (ERPs) were recorded from one midline and three pairs of lateral electrodes while subjects determined whether a pair of sequentially presented pictures had rhyming or nonrhyming names. During the 1.56-set interval between the two pictures, the slow ERP wave recorded over the left hemisphere was more negative-going than that over the right, especially at frontal electrodes. The ERPs evoked by the second picture differed as a function of whether its name rhymed with its predecessor. This difference, taking the form of increased negativity in ERPs to nonrhyming items, had an earlier onset and a greater magnitude at right than at left hemisphere electrodes. This pattern of ERP asymmetries is qualitatively similar to that found when words are rhymematched. It is therefore concluded that such asymmetries do not depend on the employment of orthographic material and may reflect some aspect(s) of the phonological processing of visually presented material. 0 IWO Academic press. h.
INTRODUCTION
Neuropsychological evidence suggests that the phonological processing of visually presented material involves processes which are strongly lateralized to the left hemisphere in the majority of individuals (Coltheart, 1980, 1983; Patterson & Besner, 1984; Zaidel & Peters, 1981). A task such as rhyme-judgement, which requires subjects to determine whether two words share the same terminal phonology, would thus be expected to rely heavily on the left hemisphere. This task has therefore been used in several previous studies investigating the relationship of lateralized cognitive processes to lateral asymmetries in scalp-recorded event-related potentials (ERPs). The basic experimental paradigm employed in the studies reviewed below involves the sequential presentation of two items, separated by a This work was supported the U.K. We thank Karalyn reprint requests to M. D. chology, University of St.
by the Wellcome Trust and the Medical Research Council of Patterson for providing some of the stimulus materials. Address Rugg, Wellcome Brain Research Group, Department of PsyAndrews, Fife KY16 9JU, United Kingdom. 424
0093-934x/90 $3.00 Copyright All rights
0 1990 by Academic Press, Inc. of reproduction in any form reserved
ERPS AND PICTURE-MATCHING
425
short period of time (typically 1 or 1.56 set), with the requirement to determine whether the second item rhymes with the first. A series of experiments (Barrett & Rugg, 1989a; Rugg, 1984a,b, 1985a; Rugg & Barrett, 1987) has demonstrated that ERPs recorded during this task show a characteristic pattern of asymmetries, involving two different regions of the ERP waveform. The earlier of these asymmetries, which occurs during the interval between the two items, appears consistent with the neuropsychological evidence concerning the lateralization of processing during rhyme-matching. The direction of the later asymmetry, occurring in the ERPs evoked by the second item, seems at first sight to be at variance with this evidence. The first of these ERP asymmetries involves the slow negative wave (the “contingent negative variation” or CNV) that develops during the interval between the two words. This wave is more negative-going from left than from right hemisphere electrodes, especially at frontal and temporal sites. It has been suggested that this asymmetry reflects the preeminence of left hemisphere mechanisms in the encoding and short-term storage of the phonological information necessary to perform the rhymematching task (Rugg, 1984a). This interpretation is strengthened by the finding that the CNV that develops between sequentially presented faces in face-matching tasks shows an asymmetry in the opposite direction (Barrett, Rugg, & Perrett, 1988; Barrett & Rugg, 1989b). Because facematching would be expected to engage processes lateralized to the right hemisphere (Ellis, 1983), this gives support to the idea that CNV asymmetries signify the engagement of lateralized material- or processingspecific mechanisms. The other ERP asymmetry occuring during rhyme-matching follows the presentation of the second word. The ERP waveforms evoked by this word diverge as a consequence of whether or not the word rhymes with the initial item. This rhyme/nonrhyme difference takes the form of a more negative-going waveform in ERPs from nonrhyming words, and appears to involve the modulation of a negative-going wave peaking around 450 msec poststimulus (N450). This difference in the ERPs evoked by rhyming and nonrhyming words is larger from right than from left hemisphere electrodes, especially over frontal and temporal regions of the scalp. The negative wave modulated by the rhyme/nonrhyme manipulation, “N450,” may be allied to late negative ERP waves, such as the “N400,“’ I The terminology applied to late negative waves sensitive to linguistic manipulations is confusing. In the present paper, “N400” will be used to designate negativities modulated by semantic manipulations, and “N450” will refer to the negative wave modulated by phonological variables. These terms are descriptive only and should not be taken to imply that the waves to which they refer have different neural sources.
426
BARRETT
AND
RUGG
which have been shown to be sensitive to manipulations of semantic priming. Both in sentence (e.g., Kutas & Hillyard, 1980, 1984) and single word (Bentin, McCarthy, & Wood, 1985; Rugg, 1985b) priming experiments, N400 is larger in ERPs evoked by unprimed than by primed words, and it has been suggested that the amplitude of N400 is inversely proportional to the degree that a word has been semantically primed by its preceding context (Kutas & Hillyard, 1984). Unlike the modulation of N450 in rhyme-matching, however, the N400 effects observed in studies manipulating semantic relationships either do not differ significantly between the two hemispheres or are only slightly lateralized to right hemisphere electrodes (Barrett & Rugg, 1987; Kutas, Van Petten, & Besson, 1988). Extrapolating from the work investigating the influence of semantic priming on N400, Rugg (1984b) suggested that the modulation of N450 in the rhyme-judgement task may reflect the differing extents to which rhyming and nonrhyming words have been primed by the preceding item. Rugg and Barrett (1987) further suggested that N450 modulation in rhymematching is controlled not by the phonological relationship between the two members of a word pair but by the orthographic relationship. The “orthographic priming hypothesis” was inspired by the fact that while the number of rhyming word pairs in English is very large, the number of terminal orthographic segments that the members of such pairs can possess is relatively small. For example, almost all words that rhyme with PALE will end in -ALE, -AIL, or -EIL. Rugg and Barrett suggested that, in the rhyme-judgement task, the rhyme-sensitive N450 is modulated by whether the second word contains an orthographic segment primed by its predecessor. As virtually all rhyming words contain such a segment, but most nonrhyming words do not, N450 would be larger for nonrhyming than rhyming words. In their initial experiments, Rugg and Barrett (1987) obtained data consistent with the orthographic priming hypothesis. Nonrhyming words which shared the same terminal orthographic segment (e.g., COUGHBOUGH) did not evoke an asymmetric N450 wave, indicating that conflicting phonologies are not always sufficient to elicit the “rhyme-sensitive” N450. However, the results of the final experiment were not consistent with the orthographic priming hypothesis. This experiment employed nonrhyming word pairs in which the second item shared its terminal orthographic segment with another word which did rhyme with the first item (e.g., STAKE-FREAK [c.f. BREAK]). According to the orthographic priming hypothesis, such nonrhyming words contain a primed orthographic segment, and thus the ERPs which they evoke should not contain an asymmetrically enhanced N450 wave. This prediction was not borne out; “STAKE-FREAK” pairs gave rise to the same pattern of ERP effects in the N450 latency range as controls. Rugg
ERPS AND PICTURE-MATCHING
427
and Barrett therefore suggested that the mechanism responsible for modulating N450 is sensitive to the general level of “congruity” between word pairs, such that either a highly similar phonology or a highly similar orthography is sufficient to attenuate the N450 recorded from right hemisphere electrodes. Rugg and Barrett’s (1987) study casts doubt on whether the modulation of ERPs by rhyming and nonrhyming word-pairs is determined by the relationship between the words’ orthographic properties. The present experiment pursues this issue by employing pictorial stimuli in a rhymejudgement task. As such stimuli contain no explicit orthographic information, any effects on ERPs of the rhyme/nonrhyme manipulation can be attributed with some confidence to the phonological attributes of the items’ names. Therefore if these effects are similar to those observed when words are rhyme-matched, this would provide strong evidence in favor of the idea that asymmetrical rhyme/nonrhyme differences in ERPs reflect the outcome of phonological rather than orthographic processing. The use of pictorial stimuli also allows further investigation of the CNV asymmetries observed in previous studies of rhyme-matching. If these asymmetries do indeed reflect the lateralization of phonological processing, they should be as apparent when phonological information is derived from pictures as when it is derived from words. METHOD Subjects. Ten female and six male right-handed (as defined by writing hand) young adults participated in this experiment. Stimuli. One hundred and sixty black and white slides, each portraying a line drawing of a common object, were used to form a set of 80 “rhyming” stimulus pairs. A rhyming pair consisted of two pictures with rhyming names (e.g., spoon-moon, sock-clock).’ Two stimulus lists were constructed using these items. List I was constructed by randomly designating 40 of the pool of 80 pairs as rhyming pairs, and then rearranging the remainder so that these were nonrhyming. These 80 pairs of pictures, 40 rhyming and 40 nonrhyming. were then randomly ordered to form List I. List 2 was constructed by employing as rhyming stimuli those pairs of pictures that had been rearranged to provide the nonrhyming stimuli for List I, and rearranging the members of the rhyming pairs of List I so as to provide the 40 nonrhyming simuli. These 80 items were then randomly ordered to form List 2. Thus no picture was repeated within a list, but across the lists each appeared once in a nonrhyming and once in a rhyming pair. Half of the subjects were shown List I, and the remainder saw List 2, thereby eliminating any possible confound between items and the rhyme/nonrhyme manipulation. A further 5 rhyming and 5 nonrhyming picture pairs, none of which was employed in the experimental lists, were used to form a practice list. The stimulus slides were displayed on a TV monitor, and subtended a visual angle of approximately 2” at the viewing distance of 90 cm. Each slide was presented for 300 msec, with an interval of 1560 msec between the onset of the first (Sl) and second (S2) stimulus of a pair. The interval between consecutive Sl onsets was 8.5 sec. A small central fixation dot was continuously present on the monitor. A row of crosses, easily visible within ’ Copies of the stimuli are available from the authors upon request.
428
BARRETT
AND RUGG
subjects’ peripheral vision, was presented on a separate TV monitor immediately below the one on which the pictures were presented. The appearance of these crosses indicated when eye movements were permissible. They were displayed from 900 msec after the onset of S2 until 600 msec before the onset of SI on the subsequent trial. Procedure. Following electrode application, subjects were seated in front of the TV monitors, with the index fingers of each hand resting on laterally positioned microswitch push-buttons. They were instructed to respond quickly and accurately with one hand when the names of the two stimuli in a pair rhymed, and with the other hand when the stimuli did not rhyme. The hands used by a subject for rhyme and nonrhyme reponses remained the same throughout the experimental session, with the hand used for each response being counterbalanced across subjects. Subjects were also instructed to keep eye and body movements to a minimum throughout each trial, to maintain fixation on the dot in the center of the presentation window, and to blink only when the crosses were present on the lower monitor. A short break was given after 40 trials. ERP recording. EEG was recorded, with reference to linked mastoids, from a parietal midline electrode (the Pz site of the International ten twenty system; Jasper, 1958), and from lateral frontal, temporal, and parietal locations. Frontal electrodes were 75% of the distance from Fz to F7 on the left and F8 on the right, temporal electrodes were 75% of the distance from Cz to T3 on the left and T4 on the right, and parietal electrodes were 75% of the distance from Pz to TS on the left and T6 on the right. Cz served as ground and, in subjects 1 to 10, EOG was recorded between electrodes placed on the outer canthus of the left eye, and above the right eyebrow. As the experiment progressed, it became clear that the lateral ERP asymmetry emerging during the SI-S2 interval was typically frontally distributed, and that it would be necessary to take additional steps to eliminate the possibility that this asymmetry resulted from systematic horizontal eye-movement. Therefore, in subjects 11 to 14, an EOG montage more sensitive to horizontal movements was employed, by placing the right-sided EOG electrode more laterally and increasing the gain of EOG channel from 5K to 20K. In subjects I5 and I6 additional sensitivity to horizontal eye-movements was achieved by omitting the Pz channel and recording vertical and horizontal EOG on separate channels. In these two subjects, vertical EOG was recorded with electrodes above and below the left eye and horizontal EOG, with electrodes on the outer canthus of each eye. Interelectrode impedances were less than 5 kohm, and all channels were amplified with a bandwidth of 0.03-30 Hz (3dB points). The sampling rate was 1 point per IO msec, beginning 100 msec before the onset of Sl and continuing for 2460 msec thereafter. Separate ERPs were formed for each subject from the trials of each experimental condition. Only trials on which a correct behavioral response was made, and no eye movement artifact detected, were used to form the averages.
RESULTS
The ERPs were quantified by measuring the mean amplitude of selected latency regions of the waveforms with respect to a lOO-msec prestimulus baseline. These data were then subjected to repeated-measures ANOVA, employing the Greenhouse-Geisser correction for inhomogeneity of covariance. Post hoc tests were performed with the Tukey HSD test for pairwise comparisons between means and Scheffe’s procedure for decomposing interactions. The significance level for all ANOVAs and post hoc analyses was set at p < 0.05. Post-S1 analysis. Grand average post-S1 waveforms from lateral electrode sites are shown in Fig. 1, collapsed across rhyming and nonrhyming
ERPS AND PICTURE-MATCHING
I
’
Sl
429
I
1560msec
+ I 5UV
’
s2
-
L. HEM.
----
R. HEM.
FIG. 1. Grand average waveforms from lateral electrodes, aligned on a pre-Sl baseline and averaged over rhyming and nonrhyming trials. F, T, and P indicate frontal, temporal, and parietal sites, respectively.
conditions. After a series of transient deflections over approximately the first 400 msec following S 1, the waveforms are characterized by a prominent, frontally distributed negative-going deflection, followed by a slow wave lasting until S2. At frontal sites, this wave is positive-going, and substantially more negative over the left hemisphere from around 900 msec until the end of the post-S2 epoch. At the parietal sites there appears to be a tendency for the earlier region of the slow wave to show a reversal of this asymmetry. The waveforms during the S 1-S2 interval were quantified by measuring the mean amplitude of the regions 600-900 msec and 900-1560 msec post-S 1, to capture the early and late asymmetries apparent in Fig. I. These data are shown in Table 1. They were subjected to repeatedmeasures ANOVA with factors of hemisphere (left/right) and site (frontal/temporal/parietal). Analysis of the 600- to 900-msec region revealed no significant effects. ANOVA of the 900- to 1560-msec data gave rise to significant effects of hemisphere (F(1, 15) = 6.03, p < .05, M,, = 6.88) and site (F(2, 30) =
430
BARRETT
MEAN
AMPLITUDE
(MICROVOLTS)
TABLE
1
OF THE 600-
TO
POST-Sl WAVEFORMS, COLLAPSED
REGIONS OF THE CONDITIONS
500-900 900-1560
AND RUGG
900-
AND
900-
UNDER
TO ~~~O-MSEC
RHYMING
AND
LATENCY
NONRHYMING
LF
LT
LP
RF
RT
RP
1.15 2.18
1.82 0.26
1.08 -0.56
1.20 4.98
1.31 1.71
-0.43 -0.86
18.71, e = .67, p < .OOl, M,, = 7.97) and to a significant interaction between these factors (F(2, 30) = 13.02, e = .73, p = .OOl, MS, = 1.48). As can be seen from Table 1, the hemispheric asymmetries in this region of the waveform are maximal at frontal sites, and essentially nonexistent at the parietal electrodes. Tukey tests indicated that only the frontal asymmetry was significant (q(2, 15) = 4.27, p < .Ol). Post-S2 analysis. Figure 2 shows the grand average post-S2 waveforms, aligned around a lOO-msec pre-S2 baseline. Rhyme/nonrhyme differences take the form of more negative-going waveforms under the nonrhyming condition. These differences appear to onset earlier over the right than the left hemisphere and to be larger and more sustained at
-RHYME .------ NON-RHYME
w d LP
*,.....a.‘-..
+ I suv
RPJCI ii--+--
400msec
FIG. 2. Grand average post-S2 waveforms, aligned on a pre-S2 baseline. Pz signifies the parietal midline. LF, RF, LT, RT, LP, and RP indicate left and right frontal, temporal, and parietal sites, respectively. Note that the Pz waveform is based on only 14 subjects, this channel being used in 2 others for horizontal EOG (see text).
431
ERPS AND PICTURE-MATCHING
right frontal and temporal sites than at the corresponding left hemisphere electrodes. To assess the reliability of the apparent hemispheric asymmetry in the onset of the rhyme/nonrhyme differences, the mean amplitude of the 300- to 500-msec latency region of these waveforms was measured and analyzed. The remainder of the waveforms was quantified by measuring the 500- to 900-msec latency region. The data from these two latency regions are shown in Table 2 for the lateral electrode sites. ANOVAs on these data employed the factors of condition (rhyme/nonrhyme), hemisphere (left/right), and site (frontal/temporal/parietal). ANOVA of the 300- to 500-msec interval revealed a significant effect of site (F(2, 30) = 27.03, e = .57, p < .OOl, M,, = 26.24), and a significant interaction between condition and hemisphere (F(1, 15) = 10.33, p < .Ol, M,, = 1.49). As can be seen from Table 2, rhyme/nonrhyme differences in this region of the waveform are larger from right than from left hemisphere electrodes. A Tukey test revealed that the mean rhyme/nonrhyme difference from right hemisphere electrodes was significant (q(2, 15) = 3.12, p < .05), while that from left hemisphere sites was not (q(2, 15) = .64). Analysis of the 500- to 900-msec data gave rise to significant main effects of condition (F(1, 15) = 33.68, p < .OOl, M,, = 14.15) and site (F(2,30) = 15.21, e = .56, p = .OOl , M,, = 22.43). In addition, significant interactions were obtained between condition and hemisphere (F( 1, 15) = 8.3 1, p < .025, Mse = 1.84), condition and site (F(2, 30) = 1 I. 18, e = .54, p < .005, M,, = 4.42), and condition, hemisphere, and site (F(2, 30) = 3.97, e = .74, p < .05, M,, = 0.70). The condition by hemisphere interaction reflects the larger rhyme/nonrhyme differences over the right compared to the left hemisphere. The three-way interaction indicates that this asymmetry in rhyme/nonrhyme effects differs across electrodes. As shown in Table 2, these effects are asymmetric across frontal and temporal electrodes, but of almost equal magnitude at the
TABLE MEAN
AMPLITUDE
300-500 Rhyme Nonrhyme d
500-900 Rhyme Nonrhyme d
2
(MICROVOLTS) OF THE 300- TO 500- AND 500- TO 900-MSEC REGIONS OF THE POST-%? WAVEFORMS
LATENCY
LF
LT
LP
RF
RT
RP
-2.21 -2.46 0.25 0.81 0.14 0.67
- 1.46 - 1.55 0.09 3.86 1.72 2.14
3.84 3.32 0.52 7.89 2.94 4.95
- 1.20 -2.42 1.22 1.64 -0.87 2.51
-0.32 -1.71 1.39 4.06 0.58 3.48
5.36 3.71 1.65 7.24 2.08 5.16
432
BARRETT
AND
RUGG
parietal electrodes. This impression was confirmed by post hoc Scheffe tests, which revealed a significant condition by hemisphere interaction for the data from frontal and temporal electrodes (F(1, 15) = 10.99, p < .Ol), but not for the data from the parietal sites (F(1, 15) = < 1). Analysis of horizontal EOG. The strongly frontal distribution of the asymmetries observed in the Sl-S2 interval raises the question of whether these asymmetries are of electrooculographic rather than cerebral origin. The direction of the ERP asymmetries is consistent with what might be expected if subjects were systematically to scan from leftto-right during the post-S1 period, and it is possible that lateral eye movements in this direction are triggered by demanding verbal tasks (Springer & Deutsch, 1985). Inspection of the EOG channels from the initial 14 subjects (especially subjects 11-14, in whom EOG was recorded with a montage intended to maximize the detection of horizontal movements) suggested that post-S1 ERP asymmetries were not the result of systematic eye-movements. This impression was confirmed in subjects 15 and 16, in whom a separate horizontal EOG channel was recorded. The data from these two subjects are shown in Fig. 3, where it can be seen that substantial frontally distributed ERP asymmetries exist in the absence of any detectable systematic horizontal EOG deflection. These data, along with the lack of evidence for systematic eye-movement during the Sl-S2 interval in the remainder of the subjects, strongly suggest that the Sl-S2 asymmetries observed in this task are of cerebral origin. Behavioral data. RT and accuracy data are shown in Table 3. Analysis of the RTs revealed a significant effect of condition (F(1, 15) = 73.31, p < 0.001, M,, = 5801.00), reflecting faster responses on rhyming trials. No significant effect emerged in the analysis of the error data. DISCUSSION
ERPs evoked by sequentially presented pictures in a rhyme-judgement task exhibit reliable interhemispheric asymmetries in two regions of the waveform: during the interval between the presentation of the two items, when the ERPs are more negative-going over the left hemisphere, and following the second item, when rhyme/nonrhyme ERP differences emerge earlier, and are larger, over the right hemisphere. The asymmetries in both regions are qualitatively similar to those occurring when visually presented words are rhyme-matched (e.g., Barrett & Rugg, 1989a). Asymmetries
during the SI-S2
Interval
Two major differences exist between the post-S1 asymmetry observed in the present experiment and the asymmetries found in those studies of rhyme-matching which employed words. First, the onset latency of this asymmetry is substantially later than in the previous experiments.
433
ERPS AND PICTURE-MATCHING
A
B Ir
EOG -+,
I Sl
160msec Sl --z---
,I --
’ 560msec
I s2
-L. HEM ‘-----. R. HEM
1;“”
FIG. 3. ERP and horizontal EOG waveforms from two subjects (A and B), averaged over rhyming and nonrhyming trials, illustrating frontally distributed interhemispheric asymmetries in the absence of systematic horizontal eye-movement.
For example, in the experiment of Rugg (1985a), the left hemisphere was more negative-going than the right within 400 msec of the presentation of Sl. By contrast, in the present study, ERPs from the left hemisphere did not begin to go more negative than the right until around 900 msec after Sl onset. Assuming that this asymmetry reflects the onset of some aspect of phonological processing (see below), its relatively late onset in the present study suggests that it takes longer to derive phonological information from pictures than from words. This is consistent with the finding that RTs in this experiment were some 200 to 300 msec longer than those typically found when word-pairs are rhyme-matched. It is TABLE MEAN
(MSEC) RHVMING
RT % correct
AND AND
3
PERCENTAGE NONRHYMING
CORRECT
FOR
TRIALS
Rhyme
Nonrhyme
923 84.5
1153 89.5
434
BARRETT
AND
RUGG
also consistent with the fact that naming latencies are invariably longer for pictures than for words (Potter & Faulconer, 1975; Young, McWeeny, Ellis, & Hay, 1986). The second difference between the present post-S1 asymmetry and that found during word-matching concerns the anterior/posterior distribution of the effect. When rhyme-matching is performed on words, the post-S1 asymmetry is of approximately equal magnitude at frontal and temporal sites. In contrast, the asymmetry found in the present experiment has a strongly frontal distribution. Two possible explanations could account for these differences in scalp distribution between word- and picture-matching. They could reflect a difference in the neural substrate of the phonological processing of pictures and words, such that pictures engage more anterior brain regions. This might reflect a greater contribution from the language production system to the derivation and maintainance of picture as opposed to word names. Alternatively, the differing distributions of the post-S1 asymmetries in word- and picture-matching may reflect the presence in the picture task of a posteriorly distributed slow-wave asymmetry that overlaps and partially cancels the frontotemporal asymmetry. This possibility gains support from the finding that when faces are employed in sequential matching tasks, a posteriorly distributed, right-more-negative-than-left ERP waveform develops during the interstimulus interval (Barrett & Rugg, 1989b; Barrett et al., 1988). It may be that this asymmetry is evoked by nonverbal stimuli other than faces (e.g., pictures), and that it reflects the activation by such stimuli of processes lateralized to the right hemisphere. The nonsignificant trend in the present experiment towards greater negativity over right than over left parietal regions early in the interstimulus interval (see Fig. 1) is consistent with this possibility. Despite the differences between the present and previous experiments, it is clear that more negative-going ERPs from the left hemisphere during rhyme-matching do not depend upon the employment of words. Common to the rhyme-matching of pictures and words is the need to derive and maintain phonological information. It is therefore arguable that the ERP asymmetries developing during the interstimulus interval in these tasks reflect the activation of regions of the left hemisphere that subserve some aspect(s) of phonological processing common to picture and word names. Post-S2
Effects
As can be seen in Fig. 2, the rhyme/nonrhyme manipulation resulted in a marked modulation of the Pz-maximum late positive component (WOO), which is some 5 PV larger when evoked by rhyming than nonrhyming stimuli. The modulation of WOO clearly contributes significantly to the rhyme/nonrhyme effects observed in this study, particularly at parietal sites for the second of the two latency regions analyzed. How-
ERPS
AND
PICTURE-MATCHING
435
ever, it is not the only basis for the rhyme/nonrhyme effects observed in the post-S2 waveforms. These effects initially have a scalp distribution quite different from the Pz-maximum P700, in that they are of almost equal magnitude at frontal, temporal, and parietal sites. And, unlike later regions of the waveforms, the asymmetry of the early effects is equally extreme at all three electrode pairs (see Table 2). These effects most likely reflect the essentially unilateral modulation of an N450-like comby the subsequent bilateral, posterior-maxponent , “uncontaminated” imum, P700 wave. The post-S2 data indicate that asymmetries in rhyme/nonrhyme ERP differences do not depend upon the employment of orthographic material. These data are thus consistent with the view that asymmetrical rhyme/nonrhyme ERP differences reflect the outcome of phonological rather than orthographic processing. However, the influence of orthography cannot be ruled out entirely, as it is possible that activation of the phonological form of a picture name always entails the coactivation of its orthographic representation. Whether such coactivation occurs could perhaps be determined by testing whether, in a rhyme-judgement task, pictures that share orthographically similar nonrhyming names (e.g., COMB-BOMB) are harder to process than other nonrhyming items, as is the case for visually (Rugg & Barrett, 1987) and auditorily presented (Seidenberg & Tanenhaus, 1979; Donnenwerth-Nolan, Tanenhaus, & Seidenberg, 1981) words. If orthography does influence rhyme-judgements on picture names, future experiments attempting to determine the relative importance of orthographic and phonological variables with respect to rhyme/nonrhyme ERP effects may have to employ illiterate subjects! As discussed elsewhere (Rugg, 1984a; Rugg & Barrett, 1987; Rugg, Kok, Barrett, & Fischler, 1986), the lateralization of an ERP effect on the scalp does not necessarily entail that the generators of the effect are lateralized to the underlying hemisphere. However, in the absence of evidence to the contrary, this is a reasonable assumption. In the present case, the implication of this assumption is that the right hemisphere is more sensitive to variations in the phonological properties of words and picture names than is the left. This might indicate a more important role for the right hemisphere in phonological processing than suggested by present neuropsychological evidence. Alternatively, and much more likely, rhyme/nonrhyme ERP effects may reflect differences in the activity of brain systems responsive to the outcome of phonological processing, rather than in systems responsible for the processing per se. For example, as noted in the Introduction, Rugg and Barrett (1987) have suggested that N450 is sensitive to the degree of congruity between word pairs. Rugg and Barrett also speculated on why ERPs were modulated more asymmetrically by variations in phonological than semantic rela-
436
BARRETT
AND RUGG
tionships. They proposed that this was because the generators of N400/450 in the right hemisphere are responsive to congruity information from both semantic and nonsemantic processing domains, while the homologous generators in the left hemisphere are modulated almost exclusively by variations in semantic congruity. The data from the present experiment are consistent with this suggestion, as is the fact that when pictures are employed in a semantic matching task, match/nonmatch ERP effects are distributed bilaterally (Barrett & Rugg, 1989~). In conclusion, this experiment demonstrates that ERPs evoked in a pictorial phonological matching task exhibit qualitative similarities to those evoked by words in an analogous task. This suggests that some of the processes activated when picture and word pairs are rhymematched have common or closely related neural substrates. REFERENCES Barrett, S. E., & Rugg, M. D. 1987. Event-related potentials in semantic and phonological matching tasks. Psychophysiology, 24, 577-578. Barrett. S. E., & Rugg, M. D. 1989a. Asymmetries in event-related brain potentials during rhyme-matching: Confirmation of the null effects of handedness. Neuropsychologiu, 17, 539-548.
Barrett, S. E., & Rugg, M. D. 1989b. Event-related potentials and the semantic matching of faces. Neuropsychologia, 27, 913-922. Barrett, S. E., & Rugg, M. D. 1989~. Event-related potentials and the semantic matching of pictures, submitted for publication. Barrett, S. E., Rugg, M. D., & Perrett, D. 1. 1988. Event-related potentials and the matching of familiar and unfamiliar faces. Neuropsychologia, 26, 105-I 17. Bentin, S., McCarthy, G., & Wood, C. C. 1985. Event-related potentials, lexical decision and semantic priming. Electroencephalography and clinical Neurophysiology, 60, 343355. Coltheart, M. 1980. Deep dyslexia: A right hemisphere hypothesis. In M. Coltheart, K. Patterson, & J. C. Marshall (Eds.), Deep dyslexia. London: Routledge & Kegan Paul. Coltheart, M. 1983. The right hemisphere and disorders of reading. In A. W. Young (Ed.), Functions of the right cerebral hemisphere. London, Academic Press. Donnenwerth-Nolan, S., Tanenhaus, M. K., & Seidenberg, M. S. 1981. Multiple code activation in word recognition: Evidence from rhyme-monitoring. Journal of Experimental Psychology: Human Learning and Memory, 7, 170-180. Ellis, H. 1983. The role of the right hemisphere in face perception. In A. W. Young (Ed.), Functions of the right cerebral hemisphere. London, Academic Press. Jasper, H. H. 1958. The ten twenty system electrode system of the International Federation. Electroencephalography
and
Clinical
Neurophysiolology,
10, 371-375.
Kutas, M., & Hillyard, S. A. 1980. Reading senseless sentences: Brain potentials reflect semantic incongruity. Scienae, 207, 203-205. Kutas, M., & Hillyard, S. A. 1984. Brain potentials during reading reflect word expectancy and semantic association. Nature(London), 307, 161-163. Kutas, M., Van Petten, C., & Besson, M. 1988. Event-related potential asymmetries during the reading of sentences. Electroencephalography and Clinical Neurophysiology, 69, 218-233. Patterson, K., & Besner, D. 1984. Is the right hemisphere literate? Cognitive Neuropsychology,
1, 315-341.
ERPS AND PICTURE-MATCHING Potter, M. C., & Faulconer, ture(London),
B. A. 1975. Time to understand pictures and words. Na-
253, 437-438.
Rugg, M. D. 1984a. Event-related Language,
431
potentials in phonological
matching tasks. Brain
and
23, 225-240.
Rugg, M. D. 1984b. Event-related potentials and the phonological processing of words and non-words. Neuropsychologia, 22, 435-443. Rugg, M. D. 1985a. The effects of handedness on event-related potentials in a rhymematching task. Neuropsychologiu, 23, 765-775. Rugg, M. D. 1985b. The effects of semantic priming and word repetition on event-related potentials. Psychophysiology, 22, 642-647. Rugg, M. D., & Barrett, S. E. 1987. Event-related potentials and the interaction between orthographic and phonological information in a rhyme-judgment task. Brain and Language, 32, 336-361. Rugg, M. D., Kok, A., Barrett, G., & Fischler, I. 1986. ERPs associated with language and hemispheric specialization. In W. C. McCallum, R. Zappoli, and F. Denoth (Eds.), Cerebral psychophysiology: Studies in event-related potentials. Amsterdam: Elsevier, 1986. Seidenberg, M. S., & Tanenhaus, M. K. 1979. Orthographic effects on rhyme monitoring. Journal of Experimental Psychology: Human Learning and Memory, 5, 546-554. Springer, S. P., & Deutsch, G. 1985. Left brain, right brain. New York: W.H. Freeman. 2nd ed. Young, A. W., McWeeny, K. H., Ellis, A. W., & Hay, D. C. 1986. Naming and categorising faces and written names. Quarterly Journal of Experimental Psychology A, 38, 297318. Zaidel, E., & Peters, A. M. 1981. Phonological encoding and ideographic reading by the disconnected right hemisphere: Two case studies. Brain and Language, 14, 205-234.