Deriving meaning: Distinct neural mechanisms for metaphoric, literal, and non-meaningful sentences

Brain and Language 100 (2007) 150–162 www.elsevier.com/locate/b&l

Deriving meaning: Distinct neural mechanisms for metaphoric, literal, and non-meaningful sentences Argyris K. Stringaris a,*,1, Nicholas C. Medford a,1, Vincent Giampietro b, Michael J. Brammer b, Anthony S. David a a

Section of Cognitive Neuropsychiatry, Institute of Psychiatry, KingÕs College London, Denmark Hill, London SE5 8AF, UK b Brain Image Analysis Unit, Institute of Psychiatry, KingÕs College London, Denmark Hill, London SE5 8AF, UK Accepted 1 August 2005 Available online 13 September 2005

Abstract In this study, we used a novel cognitive paradigm and event-related functional magnetic resonance imaging (ER-fMRI) to investigate the neural substrates involved in processing three different types of sentences. Participants read either metaphoric (Some surgeons are butchers), literal (Some surgeons are fathers), or non-meaningful sentences (Some surgeons are shelves) and had to decide whether they made sense or not. We demonstrate that processing of the different sentence types relied on distinct neural mechanisms. Activation of the left inferior frontal gyrus (LIFG), BA 47, was shared by both non-meaningful and metaphoric sentences but not by literal sentences. Furthermore, activation of the left thalamus appeared to be specifically involved in deriving meaning from metaphoric sentences despite lack of reaction times differences between literals and metaphors. We assign this to the ad hoc concept construction and open-endedness of metaphoric interpretation. In contrast to previous studies, our results do not support the view the right hemispheric is specifically involved in metaphor comprehension. Ó 2005 Elsevier Inc. All rights reserved. Keywords: Metaphor; Figurative; Literal; Meaningfulness; Thalamus; Frontal cortex; Parietal cortex; Right hemisphere

1. Introduction What does it take to decide whether or not a statement makes sense? Are different neural mechanisms required for understanding different types of sentences? Literal sentences (such as ‘‘some men are young’’) may be regarded as prototypical meaningful utterances. A standard view, dating back to antiquity, states that when reading a sentence, the first attempt is at extracting a literal meaning and if this is found to be defective, the reader proceeds to consider alternative interpretations rooted in metaphor, humour or irony (Aristotle, 1952; Grice, *

Corresponding author. Fax: +44 020 7 848 0572. E-mail address: [email protected] (A.K. Stringaris). 1 Both authors contributed equally to the accomplishment of this work. 0093-934X/$ - see front matter Ó 2005 Elsevier Inc. All rights reserved. doi:10.1016/j.bandl.2005.08.001

1975; Searle, 1993). According to this view, a sentence of the type ‘‘Some men are lions’’ would first be recognized as literally false before its metaphoric meaning was derived. This ‘‘primacy of the literal’’ doctrine implies an additional computational burden for the interpretation of figurative and ironic phrases. However, metaphoric, idiomatic, and ironic utterances are abundant in every day discourse and are readily understood by listeners. Indeed, empirical research on figurative language over the last few decades has found that literal and figurative phrases follow the same time-course and that readers cannot ignore figurative meaning in favour of a literal interpretation (Glucksberg, Gildea, & Bookin, 1982; McElree & Nordlie, 1999). This is true even for relatively unfamiliar metaphors, and appears to be dependent on the aptness of the figurative expression and the relative metaphoric salience of words involved (Blasko

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

& Connine, 1993; Giora, 1999, 2003). These results have led to the currently prevailing view that metaphorical meanings are understood as automatically as their literal counterparts (Glucksberg, 2003). Equivalence at the behavioural level may conceal differences at the neural processing level and it is in such circumstances that knowledge of functional neuroanatomy can make a significant contribution to cognitive science. There is considerable debate surrounding the brain localization of figurative language processing. In early lesion studies, Winner and Gardner (1977), using a picture matching task, showed that patients with right hemispheric (RH) damage had more deficits in processing metaphors than those with left hemispheric lesions. In addition, Brownell, Simpson, Bihrle, Potter, and Gardner (1990) suggested a special role for the right hemisphere in lexical semantic processes related to metaphor comprehension. However, in Winner and GardnerÕs (1977) study, when the same patients with right hemisphere damage were asked to explicate the meanings of metaphors in a verbal task they performed correctly. In keeping with this finding, Rinaldi, Marangolo, and Baldassari (2005) reported that right-hemisphere damaged patients tended to inappropriately select literal over metaphoric meanings only in a picture matching task but not in a verbal task. In addition, recent findings have challenged the notion that RH damage selectively impairs understanding of verbal figurative language (Gagnon, Goulet, Giroux, & Joanette, 2003; Giora, Zaidel, Soroker, Batori, & Kasher, 2000; Tompkins, Boada, & McGarry, 1992). It is possible that methodological issues related to matching of stimuli across experimental conditions and additional visuo-perceptual deficits of RH patients could account for the differences observed. A study employing lateralized presentation of metaphoric and literal targets has also suggested an enhanced role of the RH in metaphor comprehension (Anaki, Faust, & Kravetz, 1998), although a further study has found that hemispheric lateralization of metaphors is related to the type of task used, e.g., whether the task involves the presentation of single words, as opposed to whole sentences (Faust & Weisper, 2000; see also Papagno and Caporali, this volume). In addition, recent findings from repetitive transcranial magnetic stimulation (rTMS) experiments indicate that left, rather than right, temporal lobe stimulation affected comprehension of idioms (Papagno, Oliveri, & Romero, 2002). Conversely, a PET study conducted with 6 subjects (Bottini et al., 1994) demonstrated extensive right hemispheric involvement for the interpretation of figurative as opposed to literal sentences. However, this study utilized complex sentences, which were not well matched across the two conditions. Subjects in that study made significantly more mistakes interpreting metaphoric compared to literal sentences. A more recent event-related functional magnetic resonance

151

(ER-fMRI) study indicated that reading metaphoric sentences, in contrast to carefully matched literal sentences, lead to increased activation in left frontal and temporal brain regions (Rapp, Leube, Erb, Grodd, & Kircher, 2004). Similarly, a study by Lee and Dapretto (manuscript submitted) using fMRI demonstrates that processing of metaphoric semantic relationships mainly lead to activation of left rather than right prefrontal and temporoparietal regions. The authors argue that previous findings suggesting a selective role of the right hemisphere in comprehension of figurative language, might be reflective of task complexity and not specific to processing of metaphors. Although at first sight, semantic decisions appear to be binary, i.e., a given utterance can either make sense or not make sense, it has been suggested that there are significant qualitative differences in the extraction of meaning from different types of sentences. For example, despite the functional equivalence of metaphoric and literal sentences, as witnessed by equal reaction times between the two conditions, current cognitive models suggest differences in the processing requirements between literal and figurative language. One of the most influential such models treats metaphors as attributive assertions: in a sentence such as ‘‘My job is a jail,’’ ‘‘job’’ and ‘‘jail’’ are members of a common attributive category of unpleasant and confining situations (Glucksberg & Keysar, 1993). It is claimed that metaphoric expressions differ from similes in that they have more expressive force (Glucksberg, 2003). Furthermore, it has been claimed that metaphors differ from literal sentences in that they are ‘‘open ended’’ (Black, 1993; Boyd, 1993), implying that their meaning is less well circumscribed and more flexible, and that they are not readily paraphrasable into literal expressions (Searle, 1993). Indeed, in Gibbs (1992), idiomatic expressions, while being understood more quickly than their literal interpretations, involved a wider range of entailments. We predicted that differences in the quality of meaning of sentences would be reflected in their respective brain representations and that fMRI would enable us to highlight this distinction, even when traditional behavioural measures of performance were similar. For this purpose, we compared two of the most commonly occurring sentence types, literal and metaphoric, and contrasted them to non-meaningful sentences. Subjects were asked to decide whether sentences presented sequentially ‘‘made sense or not’’ and indicate their decision by a button press (for a similar task, see Gernsbacher, Keysar, Robertson, & Werner, 2001). Reaction times and BOLD-signal changes were measured. In contrast to a recent imaging study on metaphors, where subjects were asked to judge on the affective salience of figurative as opposed to literal sentences (Rapp et al., 2004), our task was more explicit, focussing on the attempt to extract meaning from sentences. Further-

152

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

more, our task was complemented by the presence of non-meaningful phrases. The advantage of this design is twofold. First, it introduces more ambiguity to the task, prompting subjects to distinguish between different sentence types strictly on the basis of semantics. Second, it allows for contrasts between successful and unsuccessful quests to comprehend a sentence. By using ERfMRI, we were able to present stimuli in random order and thus rule out anticipation effects, and also exclude subjectsÕ errors from our imaging analysis. Our main hypothesis was that although processing of metaphoric sentences compared to their literal counterparts would not differ in terms of reaction times, it would nevertheless involve increased activation of brain areas implicated in processing of linguistic information and retrieval of semantic knowledge in particular (Duncan & Owen, 2000; Thompson-Schill, DÕEsposito, Aguirre, & Farah, 1997). We therefore predicted that areas of the left inferior frontal gyrus (LIFG) would show increased activation for the processing of metaphoric, when compared to literal, statements. Furthermore, we assumed that judging a syntactically intact sentence to be non-meaningful would also require increased effort and access to semantic stores, and this would also be reflected by activation in the LIFG when compared to literal statements. In addition, we hypothesized that, since processing of metaphoric statements is thought to require recognition of common attributive categories, this would involve a neural network allowing for the integration of multisensory input. Also, based on previous experimental evidence (Gagnon et al., 2003; Papagno et al., 2002; Rapp et al., 2004), we expected that matching of stimuli across conditions with respect to basic linguistic parameters would confirm that differential activation of the right hemisphere is not involved in metaphor comprehension.

2. Methods 2.1. Participants For the fMRI study, the participants were 11 self-designated right-handed male volunteers with no history of psychiatric or neurological illnesses who were native English speakers.2 All provided written informed consent in accordance with procedures laid down by the local research ethics committee. Mean age was 33.3 years (SD 8 years) and mean verbal IQ was 116 (SD 6), as 2 To avoid any potential confounding effects related to genderspecific language processing differences, this study was confined to male subjects. While there is no compelling evidence from behavioural studies to suggest that metaphor comprehension differs between male and female subjects, future functional imaging studies would be useful to address this issue.

assessed by using the National Adult Reading Test (Nelson & Willison, 1982). For the outside the scanner phase, additional 20 right-handed subjects (10 men, 10 women) were tested (mean age, 33.0 years (SD 8.5 years); mean IQ, 117, (SD 7)). 2.2. Experimental stimuli and design of the task Triplets of sentences of the form ‘‘some X are Y’’ were constructed for the three experimental conditions: literal (LIT), metaphoric (MET), and non-meaningful (NONMEAN). The stem of the sentence ‘‘Some X are. . .’’ was identical across conditions within a given triplet, the last word varying; for example a sentence stem such ‘‘Some surgeons are. . .’’ would be followed by ‘‘fathers’’ for the LIT, ‘‘butchers’’ for the MET, and ‘‘shelves’’ for the NONMEAN conditions, respectively. These last words were matched to within one letter for length and also to within one standard deviation, for the following psycholinguistic norms using the MRC Psycholinguistic database (http://www.psy.uwa.edu.au/ mrcdatabase/uwa_mrc.htm): imageability, familiarity, Kucera-Francis written frequency, and concreteness. The initial corpus of sentences consisted of 100 triplets. Following assessment of their comprehensibility, 30 sentences were selected for the outside-the-scanner version of the task, of which 25 sentences were used for the fMRI version. Construction of metaphoric sentences was based on expressions commonly used in English. Sentences were presented for a fixed duration of 1.5 s each according to a ‘‘true’’ random sequence of numbers generated from a random number service (www. random.org). Intervals between stimuli were variable following a Poisson distribution around an average interstimulus interval of 7 s. This ‘‘jitter’’ was introduced to increase trial variance and avoid concealment of signal information due to overlap of the haemodynamic response in ER-fMRI experiments (Donaldson & Buckner, 2001; Surguladze et al., 2003). During the interstimulus intervals, a fixation cross was present on the screen, which served as a baseline condition for the haemodynamic response. (Please refer to the Image analysis section for details of percentage BOLD—blood oxygenation level dependent—signal change calculations.) 2.3. Experimental procedure Subjects were given instructions prior to performing the test. They were asked to read each presented sentence silently and decide as fast and as accurately as possible whether this ‘‘made sense or not,’’ indicating their decision by pressing one of two buttons. They were advised that sentences may either be meaningful in a formal or colloquial way, or non-meaningful. They were given illustrative examples of sentences not included in

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

the study proper. For the experiments outside the scanner, subjects were seated in front of a computer screen. For experiments in the scanner, sentences were presented to the subjects through a standard mirror system of presentation and legibility of the items was ascertained prior to commencement of the task. For both the outside the scanner and fMRI versions, the button box was placed in the subjectsÕ right hand. 2.4. Analysis of behavioural data For both the fMRI and the outside the scanner versions of the task, only ‘‘correct’’ responses, i.e., the ‘‘makes sense’’ responses for metaphors and literal sentences, and the ‘‘doesnÕt make sense’’ response for the non-meaningful, were included in the analysis. Behavioural data from each subject were averaged across each condition following logarithmic transformation to deal with reaction time outliers (Ratcliff, 1993). Results across the three conditions were compared using a repeated measures ANOVA followed by the Tukey– Kramer test for multiple comparisons, and CohenÕs d effect sizes values were calculated (Cohen, 1992; Rosnow & Rosenthal, 1996; Rosnow, Rosenthal, & Rubin, 2000; Thalheimer & Cook, 2002). 2.5. Image acquisition Gradient echo echoplanar imaging (EPI) data were acquired on a GE Signa 1.5 T system (General Electric, Milwaukee WI, USA). A quadrature birdcage headcoil was used for RF transmission and reception. Hundred T2*-weighted images depicting blood oxygenation level dependent (BOLD) contrast (Ogawa, Lee, Kay, & Tank, 1990) were acquired over the entire duration of the task at each of 22 near-axial non-contiguous 5 mm thick planes parallel to the intercommissural (AC-PC) line: TE 40 ms, TR 2 s, in-plane resolution 5 mm, interslice gap 0.5 mm. This EPI dataset provided almost complete brain coverage. An inversion recovery EPI dataset was also acquired. This was a 43 near-axial slice image; with 3 mm slices and 0.3 mm slice skip parallel to the AC-PC (TE 80 ms, TI 180 ms, TR 16 s, in-plane resolution 1.5 mm). This high-resolution inversion recovery EPI gives excellent soft tissue to CSF contrast for a template image onto which the lower-resolution functional data were mapped. The IR-EPI template has the same bandwidth as the low-resolution functional scans to avoid any mismatching of functional to anatomical data as they both have the same inherent geometric distortion. 2.6. Individual brain activation maps Data were analyzed with software developed at the Institute of Psychiatry, KingÕs College London, using a non-parametric approach. Data were first processed

153

(Bullmore et al., 1999a) to minimize motion-related artefacts. A 3D volume consisting of the average intensity at each voxel over the whole experiment was calculated and used as a template. The 3D image volume at each time-point was then realigned to this template by computing the combination of rotations (around the x, y, and z axes) and translations (in x, y, and z) that maximized the correlation between the image intensities of the volume in question and the template. Following realignment, data were then smoothed using a Gaussian filter (FWHM 7.2 mm) to improve the signal to noise characteristics of the images. Responses to the experimental paradigms were then detected by first convolving each component of the experimental design with each of two gamma variate functions (peak responses at 4 and 8 s, respectively). The best fit between the weighted sum of these convolutions and the time series at each voxel was computed using the constrained BOLD effect model suggested by Friman, Borga, Lundberg, and Knuttson (2003). This reduces the possibility of the model fitting procedure giving rise to mathematically plausible but physiologically implausible results. Following computation of the model fit, a goodness of fit statistic was computed. This consisted of the ratio of the sum of squares of deviations from the mean image intensity (over the whole time series) due to the model to the sum of squares of deviations due to the residuals (SSQratio). This statistic is used to overcome the problem inherent in the use of the F (variance ratio) statistic that the residual degrees of freedom are often unknown in fMRI time series due to the presence of coloured noise in the signal. Following computation of the observed SSQratio at each voxel, the data are permuted by the wavelet-based method described and extensively characterized in Bullmore et al. (2001). Repeated application of this method at each voxel followed by recomputation of the SSQratio from the permuted data allows (by combination of results over all intracerebral voxels) the data-driven calculation of the null distribution of SSQratios under the assumption of no experimentally determined response. Using this distribution, it is possible to calculate the critical value of SSQratio needed to threshold the maps at any desired type I error rate. The detection of activated voxels is extended from voxel to cluster level using the method described in detail by Bullmore et al. (1999b). In addition to the SSQratio, the size of the BOLD response to each experimental condition is computed for each individual at each voxel as a percentage of the mean resting image intensity level. To calculate the BOLD effect size, the difference between the maximum and minimum values of the fitted model for each condition is expressed as a percentage of the mean image intensity level over the whole time series.

154

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

2.7. Group maps The observed and permuted SSQratio maps for each individual, as well as the BOLD effect size maps are transformed into the standard space of Talairach and Tournoux (1988) using the two stage warping procedure described in detail in Brammer et al. (1997). This involves first computing the average image intensity map for each individual over the course of the experiment. The transformations required to map this image to the structural scan for each individual in the first instance and then from structural space to the Talairach template are subsequently computed by maximizing the correlation between the images at each stage. The SSQratio and BOLD effect size maps are then transformed into Talairach space, using these transformations. Group activation maps are then computed by determining the median SSQratio at each voxel (over all individuals) in the observed and permuted data maps (medians are used to minimize outlier effects). The distribution of median SSQratios over all intracerebral voxels from the permuted data is then used to derive the null distribution of SSQratios and this can be thresholded to produce group activation maps at any desired voxel or cluster-level type I error rate. Cluster level maps are thresholded at <1 expected type I error cluster per brain. The computation of a standardized measure of effect SSQratio at the individual level, followed by analysis of the median SSQratio maps over all individuals treats intra and inter subject variations in effect separately, constituting a mixed effect approach to the analysis, which is deemed desirable in fMRI. 2.8. Sensitivity of detection of fMRI responses To assess the ability of the above analysis software to detect activations, an extension of the technique previously described by Desco, Hernandez, Santos, and Brammer (2001) was used. This involved embedding artificial activations in resting state fMRI data. Artificial fMRI responses were produced using the Balloon model described by Buxton, Wong, and Frank (1998) in the region of the hippocampus (bilateral), extrastriate visual cortex (bilateral), left inferior frontal cortex, and anterior cingulate gyrus. The decision to embed activations using a physiological model and analyze using a pair of gamma variate functions was taken to bias detection excessively by using the same method for embedding and analysis. Combinations of gamma functions are commonly used to model BOLD effects in fMRI analysis. The activation sizes simulated (spatial extents) were comparable with those commonly detected in these regions in fMRI experiments on encoding recall, motion perception and verbal fluency (500–1000 mm3). BOLD effect sizes of up to 1% were simulated with block designs (10 alternating on/off blocks of 10 images each)

and randomized event-related designs with 10 or 50 events per experiment. Activations were embedded in the raw fMRI data for 6 subjects with reference to the available anatomy of the images, and data were then processed through the individual and group analysis steps described above. The threshold for detection of responses for all designs occurred with a BOLD effect of 0.1–0.15%. With a 1% effect size, approximately 90% of the embedded network was detected with the block design, 70% with 50 trials and 50% with 10 trials in the event-related simulations. At an effect size of 0.5%, these figures fell to 65% (block), 40% (50 events), and 20% (10 events). 2.9. Group comparisons Comparisons of responses between experimental conditions are performed by fitting the data at each intracerebral voxel at which all subjects have non-zero data using a linear model of the type Y ¼ a þ bX þ e; where Y is the vector of BOLD effect sizes for each individual, X is the contrast matrix for the particular intercondition contrasts required, a is the mean effect across all individuals in the various conditions, b is the computed condition difference, and e is a vector of residual errors. The model is fitted by minimizing the sum of absolute deviations rather than the sums of squares to reduce outlier effects. The null distribution of b is computed by permuting data between conditions/groups (assuming the null hypothesis of no effect of experimental condition) and refitting the above model. Group difference maps are computed as described above at voxel or cluster level by appropriate thresholding of the null distribution of b. In this paper, BOLD effect maps were used to compute significant condition differences rather than standardized measures, such as SSQratio, F or t, as these contain explicit noise components (error SSQ or error variance), raising the possibility that group differences resulting from F, SSQratio or t comparisons could reflect differences in noise rather then signal.

3. Results 3.1. Behavioural data Behavioural results from the outside the scanner and in the scanner experiments are summarized in Table 1 and highlighted in Fig. 1. In brief, reaction times for metaphoric and literal sentences did not show a statistically significant difference (p > .05). Conversely, reaction times for comprehension of non-meaningful sentences differed significantly compared to both literal and metaphoric sentences (p < .001 and p < .05, respectively).

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

155

Table 1 Mean age, mean verbal IQ, reaction times (RT) with respective standard deviations (SD) in milliseconds and logarithmically transformed RT (log RT) and accuracy data of subjects Offline

Age

VIQ

n = 20

33

117

Online

Age

VIQ

n = 11

32

116

Mean RT

SD

log Mean RT

log SD

Accuracy (%)

ANOVA

F(2/19) = 12.069

p = .001

MET LIT NONMEAN

1229.9 1149.7 1308.8

338.7 324.4 379.5

3.0605 3.0329 3.0903

0.1084 0.1082 0.1173

74.2 92 90

MET vs LIT MET vs NONMEAN LIT vs NONMEAN

d = 0.25 d = 0.2639 d = 0.5

p > .05 p < .05 p < .001

SD

log Mean RT 3.1387 3.1259 3.1783

log SD

Accuracy (%) 78 98 89

ANOVA

F(2/20) = 4.262

p = .0395

MET LIT NONMEAN

Mean RT 1460.2 1392.3 1576.7

MET vs LIT MET vs NONMEAN LIT vs NONMEAN

d = 0.25 d = 0.34 d = .625

p > .05 p > .05 p < .05

237 265 358

0.075 0.079 0.092

d Stands for effect size, where values of 0.20 are considered small; 0.50 medium; and 0.80 large (Cohen, 1992); statistical significance is reached when p < .05.

A

B

Fig. 1. Behavioural results from experiments performed outside the scanner (A, n = 20) and during fMRI (B, n = 12). Logarithmically transformed reaction times (log RT) and respective standard deviations are depicted.

3.2. fMRI results

3.4. Literal sentences

An overview of all activations obtained in the contrast between conditions is provided in Table 2 and highlighted in Fig. 2.

Comprehension of literal sentences showed strong activation of the right precentral gyrus (BA 4) when contrasted to either MET or NONMEAN sentences. There was also activation of the medial prefrontal cortex (BA 11) and the right inferior frontal gyrus (RIFG; BA 44/ 45) when comprehension of literal sentences was contrasted to metaphors and non-meaningful sentences, respectively. There were also activations of the left cerebellum, right precuneus (BA 7), and the right inferior temporal gyrus next to the middle occipital gyrus (BA 37/19) when LIT were contrasted to MET. Contrasting LIT sentences with NONMEAN phrases also activated BA 21 in the left temporal lobe. Furthermore, there was activation in the occipital cortex in the LIT > NONMEAN condition.

3.3. Metaphoric sentences Comprehension of MET sentences contrasted to both LIT and NONMEAN sentences revealed activation of the left thalamus. In addition, there was activation in the left inferior frontal gyrus (LIFG) at BA 47 and in the right middle temporal gyrus (MTG) next to the middle occipital gyrus (MOG) at BA 39/19 in the MET > LIT condition. Furthermore, there was widespread activation in the supplementary motor cortex (BA 6) and the right cerebellum as well as in the primary visual cortex bilaterally for the MET > LIT comparison. Also, there was activation in the inferior parietal lobule close to the occipital cortex (BA 40/19) when metaphors were contrasted to literal sentences.

3.5. Non-meaningful sentences Contrasting the non-meaningful sentences with metaphoric and literal sentences reveals very extensive activa-

156

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

Table 2 3D cluster-based activations observed from contrasts of the three experimental conditions with respective number of voxels and probabilities (p values) Number of voxels

Tal (x)

Tal (y)

Tal (z)

Probability

BA

Side

Cerebral region

MET > LIT 12 17 10 8 10 9 6 41 7

29 14 7 43 32 21 25 51 29

59 81 15 29 67 81 66 11 59

29 13 4 2 15 20 25 36 42

0.001309 0.001047 0.002618 0.002618 0.005497 0.004974 0.008377 0.000262 0.007592

71 71 67 47 39/19 18 18 6 40/19

L R L L R L R L L

Cerebellum Cerebellum Thalamus Inferior frontal gyrus Middle temporal gyrus/middle occipital gyrus Primary visual (peristriate) cortex (V2,V3) Primary visual (peristriate) cortex (V2,V3) Precentral gyrus Inferior parietal lobe

7

15

4

0.007790

67

L

Thalamus

18 18

L L R L R L R L L R R L R L R

Fusiform gyrus Fusiform gyrus Cerebellum Inferior frontal gyrus (BrocaÕs) Posterior cingulate gyrus Cuneus Inferior frontal gyrus (BrocaÕs) Inferior frontal gyrus (BrocaÕs) Cuneus Inferior temporal gyrus/middle occipital gyrus Precuneus Precuneus Inferior parietal lobule Precuneus Lobus paracentralis Medial prefrontal cortex GFd Inferior parietal lobule

MET > NONMEAN 13 NONMEAN > MET 10 10 26 26 9 28 13 5 10 45 16 7 30 50 13 55 99

25 40 43 32 14 11 47 43 14 47 11 26 32 4 11 0.00 40

85 70 59 26 52 77 4 18 74 63 63 44 52 56 33 7 26

13 13 18 7 9 9 20 20 31 7 20 37 42 37 48 48 42

0.001778 0.002890 0.000222 0.000222 0.002445 0.000222 0.003112 0.005335 0.004001 0.000222 0.000222 0.007335 0.000222 0.000222 0.001556 0.000222 0.000222

47 29 17 44 44 19 37/19 31 31 40 23 5 6 40

LIT > MET 10 17 6 10 12 17 24

32 22 0 47 29 7 40

63 78 56 63 78 41 11

18 12 13 7 15 42 42

0.005159 0.002150 0.009243 0.004944 0.004084 0.001935 0.001290

71 71 11 37/19 71 7 4

R L R R

Cerebellum Cerebellum Medial prefrontal cortex GFd Inferior temporal gyrus/middle occipital gyrus Cerebellum Precuneus Precentral gyrus

LIT > NONMEAN 7 16 7 17

51 22 47 36

7 74 15 7

7 15 20 42

0.004070 0.002713 0.009722 0.004296

21 18 44/45 4

L L R R

Middle temporal gyrus Cuneus Inferior frontal gyrus (BrocaÕs) Precentral gyrus

NONMEAN > LIT 14 15 15 8 7 18 41 9 10 22 30 11 133

18 40 32 32 47 22 14 54 29 29 4 0 40

56 59 18 48 52 78 67 4 56 59 7 56 30

40 18 2 2 4 9 20 20 37 42 48 53 48

0.002665 0.000485 0.000242 0.004846 0.006058 0.000485 0.000242 0.000727 0.001212 0.000727 0.000242 0.004604 0.000242

71 71 47 10 37 17 31 4 39/19 40/19 6 18 40

L R L R L L R R L R L

Cerebellum Cerebellum Inferior frontal gyrus Middle frontal gyrus Middle temporal gyrus Cuneus Precuneus Precentral gyrus Angular gyrus Inferior parietal lobe Middle frontal gyrus Precuneus Lobus parietalis inferior

BA stands for Broadman area.

L L L

L

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

157

Fig. 2. (A) Activation of the left thalamus for metaphors. Top panel shows coronal (left) and transversal (right) section images observed in the MET > LIT contrast (z coordinate = 4). Lower panel shows left thalamic activation in the MET > NONMEAN contrast (z coordinate = 4). Note that activation of the left IFG is only observed when metaphors are contrasted with literal sentences but not with the non-meaningful (top panel). (B) Activations for non-meaningful sentences contrasted to literal sentences. Axial images (left) show BOLD signal increases in the anterior cingulate, paracentral lobule, left precuneus, and regions of the inferior parietal lobule bilaterally (z coordinate = 42).

tion in areas of the parietal cortex and precuneus bilaterally, although activation at the inferior parietal lobule was clearly more pronounced in the left hemisphere. In addition, activation of BA 47 in the LIFG was observed for NONMEAN sentences when contrasted to either LIT or MET phrases. A further activation of BA 6 in the left middle prefrontal gyrus was observed for NONMEAN sentences contrasted to either MET or LIT sentences. Processing of NONMEAN sentences led to activation of the RIFG (BA 44/45) when contrasted to MET and to activation of BA 10 in the right middle frontal gyrus when contrasted to LIT sentences. Activations common to both the NONMEAN > LIT and NONMEAN > MET conditions included the right and left cerebellum and parts of the occipital cortex. Also, there was extensive activation in the right inferior temporal gyrus next to the middle occipital gyrus (BA 37/19) for the comparison between NONMEAN and MET sentences. Activation of the left middle temporal cortex (BA 37) was seen when NONMEAN were contrasted to LIT sentences.

4. Discussion The results presented here support our prediction that grasping the meaning of utterances engages different brain areas depending on the type of sentence involved. Contrasting two of the most commonly

occurring sentence types, LIT and MET, shows that, in addition to increased recruitment from areas classically involved in semantic processing, i.e., BA 47 in the LIFG, the left thalamus is involved in the processing of metaphors. Furthermore, judging a sentence to be non-meaningful requires significantly more processing time, and involves an extensive parietal cortical network in addition to the activation of areas implicated in conflict monitoring and decision-making. Activation in BA 47 of the LIFG was observed in both MET and NONMEAN sentences when contrasted to LIT sentences, but was also apparent in the NONMEAN > MET contrast. This finding is in accordance with previous findings suggesting a relative specificity of BA 47 for semantic tasks (Costafreda, Fu, Lee, Brammer, & David, 2003; McDermott, Petersen, Watson, & Ojemann, 2003). Our finding of an increased demand from the LIFG in MET and NONMEAN phrases as compared to LIT sentences may be taken to reflect a more extensive search for semantic information, in keeping with the hypothesis that the LIFG mediates retrieval and/or selection of semantic knowledge (Duncan & Owen, 2000; Fiez, Petersen, Cheney, & Raichle, 1992; Kapur et al., 1994; Thompson-Schill et al., 1997). Alternatively, this finding might be attributable to an increased demand for control during retrieval as suggested by Wagner, Pare-Blagoev, Clark, and Poldrack (2001). Either way, the finding of increased activation in the LIFG for both MET and NONMEAN as opposed to LIT sentences

158

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

suggests that additional semantic processing capacities were required in the former. This is analogous to experimental tasks involving semantically incongruent versus congruent word presentation following a priming clause; indeed in an ER-fMRI study, Kiehl, Laurens, and Liddle (2002) have shown that, in a semantic task known to elicit an ERP-response at 400 ms post-stimulus (N400), activation of bilateral inferior frontal gyrus was significantly stronger for incongruous sentence-terminating words. Interestingly, metaphoric sentences have also been shown to elicit larger N400 responses when compared to literal sentences (Coulson & Van Petten, 2002; Pynte, Besson, Robichon, & Poli, 1996) Therefore, it is conceivable that the observed activation in the LIFG reflects an attempt to resolve semantically ‘‘unexpected’’ sentences by controlled retrieval from semantic stores. The gradation of the LIFG response, being stronger for NONMEAN than meaningful MET sentences would then reflect the degree of conflict and increased demand for retrieval for NONMEAN phrases. This is supported by our finding that anterior cingulate and middle prefrontal cortex, known to be associated with conflict monitoring (Bush, Luu, & Posner, 2000; Kerns et al., 2004), were significantly active only in the NONMEAN > MET and NONMEAN > LIT comparisons. Our data demonstrate that processing of NONMEAN sentences compared to both MET and LIT leads to significantly stronger activation in the inferior parietal lobule and the precuneus bilaterally. We propose that activation in these brain areas reflects the attempt to extract potential meaning from the NONMEAN sentences by deploying an extensive search mechanism. Parietal areas, in particular the precuneus, have been consistently implicated in memory-related imagery (Bottini et al., 1994; Kosslyn, 1994; Fletcher et al., 1995; Fletcher, Shallice, Frith, Frackowiak, & Dolan, 1996). In addition, recent findings indicate that evocation of imagery is an important component of sentence understanding and deductive reasoning (Just, Newman, Keller, McEleney, & Carpenter, 2004; Knauff, Mulack, Kassubek, Salih, & Greenlee, 2002), and that parietal lobe activation is observed during transitive inference (Acuna, Eliassen, Donoghue, & Sanes, 2002). It is possible that evocation of imagery and spatial representations are an important part of the attempt to arrive at the meaning of sentences that are not readily understood, such as our NONMEAN set of sentences in this experiment. Alternatively, the activations observed in NONMEAN could be interpreted as the result of the very inability to arrive at meaning rather than the search for it. In both cases, it appears that attempting to comprehend semantically anomalous sentences involves a strategy which probably entails comparing sentence clues against existing information that includes spatial and image-based representations. However, we

also observed activation of the right precuneus in the LIT > MET condition, which is not readily accounted for by imageability. The most striking finding is the differentially increased activation observed in the thalamus during comprehension of MET sentences. A role for the thalamus in language is suggested by data from patients with thalamic lesions and from electrophysiological studies (Johnson & Ojemann, 2000; Lhermitte, 1984; Ojemann & Ward, 1971; Ojemann, Fedio, & Van Buren, 1968). Vascular lesions of the thalamus in the dominant hemisphere have been shown consistently to produce language deficits, most notably anomia, a finding that has been interpreted as reflecting damage to networks involved in attentional gating and working memory (Nadeau & Crosson, 1997; Schmahmann, 2003). While one might suggest that reading metaphoric as opposed to literal sentences requires increased recruitment of attention, this would fail to explain why there is also increased thalamic activation in MET compared to NONMEAN sentences, particularly given the behavioural data obtained here suggest NONMEAN sentences to have been the most difficult to process. We propose that instead of reflecting increased linguistic demand, thalamic activation observed in our study, is a specific feature of the processes involved in identifying attributive categories in the semantic network. Interpretation of a sentence such as ‘‘some men are lions,’’ involves the identification of an emerging object in the semantic system, probably in terms of the class inclusion assertions proposed by Glucksberg and Keysar (1993), which accommodates features of both the words ‘‘men’’ and ‘‘lions,’’ e.g., ‘‘courageous person.’’ This implies that the association of the two words is made as a holistic conjunction of the constituent terms, thus yielding a newly constructed, ad hoc representation (see also Asch & Ebenholtz, 1962; Gernsbacher et al., 2001; Kahana, 2002). In contrast, the interpretation of a corresponding literal sentence, e.g., ‘‘Some men are soldiers’’ does not necessitate resorting to a newly constructed, ad hoc concept, but can be resolved compositionally by simply juxtaposing the two words ‘‘men’’ and ‘‘soldiers.’’ In this sense, the type of association made during the interpretation of metaphors is close to the concept of non-compositional representations as suggested by Fodor and Pylyshyn (1988). A prototypical example of this model is the association made between words such as ‘‘computer’’ and ‘‘virus,’’ which would fuse to produce ‘‘computer virus,’’ a novel semantic object. In contrast, association by juxtaposition is performed in the case of word pairs such as ‘‘salt’’ and ‘‘pepper,’’ which usually requires no fusion of the constituent terms. Recent empirical data demonstrate that the brain representations of compositional and non-compositional associations differ (Kounios, Smith, Yang, Bachman, & DÕEsposito, 2001). In an fMRI study addressing this issue, Kraut et al. (2002a)

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

have shown that activation of the left thalamus is specific for a task requiring non-compositional fusion, but not for tasks that involve compositional association or the identification of superordinate categories. The authors have also shown thalamic activation to be present in a similar task of non-compositional fusion where multimodal feature stimuli (picture–word associations) were used (Kraut et al., 2002b). In addition, simultaneous recordings from thalamic and scalp electrodes in a human, during a task requiring non-compositional associations, demonstrated functional interactions between thalamic and cortical rhythms, suggesting a neural network underlying semantic recall (Slotnick, Moo, Kraut, Lesser, & Hart, 2002). Taken together these data imply a significant role of the thalamus during non-compositional associations, although the precise mechanism by which this is accomplished remains unclear. It is obvious that the neuroanatomical location of the thalamus and the extensive thalamocortical, interthalamic, and corticothalamic connections plays an important part in this process. The thalamus, as a relay station, probably acts not only to coordinate but also to modulate cortical processing (Guillery & Sherman, 2002). It has also been argued that fast activities in thalamocortical/ corticothalamic connections allow for the interaction between remote cortical regions (Lumer, Edelman, & Tononi, 1997a, 1997b; Steriade, 2000). Such processes would allow for the integration of activities in multiple cortical areas during semantic recall in the MET condition. Activations were observed for all three conditions in the right temporal cortex. BOLD signal increase in the middle temporal gyrus close to the middle occipital lobe (BA 39/19) was observed in the MET > LIT contrast, whereas an activation in the right temporo-occipital junction (BA 37/19) was observed in the NONMEAN > MET and LIT > MET contrasts. It is possible that activations in this area reflect attempts for integration of audiovisual information during comprehension (Calvert, Campbell, & Brammer, 2000; Giraud & Truy, 2002), although it remains unclear why the corresponding left hemispheric areas do not show significant activity in our study. However, there was significant activation of the left middle temporal gyrus (BA 21), confined only to the comprehension of LIT sentences compared to NONMEAN. A number of studies suggest that this region is important for speech comprehension (Giraud et al., 2004; Narain et al., 2003). Some lesion studies (Winner & Gardner, 1977) and a previous neuroimaging study (Bottini et al., 1994) have suggested that comprehension of figurative language, as opposed to literal language, is dependent on the right hemisphere. However, studies on processing of idioms (Papagno et al., 2002), lesion studies (Gagnon et al., 2003; Giora et al., 2000), and two recent imaging study (Lee and Dapretto, manuscript submitted; Rapp et al.,

159

2004) on metaphors have disputed this claim. Our study also failed to find evidence supporting a predominant role of right hemispheric structures for the comprehension of figurative language. In keeping with recent studies, we also show that the LIFG is significantly more active during processing of metaphors. Recent findings support the view that rather than metaphoric language being processed in the right hemisphere, it is non-salient meanings that place an increased demand on the RH (Giora et al., 2000; Mashal, Faulst, Hendler & JungBeeman, 2007). It is, therefore, possible that the hemispheric asymmetries found in previous studies of metaphors, reflect insufficient matching of stimuli salience across conditions rather than a basic difference between metaphoric and literal sentence processing. In our study, we observed activation of the right inferior frontal gyrus (BA 44/45) in the LIT > NONMEAN contrast, but not in the LIT > MET contrast. We hypothesize that this finding may be due to the need to compare a sentence to immediate context (Caplan & Dapretto, 2001). Lack of differential activation of the RIFG in the LIT > MET comparison also argues against its specificity for either the MET or LIT condition. We also found activation in the right precentral/premotor cortex (BA 4/6) for the LIT > MET and LIT > NONMEAN comparisons. This may be seen as surprising, given that our subjects used their right hand to indicate their response, however we note that similar activations of the ipsilateral premotor areas have been previously reported during sentence processing (Friederici, Ruschemeyer, Hahne, & Fiebach, 2003). Taken together, our data indicate that grasping the meaning of a sentence, far from being a unitary process, depends on the type of sentence involved. In addition, our data raise a further, more fundamental issue, namely the dissociation between behavioural and imaging results. In this study, sentences were carefully matched for the MET and LIT condition, and as we had predicted and as was to be expected from the relevant literature, reaction times between these two conditions were not significantly different. However, the imaging results indicate that the neural substrates underlying metaphoric and literal sentence processing do differ. How can this apparent antinomy be resolved? Clearly, behavioural experiments and fMRI scanning measure different things. It could be argued that reaction times from behavioural experiments represent a very robust, albeit crude measure of cognitive activity, whereas fMRI provides insight into the neuroanatomical correlates of such activity. In other words, the finding that two cognitive tasks have similar reaction times does not necessarily imply that they are served by common brain pathways. However, equivalence at the behavioural level is useful to minimize the confounding effects of relative difficulty for each given task. Of particular importance for the present study is that reaction times can be seen as a mea-

160

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

sure of relative salience, as has been suggested in the past (Giora, 2003). Therefore, it is unlikely that the differences in brain activations observed between metaphors and literal sentences in this study are effects of salience. Our study was not designed to address this issue, however, as noted above, recent imaging studies suggest that right hemispheric involvement is more extensive when processing low-salience linguistic items (Mashal et al., 2005). An important aspect of the current study is that it was specifically designed to examine brain mechanisms involved during judgements of meaningfulness, as opposed to implicit processing of metaphoric sentences. Also, by excluding erroneous responses, we were able to concentrate on differences between successfully arriving at meaning as opposed to deeming a sentence as meaningful or non-meaningful. Our finding of extensive parietal activation in the NONMEAN condition supports the view that distinct prefrontal regions may dynamically and selectively interact with domain specific posterior regions (Miller, 2000). This might be seen as a representation of an effort to extract the meaning out of sentences with incongruous elements. This result is also interesting in light of recent findings from in vivo diffusion tensor tractography suggesting that frontal and temporal language areas may be interconnected through an indirect pathway involving areas of the inferior parietal cortex (Catani, Jones, & Ffytche, 2005). Importantly, present data support the view that understanding metaphors involve construction of an attributive category, a concept that resembles object activation in the semantic system. Our results are also in line with previous findings pointing to an important role of thalamic activation as an integrative and modulatory device (see Kraut et al., 2002a, 2002b). We propose that this transthalamic processing mechanism may be sufficient to explain why, despite an increased demand for semantic processing as reflected by larger N400 amplitudes and increased LIFG activation, metaphoric and literal sentences are eventually computed in equal time. Fast reciprocal connections between cortical and thalamic regions may be responsible for this (Steriade, 2000). We also assume that such a mode of processing for metaphors, depending on the fast interplay of multiple cortical regions for recruitment of information, would lead to a rich, but less circumscribed representation of meaning. Clearly this is a tentative explanation and would require further experimental work to be corroborated. Given that our finding of LIFG activation for processing of metaphors appears to be a stable result across a number of well designed studies (Ahrens et al., this volume; Lee and Dapretto, in press; Rapp et al., 2004), the question is whether the thalamus is actually required for this type of cognitive processing. Ours is the only one of these studies where metaphor processing is associated with

both LIFG and thalamus activation. We suggest that this may be an effect of methodology, as our study is the only event-related fMRI study which also involves an explicit judgement of meaningfulness; the study by Ahrens et al. (this volume) was block designed, whereas Rapp et al. (2004) employed a paradigm based on implicit linguistic processing involving judgement of emotional salience. Our finding of left thalamic activation was robust and specific to metaphors, and thus we propose that the thalamus may be part of the neural circuitry underlying the repeatedly described phenomenon of open-endedness in metaphoric meanings (Black, 1993; Boyd, 1993; Gibbs, 1992). Acknowledgments The authors are indebted to Professor Sam Glucksberg from Princeton University for his helpful comments on an earlier draft of this manuscript.

References Acuna, B. D., Eliassen, J. C., Donoghue, J. P., & Sanes, J. N. (2002). Frontal and parietal lobe activation during transitive inference in humans. Cerebral Cortex, 12, 1312–1321. Anaki, D., Faust, M., & Kravetz, S. (1998). Cerebral hemispheric asymmetries in processing lexical metaphors. Neuropsychologia, 36, 691–700. Aristotle (1952). Poetics. Translated by I. Bywater. In W. D. Ross (Ed.) The works of Aristotle (Vol. 11), Rhetorica, de rhetorika ad Alexandrum, poetical. Oxford: Clarendon Press. Asch, S. E., & Ebenholtz, S. M. (1962). The principle of associative symmetry. Proceedings of the American Philosophical Society, 106, 153–163. Black, M. (1993). More about metaphor. In A. Ortony (Ed.), Metaphor and thought (2nd ed., pp. 401–424). Cambridge, MA: Cambridge University Press. Blasko, D. G., & Connine, C. M. (1993). Effects of familiarity and aptness on metaphor processing. Journal of Experimental Psychology. Learning, Memory, and Cognition, 19, 295–308. Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P., Scarpa, P., Frackowiak, R. S., & Frith, C. D. (1994). The role of the right hemisphere in the interpretation of figurative aspects of language. A positron emission tomography activation study. Brain, 117, 1241–1253. Boyd, R. (1993). Metaphor and theory change. In A. Ortony (Ed.), Metaphor and thought (2nd ed, pp. 401–424). Cambridge, MA: Cambridge University Press. Brammer, M. J., Bullmore, E. T., Simmons, A., Williams, S. C. R., Grasby, P. M., Howard, R. J., Woodruff, P. W. R., & Rabe-Hesketh, S. (1997). Generic brain activation mapping in fMRI, a nonparametric approach. Magnetic Resonance Imaging, 15, 763–770. Brownell, H. H., Simpson, T. L., Bihrle, A. M., Potter, H. H., & Gardner, H. (1990). Appreciation of metaphoric alternative word meanings by left and right brain-damaged patients. Neuropsychologia, 28, 375–383. Bullmore, E. T., Brammer, M. J., Rabe-Hesketh, S., Curtis, V., Morris, R. G., Williams, S. C. R., Sharma, T., & McGuirre, P. K. (1999a). Methods for diagnosis and treatment of stimulus-correlated motion in generic brain activation studies using fMRI. Human Brain Mapping, 7, 38–48.

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162 Bullmore, E. T., Suckling, J., Overmeyer, S., Rabe-Hesketh, S., Taylor, E., & Brammer, M. J. (1999b). Global, voxel and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain. IEEE Transactions on Medical Imaging, 18, 32–42. Bullmore, E. T., Long, C., Suckling, J., Fadili, J., Calvert, G. A., Zelaya, F., Carpenter, T. A., & Brammer, M. J. (2001). Coloured noise and computational inference in neurophysiological (fMRI) time series analysis. Resampling methods in time and wavelet domains. Human Brain Mapping, 12, 61–78. Bush, G., Luu, P., & Posner, M. I. (2000). Cognitive and emotional influences in anterior cingulate cortex. Trends in Cognitive Science, 4, 215–222. Buxton, R. B., Wong, E. C., & Frank, L. R. (1998). Dynamics of blood flow and oxygenation changes during brain activation, the balloon model. Magnetic Resonance in Medicine, 39, 855–864. Calvert, G. A., Campbell, R., & Brammer, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10, 649–657. Caplan, R., & Dapretto, M. (2001). Making sense during conversation, an fMRI study. Neuroreport, 12, 3625–3632. Catani, M., Jones, D. K., & Ffytche, D. H. (2005). Perisylvian language networks of the human brain. Annals of Neurology, 57, 8–16. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. Costafreda, S., Fu, C. H. Y., Lee, L., Brammer, M. J., & David, A. S. (2003). A meta-analysis of fMRI studies of verbal fluency, segregation of activation within inferior frontal gyrus in healthy individuals and people with schizophrenia. Schizophrenia Research, 60(1 Supp 1), 215–216. Coulson, S., & Van Petten, C. (2002). Conceptual integration and metaphor. An event related study. Memory & Cognition, 30, 958–968. Desco, M., Hernandez, J. A., Santos, A., & Brammer, M. (2001). Multiresolution analysis in fMRI, sensitivity and specificity in the detection of brain activation. Human Brain Mapping, 14, 16–27. Donaldson, D. I., & Buckner, R. L. (2001). Effective paradigm design. In P. Jezzard, P. M. Matthews, & S. M. Smith (Eds.), Functional MRI, an introduction to methods (pp. 177–195). Oxford: Oxford University Press. Duncan, J., & Owen, A. M. (2000). Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends in Neuroscience, 23, 475–483. Faust, M., & Weisper, S. (2000). Understanding metaphoric sentences in the two cerebral hemispheres. Brain and Cognition, 43, 186–191. Fiez, J. A., Petersen, S. E., Cheney, M. K., & Raichle, M. E. (1992). Impaired non-motor learning and error detection associated with cerebellar damage. A single case study. Brain, 115(Pt 1), 155–178. Fletcher, P. C., Frith, C. D., Baker, S. C., Shallice, T., Frackowiak, R. S., & Dolan, R. J. (1995). The mindÕs eye—precuneus activation in memory-related imagery. Neuroimage, 2, 195–200. Fletcher, P. C., Shallice, T., Frith, C. D., Frackowiak, R. S., & Dolan, R. J. (1996). Brain activity during memory retrieval. The influence of imagery and semantic cueing. Brain, 119, 1587–1596. Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture. A critical analysis. Cognition, 18, 3–71. Friederici, A. D., Ruschemeyer, S. A., Hahne, A., & Fiebach, C. (2003). The role of left inferior frontal and superior temporal cortex in sentence comprehension, localizing syntactic and semantic processes. Cerebral Cortex, 13, 170–177. Friman, O., Borga, M., Lundberg, P., & Knuttson, H. (2003). Adaptive analysis of fMRI data. Neuroimage, 19, 837–845. Gagnon, L., Goulet, P., Giroux, F., & Joanette, Y. (2003). Processing of metaphoric and non-metaphoric alternative meanings of words after right- and left-hemispheric lesion. Brain and Language, 87, 217–222.

161

Gernsbacher, M. A., Keysar, B., Robertson, R. W., & Werner, N. K. (2001). The role of suppression and enhancement in understanding metaphors. Journal of Memory and Language, 45, 433–450. Gibbs, R. (1992). What do idioms really mean?. Journal of Memory and Language 31, 485–506. Giora, R. (1999). On the priority of salient meanings, studies of literal and figurative language. Journal of Pragmatics, 31, 919–929. Giora, R. (2003). On our mind: Salience, context, and figurative language. New York: Oxford University Press. Giora, R., Zaidel, E., Soroker, N., Batori, G., & Kasher, A. (2000). Differential effects of right and left hemisphere damage on understanding sarcasm and metaphor. Metaphor and Symbol, 15, 63–83. Giraud, A. L., & Truy, E. (2002). The contribution of visual areas to speech comprehension, a PET study in cochlear implants patients and normal-hearing subjects. Neuropsychologia, 40, 1562–1569. Giraud, A. L., Kell, C., Thierfelder, C., Sterzer, P., Russ, M. O., Preibisch, C., & Kleinschmidt, A. (2004). Contributions of sensory input, auditory search and verbal comprehension to cortical activity during speech processing. Cerebral Cortex, 14, 247–255. Glucksberg, S. (2003). The psycholinguistics of metaphor. Trends in Cognitive Science, 7, 92–96. Glucksberg, S., Gildea, P., & Bookin, H. B. (1982). On understanding nonliteral speech, can people ignore metaphors? Journal of Verbal Learning & Verbal Behavior, 21, 85–98. Glucksberg, S., & Keysar, B. (1993). How metaphors work. In A. Ortony (Ed.), Metaphor and thought (2nd ed., pp. 401–424). Cambridge, MA: Cambridge University Press. Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.). Speech acts. Syntax and semantics (Vol. 3, pp. 41–58). New York: Academic Press. Guillery, R. W., & Sherman, S. M. (2002). Thalamic relay functions and their role in corticocortical communication, generalizations from the visual system. Neuron, 33, 163–175. Johnson, M. D., & Ojemann, G. A. (2000). The role of the human thalamus in language and memory, evidence from electrophysiological studies. Brain and Cognition, 42, 218–230. Just, M. A., Newman, S. D., Keller, T. A., McEleney, A., & Carpenter, P. A. (2004). Imagery in sentence comprehension, an fMRI study. Neuroimage, 21, 112–124. Kahana, M. J. (2002). Associational symmetry and memory theory. Memory & Cognition, 30, 823–840. Kapur, S., Rose, R., Liddle, P. F., Zipursky, R. B., Brown, G. M., Stuss, D., Houle, S., & Tulving, E. (1994). The role of the left prefrontal cortex in verbal processing, semantic processing or willed action? Neuroreport, 5, 2193–2196. Kerns, J. G., Cohen, J. D., MacDonald, A. W., III, Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004). Anterior cingulate conflict monitoring and adjustments in control. Science, 303, 1023–1026. Kiehl, K. A., Laurens, K. R., & Liddle, P. F. (2002). Reading anomalous sentences, an event-related fMRI study of semantic processing. Neuroimage, 17, 842–850. Knauff, M., Mulack, T., Kassubek, J., Salih, H. R., & Greenlee, M. W. (2002). Spatial imagery in deductive reasoning, a functional MRI study. Brain Research. Cognitive Brain Research, 13, 203–212. Kosslyn, S. M. (1994). Image and brain. The resolution of the imagery debate. Cambridge, MA: MIT Press. Kounios, J., Smith, R. W., Yang, W., Bachman, P., & DÕEsposito, M. (2001). Cognitive association formation in human memory revealed by spatiotemporal brain imaging. Neuron, 29, 297–306. Kraut, M. A., Kremen, S., Moo, L. R., Segal, J. B., Calhoun, V., & Hart, J. Jr., (2002a). Object activation in semantic memory from visual multimodal feature input. Journal of Cognitive Neuroscience, 14, 37–47. Kraut, M. A., Kremen, S., Segal, J. B., Calhoun, V., Moo, L. R., & Hart, J. Jr., (2002b). Object activation from features in the semantic system. Journal of Cognitive Neuroscience, 14, 24–36.

162

A.K. Stringaris et al. / Brain and Language 100 (2007) 150–162

Lee, S. S. & Dapretto, M. (in press). Metaphorical vs. Literal word meanings: fMRI evidence against a selective role of the right hemisphere. Neuroimage. Lhermitte, F. (1984). Language disorders and their relationship to thalamic lesions. Advances in Neurology, 42, 99–113. Lumer, E. D., Edelman, G. M., & Tononi, G. (1997a). Neural dynamics in a model of the thalamocortical system. II. The role of neural synchrony tested through perturbations of spike timing. Cerebral Cortex, 7, 228–236. Lumer, E. D., Edelman, G. M., & Tononi, G. (1997b). Neural dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex, 7, 207–227. Mashal, N., Faust, M., Hendler, T., & Jung-Beeman, M. (2007). An fMRI investigation of the neural correlates underlying the processing of novel metaphoric expressions. Brain and Language 100, 115–126. McDermott, K. B., Petersen, S. E., Watson, J. M., & Ojemann, J. G. (2003). A procedure for identifying regions preferentially activated by attention to semantic and phonological relations using functional magnetic resonance imaging. Neuropsychologia, 41, 293–303. McElree, B., & Nordlie, J. (1999). Literal and figurative interpretations are computed in equal time. Psychonomic Bulletin & Review, 6, 486–494. Miller, E. K. (2000). The prefrontal cortex and cognitive control. Nature Reviews. Neuroscience, 1, 59–65. Nadeau, S. E., & Crosson, B. (1997). Subcortical aphasia. Brain and Language, 58, 355–402. Narain, C., Scott, S. K., Wise, R. J., Rosen, S., Leff, A., Iversen, S. D., & Matthews, P. M. (2003). Defining a left-lateralized response specific to intelligible speech using fMRI. Cerebral Cortex, 13, 1362–1368. Nelson, H. E., & Willison, J. (1982). National adult reading test (2nd ed.). England: Nelson Publishing Co. Ogawa, S., Lee, T. M., Kay, A. R., & Tank, D. W. (1990). Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proceedings of the National Academy of Sciences of the United States of America, 87, 9868–9872. Ojemann, G. A., Fedio, P., & Van Buren, J. M. (1968). Anomia from pulvinar and subcortical parietal stimulation. Brain, 91, 99–116. Ojemann, G. A., & Ward, A. A. Jr., (1971). Speech representation in ventrolateral thalamus. Brain, 94, 669–680. Papagno, C., Oliveri, M., & Romero, L. (2002). Neural correlates of idiom comprehension. Cortex, 38, 895–898. Pynte, J., Besson, M., Robichon, F. H., & Poli, J. (1996). The timecourse of metaphor comprehension, an event-related potential study. Brain and Language, 55, 293–316. Rapp, A. M., Leube, D. T., Erb, M., Grodd, W., & Kircher, T. T. J. (2004). Neural correlates of metaphor processing. Cognitive Brain Research.

Ratcliff, R. (1993). Methods for dealings with reaction time outliers. Psychological Bulletin, 114, 510–532. Rinaldi, M. C., Marangolo, P., & Baldassari, F. (2005). Metaphor comprehension in right hemisphere damaged patients with visuoverbal and verbal material: A dissociation (re)considered. Cortex, 40, 479–490. Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternull on other peopleÕs published data, General procedures for research consumers. Psychological Methods, 1, 331–340. Rosnow, R. L., Rosenthal, R., & Rubin, D. B. (2000). Contrasts and correlations in effect-size estimation. Psychological Science, 11, 446–453. Schmahmann, J. D. (2003). Vascular syndromes of the thalamus. Stroke, 34, 2264–2278. Searle, J. R. (1993). Metaphor. In A. Ortony (Ed.), Metaphor and thought (2nd ed., pp. 83–111). Cambridge, MA: Cambridge University Press. Slotnick, S. D., Moo, L. R., Kraut, M. A., Lesser, R. P., & Hart, J. Jr., (2002). Interactions between thalamic and cortical rhythms during semantic memory recall in human. Proceedings of the National Academy of Sciences of the United States of America, 99, 6440–6443. Steriade, M. (2000). Corticothalamic resonance, states of vigilance and mentation. Neuroscience, 101, 243–276. Surguladze, S. A., Brammer, M. J., Young, A. W., Andrew, C., Travis, M. J., Williams, S. C., & Phillips, M. L. (2003). A preferential increase in the extrastriate response to signals of danger. Neuroimage, 19, 1317–1328. Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human brain. Stuttgart: Thieme. Thalheimer, W., & Cook, S. (2002) August. How to calculate effect sizes from published research articles: A simplified methodology. Retrieved February 16, 2004 from . Thompson-Schill, S. L., DÕEsposito, M., Aguirre, G. K., & Farah, M. J. (1997). Role of left inferior prefrontal cortex in retrieval of semantic knowledge, a reevaluation. Proceedings of the National Academy of Sciences of the United States of America, 94, 14792–14797. Tompkins, C. A., Boada, R., & McGarry, K. (1992). The access and processing of familiar idioms by brain-damaged and normally aging adults. Journal of Speech and Hearing Research, 35, 626–637. Wagner, A. D., Pare-Blagoev, E. J., Clark, J., & Poldrack, R. A. (2001). Recovering meaning, left prefrontal cortex guides controlled semantic retrieval. Neuron, 31, 329–338. Winner, E., & Gardner, H. (1977). The comprehension of metaphor in brain-damaged patients. Brain, 100, 717–729.