Resolving sentence ambiguity with planning and working memory resources: Evidence from fMRI

www.elsevier.com/locate/ynimg NeuroImage 37 (2007) 361 – 378

Resolving sentence ambiguity with planning and working memory resources: Evidence from fMRI Susana Novais-Santos, a James Gee, a,b Maliha Shah, c Vanessa Troiani, c Melissa Work, c and Murray Grossman c,⁎ a

Department of Bioengineering, University of Pennsylvania, USA Department of Radiology, University of Pennsylvania, USA c Department of Neurology-2 Gibson, University of Pennsylvania School of Medicine, 3400 Spruce St., Philadelphia, PA 19104-4283, USA b

Received 16 December 2006; revised 24 February 2007; accepted 14 March 2007 Available online 13 May 2007

We used functional magnetic resonance imaging (fMRI) to test competing claims about the role of executive resources during the disambiguation of a sentence featuring a temporary structural ambiguity. Written sentences with a direct object (DO) structure or a sentential complement (SC) structure were shown to 19 healthy, right-handed, young adults in a phrase-by-phrase manner. These sentences contained a main verb that is statistically more likely to be associated with a DO structure or an SC structure. Half of each type of sentence also contained an extra phrase strategically located to stress working memory prior to disambiguating the sentence. We found that sentences featuring a less consistent verb-structure mapping recruit greater dorsolateral prefrontal cortex (dlPFC) activation than sentences with a more consistent verb-structure mapping, implicating strategic on-line planning during resolution of a temporary structural ambiguity. By comparison, we observed left inferior parietal cortex (IPC) activation in sentences with an increased working memory demand compared to sentences with a low working memory load. These findings are consistent with a large-scale neural network for sentence processing that recruits distinct planning and working memory processing resources as needed to support the comprehension of sentences. Published by Elsevier Inc. Keywords: Sentence comprehension; Executive; fMRI; Prefrontal; Parietal

Introduction Unraveling the complex neural substrate underlying the human language comprehension system has been a major goal of neuroscience during the past century. Recent observations emphasize

⁎ Corresponding author. Fax: +1 215 349 8464. E-mail address: [email protected] (M. Grossman). Available online on ScienceDirect (www.sciencedirect.com). 1053-8119/$ - see front matter. Published by Elsevier Inc. doi:10.1016/j.neuroimage.2007.03.077

that executive resources play a substantial role during sentence comprehension. This has led to a two-component model of sentence processing that distinguishes between core sentence processing mechanisms and executive resources like working memory and strategic planning that support sentence comprehension (Friederici, 2002; Wingfield and Grossman, 2006). From this perspective, the core sentence processing mechanism in left peri-Sylvian cortex recruits resources such as working memory (WM), planning/ switching and selective attention as needed. These resources are supported in other, non-peri-Sylvian brain regions. The present study uses functional magnetic resonance imaging (fMRI) to examine this model during the comprehension of sentences with a temporary structural ambiguity. Functional neuroimaging studies of syntactic processing have shown activation of core peri-Sylvian language areas, namely left inferior frontal (IFC) and posterolateral temporal (PLTC) cortices, during tasks such as grammaticality judgments and answering probes about complex sentences (Ben-Shachar et al., 2004; Cooke et al., 2006; Cooke et al., 2001; Kuperberg et al., 2003; Ni et al., 2000). Experimental factors such as the modality of sentence presentation (visual or auditory) (Caplan et al., 1999; Friederici et al., 2000; Michael et al., 2001), the type of response (comprehension or expression) (Indefrey et al., 2001) and the age of the subjects (Caplan et al., 2003; Grossman et al., 2002) introduce subtle differences in the location of peak neuroanatomic activation, but over-all the results consistently emphasize the contribution of periSylvian frontal and temporal regions of the left hemisphere during sentence processing. The mental representation of an unfolding sentence appears to be constructed incrementally, moreover. As each successive word is encountered in this first-pass analysis, it is rapidly integrated into an evolving interpretation. Some have referred to this as the principle of “minimal attachment” (Ferreira and Henderson, 1991; Frazier and Rayner, 1982). However, this is not a straightforward task, as the syntactic and thematic relationships between a word and the remainder of the sentence are often undefined when that word is

362

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

first encountered, and can be specified only after additional material from the sentence has emerged. The process of sentence construction can be studied by examining the resolution of a temporary structural ambiguity. Consider a sentence like “The citizens heard the election result was fixed.” As this sentence emerges over time, for example, a listener is likely to interpret “the election result” initially as the direct object (DO) of “heard.” However, as the sentence is completed, it becomes apparent that “the election result” is the subject in a sentential complement (SC) that includes the verb “was fixed.” While it is possible to disambiguate a sentence by inserting the complementizer “that” (e.g. “The citizens heard that the election result was fixed”), text corpora statistics show that such complementizers are commonly omitted (Elsness, 1984; McDavid, 1964; Thompson and Mulac, 1991). Recent reviews have identified two main theories of syntactic processing: Parallel processing, which emphasizes working memory, and serial processing, which emphasizes on-line decisionmaking (Gibson and Pearlmutter, 2000; Lewis, 2000). The parallel processing approach posits that, when parsing a sentence, we build multiple representations of all possible sentence structures, and maintain them in WM. According to this theory, we compare the alternate constructions in WM and then interpret the sentence by choosing the most plausible structure (Earley, 1970; Gibson, 1998; Gibson and Pearlmutter, 1998; Gibson and Pearlmutter, 2000; Jurafsky, 1996; Just and Carpenter, 1992; MacDonald et al., 1994; Pearlmutter et al., 1999; Spivey and Tanenhaus, 1998; Stevenson, 1994). Mason et al. (2003) found experimental evidence for this parallel method of sentence processing in their fMRI study of ambiguous sentences. They observed higher levels of brain activation in left IFC and left PLTC when an ambiguous sentence is resolved in favor of either the preferred (more probable) or the unpreferred (less probable) interpretation, as compared to an unambiguous sentence. The authors explain their findings on the basis of a ranked parallel model, which always consumes massive WM resources during sentence disambiguation, since all possible structural interpretations must be constructed and temporarily maintained in an active state regardless of sentence type. A similar conclusion was reached on the basis of inferior frontal activation in a study that assessed structurally ambiguous sentences in individuals with high and low WM spans (Fiebach et al., 2005). The IFC component of verbal WM has been associated with a rehearsal mechanism (Jonides et al., 1998; Smith and Jonides, 1998a,b). Other work with unambiguous sentences has shown that, in the course of sentence processing, WM demands are also supported by a phonological buffer associated with inferior parietal cortex (IPC) (Cooke et al., 2006; Fiebach et al., 2001; Grossman et al., 2002; Stowe et al., 1998). The second processing approach uses on-line planning and subgoal decision-making to disambiguate sentence structure, based on the probabilistic information conveyed by lexical biases in the sentence (Ferreira and Clifton, 1986; Ferreira and Henderson, 1990; Garnsey et al., 1997; Traxler et al., 1998; Trueswell et al., 1994; Trueswell et al., 1993). The temporary structural ambiguity of the sentence “The citizens heard the election result was fixed” is set up by the fact that the verb “heard” is biased statistically in its more frequent association with a DO sentence structure, even though it is embedded in this instance in a sentence with an SC structure. By comparison, the sentence “The citizens claimed the election result was fixed” is less likely to evoke a garden path interpretation since the verb “claimed” is associated more often with an SC type of sentence structure. This kind of lexical bias can be used on-line to

anticipate the most likely sentence structure. While the parallel method computes and maintains all possible parses, the serial approach only follows the most probable one. This leads to reduced WM demands, but at the cost of greater decision-making and planning. Serial models access the mental lexicon to obtain a broad range of information about the words being encountered, including lexical bias (Garnsey et al., 1997), semantic plausibility and pragmatics (Traxler et al., 1998), in order to determine the most probable interpretation, which is then pursued. For example, in an experiment investigating the DO/SC ambiguity with a word-by-word self-paced reading task, Holmes (1987) observed that readers were clearly led astray in case the unfolding sentence structure did not meet their expectations. Subjects used information provided by the verb bias, as well as the plausibility of the nominal phrase as a direct object, in order to choose what interpretation to follow. In contrast, in parallel accounts of sentence processing, when the reader encounters a word, all of the semantic primitives associated with the word receive some activation (Traxler et al., 1998), which activates all possible structure interpretations, regardless of context and specific lexical properties of sentence constituents. The serial account of sentence processing thus results in increased costs of decision making, as a decision has to be made regarding what interpretation to follow. Non-linguistic studies of planning and switching implicate dorsolateral prefrontal cortex (dlPFC) (Braver and Bongiolatti, 2002; Paulus et al., 2001; Rowe and Passingham, 2001; Savage et al., 2001; Seger et al., 2000; Smith et al., 2001; Sylvester et al., 2003). In one study of ambiguous sentence processing, basal ganglia activation was reported (Noppeney and Price, 2004), an area frequently co-activated with dlPFC as part of a fronto-striatal loop that may play a role in error detection (Ullsperger and von Cramon, 2006). However, previous imaging assessments of structural disambiguation have not systematically evaluated the role of verb biases and probabilistic decision-making during sentence processing. In the present study, we investigate whether these same resources are used in association with linguistic tasks. To achieve our goal, we modified the materials of Garnsey et al. (1997) to examine the role of verb bias and WM load on sentence interpretation. We manipulated the compatibility between sentence structure and verb bias, on the one hand, and we also inserted additional, strategically placed material to stress WM independently of grammatical structure in half of the sentences of each type. Moreover, our behavioral technique during fMRI scanning minimized task-related resources that could confound interpretation of the results, since subjects passively viewed grammatically correct, semantically coherent written sentences presented in a phrase-byphrase manner. We hypothesized greater frontal recruitment during sentences with a temporary structural ambiguity. This may be due to increased IFC activation for all sentence types if their interpretation depends on WM. Alternately, dlPFC activation may be seen under a condition of low consistency between a verb and its sentence context, relative to high consistency between a verb and its sentence context, if planning and decision-making contribute to the resolution of a temporary structural ambiguity in a sentence. Moreover, we expected left IPC activation associated with the WM component of sentence processing, regardless of the type of sentence. We acknowledge that WM is always needed to read sentences. However, in this study, we are measuring differences in relative WM load rather than absolute values. We operationalize WM by including the additional phrase and predict IPC activation

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

under conditions of relatively higher WM load. Such observation will allow us to infer that IPC is a neural resource for WM, which is activated when there is a relatively higher demand for WM. Methods Subjects In our behavioral study we tested 20 young (4 males and 16 females), healthy adults, matched for age and education (mean (± SD) age = 21.4 (± 0.5) years; mean (± SD) education = 15.0 (± 0.05) years) with the imaged subjects. For the imaging experiment, 19 volunteers (13 males and 6 females, aged 18–30 (mean (± SD) = 20.8 (±0.5) years), education level 13–21 years (mean (± SD) = 15.6 (± 0.5) years) were scanned. All participants were right-handed native English speakers. All subjects were young and healthy, and were screened medically in order to insure they were not taking any psychoactive medications and had no neurologic or psychiatric disorders. Each subject participated in an informed consent procedure approved by the Institutional Review Board (IRB) at the University of Pennsylvania. Materials In this experiment, we manipulated the type of sentence structure, the syntactic bias associated with the main verb of a sentence, the syntactic bias associated with the post-verbal noun, and the working memory demands associated with these materials. Sentence structure The sentence type is determined by the syntactic frame in which a verb and the other words of a sentence are embedded. Sentences in our experiment had either an SC structure or a DO structure. For example, consider the following: 1. The worried friar asserted the belief would be justified (SC). 2. The diligent disciple asserted the belief readily (DO). Sentence 1 has an SC structure, whereas sentence 2 has a DO structure. In the former, the verb “asserted” is followed by the sentential complement “the belief would be justified.” In this case, “the belief ” is the head noun of that subordinate clause. In the latter sentence, “the belief ” is the direct object of the verb. The entire set of sentences is presented in Appendix A. Verb category Verbs are biased statistically to occur with a varying probability in one of these sentence contexts. Some verbs are more likely to be used in sentences in which they are followed by a direct object, and are hence more often encountered in a DO structure. For instance, the verb “assert” normally entails an object being asserted and is thus found more frequently in sentences of the type “The honest witness asserted the truth forcefully” than in sentences like “The honest witness asserted the truth was important.” A DO verb can nevertheless occur in an SC structure without disturbing sentence coherence. By comparison, other verbs are seen more commonly in sentence constructions that contain a sentential complement and thus have an SC statistical bias. For example, one is more likely to

363

encounter the verb “prove” in a sentence with an SC structure such as “The gifted lawyer proved the argument was unfounded” than in a DO structure like “The gifted lawyer proved the argument.” SCbias verbs can occur in a DO sentence structure without disturbing sentence coherence as well. Finally, we also used a class of verbs without a sentence bias, having an equal probability of appearing in a DO sentence or an SC sentence. This is to diminish the possibility that subjects would explicitly detect that a subset of the verbs have a structural bias that is less compatible with the syntactic structure of the sentence. For instance, the verb “acknowledge” falls into this category, since it is associated as often with a DO structure like “The sales clerk acknowledged the error honestly” as with an SC structure like “The sales clerk acknowledged the error was minor.” We refer to this as an EQ-bias verb. Verbs were classified as SC-biased if they were found at least twice as often with an embedded sentential complement as with a direct object. The reverse was true for verbs categorized as DObiased. Verbs were classified as EQ-biased if they occurred approximately equally often with sentential complements and with direct objects (with a difference not greater than 15%). We based our choice on the verb ratings produced by Connine et al. (1984) and on their frequency of occurrence. For this experiment, we selected 10 verbs of each type. The three sets of verbs differed in SC-bias (F(2,45) = 29.38, Mse = 276, p b 0.01) and in DO-bias (F(2,45) = 91.26, Mse = 173, p b 0.01). The mean preference strength of each type of verb towards each completion is summarized in Table 1. The three sets of verbs did not differ significantly in letter length (F = 2) or frequency of occurrence (Francis and Kucera, 1982) (F = 1). The post-verbal noun was selected so that any bias was equally distributed over each verb–sentence structure relationship. Some verbs of each type were chosen to be polysemic, i.e. to have more than one meaning. For instance the verb “maintain” is used both in the sense of providing maintenance (“maintained the machinery”) and of holding an opinion (“maintained the machinery was obsolete”). The number of polysemic verbs is low and equally distributed over verb types (two polysemic verbs of type DO and two of type SC). The entire set of verbs is presented in Appendix B. All six combinations of verb–sentence structure are grammatically correct and semantically coherent. They appear with equal frequency in our set of materials, hence giving rise to moreconsistent or less-consistent stimuli: verbs of type DO are more consistent with a DO sentence context and less consistent with an SC sentence structure; SC-biased verbs are more consistent with an SC sentence frame and less consistent with a DO sentence context; and EQ verbs are equally consistent with both DO and SC sentence structures, and were thus considered as belonging to the same category as the sentential context in which they were embedded. For analytic purposes, we treated these as “more consistent” stimuli. Table 1 Mean (±SD) preference strength of each type of verb towards each completion Verb bias

Completion

Mean (± SD) preference strength

SC DO SC DO EQ EQ

SC DO DO SC DO SC

0.74 (0.09) 0.71 (0.14) 0.15 (0.10) 0.13 (0.07) 0.41 (0.08) 0.45 (0.10)

364

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

Noun category Another relevant factor in our study is the plausibility of the post-verbal noun phrase as a direct object in the particular sentence under consideration. Similarly to verbs, nouns also have varying affinities for appearing in direct object contexts and sentential complement contexts. Consequently, the choice of the post-verbal noun in a sentence with a temporary structural ambiguity can greatly influence the initial interpretation of the sentence. For instance, the noun “money” is far more plausible as a direct object of “accepted” than the noun “fire” in the following DO-type sentences: “The talented photographer accepted the money” vs. “The talented photographer accepted the fire”. Likewise, in an SCtype sentence containing a DO-biased verb, the type of post-verbal noun has a considerable influence on the sentence interpretation. For example, the sentences “The talented photographer accepted the money was spurious” and “The talented photographer accepted the fire was hazardous” have the same structural frame (SC); however, the particular choice of noun biases their interpretations in slightly different directions. As was done for sentence and verb type, materials were designed so that noun types were also matched for frequency. Half of the nouns used in the study had an SC bias and the remaining a DO bias. Noun classification (SC or DO) was based on their plausibility as direct objects of each verb. Garnsey et al. (1997) determined noun plausibility with a norming study, in which subjects were asked to rate sentences like the following on a 7-point scale (7 = very plausible): “The senior senator regretted the decision” and “The senior senator regretted the reporter”. A high score would mean that the post-verbal noun is a plausible direct object of its predecessing verb. Nouns were categorized as more or less plausible direct objects (thus, DO or SC) if they met the following criterion: The difference in ratings between the more plausible and less plausible direct object version for each verb was at least 2.5 on the 7-point scale. We based our choice on the noun ratings produced by Garnsey et al. (1997). For this experiment, we selected 60 nouns, 30 of each type (DO or SC). The choice of nouns in Garnsey et al. (1997) was, however, not completely suitable for our purposes. In our study, we want to address the processing of sentences with temporary structural ambiguities. These ambiguities often emerge due to an inconsistency between the noun type and the sentence frame in which it is embedded. Therefore, if the noun is clearly SC- or DObiased, such ambiguity may not occur altogether. We thus had to change some of the post-verbal nouns (18 in total) provided by Garnsey et al. (1997), as they were not ambiguous enough to generate the garden path effect. For instance, we had to change one of the nouns following the verb “declared”. In the original version, the two sentences were as follows: “The new mayor declared the holiday would be a festive occasion” and “The new mayor declared the potholes would be repaired in June”. In this case, the post-verbal noun “the potholes” does not raise a temporary structural ambiguity, since it obviously cannot be the direct object of the verb “declared”. The reader immediately understands that it is, rather, the subject of an unfolding sentential complement structure. There is, consequently, no garden path effect. We used a different post-verbal noun, in order to create a DO/SC ambiguity. The 18 nouns we introduced into the study were also rated for plausibility, following the aforementioned norming method, by a group of 15 (3 males and 12 females) healthy, young (aged 22–27, mean (±SD) = 24.5 (± 0.5) years) native English speakers. In order to avoid priming and familiarity effects, which could have emerged in case part of the

materials had been disclosed to the participants prior to the study, we asked a separate group of subjects to perform this rating. Nouns did not differ significantly in length (Fs b 1) or frequency of occurrence (Fs b 2). The whole set of nouns, along with their postverbal plausibility ratings, is presented in Appendix C. Working memory Half of each type of sentence had an additional WM component. In order to make subjects retain information about the verb and the post-verbal nominal phrase in memory for a longer period of time before sentence structure is resolved, we inserted an extra WM phrase between the post-verbal nominal phrase and the last (disambiguating) phrase. We opted for inserting the WM phrase just prior to disambiguation, as opposed to an earlier location within the sentence, such as between the verb and the noun, in order to avoid confounds that could thereby emerge from disrupting the verb–noun structure. Experimental procedure In the present experiment we used the phrase-by-phrase method, using a “moving window” display. This approach has been used previously to assess sentence processing in cognitive (Just et al., 1992) and imaging (Cooke et al., 2006) studies. Sentences were presented to the subjects in a written, phrase-by-phrase manner, in the following format: [initial-phrase] [verb-phrase] [noun-phrase] ([WM-phrase]) [concluding-phrase]. An example of sentence presentation according to the experimental design is provided in Table 2. The sentence structure used in our experiment is as follows: the first nominal phrase (the subject) always contains three words (“the” + adjective + noun); the verbal phrase is always a verb in the simple past (e.g. “confirmed”); the second nominal phrase (the postverbal noun) is always a two-word construct (“the” + noun); the variable working memory phrase is a four-word long prepositional phrase (e.g. “in upstate New York”); the last (disambiguating) phrase consists of an adverbial phrase (e.g. “cheerfully”, “a thousand years ago”) or a brief verbal phrase (e.g. “were fair”, “was a festive occasion”), the length of which (in syllables) was counterbalanced across conditions, so that it would not affect the results. We used written materials to minimize the risk of biasing sentence structural interpretation derived from oral prosody. The six types of sentences were equally distributed in a random order over eight blocks (30 sentences per block), and each verb appeared exactly once in each block. Sentences were presented in a pseudorandom order within each block. Sentence material was viewed passively to

Table 2 Example of sentence presentation according to experimental design a Initial phrase

Verb phrase

Noun phrase

The diligent disciple The old nun

Asserted

The belief

Asserted

The belief

Optional WM phrase

Concluding phrase Readily

In various religious cults

Authoritatively

a Example of two sentences used in our experiment, presented according to the design. Both stimuli are “more consistent” (DO verb and DO noun in a DO sentence frame). The first sentence is a stimulus “without WM”, whereas the second is a stimulus “with WM”.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

minimize task-related resources involved in decision-making. In order to insure that the subjects were attending to the task and processing the information, 20% of sentences in each block (counterbalanced across conditions) were probed for a simple fact. The random nature of these probes prevented the participants from expecting a specific type of question and developing a strategy to answer it, as we randomly probed different factual aspects of each probed sentence. For example, the sentence “The rebelling youngsters protested the policy fearlessly” was followed by the probe “Did the rebelling youngsters protest the policy?”. Participants responded to such yes-or-no questions by a simple right- or left-hand button press during the behavioral experiment or the scanning procedure, immediately after the probe. These trials were not excluded from the behavioral or functional analyses. As probes always occurred after the sentence data had already been collected, they did not interfere with that item. Therefore, insofar as our experiment was concerned, probed items were no different from unprobed items when they were read. We did exclude the probes themselves and the responses to the probes. In the imaging experiment each phrase was seen for a fixed 3-s interval. The MRI scanner is equipped with stimulus delivery and monitoring systems for fMRI research so that the sentence stimuli can be presented to subjects while in the bore of the scanner and responses to stimuli can be monitored. These systems include Epson 8100 3-LCD projectors with Buhl long-throw lenses for rearview/rear-projection onto Mylar screens. The projector is housed in a custom RF shield box with filtered power receptacles. Images are viewed by the subject through mirrors mounted on the head coils. Responses are monitored using FORP fiber optic button boxes. A laptop computer outside the scanner room used E-Prime presentation software (version 1.1) to present stimuli and record response accuracy. In the behavioral experiment, materials were presented by a laptop computer with E-Prime presentation software (version 1.1). Subjects were instructed to press the space bar to advance through the phrases of a sentence, and we monitored these latencies from the onset of phrase presentation. Subjects were instructed to read each sentence rapidly but carefully since we would be randomly probing the content of some sentences. Subjects saw the phrase “GET READY” on the screen at the start of the experiment, after which the first phrase of the first sentence was displayed. A hash mark (“#”) was shown for 3 s between stimuli, that is, between the last phrase of a sentence and the first phrase of the subsequent sentence in case there was no probe. If there was a probe question, that separator appeared right after the probe and before the following stimulus. Imaging data acquisition and analysis procedure This study was performed with a 3.0 Tesla Siemens Trio wholebody human MRI scanner at the Center for Advanced Magnetic Resonance Imaging and Spectroscopy (CAMRIS) in the Department of Radiology of the University of Pennsylvania. We used a standard clinical quadrature radiofrequency head coil. In order to minimize and restrict head motion, firm foam padding was used. Each imaging protocol began with MR structural scans to determine regional anatomy: localizer images (TR = 20 ms, TE = 5 ms, 192 × 192 matrix) and T1-weighted MPRAGE images (TR = 1620 ms, TE = 3.87 ms, 192 × 256 matrix). BOLD fMRI monitored brain activation during sentence presentation. A T2*-weighted echoplanar BOLD paradigm was adopted for acquiring functional

365

images, with the following parameters: TR = 3000 ms, TE = 30 ms, a rectangular FOV of 20 × 15 cm, flip angle of 90°, 3-mm slice thickness and a 64 × 64 matrix, resulting in a voxel size of 3 × 3 × 3 mm. The data were analyzed off-line using SPM2 (Wellcome Department of Imaging Neuroscience). Each data set was registered and aligned to the first image in the series, and the brain volumes were normalized by registration to the T1 template (Evans et al., 1993) of 305 averaged brain volumes (Frackowiak et al., 1997) using a 12-parameter affine non-linear registration using 12 non-linear iterations and 7 × 8 × 7 basis functions. The volumes were normalized to Talairach and Tournoux brain coordinates (Talairach and Tournoux, 1988). An event-related design acquired data for each phrase. Onset of phrase presentation was synchronized with the scanner so that we could acquire the onset of the hemodynamic response for each stimulus presentation. By performing such synchronization, we could accurately determine the BOLD effect elicited by the presentation of each phrase. The fMRI time series was modeled using a hemodynamic response function (HRF) with a temporal derivative (Friston et al., 1998) and a time-to-peak parameter of 5 s. Intrinsic temporal correlations were eliminated by estimating the autocorrelation structure using a first order autoregressive model (AR(1)) (Friston et al., 2000). We used a subtraction procedure in our image analyses to isolate effects of interest. In particular, we were interested in describing the activation pattern during the last phrase of each sentence, where the temporary structural ambiguity is resolved. To insure that the activation was due to the disambiguation process itself, rather than some other aspect of written language processing, we subtracted the activation resulting from the presentation of each initial, unbiased phrase of each sentence from the activation associated with the terminal, disambiguating phrase of each sentence. Subsequently, we contrasted these subtracted activations across types of sentences to identify anatomical effects associated with each hypothesized condition of interest. Since we compared the terminal phrase across conditions after having subtracted the initial phrase for each condition, we safely ruled out the possibility of attributing some of the observed activation patterns to linguistic wrap-up effects. A random-effects statistical model (Penny and Holmes, 2003) was used that, first, performed the contrast of activation associated with the terminal phrase minus the initial phrase for each condition of interest in each subject. These subtractions were then contrasted across conditions of interest within each subject. Finally, these contrasts were examined across individuals in a second-level analysis using a 1-sample t-test procedure. Using the SPM random-effects model (Holmes and Friston, 1998), we contrasted activation patterns such as “more consistent” vs. “less consistent” stimuli and “high working memory” vs. “low working memory” for the group. We provide contrasts with a height threshold that is significant at the p b 0.001 level uncorrected, corresponding to a z-score N 3.09, and an extent threshold of at least 20 voxels. Although relatively liberal, this significance threshold is justified because we are contrasting activations that are controlled by contrasts with an unbiased baseline (the initial phrase of the sentence). Results Behavioral study We found that the terminal phrase of a less consistent stimulus takes longer to read than that of a more consistent stimulus (Table

366

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

3). No other reading time differences were found. We analyzed the roles of temporary structural ambiguity and working memory (WM) using an analysis of variance (ANOVA) with a consistency (2: more-consistent vs. less-consistent) × WM (2: without additional WM phrase vs. with an additional WM phrase) design. Our design contains two sentence types (DO and SC), three verb types (DO, SC and EQ) and two noun types (DO and SC). However, in our analysis, we have focused on the consistency between verb and sentence, having thus reduced the first two factors (sentence and verb types) to two conditions only: “more consistent” and “less consistent”. The only other variable in the analysis is WM: “with WM” and “without WM”. So, we only have four conditions (a 2 × 2 design). There was a significant main effect for consistency [F(1,19) = 9.18; p = 0.007] (obtained by collapsing over WM variables, i.e. “without WM” and “with WM” stimuli). We did not see a significant main effect for WM [F(1,19) = 0.84; p = 0.37] (measured across consistency) or a significant interaction effect between WM and consistency factors [F(2,19) = 1.53; p = 0.23]. There may have been a difference between the imaging data and the behavioral data because manners of presentation were not identical—whereas the behavioral experiment used a self-paced moving-window presentation, in the imaging study all phrases were presented for 3 s. Even though we found no effect for WM in the behavioral study, our a priori hypothesis was that there might be a main effect for WM and that consistency and WM might interact. Therefore, we contrasted WM conditions within less-consistent and more-consistent stimuli. This revealed a difference between “more-consistent, with WM” and “more-consistent, without WM” stimulus types [t(19) = 3.33; p = 0.004]. We also compared stimuli of the same WM type but different consistency levels, and observed that there is a significant difference between “more-consistent, without WM” and “less-consistent, without WM” stimulus types (p = 0.004), but not between “more-consistent, with WM” and “less-consistent, with WM” stimulus types (p = 0.262). Accuracy in responding to probes was comparable and very high for all stimuli (see Table 3). fMRI activation study Less-consistent stimuli were associated with dlPFC activation. We compared cortical activation patterns elicited by “more consistent” stimuli with those resulting from “less consistent” stimuli. The contrasts of the disambiguating phrase minus the initial, neutral phrase for “more consistent” stimuli and “less consistent” stimuli are illustrated in Figs. 1A–C, and the location and statistical attributes of the peak voxel in each cluster are summarized in Table 4. We observed greater left dlPFC and bilateral IFC activation during the disambiguating phrase for the “more consistent” stimuli (Fig. 1A), and bilateral dlPFC and bilateral IFC activation for the “less consistent” stimuli (Fig. 1B). Fig. 1C and Table 4 also summarize the direct subtraction of “less-consistent” minus “more-consistent” conditions. This contrast showed activation in bilateral dlPFC.

Fig. 1. Activation for more consistent and less consistent stimuli1 and activation for more WM minus less WM2. Note: 1. Distribution of activation during the last phrase (relative to the first phrase) for stimuli in which verb and sentence structure are “more consistent” (A) or “less consistent” (B). Panel C depicts the contrast of these two conditions (“less-consistent” minus “more-consistent”). 2. Distribution of activation during the last phrase (relative to the first phrase) for the contrast of stimuli that contain an additional WM phrase minus stimuli that do not contain an additional WM phrase (D).

No areas showed higher activation levels for “more consistent” than for “less consistent” stimuli. Percent signal change measurements for the activated regions, for both “more consistent” and “less consistent” stimuli, are summarized in Table 5. We also investigated more detailed effects in the sentence materials unrelated to our main hypothesis, such as manipulating the type of verb in itself, as well as the effect of manipulating the

Table 3 Mean (±SD) response latencies for more consistent and less consistent verb–sentence stimuli a Stimulus type

Main effect

Without WM

With WM

Probe accuracy

More consistent Less consistent

671.78 (±257.22) 704.64 (±254.98)

660.84 (±261.35) 703.66 (±266.80)

682.73 (±258.47) 705.63 (±246.50)

95.33 (±7.83) 91.36 (±7.31)

a Results for main effect (first column) were obtained by collapsing over WM variables (“without WM” and “with WM” stimuli). Results for “without WM” and “with WM” columns were obtained by measuring latencies separately for each condition. Latency values are in ms. Probe accuracy is percent correct.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Table 4 Neural activation patterns for more consistent and less consistent stimuli Contrast More consistent

a

Less consistent a

Less consistent−more consistent b

Activation locus

Brodmann area (BA)

Coordinates (x, y, z)

z-score

L IFC R IFC L dlPFC L IFC L dlPFC R IFC R dlPFC L dlPFC R dlPFC

47 47 10 47 9 47 9 9 9

(− 48, 19, − 8) (36, 11, − 11) (− 44, 47, − 2) (− 48, 15, − 7) (− 48, 9, 33) (40, 11, − 11) (44, 25, 28) (− 48, 9, 33) (44, 25, 28)

4.81 3.94 3.36 3.85 3.72 3.70 3.34 3.21 3.19

367

Table 6 Neural activation for more working memory and less working memory stimuli Contrast

Activation Brodmann Coordinates locus area (BA) (x, y, z)

With WM−without L IPC WM a Less consistent, with L IPC WM−less consistent, without WM a

z-score

40

(−59, −53, 25) 3.40

40

(−55, −44, 36) 3.45

a These findings involve contrasts of the final phrase where a temporary structural ambiguity is resolved minus the initial, unbiased phrase of the sentence.

a

These findings involve contrasts of the phrase where the structural ambiguity is resolved minus the initial, unbiased phrase of the sentence. b This contrast involves the resolving phrase minus the initial phrase of each type of sentence.

consistency between the verb and the post-verbal noun. These supplementary results are presented in Appendix D. We investigated the WM main effect by evaluating cortical activation patterns that emerged during the terminal phrase following an additional WM phrase minus the initial phrase of these sentences. This was compared with activation elicited by the terminal phrase of a sentence without the extra WM phrase minus the initial phrase. This contrast, illustrated in Fig. 1D, shows left IPC activation. Table 6 summarizes the location and statistical attributes of the peak voxel in this cluster. We also investigated the interaction between consistency and WM effects. In addition to the main effect of WM, we found an interaction with “less consistent” sentences. We observed that IPC showed increased activation for the WM contrast during presentation of stimuli with less-consistent verb–sentence structure features. Stimuli with more-consistent verb–sentence structure features also elicited left IPC activation, but it did not exceed our statistical threshold. Discussion When a temporary structural ambiguity emerges from a conflict between a verb and its sentence context, executive resources are recruited to resolve the ambiguity. Our findings indicate that these resources are supported in large part by dlPFC. Greater dlPFC activation for the less consistent condition is compatible with the hypothesis that on-line decision-making uses lexical biases in sentences to construct an interpretable sentence structure. WM demands appear to be associated with a different brain region-left IPC. Taken together, these results indicate that resolving a temporary structural ambiguity appears to depend on a large-scale neural network involving multiple components. We address each of the observed activations below. Table 5 Percent signal change for more consistent and less consistent stimuli Contrast

L IFC

L dlPFC

R IFC

R dlPFC

More consistent a Less consistent a

0.232 0.742

0.194 0.463

0.200 0.406

0.056 0.303

a These findings refer to the percent signal change in activation between the final phrase, where a temporary structural ambiguity is resolved, and the initial, unbiased phrase of the sentence.

Dorsolateral prefrontal cortex We observed increased activation in dlPFC under a condition of low consistency between a verb and its sentence context, relative to high consistency between a verb and its sentence context. The fact that additional resources are required for a “less consistent” sentence compared to a “more consistent” sentence is less compatible with the “massive working memory” account of structural ambiguity resolution, where all possible sentence structures are maintained in working memory during the processing of all types of sentences. From this perspective, both “more consistent” and “less consistent” stimuli require equal WM resources since all possible interpretations of both types must be constructed, and this will involve equal numbers of representations. Instead, greater activation for “less consistent” stimuli is more compatible with the additional planning required for these stimuli compared to “more consistent” stimuli. dlPFC recruitment for non-linguistic materials is consistent with this account. For example, activation of this area is seen during top-down decision-making on Stroop-like tasks (Braver and Bongiolatti, 2002; MacDonald et al., 2000; Paulus et al., 2001). While WM-demanding tasks have been shown to activate dlPFC (Braver et al., 1997; Callicott et al., 1999; Cohen et al., 1997; Garavan et al., 2000), other work suggests that dlPFC plays an active role in the strategic manipulation of information being held in WM (Barch et al., 1997; Prabhakaran et al., 2000; Rowe and Passingham, 2001; Smith et al., 2001; Sylvester et al., 2003). A language production task noted dlPFC activation during grammatically complex sentences involving “Jabberwocky-like” stimuli in content word positions (Indefrey et al., 2001), but the feature of grammatical complexity alone may not explain dlPFC activation since the synthesis of artificial, Jabberwocky material into complex sentences may require complex planning and decision-making. It is beyond the scope of this report to compare directly the decision-making resources implicated in sentential and non-linguistic tasks. Greater activation of dlPFC for “less consistent” stimuli than for “more consistent” stimuli nevertheless appears to be compatible with previous, non-linguistic work demonstrating the contribution of this area to decision-making and planning. We cannot, however, exclude the possibility that some of the difference between conditions may be due in part to factors related to lexical content beyond those that we did control. Furthermore, our behavioral results also provide us with evidence compatible with a serial approach supported by dlPFC. Subjects take longer to read “less consistent” than “more consistent” stimuli. Moreover, the presence of an additional WM phrase elicits a significant difference in reading times for “more consistent” sti-

368

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

muli, but not for “less consistent” stimuli. Additional resources are recruited for processing all “less consistent” stimuli, from our perspective, thus obscuring the ability to detect behavioral changes that can be attributed solely to the WM manipulation. However, for “more consistent” sentences, it is possible to observe increased demands in the “with WM” condition only, if WM is dissociated from the decision-making resources involved in verb–sentence consistency. Inferior frontal cortex In the present study, we observed up-regulation of IFC bilaterally under both “more-consistent” and “less-consistent” conditions. This indicates that IFC is involved as a ubiquitous component supporting sentence processing. Based on observations of aphasics following stroke (Tramo et al., 1988; Zurif, 1996), neuroimaging experiments in healthy adults have associated left IFC with grammatically dependent aspects of sentence processing (BenShachar et al., 2004; Caplan et al., 1998; Caplan et al., 1999; Caplan et al., 2000; Cooke et al., 2001; Inui et al., 1998; Kang et al., 1999; Keller et al., 2001; Kuperberg et al., 2003; Kuperberg et al., 2000; Moro et al., 2001; Newman et al., 2001; Newman et al., 2003; Ni et al., 2000; Stromswold et al., 1996). Other studies have associated left IFC with verbal WM (Barch et al., 1997; Braver et al., 2001; Cohen et al., 1997; Smith and Jonides, 1999). Histological work has subdivided IFC into two neuroanatomically distinct segments – dorsal IFC (dIFC) and ventral IFC (vIFC) – based on the connectivity patterns and cytoarchitectonic characteristics of these regions (Amunts et al., 1999). These two anatomically distinct subregions may thus perform distinct functions: vIFC appears to contribute more to the grammatical component of sentence processing, whereas dIFC is more important for resource-dependent aspects of sentence processing, such as the WM component (Cooke et al., 2006; Cooke et al., 2001; Grossman et al., 2002). The activation peak in our study was located in BA 47 in the ventral part of left IFC, associating left IFC activation, at least in part, with syntactic processing of the stimulus sentences. This impression is consistent with the fact that we did not observe IFC activation during the WM contrasts within either low-consistency or highconsistency sentences. We observed right IFC activation during these sentence materials as well. Previous work associated right IFC activation during sentence processing with the resource demands of a difficult sentence rather than a language-specific process such as grammatical processing (Just et al., 1996). However, the right-sided locus of this activation is less likely to be implicated in the WM needed to maintain multiple grammatical structures in an active state during processing, since this area has been associated with visual, rather than verbal, WM (Baddeley, 1992; Jonides et al., 1993; Smith and Jonides, 1998b; Smith et al., 1996). Inferior parietal cortex Left IPC appears to support a component of WM (Awh et al., 1996; Chein and Fiez, 2001; Jonides et al., 1998; Marshuetz et al., 2000; Paulesu et al., 1993; Smith and Jonides, 2002). This area may serve as a phonological buffer, temporarily storing phonological input information from the verb phrase until the structure of the sentence can be resolved by the final phrase. In this study, we expected to see phonological WM associated with IPC activation. Jonides et al. (1998) obtained activation of BAs 7 and 40 for WM

tasks, and concluded that these parietal regions are part of a cortical network that mediates short-term storage and retrieval of phonologically coded verbal material. Our results are in agreement with these, since we found BA 40 to be consistently recruited for conditions with high WM demands. Lesion studies further corroborate these results: deficits in verbal WM are caused by a lesion in left posterior parietal cortex and surrounding tissue (Saffran and Marin, 1975; Shallice and Vallar, 1990; Warrington et al., 1971). We may not have seen left IFC activation during WM demands, as in previous imaging studies of sentence processing (Cooke et al., 2006; Cooke et al., 2001), because IFC was subtracted out by its occurrence in both sentence conditions. Alternately, previous work showing dIFC activation for WM during sentence processing (Cooke et al., 2006; Cooke et al., 2001) inserted additional material between the head noun and the main verb of the sentence. This may have stressed WM during sentence processing in a manner that differs from our study, which may explain the absence of an effect for WM in our behavioral study. In that work, the extra phrase was placed much earlier in the sentence, thus lengthening the distance between the gap and the co-indexed noun, which had a direct effect on the gap-filling component of sentence processing. This is different from the manipulation done in the sentence materials in our study, where additional material was inserted after the main verb, in a manner that does not interrupt or lengthen a co-indexed gap. For example, temporary maintenance in a phonological buffer may be particularly useful during on-line decision making to support recovery from an incorrect structural interpretation. This may explain greater left IPC activation for “less consistent” stimuli. There are several interpretations of the observed laterality effect. Some consider that resources regarding language are lateralized to the left hemisphere, but some resources used during sentence processing may be material-neutral and bilateral (Just and Carpenter, 1992; Just et al., 1996). We see that IPC activation is lateralized to the left hemisphere, which is more consistent with this resource being a material-specific phonological buffer, as suggested in other work (Jonides et al., 1998). The area activated for the WM manipulation is left IPC, and this is associated with the inclusion of the additional phrase: we do not see IPC activation during the manipulation associated with decision-making during resolution of the conflict between a verb and its sentence structure. Thus, manipulating the WM load showed that additional activation results from this extra phrase independently of the activation associated with solving a temporary structural ambiguity, and that it is in a different part of the brain. Furthermore, in another paper in preparation, we find that there is no accumulation of activation associated with WM during sentence processing up to the point of disambiguation. These observations provide us with further evidence consistent with a serial processing approach. We infer that this disambiguation is related at least in part to dlPFC. Neuroanatomical studies have found extensive interconnections between prefrontal (especially dlPFC) and parietal areas (Mesulam et al., 1977; Pandya and Seltzer, 1982; Petrides and Pandya, 1984). Furthermore, these two cortical regions project to common cortical and subcortical areas (Selemon and Goldman-Rakic, 1988), and primates show the co-activation of these regions during WM tasks (Friedman and Goldman-Rakic, 1994). It is neuroanatomically plausible to suggest that there is a systematic coactivation of these regions during processing of sentences with a temporary structural ambiguity and high working memory load, as we observed in our study.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Appendix A (continued)

Conclusion

No. V

The findings of the present study are consistent with a structure building process during sentence comprehension that involves at least serial, on-line decision-making. This process is supported by dlPFC. Moreover, WM resources that contribute to sentence comprehension are supported by IPC. These resources appear to play a role in a large-scale neural network that supports language processing. Observations such as these are consistent with a twocomponent model of sentence processing: the core peri-Sylvian language processing regions that include IFC, and a set of executive resources that involves at least dlPFC and IPC. These resources are activated as needed, when the core regions require additional support to process a given stimulus. Acknowledgments This work was supported in part by NIH (NS35867, AG17586, NS44266 and AG15116) and the Dana Foundation. Susana Novais-Santos is also supported by the Portuguese Foundation for Science and Technology (SFRH/BD/10205/2002).

Appendix A. Stimulus sentences The table below contains all 240 stimuli used in the study, and indicates the types of verb (V), noun (N) and sentence (S), the absence (“−”) or presence (“+”) of an additional WM phrase, as well as the level of consistency (C) [more (“M”) or less (“L”)], for each stimulus No. V

WM C

Sentence

1

DO DO SC

−

L

2

DO DO SC

+

L

3

DO DO DO −

M

4

DO DO DO +

M

5

DO SC

SC

−

L

6

DO SC

SC

+

L

7

DO SC

DO −

M

8

DO SC

DO +

M

9

DO DO SC

−

L

10

DO DO SC

+

L

11

DO DO DO −

M

12

DO DO DO +

M

13

DO SC

SC

−

L

14

DO SC

SC

+

L

The worried friar asserted the belief would be justified. The concerned priest asserted the belief in stone-idol worship was ungrounded. The diligent disciple asserted the belief readily. The old nun asserted the belief in various religious cults authoritatively. The passionate artist asserted the truth would be difficult. The city-sweeper asserted the truth about minimum wage payments was shocking. The loud newsreader asserted the truth cheerfully. The sleepy student asserted the truth about cheating on tests merrily. The CIA director confirmed the rumor should be stopped now. The town gossip confirmed the rumor about the newly-weds was false. The friendly neighbor confirmed the rumor in an instant. The talkative servant confirmed the rumor about the prime minister slowly. The financial assistant confirmed the money should be managed better. The newspaper agency confirmed the money from the local bank had been stolen. The new accountant confirmed the money immediately.

15

N

DO SC

369

S

DO −

M

N

S

WM C

16

DO SC

DO +

17

DO DO SC

−

18

DO DO SC

+

19

DO DO DO −

20

DO DO DO +

21

DO SC

SC

−

22

DO SC

SC

+

23

DO SC

DO −

24

DO SC

DO +

25

DO DO SC

−

26

DO DO SC

+

27

DO DO DO −

28

DO DO DO +

29

DO SC

SC

−

30

DO SC

SC

+

31

DO SC

DO −

32

DO SC

DO +

33

DO DO SC

−

34

DO DO SC

+

35

DO DO DO −

36

DO DO DO +

37

DO SC

SC

−

38

DO SC

SC

+

39

DO SC

DO −

40

DO SC

DO +

41

DO DO SC

−

42

DO DO SC

+

43

DO DO DO −

Sentence

M The computing program confirmed the money from New York instantaneously. L The angry father emphasized the shortcomings were numerous. L The fire fighter emphasized the shortcomings of the faulty hoses were increasing. M The impoverished archaeologist emphasized the shortcomings forcefully. M The experienced judge emphasized the shortcomings of the legal system with concern. L The talented photographer emphasized the schools were ignoring photography. L The irate parents emphasized the schools across the entire nation had uninterested teachers. M The finance minister emphasized the schools when addressing the public. M The teaching inspectors emphasized the schools with strong sports teams in their annual report. L The hitch-hiker heard the story was not true. L The primary suspect heard the story given by his friend was a lie. M The busy psychiatrist heard the story briefly. M The doting grandfather heard the story with all its exaggerations unhappily. L The naughty boy heard the neighbors were evil. L The nosy butler heard the neighbors on the first floor were nice people. M The bus-conductor heard the neighbors daily. M The cleaning-woman heard the neighbors from the adjoining house each night. L The maintenance people insured the house would not collapse again. L The young architect insured the house with the expensive interior was sold. M The new owners insured the house quickly. M The old proprietor insured the house in upstate New York as soon as possible. L The honeymoon couple insured the river would never flood their property. L The aged gardener insured the river with the varying current would not wash away his daisies. M The eccentric landlord insured the river in court. M The wicked baron insured the river on his huge estate at once. L The stubborn technician maintained the machinery was state-of-the-art. L The displeased client maintained the machinery with the fancy gadgets was obsolete. M The confident engineer maintained the machinery diligently. (continued on next page)

370

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Appendix A (continued)

Appendix A (continued) No. V

N

S

WM C

44

DO DO DO +

45

DO SC

SC

−

46

DO SC

SC

+

47

DO SC

DO −

48

DO SC

DO +

49

DO DO SC

−

50

DO DO SC

+

51

DO DO DO −

52

DO DO DO +

53

DO SC

SC

−

54

DO SC

SC

+

55

DO SC

DO −

56

DO SC

DO +

57

DO DO SC

−

58

DO DO SC

+

59

DO DO DO −

60

DO DO DO +

61

DO SC

SC

−

62

DO SC

SC

+

63

DO SC

DO −

64

DO SC

DO +

65

DO DO SC

−

66

DO DO SC

+

67

DO DO DO −

68

DO DO DO +

69

DO SC

SC

−

70

DO SC

SC

+

Sentence

M The partisan architect maintained the machinery without the spare parts at the cost of the others. L The arrogant candidate maintained the debate was easy to win. L The persistent opponent maintained the debate over the ambiguous issue would continue. M The disagreeing students maintained the debate stubbornly. M The quarreling parents maintained the debate over their children's futures unendingly. L The journal editor printed the article was slanderous. L The famous actress printed the article on her scandalous affair was untrue. M The pretty popstar printed the article happily. M The ecstatic editor printed the article on Superman's daring adventures gleefully. L The bitter divorcee printed the statement had been irresponsible. L The annoyed author printed the statement of the callous media was ruining sales of his book. M The temperamental publisher printed the statement hastily. M The enthusiastic lawyer printed the statement of his client's complaint painstakingly. L The art critic wrote the interview was tedious. L The cynical newsperson wrote the interview with the popular celebrity was a huge success. M The pensive screenwriter wrote the interview thoughtfully. M The acclaimed writer wrote the interview with the presidential candidate beautifully. L The art collector wrote the paper was a clever forgery. L The nutty thief wrote the paper with the crumpled edges was easy to burn. M The versatile poet wrote the paper brilliantly. M The ugly madman wrote the paper on popular classical music proudly. L The surgical nurses protested the policy was not fair to patients. L The enraged mob protested the policy on public welfare issues was being overlooked. M The rebelling youngsters protested the policy fearlessly. M The ruthless instigators protested the policy on universal adult franchise without hesitating. L The hospital authorities protested the actions were not fair. L The upset families protested the actions of the renowned hospital were illegal.

No. V

N

S

WM C

71

DO SC

DO −

72

DO SC

DO +

73

DO DO SC

−

74

DO DO SC

+

75

DO DO DO −

76

DO DO DO +

77

DO SC

SC

−

78

DO SC

SC

+

79

DO SC

DO −

80

DO SC

DO +

81

EQ

DO SC

−

82

EQ

DO SC

+

83

EQ

DO DO −

84

EQ

DO DO +

85

EQ

SC

SC

−

86

EQ

SC

SC

+

87

EQ

SC

DO −

88

EQ

SC

DO +

89

EQ

DO SC

−

90

EQ

DO SC

+

91

EQ

DO DO −

92

EQ

DO DO +

93

EQ

SC

SC

−

94

EQ

SC

SC

+

95

EQ

SC

DO −

96

EQ

SC

DO +

Sentence

M The underpaid surgeons protested the actions vigorously. M The overworked staff protested the actions of the new doctor for a week. L The trained referees warned the spectators were rowdy. L The security personnel warned the spectators at the state championships were agitated. M The performing artist warned the spectators before starting the show. M The weather forecasters warned the spectators of the baseball game in advance. L The news commentators warned the nation would be vulnerable. L The impartial secretary warned the nation in such bad times would face financial problems. M The ancient philosopher warned the nation a thousand years ago. M The zealous patriot warned the nation with the uncaring aristocracy well in advance. M The overjoyed fiancée announced the wedding was a big event. M The wedding planners announced the wedding with all its trimmings was costly. M The eloped couple announced the wedding the next day. M The inquisitive reporter announced the wedding of Mary and John without permission. M The disappointed client announced the flowers were wilted. M The blushing schoolgirl announced the flowers with the lovely scent were beautiful. M The friendly florist announced the flowers with pride. M The charming housemaid announced the flowers in the pretty basket in a booming voice. M The new mayor declared the holiday was a festive occasion. M The stern grandmother declared the holiday of over a week was unnecessary. M The respected senator declared the holiday happily. M The exhausted mother declared the holiday before the final exams without regret. M The road minister declared the winner was absent. M The exasperated driver declared the winner of the new limousine was ungrateful. M The amused pedestrians declared the winner unanimously. M The naval officer declared the winner of the Grand Prix on the day of the event.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Appendix A (continued)

Appendix A (continued) No. V

N

S

97

EQ

DO SC

−

98

EQ

DO SC

+

99

EQ

DO DO −

100 EQ

DO DO +

101 EQ

SC

SC

−

102 EQ

SC

SC

+

103 EQ

SC

DO −

104 EQ

SC

DO +

WM C

105 EQ

DO SC

−

106 EQ

DO SC

+

107 EQ

DO DO −

108 EQ

DO DO +

109 EQ

SC

SC

−

110 EQ

SC

SC

+

111 EQ

SC

DO −

112 EQ

SC

DO +

113 EQ

DO SC

−

114 EQ

DO SC

+

115 EQ

DO DO −

116 EQ

DO DO +

117 EQ

SC

SC

−

118 EQ

SC

SC

+

119 EQ

SC

DO −

120 EQ

SC

DO +

121 EQ

DO SC

−

122 EQ

DO SC

+

123 EQ

DO DO −

124 EQ

DO DO +

371

Sentence

M The crooked politician denied the accusation was true. M The sales clerk denied the accusation of grand jewelry theft was valid. M The flower girl denied the accusation flatly. M The guilty servant denied the accusation of stealing precious goods strongly. M The defense attorney denied the election was rigged. M The obstinate judges denied the election of the twenty contestants was pre-determined. M The previous president denied the election emphatically. M The insecure monarch denied the election of a democratic council immediately. M The gifted ice-skater doubted the judges were fair. M The nervous defendant doubted the judges of the Supreme Court were unbiased. M The skeptical superintendent doubted the judges from the very beginning. M The shrewd president doubted the judges of the state government highly. M The famous singer doubted the storm would stop the show. M The weather channel doubted the storm with its torrential downpours would cease. M The arrogant pilot doubted the storm after looking at the sky. M The first mate doubted the storm of supposedly great magnitude on seeing the sunshine. M The petrified cousin feared the tantrums would intensify. M The young babysitter feared the tantrums with kicking and squealing would upset the newborn. M The gentle aunt feared the tantrums greatly. M The timid headmistress feared the tantrums of my best friend intensely. M The anxious bride feared the dress was torn. M The illustrious tailor feared the dress with the bright sequins was ripped. M The schizophrenic girl feared the dress with all her heart. M The prominent filmstar feared the dress with the low neckline very much. M The well-known spokesperson guaranteed the product was new. M The persistent salesman guaranteed the product at the new store was excellent. M The talented designer guaranteed the product personally. M The insurance company guaranteed the product with the latest modifications completely.

No. V

N

S

WM C

125 EQ

SC

SC

−

126 EQ

SC

SC

+

127 EQ

SC

DO −

128 EQ

SC

DO +

129 EQ

DO SC

−

130 EQ

DO SC

+

131 EQ

DO DO −

132 EQ

DO DO +

133 EQ

SC

SC

−

134 EQ

SC

SC

+

135 EQ

SC

DO −

136 EQ

SC

DO +

137 EQ

DO SC

-

138 EQ

DO SC

+

139 EQ

DO DO −

140 EQ

DO DO +

141 EQ

SC

SC

−

142 EQ

SC

SC

+

143 EQ

SC

DO −

144 EQ

SC

DO +

145 EQ

DO SC

−

146 EQ

DO SC

+

147 EQ

DO DO −

148 EQ

DO DO +

149 EQ

SC

SC

−

150 EQ

SC

SC

+

151 EQ

SC

DO −

152 EQ

SC

DO +

Sentence

M The anxious shopkeeper guaranteed the job would be fun. M The chic restaurant guaranteed the job of cleaning the dishes was well-paying. M The well-to-do artiste guaranteed the job herself. M The dishonest trader guaranteed the job at the Stock Exchange to his client. M The desk clerk guessed the name was scribbled. M The police sergeant guessed the name with the five syllables was fake. M The fortune-teller guessed the name in a jiffy. M The bright receptionist guessed the name of my best friend in an instant. M The intelligent boy guessed the solution was simple. M The engaged couple guessed the solution to all their problems was to get married. M The child genius guessed the solution easily. M The eccentric mathematician guessed the solution to the geometry problem at once. M The famous novelist knew the material was unusual. M The education board knew the material on advanced quantum physics was complicated. M The industrious child knew the material by heart. M The tense interviewee knew the material from the prescribed textbook by heart. M The practiced orator knew the pause was a mistake. M The fumbling actor knew the pause of three seconds was noticed. M The talented musician knew the pause immediately. M The devoted composer knew the pause of two full beats well. M The publicity agent predicted the problem was troublesome. M The worried statistician predicted the problem of too many overheads could be devastating. M The international advisors predicted the problem in advance. M The skeptical in-laws predicted the problem in the hasty marriage the first day itself. M The fashion designer predicted the clothes would sell. M The fussy supermodel predicted the clothes on the summer line would be unpopular. M The fashion guru predicted the clothes unsuccessfully. M The press reporter predicted the clothes at the fashion festival accurately. (continued on next page)

372

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Appendix A (continued)

Appendix A (continued) No. V

N

S

WM C

153 EQ

DO SC

−

154 EQ

DO SC

+

155 EQ

DO DO −

156 EQ

DO DO +

157 EQ

SC

SC

−

158 EQ

SC

SC

+

159 EQ

SC

DO −

160 EQ

SC

DO +

161 SC

DO SC

−

162 SC

DO SC

+

163 SC

DO DO −

164 SC

DO DO +

165 SC

SC

SC

−

166 SC

SC

SC

+

167 SC

SC

DO −

168 SC

SC

DO +

169 SC

DO SC

−

170 SC

DO SC

+

171 SC

DO DO −

172 SC

DO DO +

173 SC

SC

SC

−

174 SC

SC

SC

+

175 SC

SC

DO −

176 SC

SC

DO +

177 SC

DO SC

−

178 SC

DO SC

+

Sentence

M The skilled negotiator sensed the conflict was ongoing. M The marriage counselor sensed the conflict on such important issues was unsettled. M The third person sensed the conflict immediately. M The shrewd solicitor sensed the conflict in his clients' interests immediately. M The powerful government sensed the tension would be resolved. M The soldier's family sensed the tension between the warring nations was heightening. M The palm reader sensed the tension intuitively. M The perceptive teenager sensed the tension between her closest friends at once. M The sympathetic teacher admitted the schoolgirl was careless. M The caring aunt admitted the schoolgirl with the two pigtails was actually very naughty. L The selective college admitted the schoolgirl in the end. L The admissions committee admitted the schoolgirl with the excellent grades right away. M The inconsiderate pilot admitted the airplane was not on time. M The flight attendant admitted the airplane with the extra facilities was not as fast. L The control tower admitted the airplane at once. L The hangar operator admitted the airplane with the damaged propeller unquestioningly. M The office manager indicated the trouble was not insurmountable. M The police chief indicated the trouble at the main headquarters was serious. L The burnt houses indicated the trouble for all to see. L The crumbling economy indicated the trouble of the unfortunate citizens to the whole world. M The supercilious lieutenant indicated the remark was impertinent. M The stiff colonel indicated the remark about the armed forces was inappropriate. L The college professor indicated the remark with a nod. L The excelling student indicated the remark on his report card with immense pride. M The weary traveler claimed the luggage was stolen. M The airport guards claimed the luggage without the orange tag was unidentified.

No. V

N

S

179 SC

DO DO −

L

180 SC

DO DO +

L

181 SC

SC

SC

−

M

182 SC

SC

SC

+

M

183 SC

SC

DO −

L

184 SC

SC

DO +

L

185 SC

DO SC

−

M

186 SC

DO SC

+

M

187 SC

DO DO −

L

188 SC

DO DO +

L

189 SC

SC

SC

−

M

190 SC

SC

SC

+

M

191 SC

SC

DO −

L

192 SC

SC

DO +

L

193 SC

DO SC

−

M

194 SC

DO SC

+

M

195 SC

DO DO −

L

196 SC

DO DO +

L

197 SC

SC

SC

−

M

198 SC

SC

SC

+

M

199 SC

SC

DO −

L

200 SC

SC

DO +

L

201 SC

DO SC

−

M

202 SC

DO SC

+

M

203 SC

DO DO −

L

204 SC

DO DO +

L

205 SC

SC

−

M

SC

WM C

Sentence The tired passenger claimed the luggage eagerly. The brazen thief claimed the luggage with the expensive jewels without hesitation. The handicapped gentleman claimed the attendant was surly. The eight-year-old claimed the attendant in the blue dress was mean to her. The infatuated emperor claimed the attendant for his own. The turbulent sea claimed the attendant from the suburban area in a plane crash. The account executive concluded the speech was a disaster. The judicial council concluded the speech on critical legal issues was unresearched. The university head concluded the speech hastily. The overworked social worker concluded the speech on basic civil rights extremely quickly. The tax inspectors concluded the issue was unresolvable. The bankrupt client concluded the issue with the bank manager had worked against him. The docile wife concluded the issue quickly. The bankers' meeting concluded the issue with the overseas branch at once. The cab driver assumed the blame belonged to the pedestrian. The irresponsible architect assumed the blame for the building's collapse didn't lie with him. The responsible grandson assumed the blame squarely. The protective brother assumed the blame for the car accident in place of his sister. The Hollywood actress assumed the appearance was temporary. The harrowed uncle assumed the appearance of his wild nephew would soon improve. The heavyweight champion assumed the appearance reluctantly. The clever mime assumed the appearance of an old lady very convincingly. The astute jeweler figured the prices were steep. The poor farmer figured the prices of large Idaho potatoes would fall lower. The confirmed shopper figured the prices without looking at the price tags. The miserly housewife figured the prices of the latest clothes before asking the assistant. The dedicated anthropologist figured the answer was right there.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Appendix A (continued)

Appendix A (continued) No. V

N

S

WM C

206 SC

SC

SC

+

207 SC

SC

DO −

208 SC

SC

DO +

209 SC

DO SC

−

210 SC

DO SC

+

211 SC

DO DO −

212 SC

DO DO +

213 SC

SC

SC

−

214 SC

SC

SC

+

215 SC

SC

DO −

216 SC

SC

DO +

217 SC

DO SC

−

218 SC

DO SC

+

219 SC

DO DO −

220 SC

DO DO +

221 SC

SC

SC

−

222 SC

SC

SC

+

223 SC

SC

DO −

224 SC

SC

DO +

225 SC

DO SC

−

226 SC

DO SC

+

227 SC

DO DO −

228 SC

DO DO +

229 SC

SC

SC

−

230 SC

SC

SC

+

231 SC

SC

DO −

232 SC

SC

DO +

373

Sentence

M The ambitious salesgirl figured the answer to all her problems was to finish her education. L The eighth grader figured the answer eventually. L The eager archaeologist figured the answer to the temple inscriptions without delay. M The union leader implied the raise was needed. M The older employees implied the raise of a hundred dollars was welcome. L The caring boss implied the raise through indirect comments. L The new administration implied the raise in crude oil prices very subtly. M The fearless rebels implied the weather was responsible. M The agricultural community implied the weather with its unreliable nature was dangerous. L The lazy fisherman implied the weather shamelessly. L The highway patrol implied the weather of the past few days conveniently. M The young craftsman realized the mistake was grave. M The college professor realized the mistake on the midterm exam was genuine. L The foolish goldsmith realized the mistake after adding the final touches to his work. L The federal agent realized the mistake of blaming the system at that moment. M The dispassionate arbitrator realized the situation was hopeless. M The observant handyman realized the situation with the chipped tools was worsening. L The querulous wife realized the situation too late. L The foster-parents realized the situation between their two children too late. M The conservative scriptwriter suggested the scene was explicit. M The actress's mother suggested the scene with the controversial comments was unnecessary. L The celebrated author suggested the scene eagerly. L The director's cook suggested the scene with the kitchen backdrop hopefully. M The royal household suggested the mansion was unsuitable. M The overworked sweeper suggested the mansion with the four floors was unmanageable. L The married couple suggested the mansion excitedly. L The hopeful house-agent suggested the mansion with the antique furniture eagerly.

No. V

N

S

WM C

233 SC

DO SC

−

234 SC

DO SC

+

235 SC

DO DO −

236 SC

DO DO +

237 SC

SC

SC

−

238 SC

SC

SC

+

239 SC

SC

DO −

240 SC

SC

DO +

Sentence

M The factory owner suspected the workers were going on strike. M The silent attorney suspected the workers from the neighboring suburbs were objecting. L The police chief suspected the workers instinctively. L The deputy-in-charge suspected the workers at the crime scene without reason. M The prodigal son suspected the cash was thinning out. M The jealous cousin suspected the cash in his brother's care was plentiful. L The demented inmate suspected the cash nervously. L The security panel suspected the cash in the white envelope on sight.

Appendix B. Stimulus verbs The table below contains all verbs used in the study, along with their DO-preference and SC-preference (in a 0 to 1 scale). Verb bias

Verb

DO-preference

SC-preference

DO

Asserted Confirmed Emphasized Heard Insured Maintained Printed Protested Warned Wrote Admitted Assumed Claimed Concluded Figured Implied Indicated Realized Suggested Suspected Announced Declared Denied Doubted Feared Guaranteed Guessed Knew Predicted Sensed

0.64 0.71 0.75 0.74 0.84 0.74 0.78 0.58 0.74 0.87 0.09 0.09 0.06 0.14 0.08 0.06 0.21 0.14 0.18 0.29 0.49 0.44 0.33 0.42 0.32 0.46 0.39 0.31 0.45 0.53

0.31 0.25 0.19 0.16 0.13 0.23 0.01 0.11 0.11 0.00 0.60 0.89 0.69 0.80 0.46 0.90 0.70 0.77 0.61 0.68 0.48 0.52 0.27 0.56 0.48 0.50 0.25 0.46 0.51 0.45

SC

EQ

Appendix C. Stimulus nouns The tables below contain all nouns used in the study, along with the verb that precedes them, as well as the post-verbal noun

374

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

plausibility ratings. A high score (maximum: 7) means that the post-verbal noun is a plausible direct object (DO) of its predecessing verb; a low score indicates an SC-biased noun. Nouns following DO-bias verbs

Verb

Noun

Rating

Feared

Verb

Noun

Rating

Guessed

Asserted

Belief Truth Rumor Money Shortcomings Schools Story Neighbors House River Machinery Debate Article Statement Interview Paper Policy Actions Spectators Nation

5.06 2.42 6.89 4.24 6.38 3.71 6.53 3.97 7.00 2.47 6.35 3.00 6.75 3.18 6.11 3.59 6.58 2.86 6.51 3.74

Knew

Tantrums Dress Product Job Name Solution Material Pause Problem Clothes Conflict Tension

6.47 3.65 6.79 4.17 6.26 3.73 6.79 2.54 6.53 2.74 6.74 4.22

Confirmed Emphasized Heard Insured Maintained Printed Wrote Protested Warned

Nouns following SC-bias verbs Verb

Noun

Rating

Admitted

Schoolgirl Airplane Blame Appearance Luggage Attendant Speech Issue Prices Answer Raise Weather Trouble Remark Mistake Situation Scene Mansion Workers Cash

6.11 2.12 6.35 2.67 6.89 2.29 6.39 3.72 6.42 3.58 4.95 2.29 6.07 3.46 6.41 2.16 5.93 2.31 6.58 3.29

Assumed Claimed Concluded Figured Implied Indicated Realized Suggested Suspected

Nouns following EQ-bias verbs Verb

Noun

Rating

Announced

Wedding Flowers Holiday Winner Accusation Election Judges Storm

6.76 2.32 6.08 3.46 6.83 3.49 6.47 3.59

Declared Denied Doubted

Guaranteed

Predicted Sensed

Appendix D. Verb and noun analyses D.1. Verb type We investigated effects associated with manipulating the type of verb. These contrasts examined activations during the final phrase minus the initial, unbiased phrase. Because of the number of items, there was insufficient power to analyze each of these types of verbs in each sentence context. Activation patterns for these contrasts are illustrated in Fig. 2, and the location and statistical attributes of the peak voxel in each cluster are summarized in Table A1. For stimuli with a DO-biased verb, we observed bilateral IFC and right dlPFC activation (Fig. 2A). For stimuli with an SC verb, we observed activation in left IFC and bilateral dlPFC (Fig. 2B). Bilateral IFC activation remains after direct subtraction of “SC verb” stimuli from “DO verb” stimuli (Fig. 2C). Left dlPFC activation remains after direct subtraction of “DO verb” stimuli from “SC verb” stimuli (Fig. 2D). Different activation patterns emerge for DO verbs and SC verbs. The former type expects to be followed by a noun that fills the grammatical role of direct object and the thematic role of theme (or object) of the action described by the verb. This is the canonical form of a sentence in English, which is an SVO (subject-verbobject) language. This may be consistent with the possibility that, in comparison to SC verbs, DO verbs should elicit higher activation in the core language region-Broca's area. Other studies have shown activation of the right hemisphere homologue of Broca's area during sentence processing (Carpenter et al., 1999; Embick et al., 2000; Just et al., 1996). One proposal relates this to greater grammatical difficulty (Just et al., 1996), although it is not clear that DO verbs are more demanding than SC verbs. SC verbs, on the other hand, usually entail, or presuppose, a more complex predicate-argument structure (Shapiro et al., 1987, 1989), having more processing nodes involved in their representation and thus a more complex binding (Chomsky, 1981). We observed increased left dlPFC activation elicited by this type of verbs, when compared to DO verbs. This finding may suggest that a strategy is being devised to interpret the sentence with a more complex syntactical framework in mind. In our behavioral analysis, we observed no difference in verb reading times during verb presentation per se, which was expected, since verb types did not differ significantly in length. However, we measured a significant reaction time difference during sentence disambiguation when taking into consideration the sentential context in which each verb type is embedded, and thus the consistency level of the stimulus (i.e. “more consistent” or “less consistent”), as discussed in the main text.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

375

Fig. 2. Distribution of activation during the last phrase (relative to the first phrase) for stimuli with a DO verb (A) and with an SC verb (B). The bottom two panels depict the contrasts of these two conditions: “DO verb” stimuli minus “SC verb” stimuli (C) and “SC verb” stimuli minus “DO verb” stimuli (D).

D.2. Verb–noun consistency We also investigated the effect of manipulating the consistency between the verb and the post-verbal noun. Behavioral analyses show that subjects take longer to process sentences in which the noun is less consistent with its preceding verb (mean (± SD) = 729.19 (± 273.39) ms) than to process stimuli in which the postverbal noun is more consistent with the verb (mean (± SD) = 665.15 (± 247.95) m). Activation patterns for these stimuli are illustrated in Fig. 3, and the location and statistical attributes of the peak voxel in each cluster are summarized in Table A2. For stimuli Table A1 Neural activation patterns for “DO verb” and “SC verb” stimuli Contrast

Activation locus

Brodmann area (BA)

Coordinates (x, y, z)

z-score

DO verb a

L IFC R IFC R dlPFC L dlPFC R dlPFC L IFC L IFC R IFC L dlPFC

47 47 9 9 6 47 47 47 8

(−40, 19, − 11) (40, 11, − 14) (44, 25, 28) (−48, 13, 32) (24, − 5, 59) (−40, 19, − 11) (−44, 19, − 8) (44, 23, − 1) (−32, 18, 43)

4.64 4.46 4.17 3.77 3.42 4.17 3.34 3.18 3.13

SC verb a

DO verb–SC verb b SC verb–DO verb b

a These findings involve contrasts of the phrase where the structural ambiguity is resolved minus the initial, unbiased phrase of the sentence. b This contrast involves the resolving phrase minus the initial phrase of each type of sentence.

Fig. 3. Distribution of activation during the last phrase (relative to the first phrase) for stimuli in which the noun is “more consistent” with the preceding verb (A) and in which the noun is “less consistent” with the verb (B). Panel C depicts the contrast of these two conditions (“less consistent verb–noun” minus “more consistent verb–noun”).

in which the noun is more consistent with its preceding verb, we observed bilateral middle temporal cortex (MTC) and left IFC activation (Fig. 3A). For stimuli in which the noun is less Table A2 Neural activation patterns for “more consistent verb–noun” and “less consistent verb–noun” stimuli Contrast

Activation locus

Brodmann area (BA)

Coordinate (x, y, z)

z-score

More consistent VN a

R MTC L MTC L IFC L dlPFC L IFC L STC L PCC

21 21 47 9 45 39 29

(55, − 47, 2) (− 67, − 28, − 8) (− 32, 19, −4) (− 51, 17, 29) (− 51, 24, 21) (− 59, − 61, 21) (− 16, − 49, 21)

3.85 3.43 3.18 3.09 3.05 3.16 3.11

Less consistent VN a Less consistent VN–more consistent VN b

a These findings involve contrasts of the phrase where the structural ambiguity is resolved minus the initial, unbiased phrase of the sentence. b This contrast involves the resolving phrase minus the initial phrase of each type of sentence.

376

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

consistent with the verb, we observed activation in left dlPFC and in left IFC, although the latter was just below the significant threshold (Fig. 3B). Distinct activation patterns emerge for conditions in which the nominal phrase is more consistent or less consistent with its preceding verbal phrase. The former condition elicits higher activation in language regions-Broca's area and Wernicke's area (as well as its right homologue). One possibility is that when the noun fills the grammatical role it was expected to play, core language regions are more strongly activated. It is important to mention that both conditions recruit left IFC, as expected in a sentence processing task. In the “less consistent verb–noun” condition, however, left IFC activity is just below the significant threshold (p b 0.0012; z-score = 3.05) and is thus not depicted in the figure. On the other hand, when the noun is less consistent with the preceding verb, the thematic expectations are not met, and thus the reader has to re-construct the representation of the sentence. This additional processing should be translated into recruitment of additional executive resources. We observed increased left dlPFC activation for these stimuli, possibly reflecting the planning needed to resolve the “less consistent verb–noun” condition. We subsequently compared these two conditions directly, by performing direct subtractions of these contrasts. No areas showed higher activation levels for the direct subtraction “more consistent verb–noun” minus “less consistent verb–noun” stimuli. Direct subtraction of “less consistent verb–noun” minus “more consistent verb–noun” stimuli elicited activation in left superior temporal cortex (STC) and left posterior cingulate cortex (PCC). Activation patterns for these stimuli are illustrated in Fig. 3, and the location and statistical attributes of the peak voxel in each cluster are summarized in Table A2. References Amunts, K., Schleicher, A., Burgel, U., Mohlberg, H., Uylings, H., Zilles, K., 1999. Broca's region revisited: cytoarchitecture and intersubject variability. J. Comp. Neurol. 412 (2), 319–341. Awh, E., Jonides, J., Smith, E.E., Schumacher, E.H., Koeppe, R.A., Katz, S., 1996. Dissociation of storage and rehearsal in verbal working memory: evidence from positron emission tomography. Psychol. Sci. 7 (1), 25–31. Baddeley, A.D., 1992. Working memory. Science 255 (5044), 556–559. Barch, D.M., Braver, T.S., Nystrom, L.E., Forman, S.D., Cohen, J.D., 1997. Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia 35 (10), 1373–1380. Ben-Shachar, M., Palti, D., Grodzinsky, Y., 2004. Neural correlates of syntactic movement: converging evidence from two fMRI experiments. NeuroImage 21 (4), 1320–1336. Braver, T.S., Bongiolatti, S.R., 2002. The role of frontopolar cortex in subgoal processing during working memory. NeuroImage 15 (3), 523–536. Braver, T.S., Cohen, J.D., Nystrom, L.E., Jonides, J., Smith, E.E., Noll, D.C., 1997. A parametric study of prefrontal cortex involvement in human working memory. NeuroImage 5 (1), 49–62. Braver, T.S., Barch, D.M., Kelley, W.M., Buckner, R.L., Cohen, N.J., Miezin, F. M., et al., 2001. Direct comparison of prefrontal cortex regions engaged by working and long-term memory tasks. NeuroImage 14 (1), 48–59. Callicott, J.H., Mattay, V.S., Bertolino, A., Finn, K., Coppola, R., Frank, J.A., Goldberg, T.E., Weinberger, D.R., 1999. Physiological characteristics of capacity constraints in working memory as revealed by functional MRI. Cereb. Cortex 9 (1), 20–26. Caplan, D., Alpert, N., Waters, G.S., 1998. Effects of syntactic structure and

proposition number on patterns of regional cerebral blood flow. J. Cogn. Neurosci. 10 (4), 541–552. Caplan, D., Alpert, N., Waters, G.S., 1999. PET studies of syntactic processing with auditory sentence presentation. NeuroImage 9 (3), 343–351. Caplan, D., Alpert, N., Waters, G.S., Olivieri, A., 2000. Activation of Broca's area by syntactic processing under conditions of concurrent articulation. Hum. Brain Mapp. 9 (2), 65–71. Caplan, D., Waters, G.S., Alpert, N., 2003. Effects of age and speed of processing on rCBF correlates of syntactic processing in sentence comprehension. Hum. Brain Mapp. 19 (2), 112–131. Carpenter, P.A., Just, M.A., Keller, T.A., Eddy, W.F., Thulborn, K.R., 1999. Time course of fMRI-activation in language and spatial networks during sentence comprehension. NeuroImage 10, 216–224. Chein, J.M., Fiez, J.A., 2001. Dissociation of verbal working memory system components using a delayed serial recall task. Cereb. Cortex 11 (11), 1003–1014. Chomsky, N., 1981. Lectures on Government and Binding: The Pisa Lectures. Mouton de Gruyter. Cohen, J.D., Perlstein, W.M., Braver, T.S., Nystrom, L.E., Noll, D.C., Jonides, J., Smith, E.E., 1997. Temporal dynamics of brain activation during a working memory task. Nature 386 (6625), 604–608. Connine, C., Ferreira, F., Jones, C., Clifton, C., Frazier, L., 1984. Verb frame preferences: descriptive norms. J. Psycholinguist. Res. 13 (4), 307–319. Cooke, A., Zurif, E.B., DeVita, C., Alsop, D., Koenig, P., Detre, J., Gee, J., Pinango, M., Balogh, J., Grossman, M., 2001. Neural basis for sentence comprehension: grammatical and short-term memory components. Hum. Brain Mapp. 15 (2), 80–94. Cooke, A., Grossman, M., DeVita, C., Gonzalez-Atavales, J., Moore, P., Chen, W., Gee, J., Detre, J., 2006. Large-scale neural network for sentence processing. Brain Lang. 96 (1), 14–36. Earley, J., 1970. An efficient context-free parsing algorithm. Commun. ACM 13 (2), 94–102. Elsness, J., 1984. That or zero? A look at the choice of object clause connective in a corpus of American English. Eng. Stud. 65 (6), 519–533. Embick, D., Marantz, A., Miyashita, Y., O'Neil, W., Sakai, K.L., 2000. A syntactic specialization for Broca's area. Proc. Natl. Acad. Sci. 97 (11), 6150–6154. Evans, A.C., Collins, D.L., Mills, S.R., Brown, E.D., Kelly, R.L., Peters, T.M., 1993. 3D statistical neuroanatomical models from 305 MRI volumes. In: Klaisner, L.A. (Ed.), Nuclear Science Symposium and Medical Imaging Conference. IEEE Conference Record, vol. 1–3, pp. 1813–1817. Ferreira, F., Clifton Jr., C., 1986. The independence of syntactic processing. J. Mem. Lang. 25 (3), 348–368. Ferreira, F., Henderson, J.M., 1990. Use of verb information in syntactic parsing: evidence from eye movements and word-by-word self-paced reading. J. Exper. Psychol., Learn, Mem., Cogn. 16 (4), 555–568. Ferreira, F., Henderson, J.M., 1991. How is verb information used during syntactic parsing. In: Simpson, G.B. (Ed.), Understanding Word and Sentence. Elsevier, North-Holland, pp. 305–330. Fiebach, C.J., Schlesewsky, M., Friederici, A.D., 2001. Syntactic working memory and the establishment of filler-gap dependencies: insights from ERPs and fMRI. J. Psycholinguist. Res. 30 (3), 321–338. Fiebach, C.J., Schlesewsky, M., Lohmann, G., Cramon, D.Y., Friederici, A.D., 2005. Revisiting the role of Broca's area in sentence processing: syntactic integration versus syntactic working memory. Hum. Brain Mapp. 24 (2), 79–91. Frackowiak, R.S.J., Friston, K.J., Frith, C.D., Dolan, R.J., Mazziotta, J.C., 1997. Human Brain Function. Academic Press, San Diego. Francis, W.N., Kucera, H., 1982. Frequency Analysis of English Usage: Lexicon and Grammar. Houghton-Mifflin, Boston. Frazier, L., Rayner, K., 1982. Making and correcting errors during sentence comprehension: eye movements in the analysis of structurally ambiguous sentences. Cogn. Psychol. 14 (2), 178–210.

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378 Friederici, A.D., 2002. Towards a neural basis of auditory sentence processing. Trends Cogn. Sci. 6 (2), 78–84. Friederici, A.D., Meyer, M., von Cramon, D.Y., 2000. Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information. Brain Lang. 74 (2), 289–300. Friedman, H.R., Goldman-Rakic, P.S., 1994. Coactivation of prefrontal cortex and inferior parietal cortex in working memory tasks revealed by 2DG functional mapping in the rhesus monkey. J. Neurosci. 14 (5), 2775–2788. Friston, K.J., Josephs, O., Rees, G., Turner, R., 1998. Nonlinear eventrelated responses in fMRI. Magn. Res. Med. 39 (1), 41–52. Friston, K.J., Josephs, O., Zarahn, E., Holmes, A.P., Rouquette, S., Poline, J.-B., 2000. To smooth or not to smooth? Bias and efficiency in fMRI time-series analysis. NeuroImage 12 (2), 196–208. Garavan, H., Ross, T.J., Li, S.-J., Stein, E.A., 2000. A parametric manipulation of central executive functioning. Cereb. Cortex 10 (6), 585–592. Garnsey, S.M., Pearlmutter, N.J., Myers, E., Lotocky, M.A., 1997. The contributions of verb bias and plausibility to the comprehension of temporarily ambiguous sentences. J. Mem. Lang. 37 (1), 58–93. Gibson, E., 1998. Linguistic complexity: locality of syntactic dependencies. Cognition 68 (1), 1–76. Gibson, E., Pearlmutter, N.J., 1998. Constraints on sentence comprehension. Trends Cogn. Sci. 2 (7), 262–268. Gibson, E., Pearlmutter, N.J., 2000. Distinguishing serial and parallel parsing. J. Psycholinguist. Res. 29 (2), 231–240. Grossman, M., Cooke, A., DeVita, C., Alsop, D., Detre, J., Gee, J.C., 2002. Age-related changes in working memory during sentence comprehension: an fMRI study. NeuroImage 15 (2), 302–317. Holmes, A.P., Friston, K.J., 1998. Generalizability, random effects, and population interference. NeuroImage 7, S754. Holmes, V.M., 1987. Syntactic parsing: in search of the garden path. In: Coltheart, M. (Ed.), Attention and Performance XII. Laurence Erlbaum Associates, London, UK. Indefrey, P., Hagoort, P., Herzog, H., Seitz, R.J., Brown, C.M., 2001. Syntactic processing in left prefrontal cortex is independent of lexical meaning. NeuroImage 14 (3), 546–555. Inui, T., Otsu, Y., Tanaka, S., Okada, T., Nishizawa, S., Konishi, J., 1998. A functional MRI analysis of comprehension processes of Japanese sentences. NeuroReport 9 (14), 3325–3328. Jonides, J., Smith, E.E., Koeppe, R.A., Awh, E., Minoshima, S., Mintun, M.A., 1993. Spatial working memory in humans as revealed by PET. Nature 363, 623–625. Jonides, J., Schumacher, E.H., Smith, E.E., Koeppe, R.A., Awh, E., ReuterLorenz, P.A., Marshuetz, C., Willis, C.R., 1998. The role of parietal cortex in verbal working memory. J. Neurosci. 18 (13), 5026–5034. Jurafsky, D., 1996. A probabilistic model of lexical and syntactic access and disambiguation. Cogn. Sci. 20 (2), 137–194. Just, M.A., Carpenter, P.A., 1992. A capacity theory of comprehension: individual differences in working memory. Psychol. Rev. 99 (1), 122–149. Just, M.A., Carpenter, P.A., Woolley, J.D., 1992. Paradigms and processes in reading comprehension. J. Exp. Psychol. Gen. 111 (2), 228–238. Just, M.A., Carpenter, P.A., Keller, T.A., Eddy, W.F., Thulborn, K.R., 1996. Brain activation modulated by sentence comprehension. Science 274 (5284), 114–116. Kang, A.M., Constable, R.T., Gore, J.C., Avrutin, S., 1999. An event-related fMRI study of implicit phrase-level syntactic and semantic processing. NeuroImage 10 (5), 555–561. Keller, T.A., Carpenter, P.A., Just, M.A., 2001. The neural bases of sentence comprehension: a fMRI examination of syntactic and lexical processing. Cereb. Cortex 11 (3), 223–237. Kuperberg, G., McGuire, P.K., Bullmore, E.T., Brammer, M., RabeHesketh, S., Wright, I.C., et al., 2000. Common and distinct neural substrates for pragmatic, semantic, and syntactic processing of spoken sentences: an fMRI study. J. Cogn. Neurosci. 12 (2), 321–341. Kuperberg, G., Holcomb, P.J., Sitnikova, T., Greve, D., Dale, A.M., Caplan,

377

D., 2003. Distinct patterns of neural modulation during the processing of conceptual and syntactic anomalies. J. Cogn. Neurosci. 15 (2), 272–293. Lewis, R.L., 2000. Falsifying serial and parallel parsing models: empirical conundrums and an overlooked paradigm. J. Psycholinguist. Res. 29 (2), 241–248. MacDonald, M.C., Pearlmutter, N.J., Seidenberg, M.S., 1994. The lexical nature of syntactic ambiguity resolution. Psychol. Rev. 101 (4), 676–703. MacDonald, A.W., Cohen, J.D., Stenger, V.A., Carter, C.S., 2000. Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 288 (5472), 1835–1838. Marshuetz, C., Smith, E.E., Jonides, J., DeGutis, J., Chenevert, T.L., 2000. Order information in working memory: fMRI evidence for parietal and prefrontal mechanisms. J. Cogn. Neurosci. 12 (Supp 2), 130–144. Mason, R.A., Just, M.A., Keller, T.A., Carpenter, P.A., 2003. Ambiguity in the brain: what brain imaging reveals about the processing of syntactically ambiguous sentences. J. Exper. Psychol., Learn, Mem., Cogn. 29 (6), 1319–1338. McDavid, V., 1964. The alternation of “that” and zero in noun clauses. Am. Speech 39 (2), 102–113. Mesulam, M.-M., van Hoesen, G.W., Pandya, D.N., Geschwind, N., 1977. Limbic and sensory connections of the inferior parietal lobule (area PG) in the rhesus monkey: a study with a new method for horseradish peroxidase histochemistry. Brain Res. 136 (3), 393–414. Michael, E.B., Keller, T.A., Carpenter, P.A., Just, M.A., 2001. fMRI investigation of sentence comprehension by eye and by ear: modality fingerprints on cognitive processes. Hum. Brain Mapp. 13 (4), 239–252. Moro, A., Tettamanti, M., Perani, D., Donati, C., Cappa, S.F., Fazio, F., 2001. Syntax and the brain: disentangling grammar by selective anomalies. NeuroImage 13 (1), 110–118. Newman, A.J., Pancheva, R., Ozawa, K., Neville, H.J., Ullman, M.T., 2001. An event-related fMRI study of syntactic and semantic violations. J. Psycholinguist. Res. 30 (3), 339–364. Newman, S.D., Just, M.A., Keller, T.A., Roth, J., Carpenter, P.A., 2003. Differential effects of syntactic and semantic processing on the subregions of Broca's area. Cogn. Brain Res. 16 (2), 297–307. Ni, W., Constable, R.T., Mencl, W.E., Pugh, K.R., Fulbright, R.K., Shaywitz, S.E., et al., 2000. An event-related neuroimaging study distinguishing form and content in sentence processing. J. Cogn. Neurosci. 12 (1), 120–133. Noppeney, U., Price, C.J., 2004. An fMRI study of syntactic adaptation. J. Cogn. Neurosci. 16 (4), 702–713. Pandya, D.N., Seltzer, B., 1982. Intrinsic connections and architectonics of posterior parietal cortex in the rhesus monkey. J. Comp. Neurol. 204 (2), 196–210. Paulesu, E., Frith, C.D., Frackowiak, R.S.J., 1993. The neural correlates of the verbal component of working memory. Nature 362 (6418), 342–345. Paulus, M.P., Hozack, N., Zauscher, B., McDowell, J.E., Frank, L., Brown, G.G., Braff, D.L., 2001. Prefrontal, parietal, and temporal cortex networks underlie decision-making in the presence of uncertainty. NeuroImage 13 (1), 91–100. Pearlmutter, N.J., Mendelsohn, A.A., 1999. Serial Versus Parallel Sentence Processing. Manuscript Northeastern University. Penny, W.D., Holmes, A.P., 2003. Random effects analysis, In: Frackowiak, R.S.J., Friston, K.J., Frith, C., Dolan, R., Price, C.J., Zeki, S., Ashburner, J., Penny, W.D. (Eds.), Human Brain Function, 2nd edition. Academic Press. Petrides, M., Pandya, D.N., 1984. Projections to the frontal cortex from the posterior parietal region in the rhesus monkey. J. Comp. Neurol. 228 (1), 105–116. Prabhakaran, V., Narayanan, K., Zhao, Z., Gabrieli, J.D., 2000. Integration of diverse information in working memory within the frontal lobe. Nat. Neurosci. 3 (1), 85–90. Rowe, J.B., Passingham, R.E., 2001. Working memory for location and time: activity in prefrontal area 46 relates to selection rather than maintenance in memory. NeuroImage 14 (1), 77–86. Saffran, E.M., Marin, S.M., 1975. Immediate memory for word lists and

378

S. Novais-Santos et al. / NeuroImage 37 (2007) 361–378

sentences in a patient with deficient auditory short-term memory. Brain Lang. 2 (4), 420–433. Savage, C.R., Deckersbach, T., Heckers, S., Wagner, A.D., Schacter, D.L., Alpert, N., et al., 2001. Prefrontal regions supporting spontaneous and directed application of verbal learning strategies. Brain 124 (1), 219–231. Seger, C.A., Poldrack, R.A., Prabhakaran, V., Zhao, M., Glover, G.H., Gabrieli, J.D.E., 2000. Hemispheric asymmetries and individual differences in visual concept learning as measured by functional MRI. Neuropsychologia 38 (9), 1316–1324. Selemon, L.D., Goldman-Rakic, P.S., 1988. Common cortical and subcortical targets of the dorsolateral prefrontal and posterior parietal cortices in the rhesus monkey: evidence for a distributed neural network subserving spatially guided behavior. J. Neurosci. 8 (11), 4049–4068. Shallice, T., Vallar, G., 1990. The impairment of auditory–verbal short-term storage. In: Vallar, G., Shallice, T. (Eds.), Neuropsychological Impairments of Short-term Memory. Cambridge Univ Press, Cambridge, UK. Shapiro, L.P., Zurif, E., Grimshaw, J., 1987. Sentence processing and the mental representation of verbs. Cognition 27 (3), 219–246. Shapiro, L.P., Zurif, E.B., Grimshaw, J., 1989. Verb processing during sentence comprehension: contextual impenetrability. J. Psycholinguist. Res. 18 (2), 223–243. Smith, E.E., Jonides, J., 1998a. Components of verbal working memory: evidence from neuroimaging. Proc. Natl. Acad. Sci. U. S. A. 95 (3), 876–882. Smith, E.E., Jonides, J., 1998b. Neuroimaging analyses of human working memory. Proc. Natl. Acad. Sci. U. S. A. 95 (20), 12061–12068. Smith, E.E., Jonides, J., 1999. Storage and executive processes in the frontal lobes. Science 283 (5408), 1657–1661. Smith, E.E., Jonides, J., 2002. Executive control and thought, In: Squire, L. (Ed.), Fundamental Neuroscience, 2nd edition. Academic Press, New York, pp. 1377–1394. Smith, E.E., Jonides, J., Koeppe, R.A., 1996. Dissociating verbal and spatial working memory using PET. Cereb. Cortex 6 (1), 11–20. Smith, E.E., Geva, A., Jonides, J., Miller, A., Reuter-Lorenz, P.A., Koeppe, R., 2001. The neural basis of task-switching in working memory: effects of performance and aging. Proc. Natl. Acad. Sci. U. S. A. 98 (4), 2095–2100. Spivey, M.J., Tanenhaus, M.K., 1998. Syntactic ambiguity resolution in discourse: modeling the effects of referential context and lexical frequency. J. Exper. Psychol., Learn, Mem., Cogn. 24 (6), 1521–1543.

Stevenson, S., 1994. Competition and recency in a hybrid network model of syntactic disambiguation. J. Psycholinguist. Res. 23 (4), 295–322. Stowe, L.A., Broere, C.AJ., Paans, A.MJ., Wijers, A.A., Mulder, G., Vaalburg, W., Zwarts, F., 1998. Localizing components of a complex task: sentence processing and working memory. NeuroReport 9 (13), 2995–2999. Stromswold, K., Caplan, D., Alpert, N., Rauch, S., 1996. Localization of syntactic comprehension by positron emission tomography. Brain Lang. 52 (3), 452–473. Sylvester, C.-Y.C., Wager, T.D., Lacey, S.C., Hernandez, L., Nichols, T.E., Smith, E.E., et al., 2003. Switching attention and resolving interference: fMRI measures of executive functions. Neuropsychologia 41 (3), 357–370. Talairach, J., Tournoux, P., 1988. Co-Planar Stereotaxic Atlas of the Human Brain. Thieme Medical Publishing Company, New York. Thompson, S., Mulac, A., 1991. The discourse conditions for the use of the complementizer “that” in conversational English. J. Pragmat. 15 (3), 237–251. Tramo, M.J., Baynes, K., Volpe, B.T., 1988. Impaired syntactic comprehension and production in Broca's aphasia: CT lesion localization and recovery patterns. Neurology 38 (1), 95–98. Traxler, M., Pickering, M., Clifton Jr., C., 1998. Adjunct attachment is not a form of lexical ambiguity resolution. J. Mem. Lang. 39 (4), 558–592. Trueswell, J.C., Tanenhaus, M.K., Kello, C., 1993. Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. J. Exper. Psychol., Learn, Mem., Cogn. 19 (3), 528–553. Trueswell, J.C., Tanenhaus, M.K., Garnsey, S.M., 1994. Semantic influences on parsing: use of thematic role information in syntactic ambiguity resolution. J. Mem. Lang. 33 (3), 285–318. Ullsperger, M., von Cramon, D.Y., 2006. The role of intact frontostriatal circuits in error processing. J. Cogn. Neurosci. 18 (4), 651–664. Warrington, E.K., Logue, V., Pratt, R.T.C., 1971. The anatomical localization of selective impairment of auditory verbal short-term memory. Neuropsychologia 9 (4), 377–387. Wingfield, A., Grossman, M., 2006. Language and the aging brain: patterns of neural compensation revealed by functional brain imaging. J. Neurophysiol. 96 (6), 2830–2839. Zurif, E.B., 1996. Brain regions of relevance to syntactic processing, In: Osherson, D. (Ed.), An Invitation to Cognitive Science, 2nd edition. MIT Press, Cambridge, pp. 381–397.