- Email: [email protected]
Grammaticality judgments on sentences with and without movement of phrasal constituents—an event-related fMRI study
Journal of Neurolinguistics 16 (2003) 301–314 www.elsevier.com/locate/jneuroling
Grammaticality judgments on sentences with and without movement of phrasal constituents—an event-related fMRI study Isabell Wartenburgera,b,*, Hauke R. Heekerena,c, Frank Burchertb, Ria De Bleserb, Arno Villringera a
Department of Neurology, Charite´, Humboldt-University, Schumannstr. 20-21, 10117 Berlin, Germany b Department of Patholinguistics, University of Potsdam, PF: 601553, 14415 Potsdam, Germany c Laboratory of Brain and Cognition, NIMH, NIH, Bldg. 10, Room 1D80, Bethesda, MD 20892-1148, USA
Abstract One of the leading neurolinguistic theories of syntactic comprehension disorders in agrammatic aphasic subjects—the Trace Deletion Hypothesis—postulates a specific impairment in processing syntactic chains, and that this function is mediated by Broca’s area. The aim of this study was to investigate whether the specific involvement of Broca’s area in processing syntactic traces can be verified using functional brain imaging. We used event-related functional magnetic resonance imaging (fMRI) while healthy subjects were asked to judge the grammaticality of visually presented sentences with and without movement of phrasal constituents. During both kinds of sentences, fMRI showed activation in language-related brain regions. Comparing both kinds of sentences did not result in differential brain activation of left frontal or temporal regions. In particular, Broca’s area was similarly activated in processing both moved and nonmoved sentences. Thus, Broca’s area seems to be involved in general syntactic processing as required by grammaticality judgments rather than having a specific function in transmitting syntactic relations. q 2003 Elsevier Science Ltd. All rights reserved. Keywords: Agrammatism; Broca’s area; Event-related functional magnetic resonance imaging; Grammaticality judgments; Syntactic traces; Trace Deletion Hypothesis
1. Introduction Interest in the neuropsychology of syntax was originally linked to the condition of agrammatic aphasia (Pick, 1898). In modern research, this condition was usually described * Corresponding author. Address: Department of Neurology, Clinical Research Group, Charite´, HumboldtUniversity, Schumannstr. 20/21, 10117 Berlin, Germany. Tel.: þ 49-30-450-560-194; fax: þ49-30-450-560-952. E-mail address: [email protected] (I. Wartenburger). 0911-6044/03/$ - see front matter q 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0911-6044(03)00028-9
302
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
as a syntactic impairment restricted to language production. Since Caramazza and Zurif’s (1976) seminal paper, it has been shown that in the majority of subjects with agrammatic speech, syntactic comprehension is affected as well. Given that semantic processing, on the contrary, was relatively preserved in comprehension as well as production, it was argued that agrammatism was characterized by a specific central syntactic impairment (Berndt & Caramazza, 1980) and that it provides evidence for the dissociability of syntax and semantics. Another pioneering paper by Linebarger, Schwartz, and Saffran (1983) disagreed with the view that syntactic representations were lost in agrammatism. The authors showed that their agrammatic subjects, who were seriously impaired in syntactic comprehension when examined with standard sentence – picture matching tasks, had relatively normal performance on judgments of grammaticality. They interpreted this as counterevidence for a total syntactic loss account, since syntactic processing was impaired under certain conditions but not others. When semantic and syntactic processing compete for the same resources, such as in sentence – picture matching tasks, which require semantic as well as syntactic interpretation, there is a tradeoff in favor of semantics. In judgments of grammaticality, however, no semantic interpretation is required and syntax is able to operate practically in isolation, thus supporting the processing account of agrammatism. However, the introduction of newer linguistic insights into aphasiology reintroduced the notion of a representational syntactic impairment in agrammatism. In his Trace Deletion Theory (TDH) of agrammatism, Grodzinsky (1986, 1995) relied on distinctions made in Chomsky’s (1981) linguistic theory, in particular, the Government and Binding framework. In this framework, a grammatical transformation involves the movement of a constituent from its base position to another derived position in the sentence. It is assumed that the former original position is substituted by a phonetically empty but syntactically effective trace. Thus, the trace is an abstract position marker which is related to the displaced constituent via co-indexation and transmits its thematic role in case of argument movement. Therefore, the trace is necessary for correct comprehension of sentences with moved phrasal constituents. With respect to agrammatism, the TDH assumes a partial but specific syntactic impairment. It postulates that the syntactic traces, which are left behind when constituents are moved, are deleted from the agrammatic syntactic representations. Therefore, agrammatic subjects should show relatively good performance in the comprehension of semantically reversible canonical sentences which do not involve movement (e.g. actives, subject relatives or subject clefts) but they should be severely impaired in noncanonical reversible sentences (e.g. passives, object relatives and object clefts). Given that the TDH was formulated as a representational account, this dissociation was expected to characterize the performance in sentence – picture matching tasks as well as grammaticality judgment tasks. This is indeed what Grodzinsky and Finkel (1998) found. The judgment of sentences containing movement was significantly more impaired than the judgment of sentences without movement. A further assumption made by Grodzinsky (2000) and in particular Grodzinsky and Finkel (1998) is that syntactic knowledge about traces is specifically localized in Broca’s area. It is now possible to verify this assumption with healthy subjects by means of neuroimaging techniques. So far, functional neuroimaging studies (using functional magnetic resonance imaging (fMRI) and Positron Emission Tomography (PET)) on
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
303
syntactic processing used a variety of tasks and most of them compared syntactic to semantic processes. While some studies (e.g. Just, Carpenter, Keller, Eddy, & Thulborn, 1996; Stowe et al., 1998) noted increased activation in left hemispheric Broca’s as well as Wernicke’s area related to syntactic complexity, most of the more recent studies supported a privileged function of the left Broca’s area in syntactic processing (Caplan, Alpert, & Waters, 1998, 1999; Caplan, Alpert, Waters, & Olivieri, 2000; Dapretto & Bookheimer, 1999; Indefrey, Hagoort, Herzog, Seitz, & Brown, 2001). Since the TDH suggests a different neuronal organization of processing sentences with and without movement, the lack of control of such structures may have obscured the somewhat controversial results of former studies. In this study we used event-related fMRI to investigate whether processing of visually presented sentences with and without movement of phrasal constituents rely on differential neuronal substrates. In particular, following the TDH, we hypothesized an increase of activation in Broca’s area during the judgment of grammaticality of moved sentences (MOV) as compared to nonmoved sentences (nonMOV) in healthy subjects. The study consisted of two parts, a pilot behavioral study and the fMRI-study. The behavioral study was meant to determine that grammatical sentences were actually judged to be grammatical, and ungrammatical ones were considered ungrammatical by healthy native speakers. Furthermore, it was used to guarantee that the two conditions, MOV and nonMOV, were comparable with respect to accuracy and reaction time. An additional function of the behavioral study was to determine the presentation time required in the fMRI-study.
2. Materials and methods 2.1. Subjects All subjects were native speakers of German and had no prior knowledge of the sentence material. Eight (six male) healthy, right-handed young (mean age 28.3; SD ¼ 3.7) adults participated in the pilot behavioral study. None of these subjects participated in the fMRI study, for which 13 (eight male) healthy, right-handed (Oldfield, 1971) young (mean age 25.5; SD ¼ 3.6) adults were recruited. They gave written informed consent prior to the investigation and were paid for participation. All experiments were performed in compliance with the relevant laws and institutional guidelines and were approved by the ethics committee of the Charite´ Berlin. 2.2. Linguistic material and task The behavioral pilot study revealed unexpected grammaticality judgments in one sentence type, namely, WH-movement out of embedded contexts, where grammatical sentences were consistently considered ungrammatical. Therefore, these structures were removed from the final material in their grammatical as well as ungrammatical versions. The final stimulus material for the fMRI-study consisted of 80 visually presented short sentences. There were two conditions: sentences with movement of the noun phrase
304
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
(MOV, 40 sentences) and sentences without movement of the noun phrase (nonMOV, 40 sentences). Both conditions contained sentences that were either correct (60%) or contained a grammatical error (40%). In the MOV condition, NP-movement was used in passives for grammatically correct sentences. Incorrect sentences involved illicit raising of an argument over another argument position (so-called Super-raising). In the nonMOV condition, sentences were used without violation of the subcategorization frame in the correct sentences. The incorrect sentences involved complements violating the subcategorization frame of the predicate (incorrect preposition or case morpheme). Examples for each type are given in Table 1. All sentences were semantically plausible and orthographically correct. They were matched across conditions for length (M ¼ 7:3; SD ¼ 0.9) and frequency of nouns, verbs and adjectives (low frequency, M ¼ 49:1; SD ¼ 25.7 per million words) (Baayen, Piepenbrock, & Rijn, 1993). In both the behavioral and the fMRI study subjects were instructed to judge the grammaticality of the sentences intuitively, i.e. without relying on rules of grammar learned in school. They were explicitly asked not to repair or adjust incorrect sentences. They had to indicate their decision as quickly and correctly as possible by left-hand button press on a two-button response box. Reaction times and accuracy were recorded using Experimental Run Time System software (ERTS 3.28). In the pilot behavioral study, sentences were presented on a computer screen in front of the subject. The sentences were pseudo-randomized and remained in view until the subject pressed the left or right response button indicating their grammaticality judgment. A fixation cross was shown for 4 s between button press and presentation of the next sentence. Table 1 Examples of sentences
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
305
In the fMRI study, the experiment was explained outside the scanner before the scanning session and the subjects had a brief training session. The sentences were presented in an event-related and pseudo-randomized manner on a back-projection screen after an initial resting-period (20 s). Each sentence was presented for 4 s (based on the results of the behavioral study, this was determined as the optimal duration). The interstimulus interval varied between 2 and 6 s (mean 4 s). 2.3. Data acquisition and analysis For the behavioral data, nonparametric Wilcoxon-Tests were performed to compare the conditions regarding reaction times and accuracy of response (p , 0:01; corrected for multiple comparisons). FMRI measurements were performed on a 1.5 T scanner (Siemens, Erlangen, Germany) with a standard head coil. Head movement was minimized using a vacuum pad. Following the scout spin echo scan, structural three dimensional data sets were acquired using a T1-weighted sagittal sequence (3d fast low-angle shot sequence; TR/TE 20/5 ms; FA 308; voxel size 1 mm3). Subsequently, 17 4-mm slices were obtained approximately parallel to the bicommissural plane (ac – pc plane) using an echo-planar sequence (TR/TE 2000/40 ms; FA 908; FOV 256 mm; matrix 64 £ 64; interslice gap 0.4 mm; in-plane resolution 4 mm2; ascending acquisition of images). To avoid a systematic bias in sampling over peristimulus time (Burock, Buckner, Woldorff, Rosen, & Dale, 1998; Dale, 1999; Price, Veltman, Ashburner, Josephs, & Friston, 1999), scans were acquired in temporal asynchrony to the task (jittered stimulus presentation). Total scanning time per subject was held relatively short (approximately 30 min) with respect to future investigation of patients. Slices covered the whole brain with exception of the most superior frontal and superior parietal lobe, inferior temporal pole and cerebellum (most superior z about 30 and most inferior z about 2 36) thus covering both, the frontal and temporal language regions. Imaging data were analyzed using the Statistical Parametric Mapping software package (SPM99; Wellcome Department of Cognitive Neurology, University College London, UK). The first three functional volumes were excluded to allow for magnetic saturation effects. Scans were slice-time corrected, realigned, normalized, and spatially smoothed by a Gaussian kernel (FWHM ¼ 8 £ 8 £ 8 mm3) using standard SPM methods (Friston et al., 1995) and a high-pass frequency filter (171 s) was applied. Time series were modeled using event-related regressors for every sentence and convolved with the hemodynamic response function. To reduce motion induced artifacts for each subject the six realignment parameters were included in the statistical model as parameters of no interest. Mixed effects analysis. Contrast images for each condition versus rest and for differences between the respective conditions were computed for each subject. To be able to generalize the observed effects to the population (Friston, Holmes, Price, Buchel, & Worsley, 1999a; Friston, Holmes, & Worsley, 1999b; Holmes & Friston, 1998), the group effects were computed using these contrast images in a mixed-effects model treating subjects as random (commonly referred to as ‘random effects’ or ‘second level’ analysis). Group analysis was performed using one-sample t tests to identify regions that showed greater activation in MOV compared to nonMOV and vice versa and employed a statistical threshold of p , 0:005; uncorrected for multiple comparisons, with an extent
306
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
threshold of 10 contiguous voxels. In a second step we separated grammatically correct and incorrect sentences, resulting in four conditions. Grammatically correct MOV and nonMOV sentences and vice versa as well as grammatically incorrect MOVp and nonMOVp sentences and vice versa were compared. Region of interest (ROI) analysis. For the ROI analysis the computed contrast images for all conditions together compared to rest (i.e. general grammatical processing vs. rest) were tested at the mixed-effects group level and the most significant voxel within Broca’s region was determined. Parameter estimates of all four conditions within this voxel were determined and percent signal changes were calculated with respect to the mean signal. For this, the individual parameter estimate for each condition was multiplied with the predictor and 100 and then divided by the parameter estimate of the session effect. The relative signal changes of all four conditions were compared using nonparametric Wilcoxon-Tests ðp , 0:01Þ: Conjunction analysis. Additionally, a fixed effect conjunction analysis was performed in order to show regions that were conjointly active during processing sentences of all conditions. A statistical threshold of p , 0:05; corrected for entire volume, multiple comparisons, and an extent threshold of 10 contiguous voxels, was applied.
3. Results 3.1. Behavioral results After removing the inaccurately judged sentences containing WH-movement out of embedded contexts, the remaining 80 sentences were examined for accuracy and response latency. Results revealed no significant difference in reaction times and accuracy between the two experimental conditions as a whole. When separately analyzed, there were no differences for the grammatical items across conditions or for the ungrammatical sentences. 90% of reaction times were less than 4 s (mean 3.2 s; SD ¼ 1.4), thus determining the time of sentence presentation in the fMRI study. Behavioral data recorded inside the scanner during the fMRI study did not reveal significant differences in reaction times and accuracy of response between MOV and nonMOV sentences (compare Fig. 1a,b). There were also no statistically significant differences between MOV and nonMOV comparing grammatically correct and incorrect sentences separately (compare Fig. 1c,d). In particular, comparing all the conditions with each other there were no significant differences. Thus, task difficulty could be considered equal in all conditions. 3.2. FMRI results All four conditions compared to rest resulted in an activation of left frontal and temporal areas, bilateral putamen, thalamus and occipital areas. Mixed effects analysis revealed that the comparison of MOV to nonMOV did not result in specific activation within or outside Broca’s area. Comparing nonMOV to MOV resulted in activation of right superior temporal sulcus (compare Table 2).
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
307
Fig. 1. Behavioral data acquired during the fMRI experiment. (a) accuracy and (b) reaction time of judging moved and nonmoved sentences as revealed by the combined analysis of grammatically correct and incorrect sentences (MOV, movement; nonMOV, without movement). (c) accuracy and (d) reaction time as revealed by the separate analysis of grammatically correct and incorrect sentences (MOV, movement correct; MOVp, movement incorrect; nonMOV, without movement correct; nonMOVp, without movement incorrect). There were no significant differences between the conditions. Mean and standard deviation are displayed. Reduced N ¼ 11 is caused by malfunction of the recording system inside the scanner.
A further separate analysis of correct and incorrect sentences resulted in the following pattern of activation (compare also Table 2):
† Grammatically correct sentences: comparison of MOV to nonMOV resulted in activation of the right insula, comparison of nonMOV to MOV resulted in activation of the left thalamus. † Grammatically incorrect sentences: comparison of MOV to nonMOV resulted in activation of the right thalamus, comparison of nonMOV to MOV resulted in activation of bilateral parahippocampal regions, right superior temporal sulcus, and medial superior parts of cerebellum.
308
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
Table 2 Contrasting processing of sentences with and without movement and vice versa Cluster size
Voxel T value
Coordinates x
MOV vs. nonMOV All sentences together Correct sentences
15
Incorrect sentences
10
nonMOV vs. MOV All sentences together
Correct sentences Incorrect sentences
y
BA
Side
Anatomical region
R
Insula
R
Thalamus
z
4.53 3.56 3.83 3.22
36 32 20 28
12 24 28 212
28 28 0 4
19
4.58
52
24
212
21
R
Superior temporal sulcus
13 22
3.40 4.49 4.59
56 212 52
8 28 28
220 24 216
21
L R
Thalamus Superior temporal sulcus
18
4.53 4.58
48 236
220 236
28 212
37
L
39
7.89
20
240
0
19/27/37
R
Parahippocampal gyrus Parahippocampal gyrus
69
3.58 4.69
20 0
256 232
24 216
4.67 3.87
212 0
232 256
216 212
L/R
Medial superior cerebellum
Correct and incorrect sentences were compared together (all sentences) as well as separately for each condition. For each contrast cluster size, T values, coordinates of the local maxima of significance within the Montreal Neurological Institute coordinate system, approximate Brodmann areas (BA), side of activation and the respective activated anatomical region are given. The reported regions were active with p , 0:005 (uncorrected) at voxel level.
ROI analysis also revealed no significant differences in terms of percent signal change between the conditions within Broca’s area (ROI at x ¼ 248; y ¼ 8; z ¼ 20), neither when comparing MOV and nonMOV inclusive grammatically correct and incorrect sentences (Fig. 2a) nor when comparing grammatically correct and incorrect sentences separately (Fig. 2b). There were no significant differences comparing all conditions with each other (Table 3). The conjunction analysis resulted in activation of left inferior frontal, left temporal, and bilateral occipital regions, left insula, left putamen and bilateral thalamus, i.e. these regions were conjointly activated in all sentence conditions (compare Table 4 and Fig. 3). Taken together, there is large overlap of cortical activation during processing MOV and nonMOV sentences. Frontal and temporal language regions of the left hemisphere
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
309
Fig. 2. Response strength in Broca’s area as reflected by parameter estimates of each condition. Within Broca’s area (ROI at x ¼ 248; y ¼ 8; z ¼ 20) there were (a) no significant differences between moved and nonmoved sentences using a combined analysis of grammatically correct and incorrect sentences (MOV, movement; nonMOV, without movement) and (b) no significant differences between MOV and nonMOV using a separate analysis of grammatically correct and incorrect sentences (MOV, movement correct; MOVp, movement incorrect; nonMOV, without movement correct; nonMOVp, without movement incorrect). BOLD percent signal change and standard error of the mean relative to the mean signal of the ROI were given for each condition.
obviously were engaged in sentence processing as required by grammaticality judgments. Contrary to our hypothesis, as based on the TDH, however, processing of MOV compared to nonMOV sentences did not result in greater activation of Broca’s area.
310
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
Table 3 Comparison of parameter estimates in Broca’s area (ROI at 248, 8, 20) over all subjects
Z P
MOVp vs. MOV
nonMOVp vs. nonMOV
nonMOV vs. MOV
nonMOV vs. MOVp
nonMOVp vs. MOV
nonMOVp vs. MOVp
21.572 0.116
21.153 0.249
21.013 0.311
21.433 0.152
21.922 0.055
20.384 0.701
There were no significant differences comparing all conditions to each other. MOV, movement correct; MOVp, movement incorrect; nonMOV, without movement correct; nonMOVp, without movement incorrect. Z and p values as revealed by the nonparametric Wilcoxon-Test are given.
4. Discussion Some functional imaging studies on syntactic processing have shown increased activation in Wernicke’s area as well as Broca’s area (Embick, Marantz, Miyashita, O’Neil, & Sakai, 2000; Friederici, Meyer, & von Cramon, 2000a; Just et al., 1996; Stowe et al., 1998; Roder, Stock, Neville, Bien, & Rosler, 2002). This finding was confirmed in the conjunction analysis of our study. However, most of the research comparing grammatical and semantic processing found Broca’s area specifically engaged in
Table 4 Results of conjunction analysis showing areas that were active during processing all sentence conditions (i.e. incorrect and correct sentences with and without movement) Cluster size
Voxel T value
Coordinates x
Conjunction of all sentence conditions 64 3.81 252 3 260 2.65 252 17 2.72 236 39 2.6 268 2.49 252 2.1 248 71 2.79 24 2.53 20 2.38 8 18 2.57 224 386 5.03 8 4.22 16 3.88 236
BA
y
z
4 8 24 24 232 244 244 220 228 228 0 272 296 292
24 16 20 4 0 8 24 0 24 4 28 4 24 4
9 44/45
21
30/17/18/19
Side
Anatomical region
L
Inferior frontal gyrus
L L
Insula Medial temporal gyrus Superior temporal sulcus
L/R
Thalamus
L L/R
Putamen Occipital regions/lingual gyrus
Cluster size, T values, coordinates of the local maxima of significance within the Montreal Neurological Institute coordinate system, approximate Brodmann areas (BA), side of activation and the respective activated anatomical region are given. The reported regions were active with p , 0:05 (corrected) at voxel level.
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
311
Fig. 3. Results of conjunction analysis showing regions that were conjointly activated during processing both moved an nonmoved sentences. Both conditions activated left inferior frontal and temporal language areas as well as occipital areas (Results of group analysis ðN ¼ 13Þ superimposed on MNI template ‘colin27’ in neurological convention (left is left); x and z coordinates are given).
grammatical processing (Dapretto & Bookheimer, 1999; Friederici, Opitz, & von Cramon, 2000b; Kang, Constable, Gore, & Avrutin, 1999; Ni et al., 2000; Moro et al., 2001; Stromswold, Caplan, Alpert, & Rauch, 1996). To examine areas of activation during syntactic processing, we selected a grammaticality judgment task, since it has been claimed that grammaticality judgments offer a relatively clear window on syntax in isolation, given that they do not require semantic processing (Linebarger et al., 1983). This view is not supported by our results, since the conjunction analysis showed that Wernicke’s area was activated in addition to Broca’s area during grammaticality judgments. Obviously, it cannot be excluded that semantics is generally involved in sentences processing, including grammaticality judgments. Whereas most neuroimaging studies have been concerned with the role of Broca’s area in syntactic computation in general, taking into account various measures of syntactic
312
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
complexity, our study investigated a stronger claim for Broca’s area made by the TDH, namely, that it has a much more specific and limited function for the computation of gapantecedent relations. We aimed to compare MOV and nonMOV sentences without the confounding effect of difficulty and controlled for difficulty/processing load of the conditions. Since there were no differences in reaction times and accuracy of reactions between the conditions in either the pilot behavioral study or the behavioral data acquired during the fMRI study, difficulty/processing load would seem to be equal in all conditions. Contrary to our expectation which was based on the assumptions of the TDH (Grodzinsky, 2000; Grodzinsky & Finkel, 1998), our data demonstrate that processing sentences with and without movement of phrasal constituents equally modulate the activation of the major language areas. In particular, grammaticality judgments of sentences with movement of phrasal constituents as compared to sentences without such movement did not result in greater activation of Broca’s area (neither in the mixed effects analysis nor in the ROI analysis). Although in the original version of the TDH, the trace-antecedent linkage was not related to working memory resources, other authors, for example Cooke et al. (2002) focused on this interaction. The authors found increased activation in Broca’s area only in noncanonical object relatives in which there was a long antecedent-gap distance, but not for the same structure with a shorter distance. The sentences in that study were presented visually in a word-by-word fashion. We used stimulus material with a relatively short distance between the trace and its antecedent and the visually presented sentences remained in view for 4 s, so that the working memory load in our task was reduced, in contrast to Cooke et al.’s (2002). Our study does not provide evidence that Broca’s area plays a special role in processing sentences with movement if there are minimal working memory demands. Given that previous functional imaging studies found Broca’s area active in general syntactic processing, it is likely that in our study it was activated in any sentence type because of the grammaticality judgment task. Thus, it might be that the modulation of Broca’s area by the moved phrasal constituent was too small to be detectable under our experimental conditions. In other words, without the ‘preactivation’ of Broca’s area, e.g. during a ‘nonsyntactic’ task such as animate/inanimate judgments and/or using a higher magnetic field strength (3 T) and/or a larger number of sentences, its modulation with respect to movement might be detectable. Thus, the specific claims made by the TDH about the neuronal substrate of grammatical dependencies in syntactic movement remain to be verified.
Acknowledgements The technical assistance of S. Heinemann, S. Kohnen and T. Fritzsche is gratefully acknowledged. This work was supported by grants from BMBF (FKZ: 01GA0202 and Berlin Neuroimaging Center), DFG (Clinical Research Group EI-207/2-3), International Leibniz Program, and Graduate Program Berlin (Scholarship Nachwuchsfoerderung).
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
313
References Baayen, R. H., Piepenbrock, R., & Rijn, H. (1993). The CELEX lexical database (Release 1) [CD-ROM]. Philadelphia, PA: Linguistic Data Consortium, University of Pennsylvania. Berndt, R. S., & Caramazza, A. (1980). Semantic operations deficits in sentence comprehension. Psychological Research, 41, 169 –177. Burock, M. A., Buckner, R. L., Woldorff, M. G., Rosen, B. R., & Dale, A. M. (1998). Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport, 9, 3735–3739. Caplan, D., Alpert, N., & Waters, G. (1998). Effects of syntactic structure and propositional number on patterns of regional cerebral blood flow. Journal of Cognitive Neuroscience, 10, 541 –552. Caplan, D., Alpert, N., & Waters, G. (1999). PET studies of syntactic processing with auditory sentence presentation. Neuroimage, 9, 343–351. Caplan, D., Alpert, N., Waters, G., & Olivieri, A. (2000). Activation of Broca’s area by syntactic processing under conditions of concurrent articulation. Human Brain Mapping, 9, 65– 71. Caramazza, A., & Zurif, E. B. (1976). Dissociation of algorithmic and heuristic processes in language comprehension: evidence from aphasia. Brain Language, 3, 572–582. Chomsky, N. (1981). Lectures on government and binding. Dordrecht: Foris. Cooke, A., Zurif, E. B., DeVita, C., Alsop, D., Koenig, P., Detre, J., Gee, J., Pinango, M., Balogh, J., & Grossman, M. (2002). Neural basis for sentence comprehension: grammatical and short-term memory components. Human Brain Mapping, 15, 80–94. Dale, A. M. (1999). Optimal experimental design for event-related fMRI. Human Brain Mapping, 8, 109–114. Dapretto, M., & Bookheimer, S. Y. (1999). Form and content: dissociating syntax and semantics in sentence comprehension. Neuron, 24, 427–432. Embick, D., Marantz, A., Miyashita, Y., O’Neil, W., & Sakai, K. L. (2000). A syntactic specialization for Broca’s area. Proceedings of the National Academy of Sciences USA, 97, 6150– 6154. Friederici, A. D., Meyer, M., & von Cramon, D. Y. (2000a). Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information. Brain Language, 75, 289– 300. Friederici, A. D., Opitz, B., & von Cramon, D. Y. (2000b). Segregating semantic and syntactic aspects of processing in the human brain: an fMRI investigation of different word types. Cerebral Cortex, 10, 698–705. Friston, K. J., Holmes, A. P., Price, C. J., Buchel, C., & Worsley, K. J. (1999a). Multisubject fMRI studies and conjunction analyses. Neuroimage, 10, 385–396. Friston, K. J., Holmes, A. P., & Worsley, K. J. (1999b). How many subjects constitute a study? Neuroimage, 10, 1–5. Friston, K., Holmes, A. P., Worsley, K., Poline, J. B., Frith, C., & Frackowiak, R. S. (1995). Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2, 189 –210. Grodzinsky, Y. (1986). Language deficits and the theory of syntax. Brain Language, 27, 135–159. Grodzinsky, Y. (1995). A restrictive theory of agrammatic comprehension. Brain Language, 50, 27–51. Grodzinsky, Y. (2000). The neurology of syntax: language use without Broca’s area. Behavioral Brain Science, 23, 1– 21. Grodzinsky, Y., & Finkel, L. (1998). The neurology of empty categories aphasics’ failure to detect ungrammaticality. Journal of Cognitive Neuroscience, 10, 281–292. Holmes, A. P., & Friston, K. J. (1998). Generalizability, random effects, and population inference. Neuroimage, 7, S754. Indefrey, P., Hagoort, P., Herzog, H., Seitz, R. J., & Brown, C. M. (2001). Syntactic processing in left prefrontal cortex is independent of lexical meaning. Neuroimage, 14, 546–555. Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1996). Brain activation modulated by sentence comprehension. Science, 274, 114– 116. Kang, A. M., Constable, R. T., Gore, J. C., & Avrutin, S. (1999). An event-related fMRI study of implicit phraselevel syntactic and semantic processing. Neuroimage, 10, 555 –561. Linebarger, M. C., Schwartz, M. F., & Saffran, E. M. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13, 361–392.
314
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
Moro, A., Tettamanti, M., Perani, D., Donati, C., Cappa, S. F., & Fazio, F. (2001). Syntax and the brain: disentangling grammar by selective anomalies. Neuroimage, 13, 110–118. Ni, W., Constable, R. T., Mencl, W. E., Pugh, K. R., Fulbright, R. K., Shaywitz, S. E., Shaywitz, B. A., Gore, J. C., & Shankweiler, D. (2000). An event-related neuroimaging study distinguishing form and content in sentence processing. Journal of Cognitive Neuroscience, 12, 120 –133. Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9, 97– 113. ¨ ber Agrammatismus als folge cerebraler herderkrankungen; ein beitrag zur lehre vom Pick, A. (1898). U Verha¨ltnis der worttaubheit. Zur taubheit. Beitra¨ge zur pathologie und pathologischen anatomie des zentralnervensystems, Berlin: Karger, p. 123 –133. Price, C. J., Veltman, D. J., Ashburner, J., Josephs, O., & Friston, K. J. (1999). The critical relationship between the timing of stimulus presentation and data acquisition in blocked designs with fMRI. Neuroimage, 10, 36–44. Roder, B., Stock, O., Neville, H., Bien, S., & Rosler, F. (2002). Brain activation modulated by the comprehension of normal and pseudo-word sentences of different processing demands: a functional magnetic resonance imaging study. Neuroimage, 15, 1003–1014. Stowe, L. A., Broere, C. A., Paans, A. M., Wijers, A. A., Mulder, G., Vaalburg, W., & Zwarts, F. (1998). Localizing components of a complex task: sentence processing and working memory. Neuroreport, 9, 2995–2999. Stromswold, K., Caplan, D., Alpert, N., & Rauch, S. (1996). Localization of syntactic comprehension by positron emission tomography. Brain Language, 52, 452–473.
Grammaticality judgments on sentences with and without movement of phrasal constituents—an event-related fMRI study Isabell Wartenburgera,b,*, Hauke R. Heekerena,c, Frank Burchertb, Ria De Bleserb, Arno Villringera a
Department of Neurology, Charite´, Humboldt-University, Schumannstr. 20-21, 10117 Berlin, Germany b Department of Patholinguistics, University of Potsdam, PF: 601553, 14415 Potsdam, Germany c Laboratory of Brain and Cognition, NIMH, NIH, Bldg. 10, Room 1D80, Bethesda, MD 20892-1148, USA
Abstract One of the leading neurolinguistic theories of syntactic comprehension disorders in agrammatic aphasic subjects—the Trace Deletion Hypothesis—postulates a specific impairment in processing syntactic chains, and that this function is mediated by Broca’s area. The aim of this study was to investigate whether the specific involvement of Broca’s area in processing syntactic traces can be verified using functional brain imaging. We used event-related functional magnetic resonance imaging (fMRI) while healthy subjects were asked to judge the grammaticality of visually presented sentences with and without movement of phrasal constituents. During both kinds of sentences, fMRI showed activation in language-related brain regions. Comparing both kinds of sentences did not result in differential brain activation of left frontal or temporal regions. In particular, Broca’s area was similarly activated in processing both moved and nonmoved sentences. Thus, Broca’s area seems to be involved in general syntactic processing as required by grammaticality judgments rather than having a specific function in transmitting syntactic relations. q 2003 Elsevier Science Ltd. All rights reserved. Keywords: Agrammatism; Broca’s area; Event-related functional magnetic resonance imaging; Grammaticality judgments; Syntactic traces; Trace Deletion Hypothesis
1. Introduction Interest in the neuropsychology of syntax was originally linked to the condition of agrammatic aphasia (Pick, 1898). In modern research, this condition was usually described * Corresponding author. Address: Department of Neurology, Clinical Research Group, Charite´, HumboldtUniversity, Schumannstr. 20/21, 10117 Berlin, Germany. Tel.: þ 49-30-450-560-194; fax: þ49-30-450-560-952. E-mail address: [email protected] (I. Wartenburger). 0911-6044/03/$ - see front matter q 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0911-6044(03)00028-9
302
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
as a syntactic impairment restricted to language production. Since Caramazza and Zurif’s (1976) seminal paper, it has been shown that in the majority of subjects with agrammatic speech, syntactic comprehension is affected as well. Given that semantic processing, on the contrary, was relatively preserved in comprehension as well as production, it was argued that agrammatism was characterized by a specific central syntactic impairment (Berndt & Caramazza, 1980) and that it provides evidence for the dissociability of syntax and semantics. Another pioneering paper by Linebarger, Schwartz, and Saffran (1983) disagreed with the view that syntactic representations were lost in agrammatism. The authors showed that their agrammatic subjects, who were seriously impaired in syntactic comprehension when examined with standard sentence – picture matching tasks, had relatively normal performance on judgments of grammaticality. They interpreted this as counterevidence for a total syntactic loss account, since syntactic processing was impaired under certain conditions but not others. When semantic and syntactic processing compete for the same resources, such as in sentence – picture matching tasks, which require semantic as well as syntactic interpretation, there is a tradeoff in favor of semantics. In judgments of grammaticality, however, no semantic interpretation is required and syntax is able to operate practically in isolation, thus supporting the processing account of agrammatism. However, the introduction of newer linguistic insights into aphasiology reintroduced the notion of a representational syntactic impairment in agrammatism. In his Trace Deletion Theory (TDH) of agrammatism, Grodzinsky (1986, 1995) relied on distinctions made in Chomsky’s (1981) linguistic theory, in particular, the Government and Binding framework. In this framework, a grammatical transformation involves the movement of a constituent from its base position to another derived position in the sentence. It is assumed that the former original position is substituted by a phonetically empty but syntactically effective trace. Thus, the trace is an abstract position marker which is related to the displaced constituent via co-indexation and transmits its thematic role in case of argument movement. Therefore, the trace is necessary for correct comprehension of sentences with moved phrasal constituents. With respect to agrammatism, the TDH assumes a partial but specific syntactic impairment. It postulates that the syntactic traces, which are left behind when constituents are moved, are deleted from the agrammatic syntactic representations. Therefore, agrammatic subjects should show relatively good performance in the comprehension of semantically reversible canonical sentences which do not involve movement (e.g. actives, subject relatives or subject clefts) but they should be severely impaired in noncanonical reversible sentences (e.g. passives, object relatives and object clefts). Given that the TDH was formulated as a representational account, this dissociation was expected to characterize the performance in sentence – picture matching tasks as well as grammaticality judgment tasks. This is indeed what Grodzinsky and Finkel (1998) found. The judgment of sentences containing movement was significantly more impaired than the judgment of sentences without movement. A further assumption made by Grodzinsky (2000) and in particular Grodzinsky and Finkel (1998) is that syntactic knowledge about traces is specifically localized in Broca’s area. It is now possible to verify this assumption with healthy subjects by means of neuroimaging techniques. So far, functional neuroimaging studies (using functional magnetic resonance imaging (fMRI) and Positron Emission Tomography (PET)) on
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
303
syntactic processing used a variety of tasks and most of them compared syntactic to semantic processes. While some studies (e.g. Just, Carpenter, Keller, Eddy, & Thulborn, 1996; Stowe et al., 1998) noted increased activation in left hemispheric Broca’s as well as Wernicke’s area related to syntactic complexity, most of the more recent studies supported a privileged function of the left Broca’s area in syntactic processing (Caplan, Alpert, & Waters, 1998, 1999; Caplan, Alpert, Waters, & Olivieri, 2000; Dapretto & Bookheimer, 1999; Indefrey, Hagoort, Herzog, Seitz, & Brown, 2001). Since the TDH suggests a different neuronal organization of processing sentences with and without movement, the lack of control of such structures may have obscured the somewhat controversial results of former studies. In this study we used event-related fMRI to investigate whether processing of visually presented sentences with and without movement of phrasal constituents rely on differential neuronal substrates. In particular, following the TDH, we hypothesized an increase of activation in Broca’s area during the judgment of grammaticality of moved sentences (MOV) as compared to nonmoved sentences (nonMOV) in healthy subjects. The study consisted of two parts, a pilot behavioral study and the fMRI-study. The behavioral study was meant to determine that grammatical sentences were actually judged to be grammatical, and ungrammatical ones were considered ungrammatical by healthy native speakers. Furthermore, it was used to guarantee that the two conditions, MOV and nonMOV, were comparable with respect to accuracy and reaction time. An additional function of the behavioral study was to determine the presentation time required in the fMRI-study.
2. Materials and methods 2.1. Subjects All subjects were native speakers of German and had no prior knowledge of the sentence material. Eight (six male) healthy, right-handed young (mean age 28.3; SD ¼ 3.7) adults participated in the pilot behavioral study. None of these subjects participated in the fMRI study, for which 13 (eight male) healthy, right-handed (Oldfield, 1971) young (mean age 25.5; SD ¼ 3.6) adults were recruited. They gave written informed consent prior to the investigation and were paid for participation. All experiments were performed in compliance with the relevant laws and institutional guidelines and were approved by the ethics committee of the Charite´ Berlin. 2.2. Linguistic material and task The behavioral pilot study revealed unexpected grammaticality judgments in one sentence type, namely, WH-movement out of embedded contexts, where grammatical sentences were consistently considered ungrammatical. Therefore, these structures were removed from the final material in their grammatical as well as ungrammatical versions. The final stimulus material for the fMRI-study consisted of 80 visually presented short sentences. There were two conditions: sentences with movement of the noun phrase
304
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
(MOV, 40 sentences) and sentences without movement of the noun phrase (nonMOV, 40 sentences). Both conditions contained sentences that were either correct (60%) or contained a grammatical error (40%). In the MOV condition, NP-movement was used in passives for grammatically correct sentences. Incorrect sentences involved illicit raising of an argument over another argument position (so-called Super-raising). In the nonMOV condition, sentences were used without violation of the subcategorization frame in the correct sentences. The incorrect sentences involved complements violating the subcategorization frame of the predicate (incorrect preposition or case morpheme). Examples for each type are given in Table 1. All sentences were semantically plausible and orthographically correct. They were matched across conditions for length (M ¼ 7:3; SD ¼ 0.9) and frequency of nouns, verbs and adjectives (low frequency, M ¼ 49:1; SD ¼ 25.7 per million words) (Baayen, Piepenbrock, & Rijn, 1993). In both the behavioral and the fMRI study subjects were instructed to judge the grammaticality of the sentences intuitively, i.e. without relying on rules of grammar learned in school. They were explicitly asked not to repair or adjust incorrect sentences. They had to indicate their decision as quickly and correctly as possible by left-hand button press on a two-button response box. Reaction times and accuracy were recorded using Experimental Run Time System software (ERTS 3.28). In the pilot behavioral study, sentences were presented on a computer screen in front of the subject. The sentences were pseudo-randomized and remained in view until the subject pressed the left or right response button indicating their grammaticality judgment. A fixation cross was shown for 4 s between button press and presentation of the next sentence. Table 1 Examples of sentences
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
305
In the fMRI study, the experiment was explained outside the scanner before the scanning session and the subjects had a brief training session. The sentences were presented in an event-related and pseudo-randomized manner on a back-projection screen after an initial resting-period (20 s). Each sentence was presented for 4 s (based on the results of the behavioral study, this was determined as the optimal duration). The interstimulus interval varied between 2 and 6 s (mean 4 s). 2.3. Data acquisition and analysis For the behavioral data, nonparametric Wilcoxon-Tests were performed to compare the conditions regarding reaction times and accuracy of response (p , 0:01; corrected for multiple comparisons). FMRI measurements were performed on a 1.5 T scanner (Siemens, Erlangen, Germany) with a standard head coil. Head movement was minimized using a vacuum pad. Following the scout spin echo scan, structural three dimensional data sets were acquired using a T1-weighted sagittal sequence (3d fast low-angle shot sequence; TR/TE 20/5 ms; FA 308; voxel size 1 mm3). Subsequently, 17 4-mm slices were obtained approximately parallel to the bicommissural plane (ac – pc plane) using an echo-planar sequence (TR/TE 2000/40 ms; FA 908; FOV 256 mm; matrix 64 £ 64; interslice gap 0.4 mm; in-plane resolution 4 mm2; ascending acquisition of images). To avoid a systematic bias in sampling over peristimulus time (Burock, Buckner, Woldorff, Rosen, & Dale, 1998; Dale, 1999; Price, Veltman, Ashburner, Josephs, & Friston, 1999), scans were acquired in temporal asynchrony to the task (jittered stimulus presentation). Total scanning time per subject was held relatively short (approximately 30 min) with respect to future investigation of patients. Slices covered the whole brain with exception of the most superior frontal and superior parietal lobe, inferior temporal pole and cerebellum (most superior z about 30 and most inferior z about 2 36) thus covering both, the frontal and temporal language regions. Imaging data were analyzed using the Statistical Parametric Mapping software package (SPM99; Wellcome Department of Cognitive Neurology, University College London, UK). The first three functional volumes were excluded to allow for magnetic saturation effects. Scans were slice-time corrected, realigned, normalized, and spatially smoothed by a Gaussian kernel (FWHM ¼ 8 £ 8 £ 8 mm3) using standard SPM methods (Friston et al., 1995) and a high-pass frequency filter (171 s) was applied. Time series were modeled using event-related regressors for every sentence and convolved with the hemodynamic response function. To reduce motion induced artifacts for each subject the six realignment parameters were included in the statistical model as parameters of no interest. Mixed effects analysis. Contrast images for each condition versus rest and for differences between the respective conditions were computed for each subject. To be able to generalize the observed effects to the population (Friston, Holmes, Price, Buchel, & Worsley, 1999a; Friston, Holmes, & Worsley, 1999b; Holmes & Friston, 1998), the group effects were computed using these contrast images in a mixed-effects model treating subjects as random (commonly referred to as ‘random effects’ or ‘second level’ analysis). Group analysis was performed using one-sample t tests to identify regions that showed greater activation in MOV compared to nonMOV and vice versa and employed a statistical threshold of p , 0:005; uncorrected for multiple comparisons, with an extent
306
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
threshold of 10 contiguous voxels. In a second step we separated grammatically correct and incorrect sentences, resulting in four conditions. Grammatically correct MOV and nonMOV sentences and vice versa as well as grammatically incorrect MOVp and nonMOVp sentences and vice versa were compared. Region of interest (ROI) analysis. For the ROI analysis the computed contrast images for all conditions together compared to rest (i.e. general grammatical processing vs. rest) were tested at the mixed-effects group level and the most significant voxel within Broca’s region was determined. Parameter estimates of all four conditions within this voxel were determined and percent signal changes were calculated with respect to the mean signal. For this, the individual parameter estimate for each condition was multiplied with the predictor and 100 and then divided by the parameter estimate of the session effect. The relative signal changes of all four conditions were compared using nonparametric Wilcoxon-Tests ðp , 0:01Þ: Conjunction analysis. Additionally, a fixed effect conjunction analysis was performed in order to show regions that were conjointly active during processing sentences of all conditions. A statistical threshold of p , 0:05; corrected for entire volume, multiple comparisons, and an extent threshold of 10 contiguous voxels, was applied.
3. Results 3.1. Behavioral results After removing the inaccurately judged sentences containing WH-movement out of embedded contexts, the remaining 80 sentences were examined for accuracy and response latency. Results revealed no significant difference in reaction times and accuracy between the two experimental conditions as a whole. When separately analyzed, there were no differences for the grammatical items across conditions or for the ungrammatical sentences. 90% of reaction times were less than 4 s (mean 3.2 s; SD ¼ 1.4), thus determining the time of sentence presentation in the fMRI study. Behavioral data recorded inside the scanner during the fMRI study did not reveal significant differences in reaction times and accuracy of response between MOV and nonMOV sentences (compare Fig. 1a,b). There were also no statistically significant differences between MOV and nonMOV comparing grammatically correct and incorrect sentences separately (compare Fig. 1c,d). In particular, comparing all the conditions with each other there were no significant differences. Thus, task difficulty could be considered equal in all conditions. 3.2. FMRI results All four conditions compared to rest resulted in an activation of left frontal and temporal areas, bilateral putamen, thalamus and occipital areas. Mixed effects analysis revealed that the comparison of MOV to nonMOV did not result in specific activation within or outside Broca’s area. Comparing nonMOV to MOV resulted in activation of right superior temporal sulcus (compare Table 2).
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
307
Fig. 1. Behavioral data acquired during the fMRI experiment. (a) accuracy and (b) reaction time of judging moved and nonmoved sentences as revealed by the combined analysis of grammatically correct and incorrect sentences (MOV, movement; nonMOV, without movement). (c) accuracy and (d) reaction time as revealed by the separate analysis of grammatically correct and incorrect sentences (MOV, movement correct; MOVp, movement incorrect; nonMOV, without movement correct; nonMOVp, without movement incorrect). There were no significant differences between the conditions. Mean and standard deviation are displayed. Reduced N ¼ 11 is caused by malfunction of the recording system inside the scanner.
A further separate analysis of correct and incorrect sentences resulted in the following pattern of activation (compare also Table 2):
† Grammatically correct sentences: comparison of MOV to nonMOV resulted in activation of the right insula, comparison of nonMOV to MOV resulted in activation of the left thalamus. † Grammatically incorrect sentences: comparison of MOV to nonMOV resulted in activation of the right thalamus, comparison of nonMOV to MOV resulted in activation of bilateral parahippocampal regions, right superior temporal sulcus, and medial superior parts of cerebellum.
308
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
Table 2 Contrasting processing of sentences with and without movement and vice versa Cluster size
Voxel T value
Coordinates x
MOV vs. nonMOV All sentences together Correct sentences
15
Incorrect sentences
10
nonMOV vs. MOV All sentences together
Correct sentences Incorrect sentences
y
BA
Side
Anatomical region
R
Insula
R
Thalamus
z
4.53 3.56 3.83 3.22
36 32 20 28
12 24 28 212
28 28 0 4
19
4.58
52
24
212
21
R
Superior temporal sulcus
13 22
3.40 4.49 4.59
56 212 52
8 28 28
220 24 216
21
L R
Thalamus Superior temporal sulcus
18
4.53 4.58
48 236
220 236
28 212
37
L
39
7.89
20
240
0
19/27/37
R
Parahippocampal gyrus Parahippocampal gyrus
69
3.58 4.69
20 0
256 232
24 216
4.67 3.87
212 0
232 256
216 212
L/R
Medial superior cerebellum
Correct and incorrect sentences were compared together (all sentences) as well as separately for each condition. For each contrast cluster size, T values, coordinates of the local maxima of significance within the Montreal Neurological Institute coordinate system, approximate Brodmann areas (BA), side of activation and the respective activated anatomical region are given. The reported regions were active with p , 0:005 (uncorrected) at voxel level.
ROI analysis also revealed no significant differences in terms of percent signal change between the conditions within Broca’s area (ROI at x ¼ 248; y ¼ 8; z ¼ 20), neither when comparing MOV and nonMOV inclusive grammatically correct and incorrect sentences (Fig. 2a) nor when comparing grammatically correct and incorrect sentences separately (Fig. 2b). There were no significant differences comparing all conditions with each other (Table 3). The conjunction analysis resulted in activation of left inferior frontal, left temporal, and bilateral occipital regions, left insula, left putamen and bilateral thalamus, i.e. these regions were conjointly activated in all sentence conditions (compare Table 4 and Fig. 3). Taken together, there is large overlap of cortical activation during processing MOV and nonMOV sentences. Frontal and temporal language regions of the left hemisphere
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
309
Fig. 2. Response strength in Broca’s area as reflected by parameter estimates of each condition. Within Broca’s area (ROI at x ¼ 248; y ¼ 8; z ¼ 20) there were (a) no significant differences between moved and nonmoved sentences using a combined analysis of grammatically correct and incorrect sentences (MOV, movement; nonMOV, without movement) and (b) no significant differences between MOV and nonMOV using a separate analysis of grammatically correct and incorrect sentences (MOV, movement correct; MOVp, movement incorrect; nonMOV, without movement correct; nonMOVp, without movement incorrect). BOLD percent signal change and standard error of the mean relative to the mean signal of the ROI were given for each condition.
obviously were engaged in sentence processing as required by grammaticality judgments. Contrary to our hypothesis, as based on the TDH, however, processing of MOV compared to nonMOV sentences did not result in greater activation of Broca’s area.
310
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
Table 3 Comparison of parameter estimates in Broca’s area (ROI at 248, 8, 20) over all subjects
Z P
MOVp vs. MOV
nonMOVp vs. nonMOV
nonMOV vs. MOV
nonMOV vs. MOVp
nonMOVp vs. MOV
nonMOVp vs. MOVp
21.572 0.116
21.153 0.249
21.013 0.311
21.433 0.152
21.922 0.055
20.384 0.701
There were no significant differences comparing all conditions to each other. MOV, movement correct; MOVp, movement incorrect; nonMOV, without movement correct; nonMOVp, without movement incorrect. Z and p values as revealed by the nonparametric Wilcoxon-Test are given.
4. Discussion Some functional imaging studies on syntactic processing have shown increased activation in Wernicke’s area as well as Broca’s area (Embick, Marantz, Miyashita, O’Neil, & Sakai, 2000; Friederici, Meyer, & von Cramon, 2000a; Just et al., 1996; Stowe et al., 1998; Roder, Stock, Neville, Bien, & Rosler, 2002). This finding was confirmed in the conjunction analysis of our study. However, most of the research comparing grammatical and semantic processing found Broca’s area specifically engaged in
Table 4 Results of conjunction analysis showing areas that were active during processing all sentence conditions (i.e. incorrect and correct sentences with and without movement) Cluster size
Voxel T value
Coordinates x
Conjunction of all sentence conditions 64 3.81 252 3 260 2.65 252 17 2.72 236 39 2.6 268 2.49 252 2.1 248 71 2.79 24 2.53 20 2.38 8 18 2.57 224 386 5.03 8 4.22 16 3.88 236
BA
y
z
4 8 24 24 232 244 244 220 228 228 0 272 296 292
24 16 20 4 0 8 24 0 24 4 28 4 24 4
9 44/45
21
30/17/18/19
Side
Anatomical region
L
Inferior frontal gyrus
L L
Insula Medial temporal gyrus Superior temporal sulcus
L/R
Thalamus
L L/R
Putamen Occipital regions/lingual gyrus
Cluster size, T values, coordinates of the local maxima of significance within the Montreal Neurological Institute coordinate system, approximate Brodmann areas (BA), side of activation and the respective activated anatomical region are given. The reported regions were active with p , 0:05 (corrected) at voxel level.
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
311
Fig. 3. Results of conjunction analysis showing regions that were conjointly activated during processing both moved an nonmoved sentences. Both conditions activated left inferior frontal and temporal language areas as well as occipital areas (Results of group analysis ðN ¼ 13Þ superimposed on MNI template ‘colin27’ in neurological convention (left is left); x and z coordinates are given).
grammatical processing (Dapretto & Bookheimer, 1999; Friederici, Opitz, & von Cramon, 2000b; Kang, Constable, Gore, & Avrutin, 1999; Ni et al., 2000; Moro et al., 2001; Stromswold, Caplan, Alpert, & Rauch, 1996). To examine areas of activation during syntactic processing, we selected a grammaticality judgment task, since it has been claimed that grammaticality judgments offer a relatively clear window on syntax in isolation, given that they do not require semantic processing (Linebarger et al., 1983). This view is not supported by our results, since the conjunction analysis showed that Wernicke’s area was activated in addition to Broca’s area during grammaticality judgments. Obviously, it cannot be excluded that semantics is generally involved in sentences processing, including grammaticality judgments. Whereas most neuroimaging studies have been concerned with the role of Broca’s area in syntactic computation in general, taking into account various measures of syntactic
312
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
complexity, our study investigated a stronger claim for Broca’s area made by the TDH, namely, that it has a much more specific and limited function for the computation of gapantecedent relations. We aimed to compare MOV and nonMOV sentences without the confounding effect of difficulty and controlled for difficulty/processing load of the conditions. Since there were no differences in reaction times and accuracy of reactions between the conditions in either the pilot behavioral study or the behavioral data acquired during the fMRI study, difficulty/processing load would seem to be equal in all conditions. Contrary to our expectation which was based on the assumptions of the TDH (Grodzinsky, 2000; Grodzinsky & Finkel, 1998), our data demonstrate that processing sentences with and without movement of phrasal constituents equally modulate the activation of the major language areas. In particular, grammaticality judgments of sentences with movement of phrasal constituents as compared to sentences without such movement did not result in greater activation of Broca’s area (neither in the mixed effects analysis nor in the ROI analysis). Although in the original version of the TDH, the trace-antecedent linkage was not related to working memory resources, other authors, for example Cooke et al. (2002) focused on this interaction. The authors found increased activation in Broca’s area only in noncanonical object relatives in which there was a long antecedent-gap distance, but not for the same structure with a shorter distance. The sentences in that study were presented visually in a word-by-word fashion. We used stimulus material with a relatively short distance between the trace and its antecedent and the visually presented sentences remained in view for 4 s, so that the working memory load in our task was reduced, in contrast to Cooke et al.’s (2002). Our study does not provide evidence that Broca’s area plays a special role in processing sentences with movement if there are minimal working memory demands. Given that previous functional imaging studies found Broca’s area active in general syntactic processing, it is likely that in our study it was activated in any sentence type because of the grammaticality judgment task. Thus, it might be that the modulation of Broca’s area by the moved phrasal constituent was too small to be detectable under our experimental conditions. In other words, without the ‘preactivation’ of Broca’s area, e.g. during a ‘nonsyntactic’ task such as animate/inanimate judgments and/or using a higher magnetic field strength (3 T) and/or a larger number of sentences, its modulation with respect to movement might be detectable. Thus, the specific claims made by the TDH about the neuronal substrate of grammatical dependencies in syntactic movement remain to be verified.
Acknowledgements The technical assistance of S. Heinemann, S. Kohnen and T. Fritzsche is gratefully acknowledged. This work was supported by grants from BMBF (FKZ: 01GA0202 and Berlin Neuroimaging Center), DFG (Clinical Research Group EI-207/2-3), International Leibniz Program, and Graduate Program Berlin (Scholarship Nachwuchsfoerderung).
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
313
References Baayen, R. H., Piepenbrock, R., & Rijn, H. (1993). The CELEX lexical database (Release 1) [CD-ROM]. Philadelphia, PA: Linguistic Data Consortium, University of Pennsylvania. Berndt, R. S., & Caramazza, A. (1980). Semantic operations deficits in sentence comprehension. Psychological Research, 41, 169 –177. Burock, M. A., Buckner, R. L., Woldorff, M. G., Rosen, B. R., & Dale, A. M. (1998). Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport, 9, 3735–3739. Caplan, D., Alpert, N., & Waters, G. (1998). Effects of syntactic structure and propositional number on patterns of regional cerebral blood flow. Journal of Cognitive Neuroscience, 10, 541 –552. Caplan, D., Alpert, N., & Waters, G. (1999). PET studies of syntactic processing with auditory sentence presentation. Neuroimage, 9, 343–351. Caplan, D., Alpert, N., Waters, G., & Olivieri, A. (2000). Activation of Broca’s area by syntactic processing under conditions of concurrent articulation. Human Brain Mapping, 9, 65– 71. Caramazza, A., & Zurif, E. B. (1976). Dissociation of algorithmic and heuristic processes in language comprehension: evidence from aphasia. Brain Language, 3, 572–582. Chomsky, N. (1981). Lectures on government and binding. Dordrecht: Foris. Cooke, A., Zurif, E. B., DeVita, C., Alsop, D., Koenig, P., Detre, J., Gee, J., Pinango, M., Balogh, J., & Grossman, M. (2002). Neural basis for sentence comprehension: grammatical and short-term memory components. Human Brain Mapping, 15, 80–94. Dale, A. M. (1999). Optimal experimental design for event-related fMRI. Human Brain Mapping, 8, 109–114. Dapretto, M., & Bookheimer, S. Y. (1999). Form and content: dissociating syntax and semantics in sentence comprehension. Neuron, 24, 427–432. Embick, D., Marantz, A., Miyashita, Y., O’Neil, W., & Sakai, K. L. (2000). A syntactic specialization for Broca’s area. Proceedings of the National Academy of Sciences USA, 97, 6150– 6154. Friederici, A. D., Meyer, M., & von Cramon, D. Y. (2000a). Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information. Brain Language, 75, 289– 300. Friederici, A. D., Opitz, B., & von Cramon, D. Y. (2000b). Segregating semantic and syntactic aspects of processing in the human brain: an fMRI investigation of different word types. Cerebral Cortex, 10, 698–705. Friston, K. J., Holmes, A. P., Price, C. J., Buchel, C., & Worsley, K. J. (1999a). Multisubject fMRI studies and conjunction analyses. Neuroimage, 10, 385–396. Friston, K. J., Holmes, A. P., & Worsley, K. J. (1999b). How many subjects constitute a study? Neuroimage, 10, 1–5. Friston, K., Holmes, A. P., Worsley, K., Poline, J. B., Frith, C., & Frackowiak, R. S. (1995). Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2, 189 –210. Grodzinsky, Y. (1986). Language deficits and the theory of syntax. Brain Language, 27, 135–159. Grodzinsky, Y. (1995). A restrictive theory of agrammatic comprehension. Brain Language, 50, 27–51. Grodzinsky, Y. (2000). The neurology of syntax: language use without Broca’s area. Behavioral Brain Science, 23, 1– 21. Grodzinsky, Y., & Finkel, L. (1998). The neurology of empty categories aphasics’ failure to detect ungrammaticality. Journal of Cognitive Neuroscience, 10, 281–292. Holmes, A. P., & Friston, K. J. (1998). Generalizability, random effects, and population inference. Neuroimage, 7, S754. Indefrey, P., Hagoort, P., Herzog, H., Seitz, R. J., & Brown, C. M. (2001). Syntactic processing in left prefrontal cortex is independent of lexical meaning. Neuroimage, 14, 546–555. Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1996). Brain activation modulated by sentence comprehension. Science, 274, 114– 116. Kang, A. M., Constable, R. T., Gore, J. C., & Avrutin, S. (1999). An event-related fMRI study of implicit phraselevel syntactic and semantic processing. Neuroimage, 10, 555 –561. Linebarger, M. C., Schwartz, M. F., & Saffran, E. M. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13, 361–392.
314
I. Wartenburger et al. / Journal of Neurolinguistics 16 (2003) 301–314
Moro, A., Tettamanti, M., Perani, D., Donati, C., Cappa, S. F., & Fazio, F. (2001). Syntax and the brain: disentangling grammar by selective anomalies. Neuroimage, 13, 110–118. Ni, W., Constable, R. T., Mencl, W. E., Pugh, K. R., Fulbright, R. K., Shaywitz, S. E., Shaywitz, B. A., Gore, J. C., & Shankweiler, D. (2000). An event-related neuroimaging study distinguishing form and content in sentence processing. Journal of Cognitive Neuroscience, 12, 120 –133. Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9, 97– 113. ¨ ber Agrammatismus als folge cerebraler herderkrankungen; ein beitrag zur lehre vom Pick, A. (1898). U Verha¨ltnis der worttaubheit. Zur taubheit. Beitra¨ge zur pathologie und pathologischen anatomie des zentralnervensystems, Berlin: Karger, p. 123 –133. Price, C. J., Veltman, D. J., Ashburner, J., Josephs, O., & Friston, K. J. (1999). The critical relationship between the timing of stimulus presentation and data acquisition in blocked designs with fMRI. Neuroimage, 10, 36–44. Roder, B., Stock, O., Neville, H., Bien, S., & Rosler, F. (2002). Brain activation modulated by the comprehension of normal and pseudo-word sentences of different processing demands: a functional magnetic resonance imaging study. Neuroimage, 15, 1003–1014. Stowe, L. A., Broere, C. A., Paans, A. M., Wijers, A. A., Mulder, G., Vaalburg, W., & Zwarts, F. (1998). Localizing components of a complex task: sentence processing and working memory. Neuroreport, 9, 2995–2999. Stromswold, K., Caplan, D., Alpert, N., & Rauch, S. (1996). Localization of syntactic comprehension by positron emission tomography. Brain Language, 52, 452–473.