- Email: [email protected]
Differential cortical contribution of syntax and semantics: An fMRI study on two-word phrasal processing
Accepted Manuscript Differential cortical contribution of syntax and semantics: An fMRI study on two-word phrasal processing Marianne Schell, Emiliano Zaccarella, Angela D. Friederici PII:
S0010-9452(17)30297-6
DOI:
10.1016/j.cortex.2017.09.002
Reference:
CORTEX 2123
To appear in:
Cortex
Received Date: 5 February 2017 Revised Date:
17 July 2017
Accepted Date: 5 September 2017
Please cite this article as: Schell M, Zaccarella E, Friederici AD, Differential cortical contribution of syntax and semantics: An fMRI study on two-word phrasal processing, CORTEX (2017), doi: 10.1016/ j.cortex.2017.09.002. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
DIFFERENTIAL CORTICAL CONTRIBUTION OF SYNTAX AND SEMANTICS:
M AN U
SC
RI PT
AN FMRI STUDY ON TWO-WORD PHRASAL PROCESSING
TE D
Marianne Schell, Emiliano Zaccarella, Angela D. Friederici
Max Planck Institute for Human Cognitive and Brain Sciences,
AC C
EP
Department of Neuropsychology, Stephanstraße 1a | 04103 Leipzig, Germany
Correspondence should be addressed to: Marianne Schell Stephanstraße 1a, 04103 Leipzig T +49 341 9940 2238 [email protected]
1
ACCEPTED MANUSCRIPT
Abstract Linguistic expressions consist of sequences of words combined together to form phrases and sentences. The neurocognitive process handling word combination is
RI PT
drawing increasing attention among the neuroscientific community, given that the underlying syntactic and semantic mechanisms of such basic combinations— although essential to the generation of more complex structures—still need to be
SC
consistently determined. The current experiment was conducted to disentangle the neural networks supporting syntactic and semantic processing at the level of two-
M AN U
word combinations. We manipulated the combinatorial load by using words of different grammatical classes within the phrase, such that determiner-noun combinations (this ship) were used to boost neural activity in syntax-related areas, while adjective-noun combinations (blue ship) were conversely used to measure
TE D
neural response in semantic-related combinations. By means of functional magnetic resonance imaging (fMRI), we found that syntax-related processing mainly activates the most ventral part of the inferior frontal gyrus, along the frontal operculum (FOP)
EP
and anterior insula (aINS). Fine-grained analysis in BA44 confirmed that the most inferior-ventral portion is highly sensitive to syntactic computations driven by function
AC C
words. Semantic-related processing on the contrary, rather engages the anterior dorsal part of the left inferior frontal gyrus (IFG) and the left angular gyrus (AG) that is two regions which appear to perform different functions within the semantic network. Our findings suggest that syntactic and semantic contribution to phrasal formation can be already differentiated at a very basic level, with each of these two processes comprising non-overlapping areas on the cerebral cortex. Specifically, they confirm the role of the ventral IFG for the construction of syntactically legal linguistic
ACCEPTED MANUSCRIPT constructions, and the prominence of the more anterior IFG and the AG for conceptual semantics.
Keywords
RI PT
Inferior Frontal Gyrus; Angular Gyrus; Syntax; Semantics; Two-word combinatorics
Highlights
Two-word phrasal combinatorics constitute the bottom level of linguistic complexity
•
Syntactic and semantic contribution to phrasal formation is cortically differentiable
•
Syntactic processing mainly activates the ventral Inferior Frontal Gyrus (IFG)
•
Semantic processing engages the anterior-dorsal IFG and the Angular Gyrus (AG)
•
The anterior-dorsal IFG and the AG play distinct roles within the semantic network
AC C
EP
TE D
M AN U
SC
•
2
ACCEPTED MANUSCRIPT
1 Introduction Human language is characteristically defined as a highly productive cognitive ability: words can be continuously combined together to create an infinite number of new
RI PT
expressions of increasing complexity. To comprehend such novel expressions in an effective way, our brain needs to do two things: it has to assess what the meaning of the individual words mean when put together, and it has to retrieve the abstract underlying structure binding such expressions. These two processes refer to the
SC
linguistic aspects of semantics and syntax. Specifically, semantic composition drives
M AN U
the combination of the meanings of the individual words to form meaningful expressions. These combinational computations are the most fundamental mechanisms at the root of every language (Hauser, Chomsky, & Fitch, 2002). Syntactic computations, conversely, implement the rules that govern the abstract
TE D
architecture of the same expressions (Chomsky, 1995). Traditionally, neurolinguistic investigations have tried to manipulate semantic composition and syntactic computations within the context of full sentences (for a review see Friederici (2011).
EP
Semantic composition has been mostly examined either by varying the semantic load in sentences with or without pseudowords to reduce semantics information
AC C
(Humphries, Binder, Medler, & Liebenthal, 2006; Mazoyer et al., 1993; Pallier, Devauchelle, & Dehaene, 2011), by varying semantic complexity to contrast short sentences against longer sentences or narratives (Pallier et al., 2011; Stowe et al., 1998; Xu, Kemeny, Park, Frattali, & Braun, 2005), or by evaluating semantic plausibility in a specific context (Newman, Ikuta, & Burns, 2010; Zhu et al., 2013; Zhu et al., 2009). Syntactic computations, conversely, have been mainly investigated by comparing sentences to word lists lacking syntactic information (Friederici, Meyer, & Von Cramon, 2000; Humphries et al., 2006; Mazoyer et al., 1993; Pallier et al., 2011; 3
ACCEPTED MANUSCRIPT Stowe et al., 1998; Xu et al., 2005), by evaluating different degrees of syntactic complexity (Bornkessel, Zysset, Friederici, von Cramon, & Schlesewsky, 2005; Makuuchi, Bahlmann, Anwander, & Friederici, 2009), by focusing on syntactic errors (Friederici, Ruschemeyer, Hahne, & Fiebach, 2003; Vandenberghe, Nobre, & Price,
RI PT
2002), or by using syntactic priming (Segaert, Menenti, Weber, Petersson, & Hagoort, 2012). The sentential level however might not be the best approach to explore the neural basis of basic compositional processes in the semantic and
SC
syntactic domain since multiple combinations of words and complex structures tend to involve additional mechanisms external to syntactic computations and semantic
M AN U
composition. These additional mechanisms are working memory and storage (Makuuchi et al., 2009; Makuuchi & Friederici, 2013; Meyer, Obleser, Anwander, & Friederici, 2012; Santi & Grodzinsky, 2007), cognitive control and ambiguity resolution (Badre, 2008; Koechlin, Ody, & Kouneiher, 2003), and at the text level
TE D
even integration into the context of discourse (Egidi & Caramazza, 2013). More recently, researchers have started to look at compositional processing in the
EP
semantic and syntactic domain at more basic levels, investigating how the brain behaves during the combination of very simple two- or three-word phrasal structures,
AC C
such as “blue boat”, “this boat” or “on the boat”. Within the neurolinguistic tradition, the expression basic levels of linguistic processing therefore refers to the build-up of simple structures beyond single words, where the combination of independent lexical elements (e.g. an adjective and a noun, or a determiner and a noun) are combined together to form elementary phrases or sentences at the root of linguistic complexity. Evidence from different languages is now available, including English (Bemis & Pylkkanen, 2011, 2012a, 2012b, 2013; Del Prato & Pylkkanen, 2014; Westerlund, Kastner, Al Kaabi, & Pylkkanen, 2015; Westerlund & Pylkkanen, 2014; Zhang &
4
ACCEPTED MANUSCRIPT Pylkkanen, 2015), Spanish (Molinaro, Carreiras, & Duñabeitia, 2012; Molinaro, PazAlonso, Dunabeitia, & Carreiras, 2015), and German (Zaccarella & Friederici, 2015a; Zaccarella, Meyer, Makuuchi, & Friederici, 2015).
RI PT
Focusing on the semantic domain, the first of these studies (Bemis & Pylkkanen, 2011) used adjective-noun combinations and reported increased cortical activity for phrasal composition in the anterior temporal lobe, ATL (and in the ventral prefrontal cortex). In this study, participants were asked to combine together in a visual task a
SC
descriptive adjective “red” with a noun “boat” to be matched with a picture. The ATL
M AN U
was found equally active when the same types of stimuli were processed in the auditory modality (Bemis & Pylkkanen, 2012a). Given the type of stimulus used, the authors concluded that the ATL might be the region particularly engaged during semantic composition, deriving the meaning of a simple phrase like 'red boat' from the conjunction of the two simpler concepts of “redness” and “floating object”,
TE D
expressed by the adjective 'red' and the noun 'boat' respectively (Smith, 1984). The authors base their interpretation on findings from single word processing, for which
EP
activity in the ATL was reported when words had to be classified conceptually as referring to a living or non-living object (C. J. Price, Moore, Humphreys, & Wise,
AC C
1997). At the clinical level, evidence supporting a semantic role for the ATL, comes from patient studies which traditionally link the region to semantic processing deficits (e.g. semantic dementia) following temporal lobe atrophy (Bonner et al., 2009; Galton et al., 2001; Gorno-Tempini et al., 2004; Rohrer et al., 2009). Patients with semantic dementia generally show deficit in conceptual knowledge for various domains (Gorno-Tempini et al., 2011; Hodges et al., 2010; Snowden, 1995). As such, the ATL might integrate information associated with the respective concepts during the processing of words (Lambon Ralph, Sage, Jones, & Mayberry, 2010; Patterson,
5
ACCEPTED MANUSCRIPT Nestor, & Rogers, 2007; Visser, Jefferies, & Lambon Ralph, 2010). According to this view specific conceptual information (e.g. the boat’s shape, color, sound, worldknowledge, etc.) is stored in corresponding cortical areas, while the ATL serves as an
RI PT
amodal hub to fuse the different aspects of conceptual information. A second region proposed to be relevant for semantic composition is the angular gyrus, AG (Binder, Desai, Graves, & Conant, 2009). The AG was found to be involved in addition to the ATL during the construction of two-word semantic
SC
composition, regardless of modality, although the AG activated at a later time point
M AN U
than the anterior temporal region (Bemis & Pylkkanen, 2012a). Furthermore, it was shown that activation in AG varied as a function of plausibility by comparing meaningful (e.g. plaid jacket) with non-meaningful phrases (e.g. moss pony) in healthy subjects (A. R. Price, Bonner, Peelle, and Grossman (2015). The activity in AG was positively correlated with the combinatorial strength, according to a normed
TE D
measure of plausibility. This result assigns a modulating aspect to the AG during the semantic composition of words, while a more complex concept is built. Another study
EP
showed a significant functional coupling between the ATL and the AG for the processing of noun-adjective pairs, but only for the adjectives with low predictability
AC C
(implausible phrases) (Molinaro et al. (2015). At the single word level, it has recently been demonstrated that semantic decisions over word meanings appear to rely on the functional co-working between the AG and a third fundamental region of the semantic system, that is BA45 (Hartwigsen et al., 2016). Specifically, Hartwigsen and colleagues have shown that functional disruption by means of TMS over either left anterior IFG (BA45) or left AG alone does not significantly affect semantic decisions (natural vs. man-made item), suggesting that the respective other non-lesioned region can compensate for the TMS-induced effect. However, combined TMS over
6
ACCEPTED MANUSCRIPT both areas leads to significantly longer reaction times. BA45 and AG can compensate each other in evaluating semantic information for particular words, but their corresponding specific roles within the semantic network still need to be disentangled. At complex combinatorial levels, BA45 is known to be modulated by
RI PT
the processing of the verbal information in sentential constructions, which in turn guarantees the comprehension of the overall propositional meaning of a sentence (Newman et al., 2010; Zhu et al., 2009). The AG has been ascribed to reflect some
M AN U
information are integrated (Humphries et al., 2006).
SC
general form of semantic processing, in which incoming pieces of semantic
Overall, neurolinguistic evidence for semantic processes so far suggests that simple semantic composition might rely on the recruitment of regions in the ATL and the AG. In parallel, the anterior IFG (BA45) is certainly involved during semantic composition at more complex levels, when verbal information is assessed. However, it appears to
performed.
TE D
be recruited in compliance with the AG also when more basic semantic tasks are
EP
Studies focusing on the syntactic domain ask how the underlying syntactic structure of such simple two-word combinations is neurally implemented. Do perceivers
AC C
automatically build phrasal hierarchies or can semantic composition of two content words be processed without any syntactic involvement? In a first investigation of twoword phrases without any semantic information, Zaccarella et al. (2015) used twoword phrases formed by a determiner and a pseudonoun (nouns without any semantic meaning, e.g. this flirk) to specifically focus on syntactic processing. As such, the study looked at the syntactic computational effect between the two elements, once the descriptive semantic information was removed from the stimulus. The authors reported increased neural activity in the left IFG and specifically in BA 44 7
ACCEPTED MANUSCRIPT for the syntactic effect, but no significant engagement of the ATL regions. Additional analysis in BA 44 further revealed that only the ventral-anterior part was selectively responsible for syntactic processing, which they argued reflect the merge computation described in both theoretical linguistics (Chomsky, 1995) and
RI PT
evolutionary linguistics (Berwick, Friederici, Chomsky, & Bolhuis, 2013; Zaccarella & Friederici, 2016). Crucially, the sub-regional analysis in this study was grounded on a recent co-activation-based parcellation in BA44, which subdivided the region meta-
SC
analytically into five different functional clusters (Clos, Amunts, Laird, Fox, & Eickhoff, 2013). Within the area, the two posterior clusters were mostly found associated with
M AN U
phonology/overt speech (Cluster 1, dorsally) and rhythmic sequencing (Cluster 4, ventrally). The two more anterior ones were conversely linked to working memory (Cluster 2, dorsally) and linguistic tasks including syntax (Cluster 3, ventrally), while the fifth one in the inferior frontal junction was associated to task switching and
TE D
cognitive control (Cluster 5). Taken together, these results suggest a distinct role for syntactic computations in region BA44.
EP
Supporting evidence for a neural involvement of BA 44 and the frontal operculum in syntactic computations comes from many studies on sentence processing which
AC C
selectively varied the degree of syntactic complexity by comparing more complex object-first sentences versus subject-first sentences (Bornkessel et al., 2005; Fiebach, Schlesewsky, Lohmann, von Cramon, & Friederici, 2005; Grewe et al., 2005) or by employing sentences with varying degrees of embeddedness (Friederici, Bahlmann, Friedrich, & Makuuchi, 2011; Jeon & Friederici, 2013; Makuuchi et al., 2009). Moreover, lesion studies could show that atrophy in Broca’s area is related to a deficit in processing of syntactic hierarchy (Deleon et al., 2012; Rogalski et al., 2011; Wilson et al., 2010; Wilson, Galantucci, Tartaglia, & Gorno-Tempini, 2012). In
8
ACCEPTED MANUSCRIPT general, these studies converge in assigning a major role of BA 44 for syntactic processing. Against this background, it appears that contribution of syntax and semantics
to
phrasal
combinatorics
corresponds
to
functionally
distinct
RI PT
neuroanatomical regions within the left hemispheric cortex. The hypothesis that follows from this is that the processing of phrases formed by nouns preceded by open class elements—like descriptive adjectives (“red boat”)— might particularly upregulate activity in those cortical regions specifically sensitive to
SC
semantic combination. Adjectives, as content words (like nouns and verbs), primarily
M AN U
carry semantic information, and the presence of such a word in an adjective-noun pair increases the semantic processing load in the language system (Zaccarella et al., 2015) which might lead to narrowed conceptual specificity (Westerlund & Pylkkanen, 2014). However, given the fact that for German phrases the adjectiveform must be adapted according to gender, case and number of the noun (so-called
TE D
inflectional morphology), some syntactic elements are retained in semantic phrases as well (-es, in blaues Schiff/ blue ship). Conversely, the processing of phrases
EP
formed by nouns preceded by closed class items—like determiners (“this boat”)— with reduced descriptive content, would rather engage activity in syntax-related
AC C
regions, and more closely resemble pure syntactic computational processing (Zaccarella & Friederici, 2015a). So far, however, no study has directly tested whether syntactic computations and semantic compositions can be differentiated at such basic combinatorial level. The goal of the present fMRI study was to assess the functional contribution of semantics and syntax to the formation of basic two-word phrases directly in the same group of participants. In our study, we used acoustic stimuli and varied the type of word class driving the combination across two different conditions of two-word German stimuli. To identify 9
ACCEPTED MANUSCRIPT distinctive cortical regions across the brain that would implement either syntactic computation or semantic composition at the very basic two-word level, we created semantic (driven) stimuli (SEM) comprising adjective-noun phrases, and syntactic (driven) stimuli (SYN) comprising determiner-noun phrases. In addition to these
RI PT
experimental conditions we further included a third one-word non-combinatorial condition where the first word driving the phrasal effect was absent, while the second word, the noun, was kept in the identical position as in the other two conditions.
SC
Moving from previous works on the functional differentiation between phrasal effect and word accumulation effect (Zaccarella and Friederici 2015; Zaccarella et al. 2017),
M AN U
as well as recent studies on the effect of semantic processing at single-word and twoword level (Hartwigsen et al., 2016; Molinaro et al., 2015; A. R. Price et al., 2015), this additional condition served as baseline (BAS) to isolate across the brain those areas sensitive to word increase (2 words > 1word), when more syntactic information
TE D
or semantic information was added to the process. From the relative phrase-sensitive functional patterns obtained above, our first aim was that of comparing the brain activity produced by the two phrasal constructions in those regions, to search for a
EP
double dissociation between phrasal conditions and anatomical regions on the cerebral cortex. As such, we did not include any further baseline consisting of lists of
AC C
two nouns for example, since word accumulation processing for two-word lists vs. one-word lists was not within the scope of the current experiment, although already explicitly tested in other studies (Bemis & Pylkkanen, 2011; Zaccarella & Friederici, 2015b). Further, our second aim was that of directly comparing across the brain the two phrasal conditions against each other, when the baseline effect was not taken into account. These contrasts were meant to verify whether within the two networks the same anatomical regions found to be sensitive to the increased amount of
10
ACCEPTED MANUSCRIPT syntax-relevant and semantics-relevant words, remain essentially equally modulated by syntactic features (the determiner) and semantic features (the adjective) regardless of the amount of words forming the phrase. For instance, in the most anterior-ventral BA44, neural activity for two-word syntactic phrases was found to be
RI PT
significantly higher than both one-word phrases and two-word lists (Zaccarella & Friederici, 2015a), therefore speaking in favor of a pure syntactic role of this area. Conversely, it is completely unknown whether regional specificities within the
SC
semantic network can be differentiated into different functions, such that the activity in some areas vary due to the increased amount of semantic information, while some
M AN U
others only evaluate conceptual information of words regardless of the number of words. Finally, our third aim was that of testing whether at fine-grained level, the most ventral-anterior parcel within BA44 (Clos, Amunts, Laird, Fox, & Eickhoff, 2013) would have shown to be sensitive to both types of phrases, or rather active for syntax
TE D
only, as previously found when contrasting determiner-pseudoword phrases against a non-compositional word-list condition (Zaccarella & Friederici, 2015a).
EP
Based on previous literature, we expected to find neural activation for simple phrase processing in language-relevant areas of the temporo-fronto-parietal network
AC C
lateralized to the left hemisphere, including Broca’s area in the IFG, Wernicke’s area in the pSTG/STS, as well as the ATL and the AG (Catani, Jones, & Ffytche, 2005; Friederici, 2011). More specifically, we expected to find activation in syntax-related areas, namely the left IFG (BA 44), frontal operculum/anterior insula and the superior and middle temporal lobe for syntactic phrases (SYN) in which the noun combines with the determiner, independent of the contrast of interest. At a more fine-grained level we also tested the hypothesis that within BA 44 only the most anterior ventral portion of the area (Cluster 3) would be selectively active for syntactic computations
11
ACCEPTED MANUSCRIPT (Clos et al., 2013). Here we were interested to see whether this activation can also be found in a real word phrase condition, and if so, whether this would hold for syntactic phrases only (dieses Schiff/this ship), or also for semantic phrases (blaues Schiff/blue ship), since in the latter case adjectives obligatory carry inflectional
RI PT
morphemes (-es). We had two conflicting hypotheses regarding the processing of semantic phrases (SEM). On the one hand, semantic phrases (blaues Schiff/blue ship) could be processed like syntactic phrases with additional demands of semantic
SC
composition caused by the descriptive semantic information carried by the adjectives. If so, the activation pattern for semantic phrases would yield to an increase in the
M AN U
posterior IFG (BA44) due to syntactic morphology in addition to recruitment of the ATL, BA45 and the AG, as the semantic related regions (Bemis & Pylkkanen, 2011). On the other hand, semantic phrases could be processed primarily at a compositional level mainly in semantic-related areas, with syntactic information carried by the
TE D
inflectional morphology triggering automatic syntactic computations with low processing demands in BA44 (Pulvermuller, Shtyrov, Hasting, & Carlyon, 2008). In either case, we left open the issue of whether the different regions of the semantic
EP
network could be ascribed to distinct roles within the semantic system, such that the activity in some of them would vary due to the increased amount of semantic
AC C
information, while in some others it would vary as a reflection of what is more relevant to semantics for the conceptual evaluation of words, regardless of the amount of words.
12
ACCEPTED MANUSCRIPT
2 Methods 2.1 Participants Twenty-one native Germans (10 females and 11 males; mean age of 27.7 yrs.,
RI PT
ranging from 21 to 36 yrs. of age) participated in the study. All participants were right handed as assessed with the Edinburgh questionnaire (Oldfield, 1971) and reported normal hearing. None of them had any history of neurological or psychiatric disorder.
SC
Written informed consent was obtained from all participants according to the procedures approved by the Research Ethics Committee of the University of Leipzig.
M AN U
All participants received monetary compensation after completing the experiment.
2.2 Stimuli
We used two different types of two-word phrases, namely syntactically driven phrases (SYN) and semantically driven phrases (SEM), in addition to a non-
TE D
combinatorial baseline condition (BAS). The syntactically driven phrases were formed by combining nouns with determiners—function words, which carry syntactic
EP
information and which specify syntactic relations within the phrase. The semantically driven phrases were constructed by combining the same nouns with descriptive
AC C
adjectives—content words conveying additional semantic content to the noun. The non-combinatorial baseline condition consisted of single nouns only (see also Figure 1A). Throughout all conditions, 40 one-syllable nouns (20 masculine and 20 neutral) were employed, selected by using the Wortschatz Lexicon of the University of Leipzig (available at http://wortschatz.uni-leipzig.de/). Only concrete nouns with a high frequency were chosen. The mean frequency f was 10.92 (SD=2.09), where f means that the most frequent word “der” (the) is 2f times more frequent than the number of the target word. We used four color adjectives for the SEM condition and
13
ACCEPTED MANUSCRIPT four determiners for the SYN condition, together with their inflectional endings (-er for masculine nouns, -es for neutral nouns), according to the gender of the following noun. All stimuli were single-channel recorded by a trained male native speaker of German language. The speaker spoke all words individually in the form of a
RI PT
statement. Recordings were made in a sound-attenuating chamber with a resolution of 16 bits and a sampling rate of 44.1 kHz. Offline, the stimuli were edited into separate files and normalized for root mean square.
SC
Nouns had a mean duration of 0.503 s (SD=0.049). Mean duration for adjectives and
M AN U
determiners was 0.628 s (SD=0.064, Mdn=0.629, n1=8) and 0.625 s (SD=0.062, Mdn=0.620, n2=8), respectively. The distributions of adjectives and determiners showed no significant difference at p ≤ 0.05 with a U-value of 31 and a critical value of 13. To keep the final length of the stimuli constant between conditions, all recordings were embedded in a speech-shaped noise (SSN) with signal noise ratio of
TE D
6dB. Here, we used all recorded stimuli, calculated the spectral envelope of the signal and used this to shape a white noise signal. In the two-word conditions, the
EP
stimuli were concatenated with a 40 ms pause between the words and added to the SSN with the second word starting at 900 ms. The baseline condition was also added
AC C
to the SSN with the noun starting at 900 ms. This yielded to a mean length of 1.443 s (SD=0.049) for all conditions. In a final step, we also calculated the co-occurrence frequencies or ‘combinatorial strength’ of our two-word phrases as described in (A. R. Price et al., 2015). According to the authors this measure strongly correlates with the behavioral plausibility ratings of the word pair. To ensure that our conditions did not differ in terms of co-occurrence frequencies between the two words, we performed an unpaired t-test between the frequencies of adjective-noun phrases and the
14
ACCEPTED MANUSCRIPT frequencies of determiner-noun phrases. The t-test however did not reveal any frequency differences between the two conditions (t(318)=0.75, p=0.94). All stimuli were assigned to four randomized sets, so that one noun occurred only
RI PT
once for each condition, while adjectives and determiner where counterbalanced within a set. The final pool consisted of 120 stimuli, 40 for each condition and 40 filler conditions (see next section). Every participant was tested on an individually
SC
randomized version of the set.
2.3 Procedure
M AN U
Participants performed a short practice session on a desktop computer outside the scanner room on a different set of stimuli. All stimuli were presented using the software package of Presentation® (Neurobehavioural Systems, Inc., Albany, CA, USA) with air-conducted headphones (Resonance Technology, Inc., Northridge, CA,
TE D
USA). Once in the scanner, volume was adjusted to an optimal hearing level based on the feedback given by the participant. In line with previous MEG/EEG or fMRI studies testing semantic phrasal processing or syntactic phrasal processing, and
EP
discussed earlier in the introductory section, we also used an event-related design to ensure data comparability (Fig. 1B). The choice of an event-related design was
AC C
fundamentally driven by the fact that we were primarily interested in trial-related patterns of information processing between syntactic and semantic trials within the same task (Amaro & Barker, 2006), rather than aiming at uncovering sustained taskrelated activity during phrasal formation against control tasks—for which mixed designs are especially well suited, notwithstanding the introduction of independent research questions, like the involvement of more complex computations and the a priori stipulations of the optimal amount of trials for each block (Donaldson, 2004).
15
ACCEPTED MANUSCRIPT Therefore, because of the very reduced nature of the stimuli included in the experiment, together with the sparse fMRI data available, an event-related design
EP
TE D
M AN U
SC
RI PT
seems most suitable (Visscher et al., 2003).
AC C
Figure 1: Conditions and experimental design. A: Experimental conditions used for the fMRI study, SEM—semantically driven phrase (in blue), SYN—syntactically driven phrase (in red) and BAS—non-combinatorial baseline condition (in brown). B: Experimental design, every trial started with a fixation cross followed by the auditory stimulus. During the question mark, the participant had to decide if the stimulus could be part of a normal sentential structure.
In our design, each experimental trial started with a random jitter of 0, 500 or 1000 ms and was followed by the auditory stimulus. Within a reaction time window of 3 s, participants had to judge whether the stimulus within a trial of either two- or oneword length could be integrated in a normal sentential structure. Participants were
16
ACCEPTED MANUSCRIPT instructed to respond with the index and middle finger of their right hand on a twobutton response box. We used additional filler trials, which did not meet the relevant criteria (e.g. “blaues diese”, blue this), and for which subjects were requested to press the “no” button. The assignment of the buttons was counterbalanced across
RI PT
participants. The subsequent trial started after an inter-stimulus-interval of 3, 4 or 5 multiple of TR, 2.09 s, logarithmically distributed (7.733 s on average). All trials were included for further analysis. For the SYN condition—which included determiner-noun
more
specifically
in
the
most
anterior
SC
combinations—we expected to find increased neural responses in the left BA44–and ventral
section—in
the
frontal
M AN U
operculum/anterior insula, and in the superior and middle temporal gyrus independent of the contrast under analysis. With respect to the SEM condition— which conversely included adjective-noun combinations—we hypothesized that either adjective-noun phrases would have been processed in the same manner as we
TE D
expected for determiner-noun phrases—therefore triggering BA44 activity—or rather only triggering automatic syntactic analysis with low processing demands in BA44. Either way, we expected BA45, the AG and the ATL to activate as reflection of
EP
semantic processing, although we could not know whether these regions would have shown distinct patterns of activity for the different contrasts of interest, therefore
AC C
reflecting different functions within the semantic system.
2.4 Data acquisition
MRI data were obtained using a 3 Tesla Siemens Tim Trio MR scanner (Siemens Medical Systems, Erlangen, Germany) and a 12-channel headcoil at Max-Planck Institute for Human Cognitive and Brain Science in Leipzig, Germany. Functional images were acquired using a T2* weighted gradient-echo echo-planar-imaging (EPI) with 40 axial slices parallel to AC-PC line with whole brain coverage in ascending
17
ACCEPTED MANUSCRIPT direction with a functional scanning time of 27 minutes and 767 volumes. Scans had an in-plane resolution of 3 × 3 mm and a 2.5 mm slice thickness with an inter-slice gap of 0.5 mm (TR = 2.09 s, TE = 22 ms, flip angle 90°, field of view 192 mm, 64×64 matrix size). For all participants, individual high-resolution 3D T1-weighted MR scans
RI PT
acquired in previous sessions were available for normalization and co-registration.
2.5 Behavioral data analysis
SC
Reaction times were analyzed using a repeated-measures one-way ANOVA with the factor CONDITION (SYN, SEM, BAS), followed by paired t-tests between the three
corrected α = 0.05/3 = 0.016.
2.6 MRI Data analysis 2.6.1 Preprocessing
M AN U
different levels of the factor CONDITION within each cluster, with α=0.05, Bonferroni-
TE D
Preprocessing and statistical analysis were performed with SPM8 (available at www.fil.ion.ucl.ac.uk/spm, Wellcome Imaging Department, University Collage,
EP
London, UK) implemented in MATLAB 2010b (Mathworks, Inc., Sherborn, MA, USA). During the preprocessing, all functional images were realigned to the first EPI image
AC C
to correct for head motion; unwrapped to correct for geometric distortions due to susceptibility gradients based on the B0 field-map; resliced for time series correction and co-registered with the corresponding subject-specific structural T1-weighted image. We performed the normalization to standard MNI template as included in the SPM software package using a unified segmentation with a resampling size of 2 mm isotropic voxels. A Gaussian kernel of 6 mm full-width at half maximum was used to smooth the data. We applied a high-pass filter with a cut off period of 128 sec to
18
ACCEPTED MANUSCRIPT avoid low-frequency drift. The first five let-in volumes were excluded to allow for magnetic saturation effect.
2.6.2 Univariate imaging analysis
RI PT
For each participant we estimated a general linear model (Friston et al., 1995) as implemented in SPM 8 including the three conditions and the filler trials, by using a canonical hemodynamic response function. The six motion parameters from the realignment preprocessing were included to explain additional variance related to
SC
head movements. We first analyzed the BOLD response for the contrast SYN > BAS
M AN U
and SEM > BAS. First-level contrasts between conditions were run for each subject separately and then validated at second-level by means of one-sample t-test statistics. On the resulting cluster from the two contrasts, we used the Marsbar Toolbox for SPM (Brett, Anton, Valabregue, & Poline, 2002) to extract signal change percent over all trials of both syntactic phrases and semantic phrases along all voxels
TE D
within the cluster. The four resulting vectors were then entered into a 2X2 ANOVA with REGION and CONDITION as factors and a threshold set at p < 0.05. Two
EP
following paired t-tests were performed with α=0.05, Bonferroni-corrected α = 0.05/2 = 0.025. To note, an alternative analysis including a global whole-brain
AC C
analysis with combined SYN and SEM conditions against BAS—followed by ROIs analysis in the activated regions—might have compromised the data, given that crucial information on the functional activation differences between the two phrasal conditions would have been lost due to signal averaging across the two conditions during the comparison against the baseline. In a second phase, we performed two additional whole-brain contrasts (SYN > SEM and SEM > SYN) to compare syntactic phrases and semantic phrases directly. To account for the multiple comparison problem, a p < 0.001 voxel threshold was combined with a cluster extended threshold
19
ACCEPTED MANUSCRIPT of 0.05 family wise error (FWE) corrected for all the functional contrasts. The anatomical overlap between the contrasts SYN > BAS and SYN > SEM was also investigated with a conjunction analysis to test the common effects of syntactic computation between the two contrasts, as based on previous literature on local
RI PT
syntactic combinations (Zaccarella & Friederici, 2015a, 2016). We used the significant clusters of the IFG acquired in one-sample t-tests from the univariate
AND or a conjunction null hypothesis.
M AN U
2.6.3 Region-of-Interest Analysis within BA 44
SC
analysis above and calculated the overlapping voxels, which is similar to the logical
In a third phase, we additionally performed a region-of-interest analysis in BA 44, based on the hypothesis that within the region, only one sub-portion of the area (Cluster 3) would have shown some significant activity difference for the syntactic computation. Five clusters of BA44 were defined after Clos et al. (2013). We used
TE D
Marsbar to calculate the average percentage signal change over all trials of each of the three conditions along all voxels within the cluster. Fifteen paired t-tests were
EP
performed with α=0.05, Bonferroni-corrected α = 0.05/15 = 0.00333, where each
AC C
condition was compared to the other two in each of the five regions.
20
ACCEPTED MANUSCRIPT
3 Results 3.1 Behavioral Data Most of the participants performed near ceiling accuracy on all conditions. Median
RI PT
accuracy rate was 98.33% with an interquartile range of 8.99 (IQR=Q3-Q1=99.1792.50). Three participants were excluded because of poor performance, which was below 82.5% (less than Q1–1.5*IQR). The remaining 18 participants (9 females and
SC
9 males; mean age of 27.57 yrs.) were used for further analyses. Despite the high accuracy rate in the judgment task, significant differences in reaction time across the
M AN U
stimulus conditions were found F(1.118, 19.008) = 9.571; p = 0.005 (since the sphericity assumption has been violated χ2(2) = 24.87, p < 0.001, we reported the values after Greenhouse-Geisser correction).
Three paired-samples t-tests were conducted to compare the reaction times between
TE D
the conditions. All t-tests were Bonferroni-corrected for multiple comparisons. The baseline condition (one-word) with a mean reaction time of 0.673 s (SD = 0.107 s) was significantly slower than the SYN condition (M = 0.555, SD = 0.106, t(17) = -
EP
8.2845, p < 0.001) and the SEM condition (M = 0.530, SD = 0.106, t(17) = -12.7927,
AC C
p < 0.001). No significant reaction time difference was found between the SEM condition and the SYN condition (t(17) = 2.0783, p = 0.159). Notably, the reaction times data are in line with those found in previous studies manipulating the same type of experimental stimuli, where subjects were asked to process two-word phrases and single words (Bemis & Pylkkanen, 2012a). It appears that the integration between the two words in the two-word phrases is beneficial to the successful processing of the stimulus, such that for two-word phrases the noun is presented within a context, compared to one-word stimuli, where no such context is available.
21
ACCEPTED MANUSCRIPT Interestingly,
the
facilitatory
effects
of
contextual
information
have
been
demonstrated previously within the neurolinguistic tradition, showing how subjects are faster in matching a noun to a picture, when the same noun is preceded by an another word that can be integrated with it (e.g. adjective), than when it can not
RI PT
(Potter & Faulconer, 1979). This facilitatory effect at basic levels might in turn reflect those well-known facilitatory effects reported at sentential level, when grammatical strings are compared to word-list strings where no integration is possible (Bonhage,
SC
Fiebach, Bahlmann, & Mueller, 2014). Importantly, the present behavioral data speak against the possibility that higher brain activity for two-word phrases, compared to
M AN U
one-word phrases, might be due to higher effort to process the stimuli since the reverse pattern of responses is observed at the behavioral level.
3.2 MRI Data
TE D
3.2.1 Univariate imaging analysis
The activation clusters for the univariate analysis for the contrasts SYN > BAS and SEM > BAS are summarized in Table 1 and Figure 2 (top panels). We found
EP
significant activation for the contrast syntactic phrases against baseline (SYN > BAS) in the middle and posterior parts of the left IFG including the ventral BA44, BA45 and
AC C
the frontal operculum (FOP), and in ventral left superior temporal sulcus, bordering the superior section of the middle temporal gyrus. Semantic phrases against baseline (SEM > BAS) conversely activated a more anterior-dorsal region of the IFG in BA45. The ANOVA across the two different functional clusters in Broca’s area revealed a very strong interaction between the factors REGION and CONDITION (F(1,17) =33.871, p < 0.001), pointing towards a neat double dissociation within the IFG for the two phrasal constructions. One-tailed pared t-tests revealed that syntactic
22
ACCEPTED MANUSCRIPT phrases yield significantly higher activity in the most ventral BA44/45 compared to semantic phrases (t(17) = 3.92, p < 0.001), while semantic phrases yield significantly higher activity in the most dorsal BA45 compared to syntactic phrases (t(17) = 2.21,
AC C
EP
TE D
M AN U
SC
RI PT
p = 0.024).
Figure 2: Activation clusters from the univariate analysis. All contrasts were thresholded at p < 0.001 voxel-level and then 0.05 FWE-corrected at cluster-level. Volume renderings were done with MRIcron (Rorden & Brett, 2000). Diagrams show signal change percentage for each condition in each corresponding cluster. Signal change was extracted with MarsBaR (Brett et al., 2002). Error bars represent the standard error (SE). SYN (red bar); SEM (blue bar); BAS (yellow bar). See also Figure1A for a detailed explanation of the conditions.
23
ACCEPTED MANUSCRIPT The direct contrast syntactic phrases against semantic phrases (SYN > SEM) revealed activation in the ventral part of the IFG and in FOP, whereas the reverse contrast (SEM > SYN) showed activation in the left AG (Fig. 2, bottom panels).
Anatomical regions
SYN>BAS
SEM>SYN
z-score
42
4.308 3.861
z
L IFG, Pars triangularis
-42 -45
29 32
19 7
L STS/MTG
-54 -57 -63 -45 -54 -51 -54
-4 -19 -31 26 20 17 29
-5 1 7 -5 7 19 10
117
105
6.086 4.441 4.071 4.583 4.206 3.527 3.666
-33
-76
40
35
3.875
-63 -31 1 89 -51 -46 10 -66 -40 7 L IFG and FOP/aINS -51 20 -5 49 -48 23 -8 -36 11 -5 * p<0.001 uncorrected (voxel-level), p<0.05 FWE corrected (cluster-level)
4.519 4.167 4.114 3.942 3.859 3.618
L IFG and FOP/aINS
L AG
SC
y
L pSTS/MTG
EP
SYN>SEM
Cluster size*
x
M AN U
SEM>BAS
MNI coordinate
TE D
Contrast
RI PT
Table 1: Activation clusters for the different contrasts. Abbrev: L – left, IFG – Inferior frontal gyrus, STS – superior temporal sulcus, MTG – middle temporal gyrus, FOP/aINS – frontal operculum/anterior insula, AG – angular gyrus.
AC C
The activated region in the IFG for the contrast SYN > SEM—reflecting the comparison function word > content word—was smaller than the contrast SYN > BAS—reflecting function word + inflection > no word (49 voxel and 105 voxel respectively). Additional analyses covaring out reaction time effects across the different conditions revealed nearly identical functional patterns of activation as the ones discussed above (see supplementary Fig. 1 and supplementary Fig. 2).
24
ACCEPTED MANUSCRIPT We finally performed a conjunction analysis between the IFG activations for the contrasts (SYN > BAS) ∩ (SYN > SEM) to reveal commonalities for the syntactic computation driven by function words (Friston, Penny, & Glaser, 2005). Within the area, we found an overlap of 26 voxels, 53.06 % of the activated voxels during the
RI PT
SYN > SEM contrast which was also activated during the SYN > BAS contrast. The conjunction of the IFG activation between the contrasts (SYN > BAS) ∩ (SEM > BAS)
M AN U
SC
revealed no overlapping voxels. See Figure 5 with the IFG-activation for the different
EP
contrasts.
TE D
Figure 3: Overlap of the IFG activations. Sagittal, coronal, and axial cuts at slice -43, 19 and -4. All coordinates in MNI space. The yellow area shows the overlap between the SYN>SEM (red) and SYN>BAS (green) contrasts. The overlap is located in the ventral part of the IFG, including FOP (frontal operculum).
3.2.2 Region-of-Interest Analysis within BA 44
AC C
For all 5 clusters within BA44 we performed paired t-tests to see whether the activations for the two-word conditions differed significantly from the one word baseline. Only cluster CL3—the most anterior ventral one—revealed a significant difference between SYN and the non-combinatorial baseline condition, while none of the other p-values survived the Bonferroni correction (Table 2 for detailed results).
25
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
26
ACCEPTED MANUSCRIPT
1377
x
-56
y
8
z
21
SEM/SYN
0.2384
SYN/BAS SEM/BAS
CL4
CL5
2322
1863
1458
-51
-52
-55
-46
13
18
9
11
33
10
9
25
0.5085
0.0661
0.0170
0.9671
0.0857
0.0402
0.0007*
0.0603
0.0339
0.1874
0.0073
0.0974
0.7943
0.0077
AC C
percentage signal change
1917
CL3
M AN U
uncorrected p-values for conditions
mm³
TE D
center of mass
CL2
EP
volume
CL1
SC
RI PT
Table 2: Results for clusters within BA44. We defined five clusters of BA44 (top-left picture above the table) after (Clos et al., 2013). Fifteen tests were performed with α=0.05, which yielded to a Bonferroni-corrected α = 0.05/15 = 0.00333, where each condition was compared to the other two in each of the five BA44 clusters. Contrasts showing significant differences between conditions after correction are indicated with an asterisk, underlined, and marked in bold. See also (Zaccarella & Friederici, 2015a). The last row shows the diagrams reporting signal change percentage for SYN (red), SEM (blue) and BAS (yellow). Error bars denote the standard error (SE).
27
ACCEPTED MANUSCRIPT
4 Discussion The present study set out to define the neural basis of semantic and syntactic processes during the auditory processing of German two-word combinations. In our
RI PT
functional MRI paradigm, participants listened to two-word phrases in which a noun (boat, BAS) combined either with an adjective—a semantically rich contentive element (blue + boat, SEM), or with a determiner—a syntactically rich functional
SC
element (this + boat, SYN). We found activation for semantic processes (SEM > BAS contrast) in the dorsal part of anterior IFG (BA45), while syntactic processes (SYN >
M AN U
BAS contrast and SYN > SEM contrast) revealed a significant activation of the left ventral-posterior parts of the IFG (BA44), including the frontal operculum and the anterior insula. The double dissociation between the two phrases and regions of activation within the IFG was supported by a strong interaction effect in the area. The
TE D
comparison of SYN > SEM revealed a smaller activation contrast compared to SYN > BAS, which is possibly due to the remaining morphological element in the SEM condition. A further look into syntactic processes by means of a conjunction analysis
EP
between the contrasts (SYN > BAS) ∩ (SYN > SEM) yielded to an overlapping cluster in the ventral IFG region (BA44) extending to the frontal operculum. This
AC C
overlap—which reflects absence of difference between the two contrasts, mostly likely is due to the syntactic properties of the function words available in the syntactic condition. Additional syntax-related activation was found along the middle and posterior
superior
temporal
sulcus,
bordering
the
middle
temporal
gyrus
(SYN > BAS), or in the most posterior section of the posterior temporal sulcus (SYN > SEM). Conversely, the AG was activated for the SEM > SYN contrast, indicating its role in semantic processing.
28
ACCEPTED MANUSCRIPT
4.1 Syntax
and
semantics:
Contribution
of
the
IFG
and
FOP/anterior insula Activation in the left IFG was found by contrasting the two phrase conditions against the baseline condition. While for syntactic phrases the activation was located in the
RI PT
more ventral sections of the IFG, including BA44, BA45 and FOP/aINS, semantic phrases revealed a smaller activation cluster in the more anterior-dorsal part of the IFG (BA45). There was no overlap between the two activations of the IFG (as
SC
revealed by the conjunction analysis), suggesting that syntactic phrases are
M AN U
processed in a different way compared to phrases with semantic content. As such, the larger engagement of the IFG—especially the posterior part and the FOP— together with the result of the percentage signal change in the Cluster C3 of BA44 suggests a prominent role for syntax in this area during syntactic phrase processing. This effect cannot be due to mere word accumulation processing (two-words vs. one-
TE D
word), since the same activation pattern would have been expected for two-word semantic phrases vs. one-word baseline as well. This finding of a special involvement
EP
of Cluster C3 in BA 44 for syntactic phrase processing is in line with previous results (Zaccarella & Friederici, 2015b). The most straightforward interpretation of the
AC C
present data is that while two-word phrases both contain morphological inflections— which cancel each other out during direct contrast (SYN > SEM)—the availability of function words in the syntactic phrases steadily enhances syntactic processing in this area. This also explains why the opposite contrast between the two types of phrasal structures (SEM >SYN) did not reveal activation within the ventral BA44/45.
29
ACCEPTED MANUSCRIPT The semantic contrast (SEM > BAS) reported increased activity in the dorsal BA45, which did not overlap with the activation cluster for the syntactic contrast SYN > BAS. Thus brain activation for the processing of semantic phrases mainly recruits BA45, reflecting increased semantic load due to the additional semantic information carried
RI PT
by the adjective. This findings confirm the role of the anterior IFG (BA 45) for semantic processing, and suggest an involvement of this area not only for the analysis of propositional meaning for language at sentential level (Zaccarella et al.,
SC
2015), but also for basic semantic analysis of minimally complex stimuli, which modulate the region as a function of the quantity of semantic material to be
M AN U
processed. Equally important, it appears that for semantic processing at least at such basic levels, syntactic information linked to the presence of the inflectional morphology only leads to minimal activation in syntax-related regions as reflected in the hemodynamic response (to note, uncorrected p-values in BA44 tend to show that
TE D
at a finer-grained resolution activation differences between semantic phrases and baseline are indeed appreciable, see Table 2). For now, we, therefore, refrain from suggesting that phrasal constructions bypass syntactic information when not directly
EP
relevant. We do so on the basis of additional electrophysiological data indicating that incorrect inflectional morphology leaves its trace in the event-related wave-form
AC C
(Opitz, Regel, Müller, & Friederici, 2013). As such, we tend to follow the idea that syntax is a highly automatic, which can only reliably be measured under the condition of high temporal resolutions of brain activity (Hahne & Friederici, 1999; Pulvermuller et al., 2008). Compared to semantic phrases, syntactic phrases revealed a significant pattern of activity in the ventral parts of IFG (BA4), including frontal operculum and anterior insula. The ventral IFG (BA44) thus appears to be the region particularly engaged
30
ACCEPTED MANUSCRIPT during the establishment of syntactic computations forced by using a syntaxprominent element (a determiner) within the sequence, to form a syntactic phrase. The activation of the (ventral-) posterior parts for the effect of syntax processing is in line with the results of many neuroimaging studies (Bornkessel-Schlesewsky,
RI PT
Schlesewsky, & von Cramon, 2009; Friederici & Kotz, 2003; Hagoort, 2014; Makuuchi, Grodzinsky, Amunts, Santi, & Friederici, 2013; Zaccarella et al., 2015). In a more recent experiment (Goucha & Friederici, 2015) more closely inspected the
SC
role of the IFG in sentence processing, by asking subjects to attentively listen to both meaningful and meaningless sentences while preforming an acoustic monitoring
M AN U
task. When participants listened to meaningful structures, they found activation in the left IFG (BA44/45/47) as an effect of semantic processing. When all meaningful units including derivational morphology were removed from the stimulus, and content words were replaced by pseudowords, functional activation was limited to the BA44
TE D
area as an effect of syntactic analysis. Connectivity studies also suggest a separation of BA44 and BA45 in line with these fMRI findings (for an overview see Friederici and Gierhan (2013)). The dorsal pathway, connecting BA 44 and the posterior temporal
EP
cortex, is strongly associated to syntactic processing (Friederici, 2012; Friederici, Bahlmann, Heim, Schubotz, & Anwander, 2006; Wilson et al., 2011), whereas the
AC C
ventral pathway, which connects BA45 and the superior/middle temporal gyri, is rather taken to be involved in semantic representations (DeWitt & Rauschecker, 2012; Saur et al., 2008).
To note, we found FOP and the aINS supporting the processing of the syntactic computations—but not semantic composition—during the present experimental setting. The FOP/aINS region is considered to be a phylogenetically older and less specialized region than Broca’s area proper (Sanides, 1962) which is thought to
31
ACCEPTED MANUSCRIPT implement more rudimentary forms of syntactic computations which can be processed without computing hierarchical structures in specific contexts (Friederici et al. (2006). Some authors have suggested that this area might work as a word accumulation processor where words are maintained on hold prior delivering the
RI PT
sequences to those areas devoted to the construction of syntactic hierarchies (Stowe et al., 1998). Signal change in FOP/aINS is maximal in syntactically rich contexts and minimal for single words, with semantically rich contexts lying in between. The area
SC
therefore appears to recognize the categorical nature of the elements within the sequence (Friederici et al., 2000), and to string them together without implementing
M AN U
hierarchical building processes. This view is supported by findings showing that the same area was found to be active when two-word lists lacking syntactic information were compared to one-word elements (Zaccarella & Friederici, 2015a). With the present data at hand, we propose that the FOP/aINS supports the combination of two
TE D
subsequent elements, and that the activity in the area attunes to the category of the element, which is maximal when functional elements are introduced. The similar activation profiles for syntactic phrases we found in the ventral IFG and FOP are in
EP
strong agreement with a very recent study probing the intrinsic functional organization of Broca’s region, which found that the ventral BA44 and FOP together
AC C
constitute a functional module with highly similar connectivity profiles, and which the authors relate to syntactic processing. Conversely, BA45 appears to be strongly connected to semantic-related areas, including the AG (Zhang & Pylkkanen, 2015). Within this view, the engagement of the ventral parts of IFG is essentially syntactic in nature, while the processing of semantic compositions selectively engages its more anterior-dorsal part. Taken together, the present data show that the IFG is sensitive to the process of basic syntactic phrases, and they confirm the role of the inferior-
32
ACCEPTED MANUSCRIPT frontal region during syntactic structure-building processes even at the lowest level of phrasal formation.
4.2 Semantic composition: Contributions of the angular gyrus
RI PT
The activation of the AG was shown by contrasting semantic phrases (e.g. ‘red boat’) and syntactic phrases (e.g. ‘this boat’). Generally the AG is considered to be part of the heteromodal parietal association cortex (Rademacher, Galaburda, Kennedy ,
SC
Filipek, & Caviness, 1992) and appears to be activated in many different cognitive conditions (Seghier, 2013).
M AN U
Within the language domain, the function of the AG has been traditionally discussed to be semantic in nature (see the meta-analyses by Binder et al. (2009) and Vigneau et al. (2006). As such, the AG seems to be active for sentences compared to word lists (Humphries, Binder, Medler, & Liebenthal, 2007; Humphries, Love, Swinney, &
TE D
Hickok, 2005; Humphries, Willard, Buchsbaum, & Hickok, 2001). Additional evidence comes from lesion studies, which show a deficit in semantic comprehension during complex sentences (Dronkers, Wilkins, Van Valin, Redfern, & Jaeger, 2004; Hart &
EP
Gordon, 1990). Similarly, Pallier et al. (2011) showed enhanced activity in the AG when the semantic information of words was present, by parametrically increasing
AC C
the constituent size of the experimental stimuli, but not when semantic information was removed. At phrasal level, a MEG study by Bemis & Pylkkanen (2012a) compared adjective-noun phrase to simple non-compositional word lists in visual and auditory domain and found an increased activity in the left ATL followed by a left AG activity. Other studies showed higher activation in the AG during increased semantic plausibly, which reported AG engagement by comparing conventional compounds with novel, unlexicalized noun-noun compounds (Forgacs et al., 2012). This goes
33
ACCEPTED MANUSCRIPT together with the results of A. R. Price et al. (2015), who reported activation in AG during semantic integration. In their study, combinatorial plausibility (measured by phrasal frequency) in meaningful and less meaningful adjective-noun combinations modulated the degree of activity within the AG, with item analysis revealing a positive
RI PT
correlation between the beta value and the combinatorial plausibility. Similarly, a transcranial direct current stimulation study managed to speed up the comprehension for items with high plausibility, when an anodal stimulation to the left AG was
SC
performed (A. R. Price, Peelle, Bonner, Grossman, & Hamilton, 2016). In our experiment, however, while we found activity in the AG for the contrast between
M AN U
semantic phrases and syntactic phrases, our post-hoc analysis based on behavioral plausibility could not find any difference in terms of plausibility between the conditions. Therefore, an explanation in terms of a higher combinatorial plausibility does not seem to hold. At the same time, comparable functional activation was found
TE D
for both semantic phrases and single words. Therefore, we are cautious in ascribing a purely semantic composition function to AG, given that combinatorial processing would arguably not be involved for single words. This finding leads to the tentative
EP
proposal that within the semantic network, the AG serves the access for conceptual information to be evaluated. Similarly, our results could show that semantic
AC C
processing in AG is independent of any phrasal plausibility or confounds in working memory or variable amount of words. More importantly, going beyond to previous findings from the single word level (Hartwigsen et al., 2016), the present data from the phrasal level can functionally distinguish the AG from BA 45 as we found them independently active for two different semantic contrasts. Specifically, the AG checks what is more relevant to semantic processing by evaluating conceptual information of words. Conversely, significant activity difference in BA45 varies due to the increased
34
ACCEPTED MANUSCRIPT amount of semantic information, which is higher for semantic phrases compared to single words. Finally, with the present data at hand, we could not report any activation within the ATL, which opens to interesting questions regarding the exact role of the area in language processing. Future research—we believe—should
RI PT
specifically look at the functional modulations of the ATL, as generated by the use of different languages (e.g. English vs. German) and different tasks (e.g. picturematching vs. syntactic judgment).
SC
4.3 Other region in the temporal lobe for syntactic computation
M AN U
Activation in the left temporal lobe was reported for both contrasts comparing syntactic phrases to either the semantic phrases or to the baseline condition. Within the literature, the (posterior) temporal region is often proposed to be an integrative area in which syntactic and thematic-based information are assembled together
TE D
(Friederici, 2011; Grodzinsky & Amunts, 2006). Different studies have shown that the area responds more strongly when lexical-semantic information is available (Bornkessel et al., 2005; Friederici, Makuuchi, & Bahlmann, 2009; Stowe et al.,
EP
1998), or when thematic assignment needs to be evaluated (Ben-Shachar, Hendler, Kahn, Ben-Bashat, & Grodzinsky, 2003; Bornkessel et al., 2005; Constable et al.,
AC C
2004; Friederici et al., 2009; Roder, Stock, Neville, Bien, & Rosler, 2002; Zaccarella et al., 2015). In a previous study indeed such activation was shown for the contrast between prepositional phrases and word lists (Zaccarella et al., 2015). Here the authors argued that the region support thematic role assignment, and that as such it works as an integration region to map lexical information to syntactic argument hierarchies. This would also support the idea that the adjective–noun phrases do not need extensive syntactic processing. Nonetheless, the integrative nature of the area goes in hand with structural data revealing dorsal connections between the pSTG
35
ACCEPTED MANUSCRIPT and BA44 along the AF/SLF when probabilistic tractography is performed (Catani et al., 2005; Friederici & Gierhan, 2013; Perani et al., 2011; Saur et al., 2008). As such, the pSTS/STG in language integrates lexical/thematic information to syntactic hierarchical information coming from the IFG (den Ouden et al., 2012). A final remark
RI PT
concerns the more pronounced anterior activity along the superior temporal cortex we reported here for the contrast SYN>BAS. Some models of language processing assign to the anterior superior temporal cortex word category-based minimal
SC
constituent structuring, reflected as phrase structure template activation, once wordcategory information has been accessed during comprehension (Bornkessel &
M AN U
Schlesewsky, 2006). A purely syntactic-based interpretation for the region however appears to conflict with recent hemodynamic findings showing that the area responds to linguistic stimuli of increasing constituent size only when lexico-semantic information is available, but not when the same stimuli are converted into
TE D
pseudowords strings for which syntactic information alone is processed (Pallier et al., 2011). Further research is awaited to clarify the exact functional role of the more anterior superior temporal cortex—with respect to both the rostrally-located anterior
EP
temporal pole (Bemis & Pylkkanen, 2011) and the most posterior section of the temporal cortex (Zaccarella et al., 2015)—as well its structural characteristics within
AC C
the superior temporal cortical complex (Morosan et al., 2001).
36
ACCEPTED MANUSCRIPT
Conclusion The aim of this study was to get an understanding of the contribution of syntax and semantics to phrasal formation at the neural level. Our participants listened to simple
RI PT
two word phrases, while using different word classes to drive the processing to one or the other linguistic process. Our manipulations allowed us to differentiate two types of processes: syntactic computation and semantic composition. We could show that semantic phrases are processed mainly recruiting the AG and BA45 and with less
SC
activation in syntax-related areas. Syntactic phrases, on the contrary, mainly recruit
M AN U
the ventral IFG and posterior STS along the superior borders of the MTG. Overall, our data show that syntactic and semantic contribution to phrasal formation can be already differentiated at a very basic level, when two words are combined together in a phrase. In the same spirit, the restricted amount of active regions for the two processes appears to reflect the fundamentally limited resources needed to perform
TE D
phrasal processing at such basic level of linguistic complexity. The two processes do not functionally overlap at the neuro-anatomical level, and confirm the role of the
AC C
EP
ventral IFG for the construction of linguistic structures via syntactic information.
37
ACCEPTED MANUSCRIPT
References Amaro, E., Jr., & Barker, G. J. (2006). Study design in fMRI: basic principles. Brain Cogn, 60(3), 220232. doi:10.1016/j.bandc.2005.11.009 Badre, D. (2008). Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes. Trends Cogn Sci, 12(5), 193-200. doi:10.1016/j.tics.2008.02.004
RI PT
Bemis, D. K., & Pylkkanen, L. (2011). Simple composition: a magnetoencephalography investigation into the comprehension of minimal linguistic phrases. J Neurosci, 31(8), 2801-2814. doi:10.1523/JNEUROSCI.5003-10.2011 Bemis, D. K., & Pylkkanen, L. (2012a). Basic Linguistic Composition Recruits the Left Anterior Temporal Lobe and Left Angular Gyrus During Both Listening and Reading. Cereb Cortex, 23(8), 1859-1873. doi:10.1093/cercor/bhs170
SC
Bemis, D. K., & Pylkkanen, L. (2012b). Combination Across Domains: An MEG Investigation into the Relationship between Mathematical, Pictorial, and Linguistic Processing. Front Psychol, 3, 583. doi:10.3389/fpsyg.2012.00583
M AN U
Bemis, D. K., & Pylkkanen, L. (2013). Flexible composition: MEG evidence for the deployment of basic combinatorial linguistic mechanisms in response to task demands. PLoS One, 8(9), e73949. doi:10.1371/journal.pone.0073949 Ben-Shachar, M., Hendler, T., Kahn, I., Ben-Bashat, D., & Grodzinsky, Y. (2003). The neural reality of syntactic transformations: evidence from functional magnetic resonance imaging. Psychol Sci, 14(5), 433-440. doi:10.1111/1467-9280.01459 Berwick, R. C., Friederici, A. D., Chomsky, N., & Bolhuis, J. J. (2013). Evolution, brain, and the nature of language. Trends Cogn Sci, 17(2), 89-98. doi:10.1016/j.tics.2012.12.002
TE D
Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009). Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb Cortex, 19(12), 2767-2796. doi:10.1093/cercor/bhp055
EP
Bonhage, C. E., Fiebach, C. J., Bahlmann, J., & Mueller, J. L. (2014). Brain signature of working memory for sentence structure: enriched encoding and facilitated maintenance. J Cogn Neurosci, 26(8), 1654-1671. doi:10.1162/jocn_a_00566
AC C
Bonner, M. F., Vesely, L., Price, C., Anderson, C., Richmond, L., Farag, C., . . . Grossman, M. (2009). Reversal of the concreteness effect in semantic dementia. Cogn Neuropsychol, 26(6), 568579. doi:10.1080/02643290903512305 Bornkessel-Schlesewsky, I., Schlesewsky, M., & von Cramon, D. Y. (2009). Word order and Broca's region: evidence for a supra-syntactic perspective. Brain Lang, 111(3), 125-139. doi:10.1016/j.bandl.2009.09.004 Bornkessel, I., & Schlesewsky, M. (2006). The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages. Psychol Rev, 113(4), 787-821. doi:10.1037/0033-295x.113.4.787 Bornkessel, I., Zysset, S., Friederici, A. D., von Cramon, D. Y., & Schlesewsky, M. (2005). Who did what to whom? The neural basis of argument hierarchies during language comprehension. Neuroimage, 26(1), 221-233. doi:10.1016/j.neuroimage.2005.01.032 Brett, M., Anton, J.-L., Valabregue, R., & Poline, J.-B. (2002). Region of interest analysis using an SPM toolbox. Paper presented at the International Conference on Functional Mapping of the Human Brain, Sendai, Japan. Available on CD-ROM in NeuroImage, Vol 16, No 2.
38
ACCEPTED MANUSCRIPT Catani, M., Jones, D. K., & Ffytche, D. H. (2005). Perisylvian language networks of the human brain. Ann Neurol, 57(1), 8-16. doi:10.1002/ana.20319 Chomsky, N. (1995). The Minimalist Program. Cambridge, Mass.: The MIT Press. Clos, M., Amunts, K., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2013). Tackling the multifunctional nature of Broca's region meta-analytically: co-activation-based parcellation of area 44. Neuroimage, 83, 174-188. doi:10.1016/j.neuroimage.2013.06.041
RI PT
Constable, R. T., Pugh, K. R., Berroya, E., Mencl, W. E., Westerveld, M., Ni, W., & Shankweiler, D. (2004). Sentence complexity and input modality effects in sentence comprehension: an fMRI study. Neuroimage, 22(1), 11-21. doi:10.1016/j.neuroimage.2004.01.001 Del Prato, P., & Pylkkanen, L. (2014). MEG evidence for conceptual combination but not numeral quantification in the left anterior temporal lobe during language production. Front Psychol, 5, 524. doi:10.3389/fpsyg.2014.00524
SC
Deleon, J., Gesierich, B., Besbris, M., Ogar, J., Henry, M. L., Miller, B. L., . . . Wilson, S. M. (2012). Elicitation of specific syntactic structures in primary progressive aphasia. Brain Lang, 123(3), 183-190. doi:10.1016/j.bandl.2012.09.004
M AN U
den Ouden, D. B., Saur, D., Mader, W., Schelter, B., Lukic, S., Wali, E., . . . Thompson, C. K. (2012). Network modulation during complex syntactic processing. Neuroimage, 59(1), 815-823. doi:10.1016/j.neuroimage.2011.07.057 DeWitt, I., & Rauschecker, J. P. (2012). Phoneme and word recognition in the auditory ventral stream. Proc Natl Acad Sci U S A, 109(8), E505-514. doi:10.1073/pnas.1113427109 Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004). Lesion analysis of the brain areas involved in language comprehension. Cognition, 92(1-2), 145-177. doi:10.1016/j.cognition.2003.11.002
TE D
Egidi, G., & Caramazza, A. (2013). Cortical systems for local and global integration in discourse comprehension. Neuroimage, 71, 59-74. doi:10.1016/j.neuroimage.2013.01.003 Fiebach, C. J., Schlesewsky, M., Lohmann, G., von Cramon, D. Y., & Friederici, A. D. (2005). Revisiting the role of Broca's area in sentence processing: syntactic integration versus syntactic working memory. Hum Brain Mapp, 24(2), 79-91. doi:10.1002/hbm.20070
EP
Forgacs, B., Bohrn, I., Baudewig, J., Hofmann, M. J., Pleh, C., & Jacobs, A. M. (2012). Neural correlates of combinatorial semantic processing of literal and figurative noun noun compound words. Neuroimage, 63(3), 1432-1442. doi:10.1016/j.neuroimage.2012.07.029
AC C
Friederici, A. D. (2011). The brain basis of language processing from structure to function. Physiol Rev, 91, 1357-1392. doi:10.1152/physrev.00006.2011.-Language Friederici, A. D. (2012). The cortical language circuit: from auditory perception to sentence comprehension. Trends Cogn Sci, 16(5), 262-268. doi:10.1016/j.tics.2012.04.001 Friederici, A. D., Bahlmann, J., Friedrich, R., & Makuuchi, M. (2011). The neural basis of recursion of complex syntactic hierarchy. Biolinguistics, 5(1-2), 87-104. Friederici, A. D., Bahlmann, J., Heim, S., Schubotz, R. I., & Anwander, A. (2006). The brain differentiates human and non-human grammars: functional localization and structural connectivity. Proc Natl Acad Sci U S A, 103(7), 2458-2463. doi:10.1073/pnas.0509389103 Friederici, A. D., & Gierhan, S. M. (2013). The language network. Curr Opin Neurobiol, 23(2), 250-254. doi:10.1016/j.conb.2012.10.002
39
ACCEPTED MANUSCRIPT Friederici, A. D., & Kotz, S. A. (2003). The brain basis of syntactic processes: functional imaging and lesion studies. Neuroimage, 20, S8-S17. doi:10.1016/j.neuroimage.2003.09.003 Friederici, A. D., Makuuchi, M., & Bahlmann, J. (2009). The role of the posterior superior temporal cortex in sentence comprehension. Neuroreport, 20(6), 563-568. doi:10.1097/WNR.0b013e3283297dee
RI PT
Friederici, A. D., Meyer, M., & Von Cramon, D. Y. (2000). Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information. Brain Lang, 74(2), 289-300. doi:10.1006/brln.2000.2438 Friederici, A. D., Ruschemeyer, S. A., Hahne, A., & Fiebach, C. J. (2003). The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. Cereb Cortex, 13(2), 170-177. doi:10.1093/cercor/13.2.170
SC
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J.-P., Frith, C. D., & Frackowiak, R. S. J. (1995). Statistical parametric maps in functional imaging: a general linear model approach. Hum Brain Mapp, 2, 189-210. doi:10.1002/hbm.460020402 Friston, K. J., Penny, W. D., & Glaser, D. E. (2005). Conjunction revisited. Neuroimage, 25(3), 661667. doi:10.1016/j.neuroimage.2005.01.013
M AN U
Galton, C. J., Patterson, K., Graham, K., Lambon-Ralph, M. A., Williams, G., Antoun, N., . . . Hodges, J. R. (2001). Differing patterns of temporal atrophy in Alzheimer's disease and semantic dementia. Neurology, 57(2), 216-225. doi:10.1212/WNL.57.2.216 Gorno-Tempini, M. L., Dronkers, N. F., Rankin, K. P., Ogar, J. M., Phengrasamy, L., Rosen, H. J., . . . Miller, B. L. (2004). Cognition and anatomy in three variants of primary progressive aphasia. Ann Neurol, 55(3), 335-346. doi:10.1002/ana.10825
TE D
Gorno-Tempini, M. L., Hillis, A. E., Weintraub, S., Kertesz, A., Mendez, M., Cappa, S. F., . . . Grossman, M. (2011). Classification of primary progressive aphasia and its variants. Neurology, 76(11), 1006-1014. doi:10.1212/WNL.0b013e31821103e6 Goucha, T., & Friederici, A. D. (2015). The language skeleton after dissecting meaning: A functional segregation within Broca's Area. Neuroimage, 114, 294-302. doi:10.1016/j.neuroimage.2015.04.011
EP
Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., von Cramon, D. Y., & Schlesewsky, M. (2005). The emergence of the unmarked: a new perspective on the language-specific function of Broca's area. Hum Brain Mapp, 26(3), 178-190. doi:10.1002/hbm.20154
AC C
Grodzinsky, Y., & Amunts, K. (2006). Broca's Region. New York: Oxford Univ Pr. Hagoort, P. (2014). Nodes and networks in the neural architecture for language: Broca's region and beyond. Curr Opin Neurobiol, 28, 136-141. doi:10.1016/j.conb.2014.07.013 Hahne, A., & Friederici, A. D. (1999). Electrophysiological evidence for two steps in syntactic analysis. Early automatic and late controlled processes. J Cogn Neurosci, 11(2), 194-205. Hart, J., Jr., & Gordon, B. (1990). Delineation of single-word semantic comprehension deficits in aphasia, with anatomical correlation. Ann Neurol, 27(3), 226-231. doi:10.1002/ana.410270303 Hartwigsen, G., Weigel, A., Schuschan, P., Siebner, H. R., Weise, D., Classen, J., & Saur, D. (2016). Dissociating Parieto-Frontal Networks for Phonological and Semantic Word Decisions: A Condition-and-Perturb TMS Study. Cereb Cortex, 26(6), 2590-2601. doi:10.1093/cercor/bhv092 Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: what is it, who has it, and how did it evolve? Science, 298(5598), 1569-1579. doi:10.1126/science.298.5598.1569
40
ACCEPTED MANUSCRIPT Hodges, J. R., Mitchell, J., Dawson, K., Spillantini, M. G., Xuereb, J. H., McMonagle, P., . . . Patterson, K. (2010). Semantic dementia: demography, familial factors and survival in a consecutive series of 100 cases. Brain, 133(Pt 1), 300-306. doi:10.1093/brain/awp248 Humphries, C., Binder, J. R., Medler, D. A., & Liebenthal, E. (2006). Syntactic and semantic modulation of neural activity during auditory sentence comprehension. J Cogn Neurosci, 18(4), 665-679. doi:10.1162/jocn.2006.18.4.665
RI PT
Humphries, C., Binder, J. R., Medler, D. A., & Liebenthal, E. (2007). Time course of semantic processes during sentence comprehension: an fMRI study. Neuroimage, 36(3), 924-932. doi:10.1016/j.neuroimage.2007.03.059 Humphries, C., Love, T., Swinney, D., & Hickok, G. (2005). Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing. Hum Brain Mapp, 26(2), 128-138. doi:10.1002/hbm.20148
SC
Humphries, C., Willard, K., Buchsbaum, B., & Hickok, G. (2001). Role of anterior temporal cortex in auditory sentence comprehension: an fMRI study. Neuroreport, 12(8), 1749-1752. doi:10.1097/00001756-200106130-00046
M AN U
Jeon, H. A., & Friederici, A. D. (2013). Two principles of organization in the prefrontal cortex are cognitive hierarchy and degree of automaticity. Nat Commun, 4, 2041. doi:10.1038/ncomms3041 Koechlin, E., Ody, C., & Kouneiher, F. (2003). The architecture of cognitive control in the human prefrontal cortex. Science, 302(5648), 1181-1185. doi:10.1126/science.1088545 Lambon Ralph, M. A., Sage, K., Jones, R. W., & Mayberry, E. J. (2010). Coherent concepts are computed in the anterior temporal lobes. Proc Natl Acad Sci U S A, 107(6), 2717-2722. doi:10.1073/pnas.0907307107
TE D
Makuuchi, M., Bahlmann, J., Anwander, A., & Friederici, A. D. (2009). Segregating the core computational faculty of human language from working memory. Proceedings of the National Academy of Sciences, 106(20), 8362-8367. doi:10.1073/pnas.0810928106
EP
Makuuchi, M., & Friederici, A. D. (2013). Hierarchical functional connectivity between the core language system and the working memory system. Cortex, 49(9), 2416-2423. doi:10.1016/j.cortex.2013.01.007
AC C
Makuuchi, M., Grodzinsky, Y., Amunts, K., Santi, A., & Friederici, A. D. (2013). Processing noncanonical sentences in broca's region: reflections of movement distance and type. Cereb Cortex, 23(3), 694-702. doi:10.1093/cercor/bhs058 Mazoyer, B. M., Tzourio, N., Frak, V., Syrota, A., Murayama, N., Levrier, O., . . . Mehler, J. (1993). The cortical representation of speech. J Cogn Neurosci, 5(4), 467-479. doi:10.1162/jocn.1993.5.4.467 Meyer, L., Obleser, J., Anwander, A., & Friederici, A. D. (2012). Linking ordering in Broca's area to storage in left temporo-parietal regions: the case of sentence processing. Neuroimage, 62(3), 1987-1998. doi:10.1016/j.neuroimage.2012.05.052 Molinaro, N., Carreiras, M., & Duñabeitia, J. A. (2012). Semantic combinatorial processing of nonanomalous expressions. Neuroimage, 59(4), 3488-3501. doi:10.1016/j.neuroimage.2011.11.009 Molinaro, N., Paz-Alonso, P. M., Dunabeitia, J. A., & Carreiras, M. (2015). Combinatorial semantics strengthens angular-anterior temporal coupling. Cortex, 65C, 113-127. doi:10.1016/j.cortex.2015.01.004
41
ACCEPTED MANUSCRIPT Morosan, P., Rademacher, J., Schleicher, A., Amunts, K., Schormann, T., & Zilles, K. (2001). Human primary auditory cortex: cytoarchitectonic subdivisions and mapping into a spatial reference system. Neuroimage, 13(4), 684-701. doi:10.1006/nimg.2000.0715 Newman, S. D., Ikuta, T., & Burns, T., Jr. (2010). The effect of semantic relatedness on syntactic analysis: An fMRI study. Brain Lang, 113(2), 51-58. doi:10.1016/j.bandl.2010.02.001 Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9(1), 97-113.
RI PT
Opitz, A., Regel, S., Müller, G., & Friederici, A. D. (2013). Neurophysiological evidence for morphological underspecification in German strong adjective inflection. Language, 89(2), 231264. doi:10.1353/lan.2013.0033
SC
Pallier, C., Devauchelle, A. D., & Dehaene, S. (2011). Cortical representation of the constituent structure of sentences. Proc Natl Acad Sci U S A, 108(6), 2522-2527. doi:10.1073/pnas.1018711108 Patterson, K., Nestor, P. J., & Rogers, T. T. (2007). Where do you know what you know? The representation of semantic knowledge in the human brain. Nat Rev Neurosci, 8(12), 976-987. doi:10.1038/nrn2277
M AN U
Perani, D., Saccuman, M. C., Scifo, P., Anwander, A., Spada, D., Baldoli, C., & al., e. (2011). Neural language networks at birth. Proc Natl Acad Sci U S A, 108(38), 16056-16061. doi:10.1073/pnas.1102991108. Potter, M. C., & Faulconer, B. A. (1979). Understanding noun phrases. Journal of Verbal Learning and Verbal Behavoir, 18, 509-521.
TE D
Price, A. R., Bonner, M. F., Peelle, J. E., & Grossman, M. (2015). Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular gyrus. J Neurosci, 35(7), 32763284. doi:10.1523/JNEUROSCI.3446-14.2015 Price, A. R., Peelle, J. E., Bonner, M. F., Grossman, M., & Hamilton, R. H. (2016). Causal Evidence for a Mechanism of Semantic Integration in the Angular Gyrus as Revealed by High-Definition Transcranial Direct Current Stimulation. J Neurosci, 36(13), 3829-3838. doi:10.1523/JNEUROSCI.3120-15.2016
EP
Price, C. J., Moore, C. J., Humphreys, G. W., & Wise, R. J. (1997). Segregating Semantic from Phonological Processes during Reading. J Cogn Neurosci, 9(6), 727-733. doi:10.1162/jocn.1997.9.6.727
AC C
Pulvermuller, F., Shtyrov, Y., Hasting, A. S., & Carlyon, R. P. (2008). Syntax as a reflex: neurophysiological evidence for early automaticity of grammatical processing. Brain Lang, 104(3), 244-253. doi:10.1016/j.bandl.2007.05.002 Rademacher, J., Galaburda, A. M., Kennedy , D. N., Filipek, P. A., & Caviness, V. S. (1992). Human Cerebral Cortex: Localization, Parcellation, and Morphometry with Magnetic Resonance Imaging. J Cogn Neurosci, 4(4), 352-374. doi:10.1162/jocn.1992 Roder, B., Stock, O., Neville, H., Bien, S., & Rosler, F. (2002). Brain activation modulated by the comprehension of normal and pseudo-word sentences of different processing demands: a functional magnetic resonance imaging study. Neuroimage, 15(4), 1003-1014. doi:10.1006/nimg.2001.1026 Rogalski, E., Cobia, D., Harrison, T. M., Wieneke, C., Thompson, C. K., Weintraub, S., & Mesulam, M. M. (2011). Anatomy of language impairments in primary progressive aphasia. J Neurosci, 31(9), 3344-3350. doi:10.1523/jneurosci.5544-10.2011
42
ACCEPTED MANUSCRIPT Rohrer, J. D., Warren, J. D., Modat, M., Ridgway, G. R., Douiri, A., Rossor, M. N., . . . Fox, N. C. (2009). Patterns of cortical thinning in the language variants of frontotemporal lobar degeneration. Neurology, 72(18), 1562-1569. doi:10.1212/WNL.0b013e3181a4124e Rorden, C., & Brett, M. (2000). Stereotaxic display of brain lesions. Behav Neurol, 12(4), 191-200. doi:10.1155/2000/421719 Sanides, F. (1962). Die Architektonik des Menschlichen Stirnhirns. Berlin Heidelberg: Springer.
RI PT
Santi, A., & Grodzinsky, Y. (2007). Working memory and syntax interact in Broca's area. Neuroimage, 37(1), 8-17. doi:10.1016/j.neuroimage.2007.04.047 Saur, D., Kreher, B. W., Schnell, S., Kummerer, D., Kellmeyer, P., Vry, M. S., . . . Weiller, C. (2008). Ventral and dorsal pathways for language. Proc Natl Acad Sci U S A, 105(46), 18035-18040. doi:10.1073/pnas.0805234105
SC
Segaert, K., Menenti, L., Weber, K., Petersson, K. M., & Hagoort, P. (2012). Shared syntax in language production and language comprehension--an FMRI study. Cereb Cortex, 22(7), 1662-1670. doi:10.1093/cercor/bhr249
M AN U
Seghier, M. L. (2013). The angular gyrus: multiple functions and multiple subdivisions. Neuroscientist, 19(1), 43-61. doi:10.1177/1073858412440596 Smith, E. E. (1984). Conceptual combination with prototype concepts. Cognitive Sci, 8, 337-361. doi:10.1207/s15516709cog0804_2 Snowden, J. S. (1995). Semantic dementia: a form of circumscribed cerebral atrophy. Neurocase, 1(1). doi:10.1093/neucas/1.1.39-y
TE D
Stowe, L. A., Broere, C. A., Paans, A. M., Wijers, A. A., Mulder, G., Vaalburg, W., & Zwarts, F. (1998). Localizing components of a complex task: sentence processing and working memory. Neuroreport, 9(13), 2995-2999. doi:10.1097/00001756-199809140-00014 Vandenberghe, R., Nobre, A. C., & Price, C. J. (2002). The response of left temporal cortex to sentences. J Cogn Neurosci, 14(4), 550-560. doi:10.1162/08989290260045800
EP
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., . . . Tzourio-Mazoyer, N. (2006). Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing. Neuroimage, 30(4), 1414-1432. doi:10.1016/j.neuroimage.2005.11.002
AC C
Visser, M., Jefferies, E., & Lambon Ralph, M. A. (2010). Semantic processing in the anterior temporal lobes: a meta-analysis of the functional neuroimaging literature. J Cogn Neurosci, 22(6), 10831094. doi:10.1162/jocn.2009.21309 Westerlund, M., Kastner, I., Al Kaabi, M., & Pylkkanen, L. (2015). The LATL as locus of composition: MEG evidence from English and Arabic. Brain Lang, 141, 124-134. doi:10.1016/j.bandl.2014.12.003 Westerlund, M., & Pylkkanen, L. (2014). The role of the left anterior temporal lobe in semantic composition vs. semantic memory. Neuropsychologia, 57C, 59-70. doi:10.1016/j.neuropsychologia.2014.03.001 Wilson, S. M., Dronkers, N. F., Ogar, J. M., Jang, J., Growdon, M. E., Agosta, F., . . . Gorno-Tempini, M. L. (2010). Neural correlates of syntactic processing in the nonfluent variant of primary progressive aphasia. J Neurosci, 30(50), 16845-16854. doi:10.1523/JNEUROSCI.254710.2010 Wilson, S. M., Galantucci, S., Tartaglia, M. C., & Gorno-Tempini, M. L. (2012). The neural basis of syntactic deficits in primary progressive aphasia. Brain Lang, 122(3), 190-198. doi:10.1016/j.bandl.2012.04.005
43
ACCEPTED MANUSCRIPT Wilson, S. M., Galantucci, S., Tartaglia, M. C., Rising, K., Patterson, D. K., Henry, M. L., . . . GornoTempini, M. L. (2011). Syntactic processing depends on dorsal language tracts. Neuron, 72(2), 397-403. doi:10.1016/j.neuron.2011.09.014 Xu, J., Kemeny, S., Park, G., Frattali, C., & Braun, A. (2005). Language in context: emergent features of word, sentence, and narrative comprehension. Neuroimage, 25(3), 1002-1015. doi:10.1016/j.neuroimage.2004.12.013
RI PT
Zaccarella, E., & Friederici, A. D. (2015a). Merge in the Human Brain: A Sub-Region Based Functional Investigation in the Left Pars Opercularis. Front Psychol, 6, 1818. doi:10.3389/fpsyg.2015.01818 Zaccarella, E., & Friederici, A. D. (2015b). Reflections of word processing in the insular cortex: A subregional parcellation based functional assessment. Brain Lang, 142, 1-7. doi:10.1016/j.bandl.2014.12.006
SC
Zaccarella, E., & Friederici, A. D. (2016). The neurobiological nature of syntactic hierarchies. Neurosci Biobehav Rev. doi:10.1016/j.neubiorev.2016.07.038
M AN U
Zaccarella, E., Meyer, L., Makuuchi, M., & Friederici, A. D. (2015). Building by Syntax: The Neural Basis of Minimal Linguistic Structures. Cereb Cortex. doi:10.1093/cercor/bhv234 Zhang, L., & Pylkkanen, L. (2015). The interplay of composition and concept specificity in the left anterior temporal lobe: An MEG study. Neuroimage, 111C, 228-240. doi:10.1016/j.neuroimage.2015.02.028 Zhu, Z., Feng, G., Zhang, J. X., Li, G., Li, H., & Wang, S. (2013). The role of the left prefrontal cortex in sentence-level semantic integration. Neuroimage, 76, 325-331. doi:10.1016/j.neuroimage.2013.02.060
AC C
EP
TE D
Zhu, Z., Zhang, J. X., Wang, S., Xiao, Z., Huang, J., & Chen, H. C. (2009). Involvement of left inferior frontal gyrus in sentence-level semantic integration. Neuroimage, 47(2), 756-763. doi:10.1016/j.neuroimage.2009.04.086
44
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
45
S0010-9452(17)30297-6
DOI:
10.1016/j.cortex.2017.09.002
Reference:
CORTEX 2123
To appear in:
Cortex
Received Date: 5 February 2017 Revised Date:
17 July 2017
Accepted Date: 5 September 2017
Please cite this article as: Schell M, Zaccarella E, Friederici AD, Differential cortical contribution of syntax and semantics: An fMRI study on two-word phrasal processing, CORTEX (2017), doi: 10.1016/ j.cortex.2017.09.002. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
DIFFERENTIAL CORTICAL CONTRIBUTION OF SYNTAX AND SEMANTICS:
M AN U
SC
RI PT
AN FMRI STUDY ON TWO-WORD PHRASAL PROCESSING
TE D
Marianne Schell, Emiliano Zaccarella, Angela D. Friederici
Max Planck Institute for Human Cognitive and Brain Sciences,
AC C
EP
Department of Neuropsychology, Stephanstraße 1a | 04103 Leipzig, Germany
Correspondence should be addressed to: Marianne Schell Stephanstraße 1a, 04103 Leipzig T +49 341 9940 2238 [email protected]
1
ACCEPTED MANUSCRIPT
Abstract Linguistic expressions consist of sequences of words combined together to form phrases and sentences. The neurocognitive process handling word combination is
RI PT
drawing increasing attention among the neuroscientific community, given that the underlying syntactic and semantic mechanisms of such basic combinations— although essential to the generation of more complex structures—still need to be
SC
consistently determined. The current experiment was conducted to disentangle the neural networks supporting syntactic and semantic processing at the level of two-
M AN U
word combinations. We manipulated the combinatorial load by using words of different grammatical classes within the phrase, such that determiner-noun combinations (this ship) were used to boost neural activity in syntax-related areas, while adjective-noun combinations (blue ship) were conversely used to measure
TE D
neural response in semantic-related combinations. By means of functional magnetic resonance imaging (fMRI), we found that syntax-related processing mainly activates the most ventral part of the inferior frontal gyrus, along the frontal operculum (FOP)
EP
and anterior insula (aINS). Fine-grained analysis in BA44 confirmed that the most inferior-ventral portion is highly sensitive to syntactic computations driven by function
AC C
words. Semantic-related processing on the contrary, rather engages the anterior dorsal part of the left inferior frontal gyrus (IFG) and the left angular gyrus (AG) that is two regions which appear to perform different functions within the semantic network. Our findings suggest that syntactic and semantic contribution to phrasal formation can be already differentiated at a very basic level, with each of these two processes comprising non-overlapping areas on the cerebral cortex. Specifically, they confirm the role of the ventral IFG for the construction of syntactically legal linguistic
ACCEPTED MANUSCRIPT constructions, and the prominence of the more anterior IFG and the AG for conceptual semantics.
Keywords
RI PT
Inferior Frontal Gyrus; Angular Gyrus; Syntax; Semantics; Two-word combinatorics
Highlights
Two-word phrasal combinatorics constitute the bottom level of linguistic complexity
•
Syntactic and semantic contribution to phrasal formation is cortically differentiable
•
Syntactic processing mainly activates the ventral Inferior Frontal Gyrus (IFG)
•
Semantic processing engages the anterior-dorsal IFG and the Angular Gyrus (AG)
•
The anterior-dorsal IFG and the AG play distinct roles within the semantic network
AC C
EP
TE D
M AN U
SC
•
2
ACCEPTED MANUSCRIPT
1 Introduction Human language is characteristically defined as a highly productive cognitive ability: words can be continuously combined together to create an infinite number of new
RI PT
expressions of increasing complexity. To comprehend such novel expressions in an effective way, our brain needs to do two things: it has to assess what the meaning of the individual words mean when put together, and it has to retrieve the abstract underlying structure binding such expressions. These two processes refer to the
SC
linguistic aspects of semantics and syntax. Specifically, semantic composition drives
M AN U
the combination of the meanings of the individual words to form meaningful expressions. These combinational computations are the most fundamental mechanisms at the root of every language (Hauser, Chomsky, & Fitch, 2002). Syntactic computations, conversely, implement the rules that govern the abstract
TE D
architecture of the same expressions (Chomsky, 1995). Traditionally, neurolinguistic investigations have tried to manipulate semantic composition and syntactic computations within the context of full sentences (for a review see Friederici (2011).
EP
Semantic composition has been mostly examined either by varying the semantic load in sentences with or without pseudowords to reduce semantics information
AC C
(Humphries, Binder, Medler, & Liebenthal, 2006; Mazoyer et al., 1993; Pallier, Devauchelle, & Dehaene, 2011), by varying semantic complexity to contrast short sentences against longer sentences or narratives (Pallier et al., 2011; Stowe et al., 1998; Xu, Kemeny, Park, Frattali, & Braun, 2005), or by evaluating semantic plausibility in a specific context (Newman, Ikuta, & Burns, 2010; Zhu et al., 2013; Zhu et al., 2009). Syntactic computations, conversely, have been mainly investigated by comparing sentences to word lists lacking syntactic information (Friederici, Meyer, & Von Cramon, 2000; Humphries et al., 2006; Mazoyer et al., 1993; Pallier et al., 2011; 3
ACCEPTED MANUSCRIPT Stowe et al., 1998; Xu et al., 2005), by evaluating different degrees of syntactic complexity (Bornkessel, Zysset, Friederici, von Cramon, & Schlesewsky, 2005; Makuuchi, Bahlmann, Anwander, & Friederici, 2009), by focusing on syntactic errors (Friederici, Ruschemeyer, Hahne, & Fiebach, 2003; Vandenberghe, Nobre, & Price,
RI PT
2002), or by using syntactic priming (Segaert, Menenti, Weber, Petersson, & Hagoort, 2012). The sentential level however might not be the best approach to explore the neural basis of basic compositional processes in the semantic and
SC
syntactic domain since multiple combinations of words and complex structures tend to involve additional mechanisms external to syntactic computations and semantic
M AN U
composition. These additional mechanisms are working memory and storage (Makuuchi et al., 2009; Makuuchi & Friederici, 2013; Meyer, Obleser, Anwander, & Friederici, 2012; Santi & Grodzinsky, 2007), cognitive control and ambiguity resolution (Badre, 2008; Koechlin, Ody, & Kouneiher, 2003), and at the text level
TE D
even integration into the context of discourse (Egidi & Caramazza, 2013). More recently, researchers have started to look at compositional processing in the
EP
semantic and syntactic domain at more basic levels, investigating how the brain behaves during the combination of very simple two- or three-word phrasal structures,
AC C
such as “blue boat”, “this boat” or “on the boat”. Within the neurolinguistic tradition, the expression basic levels of linguistic processing therefore refers to the build-up of simple structures beyond single words, where the combination of independent lexical elements (e.g. an adjective and a noun, or a determiner and a noun) are combined together to form elementary phrases or sentences at the root of linguistic complexity. Evidence from different languages is now available, including English (Bemis & Pylkkanen, 2011, 2012a, 2012b, 2013; Del Prato & Pylkkanen, 2014; Westerlund, Kastner, Al Kaabi, & Pylkkanen, 2015; Westerlund & Pylkkanen, 2014; Zhang &
4
ACCEPTED MANUSCRIPT Pylkkanen, 2015), Spanish (Molinaro, Carreiras, & Duñabeitia, 2012; Molinaro, PazAlonso, Dunabeitia, & Carreiras, 2015), and German (Zaccarella & Friederici, 2015a; Zaccarella, Meyer, Makuuchi, & Friederici, 2015).
RI PT
Focusing on the semantic domain, the first of these studies (Bemis & Pylkkanen, 2011) used adjective-noun combinations and reported increased cortical activity for phrasal composition in the anterior temporal lobe, ATL (and in the ventral prefrontal cortex). In this study, participants were asked to combine together in a visual task a
SC
descriptive adjective “red” with a noun “boat” to be matched with a picture. The ATL
M AN U
was found equally active when the same types of stimuli were processed in the auditory modality (Bemis & Pylkkanen, 2012a). Given the type of stimulus used, the authors concluded that the ATL might be the region particularly engaged during semantic composition, deriving the meaning of a simple phrase like 'red boat' from the conjunction of the two simpler concepts of “redness” and “floating object”,
TE D
expressed by the adjective 'red' and the noun 'boat' respectively (Smith, 1984). The authors base their interpretation on findings from single word processing, for which
EP
activity in the ATL was reported when words had to be classified conceptually as referring to a living or non-living object (C. J. Price, Moore, Humphreys, & Wise,
AC C
1997). At the clinical level, evidence supporting a semantic role for the ATL, comes from patient studies which traditionally link the region to semantic processing deficits (e.g. semantic dementia) following temporal lobe atrophy (Bonner et al., 2009; Galton et al., 2001; Gorno-Tempini et al., 2004; Rohrer et al., 2009). Patients with semantic dementia generally show deficit in conceptual knowledge for various domains (Gorno-Tempini et al., 2011; Hodges et al., 2010; Snowden, 1995). As such, the ATL might integrate information associated with the respective concepts during the processing of words (Lambon Ralph, Sage, Jones, & Mayberry, 2010; Patterson,
5
ACCEPTED MANUSCRIPT Nestor, & Rogers, 2007; Visser, Jefferies, & Lambon Ralph, 2010). According to this view specific conceptual information (e.g. the boat’s shape, color, sound, worldknowledge, etc.) is stored in corresponding cortical areas, while the ATL serves as an
RI PT
amodal hub to fuse the different aspects of conceptual information. A second region proposed to be relevant for semantic composition is the angular gyrus, AG (Binder, Desai, Graves, & Conant, 2009). The AG was found to be involved in addition to the ATL during the construction of two-word semantic
SC
composition, regardless of modality, although the AG activated at a later time point
M AN U
than the anterior temporal region (Bemis & Pylkkanen, 2012a). Furthermore, it was shown that activation in AG varied as a function of plausibility by comparing meaningful (e.g. plaid jacket) with non-meaningful phrases (e.g. moss pony) in healthy subjects (A. R. Price, Bonner, Peelle, and Grossman (2015). The activity in AG was positively correlated with the combinatorial strength, according to a normed
TE D
measure of plausibility. This result assigns a modulating aspect to the AG during the semantic composition of words, while a more complex concept is built. Another study
EP
showed a significant functional coupling between the ATL and the AG for the processing of noun-adjective pairs, but only for the adjectives with low predictability
AC C
(implausible phrases) (Molinaro et al. (2015). At the single word level, it has recently been demonstrated that semantic decisions over word meanings appear to rely on the functional co-working between the AG and a third fundamental region of the semantic system, that is BA45 (Hartwigsen et al., 2016). Specifically, Hartwigsen and colleagues have shown that functional disruption by means of TMS over either left anterior IFG (BA45) or left AG alone does not significantly affect semantic decisions (natural vs. man-made item), suggesting that the respective other non-lesioned region can compensate for the TMS-induced effect. However, combined TMS over
6
ACCEPTED MANUSCRIPT both areas leads to significantly longer reaction times. BA45 and AG can compensate each other in evaluating semantic information for particular words, but their corresponding specific roles within the semantic network still need to be disentangled. At complex combinatorial levels, BA45 is known to be modulated by
RI PT
the processing of the verbal information in sentential constructions, which in turn guarantees the comprehension of the overall propositional meaning of a sentence (Newman et al., 2010; Zhu et al., 2009). The AG has been ascribed to reflect some
M AN U
information are integrated (Humphries et al., 2006).
SC
general form of semantic processing, in which incoming pieces of semantic
Overall, neurolinguistic evidence for semantic processes so far suggests that simple semantic composition might rely on the recruitment of regions in the ATL and the AG. In parallel, the anterior IFG (BA45) is certainly involved during semantic composition at more complex levels, when verbal information is assessed. However, it appears to
performed.
TE D
be recruited in compliance with the AG also when more basic semantic tasks are
EP
Studies focusing on the syntactic domain ask how the underlying syntactic structure of such simple two-word combinations is neurally implemented. Do perceivers
AC C
automatically build phrasal hierarchies or can semantic composition of two content words be processed without any syntactic involvement? In a first investigation of twoword phrases without any semantic information, Zaccarella et al. (2015) used twoword phrases formed by a determiner and a pseudonoun (nouns without any semantic meaning, e.g. this flirk) to specifically focus on syntactic processing. As such, the study looked at the syntactic computational effect between the two elements, once the descriptive semantic information was removed from the stimulus. The authors reported increased neural activity in the left IFG and specifically in BA 44 7
ACCEPTED MANUSCRIPT for the syntactic effect, but no significant engagement of the ATL regions. Additional analysis in BA 44 further revealed that only the ventral-anterior part was selectively responsible for syntactic processing, which they argued reflect the merge computation described in both theoretical linguistics (Chomsky, 1995) and
RI PT
evolutionary linguistics (Berwick, Friederici, Chomsky, & Bolhuis, 2013; Zaccarella & Friederici, 2016). Crucially, the sub-regional analysis in this study was grounded on a recent co-activation-based parcellation in BA44, which subdivided the region meta-
SC
analytically into five different functional clusters (Clos, Amunts, Laird, Fox, & Eickhoff, 2013). Within the area, the two posterior clusters were mostly found associated with
M AN U
phonology/overt speech (Cluster 1, dorsally) and rhythmic sequencing (Cluster 4, ventrally). The two more anterior ones were conversely linked to working memory (Cluster 2, dorsally) and linguistic tasks including syntax (Cluster 3, ventrally), while the fifth one in the inferior frontal junction was associated to task switching and
TE D
cognitive control (Cluster 5). Taken together, these results suggest a distinct role for syntactic computations in region BA44.
EP
Supporting evidence for a neural involvement of BA 44 and the frontal operculum in syntactic computations comes from many studies on sentence processing which
AC C
selectively varied the degree of syntactic complexity by comparing more complex object-first sentences versus subject-first sentences (Bornkessel et al., 2005; Fiebach, Schlesewsky, Lohmann, von Cramon, & Friederici, 2005; Grewe et al., 2005) or by employing sentences with varying degrees of embeddedness (Friederici, Bahlmann, Friedrich, & Makuuchi, 2011; Jeon & Friederici, 2013; Makuuchi et al., 2009). Moreover, lesion studies could show that atrophy in Broca’s area is related to a deficit in processing of syntactic hierarchy (Deleon et al., 2012; Rogalski et al., 2011; Wilson et al., 2010; Wilson, Galantucci, Tartaglia, & Gorno-Tempini, 2012). In
8
ACCEPTED MANUSCRIPT general, these studies converge in assigning a major role of BA 44 for syntactic processing. Against this background, it appears that contribution of syntax and semantics
to
phrasal
combinatorics
corresponds
to
functionally
distinct
RI PT
neuroanatomical regions within the left hemispheric cortex. The hypothesis that follows from this is that the processing of phrases formed by nouns preceded by open class elements—like descriptive adjectives (“red boat”)— might particularly upregulate activity in those cortical regions specifically sensitive to
SC
semantic combination. Adjectives, as content words (like nouns and verbs), primarily
M AN U
carry semantic information, and the presence of such a word in an adjective-noun pair increases the semantic processing load in the language system (Zaccarella et al., 2015) which might lead to narrowed conceptual specificity (Westerlund & Pylkkanen, 2014). However, given the fact that for German phrases the adjectiveform must be adapted according to gender, case and number of the noun (so-called
TE D
inflectional morphology), some syntactic elements are retained in semantic phrases as well (-es, in blaues Schiff/ blue ship). Conversely, the processing of phrases
EP
formed by nouns preceded by closed class items—like determiners (“this boat”)— with reduced descriptive content, would rather engage activity in syntax-related
AC C
regions, and more closely resemble pure syntactic computational processing (Zaccarella & Friederici, 2015a). So far, however, no study has directly tested whether syntactic computations and semantic compositions can be differentiated at such basic combinatorial level. The goal of the present fMRI study was to assess the functional contribution of semantics and syntax to the formation of basic two-word phrases directly in the same group of participants. In our study, we used acoustic stimuli and varied the type of word class driving the combination across two different conditions of two-word German stimuli. To identify 9
ACCEPTED MANUSCRIPT distinctive cortical regions across the brain that would implement either syntactic computation or semantic composition at the very basic two-word level, we created semantic (driven) stimuli (SEM) comprising adjective-noun phrases, and syntactic (driven) stimuli (SYN) comprising determiner-noun phrases. In addition to these
RI PT
experimental conditions we further included a third one-word non-combinatorial condition where the first word driving the phrasal effect was absent, while the second word, the noun, was kept in the identical position as in the other two conditions.
SC
Moving from previous works on the functional differentiation between phrasal effect and word accumulation effect (Zaccarella and Friederici 2015; Zaccarella et al. 2017),
M AN U
as well as recent studies on the effect of semantic processing at single-word and twoword level (Hartwigsen et al., 2016; Molinaro et al., 2015; A. R. Price et al., 2015), this additional condition served as baseline (BAS) to isolate across the brain those areas sensitive to word increase (2 words > 1word), when more syntactic information
TE D
or semantic information was added to the process. From the relative phrase-sensitive functional patterns obtained above, our first aim was that of comparing the brain activity produced by the two phrasal constructions in those regions, to search for a
EP
double dissociation between phrasal conditions and anatomical regions on the cerebral cortex. As such, we did not include any further baseline consisting of lists of
AC C
two nouns for example, since word accumulation processing for two-word lists vs. one-word lists was not within the scope of the current experiment, although already explicitly tested in other studies (Bemis & Pylkkanen, 2011; Zaccarella & Friederici, 2015b). Further, our second aim was that of directly comparing across the brain the two phrasal conditions against each other, when the baseline effect was not taken into account. These contrasts were meant to verify whether within the two networks the same anatomical regions found to be sensitive to the increased amount of
10
ACCEPTED MANUSCRIPT syntax-relevant and semantics-relevant words, remain essentially equally modulated by syntactic features (the determiner) and semantic features (the adjective) regardless of the amount of words forming the phrase. For instance, in the most anterior-ventral BA44, neural activity for two-word syntactic phrases was found to be
RI PT
significantly higher than both one-word phrases and two-word lists (Zaccarella & Friederici, 2015a), therefore speaking in favor of a pure syntactic role of this area. Conversely, it is completely unknown whether regional specificities within the
SC
semantic network can be differentiated into different functions, such that the activity in some areas vary due to the increased amount of semantic information, while some
M AN U
others only evaluate conceptual information of words regardless of the number of words. Finally, our third aim was that of testing whether at fine-grained level, the most ventral-anterior parcel within BA44 (Clos, Amunts, Laird, Fox, & Eickhoff, 2013) would have shown to be sensitive to both types of phrases, or rather active for syntax
TE D
only, as previously found when contrasting determiner-pseudoword phrases against a non-compositional word-list condition (Zaccarella & Friederici, 2015a).
EP
Based on previous literature, we expected to find neural activation for simple phrase processing in language-relevant areas of the temporo-fronto-parietal network
AC C
lateralized to the left hemisphere, including Broca’s area in the IFG, Wernicke’s area in the pSTG/STS, as well as the ATL and the AG (Catani, Jones, & Ffytche, 2005; Friederici, 2011). More specifically, we expected to find activation in syntax-related areas, namely the left IFG (BA 44), frontal operculum/anterior insula and the superior and middle temporal lobe for syntactic phrases (SYN) in which the noun combines with the determiner, independent of the contrast of interest. At a more fine-grained level we also tested the hypothesis that within BA 44 only the most anterior ventral portion of the area (Cluster 3) would be selectively active for syntactic computations
11
ACCEPTED MANUSCRIPT (Clos et al., 2013). Here we were interested to see whether this activation can also be found in a real word phrase condition, and if so, whether this would hold for syntactic phrases only (dieses Schiff/this ship), or also for semantic phrases (blaues Schiff/blue ship), since in the latter case adjectives obligatory carry inflectional
RI PT
morphemes (-es). We had two conflicting hypotheses regarding the processing of semantic phrases (SEM). On the one hand, semantic phrases (blaues Schiff/blue ship) could be processed like syntactic phrases with additional demands of semantic
SC
composition caused by the descriptive semantic information carried by the adjectives. If so, the activation pattern for semantic phrases would yield to an increase in the
M AN U
posterior IFG (BA44) due to syntactic morphology in addition to recruitment of the ATL, BA45 and the AG, as the semantic related regions (Bemis & Pylkkanen, 2011). On the other hand, semantic phrases could be processed primarily at a compositional level mainly in semantic-related areas, with syntactic information carried by the
TE D
inflectional morphology triggering automatic syntactic computations with low processing demands in BA44 (Pulvermuller, Shtyrov, Hasting, & Carlyon, 2008). In either case, we left open the issue of whether the different regions of the semantic
EP
network could be ascribed to distinct roles within the semantic system, such that the activity in some of them would vary due to the increased amount of semantic
AC C
information, while in some others it would vary as a reflection of what is more relevant to semantics for the conceptual evaluation of words, regardless of the amount of words.
12
ACCEPTED MANUSCRIPT
2 Methods 2.1 Participants Twenty-one native Germans (10 females and 11 males; mean age of 27.7 yrs.,
RI PT
ranging from 21 to 36 yrs. of age) participated in the study. All participants were right handed as assessed with the Edinburgh questionnaire (Oldfield, 1971) and reported normal hearing. None of them had any history of neurological or psychiatric disorder.
SC
Written informed consent was obtained from all participants according to the procedures approved by the Research Ethics Committee of the University of Leipzig.
M AN U
All participants received monetary compensation after completing the experiment.
2.2 Stimuli
We used two different types of two-word phrases, namely syntactically driven phrases (SYN) and semantically driven phrases (SEM), in addition to a non-
TE D
combinatorial baseline condition (BAS). The syntactically driven phrases were formed by combining nouns with determiners—function words, which carry syntactic
EP
information and which specify syntactic relations within the phrase. The semantically driven phrases were constructed by combining the same nouns with descriptive
AC C
adjectives—content words conveying additional semantic content to the noun. The non-combinatorial baseline condition consisted of single nouns only (see also Figure 1A). Throughout all conditions, 40 one-syllable nouns (20 masculine and 20 neutral) were employed, selected by using the Wortschatz Lexicon of the University of Leipzig (available at http://wortschatz.uni-leipzig.de/). Only concrete nouns with a high frequency were chosen. The mean frequency f was 10.92 (SD=2.09), where f means that the most frequent word “der” (the) is 2f times more frequent than the number of the target word. We used four color adjectives for the SEM condition and
13
ACCEPTED MANUSCRIPT four determiners for the SYN condition, together with their inflectional endings (-er for masculine nouns, -es for neutral nouns), according to the gender of the following noun. All stimuli were single-channel recorded by a trained male native speaker of German language. The speaker spoke all words individually in the form of a
RI PT
statement. Recordings were made in a sound-attenuating chamber with a resolution of 16 bits and a sampling rate of 44.1 kHz. Offline, the stimuli were edited into separate files and normalized for root mean square.
SC
Nouns had a mean duration of 0.503 s (SD=0.049). Mean duration for adjectives and
M AN U
determiners was 0.628 s (SD=0.064, Mdn=0.629, n1=8) and 0.625 s (SD=0.062, Mdn=0.620, n2=8), respectively. The distributions of adjectives and determiners showed no significant difference at p ≤ 0.05 with a U-value of 31 and a critical value of 13. To keep the final length of the stimuli constant between conditions, all recordings were embedded in a speech-shaped noise (SSN) with signal noise ratio of
TE D
6dB. Here, we used all recorded stimuli, calculated the spectral envelope of the signal and used this to shape a white noise signal. In the two-word conditions, the
EP
stimuli were concatenated with a 40 ms pause between the words and added to the SSN with the second word starting at 900 ms. The baseline condition was also added
AC C
to the SSN with the noun starting at 900 ms. This yielded to a mean length of 1.443 s (SD=0.049) for all conditions. In a final step, we also calculated the co-occurrence frequencies or ‘combinatorial strength’ of our two-word phrases as described in (A. R. Price et al., 2015). According to the authors this measure strongly correlates with the behavioral plausibility ratings of the word pair. To ensure that our conditions did not differ in terms of co-occurrence frequencies between the two words, we performed an unpaired t-test between the frequencies of adjective-noun phrases and the
14
ACCEPTED MANUSCRIPT frequencies of determiner-noun phrases. The t-test however did not reveal any frequency differences between the two conditions (t(318)=0.75, p=0.94). All stimuli were assigned to four randomized sets, so that one noun occurred only
RI PT
once for each condition, while adjectives and determiner where counterbalanced within a set. The final pool consisted of 120 stimuli, 40 for each condition and 40 filler conditions (see next section). Every participant was tested on an individually
SC
randomized version of the set.
2.3 Procedure
M AN U
Participants performed a short practice session on a desktop computer outside the scanner room on a different set of stimuli. All stimuli were presented using the software package of Presentation® (Neurobehavioural Systems, Inc., Albany, CA, USA) with air-conducted headphones (Resonance Technology, Inc., Northridge, CA,
TE D
USA). Once in the scanner, volume was adjusted to an optimal hearing level based on the feedback given by the participant. In line with previous MEG/EEG or fMRI studies testing semantic phrasal processing or syntactic phrasal processing, and
EP
discussed earlier in the introductory section, we also used an event-related design to ensure data comparability (Fig. 1B). The choice of an event-related design was
AC C
fundamentally driven by the fact that we were primarily interested in trial-related patterns of information processing between syntactic and semantic trials within the same task (Amaro & Barker, 2006), rather than aiming at uncovering sustained taskrelated activity during phrasal formation against control tasks—for which mixed designs are especially well suited, notwithstanding the introduction of independent research questions, like the involvement of more complex computations and the a priori stipulations of the optimal amount of trials for each block (Donaldson, 2004).
15
ACCEPTED MANUSCRIPT Therefore, because of the very reduced nature of the stimuli included in the experiment, together with the sparse fMRI data available, an event-related design
EP
TE D
M AN U
SC
RI PT
seems most suitable (Visscher et al., 2003).
AC C
Figure 1: Conditions and experimental design. A: Experimental conditions used for the fMRI study, SEM—semantically driven phrase (in blue), SYN—syntactically driven phrase (in red) and BAS—non-combinatorial baseline condition (in brown). B: Experimental design, every trial started with a fixation cross followed by the auditory stimulus. During the question mark, the participant had to decide if the stimulus could be part of a normal sentential structure.
In our design, each experimental trial started with a random jitter of 0, 500 or 1000 ms and was followed by the auditory stimulus. Within a reaction time window of 3 s, participants had to judge whether the stimulus within a trial of either two- or oneword length could be integrated in a normal sentential structure. Participants were
16
ACCEPTED MANUSCRIPT instructed to respond with the index and middle finger of their right hand on a twobutton response box. We used additional filler trials, which did not meet the relevant criteria (e.g. “blaues diese”, blue this), and for which subjects were requested to press the “no” button. The assignment of the buttons was counterbalanced across
RI PT
participants. The subsequent trial started after an inter-stimulus-interval of 3, 4 or 5 multiple of TR, 2.09 s, logarithmically distributed (7.733 s on average). All trials were included for further analysis. For the SYN condition—which included determiner-noun
more
specifically
in
the
most
anterior
SC
combinations—we expected to find increased neural responses in the left BA44–and ventral
section—in
the
frontal
M AN U
operculum/anterior insula, and in the superior and middle temporal gyrus independent of the contrast under analysis. With respect to the SEM condition— which conversely included adjective-noun combinations—we hypothesized that either adjective-noun phrases would have been processed in the same manner as we
TE D
expected for determiner-noun phrases—therefore triggering BA44 activity—or rather only triggering automatic syntactic analysis with low processing demands in BA44. Either way, we expected BA45, the AG and the ATL to activate as reflection of
EP
semantic processing, although we could not know whether these regions would have shown distinct patterns of activity for the different contrasts of interest, therefore
AC C
reflecting different functions within the semantic system.
2.4 Data acquisition
MRI data were obtained using a 3 Tesla Siemens Tim Trio MR scanner (Siemens Medical Systems, Erlangen, Germany) and a 12-channel headcoil at Max-Planck Institute for Human Cognitive and Brain Science in Leipzig, Germany. Functional images were acquired using a T2* weighted gradient-echo echo-planar-imaging (EPI) with 40 axial slices parallel to AC-PC line with whole brain coverage in ascending
17
ACCEPTED MANUSCRIPT direction with a functional scanning time of 27 minutes and 767 volumes. Scans had an in-plane resolution of 3 × 3 mm and a 2.5 mm slice thickness with an inter-slice gap of 0.5 mm (TR = 2.09 s, TE = 22 ms, flip angle 90°, field of view 192 mm, 64×64 matrix size). For all participants, individual high-resolution 3D T1-weighted MR scans
RI PT
acquired in previous sessions were available for normalization and co-registration.
2.5 Behavioral data analysis
SC
Reaction times were analyzed using a repeated-measures one-way ANOVA with the factor CONDITION (SYN, SEM, BAS), followed by paired t-tests between the three
corrected α = 0.05/3 = 0.016.
2.6 MRI Data analysis 2.6.1 Preprocessing
M AN U
different levels of the factor CONDITION within each cluster, with α=0.05, Bonferroni-
TE D
Preprocessing and statistical analysis were performed with SPM8 (available at www.fil.ion.ucl.ac.uk/spm, Wellcome Imaging Department, University Collage,
EP
London, UK) implemented in MATLAB 2010b (Mathworks, Inc., Sherborn, MA, USA). During the preprocessing, all functional images were realigned to the first EPI image
AC C
to correct for head motion; unwrapped to correct for geometric distortions due to susceptibility gradients based on the B0 field-map; resliced for time series correction and co-registered with the corresponding subject-specific structural T1-weighted image. We performed the normalization to standard MNI template as included in the SPM software package using a unified segmentation with a resampling size of 2 mm isotropic voxels. A Gaussian kernel of 6 mm full-width at half maximum was used to smooth the data. We applied a high-pass filter with a cut off period of 128 sec to
18
ACCEPTED MANUSCRIPT avoid low-frequency drift. The first five let-in volumes were excluded to allow for magnetic saturation effect.
2.6.2 Univariate imaging analysis
RI PT
For each participant we estimated a general linear model (Friston et al., 1995) as implemented in SPM 8 including the three conditions and the filler trials, by using a canonical hemodynamic response function. The six motion parameters from the realignment preprocessing were included to explain additional variance related to
SC
head movements. We first analyzed the BOLD response for the contrast SYN > BAS
M AN U
and SEM > BAS. First-level contrasts between conditions were run for each subject separately and then validated at second-level by means of one-sample t-test statistics. On the resulting cluster from the two contrasts, we used the Marsbar Toolbox for SPM (Brett, Anton, Valabregue, & Poline, 2002) to extract signal change percent over all trials of both syntactic phrases and semantic phrases along all voxels
TE D
within the cluster. The four resulting vectors were then entered into a 2X2 ANOVA with REGION and CONDITION as factors and a threshold set at p < 0.05. Two
EP
following paired t-tests were performed with α=0.05, Bonferroni-corrected α = 0.05/2 = 0.025. To note, an alternative analysis including a global whole-brain
AC C
analysis with combined SYN and SEM conditions against BAS—followed by ROIs analysis in the activated regions—might have compromised the data, given that crucial information on the functional activation differences between the two phrasal conditions would have been lost due to signal averaging across the two conditions during the comparison against the baseline. In a second phase, we performed two additional whole-brain contrasts (SYN > SEM and SEM > SYN) to compare syntactic phrases and semantic phrases directly. To account for the multiple comparison problem, a p < 0.001 voxel threshold was combined with a cluster extended threshold
19
ACCEPTED MANUSCRIPT of 0.05 family wise error (FWE) corrected for all the functional contrasts. The anatomical overlap between the contrasts SYN > BAS and SYN > SEM was also investigated with a conjunction analysis to test the common effects of syntactic computation between the two contrasts, as based on previous literature on local
RI PT
syntactic combinations (Zaccarella & Friederici, 2015a, 2016). We used the significant clusters of the IFG acquired in one-sample t-tests from the univariate
AND or a conjunction null hypothesis.
M AN U
2.6.3 Region-of-Interest Analysis within BA 44
SC
analysis above and calculated the overlapping voxels, which is similar to the logical
In a third phase, we additionally performed a region-of-interest analysis in BA 44, based on the hypothesis that within the region, only one sub-portion of the area (Cluster 3) would have shown some significant activity difference for the syntactic computation. Five clusters of BA44 were defined after Clos et al. (2013). We used
TE D
Marsbar to calculate the average percentage signal change over all trials of each of the three conditions along all voxels within the cluster. Fifteen paired t-tests were
EP
performed with α=0.05, Bonferroni-corrected α = 0.05/15 = 0.00333, where each
AC C
condition was compared to the other two in each of the five regions.
20
ACCEPTED MANUSCRIPT
3 Results 3.1 Behavioral Data Most of the participants performed near ceiling accuracy on all conditions. Median
RI PT
accuracy rate was 98.33% with an interquartile range of 8.99 (IQR=Q3-Q1=99.1792.50). Three participants were excluded because of poor performance, which was below 82.5% (less than Q1–1.5*IQR). The remaining 18 participants (9 females and
SC
9 males; mean age of 27.57 yrs.) were used for further analyses. Despite the high accuracy rate in the judgment task, significant differences in reaction time across the
M AN U
stimulus conditions were found F(1.118, 19.008) = 9.571; p = 0.005 (since the sphericity assumption has been violated χ2(2) = 24.87, p < 0.001, we reported the values after Greenhouse-Geisser correction).
Three paired-samples t-tests were conducted to compare the reaction times between
TE D
the conditions. All t-tests were Bonferroni-corrected for multiple comparisons. The baseline condition (one-word) with a mean reaction time of 0.673 s (SD = 0.107 s) was significantly slower than the SYN condition (M = 0.555, SD = 0.106, t(17) = -
EP
8.2845, p < 0.001) and the SEM condition (M = 0.530, SD = 0.106, t(17) = -12.7927,
AC C
p < 0.001). No significant reaction time difference was found between the SEM condition and the SYN condition (t(17) = 2.0783, p = 0.159). Notably, the reaction times data are in line with those found in previous studies manipulating the same type of experimental stimuli, where subjects were asked to process two-word phrases and single words (Bemis & Pylkkanen, 2012a). It appears that the integration between the two words in the two-word phrases is beneficial to the successful processing of the stimulus, such that for two-word phrases the noun is presented within a context, compared to one-word stimuli, where no such context is available.
21
ACCEPTED MANUSCRIPT Interestingly,
the
facilitatory
effects
of
contextual
information
have
been
demonstrated previously within the neurolinguistic tradition, showing how subjects are faster in matching a noun to a picture, when the same noun is preceded by an another word that can be integrated with it (e.g. adjective), than when it can not
RI PT
(Potter & Faulconer, 1979). This facilitatory effect at basic levels might in turn reflect those well-known facilitatory effects reported at sentential level, when grammatical strings are compared to word-list strings where no integration is possible (Bonhage,
SC
Fiebach, Bahlmann, & Mueller, 2014). Importantly, the present behavioral data speak against the possibility that higher brain activity for two-word phrases, compared to
M AN U
one-word phrases, might be due to higher effort to process the stimuli since the reverse pattern of responses is observed at the behavioral level.
3.2 MRI Data
TE D
3.2.1 Univariate imaging analysis
The activation clusters for the univariate analysis for the contrasts SYN > BAS and SEM > BAS are summarized in Table 1 and Figure 2 (top panels). We found
EP
significant activation for the contrast syntactic phrases against baseline (SYN > BAS) in the middle and posterior parts of the left IFG including the ventral BA44, BA45 and
AC C
the frontal operculum (FOP), and in ventral left superior temporal sulcus, bordering the superior section of the middle temporal gyrus. Semantic phrases against baseline (SEM > BAS) conversely activated a more anterior-dorsal region of the IFG in BA45. The ANOVA across the two different functional clusters in Broca’s area revealed a very strong interaction between the factors REGION and CONDITION (F(1,17) =33.871, p < 0.001), pointing towards a neat double dissociation within the IFG for the two phrasal constructions. One-tailed pared t-tests revealed that syntactic
22
ACCEPTED MANUSCRIPT phrases yield significantly higher activity in the most ventral BA44/45 compared to semantic phrases (t(17) = 3.92, p < 0.001), while semantic phrases yield significantly higher activity in the most dorsal BA45 compared to syntactic phrases (t(17) = 2.21,
AC C
EP
TE D
M AN U
SC
RI PT
p = 0.024).
Figure 2: Activation clusters from the univariate analysis. All contrasts were thresholded at p < 0.001 voxel-level and then 0.05 FWE-corrected at cluster-level. Volume renderings were done with MRIcron (Rorden & Brett, 2000). Diagrams show signal change percentage for each condition in each corresponding cluster. Signal change was extracted with MarsBaR (Brett et al., 2002). Error bars represent the standard error (SE). SYN (red bar); SEM (blue bar); BAS (yellow bar). See also Figure1A for a detailed explanation of the conditions.
23
ACCEPTED MANUSCRIPT The direct contrast syntactic phrases against semantic phrases (SYN > SEM) revealed activation in the ventral part of the IFG and in FOP, whereas the reverse contrast (SEM > SYN) showed activation in the left AG (Fig. 2, bottom panels).
Anatomical regions
SYN>BAS
SEM>SYN
z-score
42
4.308 3.861
z
L IFG, Pars triangularis
-42 -45
29 32
19 7
L STS/MTG
-54 -57 -63 -45 -54 -51 -54
-4 -19 -31 26 20 17 29
-5 1 7 -5 7 19 10
117
105
6.086 4.441 4.071 4.583 4.206 3.527 3.666
-33
-76
40
35
3.875
-63 -31 1 89 -51 -46 10 -66 -40 7 L IFG and FOP/aINS -51 20 -5 49 -48 23 -8 -36 11 -5 * p<0.001 uncorrected (voxel-level), p<0.05 FWE corrected (cluster-level)
4.519 4.167 4.114 3.942 3.859 3.618
L IFG and FOP/aINS
L AG
SC
y
L pSTS/MTG
EP
SYN>SEM
Cluster size*
x
M AN U
SEM>BAS
MNI coordinate
TE D
Contrast
RI PT
Table 1: Activation clusters for the different contrasts. Abbrev: L – left, IFG – Inferior frontal gyrus, STS – superior temporal sulcus, MTG – middle temporal gyrus, FOP/aINS – frontal operculum/anterior insula, AG – angular gyrus.
AC C
The activated region in the IFG for the contrast SYN > SEM—reflecting the comparison function word > content word—was smaller than the contrast SYN > BAS—reflecting function word + inflection > no word (49 voxel and 105 voxel respectively). Additional analyses covaring out reaction time effects across the different conditions revealed nearly identical functional patterns of activation as the ones discussed above (see supplementary Fig. 1 and supplementary Fig. 2).
24
ACCEPTED MANUSCRIPT We finally performed a conjunction analysis between the IFG activations for the contrasts (SYN > BAS) ∩ (SYN > SEM) to reveal commonalities for the syntactic computation driven by function words (Friston, Penny, & Glaser, 2005). Within the area, we found an overlap of 26 voxels, 53.06 % of the activated voxels during the
RI PT
SYN > SEM contrast which was also activated during the SYN > BAS contrast. The conjunction of the IFG activation between the contrasts (SYN > BAS) ∩ (SEM > BAS)
M AN U
SC
revealed no overlapping voxels. See Figure 5 with the IFG-activation for the different
EP
contrasts.
TE D
Figure 3: Overlap of the IFG activations. Sagittal, coronal, and axial cuts at slice -43, 19 and -4. All coordinates in MNI space. The yellow area shows the overlap between the SYN>SEM (red) and SYN>BAS (green) contrasts. The overlap is located in the ventral part of the IFG, including FOP (frontal operculum).
3.2.2 Region-of-Interest Analysis within BA 44
AC C
For all 5 clusters within BA44 we performed paired t-tests to see whether the activations for the two-word conditions differed significantly from the one word baseline. Only cluster CL3—the most anterior ventral one—revealed a significant difference between SYN and the non-combinatorial baseline condition, while none of the other p-values survived the Bonferroni correction (Table 2 for detailed results).
25
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
26
ACCEPTED MANUSCRIPT
1377
x
-56
y
8
z
21
SEM/SYN
0.2384
SYN/BAS SEM/BAS
CL4
CL5
2322
1863
1458
-51
-52
-55
-46
13
18
9
11
33
10
9
25
0.5085
0.0661
0.0170
0.9671
0.0857
0.0402
0.0007*
0.0603
0.0339
0.1874
0.0073
0.0974
0.7943
0.0077
AC C
percentage signal change
1917
CL3
M AN U
uncorrected p-values for conditions
mm³
TE D
center of mass
CL2
EP
volume
CL1
SC
RI PT
Table 2: Results for clusters within BA44. We defined five clusters of BA44 (top-left picture above the table) after (Clos et al., 2013). Fifteen tests were performed with α=0.05, which yielded to a Bonferroni-corrected α = 0.05/15 = 0.00333, where each condition was compared to the other two in each of the five BA44 clusters. Contrasts showing significant differences between conditions after correction are indicated with an asterisk, underlined, and marked in bold. See also (Zaccarella & Friederici, 2015a). The last row shows the diagrams reporting signal change percentage for SYN (red), SEM (blue) and BAS (yellow). Error bars denote the standard error (SE).
27
ACCEPTED MANUSCRIPT
4 Discussion The present study set out to define the neural basis of semantic and syntactic processes during the auditory processing of German two-word combinations. In our
RI PT
functional MRI paradigm, participants listened to two-word phrases in which a noun (boat, BAS) combined either with an adjective—a semantically rich contentive element (blue + boat, SEM), or with a determiner—a syntactically rich functional
SC
element (this + boat, SYN). We found activation for semantic processes (SEM > BAS contrast) in the dorsal part of anterior IFG (BA45), while syntactic processes (SYN >
M AN U
BAS contrast and SYN > SEM contrast) revealed a significant activation of the left ventral-posterior parts of the IFG (BA44), including the frontal operculum and the anterior insula. The double dissociation between the two phrases and regions of activation within the IFG was supported by a strong interaction effect in the area. The
TE D
comparison of SYN > SEM revealed a smaller activation contrast compared to SYN > BAS, which is possibly due to the remaining morphological element in the SEM condition. A further look into syntactic processes by means of a conjunction analysis
EP
between the contrasts (SYN > BAS) ∩ (SYN > SEM) yielded to an overlapping cluster in the ventral IFG region (BA44) extending to the frontal operculum. This
AC C
overlap—which reflects absence of difference between the two contrasts, mostly likely is due to the syntactic properties of the function words available in the syntactic condition. Additional syntax-related activation was found along the middle and posterior
superior
temporal
sulcus,
bordering
the
middle
temporal
gyrus
(SYN > BAS), or in the most posterior section of the posterior temporal sulcus (SYN > SEM). Conversely, the AG was activated for the SEM > SYN contrast, indicating its role in semantic processing.
28
ACCEPTED MANUSCRIPT
4.1 Syntax
and
semantics:
Contribution
of
the
IFG
and
FOP/anterior insula Activation in the left IFG was found by contrasting the two phrase conditions against the baseline condition. While for syntactic phrases the activation was located in the
RI PT
more ventral sections of the IFG, including BA44, BA45 and FOP/aINS, semantic phrases revealed a smaller activation cluster in the more anterior-dorsal part of the IFG (BA45). There was no overlap between the two activations of the IFG (as
SC
revealed by the conjunction analysis), suggesting that syntactic phrases are
M AN U
processed in a different way compared to phrases with semantic content. As such, the larger engagement of the IFG—especially the posterior part and the FOP— together with the result of the percentage signal change in the Cluster C3 of BA44 suggests a prominent role for syntax in this area during syntactic phrase processing. This effect cannot be due to mere word accumulation processing (two-words vs. one-
TE D
word), since the same activation pattern would have been expected for two-word semantic phrases vs. one-word baseline as well. This finding of a special involvement
EP
of Cluster C3 in BA 44 for syntactic phrase processing is in line with previous results (Zaccarella & Friederici, 2015b). The most straightforward interpretation of the
AC C
present data is that while two-word phrases both contain morphological inflections— which cancel each other out during direct contrast (SYN > SEM)—the availability of function words in the syntactic phrases steadily enhances syntactic processing in this area. This also explains why the opposite contrast between the two types of phrasal structures (SEM >SYN) did not reveal activation within the ventral BA44/45.
29
ACCEPTED MANUSCRIPT The semantic contrast (SEM > BAS) reported increased activity in the dorsal BA45, which did not overlap with the activation cluster for the syntactic contrast SYN > BAS. Thus brain activation for the processing of semantic phrases mainly recruits BA45, reflecting increased semantic load due to the additional semantic information carried
RI PT
by the adjective. This findings confirm the role of the anterior IFG (BA 45) for semantic processing, and suggest an involvement of this area not only for the analysis of propositional meaning for language at sentential level (Zaccarella et al.,
SC
2015), but also for basic semantic analysis of minimally complex stimuli, which modulate the region as a function of the quantity of semantic material to be
M AN U
processed. Equally important, it appears that for semantic processing at least at such basic levels, syntactic information linked to the presence of the inflectional morphology only leads to minimal activation in syntax-related regions as reflected in the hemodynamic response (to note, uncorrected p-values in BA44 tend to show that
TE D
at a finer-grained resolution activation differences between semantic phrases and baseline are indeed appreciable, see Table 2). For now, we, therefore, refrain from suggesting that phrasal constructions bypass syntactic information when not directly
EP
relevant. We do so on the basis of additional electrophysiological data indicating that incorrect inflectional morphology leaves its trace in the event-related wave-form
AC C
(Opitz, Regel, Müller, & Friederici, 2013). As such, we tend to follow the idea that syntax is a highly automatic, which can only reliably be measured under the condition of high temporal resolutions of brain activity (Hahne & Friederici, 1999; Pulvermuller et al., 2008). Compared to semantic phrases, syntactic phrases revealed a significant pattern of activity in the ventral parts of IFG (BA4), including frontal operculum and anterior insula. The ventral IFG (BA44) thus appears to be the region particularly engaged
30
ACCEPTED MANUSCRIPT during the establishment of syntactic computations forced by using a syntaxprominent element (a determiner) within the sequence, to form a syntactic phrase. The activation of the (ventral-) posterior parts for the effect of syntax processing is in line with the results of many neuroimaging studies (Bornkessel-Schlesewsky,
RI PT
Schlesewsky, & von Cramon, 2009; Friederici & Kotz, 2003; Hagoort, 2014; Makuuchi, Grodzinsky, Amunts, Santi, & Friederici, 2013; Zaccarella et al., 2015). In a more recent experiment (Goucha & Friederici, 2015) more closely inspected the
SC
role of the IFG in sentence processing, by asking subjects to attentively listen to both meaningful and meaningless sentences while preforming an acoustic monitoring
M AN U
task. When participants listened to meaningful structures, they found activation in the left IFG (BA44/45/47) as an effect of semantic processing. When all meaningful units including derivational morphology were removed from the stimulus, and content words were replaced by pseudowords, functional activation was limited to the BA44
TE D
area as an effect of syntactic analysis. Connectivity studies also suggest a separation of BA44 and BA45 in line with these fMRI findings (for an overview see Friederici and Gierhan (2013)). The dorsal pathway, connecting BA 44 and the posterior temporal
EP
cortex, is strongly associated to syntactic processing (Friederici, 2012; Friederici, Bahlmann, Heim, Schubotz, & Anwander, 2006; Wilson et al., 2011), whereas the
AC C
ventral pathway, which connects BA45 and the superior/middle temporal gyri, is rather taken to be involved in semantic representations (DeWitt & Rauschecker, 2012; Saur et al., 2008).
To note, we found FOP and the aINS supporting the processing of the syntactic computations—but not semantic composition—during the present experimental setting. The FOP/aINS region is considered to be a phylogenetically older and less specialized region than Broca’s area proper (Sanides, 1962) which is thought to
31
ACCEPTED MANUSCRIPT implement more rudimentary forms of syntactic computations which can be processed without computing hierarchical structures in specific contexts (Friederici et al. (2006). Some authors have suggested that this area might work as a word accumulation processor where words are maintained on hold prior delivering the
RI PT
sequences to those areas devoted to the construction of syntactic hierarchies (Stowe et al., 1998). Signal change in FOP/aINS is maximal in syntactically rich contexts and minimal for single words, with semantically rich contexts lying in between. The area
SC
therefore appears to recognize the categorical nature of the elements within the sequence (Friederici et al., 2000), and to string them together without implementing
M AN U
hierarchical building processes. This view is supported by findings showing that the same area was found to be active when two-word lists lacking syntactic information were compared to one-word elements (Zaccarella & Friederici, 2015a). With the present data at hand, we propose that the FOP/aINS supports the combination of two
TE D
subsequent elements, and that the activity in the area attunes to the category of the element, which is maximal when functional elements are introduced. The similar activation profiles for syntactic phrases we found in the ventral IFG and FOP are in
EP
strong agreement with a very recent study probing the intrinsic functional organization of Broca’s region, which found that the ventral BA44 and FOP together
AC C
constitute a functional module with highly similar connectivity profiles, and which the authors relate to syntactic processing. Conversely, BA45 appears to be strongly connected to semantic-related areas, including the AG (Zhang & Pylkkanen, 2015). Within this view, the engagement of the ventral parts of IFG is essentially syntactic in nature, while the processing of semantic compositions selectively engages its more anterior-dorsal part. Taken together, the present data show that the IFG is sensitive to the process of basic syntactic phrases, and they confirm the role of the inferior-
32
ACCEPTED MANUSCRIPT frontal region during syntactic structure-building processes even at the lowest level of phrasal formation.
4.2 Semantic composition: Contributions of the angular gyrus
RI PT
The activation of the AG was shown by contrasting semantic phrases (e.g. ‘red boat’) and syntactic phrases (e.g. ‘this boat’). Generally the AG is considered to be part of the heteromodal parietal association cortex (Rademacher, Galaburda, Kennedy ,
SC
Filipek, & Caviness, 1992) and appears to be activated in many different cognitive conditions (Seghier, 2013).
M AN U
Within the language domain, the function of the AG has been traditionally discussed to be semantic in nature (see the meta-analyses by Binder et al. (2009) and Vigneau et al. (2006). As such, the AG seems to be active for sentences compared to word lists (Humphries, Binder, Medler, & Liebenthal, 2007; Humphries, Love, Swinney, &
TE D
Hickok, 2005; Humphries, Willard, Buchsbaum, & Hickok, 2001). Additional evidence comes from lesion studies, which show a deficit in semantic comprehension during complex sentences (Dronkers, Wilkins, Van Valin, Redfern, & Jaeger, 2004; Hart &
EP
Gordon, 1990). Similarly, Pallier et al. (2011) showed enhanced activity in the AG when the semantic information of words was present, by parametrically increasing
AC C
the constituent size of the experimental stimuli, but not when semantic information was removed. At phrasal level, a MEG study by Bemis & Pylkkanen (2012a) compared adjective-noun phrase to simple non-compositional word lists in visual and auditory domain and found an increased activity in the left ATL followed by a left AG activity. Other studies showed higher activation in the AG during increased semantic plausibly, which reported AG engagement by comparing conventional compounds with novel, unlexicalized noun-noun compounds (Forgacs et al., 2012). This goes
33
ACCEPTED MANUSCRIPT together with the results of A. R. Price et al. (2015), who reported activation in AG during semantic integration. In their study, combinatorial plausibility (measured by phrasal frequency) in meaningful and less meaningful adjective-noun combinations modulated the degree of activity within the AG, with item analysis revealing a positive
RI PT
correlation between the beta value and the combinatorial plausibility. Similarly, a transcranial direct current stimulation study managed to speed up the comprehension for items with high plausibility, when an anodal stimulation to the left AG was
SC
performed (A. R. Price, Peelle, Bonner, Grossman, & Hamilton, 2016). In our experiment, however, while we found activity in the AG for the contrast between
M AN U
semantic phrases and syntactic phrases, our post-hoc analysis based on behavioral plausibility could not find any difference in terms of plausibility between the conditions. Therefore, an explanation in terms of a higher combinatorial plausibility does not seem to hold. At the same time, comparable functional activation was found
TE D
for both semantic phrases and single words. Therefore, we are cautious in ascribing a purely semantic composition function to AG, given that combinatorial processing would arguably not be involved for single words. This finding leads to the tentative
EP
proposal that within the semantic network, the AG serves the access for conceptual information to be evaluated. Similarly, our results could show that semantic
AC C
processing in AG is independent of any phrasal plausibility or confounds in working memory or variable amount of words. More importantly, going beyond to previous findings from the single word level (Hartwigsen et al., 2016), the present data from the phrasal level can functionally distinguish the AG from BA 45 as we found them independently active for two different semantic contrasts. Specifically, the AG checks what is more relevant to semantic processing by evaluating conceptual information of words. Conversely, significant activity difference in BA45 varies due to the increased
34
ACCEPTED MANUSCRIPT amount of semantic information, which is higher for semantic phrases compared to single words. Finally, with the present data at hand, we could not report any activation within the ATL, which opens to interesting questions regarding the exact role of the area in language processing. Future research—we believe—should
RI PT
specifically look at the functional modulations of the ATL, as generated by the use of different languages (e.g. English vs. German) and different tasks (e.g. picturematching vs. syntactic judgment).
SC
4.3 Other region in the temporal lobe for syntactic computation
M AN U
Activation in the left temporal lobe was reported for both contrasts comparing syntactic phrases to either the semantic phrases or to the baseline condition. Within the literature, the (posterior) temporal region is often proposed to be an integrative area in which syntactic and thematic-based information are assembled together
TE D
(Friederici, 2011; Grodzinsky & Amunts, 2006). Different studies have shown that the area responds more strongly when lexical-semantic information is available (Bornkessel et al., 2005; Friederici, Makuuchi, & Bahlmann, 2009; Stowe et al.,
EP
1998), or when thematic assignment needs to be evaluated (Ben-Shachar, Hendler, Kahn, Ben-Bashat, & Grodzinsky, 2003; Bornkessel et al., 2005; Constable et al.,
AC C
2004; Friederici et al., 2009; Roder, Stock, Neville, Bien, & Rosler, 2002; Zaccarella et al., 2015). In a previous study indeed such activation was shown for the contrast between prepositional phrases and word lists (Zaccarella et al., 2015). Here the authors argued that the region support thematic role assignment, and that as such it works as an integration region to map lexical information to syntactic argument hierarchies. This would also support the idea that the adjective–noun phrases do not need extensive syntactic processing. Nonetheless, the integrative nature of the area goes in hand with structural data revealing dorsal connections between the pSTG
35
ACCEPTED MANUSCRIPT and BA44 along the AF/SLF when probabilistic tractography is performed (Catani et al., 2005; Friederici & Gierhan, 2013; Perani et al., 2011; Saur et al., 2008). As such, the pSTS/STG in language integrates lexical/thematic information to syntactic hierarchical information coming from the IFG (den Ouden et al., 2012). A final remark
RI PT
concerns the more pronounced anterior activity along the superior temporal cortex we reported here for the contrast SYN>BAS. Some models of language processing assign to the anterior superior temporal cortex word category-based minimal
SC
constituent structuring, reflected as phrase structure template activation, once wordcategory information has been accessed during comprehension (Bornkessel &
M AN U
Schlesewsky, 2006). A purely syntactic-based interpretation for the region however appears to conflict with recent hemodynamic findings showing that the area responds to linguistic stimuli of increasing constituent size only when lexico-semantic information is available, but not when the same stimuli are converted into
TE D
pseudowords strings for which syntactic information alone is processed (Pallier et al., 2011). Further research is awaited to clarify the exact functional role of the more anterior superior temporal cortex—with respect to both the rostrally-located anterior
EP
temporal pole (Bemis & Pylkkanen, 2011) and the most posterior section of the temporal cortex (Zaccarella et al., 2015)—as well its structural characteristics within
AC C
the superior temporal cortical complex (Morosan et al., 2001).
36
ACCEPTED MANUSCRIPT
Conclusion The aim of this study was to get an understanding of the contribution of syntax and semantics to phrasal formation at the neural level. Our participants listened to simple
RI PT
two word phrases, while using different word classes to drive the processing to one or the other linguistic process. Our manipulations allowed us to differentiate two types of processes: syntactic computation and semantic composition. We could show that semantic phrases are processed mainly recruiting the AG and BA45 and with less
SC
activation in syntax-related areas. Syntactic phrases, on the contrary, mainly recruit
M AN U
the ventral IFG and posterior STS along the superior borders of the MTG. Overall, our data show that syntactic and semantic contribution to phrasal formation can be already differentiated at a very basic level, when two words are combined together in a phrase. In the same spirit, the restricted amount of active regions for the two processes appears to reflect the fundamentally limited resources needed to perform
TE D
phrasal processing at such basic level of linguistic complexity. The two processes do not functionally overlap at the neuro-anatomical level, and confirm the role of the
AC C
EP
ventral IFG for the construction of linguistic structures via syntactic information.
37
ACCEPTED MANUSCRIPT
References Amaro, E., Jr., & Barker, G. J. (2006). Study design in fMRI: basic principles. Brain Cogn, 60(3), 220232. doi:10.1016/j.bandc.2005.11.009 Badre, D. (2008). Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes. Trends Cogn Sci, 12(5), 193-200. doi:10.1016/j.tics.2008.02.004
RI PT
Bemis, D. K., & Pylkkanen, L. (2011). Simple composition: a magnetoencephalography investigation into the comprehension of minimal linguistic phrases. J Neurosci, 31(8), 2801-2814. doi:10.1523/JNEUROSCI.5003-10.2011 Bemis, D. K., & Pylkkanen, L. (2012a). Basic Linguistic Composition Recruits the Left Anterior Temporal Lobe and Left Angular Gyrus During Both Listening and Reading. Cereb Cortex, 23(8), 1859-1873. doi:10.1093/cercor/bhs170
SC
Bemis, D. K., & Pylkkanen, L. (2012b). Combination Across Domains: An MEG Investigation into the Relationship between Mathematical, Pictorial, and Linguistic Processing. Front Psychol, 3, 583. doi:10.3389/fpsyg.2012.00583
M AN U
Bemis, D. K., & Pylkkanen, L. (2013). Flexible composition: MEG evidence for the deployment of basic combinatorial linguistic mechanisms in response to task demands. PLoS One, 8(9), e73949. doi:10.1371/journal.pone.0073949 Ben-Shachar, M., Hendler, T., Kahn, I., Ben-Bashat, D., & Grodzinsky, Y. (2003). The neural reality of syntactic transformations: evidence from functional magnetic resonance imaging. Psychol Sci, 14(5), 433-440. doi:10.1111/1467-9280.01459 Berwick, R. C., Friederici, A. D., Chomsky, N., & Bolhuis, J. J. (2013). Evolution, brain, and the nature of language. Trends Cogn Sci, 17(2), 89-98. doi:10.1016/j.tics.2012.12.002
TE D
Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009). Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb Cortex, 19(12), 2767-2796. doi:10.1093/cercor/bhp055
EP
Bonhage, C. E., Fiebach, C. J., Bahlmann, J., & Mueller, J. L. (2014). Brain signature of working memory for sentence structure: enriched encoding and facilitated maintenance. J Cogn Neurosci, 26(8), 1654-1671. doi:10.1162/jocn_a_00566
AC C
Bonner, M. F., Vesely, L., Price, C., Anderson, C., Richmond, L., Farag, C., . . . Grossman, M. (2009). Reversal of the concreteness effect in semantic dementia. Cogn Neuropsychol, 26(6), 568579. doi:10.1080/02643290903512305 Bornkessel-Schlesewsky, I., Schlesewsky, M., & von Cramon, D. Y. (2009). Word order and Broca's region: evidence for a supra-syntactic perspective. Brain Lang, 111(3), 125-139. doi:10.1016/j.bandl.2009.09.004 Bornkessel, I., & Schlesewsky, M. (2006). The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages. Psychol Rev, 113(4), 787-821. doi:10.1037/0033-295x.113.4.787 Bornkessel, I., Zysset, S., Friederici, A. D., von Cramon, D. Y., & Schlesewsky, M. (2005). Who did what to whom? The neural basis of argument hierarchies during language comprehension. Neuroimage, 26(1), 221-233. doi:10.1016/j.neuroimage.2005.01.032 Brett, M., Anton, J.-L., Valabregue, R., & Poline, J.-B. (2002). Region of interest analysis using an SPM toolbox. Paper presented at the International Conference on Functional Mapping of the Human Brain, Sendai, Japan. Available on CD-ROM in NeuroImage, Vol 16, No 2.
38
ACCEPTED MANUSCRIPT Catani, M., Jones, D. K., & Ffytche, D. H. (2005). Perisylvian language networks of the human brain. Ann Neurol, 57(1), 8-16. doi:10.1002/ana.20319 Chomsky, N. (1995). The Minimalist Program. Cambridge, Mass.: The MIT Press. Clos, M., Amunts, K., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2013). Tackling the multifunctional nature of Broca's region meta-analytically: co-activation-based parcellation of area 44. Neuroimage, 83, 174-188. doi:10.1016/j.neuroimage.2013.06.041
RI PT
Constable, R. T., Pugh, K. R., Berroya, E., Mencl, W. E., Westerveld, M., Ni, W., & Shankweiler, D. (2004). Sentence complexity and input modality effects in sentence comprehension: an fMRI study. Neuroimage, 22(1), 11-21. doi:10.1016/j.neuroimage.2004.01.001 Del Prato, P., & Pylkkanen, L. (2014). MEG evidence for conceptual combination but not numeral quantification in the left anterior temporal lobe during language production. Front Psychol, 5, 524. doi:10.3389/fpsyg.2014.00524
SC
Deleon, J., Gesierich, B., Besbris, M., Ogar, J., Henry, M. L., Miller, B. L., . . . Wilson, S. M. (2012). Elicitation of specific syntactic structures in primary progressive aphasia. Brain Lang, 123(3), 183-190. doi:10.1016/j.bandl.2012.09.004
M AN U
den Ouden, D. B., Saur, D., Mader, W., Schelter, B., Lukic, S., Wali, E., . . . Thompson, C. K. (2012). Network modulation during complex syntactic processing. Neuroimage, 59(1), 815-823. doi:10.1016/j.neuroimage.2011.07.057 DeWitt, I., & Rauschecker, J. P. (2012). Phoneme and word recognition in the auditory ventral stream. Proc Natl Acad Sci U S A, 109(8), E505-514. doi:10.1073/pnas.1113427109 Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004). Lesion analysis of the brain areas involved in language comprehension. Cognition, 92(1-2), 145-177. doi:10.1016/j.cognition.2003.11.002
TE D
Egidi, G., & Caramazza, A. (2013). Cortical systems for local and global integration in discourse comprehension. Neuroimage, 71, 59-74. doi:10.1016/j.neuroimage.2013.01.003 Fiebach, C. J., Schlesewsky, M., Lohmann, G., von Cramon, D. Y., & Friederici, A. D. (2005). Revisiting the role of Broca's area in sentence processing: syntactic integration versus syntactic working memory. Hum Brain Mapp, 24(2), 79-91. doi:10.1002/hbm.20070
EP
Forgacs, B., Bohrn, I., Baudewig, J., Hofmann, M. J., Pleh, C., & Jacobs, A. M. (2012). Neural correlates of combinatorial semantic processing of literal and figurative noun noun compound words. Neuroimage, 63(3), 1432-1442. doi:10.1016/j.neuroimage.2012.07.029
AC C
Friederici, A. D. (2011). The brain basis of language processing from structure to function. Physiol Rev, 91, 1357-1392. doi:10.1152/physrev.00006.2011.-Language Friederici, A. D. (2012). The cortical language circuit: from auditory perception to sentence comprehension. Trends Cogn Sci, 16(5), 262-268. doi:10.1016/j.tics.2012.04.001 Friederici, A. D., Bahlmann, J., Friedrich, R., & Makuuchi, M. (2011). The neural basis of recursion of complex syntactic hierarchy. Biolinguistics, 5(1-2), 87-104. Friederici, A. D., Bahlmann, J., Heim, S., Schubotz, R. I., & Anwander, A. (2006). The brain differentiates human and non-human grammars: functional localization and structural connectivity. Proc Natl Acad Sci U S A, 103(7), 2458-2463. doi:10.1073/pnas.0509389103 Friederici, A. D., & Gierhan, S. M. (2013). The language network. Curr Opin Neurobiol, 23(2), 250-254. doi:10.1016/j.conb.2012.10.002
39
ACCEPTED MANUSCRIPT Friederici, A. D., & Kotz, S. A. (2003). The brain basis of syntactic processes: functional imaging and lesion studies. Neuroimage, 20, S8-S17. doi:10.1016/j.neuroimage.2003.09.003 Friederici, A. D., Makuuchi, M., & Bahlmann, J. (2009). The role of the posterior superior temporal cortex in sentence comprehension. Neuroreport, 20(6), 563-568. doi:10.1097/WNR.0b013e3283297dee
RI PT
Friederici, A. D., Meyer, M., & Von Cramon, D. Y. (2000). Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information. Brain Lang, 74(2), 289-300. doi:10.1006/brln.2000.2438 Friederici, A. D., Ruschemeyer, S. A., Hahne, A., & Fiebach, C. J. (2003). The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. Cereb Cortex, 13(2), 170-177. doi:10.1093/cercor/13.2.170
SC
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J.-P., Frith, C. D., & Frackowiak, R. S. J. (1995). Statistical parametric maps in functional imaging: a general linear model approach. Hum Brain Mapp, 2, 189-210. doi:10.1002/hbm.460020402 Friston, K. J., Penny, W. D., & Glaser, D. E. (2005). Conjunction revisited. Neuroimage, 25(3), 661667. doi:10.1016/j.neuroimage.2005.01.013
M AN U
Galton, C. J., Patterson, K., Graham, K., Lambon-Ralph, M. A., Williams, G., Antoun, N., . . . Hodges, J. R. (2001). Differing patterns of temporal atrophy in Alzheimer's disease and semantic dementia. Neurology, 57(2), 216-225. doi:10.1212/WNL.57.2.216 Gorno-Tempini, M. L., Dronkers, N. F., Rankin, K. P., Ogar, J. M., Phengrasamy, L., Rosen, H. J., . . . Miller, B. L. (2004). Cognition and anatomy in three variants of primary progressive aphasia. Ann Neurol, 55(3), 335-346. doi:10.1002/ana.10825
TE D
Gorno-Tempini, M. L., Hillis, A. E., Weintraub, S., Kertesz, A., Mendez, M., Cappa, S. F., . . . Grossman, M. (2011). Classification of primary progressive aphasia and its variants. Neurology, 76(11), 1006-1014. doi:10.1212/WNL.0b013e31821103e6 Goucha, T., & Friederici, A. D. (2015). The language skeleton after dissecting meaning: A functional segregation within Broca's Area. Neuroimage, 114, 294-302. doi:10.1016/j.neuroimage.2015.04.011
EP
Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., von Cramon, D. Y., & Schlesewsky, M. (2005). The emergence of the unmarked: a new perspective on the language-specific function of Broca's area. Hum Brain Mapp, 26(3), 178-190. doi:10.1002/hbm.20154
AC C
Grodzinsky, Y., & Amunts, K. (2006). Broca's Region. New York: Oxford Univ Pr. Hagoort, P. (2014). Nodes and networks in the neural architecture for language: Broca's region and beyond. Curr Opin Neurobiol, 28, 136-141. doi:10.1016/j.conb.2014.07.013 Hahne, A., & Friederici, A. D. (1999). Electrophysiological evidence for two steps in syntactic analysis. Early automatic and late controlled processes. J Cogn Neurosci, 11(2), 194-205. Hart, J., Jr., & Gordon, B. (1990). Delineation of single-word semantic comprehension deficits in aphasia, with anatomical correlation. Ann Neurol, 27(3), 226-231. doi:10.1002/ana.410270303 Hartwigsen, G., Weigel, A., Schuschan, P., Siebner, H. R., Weise, D., Classen, J., & Saur, D. (2016). Dissociating Parieto-Frontal Networks for Phonological and Semantic Word Decisions: A Condition-and-Perturb TMS Study. Cereb Cortex, 26(6), 2590-2601. doi:10.1093/cercor/bhv092 Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: what is it, who has it, and how did it evolve? Science, 298(5598), 1569-1579. doi:10.1126/science.298.5598.1569
40
ACCEPTED MANUSCRIPT Hodges, J. R., Mitchell, J., Dawson, K., Spillantini, M. G., Xuereb, J. H., McMonagle, P., . . . Patterson, K. (2010). Semantic dementia: demography, familial factors and survival in a consecutive series of 100 cases. Brain, 133(Pt 1), 300-306. doi:10.1093/brain/awp248 Humphries, C., Binder, J. R., Medler, D. A., & Liebenthal, E. (2006). Syntactic and semantic modulation of neural activity during auditory sentence comprehension. J Cogn Neurosci, 18(4), 665-679. doi:10.1162/jocn.2006.18.4.665
RI PT
Humphries, C., Binder, J. R., Medler, D. A., & Liebenthal, E. (2007). Time course of semantic processes during sentence comprehension: an fMRI study. Neuroimage, 36(3), 924-932. doi:10.1016/j.neuroimage.2007.03.059 Humphries, C., Love, T., Swinney, D., & Hickok, G. (2005). Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing. Hum Brain Mapp, 26(2), 128-138. doi:10.1002/hbm.20148
SC
Humphries, C., Willard, K., Buchsbaum, B., & Hickok, G. (2001). Role of anterior temporal cortex in auditory sentence comprehension: an fMRI study. Neuroreport, 12(8), 1749-1752. doi:10.1097/00001756-200106130-00046
M AN U
Jeon, H. A., & Friederici, A. D. (2013). Two principles of organization in the prefrontal cortex are cognitive hierarchy and degree of automaticity. Nat Commun, 4, 2041. doi:10.1038/ncomms3041 Koechlin, E., Ody, C., & Kouneiher, F. (2003). The architecture of cognitive control in the human prefrontal cortex. Science, 302(5648), 1181-1185. doi:10.1126/science.1088545 Lambon Ralph, M. A., Sage, K., Jones, R. W., & Mayberry, E. J. (2010). Coherent concepts are computed in the anterior temporal lobes. Proc Natl Acad Sci U S A, 107(6), 2717-2722. doi:10.1073/pnas.0907307107
TE D
Makuuchi, M., Bahlmann, J., Anwander, A., & Friederici, A. D. (2009). Segregating the core computational faculty of human language from working memory. Proceedings of the National Academy of Sciences, 106(20), 8362-8367. doi:10.1073/pnas.0810928106
EP
Makuuchi, M., & Friederici, A. D. (2013). Hierarchical functional connectivity between the core language system and the working memory system. Cortex, 49(9), 2416-2423. doi:10.1016/j.cortex.2013.01.007
AC C
Makuuchi, M., Grodzinsky, Y., Amunts, K., Santi, A., & Friederici, A. D. (2013). Processing noncanonical sentences in broca's region: reflections of movement distance and type. Cereb Cortex, 23(3), 694-702. doi:10.1093/cercor/bhs058 Mazoyer, B. M., Tzourio, N., Frak, V., Syrota, A., Murayama, N., Levrier, O., . . . Mehler, J. (1993). The cortical representation of speech. J Cogn Neurosci, 5(4), 467-479. doi:10.1162/jocn.1993.5.4.467 Meyer, L., Obleser, J., Anwander, A., & Friederici, A. D. (2012). Linking ordering in Broca's area to storage in left temporo-parietal regions: the case of sentence processing. Neuroimage, 62(3), 1987-1998. doi:10.1016/j.neuroimage.2012.05.052 Molinaro, N., Carreiras, M., & Duñabeitia, J. A. (2012). Semantic combinatorial processing of nonanomalous expressions. Neuroimage, 59(4), 3488-3501. doi:10.1016/j.neuroimage.2011.11.009 Molinaro, N., Paz-Alonso, P. M., Dunabeitia, J. A., & Carreiras, M. (2015). Combinatorial semantics strengthens angular-anterior temporal coupling. Cortex, 65C, 113-127. doi:10.1016/j.cortex.2015.01.004
41
ACCEPTED MANUSCRIPT Morosan, P., Rademacher, J., Schleicher, A., Amunts, K., Schormann, T., & Zilles, K. (2001). Human primary auditory cortex: cytoarchitectonic subdivisions and mapping into a spatial reference system. Neuroimage, 13(4), 684-701. doi:10.1006/nimg.2000.0715 Newman, S. D., Ikuta, T., & Burns, T., Jr. (2010). The effect of semantic relatedness on syntactic analysis: An fMRI study. Brain Lang, 113(2), 51-58. doi:10.1016/j.bandl.2010.02.001 Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9(1), 97-113.
RI PT
Opitz, A., Regel, S., Müller, G., & Friederici, A. D. (2013). Neurophysiological evidence for morphological underspecification in German strong adjective inflection. Language, 89(2), 231264. doi:10.1353/lan.2013.0033
SC
Pallier, C., Devauchelle, A. D., & Dehaene, S. (2011). Cortical representation of the constituent structure of sentences. Proc Natl Acad Sci U S A, 108(6), 2522-2527. doi:10.1073/pnas.1018711108 Patterson, K., Nestor, P. J., & Rogers, T. T. (2007). Where do you know what you know? The representation of semantic knowledge in the human brain. Nat Rev Neurosci, 8(12), 976-987. doi:10.1038/nrn2277
M AN U
Perani, D., Saccuman, M. C., Scifo, P., Anwander, A., Spada, D., Baldoli, C., & al., e. (2011). Neural language networks at birth. Proc Natl Acad Sci U S A, 108(38), 16056-16061. doi:10.1073/pnas.1102991108. Potter, M. C., & Faulconer, B. A. (1979). Understanding noun phrases. Journal of Verbal Learning and Verbal Behavoir, 18, 509-521.
TE D
Price, A. R., Bonner, M. F., Peelle, J. E., & Grossman, M. (2015). Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular gyrus. J Neurosci, 35(7), 32763284. doi:10.1523/JNEUROSCI.3446-14.2015 Price, A. R., Peelle, J. E., Bonner, M. F., Grossman, M., & Hamilton, R. H. (2016). Causal Evidence for a Mechanism of Semantic Integration in the Angular Gyrus as Revealed by High-Definition Transcranial Direct Current Stimulation. J Neurosci, 36(13), 3829-3838. doi:10.1523/JNEUROSCI.3120-15.2016
EP
Price, C. J., Moore, C. J., Humphreys, G. W., & Wise, R. J. (1997). Segregating Semantic from Phonological Processes during Reading. J Cogn Neurosci, 9(6), 727-733. doi:10.1162/jocn.1997.9.6.727
AC C
Pulvermuller, F., Shtyrov, Y., Hasting, A. S., & Carlyon, R. P. (2008). Syntax as a reflex: neurophysiological evidence for early automaticity of grammatical processing. Brain Lang, 104(3), 244-253. doi:10.1016/j.bandl.2007.05.002 Rademacher, J., Galaburda, A. M., Kennedy , D. N., Filipek, P. A., & Caviness, V. S. (1992). Human Cerebral Cortex: Localization, Parcellation, and Morphometry with Magnetic Resonance Imaging. J Cogn Neurosci, 4(4), 352-374. doi:10.1162/jocn.1992 Roder, B., Stock, O., Neville, H., Bien, S., & Rosler, F. (2002). Brain activation modulated by the comprehension of normal and pseudo-word sentences of different processing demands: a functional magnetic resonance imaging study. Neuroimage, 15(4), 1003-1014. doi:10.1006/nimg.2001.1026 Rogalski, E., Cobia, D., Harrison, T. M., Wieneke, C., Thompson, C. K., Weintraub, S., & Mesulam, M. M. (2011). Anatomy of language impairments in primary progressive aphasia. J Neurosci, 31(9), 3344-3350. doi:10.1523/jneurosci.5544-10.2011
42
ACCEPTED MANUSCRIPT Rohrer, J. D., Warren, J. D., Modat, M., Ridgway, G. R., Douiri, A., Rossor, M. N., . . . Fox, N. C. (2009). Patterns of cortical thinning in the language variants of frontotemporal lobar degeneration. Neurology, 72(18), 1562-1569. doi:10.1212/WNL.0b013e3181a4124e Rorden, C., & Brett, M. (2000). Stereotaxic display of brain lesions. Behav Neurol, 12(4), 191-200. doi:10.1155/2000/421719 Sanides, F. (1962). Die Architektonik des Menschlichen Stirnhirns. Berlin Heidelberg: Springer.
RI PT
Santi, A., & Grodzinsky, Y. (2007). Working memory and syntax interact in Broca's area. Neuroimage, 37(1), 8-17. doi:10.1016/j.neuroimage.2007.04.047 Saur, D., Kreher, B. W., Schnell, S., Kummerer, D., Kellmeyer, P., Vry, M. S., . . . Weiller, C. (2008). Ventral and dorsal pathways for language. Proc Natl Acad Sci U S A, 105(46), 18035-18040. doi:10.1073/pnas.0805234105
SC
Segaert, K., Menenti, L., Weber, K., Petersson, K. M., & Hagoort, P. (2012). Shared syntax in language production and language comprehension--an FMRI study. Cereb Cortex, 22(7), 1662-1670. doi:10.1093/cercor/bhr249
M AN U
Seghier, M. L. (2013). The angular gyrus: multiple functions and multiple subdivisions. Neuroscientist, 19(1), 43-61. doi:10.1177/1073858412440596 Smith, E. E. (1984). Conceptual combination with prototype concepts. Cognitive Sci, 8, 337-361. doi:10.1207/s15516709cog0804_2 Snowden, J. S. (1995). Semantic dementia: a form of circumscribed cerebral atrophy. Neurocase, 1(1). doi:10.1093/neucas/1.1.39-y
TE D
Stowe, L. A., Broere, C. A., Paans, A. M., Wijers, A. A., Mulder, G., Vaalburg, W., & Zwarts, F. (1998). Localizing components of a complex task: sentence processing and working memory. Neuroreport, 9(13), 2995-2999. doi:10.1097/00001756-199809140-00014 Vandenberghe, R., Nobre, A. C., & Price, C. J. (2002). The response of left temporal cortex to sentences. J Cogn Neurosci, 14(4), 550-560. doi:10.1162/08989290260045800
EP
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., . . . Tzourio-Mazoyer, N. (2006). Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing. Neuroimage, 30(4), 1414-1432. doi:10.1016/j.neuroimage.2005.11.002
AC C
Visser, M., Jefferies, E., & Lambon Ralph, M. A. (2010). Semantic processing in the anterior temporal lobes: a meta-analysis of the functional neuroimaging literature. J Cogn Neurosci, 22(6), 10831094. doi:10.1162/jocn.2009.21309 Westerlund, M., Kastner, I., Al Kaabi, M., & Pylkkanen, L. (2015). The LATL as locus of composition: MEG evidence from English and Arabic. Brain Lang, 141, 124-134. doi:10.1016/j.bandl.2014.12.003 Westerlund, M., & Pylkkanen, L. (2014). The role of the left anterior temporal lobe in semantic composition vs. semantic memory. Neuropsychologia, 57C, 59-70. doi:10.1016/j.neuropsychologia.2014.03.001 Wilson, S. M., Dronkers, N. F., Ogar, J. M., Jang, J., Growdon, M. E., Agosta, F., . . . Gorno-Tempini, M. L. (2010). Neural correlates of syntactic processing in the nonfluent variant of primary progressive aphasia. J Neurosci, 30(50), 16845-16854. doi:10.1523/JNEUROSCI.254710.2010 Wilson, S. M., Galantucci, S., Tartaglia, M. C., & Gorno-Tempini, M. L. (2012). The neural basis of syntactic deficits in primary progressive aphasia. Brain Lang, 122(3), 190-198. doi:10.1016/j.bandl.2012.04.005
43
ACCEPTED MANUSCRIPT Wilson, S. M., Galantucci, S., Tartaglia, M. C., Rising, K., Patterson, D. K., Henry, M. L., . . . GornoTempini, M. L. (2011). Syntactic processing depends on dorsal language tracts. Neuron, 72(2), 397-403. doi:10.1016/j.neuron.2011.09.014 Xu, J., Kemeny, S., Park, G., Frattali, C., & Braun, A. (2005). Language in context: emergent features of word, sentence, and narrative comprehension. Neuroimage, 25(3), 1002-1015. doi:10.1016/j.neuroimage.2004.12.013
RI PT
Zaccarella, E., & Friederici, A. D. (2015a). Merge in the Human Brain: A Sub-Region Based Functional Investigation in the Left Pars Opercularis. Front Psychol, 6, 1818. doi:10.3389/fpsyg.2015.01818 Zaccarella, E., & Friederici, A. D. (2015b). Reflections of word processing in the insular cortex: A subregional parcellation based functional assessment. Brain Lang, 142, 1-7. doi:10.1016/j.bandl.2014.12.006
SC
Zaccarella, E., & Friederici, A. D. (2016). The neurobiological nature of syntactic hierarchies. Neurosci Biobehav Rev. doi:10.1016/j.neubiorev.2016.07.038
M AN U
Zaccarella, E., Meyer, L., Makuuchi, M., & Friederici, A. D. (2015). Building by Syntax: The Neural Basis of Minimal Linguistic Structures. Cereb Cortex. doi:10.1093/cercor/bhv234 Zhang, L., & Pylkkanen, L. (2015). The interplay of composition and concept specificity in the left anterior temporal lobe: An MEG study. Neuroimage, 111C, 228-240. doi:10.1016/j.neuroimage.2015.02.028 Zhu, Z., Feng, G., Zhang, J. X., Li, G., Li, H., & Wang, S. (2013). The role of the left prefrontal cortex in sentence-level semantic integration. Neuroimage, 76, 325-331. doi:10.1016/j.neuroimage.2013.02.060
AC C
EP
TE D
Zhu, Z., Zhang, J. X., Wang, S., Xiao, Z., Huang, J., & Chen, H. C. (2009). Involvement of left inferior frontal gyrus in sentence-level semantic integration. Neuroimage, 47(2), 756-763. doi:10.1016/j.neuroimage.2009.04.086
44
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
45