Verbal working memory and the phonological buffer: The question of serial order

Accepted Manuscript Verbal working memory and the phonological buffer: The question of serial order Steve Majerus PII:

S0010-9452(18)30153-9

DOI:

10.1016/j.cortex.2018.04.016

Reference:

CORTEX 2318

To appear in:

Cortex

Received Date: 29 November 2017 Revised Date:

19 March 2018

Accepted Date: 24 April 2018

Please cite this article as: Majerus S, Verbal working memory and the phonological buffer: The question of serial order, CORTEX (2018), doi: 10.1016/j.cortex.2018.04.016. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

Verbal working memory and the phonological buffer: The question of serial order

RI PT

Steve Majerus

Psychology and Neuroscience of Cognition Research Unit, University of Liège

M AN U

SC

Fund for Scientific Research FNRS, 1000 Brussels, Belgium

TE D

Corresponding author:

Dr. Steve Majerus, Psychology and Neuroscience of Cognition Research Unit, Université de Liège; Boulevard du Rectorat, B33, 4000 Liège – Belgium; Tel: 0032 4 3664656; Email:

AC C

EP

[email protected]

1

ACCEPTED MANUSCRIPT

Verbal working memory and the phonological buffer:

AC C

EP

TE D

M AN U

SC

RI PT

The question of serial order

1

ACCEPTED MANUSCRIPT Abstract The concept of modality specific buffers for the temporary storage of information is a fundamental characteristic of the Working memory model proposed by Baddeley and Hitch (1974). The phonological input buffer does not make an explicit distinction between the

RI PT

identity and the serial order of memoranda, both relying on phonological codes. This review provides a critical examination of the codes and processes involved in item and serial order maintenance capabilities. On the one hand, an increasing number of studies indicate that brain

SC

injury can lead to selective impairment for the short-term retention of item versus serial order

M AN U

information. Neuroimaging studies also reveal the involvement of distinct neural substrates for the retention of item and serial order information, and possibly shared neural substrates for the retention of serial order information in verbal and visuo-spatial modalities. Other studies suggest that phonological processing areas within the dorsal language pathway can store both item and serial order information but via separate representational mechanisms. Overall

TE D

evidence indicates that serial order information in verbal working memory may be represented via multiple processes simultaneously, some being domain general and some being phonological. The phonological serial order codes appear to rely on a dorsal language

EP

pathway that has also been proposed to support a phonological buffer system, but even at this

AC C

level, distinct processes for the coding item and serial order information need to be considered.

228 words

Keywords: working memory, short-term memory, serial order, language, phonological processing, neuropsychological, neuroimaging

2

ACCEPTED MANUSCRIPT 1. Introduction The concept of buffer systems specialized in the temporary storage of modality-specific information has been one of the most fundamental features of the Working memory model proposed by Baddeley and Hitch (1974). These buffer systems do not make an explicit

RI PT

distinction between the memoranda and their serial order, i.e. their moment of occurrence within a list of stimuli. More precisely, the phonological input buffer (also called the

‘phonological store’) of the phonological loop system relies on phonological codes (Baddeley,

SC

1986). These codes, by being sound-based and inherently sequential, represent information

M AN U

about individual phonemes as well as about their serial order by keeping track of the associations between temporally adjacent items. At the neuroanatomical level, the phonological loop system has been proposed to encompass the posterior superior temporal gyrus / inferior parietal lobule (supramarginal gyrus) area and the inferior frontal cortex (Broca’s area), based on the observation that patients with lesions in these areas present with

TE D

selective impairment in verbal working memory (WM) tasks such as word span and digit span tasks (Vallar & Papagno, 2002). The posterior part of this network has been considered to represent a phonological input buffer function, while the frontal component supports the

EP

refreshing of information via subvocal articulatory rehearsal processes, and this without

AC C

distinguishing individual memory items and their serial order (Shallice & Warrington, 1977; Vallar, Di Betta, & Silveri, 1997; Vallar & Papagno, 2002; Warrington, 1981). However, the question of the representation of serial order information remains a major theoretical challenge for any model of WM. The representation of serial order information only via phonological, chaining-based mechanisms cannot account for the complexity of serial order phenomena that have been observed. Among these phenomena, the observation of a dissociation between codes for item information and codes for serial information, at behavioral, neuropsychological and neuroimaging levels, is of major theoretical interest. In 3

ACCEPTED MANUSCRIPT this review, I will examine the nature of processes involved in representing item and serial order information in verbal WM, based on behavioral, neuropsychological and neuroimaging evidence. I will show that serial order information may be represented by different processes simultaneously, some of these being domain general as they are shared by both verbal and

RI PT

visual WM modalities. I will further show that serial order information is also coded via

SC

phonological codes, but these codes need to be distinguished from item-level representations.

2. Evidence for a dissociation of item and serial order retention capacities Definition of item and serial order information in WM

M AN U

2.1.

Before examining the nature of codes and processes involved in the representation of item and serial order information, it is important that we define the different types of item and serial order information that have to processed and maintained in verbal WM tasks. Item-level

TE D

information is usually considered to reflect the verbal units that define the memoranda; these units are typically single words, but can also represent sub-word level information such as syllables and phonemes in the case of nonword recall tasks (see Table 1). At the word level,

EP

item information concerns the phonological (word form) and semantic (word meaning) aspects that define the identity of a given word and allow to distinguish it from other words

AC C

(e.g., Lee & Estes, 1981; Nairne & Kelley, 2004). On the other hand, serial order information concerns information about the temporal succession of verbal units within a list of stimuli (e.g., Lee & Estes, 1981; Nairne & Kelley, 2004). Typically, in word list immediate serial recall tasks, serial order information concerns the temporal order of occurrence of each word in the list, i.e., its within-list serial positions. Critically, the serial order of each word is in principle unpredictable, as the words are presented in arbitrary serial order. Hence serial order information needs to be temporarily represented and cannot not be encoded using structures stored in long-term memory (see Table 1). At the same time, not all aspects of serial order 4

ACCEPTED MANUSCRIPT coding require short-term storage of serial order information. The order of the phonemes within each word does not need to be represented temporarily as the long-term lexical phonological representations of the word will bind the different phonemes and their order together into a single lexical representations (see Sections 3.4, 4.1 and 4.2 for a more precise

RI PT

discussion of the possible mechanisms involved). In other words, the serial order of phonemes within a familiar word is supported by long-term serial order information (see Table 1).

SC

< INSERT TABLE 1 ABOUT HERE >

For WM tasks involving nonword stimuli, the situation is even more complex. For

M AN U

single, multisyllabic nonword repetition (e.g., Gathercole, Willis, Baddeley, & Emslie, 1994), the phonemes can be considered to reflect item information, while the sequential ordering of the phonemes reflect arbitrary serial order information that needs to be represented in verbal WM. At the same time, the ordering of phonemes and syllables can also be supported by

TE D

subsyllabic phonological knowledge about phoneme/syllable transition probabilities (phonotactic knowledge; Vitevitch & Luce, 1999) if the nonwords are composed of high frequency transition probabilities (Gathercole, 1995; Gathercole, Frankish, Pickering, &

EP

Peaker, 1999; Majerus, Van der Linden, Mulder, Meulemans, & Péters, 2004; Majerus, Martinez Perez, & Oberauer, 2012; see also below). In that case, long-term serial order

AC C

knowledge will also intervene even if the stimuli are nonwords (see Table 1). This situation is even more complex for nonword list immediate serial recall tasks in which there is an additional level of serial order representation, namely the serial position of each nonword within the list. The serial position of nonwords reflects arbitrary serial order information and needs to be represented using short-term serial order representations. In sum, different levels of serial order information characterize the type of information that intervene in verbal WM tasks. When serial order information is arbitrary, it needs to be represented using some form of short-term serial order codes, and it is in this situation that a short-term storage system, 5

ACCEPTED MANUSCRIPT such as the phonological buffer, is needed for temporarily representing serial order information. When serial order can be defined by long-term memory structures, long-term serial order knowledge will support the storage of serial order information in verbal WM tasks. Studies that have investigated the dissociation of item and serial order information in

RI PT

WM have mainly used word immediate serial recall tasks, with short-term serial order information characterizing the list-level serial order in which the words are presented. 2.2.

Behavioral evidence

SC

Several behavioral studies support a distinction of processes involved in WM for item and

M AN U

serial order information, starting with the early work by Bjork and Healy (1974). Bjork and Healy showed that item and order information appear to be forgotten at different rates. More generally, it has been shown that intra-list phonemic similarity has opposing effects on item and serial order recall, by impairing serial order recall but improving item recall (Crowder, 1979; Lian, Karlsen, & Eriksen, 2004). Similarly, when presenting list items in small

TE D

temporal groups, a benefit is observed for serial order recognition but not item recognition (Henson, Hartley, Burgess, Hitch, & Flude, 2003). Further evidence allowing us to gain a better understanding of the origin of these dissociations stems from studies that have

EP

investigated the impact of linguistic knowledge on verbal WM performance. A large number

AC C

of studies in healthy populations have shown that verbal WM performance is increased for immediate serial recall of word lists as compared to nonword lists (Hulme, Maughan, & Brown, 1991; Gathercole, Pickering, Hall, & Peaker, 2001; Majerus & Van der Linden, 2003), the so-called lexicality effect in WM. Word list immediate serial recall is also increased for word stimuli with particular rich or easy-to-access lexical and semantic longterm memory representations such as high versus low frequency words, or concrete versus abstract word stimuli (Hulme et al., 1997, 2003; Majerus & Van der Linden, 2003; Walker & Hulme, 1999). Critically, linguistic knowledge appears to support recall of items more than 6

ACCEPTED MANUSCRIPT recall of their serial order, with sometimes even reversed linguistic knowledge effects for serial order recall for tasks manipulating semantic similarity effects (Murdock & vom Saal, 1967; Nairne & Kelley, 2004; Poirier & Saunt-Aubin, 1996; Saint-Aubin & Poirier, 1999; Whiteman, Nairne, & Serra, 1994; see also Section 3.4 for a discussion of item and serial

RI PT

order recall dissociations for the lexicality effect). These findings support theoretical models of WM that distinguish between item and serial order representational mechanisms, by further considering that items are represented via direct interactions with linguistic knowledge, while

SC

(arbitrary) serial order information is encoded via distinct and specific mechanisms (e.g.,

also Section 3). 2.3.

Neuropsychological evidence

M AN U

Burgess & Hitch, 1999; Gupta & MacWhinney, 1997; Brown, Preece, & Hulme, 2000; see

This dissociation between item and list-level serial order retention capacities is also supported by data in brain injured patients with linguistic impairment. Patients with impairment to

TE D

semantic levels of linguistic knowledge due to neurodegenerative disease (semantic dementia) can show particularly poor item recall performance in word immediate serial recall tasks, while their serial order recall performance is preserved (after controlling for the fact that they

EP

recall fewer items and hence may present less opportunities for making serial order errors)

AC C

(Majerus, Norris, & Patterson, 2007). It should also be noted here that the vast majority of patients considered to have specific verbal WM impairment actually show a history of language impairment (aphasia) and a review showed that the severity of their verbal WM impairment strongly correlated with residual language processing abilities (Majerus, 2009). More directly, a recent study in a group of 14 patients with reduced verbal WM capacities as a result of brain injury investigated serial order and item WM abilities in this type of patients and revealed several double dissociations between item and serial order short-term retention deficits. Majerus, Attout, Artielle and Van der Kaa (2015) observed that, based on the 7

ACCEPTED MANUSCRIPT proportions of item and serial order errors in several word list immediate serial recall and serial order reconstruction tasks, 29% of patients showed relatively selective list-level serial order retention deficits while 21% of patients showed relatively selective item retention deficits, with the remaining patients showing similar levels of performance for both item and

RI PT

serial order verbal WM measures. Furthermore, item WM but not serial order WM scores correlated with the patients’ severity of aphasic symptoms, replicating and specifying the previous results by Majerus (2009). These results indicate that item and serial order short-

2.4.

Neuroimaging evidence

M AN U

strongly associated with language impairment.

SC

term retention capacities can be specifically impaired, with item WM impairment being

These neuropsychological dissociations between WM for item and WM for serial order information are also supported by neuroimaging studies in healthy adults. Several neuroimaging studies contrasted item and serial order probe recognition tasks in which lists of

TE D

verbal items (letters or words) were presented, and either item or serial order memory was tested via the presentation of item/serial order probes (Marshuetz, Smith, Jonides, DeGutis, & Chenevert, 2000; Marshuetz, Reuter-Lorenz, Smith, Jonides, & Noll, 2006; Henson, Burgess,

EP

& Frith, 2000; Majerus et al., 2006, 2010). These studies showed that the serial order

AC C

conditions recruited to a larger extent fronto-parietal cortices (including the intraparietal sulcus), as compared to item conditions which recruited to a larger extent fronto-temporal cortices (extending to the supramarginal gyrus) associated with phonological and orthographic aspects of language processing. Studies controlling in a more stringent manner for task difficulty and attentional requirements involved in item and serial order probe recognition tasks further showed a more specific involvement of the right intraparietal sulcus in the serial order WM conditions (Majerus et al., 2006, 2010). Interestingly, right intraparietal sulcus activation has also been observed for serial order probe recognition tasks 8

ACCEPTED MANUSCRIPT involving sequences of visual information such as unfamiliar faces (Majerus et al., 2007, 2010; Martinez Perez, Poncelet, Salmon, & Majerus, 2015). These results show that serial order processing does not only dissociate from item processing in WM tasks, but serial order memory appears to involve neural substrates clearly distinct from those thought to support a

2.5.

The domain general nature of serial order codes

RI PT

phonological buffer (left posterior superior temporal gyrus and supramarginal gyrus).

SC

The different studies reviewed so far indicate that item and serial order information are

supported by distinct representational mechanisms, with item information in verbal WM

M AN U

depending critically on interactions with the linguistic system. At the same time, the exact nature of the mechanisms supporting the coding of serial order information remains an open question. The results of the neuroimaging studies presented in the previous subsection suggest that these mechanisms may not be specific to verbal WM, but may rely on more domain

TE D

general processes shared with visual WM, as indicated by a common involvement of the right intraparietal sulcus in verbal and visual serial order WM. One hypothesis that has been advanced is the hypothesis of spatial coding of serial order information (Abrahamse, van

EP

Dijck, Majerus, & Fias, 2014; Ginsburg & Gevers, 2015; Ginsburg, van Dijck, Previtali, Fias, & Gevers, 2014; van Dijck & Fias, 2011). In a series of studies, van Dijck and colleagues

AC C

showed that temporal sequences automatically trigger spatial representational processes. In their seminal study, van Dijck and Fias (2011) observed that items from early serial positions in a verbal WM task (with sequential presentation of memoranda) were judged more rapidly with the left hand than the right hand when being presented for recognition and semantic judgment during the maintenance delay of the verbal WM task. Conversely, items from later serial positions were judged more rapidly with the right hand than the left hand. van Dijck, Abrahamse, Majerus and Fias (2013) observed similar results using vocal responses instead of right/left lateralized hand responses. These data indicate that temporal sequences of verbal 9

ACCEPTED MANUSCRIPT information may be represented using a spatial reference, structured from left to right. The hypothesis of domain-general, possibly spatial, serial order mechanisms is also supported by studies that investigated serial order processing in other modalities than the auditory-verbal WM modality. Hurlstone, Hitch, and Baddeley (2014) showed that hallmark serial order

RI PT

coding effects are very similar across verbal and visuo-spatial WM domains, with

transposition gradients for serial order errors marked by a locality constraint1 and increased serial order recall performance for temporally grouped versus ungrouped lists in both

SC

modalities. Some of these serial order memory effects have also been shown to characterize auditory non-verbal WM domains such as the musical WM domain. Gorin, Kowialiewski and

M AN U

Majerus (2016) showed that WM tasks probing the retention of serial order but not item information for tone sequences were negatively impacted by a concurrent rhythm reproduction task. These findings are very similar to the results obtained by Henson et al. (2003) in the verbal domain which showed that a rhythm reproduction task interferes with

TE D

serial order but not item verbal WM recognition tasks. Temporal grouping effects have also been observed in musical WM tasks, with higher performance for grouped than ungrouped tone lists (Deutsch, 1980; Gorin, Mengal & Majerus, submitted). Furthermore, locality

EP

constraints also characterize transposition gradients for serial order exchange errors in musical sequence recall tasks (Mathias, Pfordresher, & Palmer, 2014; Pfordresher, Palmer, & Jungers,

AC C

2007). Some of these serial order memory effects have recently been observed to occur also in tactile and olfactory WM tasks (Johnson & Miles, 2007; Johnson, Shaw & Miles, 2016). Critically, studies by Depoorter and Vandierendonck (2009) as well as Vandierendonck (2016) showed cross-domain interference of serial order coding: concurrent verbal and visuospatial WM tasks led to maximal interference when serial order recall rather than item recall was stressed. 1

Serial position exchange errors increase for adjacent versus more distant serial positions 10

ACCEPTED MANUSCRIPT 2.6.

Summary

In sum, although the question of the precise nature of the codes that support serial order coding across modalities is far from being resolved, there is increasing evidence for the existence of domain-general serial order coding processes which are difficult to reconcile with

SC

3. Evidence for phonological-based serial order codes

RI PT

a unique phonological coding account of serial order in verbal WM.

Although the data reviewed so far support the existence of domain-general codes involved in

M AN U

serial order coding, they do not allow rule out the existence of additional phonological codes that would support serial order coding more specifically in the verbal WM modality. I will now examine studies that address more directly the phonological nature of serial order codes in verbal WM, and the extent to which item and serial order levels of processing also need to

TE D

be distinguished at this level. 3.1. Behavioral evidence

EP

At the behavioral level, a number of studies have shown that phonological variables influence recall of serial order information, although the effects sometimes go in opposite directions

AC C

relative to recall of item information. As already mentioned in the previous section, the effect of phonological similarity on serial order recall is not null, but rather it affects serial order recall in a deleterious manner: serial order recall errors increase under conditions of phonological similarity, indicating that the phonological nature of memoranda interacts with the coding of their serial position in a memory list (Baddeley, 1966; Crowder, 1979; Lian et al., 2004). Similarly, Henson et al. (2003) showed that articulatory suppressing, blocking the rehearsal of phonological, articulatory codes, has a stronger impact on serial order recognition than on item recognition. In the same study, the authors also showed that the addition of 11

ACCEPTED MANUSCRIPT phonological interference in the form of irrelevant speech presented during encoding of memoranda led to a greater detrimental effect for serial order than item recognition. These results suggest that phonological characteristics constrain serial order recall performance, and hence must interact with the coding of serial order information. These studies however do not

RI PT

provide information about the exact nature of these interactions, and to what extent the codes involved are distinct for item and serial order aspects of memoranda. Neuropsychological evidence

SC

3.2.

Patients with so-called specific verbal WM deficits typically present with lesions in the

M AN U

inferior frontal and/or posterior superior temporal/ supramarginal area that have been associated with phonological processing (Freedmann & R. Martin, 2001; Friedrich, Glenn, & Marin, 1984; N. Martin & Saffran, 1992; Majerus, Lekeu, Van der Linden, & Salmon, 2001; Strub & Gardner, 1974; Trojano, Stanzione, & Grossi, 1992; Vallar, Basso, & Bottini, 1990;

TE D

see Majerus, 2009, for a review). These patients typically present particularly poor performance for recalling purely phonological material such as nonwords. The interaction between phonological processing and serial order recall is further supported by the fact that

EP

these patients present reduced or absent recency effects, suggesting that phonological codes contribute to serial order coding and recall at least for the most recent serial positions (e.g.,

AC C

Hanten & Martin, 2000). As already noted, Majerus et al. (2015) showed that these patients can present deficits at either item or serial order levels, or both. Although no detailed information about lesion-symptom mappings was reported in that study, most patients showed lesions in the left superior temporal and temporo-parietal areas associated with the dorsal language pathway involved in phonological processing. This study shows that lesions in phonological processing areas can lead to both phonological impairment and verbal WM deficits, but critically, specific item and serial order WM deficits can still be observed, indicating that the phonological codes contributing to serial order coding are not the same as 12

ACCEPTED MANUSCRIPT those contributing to item coding. This question was explored in a particularly direct manner in a recent direct electrical stimulation study in patients during awake neurosurgery. Papagno et al. (2017) stimulated either the left supramarginal gyrus or Broca’s area which are part of the dorsal language pathway involved in phonological processing. They observed that

RI PT

supramarginal gyrus stimulation led to an increased number of order errors while Broca’s area led to an increased number of item errors. The tasks were digit span recall tasks, with substitutions, intrusions and omission errors being considered as item errors and the

SC

proportion of items recalled in a wrong serial position being considered as order errors. The results by Papagno et al. show that the posterior superior temporal cortex and supramarginal

M AN U

gyrus appear to code serial position information not only for purely phonological material such as nonwords but also for lists of familiar words. Critically, for a subset of patients (N=3), Papagno et al. were able to show that stimulation of the left supramarginal gyrus also interfered with serial order memory for visuo-spatial sequences using difficult to verbalize

TE D

Chinese and Bengali characters as memoranda (the patients were Italian speakers). These preliminary results suggest that the left supramarginal gyrus may encode serial order information for both verbal and visuo-spatial information, but also that the coding of item and

Neuroimaging evidence

AC C

3.3.

EP

serial order information within the dorsal language pathway rely on distinct neural substrates.

Similar results have been observed in a recent neuroimaging study also focusing on the dorsal language pathway and making a distinction between the retention of item and serial order information. Unlike previous studies described so far, this study focused on retention and recall of nonwords, therefore maximizing processing at the phonological level. More specifically, Kalm and Norris (2014) looked at the neural substrates associated with the encoding and recall of 3-nonword sequences. They used a multivariate analysis approach in order to determine the neural patterns that characterized the representation of the order of the 13

ACCEPTED MANUSCRIPT nonwords within the list, as well as the neural patterns that characterized the representation of the individual nonword stimuli (considered as item information). They observed that encoding and recall of the order of the nonwords was associated with multivariate patterns in the posterior superior temporal gyrus, the supramarginal gyrus and the inferior frontal gyrus,

RI PT

while encoding and recall of nonword item information was associated with multivariate

patterns in the superior temporal gyrus, the superior temporal sulcus and the supramarginal gyrus. Critically, however, even if both order and item information appeared to be represented

SC

within areas of the dorsal language pathway, the nature of their patterns was best predicted by a positional model considering distinct codes for item and serial order information and their

M AN U

mappings (Burgess & Hitch, 1999, 2006; Brown et al., 2000; Less & Estes, 1981). Chaining models (e.g., Lewandowsky & Murdock, 1989), which do not distinguish between item and serial order codes but which represent serial order information via pairwise item-bindings, were not able to explain the neural patterns that were observed (see also Lashley, 1951, for an

TE D

early discussion of chaining accounts). Finally, we should note that nonword serial order representations were also associated with neural patterns in the right inferior parietal cortex very close to the right intraparietal sulcus areas reported in the previous section as supporting

The specific case of serial order for phonemes within nonwords

AC C

3.4.

EP

serial order coding in WM.

The different studies reviewed so far did not focus on one fundamental aspect of serial order processing, namely the coding of the serial order of phonemes within a nonword. The order of phonemes within nonword strings is at least partially coded via phonological codes. Indeed, when recalling word and nonword lists, it has been observed that nonword recall can lead to better serial order recall than word recall, after correcting for the fact that overall more word items than nonword items are recalled (Fallon, Mak, Tehan, & Daly, 2005; Guérard & SaintAubin, 2012; Jefferies, Frankish, & Lambon Ralph, 2006; Saint-Aubin & Poirier, 2000). 14

ACCEPTED MANUSCRIPT These results suggest that when items can only be processed at the phonological level, they will necessarily be processed in a sequential manner, leading to a particular strong focus on serial order encoding. However, this situation does not necessarily reflect the exclusive operation of a phonological buffer system where temporary representations of the sequential

RI PT

ordering of the nonwords would need to be maintained. As already mentioned in Section 2.1, when considering familiar phoneme sequences such as words, the bindings between the

different constituent phonemes are stored in linguistic long-term memory, via long-term

SC

connections between corresponding phonological and lexico-semantic representations of the word (e.g., Dell, Chang, & Griffin, 1999; Martin & Saffran, 1992; McClelland & Elman,

M AN U

1986). A similar mechanism also applies to nonwords. The serial order of phonemes within nonwords can be supported by statistical, phonotactic knowledge regarding phoneme cooccurrence probabilities stored in the sublexical phonological knowledge base (Vitevitch & Luce, 1999). Nonwords containing phoneme sequences that are more frequent in the

TE D

phonological structure of a given language are typically processed faster and recalled more accurately than nonwords with a less familiar structure (Gathercole et al., 1999; Thorn & Frankish, 2005). Knowledge about transitional probabilities of phoneme co-occurrences can

EP

also be acquired very quickly as shown by phonotactic incidental learning studies. Majerus et al. (2004, 2012) observed that passive listening over several minutes to a continuous sequence

AC C

of phonemes governed by an artificial phonotactic grammar is sufficient to influence subsequent nonword recall: nonwords with phoneme orderings that follow the artificial phonotactic grammar lead to higher recall performance than nonwords containing illegal phoneme ordering. These results show that recall of the ordering of phonemes within a nonword closely interacts with long-term phonological sequence knowledge. The same also applies to highly familiar phonological forms such as words. This principle of access to longterm sequential knowledge for representing serial order information when available can also

15

ACCEPTED MANUSCRIPT be extended to digit sequence recall, a recent study showing that recall success for digit lists can be predicted by the frequency of digit co-occurrences in natural languages (Jones & Macken, 2017). Summary

RI PT

3.5.

In sum, the studies reviewed in this section indicate that the posterior part of the dorsal

language pathway, involved in phonological processing, can support encoding and recall of

SC

serial order information in WM tasks, for both nonword and word lists. Critically, however, even within this pathway, the coding of list-level serial order information needs to be

M AN U

distinguished from the coding of item-level information. Furthermore, preliminary data indicate that the coding of serial order information in the left supramarginal gryus may not be specific to the coding of auditory-verbal information as this region also appears to support coding of visual serial order information. Finally, the processing of serial order information

TE D

via phonological codes, particularly at the intra-item level, closely interacts with long-term phonological knowledge about phoneme co-occurrences and transition probabilities

EP

4. Serial order processing and the phonological buffer: where should we go?

AC C

As already noted, the original phonological buffer concept, as proposed by the Baddeley and Hitch Working memory model, does not make an explicit distinction between item and serial order information, both types of information being represented via phonological codes (Baddeley & Hitch, 1974; Baddeley, 1986). At the neural level, the phonological input buffer has been proposed to be supported by the supramarginal gyrus, a posterior part of the dorsal language pathway. The studies examined in this review allow to derive the following observations: 1) item and serial order retention capacities can dissociate in patients with verbal WM impairment; 2) serial order retention at least partially involves modality-general 16

ACCEPTED MANUSCRIPT mechanisms associated with fronto-parietal cortices; 3) serial order information is in addition processed in the left supramarginal gyrus of the dorsal language pathway associated with phonological processing; 4) serial order retention mechanisms supported by the supramarginal gyrus need to be distinguished from verbal item retention mechanisms. These observations

RI PT

indicate that the codes processed by a phonological buffer system may be more complex than previously thought, especially as regards the processing of serial order information. Challenges for a phonological buffer concept

SC

4.1.

What amendments would be necessary to align a phonological buffer account with the

M AN U

empirical data reviewed here? First, it would be necessary to recognize the existence of multiple routes within a phonological buffer system. Based on their direct electrical stimulation results, Papagno et al. (2017) proposed that item information would be represented via a more anterior loop, linking directly the posterior temporal gyrus to Broca’s

TE D

area via the long segment of the arcuate fasciculus; serial order information would be represented via a more posterior loop linking the posterior temporal gyrus to the supramarginal gyrus and then to Broca’s area via the third segment of the longitudinal

EP

fasciculus (see also Catani, Jones, & ffytche, 2005). In other words, according to the data presented by Papagno et al. (2017), two separate phonological loops would maintain item and

AC C

serial order information. However, the question that arises here is whether the serial order loop can still be called a phonological loop as preliminary data reported by Papagno et al. indicate that stimulation of the supramarginal gyrus also interrupts the coding of serial order for difficult-to-verbalize visual information. Second, the phonological buffer hypothesis needs to account for the influence of linguistic knowledge on the retention of both item and serial order information. In a revised model, Baddeley, Gathercole, and Papagno (1998) proposed to connect the phonological store to phonological long-term memory knowledge. Baddeley (2000) also acknowledged 17

ACCEPTED MANUSCRIPT interactions with semantic linguistic knowledge. As shown by the data reviewed here, these interactions operate at many different levels, in a very fast and automatic manner. A phonological buffer hypothesis needs to consider direct interactions with sublexical phonotactic knowledge for the representation of phoneme sequence information, with lexical

RI PT

knowledge for binding familiar phoneme strings into word representations, and with semantic knowledge regrouping words into semantic chunks. The latter aspect is particularly relevant for the maintenance of sentence-level serial order information. Sentence recall typically leads

SC

to very large WM spans (up to 16 words, Baddeley, Vallar, & Wilson, 1987). This is due to the fact that a vast amount of linguistic knowledge from different levels can be used to

M AN U

represent not only (word) items, but also their serial order. A meaningful sentence will generate a conceptual representation of the entire sentence determining the hierarchical relations between the different elements of the sentence (R. Martin & Romani, 1994). In addition, syntactic knowledge about basic grammatical structures (e.g., a nouns is generally

TE D

followed by a verb which is followed by an object; Bock & Levelt, 1994; Garrett, 1980) will determine the succession of the different words and will constrain the serial positions in which the words can occur. In the light of these elements, one may consider that linguistic

EP

accounts of verbal WM and their variants, considering that the content of WM directly involves the temporary activation of long-term memory representations (e.g., Acheson &

AC C

MacDonald, 2009; Burgess & Hitch, 1999, 2006; Cowan, 1995; Gupta & MacWhinney, 1997; N. Martin & Saffran, 1992; Majerus et al., 2013; Majerus & D’Argembeau, 2011), represent a more economical theoretical solution than the proposal of multiple buffer loops interacting with multiple levels of linguistic representations. Third, the aspect that is probably most difficult to reconcile with a purely phonological buffer account of serial order processing is the observation of domain-general serial order coding processes. The behavioral and neuroimaging studies reviewed here suggest the 18

ACCEPTED MANUSCRIPT existence of domain-general serial ordering processes involving spatial and/or temporal positional codes, shared between verbal, visuo-spatial and musical domains, and supported by frontal and intraparietal cortices distinct from the dorsal language pathway considered to support a phonological buffer.

of item and serial order information in WM

RI PT

4.2. An integrated view of the cognitive and neural processes supporting the representation

SC

In Figure 1, I present an integrated view of the multiple cognitive and neural processes

involved in the retention of item and serial order information, as identified in this review.

M AN U

First, regions in the middle and inferior temporal gyri (green colour in Figure 1) associated with lexico-semantic knowledge (Binder, Desai, Graves, & Conant, 2009) have been shown to be actively recruited in WM tasks requiring the temporary maintenance of lexico-semantic information (Fiebach, Rissman & D’Esposito, 2006). Gray matter loss in these regions leads

TE D

to degradation of semantic knowledge (semantic dementia) and particularly poor item recall for lexico-semantic information in verbal WM tasks (Majerus et al., 2007; Papagno, Vernice, & Cecchetto, 2013; Rosen et al., 2002). Furthermore, patients with lesions/neuronal loss in

EP

these areas will also be impacted at the level of intra-item serial order recall, by showing an increase of phoneme ordering errors when repeating words due to the loss of lexico-semantic

AC C

bindings that ‘glue’ together the phonemes that define a word form (Patterson, Graham, & Hodges, 1994). Patients with loss of lexico-semantic information also tend to commit, in serial recall tasks, more errors for start-of-list items by showing a reduced primacy effect, initial items being considered to depend more on lexico-semantic activation than end-of-list items (Martin & Saffran, 1992; R. Martin et al., 1994). Regions in the superior temporal gyri (red colour in Figure 1) represent sublexical phonological knowledge about phonemes and their transition probabilities (Binder et al., 1999; Majerus et al., 2005; Scott, Catrin Blank, Rosen, & Wise, 2000), and support temporary 19

ACCEPTED MANUSCRIPT retention of item and list-level serial order information for nonword sequences, but, as we have seen, via distinct codes (Kalm & Norris, 2014). Patients with lesions in these areas will show reduced recency effects in immediate serial recall tasks, later items being considered to depend more on phonological activation than start-of-list items (Martin & Saffran, 1992; R.

RI PT

Martin & Lesch, 1996). The supramarginal gyrus has also been shown by both the

neuroimaging and electrical stimulation studies reviewed here to be involved in the coding of list-level serial order information. Kalm and Norris (2014) (see also Majerus et al., 2010)

SC

observed that the supramarginal gyrus can also support the temporary representation of item information while the electrical stimulation study by Papagno et al. (2017) suggests a stronger

M AN U

role of this region for serial order maintenance. Although the supramarginal gyrus is part of a posterior dorsal language pathway associated with phonological processes, the precise nature of supramarginal gyrus involvement in serial order processing currently still raises many questions (see Section 5). Importantly, the superior temporal and supramarginal components

phonological buffer.

TE D

of Figure 1 are those that overlap most strongly with the neural substrates proposed for a

Finally, regions outside the language processing system (blue colour in Figure 1) have

EP

been shown to support the temporary representation of list-level serial order as well as item

AC C

information, but again in a dissociative manner. Regions of the dorsal attention network, involving the intraparietal sulcus and the superior frontal gyrus have been shown to be highly sensitive to WM load (for both item and serial order information) (Ravizza, Delgado, Chein, Becker, & Fiez, 2004; Todd, Fougnie, & Marois, 2005) and have been proposed to reflect the attentional control demands associated with the maintenance of information in WM (Cowan et al., 2011; Majerus et al., 2012, 2016; Majerus, Péters, Bouffier, Cowan, & Phillips, 2017; Riggall & Postle, 2012). Importantly, a subset of these regions, the right intraparietal sulcus and the left dorsolateral prefrontal cortex, is associated more specifically with serial order 20

ACCEPTED MANUSCRIPT processing as opposed to item maintenance, and this in a domain-general manner (Henson et al., 2000; Majerus et al., 2006, 2010). < INSERT FIGURE 1 ABOUT HERE > Summary

RI PT

4.3.

In sum, multiple cognitive and neural processes are involved in the temporary retention of item and serial order information. Some of these, such as the supamarginal gyrus – inferior

SC

frontal gyrus loop, may have a more specific phonological buffering function, but even in this

of item and serial order information.

5. Conclusions

M AN U

phonological buffer system, a critical distinction needs to be made between the representation

TE D

This critical review addressed the role of a potential phonological buffer system in the temporary maintenance of verbal item and serial order information. The reviewed studies support the notion of phonological codes for supporting the storage of both types of

EP

information in WM, but they also show that the phonological codes for item-level and serialorder levels of representations are distinct. Furthermore, the studies show that serial order

AC C

information is not only represented via phonological codes, but additional, possibly domaingeneral processes further support the coding of serial order information, raising the possibility that serial order information in verbal WM tasks is coded via multiple mechanisms at the same time.

A first important question raised by this this review is the type of codes that support the representation of serial order information at the phonological level. Kalm and Norris (2014) argued that these codes may be positional. These codes could theoretically be a subset

21

ACCEPTED MANUSCRIPT of phonotactic levels of representation (see Figure 1). Distributional phonotactic knowledge allows to associate a given set of phonemes to a given position in a word, according to statistical distributional rules the language system has learned (Vitevitch & Luce, 1999). This type of phoneme-position associations is very similar to item-position bindings in an

RI PT

immediate serial recall task for word items. In other words, the phonological system does not only have the capacity to represent individual phonological units, but also to represent the temporal/positional dimensions of these phonological units (Leonard, Bouchard, & Chang,

SC

2013). These positional coding properties of the phonological system could be used to

temporarily represent serial order information about phonemes in nonwords also when no

M AN U

long-term phonotactic knowledge is available or to represent list-level serial order information for lexical phonological units such as words. However, these temporary serial order representations, based on the generation of novel phoneme-position bindings in the sublexical phonological system, and possibly also based on additional bindings between the sublexical

TE D

and the lexico-semantic systems due to interactive activation (Savill et al., 2018), would probably be unstable and subject to massive degradation, either via trace decay or interference or both. This is in line with repetition performance for novel phonological sequences which is

EP

typically quite poor even in subjects with no cognitive impairment (e.g., Gathercole et al.,

AC C

1999; Majerus & Van der Linden, 2003). At the same time, preliminary data by Papagno et al. (2017) point towards a slightly

more complex interpretation as regards serial order processing in presumably phonological processing regions such as the supramarginal gyrus as they showed that stimulation of the left supramarginal gyrus led to errors in both verbal and visual WM tasks (this finding his however based on three patients only and needs to be replicated). This finding also questions the role of the processes supported by the supramarginal gyrus more generally. In addition to phonological processing, this region has been suggested to support, at a broader level, 22

ACCEPTED MANUSCRIPT sequencing and relational processes, together with the inferior frontal gyrus region (see Figure 1). Relational processing and sequencing is an important property of many verbal tasks such as unfamiliar word reading, sentence planning, or speech output (Baddeley, 2012; Boylan, Trueswell, & Thompson-Schill, 2017; Guenther & Hickok, 2015). It has also been shown that

RI PT

the inferior frontal/supramarginal gyrus regions are particularly involved when temporal (i.e., sequential) rather than spectral changes in the speech signal have to be identified (Zaehle, Geiser, Alter, Jäncke, & Meyer, 2008). These relational and sequencing abilities are also

SC

required in non-verbal tasks each time a set of stimuli or internally generated thoughts have to be organized in a sequential manner; these abilities are impaired in patients with fronto-

M AN U

striatal damage (Robinson, Spooner, & Harrison, 2015). This more general property of relational/sequential processing of the frontal/supramarginal gyrus interface may explain its involvement in both verbal and non-verbal serial order WM. At the same time, the data presented by Kalm and Norris (2014) suggest that it is unlikely that item-by-item relational

TE D

processing provides sufficiently precise codes for representing the exact serial position of items in immediate serial recall tasks.

This review further indicates that serial order processing involves regions outside the

EP

language networks, and this more specifically in the intraparietal sulcus and the dorsolateral

AC C

prefrontal cortex. It has been suggested that the right intraparietal sulcus reflects a more specific form of spatial attention, and several studies have shown that serial order information in verbal WM is spontaneously coded according to a left-to-right spatial representation (Van Dijck & Fias, 2011). The left dorsolateral prefrontal cortex areas may support time-based representations that have also been shown to support serial order coding (Henson et al., 2000; Brown et al., 2000; Brown, Neath, & Chater, 2007; Hartley, Hurlstone, & Hitch, 2016). Currently, we do however not know whether these temporal and spatial processes supported by fronto-parietal cortices are ancillary serial order coding processes or whether they are core 23

ACCEPTED MANUSCRIPT serial order coding processes (Abrahamse, van Dijck, Majerus & Fias, 2014). It still remains to be shown whether patients with specific lesions in these areas will also show specific serial order impairment in verbal WM tasks. A study by Ravizza, Behrmann and Fiez (2005) suggests that this may indeed be the case. They described a patient with right parietal damage

RI PT

and who showed impairment in verbal and visuo-spatial WM tasks; the patient’s difficulties appeared specifically when spatial attentional processes could also determine WM task

performance. However, this study did not directly assess serial order retention abilities. On

SC

the other hand, patients with right parietal lesions and hemispatial neglect present with visuospatial WM deficits but their immediate serial recall performance for auditory-verbal

M AN U

information can be preserved (Piccardi et al., 2016). Again, we are critically missing studies that would have directly targeted serial order retention capacities in both auditory-verbal and visual modalities in these patients.

Overall, the data reviewed here suggest that multiple codes may contribute to the

TE D

representation of arbitrary serial order information whose fleeting nature renders its temporary storage very challenging. The simultaneous intervention of phonological, spatial and temporal serial order coding processes may enhance the stability of temporary serial order

EP

representations by providing multi-modal codes (see also De Belder, Van Dijck, Cappelletti,

AC C

& Fias, 2017; Fischer-Baum & McCloskey, 2015). Given the inherent difficulty of encoding arbitrary serial order information, any internal reference frame, containing a cohesive structure (such as the left-right dimension of space), may support the serial order coding in WM by providing a long-term memory structure that can organize the incoming items as a function of their order of occurrence. In this sense, serial order information may be stored in several formats and systems at the same time. It should be noted here that the question of the nature of the codes used for storing serial order information is far from being resolved, and

24

ACCEPTED MANUSCRIPT elucidating the mechanisms supporting serial order coding in verbal (and visuo-spatial) WM remains a major challenge (e.g., Hurlstone et al., 2014; Majerus & Attout, 2018). To conclude, this review supports a phonological buffer function of the posterior superior temporal / supramarginal gyrus. However, the dissociations for the representation of

RI PT

item and serial order information within this and several other regions of the dorsal language pathway proposed to support the phonological loop rule out a single code phonological buffer account. Also, the codes used for representing serial order information within this loop may

SC

not be purely phonological. Furthermore, temporary maintenance of item and serial order

M AN U

information is not restricted to this phonological loop system and involves access to linguistic knowledge structures within the wider language system as well as to extra-linguistic processes. The clarification of the codes and mechanisms used for the representation of serial order information remains a major challenge for a phonological buffer account, as well as for

AC C

EP

TE D

the concept of WM more generally.

Acknowledgments This work was supported by the Fund for Scientific Research FNRS and the government of the French-speaking community of Belgium (grants EOS 30446199 and FSR-S-SH-17/04).

25

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

26

ACCEPTED MANUSCRIPT Reference list Abrahamse, E., van Dijck, J. P., Majerus, S., & Fias, W. (2014). Finding the answer in space: the mental whiteboard hypothesis on serial order in working memory. Frontiers in Human Neuroscience, 8, 932. doi:10.3389/fnhum.2014.00932

RI PT

Acheson, D. J., & MacDonald, M. C. (2009). Verbal Working Memory and Language

Production: Common Approaches to the Serial Ordering of Verbal Information. Psychological Bulletin, 135, 50-68.

SC

Baddeley, A. D. (1966). Short-term memory for word sequences as a function of acoustic,

M AN U

semantic and formal similarity. Quarterly Journal of Experimental Psychology, 18, 362– 365.

Baddeley, A.D. (1986). Working memory: Oxford, England UK: Clarendon Press/Oxford University Press.

Baddeley, A. D. (2012). Working memory: theories, models, and controversies. Annual

TE D

Review of Psychology, 63, 1-29. doi:10.1146/annurev-psych-120710-100422 Baddeley, A. D. (2000). The episodic buffer: a new component of working memory? Trends in Cognitive Sciences, 4, 417-423.

EP

Baddeley, A. D., Gathercole, S., & Papagno, C. (1998). The phonological loop as a language

AC C

learning device. Psychological Review, 105, 158-173. Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 47-90). San Diego, CA: Academic Press.

Baddeley, A. D., Vallar, G., & Wilson, B. A. (1987). Sentence comprehension and phonological memory: some neuropsychological evidence. In M. Coltheart (Ed.), Attention and Performance Vol. XII: The Psychology of Reading (pp. 509-529). Hillsdale, NJ: Lawrence Erlbaum. Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. I. (2009). Where Is the Semantic 27

ACCEPTED MANUSCRIPT System? A Critical Review and Meta-Analysis of 120 Functional Neuroimaging Studies. Cerebral Cortex, 19, 2767-2796. Binder, J. R., Frost, J. A., Hammeke, T. A., Bellgowan, P. S. F., Rao, S. M., & Cox, R. W.

study. Journal of Cognitive Neuroscience, 11, 80-93.

RI PT

(1999). Conceptual processing during the conscious resting state: a functional MRI

Bjork, E. L., & Healy, A. F. (1974). Short-term order and item retention. Journal of Verbal Learning and Verbal Behavior, 13, 80–97.

SC

Bock, J. K., & Levelt, W. J. (1994). Language production: Grammatical encoding. In M. A. Gernsbacher (Ed.), Handbook of Psycholinguistics (pp. 945-984). Sand Diego, CA:

M AN U

Academic Press.

Boylan, C., Trueswell, J. C., & Thompson-Schill, S. L. (2017). Relational vs. attributive interpretation of nominal compounds differentially engages angular gyrus and anterior temporal lobe. Brain and Language, 169, 8-21. doi:10.1016/j.bandl.2017.01.008

TE D

Brown, G. D., Neath, I., & Chater, N. (2007). A temporal ratio model of memory. Psychological Review, 114, 539-576. doi:10.1037/0033-295X.114.3.539 Brown, G. D. A., Preece, T., & Hulme, C. (2000). Oscillator-based memory for serial order.

EP

Psychological Review, 107, 127-181.

Burgess, N., & Hitch, G. J. (1999). Memory for serial order: A network model of the

AC C

phonological loop and its timing. Psychological Review, 106, 551-581.

Burgess, N., & Hitch, G. J. (2006). A revised model of short-term memory and long-term learning of verbal sequences. Journal of Memory and Language, 55, 627-652.

Camos, V., Lagner, P., & Loaiza, V. M. (2017). Maintenance of item and order information in verbal working memory. Memory, 25, 953-968. doi:10.1080/09658211.2016.1237654 Catani, M., Jones, D. K., & ffytche, D. H. (2005). Perisylvian language networks of the human brain. Annals of Neurology, 57, 8-16. doi:10.1002/ana.20319

28

ACCEPTED MANUSCRIPT Cowan, N. (1995). Attention and memory: An integrated framework. New York: Oxford University Press. Cowan, N., Li, D., Moffitt, A., Becker, T. M., Martin, E. A., Saults, J. S., & Christ, S. E.

Neuroscience, 23, 2852-2863.

RI PT

(2011). A neural region of abstract working memory. Journal of Cognitive

Crowder, R. G. (1979). Similarity and order in memory. Psychology of Learning and Motivation - Advances in Research and Theory, 13, 319–353.

SC

https://doi.org/10.1016/S0079-7421(08)60086-9

De Belder, M., van Dijck, J. P., Cappelletti, M., & Fias, W. (2017). How serially organized

M AN U

working memory information interacts with timing. Psychological Research, 81, 1255– 1263. https://doi.org/10.1007/s00426-016-0816-8

Dell, G. S., Chang, F., & Griffin, Z. M. (1999). Connectionist models of language production : Lexical access and grammatical encoding. Cognitive Science, 23, 517–542.

TE D

https://doi.org/http://dx.doi.org/10.1016/S0364-0213(99)00014-2 Depoorter, A., & Vandierendonck, A. (2009). Evidence for modality-independent order coding in working memory. Quarterly Journal of Experimental Psychology, 62, 531-

EP

549. doi:10.1080/17470210801995002

Deutsch, D. (1980). The processing of structured and unstructured tonal sequences.

AC C

Perception and Psychophysics, 28, 381-389.

Fallon, A. B., Mak, E., Tehan, G., & Daly, C. (2005). Lexicality and phonological similarity: a challenge for the retrieval-based account of serial recall? Memory, 13, 349-356.

Fiebach, C. J., Rissman, J., & D'Esposito, M. (2006). Modulation of inferotemporal cortex during verbal working memory maintenance. Neuron, 51, 251-261. Fischer-Baum, S., & McCloskey, M. (2015). Representation of item position in immediate serial recall: evidence from intrusion errors. Journal of Experimental Psychology:

29

ACCEPTED MANUSCRIPT Learning, Memory, and Cognition, 41, 1426-1446. doi:10.1037/xlm0000102 Freedman, M. L., & Martin, R. C. (2001). Dissociable components of short-term memory and their relation to long-term learning. Cognitive Neuropsychology, 18, 193-226.

performance (pp. 263-271). New York: Academic Press.

RI PT

Garrett, M. F. (1980). The limits of accommodation. In V. Fromkin (Ed.), Errors in linguistic

Gathercole, S. E. (1995). Is nonword repetition a test of phonological memory or long-term knowledge? It all depends on the nonwords. Memory and Cognition, 23, 83-94.

SC

Gathercole, S. E., Frankish, C. R., Pickering, S. J., & Peaker, S. (1999). Phonotactic

influences on short-term memory. Journal of Experimental Psychology: Human

M AN U

Learning and Memory, 25, 84-95.

Gathercole, S. E., Pickering, S. J., Hall, M., & Peaker, S. (2001). Dissociable lexical and phonological influences on serial recognition and serial recall. The Quarterly Journal of Experimental Psychology, 54A, 1-30.

TE D

Gathercole, S. E., Willis, C. S., Baddeley, A. D., & Emslie, H. (1994). The children's test of nonword repetition: A test of phonological working memory. Memory, 2, 103-127. Ginsburg, V., & Gevers, W. (2015). Spatial coding of ordinal information in short- and long-

EP

term memory. Frontiers in Human Neuroscience, 9, 8.doi:10.3389/fnhum.2015.00008 Ginsburg, V., van Dijck, J.-P., Previtali, P., Fias, W., & Gevers, W. (2014). The impact of

AC C

verbal working memory on number-space associations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 976-986. doi:10.1037/a0036378 24707784

Gorin, S., Kowialiewski, B., & Majerus, S. (2016). Domain-Generality of Timing-Based Serial Order Processes in Short-Term Memory: New Insights from Musical and Verbal Domains. PLoS ONE, 11, e0168699. doi:10.1371/journal.pone.0168699

30

ACCEPTED MANUSCRIPT Gorin, S., Mengal, P., & Majerus, S. (2017). Temporal grouping effects in musical short-term memory. Memory, pp. 1–13. https://doi.org/10.1080/09658211.2017.1414848 Guenther, F. H., & Hickok, G. (2015). Role of the auditory system in speech production. Handbook of Clinical Neurology, 129, 161-175. doi:10.1016/B978-0-444-62630-

RI PT

1.00009-3

Guerard, K., & Saint-Aubin, J. (2012). Assessing the effect of lexical variables in backward recall. Journal of Experimental Psychology: Learning, Memory, & Cognition, 38, 312-

SC

324. doi:10.1037/a0025481

Gupta, P., & MacWhinney, B. (1997). Vocabulary acquisition and verbal short-term memory:

M AN U

computational and neural bases. Brain and Language, 59, 267-333. Hartley, T., Hurlstone, M. J., & Hitch, G. J. (2016). Effects of rhythm on memory for spoken sequences: A model and tests of its stimulus-driven mechanism. Cognitive Psychology, 87, 135-178. doi:10.1016/j.cogpsych.2016.05.001

TE D

Henson, R., Hartley, T., Burgess, N., Hitch, G., & Flude, B. (2003). Selective interference with verbal short-term memory for serial order information: A new paradigm and tests of a timing-signal hypothesis. Quarterly Journal of Experimental Psychology, 56A,

EP

1307-1334.

Henson, R. N. A., Burgess, N., & Frith, C. D. (2000). Recoding, storage, rehearsal, and

AC C

grouping in verbal short-term memory: an fMRI study. Neuropsychologia, 38, 426440.

Hulme, C., Maughan, S., & Brown, G. D. (1991). Memory for familiar and unfamiliar words: Evidence for a long-term memory contribution to short-term memory span. Journal of Memory and Language, 30, 685-701. Hurlstone, M. J., Hitch, G. J., & Baddeley, A. D. (2014). Memory for serial order across domains: An overview of the literature and directions for future research.

31

ACCEPTED MANUSCRIPT Psychological Bulletin, 140, 339-373. doi:10.1037/a0034221 24079725 Johnson, A. J., & Miles, C. (2007). Serial position functions for recognition of olfactory stimuli. Quarterly Journal of Experimental Psycho, 60, 1347-1355. doi:10.1080/17470210701515694

RI PT

Johnson, A. J., Shaw, J., & Miles, C. (2016). Tactile order memory: evidence for sequence learning phenomena found with other stimulus types. Journal of Cognitive Psychology, 28, 718-725.

SC

Jones, G., & Macken, B. (2017). Long-term associative learning predicts verbal short-term memory performance. Memory and Cognition. doi:10.3758/s13421-017-0759-3

M AN U

Kalm, K., & Norris, D. (2014). The representation of order information in auditory-verbal short-term memory. The Journal of Neuroscience, 34, 6879-6886. doi:10.1523/JNEUROSCI.4104-13.2014 24828642

Lashley, K. (1951). The problem of serial order in behavior. In L. Jeffress (Ed.), Cerebral

TE D

Mechanisms in Behavior (pp. 112-136). New York: Wiley. Lee, C. L., & Estes, W. K. (1981). Item and order information in short-term memory: Evidence for multilevel perturbation processes. Journal of Experimental Psychology:

EP

Human Learning and Memory, 7, 149-169. Lewandowsky, S., & Murdock, B. B. (1989). Memory for serial order. Psychological Review,

AC C

96, 25-57.

Lian, A., Karlsen, P. J., & Eriksen, T. B. (2004). Opposing effects of phonological similarity on item and order memory of words and nonwords in the serial recall task. Memory, 12, 314–337. https://doi.org/10.1080/096582103440000426 Majerus, S. (2009). Verbal short-term memory and temporary activation of language representations: the importance of distinguishing item and order information. In A. S. Thorn & M. Page (Eds.), Interactions between short-term and long-term memory in

32

ACCEPTED MANUSCRIPT the verbal domain (pp. 244-276). Hove, UK: Psychology Press. Majerus, S. (2013). Language repetition and short-term memory: An integrative framework. Frontiers in Human Neuroscience, 7, 357-doi: 310.3389/fnhum.2013.00357. Majerus, S., & Attout, L. (2018). Working Memory for Serial Order and Numerical

function in numerical cognition (in press). Elsevier.

RI PT

Cognition: What Kind of Association? In A. Henik & W. Fias (Eds.), Heterogeneity of

Majerus, S., Attout, L., Artielle, M. A., & Van der Kaa, M. A. (2015). The heterogeneity of

doi:10.1016/j.neuropsychologia.2015.08.010

SC

verbal short-term memory impairment in aphasia. Neuropsychologia, 77, 165-176.

M AN U

Majerus, S., Attout, L., D'Argembeau, A., Degueldre, C., Fias, W., Maquet, P., . . . Balteau, E. (2012). Attention Supports Verbal Short-Term Memory via Competition between Dorsal and Ventral Attention Networks. Cerebral Cortex, 22, 1086-1097. Majerus, S., Cowan, N., Peters, F., Van Calster, L., Phillips, C., & Schrouff, J. (2016). Cross-

TE D

Modal Decoding of Neural Patterns Associated with Working Memory: Evidence for Attention-Based Accounts of Working Memory. Cerebral Cortex, 26, 166-179. doi:10.1093/cercor/bhu189

EP

Majerus, S., & D'Argembeau, A. (2011). Verbal short-term memory reflects the organization of long-term memory: Further evidence from short-term memory for emotional words.

AC C

Journal of Memory and Language, 64, 181-197.

Majerus, S., D'Argembeau, A., Martinez, T., Belayachi, S., Van der Linden, M., Collette, F., . . . Maquet, P. (2010). The commonality of neural networks for verbal and visual shortterm memory. Journal of Cognitive Neuroscience, 22, 2570-2593. Majerus, S., Lekeu, F., Van der Linden, M., & Salmon, E. (2001). Deep dysphasia: Further evidence on the relationship between phonological short-term memory and language processing impairments. Cognitive Neuropsychology, 18, 385-410.

33

ACCEPTED MANUSCRIPT Majerus, S., Martinez Perez, T., & Oberauer, K. (2012). Two distinct origins of long-term learning effects in verbal short-term memory. Journal of Memory and Language, 66, 38-51. Majerus, S., Norris, D., & Patterson, K. (2007). What do patients with semantic dementia

RI PT

remember in verbal short-term memory? Sounds and order but not words. Cognitive Neuropsychology, 24, 131-151.

Majerus, S., Peters, F., Bouffier, M., Cowan, N., & Phillips, C. (2018). The Dorsal Attention

SC

Network Reflects Both Encoding Load and Top-down Control during Working Memory. Journal of Cognitive Neuroscience, 30, 144–159. https://doi.org/10.1162/jocn_a_01195

M AN U

Majerus, S., Poncelet, M., Van der Linden, M., Albouy, G., Salmon, E., Sterpenich, V., . . . Maquet, P. (2006). The left intraparietal sulcus and verbal short-term memory: Focus of attention or serial order? NeuroImage, 32, 880-891.

Majerus, S., & Van der Linden, M. (2003). The development of long-term memory effects on

TE D

verbal short-term memory : A replication study. British Journal of Developmental Psychology, 21, 303-310.

Majerus, S., Van der Linden, M., Collette, F., Laureys, S., Poncelet, M., Degueldre, C., . . .

EP

Salmon, E. (2005). Modulation of brain activity during phonological familiarization. Brain and Language, 92, 320-331.

AC C

Majerus, S., Van der Linden, M., Mulder, L., Meulemans, T., & Peters, F. (2004). Verbal short-term memory reflects the sublexical organization of the phonological language network: Evidence from an incidental phonotactic learning paradigm. Journal of Memory and Language, 51, 297-306. Marshuetz, C., Reuter-Lorenz, P. A., Smith, -. M.-C., Jonides, J., & Noll, D. C. (2006). Working memory for order and the parietal cortex: an event-related functional magnetic resonance imaging study. Neuroscience, 139, 311-316.

34

ACCEPTED MANUSCRIPT Marshuetz, C., Smith, E. E., Jonides, J., DeGutis, J., & Chenevert, T. L. (2000). Order information in working memory: fMRI evidence for parietal and prefrontal mechanisms. Journal of Cognitive Neuroscience, 12(S2), 130-144. Martin, N., & Saffran, E. M. (1992). A computational account of deep dysphasia: Evidence

RI PT

from a single case study. Brain and Language, 43, 240-274.

Martin, R. C., & Lesch, M. F. (1996). Associations and dissociations between language

impairment and list recall: Implications for models of STM. In S. E. Gathercole (Ed.),

SC

Models of short-term memory (pp. 149–178). East Sussex, UK: Hove.

Martin, R. C., & Romani, C. (1994). Verbal working memory and sentence comprehension: A

M AN U

multiple-components view. Neuropsychology, 8, 506-523.

Martinez Perez, T., Poncelet, M., Salmon, E., & Majerus, S. (2015). Functional alterations in order short-term memory networks in adults with dyslexia. Developmental Neuropsychology, 40, 407-429. doi:10.1080/87565641.2016.1153098

TE D

Mathias, B., Pfordresher, P. Q., & Palmer, C. (2014). Context and meter enhance long-range planning in music performance. Frontiers in Human Neuroscience, 8, 1040. doi:10.3389/fnhum.2014.01040

EP

Mathy, F., & Feldman, J. (2012). What's magic about magic numbers? Chunking and data compression in short-term memory. Cognition, 122, 346-362.

AC C

doi:10.1016/j.cognition.2011.11.003

McClelland, J. L., & Elman, J. L. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 1–86.

Murdock, B. B., & Vom Saal, W. (1967). Transpositions in short-term memory. Journal of Experimental Psychology, 74, 137–143. https://doi.org/10.1037/h0024507 Nairne, J. S., & Kelley, M. R. (2004). Separating item and order information through process dissociation. Journal of Memory and Language, 50, 113-133.

35

ACCEPTED MANUSCRIPT Papagno, C., Comi, A., Riva, M., Bizzi, A., Vernice, M., Casarotti, A., . . . Bello, L. (2017). Mapping the brain network of the phonological loop. Hum Brain Mapp, 38, 30113024. doi:10.1002/hbm.23569 Papagno, C., Vernice, M., & Cecchetto, C. (2013). Phonology without semantics? Good

RI PT

enough for verbal short-term memory. Evidence from a patient with semantic dementia. Cortex, 49, 626-636. doi:10.1016/j.cortex.2012.04.015

SC

Patterson, K. E., Graham, N., & Hodges, J. R. (1994). The impact of semantic memory loss on phonological representations. Journal of Cognitive Neuroscience, 6, 57–69. Pfordresher, P. Q., Palmer, C., & Jungers, M. K. (2007). Speed, accuracy, and serial order in sequence production. Cognitive Science, 31, 63-98. doi:10.1080/03640210709336985

M AN U

Piccardi, L., Matano, A., D'Antuono, G., Marin, D., Ciurli, P., Incoccia, C., . . . Guariglia, P. (2016). Persistence of Gender Related-Effects on Visuo-Spatial and Verbal Working Memory in Right Brain-Damaged Patients. Frontiers in Behavioural Neuroscie,ce, 10, 139. doi:10.3389/fnbeh.2016.00139

TE D

Ravizza, S. M., Behrmann, M., & Fiez, J. A. (2005). Right parietal contributions to verbal working memory: Spatial or executive? Neuropsychologia, 43, 2057-2067. Ravizza, S. M., Delgado, M. R., Chein, J. M., Becker, J. T., & Fiez, J. A. (2004). Functional

EP

dissociations within the inferior parietal cortex in verbal working memory. NeuroImage, 22, 562-573.

AC C

Riggall, A. C., & Postle, B. R. (2012). The Relationship between Working Memory Storage and Elevated Activity as Measured with Functional Magnetic Resonance Imaging. Journal of Neuroscience, 32, 12990-12998.

Robinson, G. A., Spooner, D., & Harrison, W. J. (2015). Frontal dynamic aphasia in progressive supranuclear palsy: Distinguishing between generation and fluent sequencing of novel thoughts. Neuropsychologia, 77, 62-75. doi:10.1016/j.neuropsychologia.2015.08.001

36

ACCEPTED MANUSCRIPT Rosen, H. J., Kramer, J. H., Gorno-Tempini, M. L., Schuff, N., Weiner, M., & Miller, B. L. (2002). Patterns of cerebral atrophy in primary progressive aphasia. American Journal of Geriatric Psychiatry, 10, 89-97. Saint-Aubin, J., & Poirier, M. (1999). Semantic similarity and immediate serial recall: Is there

RI PT

a detrimental effect on order information? The Quarterly Journal of Experimental Psychology, 52A, 367–394.

Saint-Aubin, J., & Poirier, M. (2000). Immediate serial recall of words and nonwords: Tests

SC

of a retrieval-based hypothesis. Psychonomic Bulletin and Review, 7, 332-340.

Savill, N., Ellis, E., Brooke, E., Koa, T., Ferguson, S., Rojas-Rodriguez, E., Arnold, D.,

M AN U

Smallwood, J., & Jefferies, E. (2018). Keeping it together: Semantic coherence stabilizes phonological sequences in short-term memory. Memory & Cognition, 46, 426-437.

Scott, S. K., Catrin Blank, C., Rosen, S., & Wise, R. J. S. (2000). Identification of a pathway

TE D

for intelligible speech in the left temporal lobe. Brain, 123, 2400-2406. Shallice, T., & Warrington, E. K. (1977). Auditory-verbal short-term memory impairment and conduction aphasia. Brain and Language, 4, 479-491.

EP

Strub, R. L., & Gardner, H. (1974). The repetition defect in conduction aphasia: Mnestic or Linguistic? Brain and Language, 1, 241-255.

AC C

Thorn, A. S., & Frankish, C. R. (2005). Long-Term Knowledge Effects on Serial Recall of Nonwords Are Not Exclusively Lexical. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 729-735.

Todd, J. J., Fougnie, D., & Marois, R. (2005). Visual short-term memory load suppresses temporo-parietal junction activity and induces inattentional blindness. Psychological Science, 16, 965-972. Trojano, L., Stanzione, M., & Grossi, D. (1992). Short-term memory and verbal learning with

37

ACCEPTED MANUSCRIPT auditory phonological coding defect: a neuropsychological case study. Brain and Cognition, 18, 12-33. Vallar, G., Basso, A., & Bottini, G. (1990). Phonological processing and sentence comprehension: A neuropsychological case study. In G. Vallar & T. Shallice (Eds.),

NY: Cambridge University Press.

RI PT

Neuropsychological impairments of short-term memory. (pp. 448-476). New York,

Vallar, G., Di Betta, A. M., & Silveri, M. C. (1997). The phonological short-term store-

SC

rehearsal system: Patterns of impairment and neural correlates. Neuropsychologia, 35, 795-812.

M AN U

Vallar, G., & Papagno, C. (2002). Neuropsychological Impairments of verbal short-term memory. In A. D. Baddeley, M. D. Kopelman, & B. A. Wilson (Eds.), The handbook of memory disorders 2nd edition (pp. 249-270). Chichester: John Wiley & Sons Ltd. van Dijck, J. P., & Fias, W. (2011). A working memory account for spatial-numerical

TE D

associations. Cognition., 119, 114-119.

van Dijck, J.-P., Abrahamse, E., Majerus, S., & Fias, W. (2013). Spatial attention interacts with serial-order retrieval from verbal working memory. Psychological Science, 24,

EP

1854-1859.

Vandierendonck, A. (2016). Modality independence of order coding in working memory:

AC C

Evidence from cross-modal order interference at recall. Quarterly Journal of Experimental Psychology, 69, 161-179. doi:10.1080/17470218.2015.1032987

Vitevitch, M. S., & Luce, P. A. (1999). Probabilistic phonotactics and neighborhood activation in spoken word recognition. Journal of Memory and Language, 40, 374408. Walker, I., & Hulme, C. (1999). Concrete words are easier to recall than abstract words: Evidence for a semantic contribution to short-term serial recall. Journal of

38

ACCEPTED MANUSCRIPT Experimental Psychology: Learning, Memory, and Cognition, 25, 1256-1271. Warrington, E. K. (1981). Neuropsychological evidence for multiple memory systems. Acta Neurologica Scandinavica, 64, 13-19.

the long-term reconstruction of order. Memory, 2, 275–294.

RI PT

Whiteman, H. L., Nairne, J. S., & Serra, M. (1994). Recognition and recall-like processes in

Zaehle, T., Geiser, E., Alter, K., Jäncke, L., & Meyer, M. (2008). Segmental processing in the human auditory dorsal stream. Brain Research, 1220, 179-190.

AC C

EP

TE D

M AN U

SC

doi:10.1016/j.brainres.2007.11.013

39

ACCEPTED MANUSCRIPT

Table 1. The different levels of item and serial order information encoded in verbal WM

Item level •

Word strings

Long-term serial order level •

Serial position of phonemes within

Short-term serial order level •

•

nonwords with highly familiar phoneme co-occurrences

•

Serial position of phonemes for

•

Serial position of nonword strings in a list

M AN U

Nonword strings

Serial position of words in a list

SC

words

Serial position of phonemes for nonwords with less familiar or novel

EP

TE D

phoneme co-occurrences

AC C

•

RI PT

tasks.

40

ACCEPTED MANUSCRIPT

RI PT

Figure Legends

Figure 1. A theoretical proposal of the different representational and processing levels and their underlying neural substrates involved in the short-term maintenance of verbal item

AC C

EP

TE D

M AN U

SC

and/or serial order information.

41

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

Figure 1

42

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

43