The mismatch-negativity (MMN) component of the auditory event-related potential to violations of abstract regularities: A review

International Journal of Psychophysiology 88 (2013) 109–123

Contents lists available at SciVerse ScienceDirect

International Journal of Psychophysiology journal homepage: www.elsevier.com/locate/ijpsycho

Review

The mismatch-negativity (MMN) component of the auditory event-related potential to violations of abstract regularities: A review Petri Paavilainen ⁎ Cognitive Brain Research Unit, Division of Cognitive and Neuropsychology, Institute of Behavioural Sciences, University of Helsinki, Finland

a r t i c l e

i n f o

Article history: Received 15 October 2012 Received in revised form 19 March 2013 Accepted 21 March 2013 Available online 28 March 2013 Keywords: Auditory information processing MMN (mismatch negativity) ERP (event-related potential) Implicit cognition Predictive processing

a b s t r a c t The mismatch-negativity (MMN) component of the event-related potential (ERP) has been extensively used to study the preattentive processing and storage of regularities in basic physical stimulus features (e.g., frequency, intensity, spatial location). However, studies reviewed in the present article reveal that the auditory analysis reflected by MMN also includes the detection and use of more complex, “abstract”, regularities based, for example, on relationships between various physical features of the stimuli or in patterns present in the auditory stream. When these regularities are violated, then MMN is elicited. Thus, the central auditory system performs even at the pre-attentive, auditory-cortex level surprisingly “cognitive” operations, such as generalization leading to simple concept formation, rule extraction and prediction of future stimuli. The information extracted often seems to be in an implicit form, not directly available to conscious processes and difficult to express verbally. It can nevertheless influence the behavior of the subject, for example, the regularity violations can temporarily impair performance in the primary task. Neural, behavioral and cognitive events associated with the development of the regularity representations are discussed. © 2013 Elsevier B.V. All rights reserved.

1. Introduction: the “abstract-feature” mismatch-negativity (MMN) studies A long-standing problem in cognitive psychology and cognitive neuroscience has been the extent to which the human brain processes information automatically, without conscious efforts and outside our attentional focus and/or consciousness. This problem has manifested itself in various forms in several research paradigms, such as the processing of subliminal stimuli, information processing during sleep, the processing of unattended-channel information in selective attention conditions, various dissociations between perception and awareness in neuropsychological patients, and the area of implicit learning and memory. The controversy has usually centered around the question concerning the depth of the information processing: For example, does the unattended-channel information receive semantic-level processing (e.g., Lachter et al., 2004) or how abstract is the knowledge acquired unintentionally in the implicit-learning paradigms (e.g., Cleeremans et al., 1998). The present review takes a further perspective on this issue by examining how diverse regularities the human brain can automatically extract from auditory stimulation. The results to be reviewed are obtained by using the mismatch-negativity (MMN) component of the event-related potential (ERP). This component has during the recent decades received increasing interest as an index of automatic information processing ⁎ Institute of Behavioural Sciences, P.O.B. 9, 00014 University of Helsinki, Finland. Tel.: + 358 9 19129406; fax: + 358 9 19129443. E-mail address: petri.paavilainen@helsinki.fi. 0167-8760/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.ijpsycho.2013.03.015

occurring in the auditory cortices (for a review, see Näätänen et al., 2007). MMN is elicited by violations in the regular aspects of the auditory stimulation. In the basic “oddball paradigm”, commonly used in MMN studies, the subject is presented at short intervals with physically constant “standard” stimuli, which are infrequently replaced by “deviant” stimuli (e.g., a tone of a different pitch). The deviant stimuli elicit MMN, which is seen in the deviant-minus-standard-stimulus ERP as a frontocentrally distributed negativity, typically peaking 150–200 ms after the onset of the deviance. According to the original interpretation of MMN (e.g., Näätänen et al., 1978; Näätänen, 1992), the physical sound features of the standard stimulus (e.g., pitch, intensity) are analyzed and encoded in short-duration memory traces in the auditory cortex. The elicitation of MMN indicates a disconcordance between the new auditory input and the sensorymemory trace of the standard stimulus. As MMN is elicited even when the auditory stimuli are not attended to, the underlying brain mechanisms are supposed to be, at least to a large extent, preattentive or “automatic”. Consequently, MMN is usually recorded in an “ignore condition”, where the subject is performing a primary task not related to the concurrent auditory stimulation (i.e., he/she is reading a book or watching a video). In an ignore condition, MMN can be recorded more purely, without contamination produced by ERP components related to attentive deviance processing (see Näätänen et al., 2011). The functional significance of the brain mechanisms underlying MMN generation was proposed to be the initiation of an involuntary attention switch to changes in the auditory environment. This would ensure the adequate processing of potentially important changes even in situations where attention is initially directed elsewhere. These brief attention switches are reflected in the

110

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

positive P3a component, often following MMN (e.g., Escera et al., 2001; Escera and Corral, 2007). On the basis of the early MMN studies, MMN was supposed to reflect the neural basis of the auditory sensory memory or “echoic memory” (see, e.g., Näätänen, 1992). This memory system stores the physical features of auditory stimulation for a few seconds during which time the attentional mechanisms can select task-relevant information from its contents for further (e.g., semantic) processing. However, as the research progressed, it became apparent that the properties of the memory system underlying MMN generation considerably differed from those of the classic echoic memory. The traces could, at least in some conditions, last considerably longer than just a few seconds (Winkler and Cowan, 2005), and to some specially important stimuli such as the phonemes of mother tongue, they could become even permanent (e.g., Näätänen et al., 1997; see also Näätänen and Winkler, 1999). Most importantly, the information encoded in the traces has proven to be far more complex than was originally supposed. A research line that emerged in the 1990s has revealed that the preattentive auditory analysis reflected by MMN is not restricted only to basic physical or “first-order” stimulus features (e.g., frequency, intensity, spatial location) of the individual auditory stimuli but rather also includes more complex, “higher-order” regularities or invariances based, for example, on relationships between various physical features (both within individual stimuli and between successive stimuli) or on rules determining the occurrence of specific stimuli in the auditory stream. In the following, research results pertinent to these questions will be reviewed. The paradigm developed in a pioneering study of Saarinen et al. (1992) demonstrates the difference between “first-order” and “higherorder” invariances (Fig. 1). The authors used series of tone pairs as their stimuli (two 60-ms tone pips separated by a 40-ms silent gap; silent inter-pair interval 640 ms). The position of the tone pairs in the frequency dimension randomly varied across 5 different levels. Thus, contrary to previous MMN studies, there was no physically identical, repetitive standard stimulus. Instead, the invariant feature of the standard pairs was the direction of the frequency change: the standard pairs were ascending (i.e., the second tone of a pair was higher in frequency than

pitch

Physical-feature paradigm:

pitch

Abstract-feature paradigm:

time Fig. 1. A schematic illustration of the difference between a classic physical-feature oddball paradigm, used to elicit MMN, and an abstract-feature paradigm. In the physical feature paradigm, a physically invariant standard stimulus (white rectangles) is repeatedly presented. It is occasionally replaced with a physically deviant stimulus, in this case, a tone of a higher pitch (black rectangles). In the abstract-feature paradigm developed by Saarinen et al. (1992), tone pairs are presented to the subject. The position of the pairs in the pitch dimension is randomly varying. Thus, there is no physically invariant standard stimulus. The standard pairs (white rectangles) have in common a higher-order, “abstract”, feature, namely, the direction of the within-pair pitch change: In this example, the standard pairs are descending whereas the deviant pairs (black rectangles) are ascending in pitch.

the first tone), whereas the deviant pairs (similarly varying randomly in the frequency dimension) were descending. Thus, the higher-order invariance was based on a rule defining the relationship between certain physical first-order attributes (in this case, frequencies) of the two tones forming a pair. An MMN was elicited by the deviant pairs in an ignore condition. This result was interpreted by the authors as showing that the preattentively formed stimulus representations were capable of encoding “abstract” attributes corresponding to simple concepts (“rise”, “fall”), that is, of deriving a common invariant feature from a set of individual varying physical events (for analogous data obtained with frequency glides, see Pardo and Sams, 1993). Picton et al. (2000) stressed two factors underlying the brain's ability to differentiate the standard and deviant stimuli in oddball paradigms. First, the incoming auditory information must be parsed into some kind of units and, secondly, the units categorized according to their probability of occurrence. These processes make it possible to extract the invariant or regular aspects in the stimulation against which the deviant units can be detected. The complexity of information extracted in the units may considerably vary depending on the stimulation. In the traditional oddball paradigm, the invariances are rather “concrete”, concerning the constancy of specific physical features, for example, the successive stimuli having the same frequency. The deviant unit simply is the deviant stimulus, differing from the invariant unit (standard stimulus) in frequency. However, in case of the more complex stimulus paradigms, reviewed in the present paper, the invariant unit usually is based on such regularities that can be only extracted by comparing features of multiple stimuli and their relationships with each other. Of course, even in the basic oddball paradigm, a few standard stimulus repetitions are needed for the brain to extract the regularity (“physically identical stimulus repeating”) from the stimulus sequence but in case of more complex invariances, there is no physically identical, repetitive standard stimulus. Instead, there may be many, physically different exemplars of “standard” stimuli, as in the Saarinen et al. (1992) study. The invariant feature, uniting all various standard stimuli, is based on some common rule that they all obey. Similarly, there can be also many physically different exemplars of “deviant stimuli”, all violating the same rule. As Winkler (2007) has pointed out, on the basis of the recent MMN findings, the traditional notion of a “standard” represented in the brain by the sensory memory trace of one or a few concrete stimuli can no more hold. The notion of the standard has to be extended from a “repetitive sound” to a “regular relationship between sounds” while the deviant, in turn, is better characterized as a “regularity violation” as opposed to “sound change”. Consequently, in the studies to-be-reviewed, it is perhaps clearer to use the terms “standard event” and “deviant event” as the “standardness” and “deviance” are not related to any individual physical stimulus per se. A specific stimulus cannot be classified as a standard or deviant event by itself, but only in relation to the previous stimuli. In some paradigms, a physically same stimulus can represent either a standard or a deviant event, depending on the immediate auditory past. In other paradigms, a stimulus that has not been encountered before can still be classified as a standard. These types of MMNs have been variously referred to as “abstract”, “higher-order”, “complex” or “categorical” MMNs (in contrast to “physical”, “concrete”, “first-order” or “simple” MMNs), as they suggest that the MMN mechanism is able to derive abstract invariances from physically varying particular instances (see also Picton et al., 2000). In the present review, the term “abstract-feature MMN” will be used as a general term to refer to the discussed phenomena as it has been adopted for most frequent usage. However, it must be admitted that the “abstractness” of the features is not always a clear-cut concept and in connection with some studies, perhaps some other term might be more appropriate. The present review is organized as follows: First, in Section 2, four paradigms used in the abstract-feature MMN studies are presented and the central findings reviewed. In a group of studies (Section 2.1), the regularity is based on a single physical feature and embedded in the relationship between the elements of the individual stimuli (as

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

the direction of the frequency change in the afore-described tone-pair paradigm). In some studies, the invariance is based on a conjunction between two different physical features (Section 2.2). In the studies reviewed in Section 2.3, the presentation order of the physically different stimuli is not randomized but obeys a certain repetitive pattern, the deviant events violating the pattern. In Section 2.4, the relationship of the MMN generation to predictive information processing is addressed by reviewing studies demonstrating especially clearly how the brain forms, on the basis of the regularities extracted from the preceding stimuli, dynamic predictions concerning the properties of the next stimulus. However, the paradigms, as classified here, are not mutually exclusive and some studies could well have been discussed under some other subsection, too. Also, as will be argued, the prediction aspect can be actually regarded as a general principle involved in all types of MMN paradigms. After presenting paradigms used in abstract-feature MMN studies, the main characteristics of the phenomenon are reviewed, including, for example, their relationship to attention and conscious deviance detection mechanisms and their effects on other ongoing information processing. Thereafter, a model of the neural, behavioral and conscious mechanisms related to processing of auditory regularities is presented. The review ends with a brief discussion on challenges of the abstract-feature MMN findings for future research and on the possible practical applications of the phenomenon.

2. Experimental paradigms 2.1. Extraction of abstract invariances within a single stimulus feature The afore-discussed Saarinen et al. (1992) study started a new line of MMN research, exploring the processing of abstract features in the brain. However, before examining the literature further, a simpler explanation for their results must be considered: perhaps the MMN obtained was just based on storing a separate memory trace for each of the physically different standard pairs, with the infrequent deviant pairs mismatching with these traces. If this were the case, no higher-order invariance extraction mechanisms need to be postulated for the MMN elicitation. However, if this explanation were true, then MMN should become smaller and eventually disappear by increasing the number of physically different standard pairs, as then each specific standard pair appears more infrequently and their traces become progressively weaker. Paavilainen et al. (1998)manipulated the range of variation of the pairs in the frequency dimension, with the number of physically different standard and deviant pairs being, in different conditions, either 1 (i.e., physical MMN), 5 (as in the Saarinen et al. study), or 10. The MMN obtained in each condition was very similar in amplitude, latency and scalp distribution. Zachau et al. (2005; see also Pardo and Sams, 1993) used an even more extensive range of variation, using 15 exemplars of both standard pairs and deviant pairs and observing a prominent MMN. Moreover, Paavilainen et al. (1999) showed that the direction of the within-pair frequency change can be extracted even when its magnitude randomly varies from tone pair to another, i.e., more irrelevant physical variation is added in the stimulation (in the Saarinen et al. study, the within-pair frequency change always was one whole step on a musical scale). These results strongly suggest that the MMN observed by Saarinen et al. (1992) indeed was based on the extraction of the “abstract”, invariant relationship, common to all physically different standard pairs. Guymenyuk et al. (2003) replicated Saarinen at al.'s results in 8–14 year-old children. Furthermore, Carral et al. (2005a) found a discriminative ERP response to deviant pairs even in newborn babies (however, their effect was positive in polarity, representing possibly an infant analog of the adult MMN). Moreover, Ruusuvirta et al. (2007) obtained a differential response to standard and deviant pairs in anaesthetized rats. These studies suggest the early ontogenetic and phylogenetic origin of the ability to process abstract auditory features.

111

Paavilainen et al. (1995) applied Saarinen et al.'s (1992) tone-pair paradigm under more demanding conditions with regard to information load. They used a dichotic-listening task, where the left-ear standard pairs were ascending and the right-ear standard pairs descending in frequency. In an ignore condition, deviant pairs (reversed in the direction of frequency change) in both ears elicited MMN, indicating that the brain was able to extract abstract attributes from a rapid dichotic stimulus stream, separately for the two ears, and even when the attributes were opposite between the ears. In their subsequent study, Paavilainen et al. (1998) investigated the processing of binaural information also in one of their conditions by delivering the two tones forming the pairs in the opposite ears. The deviant pairs continued to elicit MMN, indicating that the inputs to the two ears were in this case converged into a single abstractfeature trace. Carral et al. (2005b) studied the relationship between the MMN amplitude and the magnitude of the abstract change. Their standard-pair tones (5 physically different pairs) were of identical frequency, whereas the deviant pairs had the second tone 2, 4, 6 or 8 whole steps higher or lower than the first one. All the deviant pairs elicited MMN but its amplitude was not affected by the magnitude of abstract change. This finding differed from those obtained with physical-feature MMNs which usually increase in amplitude as a function of the magnitude of change (e.g., Tiitinen et al., 1994). As a possible reason for this discrepancy, the authors pointed out that their standard pairs followed a different rule than the deviant pairs (two identical tones vs. two different tones). Thus, the brain may have reacted rather to the qualitative than to the quantitative difference between the standard and deviant pairs. The analysis of the magnitude of the change would be left for later processing stages (the subsequent P3a component in their study increased in amplitude as a function of the abstract change). Moreover, Horváth et al.'s (2008a) results suggest that there may be actually no significant magnitude effect on the “genuine” (i.e., free from N1 confound) physical-feature MMN amplitude. Rather, the MMN amplitude may index the percentage of detected deviants, each of which elicits an MMN response of uniform amplitude. This result could also well apply to abstract-feature MMNs. Paavilainen et al. (2003) reasoned that if the MMN observed by Saarinen et al. (1992) were truly based on extracted abstract regularities, then it should not depend on what stimulus feature dimension lies under the common abstract attribute, uniting the various exemplars of the standard stimuli. In one condition, the stimulus pairs were all of same frequency but their position in the intensity dimension randomly varied. The standard and deviant pairs differed in the direction of the within-pair intensity change. The abstract intensity changes elicited a similar MMN as the abstract frequency changes did in another condition of their study. Furthermore, Peter et al. (2012) studied with abstract within-pair duration changes the processing of duration-related stress in speech and music stimuli. Their standard stimuli were pairs of either two syllables or two notes, with stress either on the first (long–short pair) or last (short–long pair) syllable/note. There were six different variations of the standard stimuli, the long element being 30% longer than the short one. The deviant stimuli, eliciting the MMN, were reversals of the standards. Paavilainen et al. (2003) studied also the processing of a “double deviant”, simultaneously including both abstract frequency and abstract intensity changes. They did not produce a larger MMN than either of the single abstract deviances alone. This result differs from those obtained for changes in simple physical features (in particular, frequency and intensity), for which the MMNs elicited by the single deviants have been at least partially additive (e.g., Schröger, 1995). The additivity is usually explained as a consequence of the involvement of different neural populations in processing different physical features. The lack of additivity with abstract features suggests that simultaneous deviations in two different abstract features are processed by a common neuronal population, even though these abstract features are based on different physical dimensions. Abstract invariances are possibly encoded in the brain into a single rule if

112

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

they systematically co-occur. This would be a compact way to store information, as in this way, a single rule can cover a large set of individual varying instances (see also Takegata et al., 2001). Invariant frequency ratios (i.e., musical intervals) can also be automatically derived from acoustically varying stimulation. In one condition of Paavilainen et al. (1999), the direction of the within-pair frequency change was the same both for the standard and deviant pairs (ascending). However, the standard pairs had a percentually constant within-pair frequency ratio between the two tones (5 musical steps), whereas the deviant pairs had either a larger (7 or 8 steps) or a smaller (2 or 3 steps) within-pair frequency ratio than that of the standard pairs. The stimulus pairs again randomly varied over a wide frequency range. An MMN was elicited by the deviant pairs. Subsequently, Stefanics et al. (2009) showed, using a generally similar paradigm, that even neonates' brains reacted to deviant pitch intervals with a similar brain response. In another condition of Paavilainen et al. (1999), single tones consisting of two overlaying frequency components (their frequencies corresponding to those used in the pair-wise presentation condition) were used. Again, MMN was obtained for deviants with larger or smaller within-tone frequency ratios. What could be the implications of the afore-reviewed studies to reallife information processing? The perceptual constancies are core phenomena in the functioning of the different sensory systems, with the visual modality providing perhaps the most obvious example. The perceived sizes, shapes and colors of visual objects do not change despite the fact that the objects can be viewed from different angles, distances and in different lightning conditions. In the auditory modality, too, the absolute values of different stimulus features reaching the ear often are not of prime importance for auditory processing. Instead, the relationships between various features, their changes and change directions often carry more important information. The extraction of invariant relationships from physically varying auditory stimulation could be of critical importance for many higher auditory functions such as the processing of speech and music. Thus, it could be expected that even the relative early auditory processing stages, reflected by the MMN, could be involved in extracting and storing this information. For example, we normally categorize phonemes correctly irrespective of considerable variation in the physical “surface” features of the speech signal resulting from the acoustically different voices of various speakers (e.g., male or female) or even in the presence of background noise or other distortions (e.g., a bad telephone line, hoarse voice). The results of Shestakova et al.'s (2002) magnetoencephalographic (MEG) study are pertinent for this issue (for other related studies, see Eulitz and Lahiri, 2004; Jacobsen et al., 2004; Phillips et al., 2000; Wang et al., 2012). Their standard stimuli belonged to a certain phonetic category (e.g., vowel /i/) but there were 150 different exemplars per each category, each pronounced by a different speaker. This added considerable acoustical variation in the stimuli. Occasional category changes (i.e., vowel /u/) elicited a prominent MMNm (magnetic equivalent of electric MMN) response at the auditory cortex. The response was stronger over the left than right hemisphere, probably due to the linguistic nature of the stimuli. For the segmentation of continuous speech into words, the syllabic stress is an important cue in many languages. For example, in English duration-related stress flags the onset of most bisyllabic words. Thus, the processing of relative duration changes might be important for speech comprehension (see Peter et al., 2012). Probably also in the music perception the processing of abstract changes in basic auditory features such as frequency, duration and intensity play an important role. In music, chords preserve their identity when they are played in different keys or with different instruments. Virtala et al. (2011) studied with MMN the processing of major/minor dichotomy, which is of central importance for the Western music. Their standard stimuli were root major triad chords, transposed into all possible keys so that 12 chord variations were equiprobably delivered in a random order. There were 4 types of deviant chords interspersed among them: root minor chords, dissonant

chords, inverted major chords and soft chords (i.e., chords that were physically less loud but otherwise similar to the standard chords). MMN was elicited by all the deviant chords except for the inverted major chords. This result is interesting as the inverted major chords actually differ physically much more from the standard major chords than did the minor chords. Obviously, the qualitative major–minor difference was in this musical context more “deviant” for the brain than was the quantitative difference between the frequency components of the chords. The authors proposed that a life-long exposure to Western music had “tuned” their subjects' brains to process especially effectively invariances typical for it (however, on the basis of this study, the alternative, although perhaps less feasible explanation, that the major/minor dichotomies are already biologically “hard-wired” in the brain cannot be ruled out). 2.2. Processing of feature conjunctions A single, physically identically repeating standard stimulus is not a prerequisite for the elicitation of even a “first-order” physical-feature MMN. Even if some physical features of the standard stimuli vary, an MMN is nevertheless elicited by deviations in constantly-kept features. For instance, Huotilainen et al. (1993) recorded an MMN for a frequency change in a stream of sounds which randomly varied in intensity, duration, rise and fall times, and number of harmonic partials. This result suggests that the various physical features are stored independently of each other: the deviants, which differ from the standard in one feature, strengthen the memory trace for the standard with regard to those features they have in common (in this case, frequency). This phenomenon is utilized in the so-called multi-feature paradigms, enabling the recording of MMN to several different physical features in a relatively short time (e.g., Leung et al., 2012; Pakarinen et al., 2010; Vuust et al., 2011). Although these results suggest that physical sound features are encoded independently of each other, the memory traces also encode more complex information on the conjunctions of features. Gomes et al. (1997; see also, e.g., Brattico et al., 2002; Ruusuvirta et al., 2003; Takegata et al., 2005; Winkler et al., 2005) had two frequently occurring standard stimuli, characterized by a conjunction of two different frequency and intensity values (F1–I1 and F2–I2). The two rare deviant stimuli, eliciting MMN, had the frequency of one of the standard stimuli and the intensity of another standard stimulus (F1–I2 and F2–I1). Thus, what was deviant in the deviant events was the rare conjunction of two features, which were frequently present in the standard stimuli. These MMN data are relevant for the long-standing debate on whether attentive processes are needed for the integration of sensory features to unitary objects (Treisman and Gelade, 1980). The elicitation of MMN in ignore conditions to deviant feature conjunctions suggests that in the auditory modality, feature binding occurs preattentively. However, higher-order information extraction is not yet necessarily needed for explaining the results of Gomes et al. (1997) and similar studies. The observed MMN could be explained by assuming that the two standards were simultaneously and separately encoded in the memory as a kind of “holistic traces” (for the discussion on the separate-feature vs. holistic memory representations, see, e.g., Grimm and Schröger, 2007; Ritter et al., 1995), including information on the combinations of their different features. The deviant stimuli could elicit MMN as they do not holistically match either trace. However, the results of Paavilainen et al. (2001) suggest that this simpler explanation cannot always hold. The authors wished to determine whether the MMN mechanism is capable of extracting invariant relationships based on abstract conjunctions between two sound features. Their stimuli, sinusoidal tone pips presented at an ISI of 400 ms, randomly varied over a large range (8 steps) in frequency and intensity dimensions, there being neither a physically constant, repetitive standard stimulus nor a physically constant repetitive feature conjunction. However, a constant relationship between the frequencies and intensities of the various standard events was defined by a linear conjunction rule “the higher the frequency of a stimulus, the louder the intensity of a

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

stimulus”. Subjects, ignoring sound stimuli, were presented with tone series including occasional deviant events that violated this regularity by following the opposite rule (for example, an occasional high-pitch, weak-intensity tone). An MMN was elicited by these deviant events. Its amplitude was, expectedly, larger for those deviant frequency–intensity combinations that were on either end positions of the linear rule than to those in the middle positions as they were more salient exemplars of rule violations. However, even deviants in the middle-positions elicited some MMN. In a control condition, the 8 frequently-occurring “standard” frequency–intensity conjunctions were randomly chosen from the 8 × 8 frequency–intensity space (instead of following the linear rule). Similarly, the 8 infrequent deviant combinations were also randomly selected. This time, no MMN was elicited. Apparently, as there was no rule governing the frequency–intensity relationships between the various standard events, the system was not able to detect the deviant events. Thus, the mere rareness of the deviant events was not a sufficient condition for the MMN elicitation. Subsequently, Houlihan and Stelmack (2012), using Paavilainen et al.'s (2001) paradigm, showed that higher mental ability (as measured with IQ scores) was associated with larger MMN amplitudes than lower mental ability. Moreover, Astikainen et al. (2013) replicated Paavilainen et al.'s results in anesthetized rats. These data suggest a fundamental similarity in processing rule-based abstract feature conjunctions in human and rat brains. The ability of the brain to preattentively bind together stimulus features varying simultaneously along multiple dimensions might be important, for example, in speech perception for categorizing auditory objects (see, e.g., Russ et al., 2007).

2.3. Pattern extraction One type of MMN studies had investigated the processing of various patterns embedded in the auditory stimulus sequences. In these studies, the different tones are not presented randomly. Instead, their presentation order follows certain rules. The first study to show MMNs reflecting such higher-order invariances was that of Nordby et al. (1988), who presented their subjects with sequences of tones alternating between two frequencies. The deviant events in the stimulus sequence were repetitions of the previous frequency (e.g., ABABABABBABABAABAB…; see also, e.g., Alain et al., 1994; Schröger et al., 1996; Sculthorpe and Campbell, 2011; Takegata et al., 2001). The tone repetitions elicited MMN. This finding challenged the simplest models explaining the MMN as a response specific to a stimulus change, as here the MMN was elicited by a repeating stimulus, i.e., physically the same stimulus as the one immediately preceding it. Subsequently, He et al. (2009) replicated the effect with 4-month-old babies, suggesting that the regularity-extraction mechanisms are functioning already at a very early age similarly as in adults. Moreover, Sculthorpe et al. (2009a) showed that MMN to these kinds of pattern violations is elicited even during REM sleep in adults. Higher mental ability (as measured with IQ scores) has been found to be associated with larger MMN amplitudes also to pattern violations (Sculthorpe et al., 2009b). Cornella et al. (2012) had two types of regularity violations in their sequences consisting of two tones alternating between two frequencies and having a perceived location between the two ears. The simple, physical-feature violations were deviations in the perceived location of the tones (produced by manipulating the inter-aural time difference): occasionally, either tone was perceived as coming from the left. The pattern violations were rare repetitions of either tone. An MMN was elicited by both types of regularity violations. Importantly, at the level of the middle-latency responses (MLR), however, a differential response (in the Na component) was obtained only to the location deviants. The repetition violations produced no differences in the MLR range components. The authors suggest that the results support the hierarchical organization of the auditory deviance detection system, the

113

simple (physical) regularity violations being processed already at earlier cortical levels than those to more complex regularities. Macdonald and Campbell (2011) used, instead of alternating high and low-pitch tones, a sequence consisting of alternating soft and loud tones of same pitch. The occasional repetitions of the loud tone acted as a relative, psychological increment compared to what the alternation rule would have predicted as the next stimulus (the soft tone). The soft-tone repetitions, in turn, acted as a relative, psychological decrement compared to the rule prediction (the loud tone). The increments and decrements were thus psychological rather than physical because they involved violations of an expectation of intensity change rather than an actual physical intensity change. The authors argued that the psychological increments and decrements should activate the memory-based change detection system, eliciting the MMN. The refractory-based transient detector system, reflected in the N1 amplitude increase (see, e.g., Näätänen, 1990), should not be activated as the physical intensities of the psychological increments and decrements were identical to that of the preceding standard. An MMN was indeed obtained to the repetitions when the intensity difference between the soft and loud tones was 27 dB (however, no MMN was elicited with 3 or 9 dB difference). The MMN peaked earlier and its amplitude was larger to psychological increments than to decrements. As a possible explanation, the authors pointed out that the perception of a stimulus increment (whether physical or psychological) might be more salient and ecologically important than that of a decrement. Alain et al. (1999a; see also Boh et al., 2011; Kuchenbuch et al., 2012) demonstrated that changes in the frequency or timing of any stimulus in a repetitive sequence of four-note pattern elicited MMN. The temporal proximity between the tones seems to determine, however, the way of how repeating auditory tone patterns are processed in the brain. Sussman and Gumenyuk (2005) presented their subjects with a recurring five-tone pattern (AAAABAAAAB…). Thus, the pitch-deviant B tone occurred as every fifth tone in the sequence. When the stimuli were presented at a fairly rapid pace (onset-to-onset time 200 ms), then no MMN was elicited by B tones, demonstrating that they were automatically grouped to a regularly repeating five-tone pattern. At stimulus rates of 400 ms and slower, the B tones elicited MMN, suggesting that the tones were individually represented, and the B tone was deviating for the brain as it had a different frequency than the repetitive A tone. However, Herholz et al.'s (2009; see also Salisbury, 2012) results suggest that with pattern regularities, grouping can also occur over longer time ranges. In their tone sequences, the four-tone standard pattern AAAB was “hidden” among longer, deviant patterns (AAAAB, AAAAAB). Although their stimulus rate was 1000 ms, stimuli violating the standard pattern (i.e., AAAA) elicited MMN. In the afore-reviewed studies, the individual tones forming the repeating pattern had a constant pitch (or intensity). One can construct also more “abstract” patterns, where the pitch of the tones can vary and the constant feature is embedded in the relationship between the pitches of the tones, i.e., the “auditory gestalt”. The extraction of such abstract patterns is of prime importance, for example, for music perception. We recognize melodies irrespective of the key into which they are transposed. Tervaniemi et al. (2001; see also Tervaniemi et al., 2006) used melodylike five-tone patterns which were randomly presented at 12 different levels in the frequency dimension (simulating a melody randomly played in different keys). The deviant patterns, having a different contour (up/down pattern of pitch changes), elicited MMN. Fujioka et al. (2004) examined in their MEG study separately the processing of contour and interval (pitch distances between successive notes) information in melodies. In the contour condition, they had 8 different equiprobable standard melodies, each consisting of 5 notes all ascending in pitch. The corresponding deviant melodies were altered so that their final note was descending. The interval condition used one five-note standard melody transposed to 8 keys from trial to trial, and on deviant trials the last note was raised by one whole note without changing the pitch contour. Both contour and interval changes elicited MMNm.

114

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

Brattico et al. (2006) presented their subjects with numerous short, unfamiliar melodies which varied in key, contour, rhythm and timbre, thus minimizing their surface-level invariance. The melodies contained either a pitch deviating from the equal-tempered chromatic scale (out-of-tune pitch) or a pitch deviating from the diatonic scale (out-of-key pitch). An MMN was elicited by both types of pitch violations, suggesting that melodies are automatically modeled by the brain as based on the pitch relations of the musical scale. Perhaps somewhat paradoxically, even a continuous random variation in an acoustic environment can, at least in some circumstances, be regarded as a “pattern” by the brain. Wolff and Schröger (2001; see also Horváth and Winkler, 2004) presented their subjects with series of 5 tones of different pitches that were equiprobably presented so that in most of the trials, the current tone differed in pitch from the preceding tone. The deviant events were rare repetitions of the preceding tone (e.g., BEACADDEBADCEBB…). The repetitions elicited MMN, suggesting that frequent change in the serial tone stimulation can be encoded as a “standard event” and, consequently, infrequent tone repetitions are processed by the brain as deviant events. However, such repetitions have not in all studies elicited MMN (Ritter et al., 1992; Näätänen and Rinne, 2002). For discussion on possible reasons for that, see Wolff and Schröger (2001) and Horváth and Winkler (2004).

Sinusoidal tones

Shepard tones

-1 µVRepetition deviant:

Ascending deviant:

300 ms

Fig. 2. ERPs elicited at Fz from the abstract-feature paradigm of Tervaniemi et al. (1994). Regularly descending sequences with an occasional tone repetition (top row) or an ascending tone (bottom row) as a regularity-violating deviant event were presented to the subject. Tones were either sinusoidal or so-called Shepard tones, the latter creating an illusion of an endlessly descending tone series. The deviant events elicited with both types of tone sequences MMN, seen in the deviant-event ERPs (thick lines) as a negative enhancement (gray area) compared with standard-event ERPs (thin lines). Stimulus onset is marked with arrows.

2.4. Predicting features of the succeeding stimuli Patterns embedded in the auditory stimulation make it also possible to predict the features of the incoming stimuli on the basis of the preceding stimuli. The predictive aspect of the neural mechanisms underlying abstract-feature MMN generation has been apparent in several studies. Tervaniemi et al.'s (1994) stimulus sequences consisted of repeating subsequences of 12 steadily descending tones, presented with a 250-ms inter-stimulus interval (ISI) between the tones (e.g., ABCDEFGHIJKLABCDEFG…). Thus, the consecutive stimuli within the subsequences followed the rule that each tone was lower in frequency than the previous one. The occasional deviant events were either an ascending tone (e.g., ABCDEFGHGHIJ…) or a tone repetition (e.g., ABCDEFGHHIJ…). Both types of events elicited MMN (see Fig. 2). It is again noteworthy that there was no physically constant, repeating standard stimulus. In contrast, all the standard events were physically different whereas the deviant events involved a physically similar stimulus that had occurred immediately before. Thus, when an abstract rule is extracted, even a physically novel stimulus (e.g., a stimulus that has not appeared in the immediate auditory past) can be classified as a standard event in case it obeys the rule established by the preceding stimuli. This would not be possible with concrete rules. In addition, the authors replicated their findings in the second part of their study where they used so-called Shepard tones (Shepard, 1964), creating an illusion of an endlessly descending tone sequence. Paavilainen et al. (2007) studied the preattentive detection of nonsalient feature contingencies between succeeding stimuli. They introduced a paradigm where certain features of the current stimulus predict features of the next stimulus. Their sound stimuli varied in two features, duration and pitch: Tones were either short (50 ms) or long (150 ms) in duration and low (1000 Hz) or high (1500 Hz) in frequency. All four combinations (short-low, short-high, long-low, long-high) were equiprobably presented with an ISI of 300 ms. The duration of each tone randomly was either short or long. The tone sequences were constructed so that the duration of each tone predicted the pitch of the next tone (see Fig. 3): if the present tone is short, then the next tone will be low and if the present tone is long, then the next tone will be high. Occasional deviant events violated these rules. For example, a high-pitched tone following a short tone is a deviant event. In this design, all the 4 different feature combinations used could represent either a standard or deviant event, depending on the duration of the preceding stimulus. The deviant events elicited, in an ignore condition, an MMN. Thus, the generation of MMN here appears to entail that (1) the rules are

represented in memory, (2) information of the duration of each tone is extracted and utilized on each trial to make, on the basis of the rule, a prediction of the pitch of the next tone and (3) if the prediction does not come true, then MMN is elicited. In the Paavilainen et al. (2007) study, the rules remained constant throughout the experiment. However, the real world is usually much more dynamic: as the auditory environment changes, new regularities emerge and older ones vanish. In order to have any ecological validity, the regularity-extraction mechanisms reflected by MMN should be able to rapidly extract and update regularities to keep in pace with the dynamics of the environment. Consequently, Bendixen et al. (2008) wished to determine how fast the rule encoding takes place. They developed a dynamic variation of Paavilainen et al.'s (2007) paradigm by changing the contingent rules repeatedly during the experiment. There were 4–19 presentations of tones conforming to the rule after which a deviant tone (following the opposite rule) occurred. The deviant stimulus was followed by 4–8 irregular (no rule) tones after which the next regular sequence started. The rule in the next sequence could be either the same as, or the opposite to that in the previous regular sequence. The authors found that only 15–20 contingent exemplars are needed for the registration of the rule and hence for the emergence of MMN. Thus, the system is indeed capable of a fairly rapid extraction of regularities in the auditory environment. Moreover, Bendixen and Schröger (2008) showed that with some other types of abstract regularities, the rule extraction can be even faster. They modified Tervaniemi et al.'s (1994) paradigm by constructing stimulus sequences that consisted of ascending and descending subsequences of tones. These “standard” sequences were terminated by a frequency change in the opposite direction (i.e., a deviant event). Deviants occurred randomly and equiprobably in the 3rd to 10th position of the subsequences. After a few irregular stimulus presentations, a new either ascending or descending subsequence started. Thus, the abstract rule continuously changed during the stimulus sequences; therefore the authors could determine how many instances of rule-governed stimuli were needed for the central auditory system for extract the rule. They found that the rule could be extracted even from very short sequences: only two constant relations between the feature values of successive stimuli (i.e. 3 ascending or descending tones) were enough for MMN elicitation. Moreover, the MMN amplitude was not related to the number of rule-governed stimuli preceding the deviant event, suggesting that the formation of abstract rules is not strengthened by repetition but rather established in an all-or-none fashion (for other types of

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

F4

115

LM

Ignore:

“Implicit Detection”:

-1 µV

“Explicit Detection”:

pitch

400 ms S

S

S S

S

S

S

D S

D

S

S

S S

D

S

S

S

D

etc.

time Fig. 3. In the Paavilainen et al. (2007) study, the sound stimuli varied in two features, duration and pitch (see illustration of the tone sequences in the bottom of the figure): Tones were either short or long in duration and low or high in pitch. The duration of each tone randomly was either short or long. The standard events (marked with S) followed the following rules: (1) if the present tone is short, then the next tone will be low and (2) if the present tone is long, then the next tone will be high. Occasional deviant events (marked with D) violated these rules. The ERPs to deviant events are presented with thick lines and those to standard events with thin lines. The deviant events elicited an MMN in ignore condition (subject watching a video), in an “implicit detection condition” (subject trying to detect the deviant events while unaware of the rules) and in an “explicit detection condition” (subject trying to detect the deviants events while informed of the rules). MMN at F4 is shaded. Left mastoid (LM) electrode shows the polarity inversion of the MMN, indicating generator sources at the auditory cortices.

studies related to the prediction hypothesis, see, e.g., Bendixen et al., 2007, 2009; Sussman and Winkler, 2001). Todd et al. (2010; see also Todd and Robinson, 2010; Todd and Mullens, 2011) studied the ability of the MMN mechanism to utilize sparsely repeated contingencies to predict the occurrence of a deviant stimulus. Their hypothesis was that if the MMN system can learn the contingencies hidden in the stimulus sequences, then the MMN amplitude to a predictable deviant should be smaller than that to a similar but nonpredictable deviant, as the regularity representations do not in the former case need updating to the same extent. Their sequences consisted of identical repeating standard tones and 4 types of rare deviant tones (loud deviants, soft deviants, duration deviants and frequency deviants). In some of their sequences, the stimuli were presented in a random order, i.e., prediction was not possible). In other sequences, there were hidden regularities in the stimulus presentation order, for example, that all duration deviants followed a loud deviant and all frequency deviants followed a soft deviant. The MMN to a duration deviant was indeed smaller in amplitude when its occurrence could be predicted by the features of the previous tone compared with the nonpredictable condition. However, for the frequency deviants, no such predictability effect was observed. The authors discuss, as a one possible explanation for this discrepancy, differences in the involvement of prefrontal cortex in modulating the processing of frequency and duration information. Paraskevopoulos et al.'s (2012) recent MEG results suggest that MMN can be used to probe the statistical learning of transitional probabilities between the elements of a complex tone series. Six different three-tone sequences were repeatedly presented in a random order, occasionally interleaved with deviant sequences where the last tone was different from the corresponding standard sequence. The deviant tones elicited MMN, indicating that the brain had learned to group the continuous tone stream into regular segments and reacted to regularity

violations (in addition, a difference in the P50 amplitude was observed between the standard and deviant sequences). The abstract-feature MMN studies reviewed above are consistent with an increasingly influential view on human information processing emphasizing its predictive nature (see, e.g., Bar, 2007; Friston, 2005). For example, perceptual processes generate hypotheses about the causal relationships of the world, based on the representations of previous experiences stored in memory. These hypotheses are tested by comparing them with the current sensory input. If the hypotheses are not met, then the representations are updated. Evidently, the predictive information processing can offer several advantages. The possibility to predict future events gives time to prepare for them. It can speed up the processing of upcoming stimuli that meet the predictions. The sound stream can be segregated into meaningful objects while its perceptual continuity is preserved. Also, missing or noisy information can be dealt with more efficiently. The regular aspects of the environment can be handled automatically, releasing limited resources of controlledprocessing system for dealing with the novel aspects (e.g., Winkler, 2007). Consequently, Winkler (2007; see also Winkler et al., 1996, 2009) proposed that the ultimate function of the memory system underlying MMN is to adjust a neural model of the various regularities of the auditory environment. On the basis of the regularities extracted, predictions are generated concerning the features of the next auditory input. If a prediction fails, then the weight of the corresponding regularity representation is decreased. According to Winkler (2007), this updating process is associated with the generation of MMN (see also Näätänen et al., 2011). In case the predictions are violated, however, then the MMN mechanism can initiate a quick involuntary attention call for the controlled-processing system to ensure effective handling of the novel situation (see, e.g., Winkler, 2007; Winkler et al., 2009).

116

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

The prediction model of MMN has recently gained considerable empirical and theoretical support (see Bendixen et al., 2012; Garrido et al., 2009; Lieder et al., 2013; Wacongne et al., 2011, 2012; Winkler et al., 2009; Winkler and Czigler, 2012; for other types of ERP evidence for the prediction hypothesis, see, e.g., Baldeweg, 2007; Bendixen et al., 2009; for an alternative theory aiming to explain MMN generation with simpler, adaptation-based neural mechanisms, see, however, May and Tiitinen, 2009). 3. Relationship to attention The dependency of (physical-feature) MMN on attention has been a subject of a long controversy. Originally, Näätänen et al. (1978; see also Näätänen, 1990, 1992) proposed that the generation of MMN is fully automatic (independent of attention), reflecting a pre-attentive deviancedetection process which occurs irrespective of whether or not the subject attends to the auditory stimuli. There is evidence for this view from different paradigms. The MMN is typically elicited in ignore conditions, e.g., the subject reading or watching a video during the auditory stimulation. Most abstract-feature MMN studies have also relied on these kinds of rather easy primary tasks. However, they do not provide compelling evidence for the attention-independence of MMN as with such tasks, the subject undoubtedly has enough attentional resources to at least occasionally monitor also the auditory stimuli. Importantly, MMNs to physicalfeature deviants have also been observed in conditions where the attentional mechanisms, presumably, are not functioning, such as during sleep (e.g., Ruby et al., 2008; for pattern violation data, see Sculthorpe et al., 2009a) or even in coma (for a review, see Daltrozzo et al., 2007). However, the relevance of these conditions to everyday information processing may be questioned. Perhaps the most natural paradigm used to study the effects of attention on MMN has been a demanding dichotic selective-listening task. During such tasks, MMN is usually elicited by physical-feature deviants presented even in the non-attended ear but smaller MMN amplitudes for unattended-ear deviants than those to attended-ear deviants has often been reported (for reviews, see Näätänen et al., 2007; Sussman et al., 2003; Sussman, 2007). The elicitation of abstract-feature MMNs during dichotic listening has been studied by Paavilainen et al. (1995). The authors used Saarinen et al.'s (1992) tone-pair paradigm, where subjects were presented with standard pairs and deviant pairs randomly to their left and right ears at a rapid rate (silent inter-pair gap 350 ms). The left-ear standard pairs were ascending and the right-ear standard pairs descending in frequency. In the attend conditions, the subject had to attend either to the left- or right-ear stimuli and press a button to the deviant pairs. In the attended ear, the deviant pairs elicited a large N2 wave, apparently composed of partially overlapping MMN and N2b (ERP correlate of target detection; see, e.g., Ritter et al., 1992) components, followed by a classic P3b (or “P300”; see Polich, 2012) component. The unattendedear data showed a different pattern in the left and right ears. When attention was focused on the left ear, MMN was elicited by (unattended) right-ear deviant pairs. However, when attention was focused on the right ear, left-ear deviant pairs did not elicit MMN. A possible explanation for this discrepancy might be that the left- and right-ear inputs are differently processed with regard to abstract attributes, with the left hemisphere being more specialized than the right hemisphere for this type of analytical processing. Therefore the processing of abstract features in the right-ear input (occurring predominantly in the left hemisphere) might be less vulnerable to strong focusing of attention to the opposite ear. In some abstract-feature MMN paradigms, attentive listening of the auditory stimuli was necessary to initiate the MMN elicitation. Thereafter MMN was elicited also in ignore conditions. Tervaniemi et al. (2001; see also Tervaniemi et al., 2006) used melody-like five-tone patterns which were randomly presented at 12 different frequency levels. The standard patterns had a melodic contour of an inverted U, whereas

the deviant patterns had a different contour, involving a repetition of a tone previously presented in the same pattern. Their study consisted of alternating presentations of three ignore (subjects watching a video) and three attend (subjects pressing button to deviant patterns) conditions. On the basis of their detection performance in the attend conditions, the subjects were classified into “accurate” and “inaccurate” groups. During the first ignore condition no MMN was elicited in either group. However, after the first attend condition, an MMN to deviant patterns emerged in the accurate group in the second ignore condition and its amplitude was still slightly increased in the third ignore condition. In contrast, in the inaccurate group, no MMN was observed at any phase of the experiment. Thus, in contrast to simpler tone patterns such as the tone pairs used by Saarinen et al. (1992), the formation of regularity representations for more complicated patterns needed some attentive listening. Only after the representations had emerged, then deviant events were automatically detected. Sussman (2007) proposed that the principal factor modulating the MMN amplitude is the auditory context rather than attention. According to her theory, attention can play a role in MMN elicitation in certain stimulus paradigms where it modifies the sound organization (see, e.g., Sussman et al., 1998, 2002; Sussman and Gumenyuk, 2005; Winkler et al., 2006). The sound organization determines the grouping of sequential sound elements (segregation or integration). The sound organization exerts an influence on the “standard formation” (i.e., regularity extraction processes), whereas the deviance-detection process, once the regularity has been extracted, is fairly independent of attention. It might be that in Tervaniemi et al.'s (2001) study, attention was needed to organize the physically varying tone patterns so that they were treated by the brain as a single “melody” played in different keys. After this regularity was extracted, the contour changes continued to elicit MMN even in the absence of attention. However, in most abstract feature-MMN experiments, attention has not been needed for MMN elicitation. Thus, it remains to be clarified in future studies for which types of abstract regularities attention is needed to initiate their extraction. 4. Long-term and expertise effects Originally, it was supposed that the physical stimulus features stored in the memory traces reflected by MMN irreversibly decay within ca. 10 s after the cessation of the stimulation (e.g., Sams et al., 1993). However, it has become apparent that the information stored in the traces may enter more durable forms of memory. For example, Cowan et al. (1993) demonstrated that the trace of the physical features of the standard stimulus that had decayed to the extent that the deviant stimulus no longer elicited MMN could be reactivated by presenting one example of the standard stimulus (“reminder”). Korzyukov et al. (2003) demonstrated that the representation of an abstract regularity which has just been vanished, can also be quickly reactivated by a single presentation of a stimulus representing the abstract regularity. Short trains of ascending tone pairs, similar to those with Saarinen et al. (1992) paradigm, were followed by a silent period of 8–12 s. This silent period made the regularity representation inactive, judging from the fact that a descending probe pair presented as the first stimulus immediately after the silence did not elicit MMN. However, if the probe pair was preceded by a single “reminding” ascending pair, then it did elicit MMN. This reactivating effect was similar to what has been previously shown for physical regularities (see Winkler and Cowan, 2005). The authors argued that their results demonstrate the existence of an intermediate phase of memory representations between the sensory buffer and long-term memory and, further, that these representations can be reactivated when similar stimuli are encountered and they are essential in object recognition, which requires both physical and categorical information. However, even longer sustaining or permanent effects on the formation of the memory representations reflected by MMN have been

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

reported. A prolonged exposure on certain regularities in the auditory environment or training to perceive them can result in permanent advantages in their processing. For example, Näätänen et al. (1997) demonstrated that the phonemes specific to the mother tongue are permanently represented in the brain. In addition, in the aforereviewed study of Tervaniemi et al. (2001; see also Herholz et al., 2011; Paraskevopoulos et al., 2012), the MMN elicitation to abstract patterns was facilitated by musical expertise: most of the subjects in the accurate group (who performed well in discriminating contour changes in the melody-like tone patterns and also showed MMN) were professional musicians, whereas the majority of the subjects in the inaccurate group were non-musicians. Also Fujioka et al. (2004) obtained a larger MMNm in musicians both to abstract contour and interval changes than in non-musicians, whereas MMNm in a control condition (simple pitch change) was similar for both groups. The training required to produce these facilitation effects may, however, have to be rather intense, as Tervaniemi et al. (2006) did not find MMN differences between amateur musicians and non-musicians to abstract changes in melodic contour and interval size. Virtala et al. (2012) compared the neural discrimination of major vs. minor chords in 13-year old children, using the same chord stimuli as those in their afore-discussed (2011) study. Again, MMN was elicited by the minor chords but not by the inverted major chords delivered in a context of root major chords. The MMN amplitude was larger for children having music as a hobby compared to those having other active hobbies. Thus, even though the adoption and processing of essential categorizations of the surrounding musical culture can be implicitly learned, musical training can speed up and facilitate the processes. van Zuijen et al. (2005) showed that musical expertise has an influence also on the processing of tone sequences containing either a temporal or a numerical regularity. In their study, sequences with temporal regularity could be divided into segments of a constant duration while the segments contained a varying number of tones. An occasional segment lengthening in time elicited MMN both in musicians and nonmusicians. Sequences with a numerical regularity, in turn, could be divided into segments containing a constant number of tones while the segments varied in duration. An occasional additional tone in the segment (i.e., a violation of numerical regularity) elicited MMN in musicians but not in non-musicians. The authors argued that temporal processing is of general importance in audition and consequently independent of musical expertise. In contrast, the auditory system of musicians is specialized to encode numerosity: organizing sounds by keeping track of the number of beats in a measure is of specific importance for music perception and performance. Consequently, numerosity violations elicited an MMN only in the musicians' brains. However, it must be acknowledged when interpreting the afore-reviewed results comparing musicians with non-musicians that one cannot conclusively disentangle the contributions of musical training and musical talent (i.e., prior musical aptitude) on the results. 5. Effects on the primary-task performance With concrete rules, it has been shown that rule violations elicit not only MMN but usually also the P3a component, reflecting brief, involuntary attention switching to the deviant event (Escera and Corral, 2007). The automatic attention switch to deviant events is biologically meaningful, as sudden changes in the environment can carry potentially important information for the organism. Moreover, the attention switch causes a momentary decrease in the processing resources allocated to the primary task which is accompanied by a behavioral impairment in performance immediately after a deviant event (for example, the prolongation of reaction times and/or decrease in hit rates; see, e.g., Schröger and Wolff, 1998a,b; Schröger et al., 2000). In situations requiring re-orientation from the detected deviance back to the task-relevant stimulus information, the P3a may be followed by RON (re-orienting negativity) component (Schröger and Wolff, 1998b).

117

Schröger et al. (2007), employing the tone-pair paradigm of Saarinen et al. (1992), demonstrated that also violations of abstract rules in unattended stimulus stream interfere with task-related processing. In the first, ignore condition of their study, the subjects ignored the sounds by watching a video. In the second, distraction condition, the subjects performed a task related to sounds but not to the rule: the duration of the second tone of each tone pair was randomly either short or long, and the subjects had to judge the second tone of each pair as being short or long and to press a button with the left or right index finger, respectively. After the distraction condition, the rule (standard pairs ascending, deviant pairs descending) was explained to the subjects. In the subsequent detection condition, their task was to attend to the sounds and press a button to rule-violating stimuli. In the ignore condition, rule violations elicited an MMN, again demonstrating that the extraction and application of abstract rules occurs unintentionally. In the distraction condition, the rule violations elicited both MMN and P3a components. Moreover, the reaction times were prolonged and hit rates decreased in trials where the rule was violated, probably due to the involuntary attention switch to rule violations, reflected by the P3a. Interviews after the distraction condition indicated that none of the subjects showed any sign of explicit knowledge of the rule. Thus, even rule violations that are not consciously noticed can still interfere with task-related processing. In the detection condition, subjects were aware of the rules. The detected rule violations elicited MMN, P3a and P3b components, whereas missed violations elicited none of these components. Thus, the voluntary, behavioral deviance detection in this case seems to be at least partially governed by the processes underlying MMN generation. Also in Bendixen and Schröger's (2008) afore-reviewed study with ascending and descending tone subsequences, the effects of rule violations on the primary task performance were studied. The duration of their tones randomly was either short or long, and subjects performed a twoalternative forced-choice discrimination task concerning the tone duration. The reaction times were prolonged and error rates increased on trials with rule violation compared to trials without rule violation. In addition to MMN, rule violations also elicited P3a and RON components, with P3a indicating an involuntary attention switch as a cause for the performance decline and the RON reflecting the attentional re-orienting to the task-relevant information. 6. Relationship to conscious deviance detection and awareness of the regularities The abstract-feature MMN studies might shed some light even on the debate concerning the possible attention-effects on MMN (see Section 3) by clarifying a related question, namely, the dependence of MMN on the conscious identification of the deviant events: Is MMN elicited by deviants even when the subject cannot consciously detect them and/or when he/ she is unaware of, and unable to express, the regularities differentiating the standard and deviant events? Such a finding would provide strong support for the automaticity of the response and also an interesting dissociation between functioning of the “brain” and “mind”, as that would indicate that some relatively early brain mechanisms can differentiate the standard and deviant events even when it is not possible for the later, conscious processes. It might also have implications, for example, to the controversial research on implicit learning, i.e., the learning of complex information in an incidental manner, without awareness of what has been learned. The result of implicit learning is implicit knowledge in the form of representations that are difficult to verbally express but that can nevertheless exert an influence on behavior (see, e.g., Butler and Berry, 2001; Cleeremans et al., 1998; Dienes and Perner, 1999). Implicit learning is usually regarded as going beyond the formation of simple associations, involving the acquisition of knowledge of some complexity, at some level of abstraction. By using physical-feature deviants in conventional oddball paradigms, it is obvious that even if the subject's attention is directed away from the

118

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

auditory stimuli, he/she usually becomes unintentionally aware of the regularity. The regularity is a very simple one (“identical stimulus repeating”), with the deviants popping clearly out against the repetitive background. Thus, MMNs elicited in commonly used ignore conditions (e.g., reading task, video watching) do not provide compelling evidence for the independence of MMN elicitation from conscious deviance detection. However, Bendixen et al. (2007) created a dynamic variation of the traditional oddball paradigm where subjects discriminated the duration of sequentially presented tones. With no relevance to the task, tones repeated or changed in frequency according to a rule unknown to subjects, simulating a more realistic auditory environment with continuously changing regularities. As soon as after two presentations of stimuli of the same frequency, a subsequent tone of a different frequency elicited MMN. However, on the basis of interviews after the experiment, no subject became aware of the rule, as their attention was directed to the duration dimension of the stimuli, with their frequency being experienced as randomly changing. A more common way to construct physical-feature deviants that are difficult to consciously discriminate from the standards is to use subliminal stimulus changes (for example, tiny pitch changes). MMN has usually decreased in amplitude when the behavioral discrimination threshold is approached and disappeared when the magnitude of deviance falls below it (although occasionally some MMN appears to be elicited even by deviants that are behaviorally not discriminated, see Näätänen et al., 2007). This result could be interpreted as suggesting that the MMN elicitation depends on conscious detection of the deviant stimulus. However, the absence of MMN to subliminal stimulus deviations could also result from the fact that the sensory input to the MMN system from the periphery is in that case already of too low resolution to allow the system to properly differentiate the standards and deviants. If, for example, already the inner-ear mechanisms cannot provide differential pitch responses for the deviant and standard, then it is obvious that no brain mechanism further up in the processing stream can do that either, no matter whether it is functioning automatically or not. However, the abstract-feature MMNs may provide a way to construct deviant events that are difficult to differentiate from standard events, without entailing the use of sub-discrimination threshold physical features in the elements of the stimuli. The basic physical elements used to build up stimuli can be easy to perceive and differentiate from each other like, for example, the pitches of the tones used in the Saarinen et al. (1992) tone-pair paradigm. Nevertheless, the higher-order invariances or rules hidden in their relationships, can be fairly difficult to consciously detect. Some indication of a dissociation between MMN elicitation and conscious awareness of the rule was already provided in the aforereviewed study of Paavilainen et al. (2001) where MMN was elicited in an ignore condition by deviant tones violating the linear conjunction rule “the higher the frequency of a stimulus, the louder the intensity of a stimulus”. After the MMN recording, a behavioral discrimination task was conducted. It was observed that although subjects could detect deviants quite well, most of them could not express the correct rule differentiating the standards and deviants, i.e., they were responding more or less intuitively. Thus, if the information extracted by the MMN mechanism was utilized in the discrimination task, it appeared to be in an implicit form, difficult to express verbally. van Zuijen et al. (2006) have studied most systematically the relationship between abstract-feature MMN elicitation and conscious stimulus discrimination by using the tone-pair paradigm of Saarinen et al. (1992). Their subjects were not told the attribute separating the standard and deviant pairs. In addition to ignore conditions, subjects participated also in an attend condition, where their task was to detect the deviant pairs and to press a reaction key to them. Prior to the attend condition, the subjects were allowed to train to detect deviants in practice sequences where a light flashed on a computer screen as a cue whenever a deviant pair was presented. In addition, they were interviewed between the conditions in order to determine whether they had noticed any

deviant events in the stimulus stream, on what grounds they were pressing the button and to what extent they were consciously aware of the rule separating the standard pairs from deviant ones. The deviant pairs proved to be to many subjects very difficult to detect as the abstract attribute, although in principle a simple one (the direction of the within-pair frequency change), does not pop out clearly against the background formed by the stimulus pairs all the time randomly varying up and down in the frequency dimension. In the beginning of the experiment, the MMN was elicited in the ignore condition both in subjects who in the subsequent interview did express awareness of the deviants (i.e., during the ignore condition, deviant events had involuntarily caught their attention) as well as in those who did not. Part of the subjects learned to detect deviant pairs intuitively: they pressed a button in the attend condition to deviant pairs but could not verbally describe how the deviants differed from the standards. In contrast, some subjects developed explicit knowledge of the stimulus structure: they could also verbally express correctly the rule they utilized in the detection task. In the attend condition, the detected deviants elicited an MMN in both groups but it was followed by the P3b only in those subjects who used explicit knowledge for detecting the deviants. The missed deviants elicited MMN in the subjects with explicit knowledge but not in those with intuitive knowledge. In addition, the MMN of subjects with explicit knowledge was larger in amplitude than that of subjects who showed only intuitive knowledge. Subjects with initially larger MMNs were more likely to develop later in the experiment explicit knowledge than subjects with an initially smaller MMN. These findings suggest that the information extracted by the mechanism generating MMN may form the basis for intuitive and explicit stimulus discrimination and decision-making. However, there were two subjects who did not learn to detect the deviants at all. Nevertheless, the deviants elicited MMN in them. Obviously, in these subjects, the output of the MMN mechanism could not, for some reason, be utilized by their behavioral and conscious discrimination processes. The pattern of results of Schröger et al. (2007) and Bendixen and Schröger (2008) is essentially similar to that of van Zuijen et al. (2006). Their rule violations also elicited an MMN even when the subjects were unaware of the rule. Additionally, these rule violations caused a performance decline in the primary task (see Section 5). In the attend condition of Schröger et al. (2007), the detected deviants elicited both MMN and P3b components. However, no MMN was elicited by the missed deviants. In Bendixen and Schröger (2008), the attended deviants elicited MMN, P3a and P3b components. In the Paavilainen et al. (2007) study (where the rules were “long tones are followed by high tones” and “short tones are followed by low tones”), the subjects' ability to consciously detect the rule-violating stimuli was also studied. After the initial ignore (video watching) condition, subjects were interviewed of whether any specific tones somehow caught their attention during the video watching. No indication of awareness of rules or of the true deviants was obtained. Thereafter, two attend conditions followed. Before the first, “implicit detection condition”, subjects were encouraged to press button to any tones that somehow sounded “odd” or deviant but the rules were not yet revealed to them. After the condition, they were again interviewed to probe their awareness of the rules and deviants. No indication of even fragmental awareness of the rules was obtained from any of the subjects. Before the second, “explicit detection condition”, the rules and how the deviants violated them were explicitly described to them and they were asked to press button to tones violating the rules. Interestingly, a fairly similar MMN was obtained in all three conditions (ignore, implicit detection, explicit detection) although they involved major manipulations with regard to the subjects' direction of attention and knowledge of the structure of the stimuli. Thus, there were no indication of top-down processing effects on the MMN system. It seems that the subjects' brains were learning the rules independently of the direction of attention and the subjects' conscious efforts and responding to rule violations by MMN generation. In the detection conditions, the subjects'

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

performance was somewhat above the chance level. However, as the subjects obviously were (before the last condition) unaware of the rules, learning occurred in an implicit form and deviants were detected more or less intuitively. The explicit awareness of the rules in the last condition did not improve the detection performance. This was probably due to the fact that as the interstimulus interval was very short (300 ms), there simply was not enough time to consciously apply rules to each tone to determine whether it violated rules or not. The MMN was followed in all conditions by a small P3a component, suggesting that at least some of the deviants elicited an involuntary attention switch. It remains an interesting possibility for further studies to determine whether these attentional switches could be somehow related to intuitive detection (implicit knowledge) of the deviant events: perhaps the activation of the brain mechanisms generating the P3a is accompanied with a vague subjective feeling that “something odd just happened in the stimulation”, resulting in an increased probability that the subjects presses the reaction key although he/she cannot explicitly express reasons for that. Further evidence for the dissociation between the neural mechanisms generating MMN and those underlying conscious deviance discrimination was obtained in Paraskevopoulos et al.'s (2012) sequence learning study. Their subjects could not in a behavioral discrimination task explicitly distinguish the standard tone sequences from the deviant ones although the latter did elicit MMN in the preceding ignore condition. 7. Locus of origin in the brain The MMN to physical deviance gets a contribution from two main brain areas: a bilateral supratemporal process at the auditory cortices and a predominantly right-hemispheric frontal process (for reviews, see Alho, 1995; Deouell, 2007; Näätänen et al., 2007). The supratemporal MMN component is, presumably, associated with the pre-perceptual detection of regularity violation, whereas the frontal component appears to be related to the involuntary attention switch caused by the violation (Näätänen et al., 2007). In EEG recordings, due to the orientation of the dipolar source at the auditory cortices, the supratemporal component reverses its polarity at electrodes below the level of Sylvian fissure (when nose is used as a reference). The location of the source on the auditory cortices may slightly vary depending on the physical feature eliciting the MMN, suggesting that the memory traces to various features are located in slightly different places (see Näätänen et al., 2007). What is the location of the regularity representations involved in the generation of MMN for abstract features? The polarity inversion offers a robust indicator of generator at the auditory cortices. It has been reported in many abstract-feature studies (Guymenyuk et al., 2003; Korzyukov et al., 2003; Macdonald and Campbell, 2011; Paavilainen et al., 2001, 2007; Sculthorpe et al., 2009a,b; Virtala et al., 2011; Wolff and Schröger, 2001; van Zuijen et al., 2005) although not in all (Bendixen and Schröger, 2008; Paavilainen et al., 1995; van Zuijen et al., 2006). The lack of a clear polarity inversion observed in a few studies, however, does not necessarily imply a locus of origin outside the auditory cortices, as it may also result from a low signal-to-noise ratio: the amplitude of the abstractfeature MMNs is often rather low compared to those elicited for physical feature changes. Furthermore, the amplitude of the positive, polarityinverted MMN is usually even in case of physical deviants only a fraction of the amplitude of the negative-polarity MMN. Brattico et al. (2006) and Schröger et al. (2007)localized their abstract-feature MMNs on the auditory cortices utilizing electrical source analysis and also MEG studies have yielded similar results (e.g., Herholz et al., 2011; Korzyukov et al., 2003; Paraskevopoulos et al., 2012; Shestakova et al., 2002; Tervaniemi et al., 2001). Moreover, Alain et al. (1999b) found different generator sources at the auditory cortices for MMNs to pitchand pattern-deviant stimuli. With speech stimuli, stronger responses over the left than the right auditory cortex has been reported (Shestakova et al., 2002). In animal experiments, abstract-feature responses have been recorded epidurally at electrodes located above the

119

rat's primary auditory cortex (Astikainen et al., 2013; Ruusuvirta et al., 2007). The involvement of frontal areas in abstract-feature MMN generation has also been reported (Alain et al., 1999b; Guymenyuk et al., 2003; Korzyukov et al., 2003; Schröger et al., 2007). Guymenyuk et al. (2003) manipulated the complexity of the abstract regularities and found that in 8–14 year-old children, the amplitude of the temporal MMN component was similar in easy and hard conditions, whereas that of the frontal MMN component was lower in the hard condition. They suggested that the effect may be related to the slow maturation of the frontal lobes. Several recent studies have indicated that deviance detection in the auditory system is implemented on multiple levels in the human brain, the MMN generation being probably a consequence of a cascade of deviance detection processes taking place at different levels (for a review, see Grimm and Escera, 2012). Differential ERP responses preceding MMN to physically deviant stimuli have been recorded at the brainstem level (Slabu et al., 2012) and in the middle-latency component range (Alho et al., 2012; Grimm et al., 2011; Leung et al., 2012; Recasens et al., 2012). However, thus far in these studies only Cornella et al. (2012) have used higher-order regularity violations. As reviewed in Section 2.3 above, they did not find evidence for pattern violation detection in the MLR range. Thus, these preliminary data may indicate that while deviance detection in physical features occurs already at the lower hierarchical levels, more complex auditory regularities may be processed only at the higher neural levels, generating the MMN (see, however, the P50 data of Paraskevopoulos et al., 2012). However, more research is clearly needed to conclusively resolve this issue. 8. A model of the neural, behavioral and conscious events associated with the processing of auditory regularities On the basis of the afore-reviewed studies, the following model (Fig. 4) is proposed: The auditory cortex is continuously extracting various regularities embedded in the auditory stimulation and storing representations of them. The formation of these preattentive representations is usually fairly rapid and dynamic, adapting fast to changes in the environment. This process generally seems to occur automatically, even without focusing attention on auditory stimuli. However, with certain types of stimuli, attention may be needed to form the regularity representation but thereafter MMN is elicited by regularity violations even in the absence of attention. A prolonged exposure to certain types of regularities (e.g., musical training, everyday language environment) may also enhance the formation of representations. The regularity representations enable the central auditory system to manage a large part of its input automatically, i.e., without requiring the limited controlled-processing resources (Winkler et al., 1996). If the auditory input contains regularity violations, then MMN will be elicited. In the present model, the elicitation of MMN may, depending on the type of regularity, even precede changes in behavioral and conscious levels. The supratemporal MMN signal itself may be related to the updating of the representations as Winkler (2007) has proposed. Here it is assumed, in line with the original view of Winkler et al. (1996), that it also serves as an “alarm signal”, transmitted to the frontal MMN generator, which may initiate an attention switch to the deviant event. As reviewed in Section 7, recent evidence indicates that regularity violations based on physical stimulus features are processed already earlier at MLR (or even ABR) levels, supporting the hierarchical deviance detection model (Grimm and Escera, 2012). It remains to be tested in future experiments whether abstract-feature processing might also take place already on the neural levels preceding those involved in MMN generation or does it always entail the involvement of later cortical auditory/association areas. Once a strong enough regularity representation has been formed, its violations may also start to exert influences observable at the behavioral level. For these purposes, higher brain mechanisms may utilize the MMN signal via the frontal attention-switching mechanisms. However,

120

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

especially in case of abstract regularities, the subject is often acting on an implicit-knowledge level, that is, more or less intuitively: if the subject's task is to detect deviant events, then his/her performance may be above the chance level but he/she is not consciously aware of the actual regularities and how the deviant events violate them. If the subject's task is to concentrate on performing a primary task, not related to the auditory regularities, then the brief involuntary attention switches associated with the MMN signal may manifest themselves in temporary impairments in the primary task performance (i.e., decrease in hit rates or prolongation of reaction times). On the electrophysiological level, they are reflected by the presence of the P3a component (however, it should be noted that the tight coupling of MMN and P3a elicitation has also been questioned, see Horváth et al., 2008b; Rinne et al., 2006). Finally, it is possible that the subject becomes fully aware of the regularities and their violations on a conscious, explicit-knowledge level. In order to reach this level, some focusing of selective attention on the auditory stimuli is usually needed. Depending on the type of regularity, some subjects may not reach this level even after prolonged attentive listening but, for example, musical expertise can help others in reaching this phase (also, depending on the regularity, an explicit description of the rules to the subject may switch his/her performance on this level). It is manifested as the subject's ability to express verbally the rules embedded in the stimulation and as a high performance level in detecting deviant events. On the electrophysiological measures, the elicitation of the P3b to deviant events indicates that they are consciously perceived by the subject. 9. Some challenges for the future research and conclusions Some questions that obviously need to be addressed in future studies involve, for example, the extent of this type of regularity extraction: how diverse and complex auditory regularities can be extracted on the processing level reflected by the MMN? In what type of condition does this

STIMULATION

MMN (MLR)

Pre-attentive neural level Auditory cortex starts to model regularities in the stimulation MMN is elicited when the regularities are violated (context updating) (depending on the type of regularity, also earlierlevel responses may be involved)

SELECTIVE ATTENTION (needed in certain stimulation paradigms?)

extraction occur automatically and when is selective attention to auditory stimuli needed? The exact relationship between the abstract-MMN mechanisms, behavioral changes and conscious awareness of the regularities (Fig. 4) needs also further clarification in better-controlled studies: Could it be possible in some circumstances that an MMN to deviant events can be elicited even when the subject's behaviorally measured detection is on a chance level, or is the emergence of MMN immediately associated with improvements in detection performance? Although the data thus far accumulated strongly suggest that MMN is elicited even when the subject is not aware of the exact regularities and their violations (and cannot express them verbally), also this theoretically important question needs further confirmation. Moreover, the possibility of topdown effects on MMN generator should be explored in more detail: Could explicit information of the regularities facilitate the formation of the regularity representations? The relationship between the generators of MMN and those of earlier deviance detection mechanisms manifesting at the MLR (or even ABR) range needs also further investigation. Are regularity violations based on simpler, physical features processed already at earlier levels and those to abstract features only at later, MMN-generation levels? Grimm et al. (2011) have, as a one alterative, suggested that there could be functional differences between the levels of deviance processing. For example, the earlier levels may be involved in generating auditory predictions and the later levels in updating regularity representations. How are the regularities extracted at the neuronal level? The leading models explaining the elicitation of MMN are based on the predictive coding framework (see, e.g., Garrido et al., 2009; Lieder et al., 2013; Wacongne et al., 2011; Winkler et al., 2009) but it is not yet clear whether they can in their present form account for all the afore-reviewed results. At least certain types of abstract-feature MMNs, such as those obtained in the Saarinen et al. paradigm, could be, in principle, explainable also within the simpler, adaptation-based mechanisms as there is evidence for neurons encoding for rising vs. falling frequency relations between sequential tones (see May and Tiitinen, 2009). However, it is also possible

hit rate RT P3a

verbal description of the regularities P3b

Behavioural level

Conscious level

The preattentive regularity representations influence behaviour

Explicit knowledge of the regularities: the subject is aware of the regularities and can verbally express them

Involuntary attention shifts to deviant events, accompanied by a momentary impairment in the primary-task performance Implicit knowledge of complex regularities: deviant-event detection above chance level, but no awareness of the rules (”intuitive performance”)

Deviant-event detection on a high level

SELECTIVE ATTENTION

Fig. 4. A hypothetical model, based on the MMN data reviewed, of the relationship between the neural, behavioral and conscious events, leading from implicit to explicit knowledge on auditory regularities. Indicators of processing occurring at each level are given on the top of the figure. See text for further details.

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

that several types of neural mechanisms are involved in the generation of the responses to regularity-violating tones, depending on the neural level and complexity of the regularities (see Grimm and Escera, 2012). The possible clinical applications for abstract-feature MMNs might also be worth studying. There have been plenty of interesting attempts to develop clinical applications for MMN to diagnose various auditory information processing disorders (for a review, see Näätänen et al., 2012) but they have been mainly based on using MMNs elicited by physicalfeature deviants. The multi-feature paradigms, enabling the recording of MMN to several different physical features in a relatively short time even without a physically identically repeating standard stimulus (e.g., Pakarinen et al., 2010) are an especially promising approach (see, e.g., Fisher et al., 2012; Lovio et al., 2012). The proper abstract-feature MMN paradigms might in some cases be even more sensitive indices as with them, it could be possible to tap the functioning of such higher-order neural mechanisms that might be more vulnerable for degenerative brain disorders. The observed associations between abstract-feature MMN amplitudes and such factors as general mental ability or certain specific abilities (e.g., musicality) may also hold promises for future practical applications, both with children and adults. In conclusion, the afore-reviewed MMN studies suggest that the central auditory system performs even at the pre-attentive, auditory-cortex level rather complex “cognitive” operations, such as generalization leading to simple concept formation, rule extraction and prediction of stimuli. The information extracted by the preattentive mechanisms often seems to be in an implicit form, not directly available to conscious processes and difficult to express verbally. Nevertheless, it can exert influences on behavior. For reviews setting the present questions in a wider context in human information processing, among other phenomena demonstrating a kind of “primitive intelligence” at the level of auditory cortex, see Näätänen et al. (2001, 2010).

Acknowledgments The author wishes to thank Professor Risto Näätänen, Professor Mari Tervaniemi and the two anonymous reviewers for their useful and constructive comments on the manuscript.

References Alain, C., Woods, D.L., Ogawa, K.H., 1994. Brain indices of automatic pattern processing. Neuroreport 6, 140–144. Alain, C., Cortese, F., Picton, T.W., 1999a. Event-related brain activity associated with auditory pattern processing. Neuroreport 10, 2429–2434. Alain, C., Achim, A., Woods, D.L., 1999b. Separate memory-related processing for auditory frequency and patterns. Psychophysiology 36, 737–744. Alho, K., 1995. Cerebral generators of mismatch negativity (MMN) and its magnetic counterpart (MMNm) elicited by sound changes. Ear and Hearing 16, 38–51. Alho, K., Grimm, S., Mateo-Leon, S., Costa-Faidella, J., Escera, C., 2012. Early processing of pitch in the human auditory system. The European Journal of Neuroscience 1–7. http://dx.doi.org/10.1111/j.1460-9568.2012.08219.x. Astikainen, P., Ruusuvirta, T., Näätänen, R., 2013. Rule-based categorization of arbitrary auditory objects in anaesthetized rats (Manuscript in preparation). Baldeweg, T., 2007. ERP repetition effects and mismatch negativity generation: a predictive coding perspective. Journal of Psychophysiology 21, 204–213. Bar, M., 2007. The proactive brain: using analogies and associations to generate predictions. Trends in Cognitive Sciences 11, 280–289. Bendixen, A., Schröger, E., 2008. Memory trace formation for abstract auditory features and its consequences in different attentional contexts. Biological Psychology 78, 231–241. Bendixen, A., Roeber, U., Schröger, E., 2007. Regularity extraction and application in dynamic auditory stimulus sequences. Journal of Cognitive Neuroscience 19, 1664–1677. Bendixen, A., Prinz, W., Horváth, J., Trujillo-Barreto, N.J., Schröger, E., 2008. Rapid extraction of auditory feature contingencies. NeuroImage 41, 1111–1119. Bendixen, A., Schröger, E., Winkler, I., 2009. I heard that coming: event-related potential evidence for stimulus-driven prediction in the auditory system. The Journal of Neuroscience 29, 8447–8451. Bendixen, A., SanMiguel, I., Schröger, E., 2012. Early electrophysiological indicators of predictive processing in audition: a review. International Journal of Psychophysiology 3, 120–131. Boh, B., Herholz, S., Lappe, C., Pantev, C., 2011. Processing of complex auditory patterns in musicians and nonmusicians. PloS One 6, e21458.

121

Brattico, E., Winkler, I., Näätänen, R., Paavilainen, P., Tervaniemi, M., 2002. Simultaneous storage of two complex sound patterns in auditory sensory memory. Neuroreport 13, 1747–1751. Brattico, E., Tervaniemi, M., Näätänen, R., Peretz, I., 2006. Musical scale properties are automatically processed in the human auditory cortex. Brain Research 1117, 162–174. Butler, L.T., Berry, D.C., 2001. Implicit memory: intention and awareness revisited. Trends in Cognitive Sciences 5, 192–197. Carral, V., Huotilainen, M., Ruusuvirta, T., Fellman, V., Näätänen, R., Escera, C., 2005a. A kind of auditory “primitive intelligence” already present at birth. The European Journal of Neuroscience 21, 3201–3203. Carral, V., Corral, M.-J., Escera, C., 2005b. Auditory event-related potentials as a function of abstract change magnitude. Neuroreport 16, 301–305. Cleeremans, A., Destrebecqz, A., Boyer, M., 1998. Implicit learning: news from the front. Trends in Cognitive Sciences 2, 406–416. Cornella, M., Leung, S., Grimm, S., Escera, C., 2012. Detection of simple and pattern regularity violations occurs at different levels of the auditory hierarchy. PloS One 7, e43604. Cowan, N., Winkler, I., Teder, W., Näätänen, R., 1993. Memory prerequisites of the mismatch negativity in the auditory event-related potential (ERP). Journal of Experimental Psychology. Learning, Memory, and Cognition 19, 909–921. Daltrozzo, J., Wioland, N., Mutschler, V., Kotchoubey, B., 2007. Predicting coma and other low responsive patients outcome using event-related brain potentials: a meta-analysis. Clinical Neurophysiology 118, 606–614. Deouell, L., 2007. The frontal generator of the mismatch negativity revisited. Journal of Psychophysiology 21, 188–203. Dienes, Z., Perner, J., 1999. A theory of implicit and explicit knowledge. The Behavioral and Brain Sciences 22, 735–808. Escera, C., Corral, M.J., 2007. Role of mismatch negativity and novelty-P3 in involuntary auditory attention. Journal of Psychophysiology 21, 251–264. Escera, C., Yago, E., Alho, K., 2001. Electrical brain responses reveal the temporal dynamics of brain events during involuntary attention switching. The European Journal of Neuroscience 14, 877–883. Eulitz, C., Lahiri, A., 2004. Neurobiological evidence for abstract phonological representations in the mental lexicon during speech recognition. Journal of Cognitive Neuroscience 16, 577–583. Fisher, D.J., Grant, B., Smith, D.M., Borracci, G., Labelle, A., Knott, V.J., 2012. Nicotine and the hallucinating brain: effects on mismatch negativity (MMN) in schizophrenia. Psychiatry Research 196, 181–187. Friston, K., 2005. A theory of cortical responses. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 360, 815–836. Fujioka, T., Trainor, L.J., Ross, B., Kakigi, R., Pantev, C., 2004. Musical training enhances automatic encoding of melodic contour and interval structure. Journal of Cognitive Neuroscience 16, 1010–1021. Garrido, M.I., Kilner, J.M., Stephan, K.E., Friston, K.J., 2009. The mismatch negativity: a review of underlying mechanisms. Clinical Neurophysiology 120, 453–463. Gomes, H., Bernstein, R., Ritter, W., Vaughan Jr., H.G., 1997. Storage of feature conjunctions in transient auditory memory. Psychophysiology 34, 712–716. Grimm, S., Escera, C., 2012. Auditory deviance detection revisited: evidence for a hierarchical novelty system. International Journal of Psychophysiology 85, 88–92. Grimm, S., Schröger, E., 2007. The processing of frequency deviations within sounds: evidence for the predictive nature of the Mismatch Negativity (MMN) system. Restorative Neurology and Neuroscience 25, 241–249. Grimm, S., Escera, C., Slabu, L., Costa-Faidella, J., 2011. Electrophysiological evidence for the hierarchical organization of auditory change detection in the human brain. Psychophysiology 48, 377–384. Guymenyuk, V., Korzyukov, O., Alho, K., Winkler, I., Paavilainen, P., Näätänen, R., 2003. Electric brain responses indicate preattentive processing of abstract acoustic regularities in children. Neuroreport 14, 1411–1415. He, C., Hotson, L., Trainor, L.J., 2009. Development of infant mismatch responses to auditory pattern changes between 2 and 4 months old. The European Journal of Neuroscience 29, 861–867. Herholz, S.C., Lappe, C., Pantev, C., 2009. Looking for a pattern: an MEG study on the abstract mismatch negativity in musicians and nonmusicians. BMC Neuroscience 10. Herholz, S.C., Boh, B., Pantev, C., 2011. Musical training modulates encoding of higherorder regularities in the auditory cortex. The European Journal of Neuroscience 34, 524–529. Horváth, J., Winkler, I., 2004. How the human auditory system treats repetition amongst change. Neuroscience Letters 368, 157–161. Horváth, J., Czigler, I., Jacobsen, T., Maess, B., Schröger, E., Winkler, I., 2008a. MMN or no MMN: no magnitude of deviance effect on the MMN amplitude. Psychophysiology 45, 60–69. Horváth, J., Winkler, I., Bendixen, A., 2008b. Do N1/MMN, P3a, and RON form a strongly coupled chain reflecting the three stages of auditory distraction? Biological Psychology 79, 139–147. Houlihan, M., Stelmack, R.M., 2012. Mental ability and mismatch negativity: pre-attentive discrimination of abstract feature conjunctions in auditory sequences. Intelligence 40, 239–244. Huotilainen, M., Ilmoniemi, R.J., Lavikainen, J., Tiitinen, H., Alho, K., Sinkkonen, J., Knuutila, J., Näätänen, R., 1993. Interaction between representations of different features of auditory sensory memory. Neuroreport 4, 1279–1281. Jacobsen, T., Schröger, E., Alter, K., 2004. Pre-attentive perception of vowel phonemes from variable speech stimuli. Psychophysiology 41, 654–659. Korzyukov, O.A., Winkler, I., Gumenyuk, V.I., Alho, K., 2003. Processing abstract auditory features in the human auditory cortex. NeuroImage 20, 2245–2258. Kuchenbuch, A., Paraskevopoulos, E., Herholz, S.C., Pantev, C., 2012. Electromagnetic correlates of musical expertise in processing of tone patterns. PloS One 7, e30171.

122

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123

Lachter, J., Forster, K.I., Ruthruff, E., 2004. Forty-five years after Broadbent (1958): still no identification without attention. Psychological Review 111, 880–913. Leung, S., Cornella, M., Grimm, S., Escera, C., 2012. Is fast auditory change detection feature specific? An electrophysiological study in humans. Psychophysiology 49, 933–942. Lieder, F., Daunizeau, J., Garrido, M.I., Friston, K.J., Stephan, K.E., 2013. Modelling trial-bytrial changes in the mismatch negativity. PLoS Computational Biology 9, e1002911. Lovio, R., Halttunen, A., Lyytinen, H., Näätänen, R., Kujala, T., 2012. Reading skill and neural processing accuracy improvement after a 3-hour intervention in preschoolers with difficulties in reading-related skills. Brain Research 1448, 42–55. Macdonald, M., Campbell, K., 2011. Effects of a violation of an expected increase or decrease in intensity on detection of change within an auditory pattern. Brain and Cognition 77, 438–445. May, P.J.C., Tiitinen, H., 2009. Mismatch-negativity (MMN), the deviance-elicited auditory deflection, explained. Psychophysiology 46, 1–57. Näätänen, R., 1990. The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. The Behavioral and Brain Sciences 13, 201–288. Näätänen, R., 1992. Attention and Brain Function.Lawrence Erlbaum Associates, Hillsdale, New Jersey 494. Näätänen, R., Rinne, T., 2002. Electric brain response to sound repetition in humans: an index of long-term-memory-trace formation? Neuroscience Letters 318, 49–51. Näätänen, R., Winkler, I., 1999. The concept of auditory stimulus representation in cognitive neuroscience. Psychological Bulletin 125, 826–859. Näätänen, R., Gaillard, A.W.K., Mäntysalo, S., 1978. Early selective-attention effect on evoked potential reinterpreted. Acta Psychologica 42, 313–329. Näätänen, R., Lehtokoski, A., Lennes, M., Cheour, M., Huotilainen, M., Iivonen, A., Vainio, M., Alku, P., Ilmoniemi, R., Luuk, A., Allik, J., Sinkkonen, J., Alho, K., 1997. Language-specific phoneme representations revealed by electric and magnetic brain responses. Nature 385, 432–434. Näätänen, R., Tervaniemi, M., Sussman, E., Paavilainen, P., Winkler, I., 2001. “Primitive intelligence” in the auditory cortex. Trends in Neurosciences 24, 283–288. Näätänen, R., Paavilainen, P., Rinne, T., Alho, K., 2007. The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical Neurophysiology 118, 2544–2590. Näätänen, R., Astikainen, P., Ruusuvirta, T., Huotilainen, M., 2010. Automatic auditory intelligence: an expression of the sensory-cognitive core of cognitive processes. Brain Research Reviews 64, 123–136. Näätänen, R., Kujala, T., Winkler, I., 2011. Auditory processing that leads to conscious perception: a unique window to central auditory processing opened by the mismatch negativity and related responses. Psychophysiology 48, 4–22. Näätänen, R., Kujala, T., Escera, C., Baldeweg, T., Kreegipuu, K., Carlson, S., Ponton, C., 2012. The mismatch negativity (MMN) — a unique window to disturbed central auditory processing in ageing and different clinical conditions. Clinical Neurophysiology 123, 424–458. Nordby, H., Roth, W.T., Pfefferbaum, A., 1988. Event-related potentials to breaks in sequences of alternating pitches or interstimulus intervals. Psychophysiology 25, 262–268. Paavilainen, P., Saarinen, J., Tervaniemi, M., Näätänen, R., 1995. Mismatch negativity to changes in abstract sound features during dichotic listening. Journal of Psychophysiology 9, 243–249. Paavilainen, P., Jaramillo, M., Näätänen, R., 1998. Binaural information can converge in abstract memory traces. Psychophysiology 35, 483–487. Paavilainen, P., Jaramillo, M., Näätänen, R., Winkler, I., 1999. Neuronal populations in the human brain extracting invariant relationships from acoustic variance. Neuroscience Letters 265, 179–182. Paavilainen, P., Simola, J., Jaramillo, M., Näätänen, R., Winkler, I., 2001. Preattentive extraction of abstract feature conjunctions from auditory stimulation as reflected by the mismatch negativity (MMN). Psychophysiology 38, 359–365. Paavilainen, P., Degerman, A., Takegata, R., Winkler, I., 2003. Spectral and temporal stimulus characteristics in the processing of abstract auditory features. Neuroreport 14, 715–718. Paavilainen, P., Arajärvi, P., Takegata, R., 2007. Preattentive detection of nonsalient contingencies between auditory features. Neuroreport 18, 159–163. Pakarinen, S., Huotilainen, M., Näätänen, R., 2010. The mismatch negativity (MMN) with no standard stimulus. Clinical Neurophysiology 121, 1043–1050. Paraskevopoulos, E., Kuchenbuch, A., Herholz, S.C., Pantev, C., 2012. Statistical learning effects in musicians and non-musicians: an MEG study. Neuropsychologia 50, 341–349. Pardo, P.J., Sams, M., 1993. Human auditory cortex responses to rising versus falling glides. Neuroscience Letters 159, 43–45. Peter, V., McArthur, G., Thompson, W.F., 2012. Discrimination of stress in speech and music: a mismatch negativity (MMN) study. Psychophysiology 49, 1590–1600. Phillips, C., Pellathy, T., Marantz, A., Yellin, E., Wexler, K., Poeppel, D., McGinnis, M., Roberts, T., 2000. Auditory cortex accesses phonological categories: an MEG mismatch study. Journal of Cognitive Neuroscience 12, 1038–1055. Picton, T.W., Alain, C., Otten, L., Ritter, W., Achim, A., 2000. Mismatch negativity: different water in the same river. Audiology & Neuro-Otology 5, 111–139. Polich, J., 2012. Neuropsychology of P300. In: Luck, S.J., Kappenman, E.S. (Eds.), The Oxford Handbook of Event-Related Potential Components. Oxford University Press, New York, pp. 159–188. Recasens, M., Grimm, S., Capilla, A., Nowak, R., Escera, C., 2012. Two sequential processes of change detection in hierarchically ordered areas in the human auditory cortex. Cerebral Cortex. http://dx.doi.org/10.1093/cercor/bhs295. Rinne, T., Särkkä, A., Degerman, A., Schröger, E., Alho, K., 2006. Two separate mechanisms underlie auditory change detection and involuntary control of attention. Brain Research 1077, 135–143.

Ritter, W., Paavilainen, P., Lavikainen, J., Reinikainen, K., Alho, K., Sams, M., Näätänen, R., 1992. Event-related potentials to repetition and change of auditory stimuli. Electroencephalography and Clinical Neurophysiology 83, 306–321. Ritter, W., Deacon, D., Gomes, H., Javitt, D.C., Vaughan Jr., H.G., 1995. The mismatch negativity of event-related potentials as a probe of transient auditory memory: a review. Ear and Hearing 16, 52–67. Ruby, P., Caclin, A., Boulet, S., Delpuech, C., Morlet, D., 2008. Odd sound processing in the sleeping brain. Journal of Cognitive Neuroscience 20, 296–311. Russ, B.E., Lee, Y.-S., Cohen, Y.E., 2007. Neural and behavioural correlates of auditory categorization. Hearing Research 229, 204–212. Ruusuvirta, T., Huotilainen, M., Fellman, V., Näätänen, R., 2003. The newborn human brain binds sound features together. Neuroreport 14, 2117–2119. Ruusuvirta, T., Koivisto, T., Wikgren, J., Astikainen, P., 2007. Processing of melodic contours in urethane-anaesthetised rats. The European Journal of Neuroscience 26, 701–703. Saarinen, J., Paavilainen, P., Schröger, E., Tervaniemi, M., Näätänen, R., 1992. Representation of abstract attributes of auditory stimuli in the human brain. Neuroreport 3, 1149–1151. Salisbury, D.S., 2012. Finding the missing stimulus mismatch negativity (MMN): emitted MMN to violations of an auditory gestalt. Psychophysiology 49, 544–548. Sams, M., Hari, R., Rif, J., Knuutila, J., 1993. The human auditory sensory memory persists about 10 s: neuromagnetic evidence. Journal of Cognitive Neuroscience 5, 363–370. Schröger, E., 1995. Processing of auditory deviants with changes in one versus two stimulus dimensions. Psychophysiology 32, 55–65. Schröger, E., Wolff, C., 1998a. Behavioural and electrophysiological effects of task-irrelevant sound change: a new distraction paradigm. Cognitive Brain Research 7, 71–87. Schröger, E., Wolff, C., 1998b. Attentional orienting and reorienting is indicated by human event-related brain potentials. Neuroreport 15, 3355–3358. Schröger, E., Tervaniemi, M., Wolff, C., Näätänen, R., 1996. Preattentive periodicity detection in auditory patterns as governed by time and intensity information. Cognitive Brain Research 4, 145–148. Schröger, E., Giard, M.H., Wolff, C., 2000. Auditory distraction: event-related potential and behavioural indices. Clinical Neurophysiology 111, 1450–1460. Schröger, E., Bendixen, A., Trujillo-Barreto, N.J., Roeber, U., 2007. Processing of abstract rule violations in audition. PloS One 2, e1131. Sculthorpe, L.D., Campbell, K.B., 2011. Evidence that the mismatch negativity to pattern violations does not vary with deviant probability. Clinical Neurophysiology 122, 2236–2245. Sculthorpe, L.D., Ouellet, D.R., Campbell, K.B., 2009a. MMN elicitation during natural sleep to violations of an auditory pattern. Brain Research 1290, 52–62. Sculthorpe, L.D., Stelmack, R.M., Campbell, K.B., 2009b. Mental ability and the effect of pattern violation discrimination on P300 and mismatch negativity. Intelligence 37, 405–411. Shepard, R.N., 1964. Circularity in judgments of relative pitch. The Journal of the Acoustical Society of America 38, 2346–2353. Shestakova, A., Brattico, E., Huotilainen, M., Galunov, V., Soloviev, A., Sams, M., Ilmoniemi, R.J., Näätänen, R., 2002. Abstract phoneme representations in the left temporal cortex: magnetic mismatch negativity study. Neuroreport 13, 1813–1816. Slabu, L., Grimm, S., Escera, C., 2012. Novelty detection in the human auditory brainstem. The Journal of Neuroscience 32, 1447–1452. Stefanics, G., Haden, G.P., Sziller, I., Balazs, L., Beke, A., Winkler, I., 2009. Newborn infants process pitch intervals. Clinical Neurophysiology 120, 304–308. Sussman, E.S., 2007. A new view on the MMN and attention debate: the role of context in processing auditory events. Journal of Psychophysiology 21, 164–175. Sussman, E.S., Gumenyuk, V., 2005. Organization of sequential sounds in auditory memory. Neuroreport 16, 1519–1523. Sussman, E., Winkler, I., 2001. Dynamic sensory updating in the auditory system. Cognitive Brain Research 12, 431–439. Sussman, E., Ritter, W., Vaughan Jr., H.G., 1998. Attention affects the organization of auditory input associated with the mismatch negativity system. Brain Research 789, 130–138. Sussman, E., Winkler, I., Huotilainen, M., Ritter, W., Näätänen, R., 2002. Top-down effects can modify the initially stimulus-driven auditory organization. Cognitive Brain Research 13, 393–405. Sussman, E., Winkler, I., Wang, W., 2003. MMN and attention: competition for deviance detection. Psychophysiology 40, 430–435. Takegata, R., Paavilainen, P., Näätänen, R., Winkler, I., 2001. Preattentive processing of spectral, temporal, and structural characteristics of acoustic regularities: a mismatch negativity study. Psychophysiology 28, 902–998. Takegata, R., Brattico, E., Tervaniemi, M., Varyagina, O., Näätänen, R., Winkler, I., 2005. Preattentive representation of feature conjunctions for simultaneous, spatially distributed auditory objects. Cognitive Brain Research 25, 169–179. Tervaniemi, M., Maury, S., Näätänen, R., 1994. Neural representations of abstract stimulus features in the human brain as reflected by the mismatch negativity. Neuroreport 5, 844–846. Tervaniemi, M., Rytkönen, M., Schröger, E., Ilmoniemi, R.J., Näätänen, R., 2001. Superior formation of cortical memory traces for melodic patterns in musicians. Learning & Memory 8, 295–300. Tervaniemi, M., Castaneda, A., Knoll, M., Uther, M., 2006. Sound processing in amateur musicians and nonmusicians: event-related potential and behavioural indices. Neuroreport 17, 1225–1228. Tiitinen, H., May, P., Reinikainen, K., Näätänen, R., 1994. Attentive novelty detection in humans is governed by pre-attentive sensory memory. Nature 370, 90–92. Todd, J., Mullens, D., 2011. Implementing conditional inference in the auditory system: what matters? Psychophysiology 48, 1434–1443. Todd, J., Robinson, J., 2010. The use of conditional inference to reduce prediction error — a mismatch negativity (MMN) study. Neuropsychologia 48, 3009–3018.

P. Paavilainen / International Journal of Psychophysiology 88 (2013) 109–123 Todd, J., Myers, R., Pirillo, R., Drysdale, K., 2010. Neurophysiological correlates of auditory perceptual inference: a mismatch negativity (MMN) study. Brain Research 1310, 113–123. Treisman, A., Gelade, G., 1980. A feature-integration theory of attention. Cognitive Psychology 12, 97–136. van Zuijen, T.L., Sussman, E., Winkler, I., Näätänen, R., Tervaniemi, M., 2005. Auditory organization of sound sequences by a temporal or numerical regularity — a mismatch negativity study comparing musicians and non-musicians. Cognitive Brain Research 23, 270–276. van Zuijen, T.L., Simoens, V.I., Paavilainen, P., Näätänen, R., Tervaniemi, M., 2006. Implicit, intuitive, and explicit knowledge of abstract regularities in a sound sequence: an eventrelated brain potential study. Journal of Cognitive Neuroscience 18, 1292–1303. Virtala, P., Berg, V., Kivioja, M., Purhonen, J., Salmenkivi, M., Paavilainen, P., Tervaniemi, M., 2011. The preattentive processing of major vs. minor chords in the human brain: a event-related potential study. Neuroscience Letters 487, 406–410. Virtala, P., Huotilainen, M., Putkinen, V., Makkonen, T., Tervaniemi, M., 2012. Musical training facilitates the neural discrimination of major vs. minor chords in 13-year-old children. Psychophysiology 49, 1125–1132. Vuust, P., Brattico, E., Glerean, E., Seppänen, M., Pakarinen, S., Tervaniemi, M., Näätänen, R., 2011. New fast mismatch negativity paradigm for determining the neural prerequisites for musical ability. Cortex 47, 1091–1098. Wacongne, C., Labyt, E., van Wassenhove, V., Bekinschtein, T., Naccache, L., Dehaene, S., 2011. Evidence for a hierarchy of predictions and prediction errors in human cortex. Proceedings of the National Academy of Sciences of the United States of America 108, 20754–20759. Wacongne, C., Changeaux, J.-P., Dehaene, S., 2012. A neuronal model of predictive coding accounting for the mismatch negativity. The Journal of Neuroscience 32, 3665–3678.

123

Wang, X.-D., Gu, F., He, K., Chen, L.-H., Chen, L., 2012. Preattentive extraction of abstract auditory rules in speech sound stream: a mismatch negativity study using lexical tones. PloS One 7, e30027. Winkler, I., 2007. Interpreting the mismatch negativity. Journal of Psychophysiology 21, 147–163. Winkler, I., Cowan, N., 2005. From sensory to long term memory: evidence from auditory memory reactivation studies. Experimental Psychology 2, 3–20. Winkler, I., Czigler, I., 2012. Evidence from auditory and visual event-related potential (ERP) studies of deviance detection (MMN and vMMN) linking predictive coding theories and perceptual object representations. International Journal of Psychophysiology 83, 132–143. Winkler, I., Karmos, G., Näätänen, R., 1996. Adaptive modelling of the unattended acoustic environment reflected in the mismatch negativity event-related potential. Brain Research 742, 239–252. Winkler, I., Czigler, I., Sussman, E., Horváth, J., Balazs, L., 2005. Preattentive binding of auditory and visual stimulus features. Journal of Cognitive Neuroscience 17, 320–339. Winkler, I., van Zuijen, T.L., Sussman, E., Horváth, J., Näätänen, R., 2006. Object representation in the human auditory system. The European Journal of Neuroscience 24, 625–634. Winkler, I., Denham, S.L., Nelken, I., 2009. Modelling the auditory scene: predictive regularity representations and perceptual objects. Trends in Cognitive Sciences 13, 532–540. Wolff, C., Schröger, E., 2001. Activation of the auditory pre-attentive change detection system by tone repetitions with fast rate. Cognitive Brain Research 10, 323–327. Zachau, S., Rinker, T., Körner, B., Kohls, G., Maas, V., Henninghausen, K., Schecker, M., 2005. Extracting rules: early and late mismatch negativity to tone patterns. Neuroreport 16, 2015–2019.