- Email: [email protected]
Consonant chords stimulate higher EEG gamma activity than dissonant chords
Neuroscience Letters 488 (2011) 101–105
Contents lists available at ScienceDirect
Neuroscience Letters journal homepage: www.elsevier.com/locate/neulet
Consonant chords stimulate higher EEG gamma activity than dissonant chords Jin Young Park a , Haeil Park c , Joong-il Kim b , Hae-Jeong Park a,b,c,∗ a
Department of Psychiatry and Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea BK21 Project for Medical Science, Yonsei University, College of Medicine, Seoul, Republic of Korea c Department of Radiology, Nuclear Medicine and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Republic of Korea b
a r t i c l e
i n f o
Article history: Received 10 June 2010 Received in revised form 28 October 2010 Accepted 3 November 2010 Keywords: Gamma band activity Consonance chord Dissonance chord Auditory coherent percept Auditory Gestalt EEG
a b s t r a c t We examined the perceptions of consonant and dissonant chords to test auditory coherent percepts that are related to gamma oscillation. Consonant chords have coherent auditory properties due to the physical relationships of their components, in contrast to dissonant chords. EEGs were measured on 18 subjects with no musical expertise while they listened to consonant chords, dissonant chords, and single-note sounds and counted the number of single tones they heard. Induced gamma band activity was observed over the right brain hemisphere 170 ms after the onset of stimuli. The induced gamma activity was significantly increased while listening to consonant chords as compared to dissonant chords. Our results suggest that the neural activity of the gamma frequency bands may reflect an auditory coherent percept generated from physical relationships of sounds. © 2010 Elsevier Ireland Ltd. All rights reserved.
According to the temporal binding theory [26], perceptual binding is implemented by the synchronization of neuronal discharges corresponding to stimuli that share physical properties and eventually lead to a coherent percept. This perceptual binding has been substantiated by neural synchronization in the gamma frequency band (30–80 Hz) in animal and human studies [8]. The perception of coherent objects is specifically associated with “induced” gamma band activity, which is not phase-locked to stimulus transients in the study of human visual perception [27]. So far, many studies have documented the induced gamma band activity for coherent visual stimuli such as Kanizsa-type illusory contours [29], coherently moving bars [21], and meaningful ambiguous stimuli [23]. Such coherence-related gamma band activity also appears in the infant brain when babies are old enough to perceive stimuli as a Gestalt [6]. Compared to studies with the visual stimuli, research on the gamma band activity for the auditory coherent perception is rarely found in previous studies [16,24]. Most auditory experiments on coherent perception were conducted using event-related potentials (ERPs), which were found to be increased for mistuned stimuli compared to harmonic complex tones [1,2,7]. In a study, Knief et al. [16] conducted an auditory experiment using coherent and non-
∗ Corresponding author at: Department of Radiology and Psychiatry, Yonsei University College of Medicine, 134 Shinchon-dong, Seodaemun-gu, Seoul 120-749, Republic of Korea. Tel.: +82 2 2228 2363; fax: +82 2 393 3035. E-mail addresses: [email protected], [email protected], [email protected] (H.-J. Park). 0304-3940/$ – see front matter © 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.neulet.2010.11.011
coherent stimuli by combining pure sinusoidal tones as an analogue to the visual Kanizsa experiment. However, they did not find any significant difference in the induced gamma activity; instead, they observed a difference in the evoked gamma features for coherent stimuli compared to non-coherent stimuli [16]. In contrast to pure tones, the perception of musical instrument tones, characterized by spectral energy distribution and temporal waveform, has a different neurophysiological basis [18,25,32]. In the musical system, instrumental tones (i.e., “notes” in the music) constitute tonal chords. They can be categorized as consonant and dissonant according to their coherence properties [9]. For example, the existence of the perfect fifth can be a factor for differentiating consonant and dissonant chords. Major (e.g. C, E, and G) and minor (e.g. C, E , and G) chords that contain the perfect fifth are regarded as more consonant, while augmented (e.g. C, E, and G# ) and diminished (e.g. C, E , and G ) chords that do not have the perfect fifth are dissonant. Perception of consonant chords in music is thought to include associative processing of pitch relationships such as Gestalt grouping [4,17,30,32]. The fundamentals and harmonics of combined tones, which do not have coherent properties, cause a breakdown of the percept of a unitary sound, thereby leading to the impression that separate tones are heard [20]. This auditory dissociation appears in the perception of dissonant chords [22]. This indicates that the perceptual mechanisms of consonant and dissonant chords differ from each other, especially in the Gestalt processing. The current study was designed to examine the difference between consonant and dissonant chords using induced gamma activities, which are known to reflect perceptual binding in visual
102
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105
modality [8,23,27,29]. We hypothesized that consonant chords, which have a higher degree of coherence [3], would elicit increased induced gamma activity compared to dissonant chords. Eighteen subjects participated in this study (nine females; mean age 23.39 ± 1.97) under the ethics guidelines of the Institutional Review Board at Yonsei University and the Declaration of Helsinki (World Medical Association: Ethical Principles for Medical Research Involving Human Subjects, 1964). All participants were free of neurological or psychiatric disorders, had normal auditory capabilities, and were right-handed. None of the participants were musicians or experts in music. All participants gave informed consent. In total, there were 35 acoustic stimuli consisting of 7 single notes, 14 consonant chords, and 14 dissonant chords. Consonant and dissonant chords were composed of three notes chosen on the basis of the Western classical theory of harmony. Consonant chords are composed of major chords and minor chords. A minor chord was produced by reducing the second note of a major chord by one semi-tone. Dissonant chords are made of augmented chords and diminished chords. An augmented chord was created by increasing the third note of a major chord by one semi tone, while a diminished chord was produced by decreasing the third note of a minor chord
by one semi-tone. The root position of each chord was located at A3, B3, C4, D4, E4, F4, and G4 [9]. The single notes are A3, B3, C4, D4, E4, F4, and G4. All sound stimuli were generated as piano sounds using NoteWorthy Composer (NoteWorthy Software, Inc., USA) and edited using Cool Edit Pro® (Syntrillioum Software, USA) (Fig. 1). We used E-prime® software (Psychology Software Tools, USA) to present stimuli. Participants were instructed to listen to the presented sound stimuli carefully, to count the number of single note sounds, and to fix their eyes on a small cross in the display monitor. Finally, they were requested to report the number of targets to keep them concentrating on the task. There were five total blocks with 105 stimuli per block. Each stimulus was presented for 700 ms, and inter-stimulus intervals ranged from 2000 ms to 3000 ms. All types of stimuli appeared randomly with equal probability. Stimuli were presented binaurally via headphones with sound levels adjusted individually to a comfortable volume. All participants reported hearing an average of 101.56 (standard deviation 6.27) single note sounds in response to hearing a total of 105 single note sounds. Electroencephalograms (EEGs) were recorded using a GRASS 15A54 amplifier (Grass Technologies, USA) with 21 sintered Au/Agelectrodes. The electrode locations, according to the international
Fig. 1. Consonant and dissonant chords. (a) Consonant chords and dissonant chords used in the current study. Consonant chords are divided into major and minor chords while dissonant chords are divided into augmented and diminished chords. (b) Waveforms of a consonant chord (Do, Mi, and Sol) and a dissonant chord (Do, Mi, and Sol#). (c) Spectrums and spectrograms of a consonant chord and a dissonant chord in (b). The left figure of (c) is for a consonant major chord and the right figure is for a dissonant augmented chord in (b). Frequencies below 1200 Hz were displayed for illustrative purpose. The spectrums and spectrograms are driven using Praat: doing phonetics by computer (version 5.1) available from http://www.praat.org.
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105
103
Fig. 2. Induced gamma band response in the right hemisphere. Time-frequency analysis of gamma oscillation induced by a consonant chord (top) and a dissonant chord (middle). Grand average of mean induced gamma band amplitude (averaged over 30–60 Hz) for consonant and dissonant chords (bottom). Gamma band activity at around 170 ms was more increased when hearing a consonant chord compared to when hearing a dissonant chord.
10–20 system, were as follows: AFz, Fp1, Fp2, Fz, F3, F4, F7, F8, Cz, C3, C4, T3, T4, Pz, P3, P4, T5, T6, Oz, O1 and O2. We also placed an electrode on each mastoid for the linked reference and a ground electrode at nasion. Eye movement activity was monitored with two additional electrodes placed supra-orbitally to both eyes and was referenced to the linked mastoids. Electrode impedances were kept below 10 k prior to data acquisition. EEG was sampled at 1000 Hz (analogue band-pass filter 0.1–100 Hz) and then stored for off-line analysis. Data were epoched from 500 ms prestimulus to 1500 ms poststimulus. Epochs containing eye-movements or other artifacts (maximum amplitude ±70 V or electrode drifts) were rejected. To investigate the power of oscillatory activity, the EEG signals were convolved with Morlet wavelets [28]. The Morlet-convolved signal shows a Gaussian envelope with a temporal standard deviation ( t ) and a spectral standard deviation ( f = 1/(2 t )) around its central frequency (f0 ):
(t, f ) = A exp(i2ft) exp
−t 2 2t 2
.
Wavelet functions should be normalized prior to convolution to provide an energy unit at all scales; for the Morlet wavelet transformation, the normalization parameter A is t −1/2 −1/4 . A wavelet family is characterized by a constant ratio (f0 / f ). We used a wavelet family with 7 as its constant ratio [28] and f0 ranging from 30 to 60 Hz in 1 Hz increments. To avoid cancelling out non-phase-locked activity when determining the mean, each single trial was first wavelet-transformed; these transformed data were subsequently squared for computing power of activity. In other words, the power [P(t, f)] of the signal in a frequency band is the square norm of the result of the convolution of a complex wavelet [(t, f)] with the signal [s(t)] [28]:
2
P(t, f ) = (t, f ) × s(t) . For analysis of gamma oscillation, we divided the brain into four regions of interest (ROI): anterior (AFz, Fz, F3, and F4), left lateral (F7, C3, T7, and P7), right lateral (F8, C4, T8, and P8), and posterior (Pz, Oz, O1, and O2) to examine the effects on the different
brain locations [5,15]. We performed a baseline correction in the prestimulus time window from 200 ms to 50 ms. For statistical analyses, we performed a two-way repeated measures ANOVA with “stimulus” (consonant chord vs. dissonant chord) and “ROI” (anterior vs. left lateral vs. right lateral vs. posterior) as the factors. If necessary, Greenhouse Geisser corrections were applied to the ANOVA analysis. The alpha level for statistical tests was determined as 0.05. All statistical analyses were carried out with the Statistical Package for the Social Sciences (SPSS) software ver. 13.0® (SPSS Inc., USA). The time-frequency plot of the induced gamma bands averaged across all channel and chord conditions revealed peaks at around 168 ms, 40 Hz, as shown in Fig. 2. For further analysis, maximum amplitudes of induced gamma activity in a frequency of 30–60 Hz in a time window around 100–250 ms after stimulus onset entered the repeated measures ANOVA (Fig. 3). The repeated measures ANOVA showed the significant main effect of stimulus type, (F(1,17) = 4.573, p = 0.047) due to enhanced activity in the gamma band by the consonant chord over the dissonant chord. The main effect of the ROI (F(1,17) = 0.265, n.s.) was not significant, but an interaction between the stimulus and the ROI (F(1,17) = 2.417, p = 0.087) tended to exist, although not significant. To clarify in which location the difference occurred, we performed paired t-tests for four regions with a Bonferroni correction and found that the induced gamma band was significantly more activated by consonant chords than dissonant chords in the right lateral region (t(17) = 3.336, p = 0.004) (Fig. 2). In addition, there is a greater tendency for increased gamma band activation in the anterior region (t(17) = 2.333, p = 0.032) by the consonant chord than the dissonant chord. No significant difference was found in the left lateral region (t(17) = 0.451, n.s.) or the posterior region (t(17) = −0.145, n.s.). We also analyzed the evoked gamma oscillation, which is phaselocked to the stimuli, but we did not observe any statistically significant differences. In this study, we found that the induced gamma activity was more pronounced when listening to a consonant chord (a coherent auditory object) than when listening to a dissonant chord. We attribute this difference in induced gamma activities between
104
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105
Fig. 3. Induced gamma band activity in response to consonant and dissonant chords at 30–60 Hz and 100–250 ms, and their statistical difference t-value map.
consonant and dissonant chords to different degrees of stimuli coherence, and thereby different degrees of perceptual binding. The peak time of induced gamma oscillations around 170 ms after stimulus onset in this study corresponds to the period of auditory Gestalt formation, such as processing of detailed pitch relations in musical chords, around 100–200 ms [17]. The induced gamma band activities in this time period are generally observed in the binding of coherent stimuli such as the visual binding [35] and the binding of multimodal stimuli [34]. The differences between perceptions of consonant and dissonant chords were significant in the lateral part of the right hemisphere. This result is consistent with functional magnetic resonance imaging studies of the non-musician group that showed more activation in the right hemisphere for consonant chords compared to dissonant chords [11,19]. The right hemisphere has also been suggested to play a role in the “Gestalt Formation” such as the auditory Gestalt processing [14,33] and global Gestalt image formation [12]. Furthermore, the frontal lobe, where increased gamma activity by consonant chords was observed, is also involved in the music perception (i.e., auditory sensory memory, auditory Gestalt formation, working memory pitch) in connection with the temporal lobe [17]. In Western music, various theories have been introduced to explain the difference between consonance and dissonance. Pythagoras’ rule, a major theory in history, proposes the frequency ratio as a determinant for the harmonicity. Helmholtz explained musical dissonance in terms of the sensation of “beats” and “roughness” [13]. It occurs when two or more concurrent components of a complex sound cannot be resolved due to its limited bandwidth associated with the mechanics of the basilar membrane. However, this psychoacoustic model cannot explain the perception of dissonance when tones are presented dichotically [3]. Although there exists disagreement about the detail of the underlying neural mechanisms, multilevel neural processing is observed in the perception of consonance. Multilevel neural processing occurs in early auditory processing stages from the auditory nerve [31] to the inferior colliculus [3] to the primary auditory cortex [9], which extract acoustic features such as pitch and roughness. This feature extraction from rough to specific is processed up to 100 ms after stimulus onset according to ERP studies [17]. Then, the acoustic information enters an auditory Gestalten stage in which a more detailed processing of the pitch relations between the tones of a chord is achieved, mainly at the auditory cortex in association with the frontal cortex around 100–200 ms after stimulus [17]. In our study, increased gamma band activity by consonant chords observed at the right temporal and frontal cortex around 170 ms may imply perceptual binding at a more central and later stage of the neural processing, which may lead to the perception of tonal consonance. In contrast to this study, Knief et al. [16] did not find induced gamma band activity for coherent tones composed of pure sinusoidal tones. This inconsistency may be due to the differences in the stimuli used in the two studies. Chords of piano tones were used in our study, whereas coherent tones composed of pure tones were
used in the study by Knief et al. [16]. Chords of piano tones used in the current study have overtones and envelope, and may require a more complex auditory binding process than combinations of pure tones. Due to the additional neural input caused by frequency bands below or above the fundamental frequency, the piano tones are considered to elicit stronger neurophysiologic activities [18] and have different neurophysiological responses compared to pure tones [10,25]. Most of the gamma band studies based on the Gestalt theory were done using visual modality [8,23,27,29]. The current study shows that induced gamma activity is observed not only in the visual Gestalt processing but also in the auditory Gestalt processing. This could provide a clue to the recent debate regarding the nature of gamma band activity whether gamma oscillation in visual cognitive processing, where it is tightly time-locked to the onset of involuntary miniature eye movements, reflects a saccadic spike potential or not [36]. Yuval-Greenberg et al. [36] proposed that the gamma oscillations generated by these eye movements are affected by the characteristics of the visual stimuli and that these differences in the characteristics lead to differences in the induced gamma band response. Since we used the auditory stimuli while the subjects were staring at a fixed point across the whole task, ocular movement-related factors different for two conditions (consonant and dissonant), including difference in the nature of visual stimuli, are not considered to severely affect the current study. Therefore, the current result supports the notion that induced gamma band response may reflect the ‘neural activity’ (not miniature saccadic dynamics) related to the coherent perception. To sum up, we found a significant increase in induced gamma oscillation for the consonant chords compared to dissonant chords. This result indicates that induced gamma band activity represents general perceptual binding, not only by the visual coherent percept but also by the auditory coherent percept. Our results shed light on clarifying the relationship between central object processing and gamma oscillation. Acknowledgements This work was supported by a grant (R0120050001052202006) from the Basic Research Program of the Korea Science & Engineering Foundation. The authors would like to thank Byoung-Kyoung Min, Ph.D. and Il Keun Lee, M.D. for their help with experiments. References [1] C. Alain, S.R. Arnott, T.W. Picton, Bottom-up and top-down influences on auditory scene analysis: evidence from event-related brain potentials, J. Exp. Psychol. Hum. Percept. Perform. 27 (2001) 1072–1089. [2] C. Alain, B.M. Schuler, K.L. McDonald, Neural activity associated with distinguishing concurrent auditory objects, J. Acoust. Soc. Am. 111 (2002) 990–995. [3] G.M. Bidelman, A. Krishnan, Neural correlates of consonance, dissonance, and the hierarchy of musical pitch in the human brainstem, J. Neurosci. 29 (2009) 13165–13171. [4] A.S Bregman, Auditory Scene Analysis: The Perceptual Organization of Sound, The MIT Press, London, 1994.
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105 [5] N.A. Busch, C.S. Herrmann, M.M. Muller, D. Lenz, T. Gruber, A cross-laboratory study of event-related gamma activity in a standard object recognition paradigm, Neuroimage 33 (2006) 1169–1177. [6] G. Csibra, G. Davis, M.W. Spratling, M.H. Johnson, Gamma oscillations and object processing in the infant brain, Science 290 (2000) 1582–1585. [7] B.J. Dyson, C. Alain, Representation of concurrent acoustic objects in primary auditory cortex, J. Acoust. Soc. Am. 115 (2004) 280–288. [8] A.K. Engel, W. Singer, Temporal binding and the neural correlates of sensory awareness, Trends Cogn. Sci. 5 (2001) 16–25. [9] Y.I. Fishman, D.H. Reser, J.C. Arezzo, M. Steinschneider, Complex tone processing in primary auditory cortex of the awake monkey. I. Neural ensemble correlates of roughness, J. Acoust. Soc. Am. 108 (2000) 235– 246. [10] Y.I. Fishman, D.H. Reser, J.C. Arezzo, M. Steinschneider, Complex tone processing in primary auditory cortex of the awake monkey. II. Pitch versus critical band representation, J. Acoust. Soc. Am. 108 (2000) 247–262. [11] A.H. Foss, E.L. Altschuler, K.H. James, Neural correlates of the Pythagorean ratio rules, Neuroreport 18 (2007) 1521–1525. [12] S. Gardini, R. De Beni, C. Cornoldi, A. Bromiley, A. Venneri, Different neuronal pathways support the generation of general and specific mental images, Neuroimage 27 (2005) 544–552. [13] H.L.F. Helmholtz, On the Sensations of Tone as a Physiological Basis for the Theory of Music, Dover Publications, New York, 1954. [14] C.S. Herrmann, A.D. Friederici, U. Oertel, B. Maess, A. Hahne, K. Alter, The brain generates its own sentence melody: a Gestalt phenomenon in speech perception, Brain Lang. 85 (2003) 396–401. [15] A. Keil, M.M. Muller, W.J. Ray, T. Gruber, T. Elbert, Human gamma band activity and perception of a gestalt, J. Neurosci. 19 (1999) 7152–7161. [16] A. Knief, M. Schulte, O. Bertran, C. Pantev, The perception of coherent and noncoherent auditory objects: a signature in gamma frequency band, Hear. Res. 145 (2000) 161–168. [17] S. Koelsch, W.A. Siebel, Towards a neural basis of music perception, Trends Cogn. Sci. 9 (2005) 578–584. [18] B. Lutkenhoner, A. Seither-Preisler, S. Seither, Piano tones evoke stronger magnetic fields than pure tones or noise, both in musicians and non-musicians, Neuroimage 30 (2006) 927–937. [19] L Minati, C. Rosazza, L. D’Incerti, E. Pietrocini, L. Valentini, V. Scaioli, C. Loveday, M.G. Bruzzone, Functional MRI/event-related potential study of sensory consonance and dissonance in musicians and nonmusicians, Neuroreport 20 (2009) 87–92. [20] B.C. Moore, B.R. Glasberg, R.W. Peters, Thresholds for hearing mistuned partials as separate tones in harmonic complexes, J. Acoust. Soc. Am. 80 (1986) 479– 483.
105
[21] M.M. Muller, M. Junghofer, T. Elbert, B. Rochstroh, Visually induced gammaband responses to coherent and incoherent motion: a replication study, Neuroreport 8 (1997) 2575–2579. [22] N. Passynkova, H. Neubauer, H. Scheich, Spatial organization of EEG coherence during listening to consonant and dissonant chords, Neurosci. Lett. 412 (2007) 6–11. [23] E. Rodriguez, N. George, J.P. Lachaux, J. Martinerie, B. Renault, F.J. Varela, Perception’s shadow: long-distance synchronization of human brain activity, Nature 397 (1999) 430–433. [24] M. Schulte, A. Knief, A. Seither-Preisler, C. Pantev, Different modes of pitch perception and learning-induced neuronal plasticity of the human auditory cortex, Neural Plast. 9 (2002) 161–175. [25] A. Seither-Preisler, K. Krumbholz, B. Lutkenhoner, Sensitivity of the neuromagnetic N100m deflection to spectral bandwidth: a function of the auditory periphery? Audiol. Neurootol. 8 (2003) 322–337. [26] W. Singer, Neuronal synchrony: a versatile code for the definition of relations? Neuron 24 (49-65) (1999) 111–125. [27] C. Tallon-Baudry, O. Bertrand, Oscillatory gamma activity in humans and its role in object representation, Trends Cogn. Sci. 3 (1999) 151–162. [28] C. Tallon-Baudry, O. Bertrand, C. Delpuech, J. Permier, Oscillatory gamma-band (30–70 Hz) activity induced by a visual search task in humans, J. Neurosci. 17 (1997) 722–734. [29] C. Tallon-Baudry, O. Bertrand, C. Delpuech, J. Pernier, Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human, J. Neurosci. 16 (1996) 4240–4249. [30] E. Terhardt, The concept of musical consonance: an link between music and pyshoacoustics, Music Perception 1 (1984) 276–295. [31] M.J. Tramo, P.A. Cariani, B. Delgutte, Representation of tonal consonance and dissonance in the temporal firing patterns of auditory nerve fibers, Soc. Neurosci. 18 (1992) 382. [32] M.J. Tramo, P.A. Cariani, B. Delgutte, L.D. Braida, Neurobiological foundations for the theory of harmony in western tonal music, Ann. N. Y. Acad. Sci. 930 (2001) 92–116. [33] W. Wapner, S. Hamby, H. Gardner, The role of the right hemisphere in the apprehension of complex linguistic materials, Brain Lang. 14 (1981) 15–33. [34] A. Widmann, T. Gruber, T. Kujala, M. Tervaniemi, E. Schroger, Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity, Cereb. Cortex 17 (2007) 2696–2702. [35] X. Wu, D. Zhang, Early induced beta/gamma activity during illusory contour perception, Neurosci. Lett. 462 (2009) 244–247. [36] S. Yuval-Greenberg, O. Tomer, A.S. Keren, I. Nelken, L.Y. Deouell, Transient induced gamma-band response in EEG as a manifestation of miniature saccades, Neuron 58 (2008) 429–441.
Contents lists available at ScienceDirect
Neuroscience Letters journal homepage: www.elsevier.com/locate/neulet
Consonant chords stimulate higher EEG gamma activity than dissonant chords Jin Young Park a , Haeil Park c , Joong-il Kim b , Hae-Jeong Park a,b,c,∗ a
Department of Psychiatry and Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea BK21 Project for Medical Science, Yonsei University, College of Medicine, Seoul, Republic of Korea c Department of Radiology, Nuclear Medicine and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Republic of Korea b
a r t i c l e
i n f o
Article history: Received 10 June 2010 Received in revised form 28 October 2010 Accepted 3 November 2010 Keywords: Gamma band activity Consonance chord Dissonance chord Auditory coherent percept Auditory Gestalt EEG
a b s t r a c t We examined the perceptions of consonant and dissonant chords to test auditory coherent percepts that are related to gamma oscillation. Consonant chords have coherent auditory properties due to the physical relationships of their components, in contrast to dissonant chords. EEGs were measured on 18 subjects with no musical expertise while they listened to consonant chords, dissonant chords, and single-note sounds and counted the number of single tones they heard. Induced gamma band activity was observed over the right brain hemisphere 170 ms after the onset of stimuli. The induced gamma activity was significantly increased while listening to consonant chords as compared to dissonant chords. Our results suggest that the neural activity of the gamma frequency bands may reflect an auditory coherent percept generated from physical relationships of sounds. © 2010 Elsevier Ireland Ltd. All rights reserved.
According to the temporal binding theory [26], perceptual binding is implemented by the synchronization of neuronal discharges corresponding to stimuli that share physical properties and eventually lead to a coherent percept. This perceptual binding has been substantiated by neural synchronization in the gamma frequency band (30–80 Hz) in animal and human studies [8]. The perception of coherent objects is specifically associated with “induced” gamma band activity, which is not phase-locked to stimulus transients in the study of human visual perception [27]. So far, many studies have documented the induced gamma band activity for coherent visual stimuli such as Kanizsa-type illusory contours [29], coherently moving bars [21], and meaningful ambiguous stimuli [23]. Such coherence-related gamma band activity also appears in the infant brain when babies are old enough to perceive stimuli as a Gestalt [6]. Compared to studies with the visual stimuli, research on the gamma band activity for the auditory coherent perception is rarely found in previous studies [16,24]. Most auditory experiments on coherent perception were conducted using event-related potentials (ERPs), which were found to be increased for mistuned stimuli compared to harmonic complex tones [1,2,7]. In a study, Knief et al. [16] conducted an auditory experiment using coherent and non-
∗ Corresponding author at: Department of Radiology and Psychiatry, Yonsei University College of Medicine, 134 Shinchon-dong, Seodaemun-gu, Seoul 120-749, Republic of Korea. Tel.: +82 2 2228 2363; fax: +82 2 393 3035. E-mail addresses: [email protected], [email protected], [email protected] (H.-J. Park). 0304-3940/$ – see front matter © 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.neulet.2010.11.011
coherent stimuli by combining pure sinusoidal tones as an analogue to the visual Kanizsa experiment. However, they did not find any significant difference in the induced gamma activity; instead, they observed a difference in the evoked gamma features for coherent stimuli compared to non-coherent stimuli [16]. In contrast to pure tones, the perception of musical instrument tones, characterized by spectral energy distribution and temporal waveform, has a different neurophysiological basis [18,25,32]. In the musical system, instrumental tones (i.e., “notes” in the music) constitute tonal chords. They can be categorized as consonant and dissonant according to their coherence properties [9]. For example, the existence of the perfect fifth can be a factor for differentiating consonant and dissonant chords. Major (e.g. C, E, and G) and minor (e.g. C, E , and G) chords that contain the perfect fifth are regarded as more consonant, while augmented (e.g. C, E, and G# ) and diminished (e.g. C, E , and G ) chords that do not have the perfect fifth are dissonant. Perception of consonant chords in music is thought to include associative processing of pitch relationships such as Gestalt grouping [4,17,30,32]. The fundamentals and harmonics of combined tones, which do not have coherent properties, cause a breakdown of the percept of a unitary sound, thereby leading to the impression that separate tones are heard [20]. This auditory dissociation appears in the perception of dissonant chords [22]. This indicates that the perceptual mechanisms of consonant and dissonant chords differ from each other, especially in the Gestalt processing. The current study was designed to examine the difference between consonant and dissonant chords using induced gamma activities, which are known to reflect perceptual binding in visual
102
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105
modality [8,23,27,29]. We hypothesized that consonant chords, which have a higher degree of coherence [3], would elicit increased induced gamma activity compared to dissonant chords. Eighteen subjects participated in this study (nine females; mean age 23.39 ± 1.97) under the ethics guidelines of the Institutional Review Board at Yonsei University and the Declaration of Helsinki (World Medical Association: Ethical Principles for Medical Research Involving Human Subjects, 1964). All participants were free of neurological or psychiatric disorders, had normal auditory capabilities, and were right-handed. None of the participants were musicians or experts in music. All participants gave informed consent. In total, there were 35 acoustic stimuli consisting of 7 single notes, 14 consonant chords, and 14 dissonant chords. Consonant and dissonant chords were composed of three notes chosen on the basis of the Western classical theory of harmony. Consonant chords are composed of major chords and minor chords. A minor chord was produced by reducing the second note of a major chord by one semi-tone. Dissonant chords are made of augmented chords and diminished chords. An augmented chord was created by increasing the third note of a major chord by one semi tone, while a diminished chord was produced by decreasing the third note of a minor chord
by one semi-tone. The root position of each chord was located at A3, B3, C4, D4, E4, F4, and G4 [9]. The single notes are A3, B3, C4, D4, E4, F4, and G4. All sound stimuli were generated as piano sounds using NoteWorthy Composer (NoteWorthy Software, Inc., USA) and edited using Cool Edit Pro® (Syntrillioum Software, USA) (Fig. 1). We used E-prime® software (Psychology Software Tools, USA) to present stimuli. Participants were instructed to listen to the presented sound stimuli carefully, to count the number of single note sounds, and to fix their eyes on a small cross in the display monitor. Finally, they were requested to report the number of targets to keep them concentrating on the task. There were five total blocks with 105 stimuli per block. Each stimulus was presented for 700 ms, and inter-stimulus intervals ranged from 2000 ms to 3000 ms. All types of stimuli appeared randomly with equal probability. Stimuli were presented binaurally via headphones with sound levels adjusted individually to a comfortable volume. All participants reported hearing an average of 101.56 (standard deviation 6.27) single note sounds in response to hearing a total of 105 single note sounds. Electroencephalograms (EEGs) were recorded using a GRASS 15A54 amplifier (Grass Technologies, USA) with 21 sintered Au/Agelectrodes. The electrode locations, according to the international
Fig. 1. Consonant and dissonant chords. (a) Consonant chords and dissonant chords used in the current study. Consonant chords are divided into major and minor chords while dissonant chords are divided into augmented and diminished chords. (b) Waveforms of a consonant chord (Do, Mi, and Sol) and a dissonant chord (Do, Mi, and Sol#). (c) Spectrums and spectrograms of a consonant chord and a dissonant chord in (b). The left figure of (c) is for a consonant major chord and the right figure is for a dissonant augmented chord in (b). Frequencies below 1200 Hz were displayed for illustrative purpose. The spectrums and spectrograms are driven using Praat: doing phonetics by computer (version 5.1) available from http://www.praat.org.
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105
103
Fig. 2. Induced gamma band response in the right hemisphere. Time-frequency analysis of gamma oscillation induced by a consonant chord (top) and a dissonant chord (middle). Grand average of mean induced gamma band amplitude (averaged over 30–60 Hz) for consonant and dissonant chords (bottom). Gamma band activity at around 170 ms was more increased when hearing a consonant chord compared to when hearing a dissonant chord.
10–20 system, were as follows: AFz, Fp1, Fp2, Fz, F3, F4, F7, F8, Cz, C3, C4, T3, T4, Pz, P3, P4, T5, T6, Oz, O1 and O2. We also placed an electrode on each mastoid for the linked reference and a ground electrode at nasion. Eye movement activity was monitored with two additional electrodes placed supra-orbitally to both eyes and was referenced to the linked mastoids. Electrode impedances were kept below 10 k prior to data acquisition. EEG was sampled at 1000 Hz (analogue band-pass filter 0.1–100 Hz) and then stored for off-line analysis. Data were epoched from 500 ms prestimulus to 1500 ms poststimulus. Epochs containing eye-movements or other artifacts (maximum amplitude ±70 V or electrode drifts) were rejected. To investigate the power of oscillatory activity, the EEG signals were convolved with Morlet wavelets [28]. The Morlet-convolved signal shows a Gaussian envelope with a temporal standard deviation ( t ) and a spectral standard deviation ( f = 1/(2 t )) around its central frequency (f0 ):
(t, f ) = A exp(i2ft) exp
−t 2 2t 2
.
Wavelet functions should be normalized prior to convolution to provide an energy unit at all scales; for the Morlet wavelet transformation, the normalization parameter A is t −1/2 −1/4 . A wavelet family is characterized by a constant ratio (f0 / f ). We used a wavelet family with 7 as its constant ratio [28] and f0 ranging from 30 to 60 Hz in 1 Hz increments. To avoid cancelling out non-phase-locked activity when determining the mean, each single trial was first wavelet-transformed; these transformed data were subsequently squared for computing power of activity. In other words, the power [P(t, f)] of the signal in a frequency band is the square norm of the result of the convolution of a complex wavelet [(t, f)] with the signal [s(t)] [28]:
2
P(t, f ) = (t, f ) × s(t) . For analysis of gamma oscillation, we divided the brain into four regions of interest (ROI): anterior (AFz, Fz, F3, and F4), left lateral (F7, C3, T7, and P7), right lateral (F8, C4, T8, and P8), and posterior (Pz, Oz, O1, and O2) to examine the effects on the different
brain locations [5,15]. We performed a baseline correction in the prestimulus time window from 200 ms to 50 ms. For statistical analyses, we performed a two-way repeated measures ANOVA with “stimulus” (consonant chord vs. dissonant chord) and “ROI” (anterior vs. left lateral vs. right lateral vs. posterior) as the factors. If necessary, Greenhouse Geisser corrections were applied to the ANOVA analysis. The alpha level for statistical tests was determined as 0.05. All statistical analyses were carried out with the Statistical Package for the Social Sciences (SPSS) software ver. 13.0® (SPSS Inc., USA). The time-frequency plot of the induced gamma bands averaged across all channel and chord conditions revealed peaks at around 168 ms, 40 Hz, as shown in Fig. 2. For further analysis, maximum amplitudes of induced gamma activity in a frequency of 30–60 Hz in a time window around 100–250 ms after stimulus onset entered the repeated measures ANOVA (Fig. 3). The repeated measures ANOVA showed the significant main effect of stimulus type, (F(1,17) = 4.573, p = 0.047) due to enhanced activity in the gamma band by the consonant chord over the dissonant chord. The main effect of the ROI (F(1,17) = 0.265, n.s.) was not significant, but an interaction between the stimulus and the ROI (F(1,17) = 2.417, p = 0.087) tended to exist, although not significant. To clarify in which location the difference occurred, we performed paired t-tests for four regions with a Bonferroni correction and found that the induced gamma band was significantly more activated by consonant chords than dissonant chords in the right lateral region (t(17) = 3.336, p = 0.004) (Fig. 2). In addition, there is a greater tendency for increased gamma band activation in the anterior region (t(17) = 2.333, p = 0.032) by the consonant chord than the dissonant chord. No significant difference was found in the left lateral region (t(17) = 0.451, n.s.) or the posterior region (t(17) = −0.145, n.s.). We also analyzed the evoked gamma oscillation, which is phaselocked to the stimuli, but we did not observe any statistically significant differences. In this study, we found that the induced gamma activity was more pronounced when listening to a consonant chord (a coherent auditory object) than when listening to a dissonant chord. We attribute this difference in induced gamma activities between
104
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105
Fig. 3. Induced gamma band activity in response to consonant and dissonant chords at 30–60 Hz and 100–250 ms, and their statistical difference t-value map.
consonant and dissonant chords to different degrees of stimuli coherence, and thereby different degrees of perceptual binding. The peak time of induced gamma oscillations around 170 ms after stimulus onset in this study corresponds to the period of auditory Gestalt formation, such as processing of detailed pitch relations in musical chords, around 100–200 ms [17]. The induced gamma band activities in this time period are generally observed in the binding of coherent stimuli such as the visual binding [35] and the binding of multimodal stimuli [34]. The differences between perceptions of consonant and dissonant chords were significant in the lateral part of the right hemisphere. This result is consistent with functional magnetic resonance imaging studies of the non-musician group that showed more activation in the right hemisphere for consonant chords compared to dissonant chords [11,19]. The right hemisphere has also been suggested to play a role in the “Gestalt Formation” such as the auditory Gestalt processing [14,33] and global Gestalt image formation [12]. Furthermore, the frontal lobe, where increased gamma activity by consonant chords was observed, is also involved in the music perception (i.e., auditory sensory memory, auditory Gestalt formation, working memory pitch) in connection with the temporal lobe [17]. In Western music, various theories have been introduced to explain the difference between consonance and dissonance. Pythagoras’ rule, a major theory in history, proposes the frequency ratio as a determinant for the harmonicity. Helmholtz explained musical dissonance in terms of the sensation of “beats” and “roughness” [13]. It occurs when two or more concurrent components of a complex sound cannot be resolved due to its limited bandwidth associated with the mechanics of the basilar membrane. However, this psychoacoustic model cannot explain the perception of dissonance when tones are presented dichotically [3]. Although there exists disagreement about the detail of the underlying neural mechanisms, multilevel neural processing is observed in the perception of consonance. Multilevel neural processing occurs in early auditory processing stages from the auditory nerve [31] to the inferior colliculus [3] to the primary auditory cortex [9], which extract acoustic features such as pitch and roughness. This feature extraction from rough to specific is processed up to 100 ms after stimulus onset according to ERP studies [17]. Then, the acoustic information enters an auditory Gestalten stage in which a more detailed processing of the pitch relations between the tones of a chord is achieved, mainly at the auditory cortex in association with the frontal cortex around 100–200 ms after stimulus [17]. In our study, increased gamma band activity by consonant chords observed at the right temporal and frontal cortex around 170 ms may imply perceptual binding at a more central and later stage of the neural processing, which may lead to the perception of tonal consonance. In contrast to this study, Knief et al. [16] did not find induced gamma band activity for coherent tones composed of pure sinusoidal tones. This inconsistency may be due to the differences in the stimuli used in the two studies. Chords of piano tones were used in our study, whereas coherent tones composed of pure tones were
used in the study by Knief et al. [16]. Chords of piano tones used in the current study have overtones and envelope, and may require a more complex auditory binding process than combinations of pure tones. Due to the additional neural input caused by frequency bands below or above the fundamental frequency, the piano tones are considered to elicit stronger neurophysiologic activities [18] and have different neurophysiological responses compared to pure tones [10,25]. Most of the gamma band studies based on the Gestalt theory were done using visual modality [8,23,27,29]. The current study shows that induced gamma activity is observed not only in the visual Gestalt processing but also in the auditory Gestalt processing. This could provide a clue to the recent debate regarding the nature of gamma band activity whether gamma oscillation in visual cognitive processing, where it is tightly time-locked to the onset of involuntary miniature eye movements, reflects a saccadic spike potential or not [36]. Yuval-Greenberg et al. [36] proposed that the gamma oscillations generated by these eye movements are affected by the characteristics of the visual stimuli and that these differences in the characteristics lead to differences in the induced gamma band response. Since we used the auditory stimuli while the subjects were staring at a fixed point across the whole task, ocular movement-related factors different for two conditions (consonant and dissonant), including difference in the nature of visual stimuli, are not considered to severely affect the current study. Therefore, the current result supports the notion that induced gamma band response may reflect the ‘neural activity’ (not miniature saccadic dynamics) related to the coherent perception. To sum up, we found a significant increase in induced gamma oscillation for the consonant chords compared to dissonant chords. This result indicates that induced gamma band activity represents general perceptual binding, not only by the visual coherent percept but also by the auditory coherent percept. Our results shed light on clarifying the relationship between central object processing and gamma oscillation. Acknowledgements This work was supported by a grant (R0120050001052202006) from the Basic Research Program of the Korea Science & Engineering Foundation. The authors would like to thank Byoung-Kyoung Min, Ph.D. and Il Keun Lee, M.D. for their help with experiments. References [1] C. Alain, S.R. Arnott, T.W. Picton, Bottom-up and top-down influences on auditory scene analysis: evidence from event-related brain potentials, J. Exp. Psychol. Hum. Percept. Perform. 27 (2001) 1072–1089. [2] C. Alain, B.M. Schuler, K.L. McDonald, Neural activity associated with distinguishing concurrent auditory objects, J. Acoust. Soc. Am. 111 (2002) 990–995. [3] G.M. Bidelman, A. Krishnan, Neural correlates of consonance, dissonance, and the hierarchy of musical pitch in the human brainstem, J. Neurosci. 29 (2009) 13165–13171. [4] A.S Bregman, Auditory Scene Analysis: The Perceptual Organization of Sound, The MIT Press, London, 1994.
J.Y. Park et al. / Neuroscience Letters 488 (2011) 101–105 [5] N.A. Busch, C.S. Herrmann, M.M. Muller, D. Lenz, T. Gruber, A cross-laboratory study of event-related gamma activity in a standard object recognition paradigm, Neuroimage 33 (2006) 1169–1177. [6] G. Csibra, G. Davis, M.W. Spratling, M.H. Johnson, Gamma oscillations and object processing in the infant brain, Science 290 (2000) 1582–1585. [7] B.J. Dyson, C. Alain, Representation of concurrent acoustic objects in primary auditory cortex, J. Acoust. Soc. Am. 115 (2004) 280–288. [8] A.K. Engel, W. Singer, Temporal binding and the neural correlates of sensory awareness, Trends Cogn. Sci. 5 (2001) 16–25. [9] Y.I. Fishman, D.H. Reser, J.C. Arezzo, M. Steinschneider, Complex tone processing in primary auditory cortex of the awake monkey. I. Neural ensemble correlates of roughness, J. Acoust. Soc. Am. 108 (2000) 235– 246. [10] Y.I. Fishman, D.H. Reser, J.C. Arezzo, M. Steinschneider, Complex tone processing in primary auditory cortex of the awake monkey. II. Pitch versus critical band representation, J. Acoust. Soc. Am. 108 (2000) 247–262. [11] A.H. Foss, E.L. Altschuler, K.H. James, Neural correlates of the Pythagorean ratio rules, Neuroreport 18 (2007) 1521–1525. [12] S. Gardini, R. De Beni, C. Cornoldi, A. Bromiley, A. Venneri, Different neuronal pathways support the generation of general and specific mental images, Neuroimage 27 (2005) 544–552. [13] H.L.F. Helmholtz, On the Sensations of Tone as a Physiological Basis for the Theory of Music, Dover Publications, New York, 1954. [14] C.S. Herrmann, A.D. Friederici, U. Oertel, B. Maess, A. Hahne, K. Alter, The brain generates its own sentence melody: a Gestalt phenomenon in speech perception, Brain Lang. 85 (2003) 396–401. [15] A. Keil, M.M. Muller, W.J. Ray, T. Gruber, T. Elbert, Human gamma band activity and perception of a gestalt, J. Neurosci. 19 (1999) 7152–7161. [16] A. Knief, M. Schulte, O. Bertran, C. Pantev, The perception of coherent and noncoherent auditory objects: a signature in gamma frequency band, Hear. Res. 145 (2000) 161–168. [17] S. Koelsch, W.A. Siebel, Towards a neural basis of music perception, Trends Cogn. Sci. 9 (2005) 578–584. [18] B. Lutkenhoner, A. Seither-Preisler, S. Seither, Piano tones evoke stronger magnetic fields than pure tones or noise, both in musicians and non-musicians, Neuroimage 30 (2006) 927–937. [19] L Minati, C. Rosazza, L. D’Incerti, E. Pietrocini, L. Valentini, V. Scaioli, C. Loveday, M.G. Bruzzone, Functional MRI/event-related potential study of sensory consonance and dissonance in musicians and nonmusicians, Neuroreport 20 (2009) 87–92. [20] B.C. Moore, B.R. Glasberg, R.W. Peters, Thresholds for hearing mistuned partials as separate tones in harmonic complexes, J. Acoust. Soc. Am. 80 (1986) 479– 483.
105
[21] M.M. Muller, M. Junghofer, T. Elbert, B. Rochstroh, Visually induced gammaband responses to coherent and incoherent motion: a replication study, Neuroreport 8 (1997) 2575–2579. [22] N. Passynkova, H. Neubauer, H. Scheich, Spatial organization of EEG coherence during listening to consonant and dissonant chords, Neurosci. Lett. 412 (2007) 6–11. [23] E. Rodriguez, N. George, J.P. Lachaux, J. Martinerie, B. Renault, F.J. Varela, Perception’s shadow: long-distance synchronization of human brain activity, Nature 397 (1999) 430–433. [24] M. Schulte, A. Knief, A. Seither-Preisler, C. Pantev, Different modes of pitch perception and learning-induced neuronal plasticity of the human auditory cortex, Neural Plast. 9 (2002) 161–175. [25] A. Seither-Preisler, K. Krumbholz, B. Lutkenhoner, Sensitivity of the neuromagnetic N100m deflection to spectral bandwidth: a function of the auditory periphery? Audiol. Neurootol. 8 (2003) 322–337. [26] W. Singer, Neuronal synchrony: a versatile code for the definition of relations? Neuron 24 (49-65) (1999) 111–125. [27] C. Tallon-Baudry, O. Bertrand, Oscillatory gamma activity in humans and its role in object representation, Trends Cogn. Sci. 3 (1999) 151–162. [28] C. Tallon-Baudry, O. Bertrand, C. Delpuech, J. Permier, Oscillatory gamma-band (30–70 Hz) activity induced by a visual search task in humans, J. Neurosci. 17 (1997) 722–734. [29] C. Tallon-Baudry, O. Bertrand, C. Delpuech, J. Pernier, Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human, J. Neurosci. 16 (1996) 4240–4249. [30] E. Terhardt, The concept of musical consonance: an link between music and pyshoacoustics, Music Perception 1 (1984) 276–295. [31] M.J. Tramo, P.A. Cariani, B. Delgutte, Representation of tonal consonance and dissonance in the temporal firing patterns of auditory nerve fibers, Soc. Neurosci. 18 (1992) 382. [32] M.J. Tramo, P.A. Cariani, B. Delgutte, L.D. Braida, Neurobiological foundations for the theory of harmony in western tonal music, Ann. N. Y. Acad. Sci. 930 (2001) 92–116. [33] W. Wapner, S. Hamby, H. Gardner, The role of the right hemisphere in the apprehension of complex linguistic materials, Brain Lang. 14 (1981) 15–33. [34] A. Widmann, T. Gruber, T. Kujala, M. Tervaniemi, E. Schroger, Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity, Cereb. Cortex 17 (2007) 2696–2702. [35] X. Wu, D. Zhang, Early induced beta/gamma activity during illusory contour perception, Neurosci. Lett. 462 (2009) 244–247. [36] S. Yuval-Greenberg, O. Tomer, A.S. Keren, I. Nelken, L.Y. Deouell, Transient induced gamma-band response in EEG as a manifestation of miniature saccades, Neuron 58 (2008) 429–441.