Functional MRI changes before and after onset of reported emotions

Psychiatry Research: Neuroimaging 132 (2004) 239 – 250 www.elsevier.com/locate/psychresns

Functional MRI changes before and after onset of reported emotions Isak Prohovnika,b,*, Pawel Skudlarskia, Robert K. Fulbrighta, John C. Gorec, Bruce E. Wexlerd a

Department of Diagnostic Radiology, Yale University School of Medicine, New Haven, CT, USA Departments of Psychiatry and Radiology, Mount Sinai School of Medicine, New York, NY, USA c Department of Radiology, Vanderbilt University, Nashville, TN, USA d Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA

b

Received 5 April 2002; received in revised form 15 May 2003; accepted 25 March 2004

Abstract The social nature of emotion is evident in the importance of facial and vocal displays in emotion-related behavior. This is the first brain-imaging study to use simulated face-to-face social interactions to evoke emotional responses and to compare valencerelated activations before and after subjective onset of emotional response. Videotapes were prepared with actresses who described happy or unhappy experiences. Functional magnetic resonance imaging (fMRI) at 1.5 T was used to acquire BOLD images in 21 healthy young adults before, after, and during viewing of the happy and sad tapes. Subjects pushed buttons to indicate the onset of subjective emotional responses. Group data were analyzed by a bootstrap randomization method after anatomical normalization. Significant activation was detected in frontal and sensory regions prior to the reported onset of emotional response, and this activity showed a marked decrease after the report of conscious emotional experience. Significant differences between happy and sad conditions were evident in multiple brain regions both before and after the reported onset of emotional response, including the middle and superior temporal gyri, the middle frontal gyrus, the caudate, and the hippocampus. Socially relevant emotional stimulation is feasible and evokes robust responses. The neural correlates of the evoked emotion are multiple, widely distributed, and inclusive of areas important in many cognitive tasks. Positive and negative emotional responses include activation of common and distinctive brain regions. D 2004 Published by Elsevier Ireland Ltd. Keywords: Emotion; Brain imaging; MRI; Affective disorders; Subjective experience

1. Introduction * Corresponding author. Current address: Bronx VAMC, MIRECC (00MH), 130 West Kingsbridge Road, Bronx, NY 10468, USA. Tel.: +1 718 584 9000x3629; fax: +1 530 380 9377. E-mail address: [email protected] (I. Prohovnik).

Emotion is a complex multidimensional phenomenon with perceptual, expressive, and subjective components. Imaging studies of emotion-related brain

0925-4927/$ - see front matter D 2004 Published by Elsevier Ireland Ltd. doi:10.1016/j.pscychresns.2004.03.005

240

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

activation have been correspondingly rich and varied, encompassing multiple components and using a wide range of stimuli and procedures to evoke the target state (see Phan et al., 2002, for a recent review). Given the wide range of experimental paradigms, it is impressive that medial aspects of the superior frontal cortex have so consistently been found to show emotion-related activation. Such activation has been seen, for example, after presentation of film clips (Lane et al., 1997a) or slides (Lane et al., 1997a,b; Royet et al., 2000), and after self-induced emotion (Lane et al., 1997a; Teasdale et al., 1999). Amygdala activation after presentation of fear-evoking stimuli has also been consistently reported (e.g., Morris et al., 1996; Whalen et al., 1998), but here, the experimental paradigms employed in different studies have been very similar. Otherwise, results have been notable for their inconsistency, with specific activations generally apparent in less than half of the studies (Phan et al., 2002), and some areas have been found to increase activity in some studies and decrease activity in others (Damasio et al., 1999; Mayberg et al., 1999; Aalto et al., 2002). This variability has been attributed to variation in experimental paradigm, stimuli and imaging methods. Despite the large number of imaging studies of emotion, reported activation differences between happy or positive and sad or negative emotional states have been limited. Happiness-evoking stimuli have been shown to lead to greater activation than both fear- and sadness-inducing stimuli in the left occipital cortex in the area of the calcarine fissure (Morris et al., 1996; George et al., 1995) and in the right middle temporal gyrus (MTG) (Lane et al., 1997a; Morris et al., 1996). Other findings have not been replicated and one study found no differences between the happy and sad conditions (Schneider et al., 1995). The most consistently activated area in studies of happiness, the basal ganglia, has also consistently been shown to become active with disgust (Phan et al., 2002). Given the obvious importance of the distinction between positive and negative emotions, the relative paucity of reported brain-activation differences between the two states raises questions about the sensitivity of experimental paradigms. The present study focused on identifying differences in brain activation associated with happy and sad

emotion-related stimulation, addressing three design features that may have contributed to inconsistency and insensitivity in previous studies. In a prototypic functional magnetic resonance imaging (fMRI) design, stimuli such as words or pictures are each presented briefly in alternating blocks of positive, negative, or neutral stimuli. While this often works well for studies of cognition, emotional responses can continue past the typical interblock period and into the next emotion block (Garrett and Maddock, 2001), thereby decreasing the power to detect differences between stimulus blocks. Such studies may be limited further by the weak ability of single words or pictures to generate powerful emotional response, compared with the sustained multi-faceted stimulation that occurs in real life. PET studies have often used more powerful cinematic stimuli, but power may be limited in those studies by averaging over the entire period of stimulus presentation regardless of when the subject actually experiences an emotional response. In this study, then, we used video clips to produce emotional responses, used the high temporal resolution of fMRI to examine activation both prior to and after the onset of emotional experience instead of averaging across the entire stimulus period, and focused on identifying differences between happy and sad emotional states. Previous studies have used short segments of popular movies as stimuli, but these often lose power and meaning when extracted from the context of the whole movie and subject response to them can be affected by whether or not the subject had seen the original movie. Instead, we produced videotapes simulating faceto-face social interaction between the subject and an actress pretending to be either very happy or very sad. Although there is no consensus on a precise definition of emotion, it is clear that a fundamental aspect of emotions is social or interpersonal, and that facial and vocal displays of emotion are the primary mechanisms for expression and recognition of emotional states. Displays such as smiling, frowning, laughing, and crying are the same in all cultures (Ekman, 1972) and are even apparent in children born blind (Charlesworth and Kreutzer, 1973) indicating they are innate and biologically important. When one individual sees even a picture of another smiling, electrical activity increases in the zygomatic muscles that lift the corners of the mouth in a smile (Dimberg, 1982; Wexler et al., 1994). Similar increases are

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

measurable in the corrugator muscles that furrow the brow in a frown when the viewer sees a picture of someone looking sad. Such automatic mimicry appears to be one way that emotional states spread from person to person, since deliberately creating a smile or a frown leads to physiological, subjective, and behavioral changes consistent with being happy or sad (Ekman et al., 1983; Laird, 1974). Thus, our expectation was that having subjects view videos of someone smiling and laughing, frowning and crying while talking directly to them would trigger automatic emotional responses in the subjects. Such responses could be enhanced by cognitive decoding of the descriptions of emotion-related events and by personal memories prompted by the induced emotion and the content of the descriptions. Finally, the purpose of this study was to examine the moment of self-reported onset of emotion. The subjects indicated by button press the onset of emotional reaction, which typically occurred within a minute of tape viewing. Tape contents were fairly constant before and after this moment; the only difference was the subjective definition, and its announcement, of the emotional experience. Thus, comparing the preceding and following short time periods may provide a sharp delineation of the onset of emotional experience.

2. Methods The subjects were 21 healthy young adults (13F, 8M, average age 33 years). All denied any more than occasional social use of alcohol or illicit drugs, and they reported no use of psychoactive substances during the 72 h prior to fMRI. All wrote with their right hands and were right-handed by self-report. None were taking CNS-active prescription drugs; had history of psychiatric illness, neurological illness or brain injury; or abnormalities on structural MRI. After description of the study, subjects gave written informed consent. Subjects were told the experiment involved measurement of brain responses related to emotions that might be triggered by watching videotapes, and that they were to indicate by button press when they first began to feel any emotional response and when that response became moderate or very strong. The subjects were not told before or after a tape what

241

emotion the actress was attempting to portray. Immediately after each tape, subjects were asked if they had had any emotional response to the tape and, if so, to describe how they felt in a few words and then to rate the average and peak intensity of their feelings on a scale of 1 to 10. Videotapes were prepared with actresses seated, looking at, and directly talking to the camera (viewer). One actress made a happy and a sad tape; another actress made a second sad tape. In the happy tape, the actress smiled frequently and spoke cheerfully about happy memories of growing up on a farm and then of a family reunion at the time of her parents’ 50th anniversary. In the sad tapes, the actresses spoke sadly about the death of a close family member, crying throughout. These personal vignettes ranged in length from 3 to 4.5 min and were preceded and followed by 30-s baseline periods of gray illumination. Subjects were told at the start of the baseline that another videotape and imaging run were beginning, but they were not told the nature of the material or what emotion to expect. Subjects indicated the onset and progressive increases in emotional responses by button pushes during tape viewing. Immediately after viewing each tape, subjects described the nature of their emotional responses and rated the peak and average intensity of their responses from 0 to 10. Half of the subjects received the tapes in the following order: Sad A, Happy A, Sad B. The other half received them as follows: Happy A, Sad B, Sad A. There were 2.5-min rests between tapes. 2.1. fMRI Subjects were positioned in the scanner with their heads stabilized by a band across the forehead. Video goggles were placed over their eyes and headphones over their ears (Magnetic Resonance Technologies) for presentation of stimulus tapes. MRI images were acquired using a 1.5 T GE Signa with resonant gradients for echo planar imaging (Advanced NMR; Wilmington, MA). Conventional T1-weighted spin echo sagittal anatomic images (TR=667 ms, TE=11 ms, FOV=24 cm, slice thickness=5 mm, gap=0, 2561281 Nex) were acquired for slice localization. Eight T1-weighted oblique axial anatomic images (TR=500 ms, TE=13 ms, FOV=4040 cm, SLT=8 mm, gap=1 mm, 2561921 Nex) were acquired

242

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

parallel to the plane transecting the anterior and posterior commissures, and functional BOLD images were obtained in these same planes using a single shot EPI gradient echo sequence (TR=2600 ms, TE=60 ms, flip angle=68, FOV=4020 cm, 1281281 Nex, slice thickness=8 mm, gap=1 mm). Measurement of changes in the center of mass of functional images over time demonstrated that no subject had movement greater than 1 voxel from the beginning to the end of functional imaging or greater than 0.5 voxel from the beginning to the end of any tape. Motion was corrected using SPM96. Corrected images were spatially filtered using a Gaussian filter with a fullwidth at half-maximum of 6.5 mm. 2.2. Data analysis The time course of induced emotion is probably complex; in this experiment, the onset and duration of emotional response varied from subject to subject, leading some subjects to have longer periods of tape viewing prior to the report of emotion and shorter periods of watching the tape after reporting the onset of emotion. In this study, we analyzed only the epochs immediately before and after the onset of reported emotional experience. The BOLD images were divided into three experimental conditions for each tape: the 30-s gray-field baseline (B1), the initial period of tape viewing prior to the self-report of emotional response (EMO 0) and the initial period of emotional response (EMO 1). In order to maintain sufficient comparability among subjects in data sampling, comparisons were not made if either epoch to be compared was less than 20 s long, and only the first 45 s of longer EMO 1 epochs were considered. To minimize type I error, only two contrasts were evaluated: EMO 0 vs. B1 and EMO 1 vs. EMO 0. These correspond to the effects of tape viewing compared with non-tape baseline (i.e., presumed nonemotional visual stimulation) and the effect of emotional response (holding visual stimulation fairly constant). Some subjects did not report any emotional response to one or two tapes (5 to the Happy tape, 2 to Sad A, and 2 to Sad B). These subjects therefore did not contribute to the EMO 1EMO 0 contrast. One subject did not report any emotional response to any tape and was dropped from the data set, resulting in n=20.

Two t-maps were created for each subject for each tape, according to the contrasts above. These t-maps and the corresponding anatomic images from individual subjects were transformed by linear interpolation into a proportional three-dimensional grid defined by Talairach and Tournoux (1988). The six slices analyzed in this experiment are at z levels of 4, +4, 12, 24, 32, and +40 mm from the ACPC line. The t-maps were not used to derive significance levels, but only to compute standard linear contrast measures. Under the null hypothesis of no effect, the expected value of the mean of this contrast across subjects is equal to zero. We used a randomization procedure to generate a distribution of task-related t-values in order to estimate significance (Manly, 1997; Skudlarski and Gore, 1999; Shaywitz et al., 1999). This procedure creates the population distribution for each voxel by repeatedly calculating the value of the contrast when the t-values of half the subjects, randomly chosen, have a reversed sign. This randomization was performed 1000 times, generating a sampling distribution of the linear contrast measures. The observed linear contrast measure, calculated without sign reversal, was assigned a P-value based on its position in this distribution. The proportion of times that the observed linear contrast measure was more extreme than the randomized linear contrast measure represents the significance. The two contrasts were first assessed for the two sad tapes together and then for the single happy tape. For each of these contrasts, randomizations were performed by reversing the sign of the t-values in randomly chosen subsets of half the subjects. Next, the statistical significance of differences between happy and sad conditions was evaluated. For this analysis, the linear contrast was a simple subtraction between the activation maps for each tape, with values reversed for the two affect conditions in randomly chosen subsets of half the subjects. Significant results were defined in areas with cluster size N20 (area N40 mm2) and all pixels exceeding Pb0.05.

3. Results Subjective emotional responses were robust during all tapes but significantly greater ( F 2,54 =5.61, Pb0.01) during both sad tapes than during the happy

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

tape (Table 1). While some subjects did not report any emotional responses on some tapes (see Section 2), no subject reported sad emotion during the happy tape or happy emotion during the sad tape. The onset of reported emotional response also varied (Table 1), being significantly faster for the sad tapes ( F 2,45=4.49, Pb0.02; the lower n is due to subjects who never pressed the button). The intensity and timing were negatively correlated for the sad tapes (BS: r= 0.50, Pb0.05; KS: r= 0.43, Pb0.10), but not for the happy tape (r=0.07). The two sad tapes were very similar in the timing and intensity of evoked emotion. Fig. 1 depicts the cerebral topography of significant BOLD changes for the EMO 0 vs. B1 and the EMO 1 vs. EMO 0 contrasts. Results of the statistical comparisons between the happy and sad conditions (the difference analysis mentioned above) are provided in Table 1 and the text below. These are not necessarily reflected in obvious differences in Fig. 1. 3.1. EMO 0–B1 contrasts There is robust activation (for both sad and happy tapes) of primary and secondary auditory and visual areas. Additional areas of activation are also evident during both tapes in more anterior portions of the cortex: in the medial superior frontal gyrus, and in the right inferior and middle frontal gyri (Table 2). Even though the subjects had not yet reported the onset of any feelings, activation of the STG was significantly greater during the happy than during the sad tapes ( Pb0.005, z=4, 12). Other brain regions showed sad activations to be stronger than happy activations during this same initial period of tape viewing. The sad but not the happy tape led to activation of the precuneus (Fig. 1B, z=40) and the angular gyrus (Fig. 1B, z=32), both in the right hemisphere, with the differences between happy and sad significant at Pb0.005 and Pb0.02, respectively. Activity changes were significantly greater during the Table 1 Intensity and timing of emotional responses (meanFS.D.) Tape

Intensity

EMO 0 duration (s)

BS KS BH

6.6F2.9 6.2F2.6 3.9F2.5

59F34 53F33 105F81

243

sad tapes in the SFG and the MFG on the left ( Pb0.01, z=12). Activation of this area is not evident during the sad tape at Pb0.005 (Fig. 1B), but is when the threshold is decreased to Pb0.01 (data not shown). Activity changes were also greater during the sad tapes in the supramarginal gyrus ( Pb0.01, z=40), although significant changes in this area were not detected for either the sad or happy tapes alone. Activation of the anterior cingulate was evident during the sad but not the happy condition (z=40), but the difference between conditions was not significant. 3.2. EMO 1–EMO 0 With the onset of conscious feelings, there were no additional activations. Instead, activity decreased in several of the areas that had increased during the initial period of tape viewing (Fig. 1A and B, column B); this reversal of activation was more consistent during the sad tape. The two emotions led to reciprocal disparities in the medial aspects of the MTG, with decreases significantly greater on the right after the reported onset of sad feelings ( Pb0.005, z= 4) and on the left after the reported onset of happy feelings ( Pb0.05, z= 4). Decreases were also greater after the reported onset of sad feelings in the right cuneus ( Pb0.005, z=4) and MOG ( Pb0.005, z=12, 24). Decreases were greater after the reported onset of happy feelings in the left angular gyrus ( Pb0.01, z=32), the precuneus ( Pb0.05, z=32, 40), the cingulate ( Pb0.005, z=32, 40), and the left SFG ( Pb0.005, z=40).

4. Discussion The present study was designed to examine differences in brain response associated with evoked happy and sad emotions. Since the stimulus tapes are similar with regard to interest, arousal, and self-regulation as well as audio-visual format and sensory input, differences between the two are likely to be related to emotional valence (Table 2). Activity during the happy tape was different from activity during the sad tapes in the MTG, STG, MFG, SFG, cingulate gyrus, supramarginal gyrus, lingual gyrus, cuneus, precuneus, angular gyrus, MOG, caudate, and hippo-

244

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

Slice Level

40

32

24

12

4

-4 Emo0-B1

Emo1-Emo0

Happy

Emo0-B1

Emo1-Emo0

Sad

Fig. 1. Areas of significant change of BOLD signal during viewing of the happy or sad tapes, superimposed on normalized T1-weighted anatomical images. The rows depict six transverse slices with their Talairach z coordinates (distance in mm from the AC–PC line). The first two columns show changes from the pre-tape baseline to the initial happy tape viewing period prior to self-report of feelings (EMO 0–B1), and changes from this initial pre-emotional response viewing period to the period during which subjects reported feeling happy (EMO 1–EMO 0). The next two columns depict the same periods for the sad tape. Colors represent positive (yellow and red) and negative (purple and blue) signal changes that exceeded the 0.05 significance level by the randomization test.

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250 Table 2 Differential emotional effects Region

Contrast EMO 0–B1

EMO 1–EMO 0

Anterior cingulate Superior frontal gyrus Middle frontal gyrus Pre-central gyrus Supramarginal gyrus Superior temporal gyrus Middle temporal gyrus

SNHa (z=12) SNHb (z=12) SNHc (L, z=32, 40) b SNH (z=4, 12, 40)

Angular gyrus Precuneus Cuneus Lingual Midlle occipital gyrus

SNHc (R, z=32) SNHc (R, z=40)

SNHb (L, z=40) HNSa (z=4, 12) HNSa, (R, z= 4) SNHd (L, z= 4) SNHb (z=32) SNHd (z=32, 40) HNSa (R, z=4) HNSa (R, z=12, 24)

Brain regions in which activation is significantly different when subjects watch the happy tape than when they watch the sad tapes during each of the following epochs: EMO 1–B1, initial period of tape viewing, prior to the reported onset of emotional responses, compared with the pre-tape baseline; EMO 1–EMO 0, initial period of reported emotional response compared with the immediately preceding period of tape viewing. a Pb0.005. b Pb0.01. c Pb0.02. d Pb0.05.

campus. Some of these differences between the happy and sad conditions were due to emotion-related increases and others to emotion-related decreases in activity. We discuss several of these regional differences in relation to previous studies of the roles of the regions in both emotion and cognition, and then consider more general issues of the number, nature, time course, and distribution of the observed valencerelated differences in regional brain activity. 4.1. Temporal lobe During the initial period of tape viewing prior to the onset of self-reported feelings of emotion, increased activity was somewhat greater in the right MTG during the happy than during the sad tapes ( Pb0.05). After the onset of self-reported sadness, activity in this area decreased, increasing the significance of the difference between happy and sad conditions. Two previous studies have also noted significantly greater activity in the right MTG during pleasant compared with unpleasant emotional states

245

(Lane et al., 1997a; Morris et al., 1996). Activity in the left MTG, in contrast, decreased after the onset of feelings of happiness and was significantly greater during this period of the sad tapes than during this period of the happy tape. One previous study found greater activity in the left MTG when subjects viewed aversive rather than neutral stimuli (Kosslyn et al., 1996). The MTG has previously been shown to be active during perceptual and cognitive operations related to emotional experience. Areas within both the left and right MTG are active when subjects watch moving visual stimuli (e.g., Martin et al., 1995; Cornette et al., 1998a,b), observe complex visual patterns (Beason-Held et al., 1998), do visual–spatial discrimination tasks (Schwartz et al., 1996), or look at pictures of human faces (Puce et al., 1996). Moreover, activation in the MTGs is greater when subjects look at faces in which the eyes are visible rather than faces in which the eyes cannot be seen (Wicker et al., 1998). Ability to verbally label emotional facial expressions is lost with lesions of the right MTG (Rapcsak et al., 1989, 1993), and intra-operative recordings have identified cells within the right MTG (and STG) that change activity when subjects are asked to verbally label facial expressions (Ojemann et al., 1992). The right MTG also appears important in sustaining and focusing attention (Samuelsson et al., 1997; Fink et al., 1997). Both the left and the right MTG are active during a variety of other language-related tasks including learning word pairs (Elger et al., 1997; Ojemann and Schoenfield-McNeill, 1998), naming objects (Martin et al., 1996; Raymer et al., 1997), generating action words (Martin et al., 1995), listening to abstract word definitions (Mellet et al., 1998), matching concepts presented in words (Vandenberghe et al., 1996), understanding metaphors (Bottini et al., 1994), and imagining sentences spoken in another person’s voice (McGuire et al., 1996). While the various functions associated with the MTG could each depend primarily on activity in different areas of the MTG, they constitute overlapping and inter-related clusters of behavioral capabilities. In the case of the perception and naming of movement, these two closely inter-related functions have been shown to activate adjacent and probably overlapping regions of the MTG (Martin et al., 1995). Moreover, large regions of the MTG were active during viewing of the emotion-generating tapes. Thus, it seems possible

246

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

that a wide variety of MTG-related functions were active during tape viewing, contributed to and were affected by emotional responses to the tapes, and were different in happy and sad conditions. Activation was markedly greater during the happy than during the sad tape in STG bilaterally. This may be due in part to the actress speaking more quickly during the happy tape, as STG activation has been shown to increase as the rate of word presentation increases (Dhankhar et al., 1997). This effect could also be caused by other social aspects of face perception (Hoffman and Haxby, 2000). This in itself does not make the enhanced STG activation irrelevant to understanding neural correlates of emotion since speech, motor, and facial expressive activity is typically greater in happy social interactions than in sad ones. Beyond this, however, Lane et al. (1997a) reported robust STG activation in happy emotional states induced by silent films or by recall of happy personal events. In addition, electrophysiological studies in monkeys (Hasselmo et al., 1989; Heywood and Cowey, 1992), cortical stimulation studies in humans (Ojemann et al., 1992), and previous fMRI studies in humans (Puce et al., 1995) have shown that areas within the STG are active during perception of faces. Moreover, activation is significantly greater when subjects view faces in which the eyes or the mouth are moving (Puce et al., 1998). Since the face in general, and the mouth and eyes in particular, are major components of affective displays, these studies provide precedents for the present finding of emotionrelated activation in the STG.

learning (e.g., visual paired-associates, Klingberg and Roland, 1998); working memory (Klingbert et al., 1997; Belger et al., 1998); recognition memory (Taylor et al., 1998); selection of responses from among alternatives (Grabowski et al., 1998; Desmond et al., 1998); and self-initiation of movement (Pedersen et al., 1998). 4.3. Cuneus The happy tape also led to greater activation than the sad tape in the cuneus (primarily on the left) and in a small region of the middle occipital gyrus. George et al. (1995) also found greater activation in the left cuneus, as well as other areas in the visual cortex (left greater than right) when subjects recalled happy events while looking at pictures of happy faces than when they recalled sad events while looking at sad faces. The cuneus, not unlike the MTG, has been shown to be active in tasks requiring detection of motion (Cheng et al., 1995; Orban et al., 1998; Shulman et al., 1998; Cornette et al., 1998a,b), perception and recognition of visual patterns (Roland and Gulyas, 1995; Hadjikhani and Roland, 1998), shifting attention between visual features (Le et al., 1998), and recall of temporal order (Cabeza et al., 1997). In addition, it has been shown to be active during pitch discrimination (Platel et al., 1997), mental arithmetic (Dehaene et al., 1996) and, in older adults, during a word recognition task (Cabeza et al., 1997). 4.4. Time course

4.2. Frontal lobe Activity was greater during the sad than during the happy tape in the MFG bilaterally, with differences evident during the initial period of tape viewing prior to the onset of self-reported feelings of either sadness or happiness. These areas have not been reported to be active in previous studies of emotion, but they have been shown to be active during a wide range of cognitive operations, including sustained attention (e.g., concentrate on a small dot of light, Lewin et al., 1996; Johannsen et al., 1997), manipulation of information (e.g., mental rotation, Cohen et al., 1996; Faillenot et al., 1997); reasoning (e.g., determining logical relations among statements, Goel et al., 1998);

Several intriguing aspects of our results are related to the time course of cerebral activation during emotional tape viewing. Watching the videotapes prior to the reported onset of emotion activated both auditory and visual sensory and association areas as expected. Location of these activations was highly similar with both sad and happy tapes. In addition to these expected sensory and perceptual areas, a medial area of the SFG was active during EMO 0 of both tapes. This area has been found to activate in both pleasant and unpleasant emotional states, but not neutral control states, in multiple previous studies using a variety of different procedures to induce emotion (e.g., Phan et al., 2002; Damasio et al., 2000;

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

Reiman et al., 1997). In our study, we believe this represents an emotional response prior to its conscious report. This is consistent with the report by Damasio et al. (2000) that in a study of recall-induced emotion, psychophysiological activity preceded subjective reports in all instances. This area has also been shown to be active during the perception and processing of the social aspects of facial communication (Hoffman and Haxby, 2000), both of which are essential components of the emotional response in our experiment. Although it is possible that this activation is associated with self-monitoring and other aspects of preparation for reporting an emotional response, such response demands were absent from previous studies that found emotion-related activation of this area (Lane et al., 1997b; Teasdale et al., 1999). Moreover, we found this same activation during the initial 30 s of tape viewing in five pilot subjects who were not asked to indicate the onset or intensity of their emotional responses, but only to watch the tapes and report afterwards how they felt (data not shown). Once the subjects indicated the onset of conscious emotional experience (EMO 1), significant changes from EMO 0 consisted of signal reductions. Most previous studies could not observe such changes because they did not clearly mark this point, or because they lacked temporal resolution. Nevertheless, there are some previous hints of similar phenomena in the literature. A similar observation has been reported from a PET study of blood flow changes immediately after subjects reported the onset of recall-generated positive and negative emotions (Damasio et al., 1999). Another study also found decreased blood flow in multiple areas several minutes after the self-reported onset of recall-generated sadness as compared with rest, including the right lateral frontal and middle temporal decreases seen in the present study (Mayberg et al., 1999). In addition, many of the decreases in activity observed in the present study were also seen in a PET study of averaged blood flow changes over 2.5-min emotionevoking film clips (Aalto et al., 2002). One possible explanation is that conscious awareness of emotion is associated with resolution rather than initiation of neural processes. It is also possible that the signal reductions are due to sympathetic arousal and associated cerebral vasoconstriction. This mechanism has been invoked to explain other seemingly para-

247

doxical findings. For example, Coghill et al. (1998) measured global CBF by H215O PET and found a decrease during capsaicin-induced pain in normal subjects. However, our signal reductions were temporally related to the reported onset of subjective feelings of emotion, differed in extent and distribution between happy and sad conditions, and were not seen in all areas (or all functional systems). These aspects argue for a specific regional effect, and against a systemic physiological confound, such as a change in respiration, pCO2, or sympathetic discharge, that would tend to exert a more uniform influence on cerebral vasculature. While eliciting several intriguing indications for future studies, this report is limited in several ways. Although the primary interest was in contrasting happy and sad response states, the two conditions shared activations related to time in the scanner, monitoring and pressing a button to report emotional experience, and looking at complex audio-visual stimuli in comparison to a plain gray screen. These commonalities would serve to minimize differences between happy and sad conditions. Furthermore, several areas of the brain that may be relevant to emotional processing (e.g., orbitofrontal cortex) could not be analyzed due to susceptibility artifacts. However, these factors are unlikely to explain the differences between the sad and happy conditions. 4.5. Conclusion Having actresses portray happy or sad feelings and describe those feelings in a simulated one-on-one interaction with the subject led to widespread, valence-specific changes in regional brain activation. The interdependence of emotion and cognition is evident in the effects of emotion on multiple brain regions known to be important in cognition and behavior. The neural correlates of emotion are multiple, widely distributed and inclusive of areas important in other functions, rather than being highly localized and specific. The multiple and distributed changes in regional brain activity associated with each emotion may have their own characteristic functional coherence (Wexler, 1986). Confronted with these data, the search for localized neural foci of emotional response or emotional state would seem to be unjustified. Widespread changes in cortical activity

248

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

are as much part of emotion as they are its effects. On the other hand, the current results suggest that imaging studies of the temporal evolution of emotional experience are now possible with fMRI, and may generate intriguing new observations.

Acknowledgments This work was supported in part by a grant from the National Institute of Mental Health to Dr. Wexler (KO2-MH1296) and an educational grant from Siemens Medical Systems to Dr. Prohovnik.

References Aalto, S., Naatanen, P., Wallius, E., Metsahonkala, L., Stenman, H., Niem, P.M., Karlsson, H., 2002. Neuroanatomical substrata of amusement and sadness: a PET activation study using film stimuli. NeuroReport 13 (1), 67 – 73. Beason-Held, L.L., Purpura, K.P., Krasuski, J.S., Maisog, J.M., Daly, E.M., Mangot, D.J., Desmond, R.E., Optican, L.M., Schapiro, M.B., VanMeter, J.W., 1998. Cortical regions involved in visual texture perception: an fMRI study. Brain Research 7, 111 – 118. Belger, A., Puce, A., Krystal, J.H., Gore, J.C., Goldman-Rakic, P., McCarthy, G., 1998. Dissociation of mnemonic and perceptual processes during spatial and nonspatial working memory using fMRI. Human Brain Mapping 6 (1), 14 – 32. Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P., Scarpa, P., Frackowiak, R.S., Frith, C.D., 1994. The role of the right hemisphere in the interpretation of figurative aspects of language. A positron emission tomography activation study. Brain 117 (Pt 6), 1241 – 1253. Cabeza, R., Grady, C.L., Nyberg, L., McIntosh, A.R., Tulving, E., Kapur, S., Jennings, J.M., Houle, S., Craik, F.I., 1997. Agerelated differences in neural activity during memory encoding and retrieval: a positron emission tomography study. Journal of Neuroscience 17 (1), 391 – 400. Charlesworth, W., Kreutzer, M., 1973. Facial expressions of infants and children. In: Ekman, P. (Ed.), Darwin and Facial Expression. Academic Press, New York. Cheng, K., Fujita, H., Kanno, I., Miura, S., Tanaka, K., 1995. Human cortical regions activated by wide-field visual motion: an H2(15)O PET study. Journal of Neurophysiology 74 (1), 413 – 427. Coghill, R.C., Sang, C.N., Berman, K.F., Bennett, G.J., Iadarola, M.J., 1998. Global cerebral blood flow decreases during pain. JCBFM 18, 141 – 147. Cohen, M.S., Kosslyn, S.M., Breiter, H.C., DiGirolamo, G.J., Thompson, W.L., Anderson, A.K., Brookheimer, S.Y., Rosen, B.R., Belliveau, J.W., 1996. Changes in cortical activity during

mental rotation. A mapping study using functional MRI. Brain 119 (Pt 1), 89 – 100. Cornette, L., Dupont, P., Rosier, A., Sunaert, S., Van Hecke, P., Michiels, J., Mortelmans, L., Orban, G.A., 1998a. Human brain regions involved in direction discrimination. Journal of Neurophysiology 79, 2749 – 2765. Cornette, L., Dupont, P., Spileers, W., Sunaert, S., Michiels, J., Van Hecke, P., Mortelmans, L., Orban, G.A., 1998b. Human cerebral activity evoked by motion reversal and motion onset: a PET study. Brain 121 (Pt 1), 143 – 157. Damasio, A.R., Grabowski, T.J., Bechara, A., et al., 1999. The contribution of subcortical nuclei to processing of emotion and feeling. NeuroImage 9, S359. Damasio, A.R., Grabowski, T.J., Bechara, A., Damasio, H., Ponto, L.L.B., Parvizi, J., Hichwa, R.D., 2000. Subcortical and cortical brain activity during the feeling of self-generated emotions. Nature Neuroscience 3, 1049 – 1056. Dehaene, S., Tzourio, N., Frak, V., Raynaud, L., Cohen, L., Mehler, J., Mazoyer, B., 1996. Cerebral activations during number multiplication and comparison: a PET study. Neuropsychologia 34 (11), 1097 – 1106. Desmond, J.E., Gabrieli, J.D., Glover, G.H., 1998. Dissociation of frontal and cerebellar activity in a cognitive task: evidence for a distinction between selection and search. NeuroImage 7 (4 Pt 1), 368 – 376. Dhankhar, A., Wexler, B.E., Fulbright, R.K., Halwes, T., Blamire, A.M., Shulman, R.G., 1997. Functional magnetic resonance imaging assessment of the human brain auditory cortex response to increasing word presentation rates. Journal of Neurophysiology 77, 76 – 483. Dimberg, U., 1982. Facial reactions to facial expressions. Psychophysiology 19, 643 – 647. Ekman, P., 1972. Universals and cultural differences in facial expressions of emotion. In: Cole, J.K. (Ed.), Nebraska Symposium on Motivation, vol. 19. University of Nebraska Press, Lincoln, NE, pp. 207 – 283. Ekman, P., Levenson, R.W., Friesen, W.V., 1983. Autonomic nervous system activity distinguishing among emotions. Science 221, 1208 – 1210. Elger, C.E., Grunwald, T., Lehnertz, K., Kutas, M., Helmstaedter, C., Brockhaus, A., Van Roost, D., Heinze, H.J., 1997. Human temporal lobe potentials in verbal learning and memory processes. Neuropsychologia 35 (5), 657 – 667. Faillenot, I., Sakata, H., Costes, N., Decety, J., Jeannerod, M., 1997. Visual working memory for shape and 3D-orientation: a PET study. NeuroReport 8 (4), 859 – 862. Fink, G.R., Halligan, P.W., Marshall, J.C., Frith, C.D., Frackowiak, R.S., Dolan, R.J., 1997. Neural mechanisms involved in the processing of global and local aspects of hierarchically organized visual stimuli. Brain 120 (Pt 10), 1779 – 1791. Garrett, A.S., Maddock, R.J., 2001. Time course of the subjective emotional response to aversive pictures: relevance to fMRI studies. Psychiatry Research 108 (1), 39 – 48. George, M.S., Ketter, T.A., Parekh, P.I., Horwitz, B., Herscovitch, P., Post, R.M., 1995. Brain activity during transient sadness and happiness in healthy women. American Journal of Psychiatry 152 (3), 341 – 351.

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250 Goel, V., Gold, B., Kapur, S., Houle, S., 1998. Neuroanatomical correlates of human reasoning. Journal of Cognitive Neuroscience 10 (3), 293 – 302. Grabowski, T.J., Damasio, H., Damasio, A.R., 1998. Premotor and prefrontal correlates of category-related lexical retrieval. NeuroImage 3, 232 – 243. Hadjikhani, N., Roland, P.E., 1998. Cross-modal transfer of information between the tactile and the visual representations in the human brain: a positron emission tomographic study. Journal of Neuroscience 18 (3), 1072 – 1084. Hasselmo, M.E., Rolls, E.T., Baylis, G.C., 1989. The role of expression and identity in face-selective responses of neurons in the temporal visual cortex of the monkey. Behavioral Brain Research 32, 203 – 218. Heywood, C.A., Cowey, A., 1992. The role of the dface-cellT area in the discrimination and recognition of faces by monkeys. Philosophical Transactions of the Royal Society of London. B 335, 31 – 38. Hoffman, E.A., Haxby, J.V., 2000. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nature Neuroscience 3, 80 – 84. Johannsen, P., Jakobsen, J., Bruhn, P., Hansen, S.B., Gee, A., Stodkilde-Jorgensen, H., Gjedde, A., 1997. Cortical sites of sustained and divided attention in normal elderly humans. NeuroImage 6 (3), 145 – 155. Klingberg, T., Roland, P.E., 1998. Right prefrontal activation during encoding, but not during retrieval, in a non-verbal pairedassociates task. Cerebral Cortex 8 (1), 73 – 79. Klingbert, G., O’Sullivan, B.T., Roland, P.E., 1997. Bilateral activation of fronto-parietal networks by incrementing demand in a working memory task. Cerebral Cortex 7 (5), 465 – 471. Kosslyn, S.M., Shin, L.M., Thompson, W.L., McNally, R.J., Rauch, S.L., Pitman, R.K., Alpert, N.M., 1996. Neural effects of visualizing and perceiving aversive stimuli: a PET investigation. NeuroReport 7 (10), 1569 – 1576. Laird, J.D., 1974. Self-attribution of emotion: the effects of expressive behavior on the quality of emotional experience. Journal of Personality and Social Psychology 29, 475 – 486. Lane, R.D., Reiman, E.M., Ahern, G.L., Schwartz, G.E., Davidson, R.J., 1997a. Neuroanatomical correlates of happiness, sadness, and disgust. American Journal of Psychiatry 154 (7), 926 – 933. Lane, R.D., Reiman, E.M., Bradley, M.M., Lang, P.J., Ahern, G.L., Davidson, R.J., Schwartz, G.E., 1997b. Neuroanatomical correlates of pleasant and unpleasant emotion. Neuropsychologia 35 (11), 1437 – 1444. Le, T.H., Pardo, J.V., Hu, X., 1998. 4 T-fMRI study of nonspatial shifting of selective attention: cerebellar and parietal contributions. Journal of Neurophysiology 79 (3), 1535 – 1548. Lewin, J.S., Friedman, L., Wu, D., Miller, D.A., Thompson, L.A., Klein, S.K., Wise, A.L., Hedera, P., Buckley, P., Meltzer, H., Friedland, R.P., Duerk, J.L., 1996. Cortical localization of human sustained attention: detection with functional MR using a visual vigilance paradigm. Journal of Computer Assisted Tomography 20 (5), 695 – 701. Manly, B., 1997. Randomization, Bootstrap and Monte Carlo Methods in Biology. Chapman & Hall, London, England.

249

Martin, A., Haxby, J.V., Lalonde, F.M., Wiggs, C.L., Ungerleider, L.G., 1995. Discrete cortical regions associated with knowledge of color and knowledge of action. Science 270 (5233), 102 – 105. Martin, A., Wiggs, C.L., Ungerleider, L.G., Haxby, J.V., 1996. Neural correlates of category-specific knowledge. Nature 379 (6566), 649 – 652. Mayberg, H.S., Liotti, M., Brannan, S.K., McGinnis, S., Mahurin, R.K., Jerabek, P.A., Silva, J.A., Tekell, J.L., Martin, C.C., Lancaster, J.L., Fox, P.T., 1999. Reciprocal limbic-cortical function and negative mood: converging Pet findings in depression and normal sadness. American Journal of Psychiatry 156 (5), 675 – 682. McGuire, P.K., Silbersweig, D.A., Wright, I., Murray, R.M., Frackowiak, R.S., Frith, C.D., 1996. The neural correlates of inner speech and auditory verbal imagery in schizophrenia: relationship to auditory verbal hallucinations. British Journal of Psychiatry 169 (2), 148 – 159. Mellet, E., Tzourio, N., Denis, M., Mazoyer, B., 1998. Cortical anatomy of mental imagery of concrete nouns based on their dictionary definition. NeuroReport 9 (5), 803 – 808. Morris, J.S., Frith, C.D., Perrett, D.I., Rowland, D., Young, A.W., Calder, A.J., Dolan, R.J., 1996. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383, 812 – 815. Ojemann, G.A., Schoenfield-McNeill, J., 1998. Neurons in human temporal cortex active with verbal associative learning. Brain and Language 64 (3), 317 – 327. Ojemann, J.G., Ojemann, G.A., Lettich, E., 1992. Neuronal activity related to faces and matching in human right nondominant temporal cortex. Brain 115 (Pt 1), 1 – 13. Orban, G.A., Dupont, P., De Bruyn, B., Vardenberghe, R., Rosier, A., Mortelmans, L., 1998. Human brain activity related to speed discrimination tasks. Experimental Brain Research 122 (1), 9 – 22. Pedersen, J.R., Johannsen, P., Bak, C.K., Kofoed, B., Saermark, K., Gjedde, A., 1998. Origin of human motor readiness field linked to left middle frontal gyrus by MEG and PET. NeuroImage 8 (2), 214 – 220. Phan, K.L., Wager, T., Taylor, S.F., Liberzon, I., 2002. Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI. NeuroImage 16, 331 – 348. Platel, H., Price, C., Baron, J.C., Wise, R., Lambert, J., Frackowiak, R.S., Lechevalier, B., Eustache, F., 1997. The structural components of music perception. A functional anatomical study. Brain 120 (Pt 2), 229 – 243. Puce, A., Allison, T., Gore, J.C., McCarthy, G., 1995. Face-sensitive regions in human extrastriate cortex studied by functional MRI. Journal of Neurophysiology 74, 1192 – 1199. Puce, A., Allison, T., Asgari, M., Gore, J.C., McCarthy, G., 1996. Differential sensitivity of human visual cortex to faces, letterstrings, and textures: a functional magnetic resonance imaging study. Journal of Neuroscience 16 (16), 5205 – 5215. Puce, A., Allison, T., Bentin, S., Gore, J.C., McCarthy, G., 1998. Temporal cortex activation in humans viewing eye and mouth movements. Journal of Neuroscience 18 (6), 2188 – 2199. Rapcsak, S.Z., Kaszniak, A.W., Rubens, A.B., 1989. Anomia for facial expressions: evidence for a category-specific

250

I. Prohovnik et al. / Psychiatry Research: Neuroimaging 132 (2004) 239–250

visual–verbal disconnection syndrome. Neuropsychologia 27, 1021 – 1041. Rapcsak, S.Z., Comer, J.F., Rubens, A.B., 1993. Anomia for facial expressions: neuropsychological mechanisms and anatomical correlates. Brain and Language 45 (2), 233 – 252. Raymer, A.M., Foundas, A.L., Maher, L.M., Greenwald, M.L., Morris, M., Rothi, L.J., Heilman, K.M., 1997. Cognitive neuropsychological analysis and neuroanatomic correlates in a case of acute anomia. Brain and Language 58 (1), 137 – 156. Reiman, E.M., Lane, R.D., Ahern, G.L., Schwartz, G.E., Davidson, R.J., Friston, K.J., Yun, L.S., Chen, K., 1997. Neuroanatomical correlates of externally and internally generated human emotion. American Journal of Psychiatry 154 (7), 918 – 925. Roland, P.E., Gulyas, B., 1995. Visual memory, visual imagery, and visual recognition of large field patterns by the human brain: functional anatomy by positron emission tomography. Cerebral Cortex 5 (1), 79 – 93. Royet, J.P., Zald, D., Versace, R., Costes, N., Lavenne, F., Koenig, O., Gervais, R., 2000. Emotional responses to pleasant and unpleasant olfactory, visual, and auditory stimuli: a positron emission tomography study. Journal of Neuroscience 20 (20), 7752 – 7759. Samuelsson, H., Jensen, C., Ekholm, S., Naver, H., Blomstrand, C., 1997. Anatomical and neurological correlates of acute and chronic visuospatial neglect following right hemisphere stroke. Cortex 33 (2), 271 – 285. Schneider, F., Gur, R.E., Mozley, L.H., Smith, R.J., Mozley, P.D., Censits, D.M., Alavi, A., Gur, R.C., 1995. Mood effects on limbic blood flow correlate with emotional self-rating: a PET study with oxygen-15 labeled water. Psychiatry Research: Neuroimaging 61, 265 – 283. Schwartz, T.H., Ojemann, G.A., Haglund, M.M., Lettich, E., 1996. Cerebral lateralization of neuronal activity during naming, reading, and line matching. Brain Research 4, 263 – 273. Shaywitz, S.E., Shaywitz, B.A., Pugh, K.R., Fulbright, R.K., Skudlarski, P., Menel, W.E., Constable, R.T., Naftolin, F.,

Palter, S.F., Marchione, K.E., Katz, L., Shankweiler, D.P., Fletcher, J.M., Lacadie, C., Keltz, M., Gore, J.C., 1999. Effect of estrogen on brain activation patterns in postmenopausal women during working memory tasks. Journal of the American Medical Association 281 (13), 1197 – 1202. Shulman, G.L., Schwarz, J., Miezin, F.M., Petersen, S.E., 1998. Effect of motion contrast on human cortical responses to moving stimuli. Journal of Neurophysiology 79 (5), 2794 – 2803. Skudlarski, P., Gore, J.C., 1999. Randomization in multi-subject fMRI studies: ROC analyses. NeuroImage 9, 591. Talairach, J., Tournoux, P., 1988. Co-planar Stereotaxic Atlas of the Human Brain. Thieme, New York. Taylor, S.F., Liberzon, I., Fig, L.M., Decker, L.R., Minoshima, S., Koeppe, R.A., 1998. The effect of emotional content on visual recognition memory: a PET activation study. NeuroImage 8 (2), 188 – 197. Teasdale, J.D., Howard, R.J., Cox, S.G., Ha, Y., Brammer, M.J., Williams, S.C., Checkley, S.A., 1999. Functional MRI study of the cognitive generation of affect. American Journal of Psychiatry 156 (2), 209 – 215. Vandenberghe, R., Price, C., Wise, R., Josephs, O., Frackowiak, R.S., 1996. Functional anatomy of a common semantic system for words and pictures. Nature 383 (6597), 254 – 256. Wexler, B.E., 1986. A model of brain function: its implications for psychiatric research. British Journal of Psychiatry 149, 202 – 209. Wexler, B.E., Levenson, L., Warrenburg, S., Price, L.H., 1994. Decreased perceptual sensitivity to emotion-evoking stimuli in depression. Psychiatry Research 51, 127 – 138. Whalen, P.J., Rauch, S.L., Etcoff, N.L., McInerney, S.C., Lee, M.B., Jenike, M.A., 1998. Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. Journal of Neuroscience 18 (1), 411 – 418. Wicker, B., Michel, F., Henaff, M.A., Decety, J., 1998. Brain regions involved in the perception of gaze: a PET study. NeuroImage 8 (2), 221 – 227.