The role of the right hemisphere in the control of speech prosody in propositional and affective contexts

BRAIN

AND

LANGUAGE

25, 19-36 (1985)

The Role of the Right Hemisphere in the Control of Speech Prosody in Propositional and Affective Contexts BARBARA E. SHAPIRO Harvard

University

and Boston

Veterans

MARTHA Boston

University

School

of Medicine

Administration

Medical

Center

DANLY

and Massachusetts

Institute

of Technology

Sixteen right-handed adult males with localized insult to either the right or left hemisphere and five control subjects without brain damage read aloud target sentences embedded in paragraphs, while intoning their voices in either a declarative, interrogative, happy, or sad mode. Acoustical analysis of the speech wave was performed. Right-anterior (pre-Rolandic) and right-central (pre- and post-Rolandic) brain-damaged patients spoke with less pitch variation and restricted intonational range across emotional and nonemotional domains, while patients with right posterior (post-Rolandic) damage had exaggerated pitch variation and intonational range across both domains. No such deficits were found in patients with left posterior damage, whose prosody was similar to that of normal control subjects. It is suggested that damage to the right hemisphere alone may result in a primary disturbance of speech prosody that may be independent of the disturbances in affect often noted in right-brain-damaged populations. 0 1985 Academic FWSS, IIIC.

This research was supported in part by funds from the Peter B. Livingston Memorial Fund Fellowship to Barbara Shapiro, NINCDS (Grant NS11408 to Edgar Zurif, Howard Gardner, and Hiram Brownell), and the Veterans Administration. We gratefully acknowledge the invaluable assistance of Kenneth N. Stevens and Keith North at MIT’s Research Laboratory of Electronics for making available the computer facilities of the Speech Communication Group. An earlier version of this paper was presented at the Academy of Aphasia, New Paltz, New York, October 1982. Barbara Shapiro is currently at the University of Miami School of Medicine, Ph.D-M.D. Program (R-123), P.O. Box 016%0, Miami, FL 33101. Send requests for reprints to Martha Danly, Wang Laboratories, Inc., 1 Industrial Ave., Lowell, MA 01851. 19 0093-934X/85 $3.00 Copyright 0 1985 by Academic Press, Inc. All rights of reproduction in any form reserved.

20

SHAPIRO

AND

DANLY

INTRODUCTION

Investigators have noted the dysprosodic (amelodic) speech of patients with damage to the right hemisphere of the brain, especially in the production of emotionally toned speech. Other researchers have noted deficits in the comprehension of emotionally toned speech (Monrad-Krohn, 1947; Ross, 1981; Ross, Hamey, delacoste-Utamsing, & Purdy, 1981; Ross & Mesulam, 1979; Tucker, Watson, & Heilman, 1977). In addition, there are many clinical reports of “flat affect” and/or an aberrant emotional quality in the behavior of patients with right brain damage (e.g., Gardner, 1975; Gardner, Ling, Flamm, & Silverman, 1975), although the specific locus of lesion within the right hemisphere has not been well specified. A major question that suggests itself is the following: does the dysprosodic, flat speech of right-brain-damaged patients reflect a true paucity in the emotional realm of these patients, or does it reflect a more intrinsic inability to control prosodic elements of speech across many domains, that may be only accidentally linked to the aberrant emotional qualities found in this population? This distinction is quite important, as it may help clarify the extent to which the flat speech we perceive in these patients is secondary to the noted emotional changes following right brain damage. A second major task is to determine those physical properties of the speech signal that correlate with the clinical perception of flat speech, thus providing an objective measure of dysprosody. In response to the first question, attempts have been made to examine, first, those aspects of behavior following right brain damage that contribute to the abnormal affect found in this population (Wapner, Hamby, & Gardner, 1981) and second, those aspects of speech prosody that are impaired in right-brain-damaged patients (Cooper, Soares, Nicol, Michelow, & Goloskies, 1984; Danly, Shapiro, & Gardner, 1982; Dordain, Degos, & Dordain, 1971; Weintraub, Mesulam, & Kramer, 1981). However, to date no systematic investigation has been undertaken comparing the speech prosody of right-brain-damaged patients in both emotional and nonemotional contexts, using an acoustical analysis of the speech wave. Hughlings Jackson (1915) proposed two basic categories of speechpropositional (concerned with linguistic aspects of speech) and affective (concerned with emotional aspects of speech). It has been shown that certain acoustical properties of speech prosody vary not only with linguistic context, but also with emotional states (Cosmides, 1983; Huttar, 1968; Lieberman & Michaels, 1962; Williams & Stevens, 1972). In propositional speech, one may vary prosody to mark the mode of an utterance. For example, declarative sentences generally contain a terminal fall in intonation, while yes-no interrogative sentences normally contain a terminal rise in intonation. In affective speech, one may vary prosody to mark the emotional quality of the speech. Thus, happy sentences are normally produced at a higher pitch, with greater intonational range and more

RIGHT

HEMISPHERE

PROSODY

21

variability than are sad sentences (Lieberman & Michaels, 1962; Williams & Stevens, 1972). While the distinctions between propositional and affective speech are not necessarily orthogonal, they offer a good test of the first question addressed. The extent to which patients with right brain damage modulate speech prosody correctly in either or both of these speech categories may help distinguish between an intrinsic, global deficit in speech melody versus one in the emotional realm only. In the former case, patients with right brain damage would be dysprosodic in the production of both propositional and affective speech, while in the latter case, they may be dysprosodic in emotional contexts only. In response to the second question, it should be noted that speech prosody refers to three major acoustical properties of the speech wave: fundamental frequency (F,,), duration, and amplitude; the perceptual correlates of these parameters include intonation (pitch), speech timing (word duration and pausing), and stress. Danly and Shapiro (1982) recently demonstrated the importance of distinguishing between clinical-perceptual and acoustical measures of speech. The speech of patients with left anterior brain damage and Broca’s aphasia has often been characterized as flat and dysprosodic. Yet we found their speech to be hypermelodic by certain acoustical measures. That is, when the speech of Broca’s aphasics was subjected to an acoustical analysis, it actually contained more variation in F,, than that of normal control subjects. Thus it is possible for a discrepancy to exist between the clinical-perceptual and acoustical measures of prosody; in order to obtain a more objective assessment of prosody, it is therefore valuable to measure the acoustic speech signal. In the present investigation, an attempt was made to correlate deficits in speech prosody across emotional and nonemotional domains with certain lesion sites in the right hemisphere. Patients with either right anterior, right central, right posterior, or left posterior unilateral brain damage were tested. Based on the results of previous research investigations (Danly, et al., 1982; Dordain et al., 1971; Weintraub et al., 1981) and clinical observations (Ross, 1981), we hypothesized that in comparison to normal control subjects, patients with either right anterior or right central damage, who have been characterized as having dysprosodic flat speech, would display lower mean FO level, restricted FO range and FO variability, and reduced capacity to modulate FO in propositional and/or affective contexts. Patients with right posterior damage were expected to display a higher than average mean F, level, exaggerated F. range and F, variability, and to accentuate modulation of F. in propositional and/or affective contexts. Patients with left posterior damage were expected to behave essentially like normal control subjects, with average mean F, level, normal F,, range and variability, and a capacity to modulate F. correctly in both propositional and affective contexts. They were included

22

SHAPIRO AND DANLY

in the present study to determine the effect of brain damage per se on these measures. Furthermore, we tested whether the ability of subjects to modulate F,, to signal the emotions happy and sad would be differentially affected by whether the sentences contained semantically neutral vs. semantically loaded affective words. METHOD Subjecrs. Subjects included 16 right-handed patients who had sustained localized cerebral insult to either the nght or left hemisphere, and 5 nonneurological control subjects. All subjects were male adults between 36 and 69 years of age. Patients were classified into three right-brain-damaged experimental groups and one left-brain-damaged control group on the basis of radiological findings (CT scan or brain scan); neuropsychological assessment; and/or clinical findings including field cuts, hemiplegia, and hemisensory loss. Experimental subject groups comprised three patients with lesions restricted to the anterior region of the right hemisphere (right anterior), three with right central lesions with primarily parietal but also including pre-Rolandic injury (right central), and five with right posterior lesions with temporoparietal injury (right posterior). Five left-posterior-brain-damaged patients with temporoparietal injury (left posterior) constituted the brain damage control group. These patients were recovering fluent asphasics with good comprehension. No control patients with either left anterior or left central damage were tested, as these are generally nonfluent patients who exhibit difficulty with a sentence reading paradigm. All patients had suffered an infarct of either hemorrhagic or emobolic origin, with the exception of one patient with a left posterior lesion of uncertain etiology. No consistent group differences were found in age, educational level, or socioeconomic status, with the exception of a mean higher educational level noted in the left posterior group. This was due to an unusually high educational attainment by one patient in this group. Materials. Stimulus items consisted of four sets (A,B,C,D) of four paragraphs each (see Appendix). The four paragraphs in each set contained one target sentence that was either declarative, interrogative, happy, or sad. (Note that the happy and sad sentences were also declarative.) The declarative and interrogative sentences thus represented a test of prosody in propositional speech; the happy and sad sentences represented a test of emotional speech. Within sets A and B, the target sentence was identical across all four paragraphs, and only the context of the paragraph cued whether the reading was to be declarative, interrogative, happy, or sad. Furthermore, the semantics of the target sentences alone were affectively neutral, thus cuing neither a declarative vs. interrogative nor a happy vs. sad reading. Any differences in F0 between target sentences within paragraph sets A or B would therefore be attributable to the influence of the context, rather than to any particular lexical item or syntactic structure within the target sentence. The target sentence (in italics) in set A was “He will be here tomorrow,” and in set B, “You wrote it last night” (see Appendix). Within sets C and D, the target sentences were not identical across the four paragraphs. As for sets A and B, the context of the paragraph cued a particular readingdeclarative, interrogative, happy, or sad. In addition, the semantics of the target sentences cued either a happy or sad reading, while the word order cued either a declarative or interrogative reading (see Appendix). In sets C and D, we were interested in determining whether the semantic loading of the target sentences would either augment or diminish the prosodic patterns associated with happy and sad sentences. Preliminary investigation (Danly et al., 1982) suggested that both normal control subjects and patients with right posterior damage tended to exaggerate prosodic variation when the semantics of the target sentences themselves contained no inherent affective tone (sets A and B). These subjects may have been attempting to enhance the emotional meaning through prosodic control when the inherent affective tone was

RIGHT HEMISPHERE

PROSODY

23

undefined. In contrast, when affective tone was cued by the semantics of the target sentences (sets C and D), the prosodic effect was relatively diminished. We must introduce a note of caution regarding the task used in the present study. All subjects were asked to simulate a particular emotional quality or propositional stance in a sentence reading task. This paradigm was specifically chosen in order to reduce the many influences on F,, variability that were unrelated to the propositional and affective distinctions we wished to test. While spontaneous speech has a greater degree of naturalness associated with it, it does not allow for control over the words spoken, their emphasis, or contextall of which can have as large an effect on F0 as the affective and propositional conditions we wished to test. Thus, any direct comparisons between individual subjects or groups of subjects using spontaneous speech would necessarily include all of the above factors that influence FO patterns, which could potentially obscure the specific influences of affect and propositional stance we hoped to control and measure. Furthermore, the validity of this paradigm gains some support from the work of Williams and Stevens (1972), who demonstrated some consistency in F,, patterns across emotionally toned phrases from simulated and spontaneous situations. Given a reading task, however, it is possible that group differences in prosodic control may be determined more by group differences in ability to produce an affective or propositional tone in a simulated situation, rather than differences in their ability to produce these spontaneously. However, we are unaware of any evidence that supports or predicts such group differences based on the ability to simulate an affective or propositional tone of voice. Procedure. Subjects were tested individually in a quiet room on two separate occasions. Paragraphs were presented to the subject in random order, with the following constraints: in session 1, subjects read one paragraph containing a propositional target sentence and one containing an emotional target sentence from sets A, B, C, and D, thus constituting eight paragraphs. In session 2, subjects read the remainder of the paragraphs. Paragraphs were presented to the subjects individually on typed sheets, with the target sentences punctuated as indicated in the Appendix, but without italics. The subject was read a standard set of instructions, asking him to read each paragraph silently, and then aloud, using as much expression or feeling as possible. Most importantly, though, he was encouraged to speak naturally. If the subject erred while reading a target sentence, he was always requested to repeat the entire paragraph. It was our impression that all of the patients understood the contents of the paragraphs, and we have no reason to believe that any particular patient group differed in comprehension level. For each of the four paragraph types, the mode of the target utterance was explicitly stated within the paragraph, making it unnecessary for the subject to infer the underlying mode of the target sentence. Each subject was allowed to read the paragraphs as many times as he wished, until he felt he had produced a good reading. Responses were recorded on a Sony TC-106A tape recorder via a Sony F-25 microphone. Acousrical analysis. Each target sentence was analyzed for FO values, using W. L. Henke’s Fundamental Period program (FPRD). The analysis was implemented on a PDP9 computer, at a sampling rate of 10 kHz, at the Research Laboratory of Electronics at MIT. The FPRD program measures fundamental periods of individual glottal cycles in the amplitude over time waveform, which are then inverted to provide an estimate of fundamental frequency over time. A second program displays a continuous plot of F0 and amplitude over time. Hard copies of the FO and amplitude plots were made, and FO measurements were taken for all peak and valley locations for each word in the target sentences. The following properties of FO which have been shown to be related to emotional states and propositional, were used as dependent measures for most of the analyses: 1. Average FO level at peaks: Sum of FO values at all peaks divided by the number of peaks per sentence.

24

SHAPIRO

AND DANLY

2. Average F0 level at valleys: Sum of F0 values at all valleys divided by the number of valleys per sentence. 3. Average F0 level at all peaks and valleys: Sum of F0 values at all peaks and valleys divided by the number of peaks and valleys per sentence. Note that measures 1 - 3 are closely related. F0 level at peaks is generally considered a more stable measure than F. at valley locations (Cooper & Sorensen, 1981). In this study, we compared all three measures. 4. F,, variability: Sum of all peak-to-valley and valley-to-peak changes in F,, level for the entire sentence. 5. F,, variability per variation: Average amount of change in F. level for each valley-topeak and peak-to-valley variation for each sentence (i.e., total amount of F, variability divided by total number of valley-peak and peak-valley variations). This provided a measure of average excursion per local F. contour. 6. F. excursion: Highest peak F. value minus the lowest valley F. value. This provided a measure of F. range. 7. F,, rise (interrogative sentences): The valley-to-peak rise in F. level associated with interrogative sentences. This measurement was taken on one word only, generally the last word in the interrogative target sentence. However, if a normal control subject produced a rise on another word in the target sentence that was at a predictable location for an interrogative rise, this location was also examined for a valley-to-peak rise in the sentences produced by the other subjects. In such a case, the word with the greatest valley-to-peak rise was used for the analysis.

RESULTS AND DISCUSSION Results are presented in the following manner. Each hypothesis outlined earlier is addressed using the FO measures detailed above. For most of the hypotheses, a three-factor Group (5) x Paragraph Set (4) x Sentence Type (2) repeated measures analysis of variance was carried out on the FO property or properties appropriate to the hypothesis. Exceptions are noted below. In each case the overall analysis of variance was followed by a set of orthogonal planned comparisons, weighted to reflect our predictions (Rosenthal & Rosnow, 1984). Hypothesis 1: Subject groups will differ in overall mean level of pitch, as measured by FO level at peaks, valleys, and combined peak and valley locations, averaged across all sentence types and all paragraph sets. Table 1 contains the mean FO values for all subject groups at peak, valley, and combined peak and valley locations. TABLE MEAN

FUNDAMENTAL

FREQUENCY (F,,) VALUES PEAK AND VALLEY LOCATIONS

1 (IN Hz) AT PEAK, FOR ALL SUBJECT

VALLEY, GROUPS

AND

COMBINED

Subject group Right anterior F. peaks F. valleys Combined F,, peaks and valleys

Right central

Normal control

Left posterior

Right posterior

142 126

122 105

162 134

132 108

182 150

135

112

147

118

163

RIGHT HEMISPHERE

PROSODY

2.5

As expected, right posterior patients had the highest mean F, level while right central patients had the lowest, with normal controls falling between them. However, the patients with left posterior damage produced much lower F0 values than expected, while the F,, values for the right anterior patients were much higher than expected, though not as high as the normal controls. The overall F for these measures was not significant. However, planned comparisons on two of the measures (F,, peaks and combined peaks and valleys) supported our predictions that patients with right posterior damage would display higher mean F,,, while normal control and left-posteriorbrain-damaged groups would display average mean F,, and patients with either right anterior or right central brain damage lowered mean F,,. Table 2 reports the F values, degrees of freedom, and significance levels of these tests. Hypothesis 2: Subject groups will differ in the amount of intonational fluctuation and range in their speech, as measured by F,, excursion, F. variability, and F. variability/variation averaged across all sentence types and all paragraph sets. Table 3 displays the mean F,, values for variability, variability per variation, and excursion measures for each subject group. It was expected that patients with right posterior brain damage would display greater F, variability and a greater F,, range than normal control and left-posteriorbrain-damaged subjects. Patients with either right anterior or right central damage were expected to display reduced F,, variability and a restricted F. range. The planned comparisons based on these predictions revealed that they were substantiated for all three measures of F,, modulation. (See Table 2.) Note that for the excursion measure, patients with right posterior damage had essentially the same mean range as the normal control group, though both groups, in addition to the left posterior group, had nearly twice as great an intonational range as either the right anterior or right central groups. Hypothesis 3: Subject groups will differ in their ability to produce the rises in F. associated with yes-no questions. This tested the subjects’ ability to intone their speech in a propositional context. It was predicted that while normal control and left-posterior-braindamaged subjects would produce an average rise in F,, right posterior patients would produce a larger F,, rise, while right anterior and right central patients would produce a greatly attenuated F. rise. Results in Table 2 indicate that these predictions were substantiated. The mean F. rise produced by normal control (64.5 Hz) and left posterior (61.5 Hz) subjects was nearly 100% greater than that produced by either right anterior (31 Hz) or right central (39.5 Hz) patients, while the rise produced by right posterior (79 Hz) patients was about 125% greater than that of

Overall Planned Overall Planned Overall Planned

Overall F Planned comparison

Sentence type x group interaction Planned comparison Planned comparison Planned comparison Planned comparison

Planned comparison

Hypothesis 2

Hypothesis 3

Hypothesis 4

Hypothesis 5

comparison

F

comparison

F

comparison

F

F

comparison

F

F comparison

Overall Planned Overall Planned Overall

Hypothesis 1

ANALYSESOF VARIANCE AND

2

2.14 8.37 2.97 11.25 3.94 14.96

2.37 6.73 2.17 5.14 1.94

F,,

variability

F(1,48) = 6.65

10.78 5.71 4.43 6.79

F(1,16) F(1,16) F(1,16) F(1,16)

peaks peaks and valleys variability per variation excursion

F,, F,, F. F.

= = = =

F(4,16) = 3.59

peaks

F(4,16) = 3.92 F(1,16) = 14.87

= = = = = =

= = = = =

F.

Interrogative rise Interrogative rise

F. F,, F,, F,, F,, F.

F(4,16) F(1,16) F(4,16) F(1,16) F(4,16) F(1,16)

l-5

variability variability variability per variation variability per variation excursion excursion

FOR TESTS OF HYPOTHESES

F(4,16) F(1,16) F(4,16) F(1,16) F(4,16)

COMPARISONS

TABLE F. peaks F. peaks Combined peaks and valleys Combined peaks and valleys F. valleys

PLANNED

.12 < .05 = .05 < .Ol = .02 < .Ol

< < < <

.005 .05 .05 .025 p < .05

p p p p

p < .03

p < .02 p < .005

p p p p p

p =

< .05 = .12 < .05 = .15

p < .lO p p p p

3 4

Y

z

0

% j;i

8

RIGHT HEMISPHERE TABLE MEAN

FUNDAMENTAL VARIATION,

FREQUENCY AND

(F,J

EXCURSION

VALUES

27

PROSODY 3

(IN

MEASURES

Hz)

FOR VARIABILITY,

FOR ALL

SUBJECT

VARIABILITY

PER

GROUPS

Subject group Right central

Normal control

Left posterior

Right posterior

162

144

207

202

263

22 51

19 52

29 90

26 78

35 90

Right anterior variability variability per variation F. excursion F. F.

the other right-brain-damaged groups. On perceptual evaluation of the interrogative F0 rise, only one patient with right brain damage (a right central patient) failed to produce an F,, rise on an interrogative target sentence. This indicates that the mean differences among subject groups in the amount of F,, rise for questions were due to rises that were produced consistently by the different subject groups, rather than a combination of normal rises and failures to rise. Hypothesis 4: Subject groups will differ in their ability to modulate F0 to signal the emotions happy and sad. This tested subjects’ ability to intone their speech in an affective context. In normal speech, happy sentences are produced with a higher mean F,, greater F0 range, and more F0 variability than sad sentences. Thus, the ability to signal a happy vs. sad emotion should be reflected in differences in the above measures for happy vs. sad sentences. Analyses of variance on measures 1 - 6 outlined above revealed a significant main effect for Sentence Type (happy vs. sad) for all six measures: F0 variability, F,, variability/variation, F0 excursion, and F,, level at peaks, valleys, and combined peaks and valleys (see Table 2 for statistics). Thus, averaged across all subject groups and paragraph sets, the happy sentences were produced with a higher mean F,,, greater F~ range, and more F,, variability than were the sad sentences (see Table 4 for mean F,, values for each subject group). The Sentence Type x Group interaction, which tested differences between groups in the ability to modulate F, to signal the happy vs. sad sentences, was significant for FO level at peak locations only, a finding which was not surprising, as F,, level at peaks is a more stable measure of average F,, while F,, level at valleys tends to be less differentiating (Cooper & Sorensen, 1981), given a reading paradigm. A series of planned comparisons on the interaction effect was performed. It was expected that while normal control and left-posterior-brain-damaged subjects would mark the happy/sad distinction through modulation of F,,

variability per variation

excursion

peaks

F.

F.

F,,

Combined F. peaks and valleys

F. valleys

variability

F,,

MEAN

FUNDAMENTAL

Sad Difference

Ham

Sad Difference

Happy

Happy Sad Difference Happy Sad Difference Happy Sad Difference Happy Sad Difference

FREQUENCY

(FO) VALUES

188 151 37 26 19 7 60 46 14 138 138 0 129 120 9 138 127 11

149 126 23 21 16 5 52 50 2 117 114 3 loo 99 1 109 105 4

Bight central 242 172 70 34 21 13 111 70 41 174 135 39 140 116 23 155 125 30

Normal control

Subject group

TABLE 4 Hz) FOR HAPPY AND SAD SENTENCES AND HAPPY-SAD

Bight anterior

(IN

248 147 101 32 19 13 88 68 20 142 115 27 111 98 13 124 105 19

Left posterior

DIFFERENCES

IN F. VALUE

328 212 116 44 26 18 107 68 39 196 161 35 153 139 14 171 149 22

Bight posterior

P

is u

2 tz

RIGHT HEMISPHERE

PROSODY

29

patients with right posterior damage would accentuate the happy/sad distinction (greater difference in FO values between happy and sad sentences), while patients with either right anterior or right central damage would show diminished capacity to mark the happy/sad distinction through F,, modulation (smaller difference in F,, values between happy and sad sentences). Table 4 contains the mean F,, values for all happy and sad sentences, and the mean difference in FO values between the happy and sad sentences, for all subject groups. Results showed that these predictions were substantiated for FO variability/variation, FO excursion, F,, level at peak locations, and F, level at peaks and valleys. In general, normal control subjects and patients with either right posterior or left posterior brain damage were able to signal either a happy or a sad sentence through modulation of F,,, while patients with right anterior or right central damage were quite limited in their ability to use FO to signal these emotions. Figure 1 contains F, contours (upper line) and amplitude measures (lower line) of the target sentence “He will be here tomorrow” in happy and sad contexts, for one subject representative of each subject group. Note that subject R.S., who had right posterior damage, showed more F, variability per variation, a greater FO range, and a higher mean FO level when the target sentence was uttered in a happy context than in a sad context. The same trend was observed in the speech of subject J.H., a normal control subject, and subject F.L., who had left posterior damage. In contrast, subject L.H., who had right central damage, did not display these differences in FO values for the happy vs. sad target sentences. The FO contours for both sentences look very similar. Subject A.S., who had right anterior damage, also failed to differentiate between the happy and sad target sentences through modulation of F,. While the above patients demonstrated common patterns of group differences among the subject groups, not all patients within each group produced identical prosody. We noted some heterogeneity within patient groups, commented upon under Conclusions. Hypothesis 5: The ability of subjects to modulate F, to signal the emotions happy and sad will be differentially affected by whether the sentences contain semantically neutral (paragraph sets A and B) vs. semantically loaded (paragraph sets C and D) affective words. Thus, some subject groups may accentuate the happy/sad distinction through modulation of speech prosody when the semantics of the sentences themselves are neutral, while others may signal the happy/sad distinction through prosody only when the semantics of the sentences themselves also reflect this distinction. A three-factor Group (5) x Paragraph Set (4) x Sentence Type (2) repeated measures analysis of variance on measures 1 - 6 outlined above revealed no significant Group x Paragraph Set x Sentence Type interactions. A series of planned comparisons was performed. Such an analysis

SHAPIRO

HAPPY CONTEXT

AND DANLY

SAD CONTEXT

Subject R.S. Right Posterior Brain Damage

Subject J.H. Normal Control

Subject F.L. Left Posterior Brain Damage

Subject L.H. Right Central Brain Damage

Subject AS. Right Anterior Brain Domoge

(SECONDS) 1. Fundamental frequency (upper line) and amplitude (lower line) contours for the target sentence “He will be here tomorrow” spoken in happy and sad contexts by one subject from each subject group. FIG.

is performed by first removing from the cell means the main effect for Group, Sentence Type, and Paragraph Set and the three first-order interaction effects, leaving only the residual second-order interaction effect (Rosenthal & Rosnow, 1984). It was predicted that patients with right posterior brain damage would modulate F,, to signal happy vs. sad sentences to a greater degree in

RIGHT

HEMISPHERE

PROSODY

31

semantically neutral than in semantically loaded contexts. In contrast, patients with either right anterior or right central brain damage would display the opposite pattern: given their limited ability to mark the happy/ sad distinction through modulation of F,, they would show more F,, modulation in semantically loaded than in semantically neutral contexts. A preliminary investigation of this question had shown that when the semantics of a sentence was relatively neutral, patients with right posterior damage appeared to enhance the emotional meaning through intonation, while patients with either right anterior or right central damage did not use this strategy. These latter groups, in fact, were more likely to use intonation to mark the happy/sad distinction when this distinction was also marked by the semantics of the target sentence. Results of this planned comparison were significant for F, variability only (Table 2). Thus, right posterior, normal control, and left posterior patients tended to signal the happy/sad distinction using variability in F, to a greater degree in the semantically neutral target sentences, while right anterior and right central patients did so more in the semantically loaded sentences. CONCLUSIONS

The present results indicate that in general, the clinical perception of flat speech that has been attributed to patients with right anterior brain damage is borne out by an acoustical analysis of the speech wave. These patients have less pitch variation in their speech, and a restricted intonational range. Patients with right central brain damage (including both pre- and post-Rolandic injury) display a similar pattern, along with a lowered mean F, level. Furthermore, this characterization appears to span both emotional and propositional domains. Patients with right posterior brain damage display just the opposite pattern: when their speech was subjected to an acoustical analysis, it was found that they speak at a higher mean F, level, and have more pitch variation and a greater intonational range than either right-anterior-, right-central-, normal control, or left-posterior-brain-damaged patients. In addition, there is some evidence to suggest that normal control subjects and patients with either left or right posterior brain damage tend to vary speech prosody to a greater degree in semantically neutral than in semantically loaded contexts. This may reflect a strategy used by these subjects to enhance the emotional meaning of the sentence, when the inherent affective tone was undefined. Patients with right anterior and right central brain damage did not use this strategy. While these patients were able to vary speech prosody to signal the happy/sad distinction, albeit to a very limited degree (see Table 3), they seem to have been aided in signaling this distinction through prosody when the emotional tone was also inherent in the meaning of the sentence itself. Patients with left posterior damage behave essentially like normal control

32

SHAPIRO

AND

DANLY

subjects in terms of F0 variability measures and F, range. In a previous study, Danly, Cooper, and Shapiro (1983) found that patients with left posterior damage produced significantly more F,, variability in a sentence reading task than did normal control subjects. However, the patients in that study were fluent aphasics with moderate to severe linguistic impairment. It was postulated at that time that the increased F. variability in that population may have been due to a combination of factors including their impaired articulatory control of F,, and increased use of emphatic stress, which would lead to higher F, values (Lehiste, 1970) and thereby increased F. variability. In the present study, we selected only mildly impaired left posterior patients, which may account for the lack of increased F. variability in their speech. Furthermore, none of these subjects displayed a high mean F. level: three spoke at a lower level than that of the normal control subjects, while the remaining two were comparable to the normal controls. To summarize our results, we agree with the conclusions of Weintraub et al. (198 1) and Cooper et al. (1984), as well as with the previous findings by Danly et al. (1982), that the prosodic deficits observed in patients with right brain damage may not be specifically tied to an affective context. These studies revealed several disturbances in speech prosody with right-brain-damaged patients in nonaffective contexts. Weintraub et al. found an inability to produce appropriate lexical and contrastive stress patterns, and Cooper et al. found elevated F, levels. Danly et al. showed that several local F,, patterns related to syntactic factors were normal, while more global F,, characteristics, such as mean F. and F. variation, were impaired in ways similar to those discussed here. In the present study, we have demonstrated faulty intonation patterns in the same patients across both propositional and affective contexts, thus suggesting an intrinsic deficit in the modulation of speech prosody that may be only fortuitously linked to the aberrant emotional qualities often found in this population. Furthermore, these conclusions have been substantiated by an acoustical analysis of the speech wave. The present results further indicate that there is a marked difference in speech prosody between those patients with damage to the anterior portions of the right hemisphere (including the right central patients) and those with damage to the posterior portions. Ross (1981) has noted that various deficits in affective speech may be linked with specific lesion sites in the right hemisphere. While the present data suggest a dissociation between the anterior and posterior portions of the right hemisphere in the control of speech prosody in general, this distinction may be overly simplified. Not all of the patients with right posterior damage displayed exaggerated intonation contours across all measures when compared to the normal control or left-posterior-brain-damaged subjects, nor did all of the patients with right anterior damage display comparatively flat

RIGHT HEMISPHERE

PROSODY

33

contours across all measures. These exceptions are most likely due to the fact that our ability to specify the anatomical correlates of speech melody is only just emerging. Thus, the rather gross subject categories we used may not have captured the relevant anatomical distinctions, thereby creating heterogeneous classifications. In addition, speakers in general tend to vary different aspects of F, in emotional situations (Williams & Stevens, 1972). To reiterate our caution regarding the task used in the present study, it is possible that the group differences in prosodic control were determined by differences in ability to produce an affective or propositional tone in a simulated rather than spontaneous situation. To clarify this issue, a future study of both spontaneous speech and read passages using the same subjects for both conditions may be useful. In conclusion, we have shown that patients with damage to the anterior portions of the right hemisphere display flat speech across both emotional and nonemotional domains, whereas those with damage to the posterior portions appear hypermelodic in these contexts. Furthermore, no such deficits were found in patients with left posterior brain damage. These findings, which have been confirmed by an acoustical analysis of the speech wave, suggest that damage to the right hemisphere alone may result in a primary disturbance of speech prosody that is not necessarily linked to the disturbances in affect often noted in right-brain-damaged populations. APPENDIX A-l Pam could not remember the message that Bob had left for her. Although she knew it was something about Ted, she could not remember it exactly. When she saw Bob approaching her from the distance, she asked what the message from Ted had been. Bob told her, “He will be here tomorrow.” Pam said, “Thanks for the message,” and continued on her way. A-2 Alice was not quite sure when Rick would be arriving from Denmark. Although he had written he would be back on Saturday, friends had said he may come as early as Friday. When the phone rang, Alice recognized the voice of Rick’s friend in Denmark at the other Rick’s end of the line. “What did you say,” she asked, “He will be here tomorrow?” friend answered “Yes,” and they finished the conversation. A-3 The family joyously awaited John’s arrival. Although he had been away from home for many years, he would be home for Christmas this year with the whole family. Everyone helped with cooking a big meal for the next day. Mother was especially happy as she announced with great delight, “He will be here tomorrow.” The family was excited as they thought of John’s arrival. A-4 Pete and his family were very sad when they left their farm after so many years. They had lost all their crops in a big flood, and could no longer make their monthly payments

34

SHAPIRO

AND DANLY

to the bank. That night, as they took their last walk around the farm, Pete asked his mother when the man from the bank would come by to take over the farm. She answered with great sorrow, “He will be here tomorrow.” The family knew they would never be able to afford another farm. B-l Sarah had received a beautiful plant from her mother-in-law for her birthday the week before. Although she thought she had sent out the thank-you note, she could not actually remember having written it. She asked her husband Jack if he remembered her having written the note. Jack thought about it for a moment and said, “You wrote it last night.” Then Sarah remembered that she had already written the note. B-2 Janet had finally written out the rent check as it was due the next day. She had been intending to write it out all week but something had always distracted her. When her husband called to remind her of the check, she told him she had written it the night before. “What did you say,” he asked her, “You wrote it last night?” Janet answered “Yes,” and asked him when he would be home for dinner. B-3 Bob and Mary waited with delight, as Nick had promised to write a song for their wedding. Nick had been planning the song for a while and had finally written it the night before. When Nick came over to visit that night, he announced that he had just written the song the night before. “That’s wonderful” said Bob, as he repeated with great joy, “You wrote it last night.” After dinner, Nick sat down at the piano and played the song for them. B-4 Jack was quite sad that his marriage to Nancy was breaking up. He was hoping Nancy would decide not to write out the divorce agreement until they had a chance to speak again. When he called her on the phone to set up a meeting, she told him she had finished writing the divorce agreement the night before. “1 see,” he said with great sorrow, “You wrote it last night.” Jack knew that the marriage was really over. C-l

Barry couldn’t decide what color hat to buy. Although the blue one was nice, the brown one looked better with his jacket. When the clerk asked him if he wanted the brown one, he thought about it for a minute. Then he said, “The brown one isJine.” The clerk handed him the brown hat, and he paid for it. c-2 Paul wasn’t sure if Frank would go to the movies with him that night or not. If Frank would not go, then Paul would make other plans. He decided to call Frank to see what his plans were. When Frank answered the phone, Paul asked him, “Will you be going tonight?” Frank answered “Yes,” and they made plans to meet at seven. c-3 Janie called Sally to invite her to attend an opening performance at the theater with her. Janie had gotten two free tickets, and knew that Sally would be happy to go with her. When Janie invited her, Sally was elated. She told Janie with great joy, “I would love fo attend.” Sally and Janie decided to meet at the theater at eight. c-4 Tom was saddened by the news he had to tell his neighbor Nick. Nick had been away on vacation, and Tom was watching his house for him. But last night the house had caught

RIGHT HEMISPHERE

35

PROSODY

on fire and was completely burned down. When Tom called Nick, he told him with great Nick said he would come home the next day. sorrow, “The whole house was destroyed.” D-l Don was not sure how to use the train system in Boston. He was told to take the green line if he wanted to get to Government Center. As Don waited on the platform, he asked the man standing next to him to let him know which train went to Government Center. As the next train was pulling into the station, the man told Don, “The next train is yours.” Don thanked the man, and boarded the train. D-2 Dan asked the salesclerk if she could show him a particular sweater. He had seen the sweater in the store the week before, but did not see it today. Dan described the sweater, and the salesclerk went in the back room to look for it. As she came out with a sweater in her hands, she asked Dan, “Zs this the right one?” Dan answered “Yes,” and bought the sweater. D-3 Jim was quite cheerful when he came home from work this day. He had asked his boss for a raise, and the boss had agreed to one. When his wife Donna came home, he told her the good news. “Guess what,” he said with great joy, “I am getting a raise.” Donna congratulated him on the raise. D-4 Doug was quite depressed when he learned he would not be able to attend the Naval Academy. His father and brothers had gone there before him, and he had been looking forward to it since high school. He called his father to tell him the bad news. Doug told his father, with great sadness, “I did not get accepted.” His father told him he was sorry to hear the news.

REFERENCES Cooper, W. E., Soares, C., Nicol, J., Michelow, D., & Goloskies, S. 1984. C!ausal intonation after unilateral brain damage. Language and Speech, in press. Cooper, W. E., & Sorensen, J. M. 1981. Fundamental frequency in sentence production. New York/Berlin: Springer-Verlag. Cosmides, L. 1983. Invariances in the acoustic expression of emotion during speech. Journal

of Experimental

Psychology:

Human

Perception

and Performance,

9, 864-

881. Danly, M., &Shapiro, B. E. 1982. Speech prosody in Broca’s aphasia. Brain and Language, 16, 171-190. Danly, M., Cooper, W. E., & Shapiro, B. E. 1983. Fundamental frequency, language processing, and linguistic structure in Wemicke’s aphasia. Brain and Language, 19, l-24. Danly, M., Shapiro, B. E., & Gardner, H. 1982. Dysprosody in right brain damaged patients: Linguistic and emotional components. Paper presented at the Academy of Aphasia, New Paltz, New York, October, 1982. Dordain, M., Degos, J. D., & Dordain, G. 1971. Troubles de la voix dans les hemiplegies gauches. Revue de Laryngologie, 92, 178-188. Gardner, H. 1975. The shuttered mind. New York: Knopf. Gardner, H., Ling, P. K., Flamm, L., & Silverman, J. 1975. Comprehension and appreciation of humorous material following brain damage. Brain, 98, 399-412.

36

SHAPIRO

AND DANLY

Huttar, G. L. 1968. Relations between prosodic variables and emotions in normal American English utterances. Journal of Speech and Hearing Research, 11, 481-487. Jackson, J. H. 1915. On affections of speech from diseases of the brain. Brain, 38, 106174. Lehiste, I. 1970. Suprasegmentals. Cambridge, MA: MIT Press. Lieberman, P., & Michaels, S. B. 1%2. Some aspects of fundamental frequency and amplitude envelope as related to the emotional content of speech. Journal of zhe Acoustical Society of America, 24, 922-927. Monrad-Krohn, G. H. 1947. Dysprosody or altered “melody of language.” Brain, 70,405423. Rice, D. G., Abroms, G. M., & Saxman, J. H. 1969. Speech and physiological correlates of “flat” affect. Archives of General Psychiatry, 20, 566-572. Rosenthal, R., & Rosnow, R. L. 1984. Essentials of behavioral research: Methods and data analysis. New York: McGraw-Hill. Ross, E. D. 1981. The aprosodias: Functional-anatomic organization of the affective components of language in the right hemisphere. Archives of Neurology, 38, 561-569. Ross, E. D., Hamey, J. H., delacoste-Utamsing, C., & Purdy, P. 1981. How the brain integrates affective and propositional language into a unified behavioral function. Archives of Neurology, 38, 745-748. Ross, E. D., & Mesulam, M. 1979. Dominant language functions of the right hemisphere? Prosody and emotional gesturing. Archives of Neurology, 35, 144-148. Tucker, D. M., Watson, R. T., & Heilman, K. M. 1977. Discrimination and evocation of affectively intoned speech in patients with right parietal disease. Neurology, 27, 947950. Wapner, W., Hamby, S., & Gardner, H. 1981. The role of the right hemisphere in the apprehension of complex linguistic material. Brain and Language, 14, 15-33. Weintraub, S., Mesulam, M., & Kramer, L. 1981. Disturbances in prosody: A right hemisphere contribution to language. Archives of Neurology, 38, 742-744. Williams, C. E., & Stevens, K. N. 1972. Emotions and speech: Some acoustical correlates. Journal of the Acoustical Society of America, 52, 1238-1250.