Co-speech hand movements during narrations: What is the impact of right vs. left hemisphere brain damage?

Neuropsychologia 93 (2016) 176–188

Contents lists available at ScienceDirect

Neuropsychologia journal homepage: www.elsevier.com/locate/neuropsychologia

crossmark

Co-speech hand movements during narrations: What is the impact of right vs. left hemisphere brain damage? ⁎

Katharina Hogrefea, , Robert Reinb,c, Harald Skomrochd, Hedda Lausbergd a Clinical Neuropsychology Research Group (EKN), Institute of Phonetics and Speech Processing (IPS), Ludwig-Maximilians-Universität München, Schellingstr. 3, 80799 Munich, Germany b Institute of Cognitive and Team/Racket Sport Research, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933 Köln, Germany c Institute of Sport Sciences, Goethe University, Ginnheimer Str. 39, 60487 Frankfurt / Main, Germany d Department of Neurology, Psychosomatic Medicine, and Psychiatry, Institute of health promotion and clinical movement science, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933 Köln, Germany

A R T I C L E I N F O

A BS T RAC T

Keywords: Hand movement behaviour Non-verbal communication Gesture Aphasia Right brain damage Left brain damage

Persons with brain damage show deviant patterns of co-speech hand movement behaviour in comparison to healthy speakers. It has been claimed by several authors that gesture and speech rely on a single production mechanism that depends on the same neurological substrate while others claim that both modalities are closely related but separate production channels. Thus, findings so far are contradictory and there is a lack of studies that systematically analyse the full range of hand movements that accompany speech in the condition of brain damage. In the present study, we aimed to fill this gap by comparing hand movement behaviour in persons with unilateral brain damage to the left and the right hemisphere and a matched control group of healthy persons. For hand movement coding, we applied Module I of NEUROGES, an objective and reliable analysis system that enables to analyse the full repertoire of hand movements independent of speech, which makes it specifically suited for the examination of persons with aphasia. The main results of our study show a decreased use of communicative conceptual gestures in persons with damage to the right hemisphere and an increased use of these gestures in persons with left brain damage and aphasia. These results not only suggest that the production of gesture and speech do not rely on the same neurological substrate but also underline the important role of right hemisphere functioning for gesture production.

1. Introduction 1.1. Gesture, speech, and laterality of hand movements During communication we produce spontaneously hand movements. These hand movements comprise irregular, continuous selftouching behaviour as well as gestures with a phase structure that are based on conceptual processes. Hand movements have been studied in different fields of research. Most attention was dedicated to the relationship of gesture, speech and communication. To date, it is widely acknowledged that gesture production interacts with speech production. The precise nature of this relationship however remains unclear. Whereas some authors argue that gesture and speech originate from the same representation and are inseparable throughout the production process (Butterworth and Hadar, 1989; McNeill, 1992,

2005), others claim that gesture and speech production are separate but closely coordinated processes (De Ruiter, 2000; Feyereisen, 1987; Kita and Özyürek, 2003; Krauss et al., 2000). In this respect, the question whether language and gesture originate from the same neural substrate is of great interest. Kimura (1973) observed that healthy speakers with right hand preference produced more communicative gestures (“free movements” in her terminology, including all movements of the hands or arms which did not result in touching the body or coming to rest) with the right hand, also regardless of the speaking topic. It was also reported that self-touching behaviour, i.e. movements that resulted in touching the person's own body or clothing, was produced equally often with the left and the right hand (Lavergne and Kimura, 1987). These findings led to the conclusion that the generation of gestures is obligatory tied to linguistic processes and that speech and gesture originate from a common neural system (compare Lausberg

⁎ Correspondence to: Clinical Neuropsychology Research Group (EKN), Institute of Phonetics and Speech Processing, Ludwig-Maximilians-Universtität München, Schellingstr. 3, 80799 Munich, Germany. E-mail address: [email protected] (K. Hogrefe).

http://dx.doi.org/10.1016/j.neuropsychologia.2016.10.015 Received 10 April 2016; Received in revised form 18 October 2016; Accepted 24 October 2016 Available online 27 October 2016 0028-3932/ © 2016 Elsevier Ltd. All rights reserved.

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

bances that are probably caused by greater lesions may primarily make use of a reduced set of gestures. Other studies investigated the expression of meaning via gesture in PWA: Some individuals with severe aphasia convey more information via gesture than via speech (Hogrefe et al., 2013) and gesture contributes to the expression of meaning in PWA (De Beer et al., in press). These studies lend empirical support to the notion that persons with aphasia use gestures as a communicative device as compensation for their reduced language expression. Further, neuropsychological disorders that have been attributed to LBD have been shown to have an impact on gesture production. Limb apraxia may lead to a disturbed production of pantomimes to command (e.g. Goldenberg, Hermsdörfer, Glindemann, Rorden, and Karnath, 2007; Goldenberg and Randerath, 2015; Tarhan et al., 2015). Furthermore, limb apraxia may impact on the intelligibility of spontaneously produced gestures in persons with severe aphasia (Feyereisen et al., 1988; Hogrefe et al., 2012). Additionally, non-verbal semantic disorders lead to a reduced diversity of gestures in this patient group (Hogrefe et al., 2012).

and Kita (2003) for a detailed critical review of Kimura's experiments). More recent research investigating handedness contradicts Kimura's notion of a right hand preference for communicative hand gestures. An equal use of the right and left hands was reported for iconic gestures (Blonder et al., 1995; Lausberg and Kita, 2003) and self-touching behaviour (Blonder et al., 1995) in healthy speakers. Furthermore, a range of studies with healthy speakers as well as splitbrain patients indicate that hand preference for gesturing is tied to gesture type in that specific gesture types are predominantly produced with either the right or the left hand (for an overview compare Lausberg, 2013, p. 33 ff). There seems to be evidence for a specialization of the left hemisphere for iconic, pantomimic, and deictic gestures as well as for self-touching behaviour that involves the manipulation of objects. In contrast, the right hemisphere seems to play an important role in the generation of rhythmic gestures, i.e. batons, and continuous self-touching behaviour of the speaker. These findings support a bilateral system responsible for gesture production and self-touching behaviour. Studies investigating persons with unilateral left hemisphere damage (LBD) or right hemisphere brain damage (RBD) support the thesis that both hemispheres contribute to hand movement behaviour. Most of these studies investigated persons with LBD following the rationale that knowledge about the impact of aphasia on gesture use may indicate the precise relationship of gesture and speech, whereas some studies also focused on persons with RBD. At present, only a few studies have compared the two groups with each other.

1.3. Gesture production in persons with right hemisphere damage It is widely acknowledged that the right cerebral hemisphere plays – amongst others – an important role for the processing and production of emotions and pragmatic aspects of communication. Persons with damage to the right hemisphere usually do not display systematic linguistic deficiencies but often a communication disorder that may affect narrative-discourse abilities, the processing of metaphors or idioms as well as the processing of prosody (e.g. Brownell et al., 1995). Côté et al. (2007) estimated that approximately 50% of the persons with damage to the right hemisphere display subsequent communication disorders. There are some descriptions of persons with RBD that deal with the display of emotions through non-verbal expression (Ross, 1981, 1996; Ross and Mesulam, 1979). Most of these studies focus on the specific characteristics of prosody, but some include also “body language”. Ross and Mesulam (1979, p. 148) report two patients who “evidenced an inability to communicate emotions through the use of facial, limb, and body gestures”. The authors use the term “agestural” for this state and claim that it accompanies the “aprosody” in patients with flattened affect. The observation that persons with RBD produce one type of communicative conceptual gestures, namely iconic gestures, at a lower rate than healthy or aphasic speakers has also been reported by Hadar et al. (1998). On the other hand, it has been described by Blonder et al. (1995) for a group and by Cocks et al. (2007) for two persons that RBD enhances the production of self-touching behaviour. Reasons for these deviant gesture use patterns have been attributed to visuo-spatial deficits (Hadar et al., 1998; McNeill and Pedelty, 1995) or viewed in close relationship with disturbed prosody (Ross and Mesulam, 1979). However, these suggestions were not confirmed by more recent studies (Cocks et al., 2007; Hogrefe et al., 2011). Taken together, even if the reported studies did not reveal very clear results with respect to hand movement behaviour, the majority supports findings from healthy speakers and split brain patients in that damage to the right hemisphere may lead to a reduction of iconic gestures and an increase in self-touching movements.

1.2. Gesture production in persons with left hemisphere damage Investigating the impact of aphasia on gesture production can reveal more insights into the relationship of gesture and speech (for an overview compare Rose (2006)). Results so far do not show a straightforward relationship between language disorder and gesture use: Some authors argue that gesture and speech break down together in persons with aphasia (PWA; e.g. Cicone et al., 1979; Glosser et al., 1986; McNeill, 1985) – a finding that supports Kimura's hypothesis – whereas other studies suggest that PWA may compensate with gestural communication for their reduced verbal output (e.g. Ahlsen, 1991; Behrmann and Penn, 1984; Beland and Ska, 1992; Herrmann et al., 1988; Hogrefe et al., 2013; Le May et al., 1988). According to the first view, gesture production is disturbed and reflects type (Cicone et al., 1979) and severity of aphasia (Glosser et al., 1986). Hence, the language impairment leads to a parallel impairment of the two modalities with gesture displaying the same characteristics as the verbal output. This view is in line with the classical concept of “asymbolia” which was introduced by Finkelnburg (1870). Finkelnburg claimed that aphasia is one particular manifestation of a general disability to display concepts by means of signs. In contrast to this view, some studies argue for a compensatory use of gestures in PWA who have been shown to produce more communicative gestures or specific gestures types than healthy speakers (Herrmann et al., 1988; Le May et al., 1988; Sekine and Rose, 2013). In a recent study, Sekine and Rose (2013) classified gestures of persons with differing degrees of aphasia severity according to twelve different gesture types. The authors analyzed if a gesture type appeared at least once in the discourse sample and found that – in contrast to the healthy control persons – PWA used the full range of gesture types. Interestingly, the only person with global aphasia included in the sample produced two gesture types only, namely deictic and emblematic gestures. This finding is in line with the study of Herrmann et al. (1988), who showed that in comparison to healthy speakers, persons with severe aphasia produce more emblematic gestures including head shakes, nods, and shoulder shrugs in conversations. These results indicate that persons with left hemisphere damage may use the full range of the gestural repertoire. However, the data further suggest that individuals with more severe language distur-

1.4. Comparisons of persons with left and right hemisphere damage To date, only few studies have compared these two groups with each other. As mentioned above, Blonder et al. (1995) showed that persons with RBD produced more self-touching movements on their own body with their right hand than did persons with LBD and persons without neurological disorders. Hadar et al. (1998) found differing rates for the production for communicative gestures in two patient groups and a control group: The persons with RBD produced fewer communicative 177

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

neuropsychological dysfunctions. 33 patients suffered from hemiparesis contralateral to their brain lesion. We only included patients with focal brain damage due to stroke or traumatic brain injury. Patients had no other neurological diseases. All patients were alert and able to follow the instructions. All LBD participants were also included in the study of Hogrefe et al. (2012). The control participants (CON, age range 33–72 years, M 52.9 ± SD 11) were paid for their participation. All participants gave informed consent to participate in the study that had been approved by the Ethical Committee of the Bavarian Medical Board. Demographic data is presented in Table 1.

gestures than healthy persons and PWA whereas PWA had a higher production rate compared to healthy speakers and persons with RBD. Also the distribution of gesture types differed between the groups with the RBD participants producing less iconic gestures than the other two groups. A recent study by Rousseaux et al. (2010) used a broader approach in applying a communication test including three main parts: participation to communication, verbal communication, and non-verbal communication to different patient groups. With respect to non-verbal communication, their main findings showed that persons with LBD showed an increase in gesture production whereas persons with RBD displayed disturbances in non-verbal pragmatic behaviour such as using affective expression at the face or the limbs, orientating gaze, and using gestures to coordinate the turn-taking (Rousseaux et al., 2010, p. 1106).

2.2. Clinical assessment 2.2.1. Assessment of hand preference Hand preference was determined with a questionnaire (Hermsdörfer et al., 1994) that is based on the Edinburgh Inventory of Handedness (Oldfield, 1971) but uses some items that are different. For instance, the questionnaire does not involve the items “writing” and “drawing” as hand preference for these items has been shown to be culturally influenced (compare Salmaso and Longoni (1985)). It contains the following items: throwing, using scissors, using a comb, using a toothbrush, using a knife (without fork), using a spoon, using a hammer, using a screwdriver, striking a match, threading a needle.

1.5. Aims of the present study In the above cited studies, methods for data elicitation and data analysis differ a lot which makes a comparison of the results difficult. Most of the studies investigated only specific gesture types, e.g. iconic gestures, instead of the full repertoire of hand movements that accompany speech. Finally, only a few studies compared individuals with LBD and individuals with RBD. To our knowledge, there is to date no study that systematically analyzed the full range of hand movement behaviour with a differentiated coding system in persons with LBD versus persons with RBD. In the present study, we aimed to fill this gap by systematically comparing hand movement behaviour in persons with LBD vs. persons with RBD as well as vs. healthy persons. For hand movement coding we applied Module I of the NEUROGES coding system (Lausberg, 2013; Lausberg and Sloetjes, 2009) which segments and classifies the ongoing stream of hand movements that accompany speech and thereby analyses the whole repertoire of hand movements. NEUROGES allows coding independently of the analysis of the concomitant speech and includes all kind of hand movements. Therefore, it is specifically suited to examine PWA. We hypothesize that gesture and language production do not depend on the same neurological substrate as it has been claimed by Kimura (1973). Consequently and in line with previous research, we expect that persons with LBD – due to their reduced verbal capacities – will produce more communicative gestures with conceptual content, i.e. hand movements in space according to NEUROGES, than healthy speakers and persons with RBD. The latter are expected to produce a higher amount of self-touching movements, as has been shown by Blonder et al. (1995) and Cocks et al. (2007) before. Furthermore, in line with previous descriptions, RBD participants are expected to produce less communicative gestures with conceptual content, i.e. hand movements in space in comparison to the two other groups.

2.2.2. Aphasia assessment in LBD participants The Aachen Aphasia Test (AAT; Huber et al., 1983) was administered to all LBD participants. It consists of a verbal communication scale and of five subtests: Token Test, naming, comprehension, repetition, and written language. Aphasia was classified following the test instructions. The verbal communication scale evaluates a participant's verbal performance in a semi-structured interview on a scale from 0 to 5. Six participants received a rating of 0 (no comprehensible verbal expression and marked reduction of verbal comprehension). Eleven participants were rated 1 (communication only by fragmentary and mostly incomprehensible utterances. The listener must infer the meaning of the utterances by deduction, questioning, and guessing), and three received a rating of 2 (conversation about familiar topic is possible with help of the communication partner, but the patients frequently fail to convey their ideas; compare Table 2). 2.2.3. Assessment of limb apraxia in LBD participants Limb apraxia was assessed by a pantomime-to-command task (Goldenberg et al., 2003, 2007). Participants were asked to mime the use of twenty common objects (e.g. cutting with scissors). The examiner named the action and the object while (s)he showed a photograph of the object. The patients' performances were video-taped. For the analysis, the examiner marked the presence of predefined features of the pantomime on an examination sheet (e.g. cutting with scissors: bended fingers with opposition of the thumb, opening and closing of hand or fingers, hand oriented perpendicular to table, movement of whole hand parallel to table). The total number of features across all items represented the total score in this task. The maximum score is 55, values below 45 are pathological (Goldenberg et al., 2007; compare Table 1).

2. Method 2.1. Participants Altogether 38 patients as well as 20 healthy controls took part in this study. All patients were recruited from the Clinic for Neuropsychology, Hospital Bogenhausen, Municipal Clinic München GmbH, where they underwent neuropsychological diagnostics and treatment. The patients suffered from unilateral focal lesions, which were caused in 37 of the cases by first-ever cerebrovascular accident (CVA) and in one case by trauma. 20 patients were suffering from left hemisphere brain damage (LBD, age range 35–68 years, M 50.9 ± SD 8.2) and 18 patients from right hemisphere brain damage (RBD, range 40–66 years, M 53.5 ± SD 8.3). All of the patients were at least three months post-onset of the brain damage (LBD, range 3–53, M 12.3 ± SD 12.4; RBD, range 3–11, M 4.5 ± SD 2.4) and they displayed other

2.2.4. Assessment of non-verbal semantic processing deficits in LBD participants Clinical diagnostics included an assessment of non-verbal semantic processing. Two subtests (BOSU 2 and BOSU 3) of the Bogenhausener Semantik-Untersuchung (BOSU; Glindemann et al., 2002), a test similar to the Pyramids and Palm Trees Test (Howard and Patterson, 1992), were administered. Each subtest consists of 10 items. The items display images of four objects, three of which have a semantic property which is missing in the fourth. In BOSU 2, the odd picture differs from the other by a main semantic feature (e.g., rabbit, pig, goat, and iron), whereas in BOSU 3, the odd picture differs from the others by a 178

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Table 1 Demographic and clinical characteristics of participants with left and right hemisphere damage as well as demographic data of the control participants. Participant

Gender

Age

Handedness

Profession

t post onset

Aetiology

Lesion site

Hemiparesis

Apraxiaa

Semantic processingb

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17 L18 L19 L20

M M F M F M M M F M M M M W M M M M F M

41 68 46 58 55 48 53 53 55 57 61 35 47 43 48 60 39 50 51 52

R 100 R 100 100 100 R 100 100 100 100 100 100 100 R 100 100 100 Ambidexter R

Engineer Estate Agent retired Housewife Mechanic Teacher Fireman Interior decorater No information No information Electrician Farmer Consultant Typographer Housewife Insurance Agent Medical Doctor Physician Technician Administrative officer Manager

7 6 3 4 12 15 24 32 4 12 7 19 5 5 3 3 53 17 11 4

hem isch isch isch isch isch isch hem isch isch isch chi isch hem isch isch isch isch isch hem

SC T SC F F,T, P, BG F, T. BG F, T. BG F, T. BG F, T, P, BG SC F, T, BG F, T, P, BG F, T F, T F, T, P, O, BG F, T, O F, T, P. BG F, T T T

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Residual Yes Yes Residual Yes Yes No No

36 45 43 24 15 50 41 42 47 51 12 45 53 49 22 52 47 7 42 46

6 0 0 7 2 1 2 1 2 2 5 1 4 2 5 1 1 8 3 3

Participant R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R18

Gender F M M M F M F M M F M M F F M F M F

Age 43 54 60 59 43 66 53 65 64 44 54 55 40 53 60 46 59 48

Handedness 100 25 100 100 100 100 82 100 100 100 100 100 100 82 100 54 100 100

Profession Software developer Insurance Agent No information Technician Buying Agent Electrician (retired) Teacher Engineer Metalworker Doctor's assistant Secretary Mechanic Biologist Salesclerk Safety/at/work expert Salesclerk Sociologist Masseuse

t post onset 7 3 3 4 9 3 3 11 5 5 3 6 3 3 3 3 3 4

Aetiology hem isch isch hem isch isch isch isch isch hem hem isch isch hem hem isch isch hem

Lesion site T F, T F T NID BG, SC SC T F, T, P, O F P, O T NID F NID T F NID

Hemiparesis Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Residual Yes Yes Yes Residual

Neglect multimodal multimodal multimodal multimodal multimodal multimodal multimodal multimodal multimodal multimodal visual multimodal multimodal Residual visual multimodal multimodal multimodal No

Visual spatial deficits Yes Yes Yes Yes Yes Yes Yes No info Yes Yes Yes No No Residual Yes Yes Yes No

Participant C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19

Gender F M F F M F M F M F M M M F F F M M M

Age 33 56 51 51 39 33 58 58 72 47 44 58 62 64 44 51 52 63 71

Handedness 100 100 100 100 100 100 82 100 100 82 100 82 100 100 100 100 66 100 100

Profession Psychologist Engineer Doctor Occupational Therapist Mechanic Singer Administrative officer Administrative officer Maintanance Man Jobless Caregiver Educator Kindergarten (retired) Administrative Officer Social education worker Engineer Tutor for school kids Engineer Physicist Miller (retired)

– – – – – – – – – – – – – – – – – – –

– – – – – – – – – – – – – – – – – – –

– – – – – – – – – – – – – – – – – –

– – – – – – – – – – – – – – – – – – –

– – – – – – – – – – – – – – – – – – –

– – – – – – – – – – – – – – – – – – –

subject: L = LHD, R = RHD; C = Control participants; gender: M = male, F = female; age: in years; t post onset: in months; handedness: laterality index or information obtained by patients themselves or their relatives: R = right hander; aetiology: isch = ischemic infarction of medial cerebral artery, hem = hemorrhagic infarction, chi = closed head injury; token test: percentile values; Apraxia: raw scores, < 45= impaired (bold); lesion sites: F = frontal; T= temporal; P= parietal; O = occipital; BG = basal ganglia; SC = subcortical lesion; NID = no imaging data available. a Pantomime-to-command test (Goldenberg et al., 2003, 2007), cut-off < 45. b BOSU 2+3 (Glindemann et al., 2002): error scores.

2.2.5. Hemiparesis, neglect, spatial deficits Information on the diagnosis of hemiparesis, neglect, and possible spatial deficits were taken from the medical reports.

subordinate semantic feature (e.g., toaster, fridge, mixer, and television). The participants are asked to indicate the one picture that makes the exception. For analysis, we used the total number of errors that were committed in the two subtests (compare Table 1).

179

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Table 2 Aphasia syndrome, verbal communication score, results of token test and naming as well as Identification Rates (taken from Hogrefe et al. (2012)) and total number of conceptual hand movements (compare Section 3.3.5) of participants with left hemisphere damage. LBD participants

Aphasia syndrome

Verbal communicationa

Token testa

Naminga

Identification Rate (Hogrefe et al., 2012)

Number of conceptual hand movements

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17 L18 L19 L20

global global Broca global global global global mTCA global Wernicke global global Wernicke global global Wernicke Broca global global global

1 0 2 1 1 0 1 1 1 2 0 0 1 1 0 1 2 1 0 1

10 65 21 33 24 21 38 33 16 48 24 2 5 22 5 2 16 13 7 7

18 6 28 19 20 6 14 29 32 48 14 6 21 26 6 28 66 17 6 20

89 78 15 28 44 61 61 61 67 89 44 89 83 78 50 78 94 22 94 72

73 58 53 36 64 154 110 74 92 103 122 87 83 74 85 153 148 38 36 65

mTCA = mixed transcortical aphasia, token test = percentile values, naming = percentile values. a Aachen Aphasia Test (Huber et al., 1983).

accompanying hand movements, which not only includes gestures but likewise discrete self-touching, fidgeting, shifts, etc. Module II provides a fine-grained analysis of the laterality of the hand movements, and Module III is most comparable to traditional gesture coding systems such as by Efron (1941) or McNeill (1992), as it focuses on the function and meaning of conceptual hand movements including gestures. As recent studies, e. g. Blonder et al. (1995; see Introduction), have reported alterations not only of gestural but also of self-touching behaviour in persons with brain damage, for the present research question we chose the Module I of the NEUROGES system (version 2009). Module I differs notably from those behavioural coding systems such as by Ekman and Friesen (1969) that operate with functional values such as the values adaptor or regulator. While Ekman & Friesen's system implies a priori assumptions about the function of hand movements, the NEUROGES system follows a primarily descriptive approach focussing on the visual appearance of the hand movement. Based on an assessment algorithm, in three subsequent assessment steps (categories) the ongoing stream of hand movements is segmented and is classified into behavioural units (compare Fig. 1). In the first evaluation step (Activation category), the stream of hand movement behaviour is segmented regarding a set of three movement criteria (presence or absence of motion, presence or absence of antigravity position, and presence or absence of muscle contraction) and thereby classified into movement units versus no-movement (rest position/posture) units. The Activation category registers the overall degree of motor activity, as measured by frequency of movement units (number/minute) in the right and left hand. The category covers traditional clinical concepts of hyper- and hypoactivity, as quantitative disorders in movement activity, such reported for anorexia nervosa, depression, attention deficit hyperactivity disorder, etc. In the second assessment step, the movement units are submitted to the Structure category analysis that focuses on the movement trajectory. Hand movements that have a transport (one-dimensional trajectory starting from rest position), a complex (complex one-way trajectory, hand shaping, movement dynamics), and a retraction phase (one-dimensional trajectory back to rest position) are labeled phasic (Lausberg, 2013). The tripartite trajectory structure reflects that the hand is transported to a specific location with the purpose to realize a

2.3. Stimulus materials and procedures 2.3.1. Stimulus material The stimulus material was the same as used in Hogrefe et al. (2012, 2013). It consisted of 10 video clips. The duration of the clips varied between 30 and 90 s. 2.3.2. Procedure The video-clips were presented on a notebook computer. Immediately after each clip participants were asked to retell the story in a vivid and imaginable manner. They were told that someone who had not seen the video should be able to infer the content of the story from their narration. The instruction did not include a command to gesture. All narrations were filmed from a frontal position. The first video-clip served as a warm-up, in which the experimenter gave feedback and asked questions to encourage participants to talk more vividly, if necessary. Throughout the narrations of the other nine videoclips, the examiner gave only confirmative comments like “yes” and “okay”. The examiner sat opposite to the patients and avoided to produce hand gestures during the examinations. 2.4. Data analysis 2.4.1. Analysis of video-taped hand movement behaviour For the purpose of this study, we analyzed hand movement behaviour during six narrations. Hand movement behaviour analysis was conducted with the NEUROGES – ELAN system, which consists of a behavioural coding system for hand movements and a media annotation tool (Lausberg and Sloetjes, 2009; www.lat-mpi.eu/tools/tools/elan). Among the existent coding systems for hand movements and gestures, the NEUROGES system has been chosen as it is objective and reliable as evidenced by a recent survey of the interrater agreement in 18 empirical studies that employed the NEUROGES system (Lausberg and Sloetjes, 2015). Most importantly for the present study, in which PWA were investigated, the NEUROGES system allows for an assessment of hand movements and gestures independent of speech. The NEUROGES system comprises three modules that build up on each other. Module I analyzes the ongoing stream of speech180

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Fig. 1. NEUROGES Module I coding algorithm; the Focus values on attached object, on separate object, and on person were not relevant for this study as they did not occur in the narrations (adpoted from Lausberg (2013)).

resting on the armrests. Finally, aborted units are movements that are disrupted during the transport phase. Fig. 2 shows examples for different forms of phasic hand movements (first row), repetitive hand movements (second row), and irregular hand movements (third row). In the third assessment step, the phasic, repetitive, and irregular units are further classified with the Focus category. The Focus category classifies phasic, repetitive, and irregular movements according to the location where the hand acts (on). In the 2009 version of the NEUROGES system (Lausberg and Sloetjes, 2009) three Focus values were distinguished: within body (acting on body-internal structures, e.g. rolling the shoulders to relax muscles), on body/on attached object (acting on the body surface or on an object that is attached to the body, e.g. scratching the skin), and in space (acting in space without touching something, e.g. indicating a direction by pointing in space; this value matches the functional definition of gestures). From the behavioural functional perspective (e.g. Efron, 1941), in space movements are gestures. On body movements include self-touch but also gestures and actions in which the gesturers touches her-/himself. On attached object movements are functional or pseudo-functional manipulation of objects that are attached to the gesturer's body, e.g. adjusting the tie or playing with the bracelet. Within body movements include all functional and pseudo-functional movements that focus on inner-body structures, e.g. that serve to stimulate muscles, tendons, and joints, such as rolling the foot, but also stirs, and involuntary movements such as tics. According to the premotor theory of attention the preparation of a movement toward an object/subject of interest necessarily implies a shift in spatial attention, as attention derives from activity of sensorimotor circuits (Rizzolatti et al., 1994; Rizzolatti and Craighero, 2010). Thus, any hand movement implies a (partial or

plan or to execute a concept, respectively, and then retracted again to rest position. Thus, the production of phasic hand movements is based on conceptual processes. From the behavioural functional perspective, phasic movements are gestures, actions, discrete (self-) touches, etc. A variant is repetitive movements, in which the complex phase is characterized by a repetitive execution of the trajectory. Just as phasic movements, repetitive hand movements rely on conceptual processes but, as motor control research evidences, during the repetition proper, routine processes are prominent. Functionally, repetitive movements are gestures, actions, or discrete (self-) touches with a persistent component, such as a series of baton gestures emphasizing prosody, hammering, or moving the hand to an itching part of the body and scratching. Both phasic and repetitive movements, which are based on conceptual processes, differ from irregular movements that lack a structured trajectory. The hand remains at a place and displays small movements, such as in fidgeting. Irregular movement activity seems to be going by itself and can be potentially continuous, e.g. ongoing during an interview being only interrupted by conceptual movements or by shifts. Functionally, irregular movements merely provide sensorimotor stimulation and there is ample empirical evidence that they serve arousal-regulation (see review in Lausberg (2013)). Phasic, repetitive, and irregular movements can be ordered according to conceptual complexity, ranging from phasic movement, which are based on conceptual processes, to repetitive movements, in which the repetitive phase proper can be performed automatically, to irregular movements, which are not based on conceptual processes. Shift movements are characterized by a one-dimensional trajectory from rest position to another. Functionally, shifts exclusively serve to adopt a new rest position, e.g moving the hands from resting in the lap to 181

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Fig. 2. Picture lexicon illustrating the NEUROGES StructureFocus values.

complete picture of our data, we have employed the following measures: Descriptive analyses were conducted on the basis of each participant's total number of hand movement for discrete hand movement units (phasic, repetitive, aborted, shift) across all six narrations. For non-discrete, irregular movements, however, each participant's total duration in seconds was used, since irregular hand movements are potentially ongoing in time and maybe displayed by some individuals continuously, only being interrupted by occasional gestures or actions. For the ANOVA statistics, absolute and relative measures were used: For the Structure values phasic, repetitive, aborted, and shift, the mean number of units across all narrations as well as the mean number of units per minute, separately for the right and left hands, were submitted to statistical analyses. For the Structure value irregular, which refers to hand movement behaviour that is potentially continuous in time (see above), the mean duration across the six narrations as well as the proportion of time per minute (both measured in number of seconds) in which this behaviour was displayed was submitted to analysis. Group comparisons were conducted with mixed linear models (ANOVA). Post-hoc tests were calculated for the following comparisons:

complete) shift of attention toward the location, where the hands acts (on). Accordingly, the three Focus values can be ordered – with regards to the gesturer – from internal to external orientation as follows: within body, on body / on attached object (hereafter simply: on body), in space. Fig. 2 shows examples for different forms of within body movements (first column), on body and on attached objects movements (second and third lines), and in space movements (fifth column). For an overview of the complete NEUROGES system, on separate object movements (fourth column) and on person movements (sixth column) are depicted as well – these two values, however, were not relevant in the present experimental design. For a full exploitation of the assessments, the analyses of the Structure and Focus categories were concatenated to StructureFocus values, resulting in six possible combinations of Structure and Focus values: phasic in space, phasic on body, repetitive in space, repetitive on body, irregular within body, irregular on body. To illustrate the StructureFocus values, Fig. 2 in Rows (Structure) and Columns (Focus) showing each possible combination of Structure and Focus in a hand movement. For each narration hand movement behaviour was assessed by two independent raters who had been intensively trained in the NEUROGES system (http://neuroges.neuroges-bast.info/certifiedraters). For an overview of guidelines we followed for the determination of interrater agreement compare Krippendorff (2004). Narrations of the first two films were used for the training of the raters. Data was divided up to two rater teams. Team 1 analyzed retellings of Mr. Bean stories and team 2 analyzed retellings of Tweety and Sylvester stories. The first rater coded 100% of the given data, the second rater coded 25% of each participant in order to establish interrater reliability.

• • • • • •

Control persons left hand – control persons right hand Control persons right hand – LBD left hand Control persons right hand – RBD right hand Control persons left hand – LBD left hand Control persons left hand – RBD right hand LBD left hand – RBD right hand

We did not perform comparisons involving the right hand of the LBD participants or the left hand of the RBD participants, since most of patients had a hemiparesis (compare Table 1).

2.4.2. Statistical analysis Different measures shed different lights on the hand movement and gesture data (Sassenberg and Helmich, 2013). To provide a most 182

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

3. Results

Table 3 EasyDiag scores for interrater agreement. Structure

Focus

EasyDiag Rater Team 1

EasyDiag Rater Team 2

Phasic

In space On body In space On body Within body On body

0.61 0.50 0.67 0.53 NA 0.71 0.51 0.60

0.46 0.35 0.57 0.40 0.12 0.44 0.34 0.30

Repetitive Irregular Shift Aborted

3.1. Descriptive data Table 4 shows the descriptive data separately for discrete hand movement units (phasic, repetitive, aborted, shift) and non-discrete, irregular movements. Since irregular hand movements are potentially ongoing in time and maybe displayed by some individuals continuously, only being interrupted by occasional gestures or actions, they are reported as total duration in seconds across all narrations. 3.2. ANOVA For each NEUROGES value, we calculated ANOVAs in order to compare the groups, the hand choice and the interactions of group and hand choice. The results are summarized in Table 5. Furthermore, post-hoc tests were calculated. Except for one, all of the LBD participants were either hemiparetic or displayed residual symptoms of a hemiparesis in the upper extremity contralateral to the lesion. Hence, we only present those comparisons involving the ipsilesional hands of the stroke participants and the right and left hands of the control persons. Initial analysis of the data included the number of months since the lesion event as an additional covariate. However, as in all cases a non-significant effect was found the covariate was dropped from further analyses. The post-hoc tests revealed the following results.

2.5. Interrater reliability Interrater Reliablity was established with EasyDiag (Holle and Rein, 2015). EasyDIAg is a modified version of Cohen's Kappa that not only takes into account the raters’ categorical agreement, e.g. if a unit is phasic or shift, but also their temporal agreement, e.g. when a unit begins and ends. Accordingly, the EasyDIAg scores are numerically lower than Cohen's Kappa scores (Skomroch, 2013). Table 3 shows the agreement scores for the eight StructureFocus values. The variation of the scores between the NEUROGES values as found in the present study is in line with a recent review on interrater reliability including 18 empirical studies that used the NEUROGES system (Lausberg and Sloetjes, 2015). The review demonstrated that the interrater agreement varied between the different NEUROGES values. Some values such as in space values were found to be easier to achieve agreement on than others such as within body. Among others, the frequency of occurrence of a value is one factor that influences the reliability of identification. Furthermore, the variability between the two rater teams in the present study ranges within the frame of reference in the review. The inter-team variability might be due to the fact that team 2 analyzed the narrations of the motion rich cartoonstimuli which might have been more difficult to code than the retellings of the more static Mr Bean-stories which were analyzed by team 1. However, while team 2 scored in general lower than team 1, both teams agreed on the fact that they scored best in the three values repetitive in space, phasic in space, and irregular on body.

3.3. Conceptual (phasic and repetitive) movements 3.3.1. Phasic in space Control participants produced with either hand a significantly higher mean number of phasic in space movements than RBD participants did with their right hands. LBD participants produced a significantly higher mean number of phasic in space movements with their left hand than RBD participants with their right hand (compare Fig. 3). Regarding the frequency of phasic in space movements (number per minute), control persons produced more of them with either hand than did RBD participants with their right hand. LBD participants produced more phasic in space movements than control persons with either hand and RBDs with their right hand (compare Fig. 4).

Table 4 Total mean number of discrete movement units including phasic, repetitive, aborted movements and shifts and total mean duration of irregular movements in seconds, respectively, in the three examination groups. CON (N =20)

LBD (N =20)

RBD (N =18)

Hand

Left

Right

Left

Right

Left

Right

Total mean number of discrete hand movements units Range Standard deviation Total mean duration of non-discrete, irregular hand movements in seconds Range Standard deviation

84.3

87.8

94.1

16.8

12.1

55.5

20–232 54.5

7–228 60.6

39–168 38.4

0–152 36.1

0–88 24.6

14–128 32.4

105.2

98.8

17.4

2.6

15.8

110.4

4.6–443 110.2

0–470 122.1

0–76.8 19.2

0–33.5 7.6

0–105.1 36.5

4.9–315 94.6

3.3.2. Phasic on body LBD participants produced a significantly higher mean number of phasic on body movements with their left hand than did control persons with either hand. Furthermore, LBD participants produced significantly more phasic on body movements per minute with their left hand than did control persons with either hand. RBD participants produced significantly more phasic on body movements per minute with their right hand than did control persons with either hand. 3.3.3. Repetitive in space LBD participants produced a significantly higher mean number of repetitive in space movements with their left hand than did RBD participants with their right hand. Regarding the frequency of repetitive in space movements, LBD participants produced significantly more of them with their left hand than RBD participants with their right hand and control persons with either hand (compare Fig. 5). 3.3.4. Repetitive on body LBD participants produced significantly more repetitive on body movements per minute than control persons with either hand. 183

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Table 5 Summary of the effects yielded by ANOVA. Effects yielded by ANOVA

Structure value phasic

Focus value

in space on body repetitive in space on body total conceptual (phasic + repetitive) irregular on body within body total irregular shift aborted

Total number of units (in case of irregular movements total number of seconds)

Frequency (units per minute; in case of irregular movements seconds per minute)

Group F(2,55) 11.4** 0.7 5.9** 3.2 8.5**

Hand F(1,338) 11.6** 4.72* 15.2** 0 14.1**

Group*Hand interaction F(2,338) 124.6** 67.7** 73.5** 56.5** 150**

Group F(2,55) 12.9** 1.1 6.7** 1.7 12.9**

Hand F(1,338) 29.6** 7* 29.8** 1.430**

Group*Hand interaction F(2,338) 185** 87.4** 99.5** 54.3** 185.1**

7.6** 5.9** 7.8** 10.6** 1.4

28.7** 0.5 28.7** 0.9 10**

74.3** 1.5 74.7** 10.7** 4.1*

7.7** 7.7** 8.1** 10** 0.3

36** 36** 37.2** 0.7 9.7**

89** 89.4** 91.4** 13.4** 10.7**

*p < 0.05, **p < 0.005.

Fig. 5. Mean frequency (number per minute) and standard deviation of repetitive in space hand movements in the three examination groups.

Fig. 3. Mean number and standard deviation of phasic in space hand movements in the three examination groups.

3.3.5. All conceptual hand movements We further conducted analyses in which all phasic and repetitive hand movements were summed up. Participants with LBD produced a significantly higher mean number of conceptual hand movements with their left hand than did RBD participants with their right hand. Furthermore, LBD participants produced significantly more conceptual hand movements per minute with their left hand than control persons with either hand and RBD participants with their right hand (compare Fig. 6). 3.3.5.1. Conceptual hand movements and aphasia in LBD participants. To test for a possible relationship of speech production and the production of conceptual hand movements within the group of PWA, we conducted a Pearson's correlation with the subtest Naming of the Aachen Aphasia Test and the total number of conceptual hand movements as well as the number of conceptual hand movements per minute. There were no significant relationships (for the total number of conceptual hand movements: r=0.33, p=0.15; for the number of conceptual hand movements per minute: r=−0.12, p=0.62). Fig. 4. Mean frequency (number per minute) and standard deviation of phasic in space hand movements in the three examination groups.

184

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Fig. 7. Mean number of seconds per minute and standard deviation of irregular hand movements in the three examination groups.

Fig. 6. Mean frequency (number per minute) and standard deviation of conceptual hand movements in the three examination groups.

3.4. Irregular movements

3.3.5.2. Conceptual hand movements and limb apraxia in LBD participants. To test for a possible relationship of limb apraxia and the production of conceptual hand movements within the persons with LBD, we conducted a Pearson's correlation with the scores of the pantomime-to-command task and the total number of conceptual hand movements as well as with the number of conceptual hand movements per minute. There were no significant relationships (for the total number of conceptual hand movements: r =0.35, p=0.13; for the number of conceptual hand movements per minute: r =0.17, p=0.48).

3.4.1. Irregular on body The post-hoc analysis of the mean duration across the narrations as well as per minute revealed that LBD participants spent significantly less time (in seconds) with irregular on body movements with their left hand than did RBD participants with their right hand or control persons with either hand.

3.4.2. Irregular within body Post-hoc analysis of the mean duration across the narrations as well as per minute revealed that LBD participants spent significantly less time (in seconds) with irregular within body with their left hand than did RBD participants with their right hand or control persons with either hand.

3.3.5.3. Conceptual hand movements and non-verbal semantic processing capacities in LBD participants. To test for a possible relationship of non-verbal semantic processing capacities and the production of conceptual hand movements within the persons with LBD, we conducted a Pearson's correlation with the error scores of the semantic processing task and the total number of conceptual hand movements as well as with number of conceptual hand movements per minute. There were no significant relationships (for the total number of conceptual hand movements: r =−0.38, p=0.10; for the number of conceptual hand movements per minute: r =−0.42, p=0.07).

3.4.3. All irregular hand movements We conducted an additional analysis of all irregular hand movements taken together: Post-hoc analysis revealed that over the time of the narrations as well as per minute LBD participants spent significantly less time with irregular hand movements with their left hand than did RBD participants with their right hand or control persons with either hand (Fig. 7).

3.3.5.4. Conceptual hand movements and comprehensibility of gestures in LBD participants. All LBD participants were included in a previous study (Hogrefe et al., 2012) in which the comprehensibility of gestures was determined. We tested for a possible relationship of number and frequency of conceptual hand movements and the comprehensibility of gestures (Identification Rate). There were no significant relationships (for the total number of conceptual hand movements: r =0.23, p=0.35; for the number of conceptual hand movements per minute: r =0.17, p=0.46). An additional analysis was conducted with a more narrow focus on functional gestures only: we conducted a Pearson's correlation with all in space movements (phasic in space and repetitive in space) and the comprehensibility of gestures. There were no significant relationships (for the total number of phasic and repetitive movements in space, i.e. gestures: r =0.28, p=0.23; for the number of gestures per minute: r =0.168, p=0.48).

3.5. Aborted movements Post-hoc comparisons showed no significant differences concerning the mean number and the frequency of aborted units between the three examination groups.

3.6. Shift movements Post-hoc comparisons showed that control persons produced a significant higher mean number of shifts with either hand than did LBD participants with their left hand. Furthermore, control persons produced significantly more shifts per minute with their left hand than did LBD participants. 185

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

related to the production of conceptual hand movements. The latter finding is in line with previous studies that showed that apraxia did not per se hinder gesture production, i.e. did not negatively impact on the frequency of gesture use (experiment 1 in Feyereisen et al. (1988), Lausberg et al. (2000, 2007) and Rose and Douglas (2003)). Our findings did not reveal a specific relationship between typical left hemispheric functions and the production of conceptual hand movements. However, other studies investigating the impact of neuropsychological disorders that have been attributed to left hemisphere processing showed that the different disorders influence different aspects of gesture production. Non-verbal semantic processing disorders lead to an overall reduction of gestural diversity with respect to movement execution (e.g. hand shapes, locations of gesture execution; Hogrefe et al., 2012). Furthermore, some studies showed a negative effect of limb apraxia on the comprehensibility of gestures (Borod et al., 1989; experiment 2 in Feyereisen et al. (1988), Hogrefe et al. (2012) and Mol et al. (2013)). The observation that in the present study the frequency of conceptual hand movements as well as the frequency of functional gestures does not show a relationship to the comprehensibility of the gestures (as has been determined in Hogrefe et al. (2012)) is surprising. A possible explanation could be that the frequency of functional gestures reflects the potential richness of gestural communication, but that this potential richness is not related to an optimal use of gestures. Hence, a gesture might be understandable and interpretable but not suited to reflect aspects of the story in question. Accordingly, LBD participants tried to convey contents of the stories via an enhanced use of gestures as compared to the other groups but they did not always succeed to choose those gestures that fit best into the context of the stories. LBD participants spent significantly less time with irregular movements and showed a lower frequency of shifts than participants of the two other groups. This finding might relate to the enhanced use of gestures leaving less opportunity for self-touching behaviour. It is plausible that in the context of the present study design requiring a renarration, the LBD participants shifted to an explicit use of the left hand for generating conceptual hand movements. The explicit volitional use of the left hand suppresses its implicit unintentional use for displaying irregular movements.

4. Discussion In the discussion we refer to the frequencies of hand movements as measured in numbers per minute and proportion of time as measured in seconds per minute as the narrations differed considerably with respect to their lengths. 4.1. Summary of main results The main findings can be summarized as follows: 1. For all phasic and repetitive movements, LBD participants showed significantly greater frequencies with their left hands compared to the other two groups, i.e. than the RBD group with their right hands and the control group with either hand. This observation also holds for the superordinate value of conceptual movements which comprise phasic and repetitive movements. For phasic movements on body the LBD group reached with the left hand a higher frequency than the control group with either hand. For all irregular movement values, LBD participants showed significant lower proportions of time than participants of the other two groups. Furthermore, LBD participants displayed lower frequencies for shifts compared to the control persons' left hands. 2. RBD participants produced fewer phasic hand movements in space compared to the other two groups. Furthermore, they produced more phasic movements on body with their right hand than the control participants with either hand. Persons with RBD produced a similar amount of irregular hand movements as healthy control persons. 3. The healthy control participants displayed each of the hand movement values equally frequent with the left and right hands. 4.2. Hand movement behaviour in LBD participants LBD participants obtained for all phasic and repetitive movements greater frequencies than the other two groups with the exception of phasic on body movements for which the RBD group reached similar frequencies. For all values of irregular movements, LBD participants obtained significant lower proportions of time than participants of the other two groups. Furthermore, they obtained a lower frequency of shifts as compared to the control persons’ left hands. The findings of the Structure category showed that LBD participants produce more conceptual movements than persons without brain damage and persons with right hemisphere damage. In particular, as has been determined through the results from the StructureFocus category, LBD participants produced more phasic and repetitive in space movements which were functionally gestures. Given that brain damage in these participants led in all cases to severe forms of aphasia, Kimura's claim that speech and “free movements”, which are equivalent to phasic and repetitive movements in space, originate from a common neural system (Kimura, 1973) as well as the often cited assumption by McNeill (1985) that gesture and speech break down in parallel in aphasia are not supported by these results. On the contrary, in the case of aphasia we find an enhanced production of conceptual hand movements including gestures. These findings support earlier studies that indicated a greater frequency of gestures in general or of specific gesture types in persons with aphasia as compared to healthy persons and persons with right hemisphere damage (Hadar et al., 1998; Herrmann et al., 1988; Le May et al., 1988; Sekine and Rose, 2013). These results might be interpreted such that LBD participants possibly tried to communicate information about the scenes using gestures. Within the LBD group, statistical analysis did not reveal a relationship of conceptual hand movements to language production capacities as tested by the subtest Naming of the Aachen Aphasia Test. However, a reduced variability with respect to the language disorder has to be assumed as all PWA displayed very severe types of aphasia. Furthermore, non-verbal semantic processing capacities as well as apraxia as diagnosed with a pantomime-to-command task were not

4.3. Hand movement behaviour in RBD participants RBD participants produced phasic in space hand movements at a lower frequency than the participants of the other two groups. This finding is in line with our hypothesis. RBD participants produced more phasic on body movements per minute than the control participants. Furthermore, – contrary to our expectations – we could not observe a greater proportion of time spent using irregular movements of the right hand in RBD participants. Hence, the finding that persons with RBD produce fewer phasic in space movements, which are functionally gestures, supports previous findings showing a reduced number of hand gestures with conceptual content in these participants (Cocks et al., 2007; Hadar et al., 1998). This suggests that the right hemisphere substantially contributes to gesture production. This assumption is in line with current evidence showing laterality preferences for a spontaneous left hand choice for the production of certain gesture types as well as for the production of conceptual gestures under specifically controlled conditions (for an overview compare Lausberg (2013), p. 29ff.). For example, results from a study with three individuals with complete callosal disconnection suggest that gestures that involve spatial imagery may be generated by the right hemisphere (Lausberg et al., 2007; Kita and Lausberg, 2008). Furthermore, the reduced output of conceptual phasic gestures parallels the observation that – as compared to healthy control persons – the content of spoken narrations in RBD participants is reduced. Previous research on discourse abilities of persons with damage to the right hemisphere showed that the discourse of these persons contains 186

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

nicative gestures, i.e. hand movements in space, in the LBD group, which comprised persons with severe aphasia, indicates that gestures may draw on different resources in the right and left hemispheres than language. According to our data, apraxia as well as non-verbal semantic processing did not influence the production of conceptual hand movements. However, in this study we focused on hand movement frequency. Results of previous studies showed that limb apraxia impacts on the comprehensibility of gestures and that non-verbal semantic disorders influence the diversity of gesture use (experiment 2, Feyereisen et al., 1988; Hogrefe et al., 2012) and this demonstrates that the left hemisphere is involved in the production of gestures. The finding that neither the frequency of conceptual hand movements nor the frequency of gestures shows a relationship to the comprehensibility of the gestures (as has been determined in Hogrefe et al. (2012)) suggests that in LBD potential gestural richness is not related to an optimal use of gestures. Future studies should investigate the relationship of different accompanying neuropsychological disorders and different aspects of gesture production in PWA in more detail. On the other hand, our results show clearly that RBD leads to a decrease in the production of gestures (i.e. phasic in space). Hence, the intact right hemisphere contributes substantially to the production of gestures. We hypothesize that a reduction of the use of gestures in persons with RBD might be related to pragmatic or executive dysfunctions in that they prevent the generation of novel conceptualizations. Future studies should analyse this relationship.

less information, fewer concepts and more irrelevant remarks (e.g. Bloom et al., 1993; Joanette et al., 1986). As the smaller amount of information was not related to a smaller amount of total verbal output, in RBD participants conceptual information in verbal narrations seems to be generally reduced. Further, as previous research and our present data indicate, this reduction of conceptual content is a multimodal phenomenon. The fact that RBD participants produced nonetheless an equal amount of repetitive in space hand movements – compared to either hand of the control participants – does not contradict this notion. In NEUROGES, the Structure values irregular, repetitive, and phasic are proposed to rely on processes with different levels of cognitive complexity ranging from non-conceptual via automatic conceptual to novel conceptual. In the RBD group presented here, only the hand movements with the Structure value phasic and the Focus value in space, which are assumed to represent a high cognitive complexity involving the generation of novel conceptualizations, show a reduced occurrence. The pragmatic deficit that is frequently present in persons with RBD appears often in combination with disorders in executive functioning (Champagne-Laveau and Joanette, 2009). For persons with disorders in executive functioning the generation of novel conceptualizations might cause problems. This may lead to a reduced production of conceptual gestures that are created spontaneously during discourse. The fact that RBD participants produced a greater amount of phasic on body movements compared to controls with either hand reflects RBD participants’ self-touching behaviour as phasic on body movements are functionally self-touch movements with the exception of some few movements that included touching of the body, e.g. a selfdeictic gesture in which the index touches the sternum. In fact, “selftouch” movements, which are conducted on the body such as stroking hair, scratching, or wiping nose, face, eyes, have been shown to be increased in RBD participants in earlier studies (Blonder et al., 1995; Cocks et al., 2007). Since RBD is often associated with a decrease in arousal, the high frequency of phasic on body movements might indicate an implicit self-regulatory strategy to increase arousal. Another explanation that was brought up by Blonder et al. (1995) could be that in humans the right hemisphere evolved mechanisms to inhibit the production of self-touching behaviour. In RBD these mechanisms may be disturbed.

Acknowledgements This work was funded by the Volkswagen Foundation grant II/ 82175 to the last author and the German Research Foundation DFG grant GO 968/3-3. Reha-Hilfe e.V. supported the work with financial aid for technical devices. We thank Henning Holle for his efforts in developing the method EasyDIag for the determination of the interrater-reliability. Irina Kreyenbrink is thanked for developing the NEUROGES picture lexicon. Furthermore, we are grateful for the valuable comments of two anonymous reviewers. References Ahlsen, E., 1991. Body communication as compensation for speech in a Wernicke's aphasic–a longitudinal study. J. Commun. Disord. 24, 1–12. Behrmann, M., Penn, C., 1984. Non-verbal communication of aphasic patients. Br. J. Disord. Commun. 19, 155–168. Beland, R., Ska, B., 1992. Interaction between verbal and gestural language in progressive aphasia: a longitudinal case study. Brain Lang. 43, 355–385. Blonder, L.X., Burns, A.F., Bowers, D., Moore, R.W., Heilman, K.M., 1995. Spontaneous gestures following right hemisphere infarct. Neuropsychologia 33, 203–213. Bloom, R., Borod, J., Obler, L., Gerstman, L., 1993. Suppression and facilitation of pragmatic performance: effects of emotional content on discourse following right and left brain damage. J. Speech Hear. Res. 36, 1227–1235. Borod, J.C., Fitzpatrick, P.M., Helm-Estabrooks, N., Goodglass, H., 1989. The relationship between limb apraxia and the spontaneous use of communicative gesture in aphasia. Brain Cogn. 10, 121–131. Brownell, H., Gardner, H., Prather, P., Martino, G., 1995. Language, communication, and the right hemisphere. In: Kirshner, H.S. (Ed.), Handbook of Neurological Speech and Language Disorders. Marcel Dekker, New York, 325–349. Butterworth, B., Hadar, U., 1989. Gesture, speech, and computational stages: a reply to McNeill. Psychol. Rev. 96, 168–174. Champagne-Laveau, M., Joanette, Y., 2009. Pragmatics, theory of mind and executive functions after a right-hemisphere lesion: different patterns of deficits. J. Neurolinguist. 22, 413–426. Cicone, M., Wapner, W., Foldi, N., Zurif, E., Gardner, H., 1979. The relation between gesture and language in aphasic communication. Brain Lang. 8, 324–349. Cocks, N., Hird, K., Kirsner, K., 2007. The relationship between right hemisphere damage and gesture in spontaneous discourse. Aphasiology 21, 299–319. Côté, H., Payer, M., Giroux, F., Joanette, Y., 2007. Towards a description of clinical communication impairment profiles following right-hemisphere damage. Aphasiology 21, 739–749. De Beer, C., Carragher, M., van Nispen, K., de Ruiter, J., Hogrefe, K., Rose, M., 2016. How much information do people with aphasia convey via gesture? Am. J SpeechLanguage Pathol., (in press). De Ruiter, J.-P., 2000. The production of gesture and speech. In: McNeill, D. (Ed.), Language and Gesture. Cambridge University Press, Cambridge, UK, 284–311. Efron, D., 1941. Gesture and Environment. King's Crown Press, New York.

4.4. Hand movement behaviour in control participants The frequency of movements of right vs. the left hand does not differ from each other for none of the comparisons conducted for the control participants. These results are in line with several previous studies (for a review compare Lausberg (2013)). These findings suggest that the influential account by Kimura (1973), who claimed that the production of “free movements”, which are equivalent to in space movements according to NEUROGES, is exclusively tied to hand preference and linguistic processes might not be tenable any more. 5. Conclusion The findings of the present study indicate that the generation of conceptual hand movements (including gestures) and language production do not rely on the same neurological substrates. Our findings as well as the outcomes of previous studies lend empirical evidence for the assumption that communicative gestures may be generated independently of linguistic processes. However, we do not doubt that in general, in individuals with normal neural and mental functions gesture and speech rely on a common communicative intention and that gesture is closely coordinated with language throughout the production process. It seems to be clear from the literature that hand movement behaviour is influenced by language and communication, i.e. interactional processes. However, the increased use of commu187

Neuropsychologia 93 (2016) 176–188

K. Hogrefe et al.

Lausberg, H., Kita, S., 2003. The content of the message influences the hand choice in cospeech gestures and in gesturing without speaking. Brain Lang. 86, 57–69. Lausberg, H., Sloetjes, H., 2009. Coding gestural behavior with the NEUROGES–ELAN system. Behav. Res. Methods 41, 841–849. Lausberg, H., Sloetjes, H., 2015. The revised NEUROGES-ELAN system: an objective and reliable interdisciplinary analysis tool for nonverbal behavior and gesture. Behav. Res. Methods. http://dx.doi.org/10.3758/s13428-015-0622-z. Lausberg, H., Davis, M., Rothenhausler, A., 2000. Hemispheric specialization in spontaneous gesticulation in a patient with callosal disconnection. Neuropsychologia 38, 1654–1663. Lausberg, H., Zaidel, E., Cruz, R.F., Ptito, A., 2007. Speech-independent production of communicative gestures: evidence from patients with complete callosal disconnection. Neuropsychologia 45, 3092–3104. Lavergne, J., Kimura, D., 1987. Hand movement asymmetry during speech: no effect of speaking topic. Neuropsychologia 25, 689–693. Le May, A., David, R., Thomas, P.A., 1988. The use of spontaneous gesture by aphasic patients. Aphasiology (2), 137–145. McNeill, D., 1985. So you think gestures are nonverbal? Psychol. Rev. 92, 350–371. McNeill, D., 1992. Hand and Mind. What Gestures Reveal about Thoughts. The University of Chicago Press, Chicago, London. McNeill, D., 2005. Gesture and Thought. The University of Chicago Press, Chicago. McNeill, D., Pedelty, L.L., 1995. Right brain and gesture. In: Emmorey, K., Reilly, J. (Eds.), Language, Gesture, and Space. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, 63–85. Mol, L., Krahmer, E., van de Sandt-Koenderman, M., 2013. Gesturing by speakers with aphasia: How does it compare? J. Speech Lang. Hear. Res. 56, 1224–1236. Oldfield, R.C., 1971. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113. Rizzolatti, G., Craighero, L., 2010. Premotor theory of attention. Scholarpedia 5 (1), 6311 . Rizzolatti, G., Riggio, L., Sheliga, B.M., 1994. Space and selective attention. In: Attention & Performance XV. MIT Press, Cambridge, MA, 231–265. Rose, M., 2006. The utility of arm and hand gestures in the treatment of aphasia. Adv. Speech-Lang. Pathol. 8, 92–109. Rose, M., Douglas, J., 2003. Limb apraxia, pantomime, and lexical gesture in aphasic speakers: preliminary findings. Aphasiology 17, 453–464. Ross, E.D., 1981. The aprosodias. Functional-anatomic organization of the affective components of language in the right hemisphere. Arch. Neurol. 38, 561–569. Ross, E.D., 1996. Hemispheric specialization for emotions, affective aspects of language and communication and the cognitive control of display behaviors in humans. Prog. Brain Res. 107, 583–594. Ross, E.D., Mesulam, M.M., 1979. Dominant language functions of the right hemisphere? Prosody and emotional gesturing. Arch. Neurol. 36, 144–148. Rousseaux, M., Daveluy, W., Kozlowski, O., 2010. Communication in conversation in stroke patients. J. Neurol. 257, 1099–1107. Salmaso, D., Longoni, A.M., 1985. Problems in the assessment of hand preference. Cortex 21, 533–549. Sassenberg, U., Helmich, I., 2013. Statistical evaluation and data presentation. In: Lausberg, H. (Ed.), Understanding Body Movement: A Guide to Empirical Research on Nonverbal Behaviour: With an Introduction to the NEUROGES Coding System. Peter Lang, Frankfurt/Main, 289–322. Sekine, K., Rose, M.L., 2013. The relationship of aphasia type and gesture production in people with aphasia. Am. J. Speech Lang. Pathol. 22, 662–672. Skomroch, H., 2013. Recommendations for assessing the level of interrater agreement. In: Lausberg, H. (Ed.), Understanding Body Movement: A Guide to Empirical Research on Nonverbal Behaviour: With an Introduction to the NEUROGES Coding System. Peter Lang, Frankfurt/Main, 279–286. Tarhan, L.Y., Watson, C.E., Buxbaum, L.J., 2015. Shared and distinct neuroanatomic regions critical for tool-related action production and recognition: evidence from 131 left-hemisphere stroke patients. J. Cogn. Neurosci. 27, 2491–2511.

Ekman, P., Friesen, W.V., 1969. The repertoire of nonverbal behaviour: categories, origins, usage and coding. Semiotica 1, 49–99. Feyereisen, P., 1987. Gestures and speech, interactions and separations: a reply to McNeill (1985). Psychol. Rev. 94, 493–498. Feyereisen, P., Barter, D., Goossens, M., Clerebaut, N., 1988. Gestures and speech in referential communication by aphasic subjects: channel use and efficiency. Aphasiology 2, 21–32. Finkelnburg, F.C., 1870. Sitzung der Niederrheinischen Gesellschaft in Bonn. Medizinische Section. Berl. Klin. Wochenschr. 7, 449–450, 460–462. Glindemann, R., Klintwort, D., Ziegler, W., Goldenberg, G., 2002. Bogenhausener Semantik-Untersuchung (BOSU). Urban & Fischer, München. Glosser, G., Wiener, M., Kaplan, E., 1986. Communicative gestures in aphasia. Brain Lang. 27, 345–359. Goldenberg, G., Randerath, J., 2015. Shared neural substrates of apraxia and aphasia. Neuropsychologia 75, 40–49. Goldenberg, G., Hartmann, K., Schlott, I., 2003. Defective pantomime of object use in left brain damage: apraxia or asymbolia? Neuropsychologia 41, 1565–1573. Goldenberg, G., Hermsdörfer, J., Glindemann, R., Rorden, C., Karnath, H.O., 2007. Pantomime of tool use depends on integrity of left inferior frontal cortex. Cereb. Cortex 17, 2769–2776. Hadar, U., Burstein, A., Krauss, R., Soroker, N., 1998. Ideational gestures and speech in brain-damaged subjects. Lang. Cogn. Process. 13, 59–76. Hermsdörfer, J., Mai, N., Rudroff, G., Münßinger, M., 1994. Untersuchung zerebraler Handfunktionsstörungen. Ein Vorschlag zur standardisierten Durchführung Vol. 6. Borgmann, Dortmund. Herrmann, M., Reichle, T., Lucius-Hoene, G., Wallesch, C.W., Johannsen-Horbach, H., 1988. Nonverbal communication as a compensative strategy for severely nonfluent aphasics? A quantitative approach. Brain Lang. 33, 41–54. Hogrefe, K., Ziegler, W., Tillmann, C., Goldenberg, G., 2011. Intonation and hand gestures in narrations of healthy speakers and speakers suffering from right hemisphere damage: a pilot study. In: Proceedings of the second conference Gesture and Speech in Interaction (GESPIN), Bielefeld, September 2011. Hogrefe, K., Ziegler, W., Weidinger, N., Goldenberg, G., 2012. Non-verbal communication in severe aphasia: influence of aphasia, apraxia, or semantic processing? Cortex 48, 952–962. Hogrefe, K., Ziegler, W., Wiesmayer, S., Weidinger, N., Goldenberg, G., 2013. The actual and potential use of gestures for communication in aphasia. Aphasiology 27, 1070–1089. Holle, H., Rein, R., 2015. EasyDIAg: a tool for easy determination of interrater agreement. Behav. Res Methods 47, 837–847. Howard, D., Patterson, K., 1992. The Pyramids and Palmtree Test. Thames Valley Test Company, Bury St Edmunds. Huber, W., Poeck, K., Weniger, D., Willmes, K., 1983. Aachener Aphasie Test (AAT). Göttingen, Hogrefe. Joanette, Y., Goulet, P., Ska, B., Nespoulous, J.L., 1986. Informative content of narrative discourse in right-brain-damaged right-handers. Brain Lang. 29, 81–105. Kimura, D., 1973. Manual activity during speaking – I. Right-handers. Neuropsychologia 11, 45–50. Kita, S., Özyürek, A., 2003. What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. J. Mem. Lang. 48, 16–32. Kita, S., Lausberg, H., 2008. Generation of co-speech gestures based on spatial imagery from the right-hemisphere: evidence from split-brain patients. Cortex 44, 131–139. Krauss, R.M., Chen, Y., Gottesmann, R.F., 2000. Lexical gestures and lexical access: a process model. In: McNeill, D. (Ed.), Language and Gesture. Cambridge University Press, Cambridge, UK, 261–283. Krippendorff, K., 2004. Content Analysis: An Introduction to Its Methodology. Sage, Newbury Park, CA. Lausberg, H. (2013). Understanding Body Movement and Gesture. A Guide to Empirical Research on Nonverbal Behaviour. Peter Lang, Frankfurt/Main

188