A further evaluation of The Aggression Questionnaire: Issues of validity and reliability

Pergamon

Behav. Res. Ther. Vol. 35, No. 1I, pp. 1047--1053, 1997 © 1997 ElsevierScienceLtd. All rights reserved Printed in Great Britain PII: S0005-7967(97)00064-8 0005-7967/97 $17.00 + 0.00

A FURTHER EVALUATION OF THE AGGRESSION QUESTIONNAIRE: ISSUES OF VALIDITY AND RELIABILITY JULIE AITKEN HARRIS Department of Psychology, Social Science Centre, The University of Western Ontario, London, Ont. N6A 5C2, Canada

(Received 17 April 1997; revised 17 June 1997)

Summary--An analysis of the four scales from Buss and Perry's (1992, Journal of Personality and Social Psychology, 63, 452459) Aggression Questionnaire was conducted. Examined was the internal consistency, test-retest reliability, the influence of social desirability, and the interrelationships with other measures of aggression. The results suggest that the four scales of The Aggression Questionnaire have moderate to high internal consistencies and are stable over seven months of testing. Social desirability was found to have a moderately high negative relationship with the aggression scales suggesting that social desirability may influence responses provided on The Aggression Questionnaire. In addition, the four aggression scales were found to be positively related to other measures of aggression including scales assessing affect instability and aggressive attitudes, as well as scales designed for clinical use, suggesting some degree of construct validity. © 1997 Elsevier Science Ltd

INTRODUCTION In 1992, Buss and Perry published an updated and psychometrically improved measure of aggression, titled The Aggression Questionnaire, which stemmed from the popular Buss-Durkee Hostility Inventory (Buss & Durkee, 1957). The Aggression Questionnaire contains four selfreport measures of aggression, including physical aggression, verbal aggression, anger, and hostility. Since the publication of The Aggression Questionnaire, the measure has been used in a variety of settings with each study, to some degree, furthering the construct validity of the aggression scales by examining how the measures are related to various behaviors and to other measures of personality. J. A. Harris, Rushton, Hampson, and Jackson (1996) examined the correlations between salivary testosterone and aggressive and pro-social personality characteristics in men and women, and found that the four scales of The Aggression Questionnaire along with two other aggression measures [one of which was non-verbal, the aggression scale from the Nonverbal Personality Questionnaire (Paunonen & Jackson, 1988)] loaded onto an aggression factor. This aggression factor was found to have a positive relationship with salivary testosterone and a negative relationship with a pro-social personality factor which included two measures of nurturance, an empathy scale, and an altruism scale. These results suggest that The Aggression Questionnaire is both related to other measures of aggression and that the dimensions of aggression assessed may be related to the gonadal sex hormone testosterone. M. B. Harris (1996) examined the relationship between The Aggression Questionnaire and self-report frequency of engaging in and being the target of aggressive acts in Hispanic and Anglo men and women, and found a positive relationship (with correlations ranging from 0.26 to 0.49) between the aggression scales and having engaged in a variety of aggressive acts, such as throwing an object at another individual or slapping someone. The positive correlations between aggression and having engaged in aggressive acts were found when participants were asked the frequency of their aggressive behaviors for the past month and if they had ever engaged in the aggressive behaviors. The four aggression scales were also found to be positively related to subjects' reports of being the target of aggression, with correlations ranging from 0.11 to 0.33. 1047

1048

J.A. Harris

Russell and Arms (1995) found that reports of enjoying watching and possibly partaking in physical fights was positively related to the physical aggression and anger subscales of The Aggression Questionnaire (only two scales were included in this study) based on interviews of men at an ice hockey game. Similarly, Archer, Holloway, and McLoughlin (1995) examined the relationship between The Aggression Questionnaire and fighting behavior in male students and unemployed men. Archer et al. found positive correlations between fighting behavior and the four aggression scales, with correlations ranging from 0.14 with the hostility scale, to 0.44 with the physical aggression scale. Also found was a negative relationship between the aggression scales and recency in which individuals had fought (correlations ranged from -0.19 to -0.44), suggesting that higher scores on the aggression scales were related to having had a fight more recently (less amount of time lapsed between test time and last fight). In addition, Archer et al. reported a positive relationship (with correlations ranging from 0.39 to 0.63) between the aggression scales and a 'toughness' measure which assessed the subjects' beliefs that men should be tough. In addition to investigations of the relationships between The Aggression Questionnaire and aggressive behaviors such as fighting, some studies have focused more on the psychometric properties of the scales. Archer, Kilpatrick, and Bramwell (1995) examined the intercorrelations between The Aggression Questionnaire scales and Gladue's (1991a,b) Aggression Inventory in British men and women. Archer et al. found that the British sample was similar to the American samples used to standardize the two aggression measures. The four scales from The Aggression Questionnaire were found to be positively related to the physical aggression scale from the Aggression Inventory, with correlations ranging from 0.18 to 0.64. The physical aggression, anger, and verbal aggression scales from The Aggression Questionnaire were negatively related to the avoidance scale from the Aggression Inventory, with correlations from -0.12 to -0.29 (hostility from The Aggression Questionnaire was positively correlated (r = 0.10), with the avoidance scale from the Aggression Inventory). In addition, the four scales from The Aggression Questionnaire were positively related to the Aggression Inventory subscales of verbal aggression and impulsiveness (correlations ranged from 0.30 to 0.51 and from 0.44 to 0.62, respectively). In creating The Aggression Questionnaire, Buss and Perry (1992) found that the aggression scales were positively related to measures of impulsiveness and competitiveness, as well as to other measures of aggression. In addition, Buss and Perry reported that the four aggression scales fit a four factor model using confirmatory factor analyses. J. A. Harris (1995) attempted to replicate the four factor structure of The Aggression Questionnaire and found that, in general, the four scales fit a four factor model, but further suggested that the fit of the model would be improved if two hostility items were removed since these item appear to assess suspicion and not hostility. Meesters, Muris, Bosma, Schouten, and Beuving (1996) also conducted a confirmatory factor analysis on The Aggression Questionnaire with a Dutch sample. Meesters et al. also found that the four factor model could be improved by removing the two hostility items suggested by J. A. Harris (1995) and also reported that the model fit could be further improved by removing one item from the verbal aggression scale. The issue of whether to omit the two items discussed by J. A. Harris (1995) and Meesters et al. (1996) has been raised by Bernstein and Gesn (1997) who have demonstrated that these two items have different (higher) mean scores than do the remaining hostility items. Bernstein and Gesn report that because these two items have higher means, the items add information to the scale by increasing the range of the construct assessed (in this case hostility). One issue that might be addressed in future research is whether the means for these two (suspiciousness) items is different because of content. If the items do relate to suspiciousness rather than hostility, then including the items would detract from the assessment of the hostility construct. A possible solution (as an alternative to omitting the items) may be to create a suspiciousness factor for inclusion in The Aggression Questionnaire. In general, the above results suggest that The Aggression Questionnaire is related to other self-report measures of aggression as well as behavioral indicators of aggressive behaviors, suggesting that the scale has some degree of construct validity. The purpose of the present study is to further explore some of the properties of The Aggression Questionnaire. In particular, re-

The aggressionquestionnaire

1049

liability (test-retest and internal consistency), the influence of the response style social desirability, and interrelations with other measures of aggression were examined in a group of women.

METHOD Participants and procedure

A group of 106 first year undergraduate university women (Mage = 20.3, SD = 4.9, range 1748 years) were administered self-report aggressive measures (see below) and a social desirability scale (administered only once) three times over the course of seven months (the first test session was in the first two weeks of September, the second in the first two weeks of January, and third in the last two weeks of March). Participants were tested in groups of approximately 15 to 20 women. The order of the measures given to the participants was varied randomly for the three test sessions and were presented to the women in a single test battery. Between the first test session and the second test session, 21 cases were lost due to attrition, and a further 15 cases were lost between the second and third test session. Measures

Subjects were administered the four subscales from The Aggression Questionnaire (Buss & Perry, 1992). The first scale, the physical aggression scale, consists of nine items, such as, 'I have threatened people I know'. The second scale is the verbal aggression scale, which has five items, for example, 'I often find myself disagreeing with people.' The third scale is the anger scale, consisting of seven items, such as, 'I flare up quickly but get over it quickly.' The fourth scale is the hostility scale, which has eight items, for example, 'I am sometimes eaten up with jealously.' Responses to the items from the four subscales were given using a five-point scale ranging from 1 extremely uncharacteristic of me to 5 extremely characteristic of me. In addition, eight subscales from the Personality Assessment Inventory (PAI; Morley, 1991) were completed by the participants. The PAl was designed specifically to provide clinical psychological information about individuals and it was thought that the relations between these clinically based scales and the personality based scales of The Aggression Questionnaire would be of interest. The first of the PAI scales measures affective instability, a measure of anger control, based on six items, such as, 'My mood can shift quite suddenly'. The second PAI scale was the verbal aggression scale, which has six items, for example, 'I tell people off when they deserve it'. The third PAl scale tested was the physical aggression scale, consisting of six items, for example, 'People are afraid of my temper'. The fourth PAI scale was the antisocial attitude scale, which has eight items, for example, 'I've deliberately damaged someone's property'. The fifth PAI scale was the self-harm scale, consisting of six items, for example, 'I sometimes do things so impulsively that I get into trouble'. The sixth scale from the PAl was the aggressive attitude scale, consisting of six items, such as, 'I have a bad temper'. The seventh PAl scale was the negative relationships scale, based on six items, such as, 'My relationships have been stormy.' The eighth PAI scale was the egocentricity scale, consisting of eight items, such as, 'I have borrowed money knowing I wouldn't pay it back.' Responses to the items for the PAl scales are given using a four-point scale ranging from 0 False, not at alltrue to 3 very true. Also administered was the Lack of Frustration Scale (Olweus, 1986). The same response format as that of the PAI items was used for this scale. The Lack of Frustration Scale consists of three items, such as, 'I become easily impatient and irritable if I have to wait.' In addition four scales from Gladue's (1991a,b) Aggression Inventory (AI) were administered to the participants. The first scale was the verbal aggression scale, consisting of six items, such as, 'When a person is unfair to me, I get angry and protest'. The second scale was the impulsivity scale, consisting of four items, such as, 'I often act before I think'. The third AI scale was the avoidance scale, consisting of two items, such as, 'Whenever someone is being unpleasant, I think it is better to be quiet than to make a fuss'. The fourth AI scale measured was the impatience scale, consisting of three items, such as, 'Others say that I lose patience easily'. Responses to the Lack of Frustration Scale and the AI scale items was the same as for the PAI items since these items were intermixed with the PAl items.

1050

J . A . Harris

In addition, participants completed the social desirability scale from the Personality Research Form (PRF; Jackson, 1989). This scale consists of 16 true/false items which are evenly keyed. An example of a positively keyed social desirability item is, 'If someone gave me too much change, I would point it out'. RESULTS AND DISCUSSION Table 1 presents the mean and SD for each of the self-report aggression questionnaires. A comparison of the scale means for The Aggression Questionnaire found in the present study with those means reported by other researchers for female subjects suggests that the present sample of Canadian women were more similar to Archer et ars. (1995) British sample for physical aggression (M = 16.61), which are both slightly less than the mean of 17.9 reported by Buss and Perry (1992) for American women. Across the three studies, each reported verbal aggression means of approximately 13 (Buss and Perry reported a mean of 13.5 and Archer et al. reported a mean of 13.36). Hostility scores in the present sample resembled the American women reported by Buss and Perry (M = 20.2) which were higher than the British sample (M = 17.47; Archer et al., 1995). Anger scores in the present sample were both slightly lower than the American sample (M = 16.7, Buss & Perry, 1992) and were lower than the British sample ( M = 17.27; Archer et al., 1995). Also listed in Table 1 are two estimates of the reliability of each measure, the internal consistency (alpha) values as well as test-retest reliabilities. In general, most of the scales had moderately high (0.6 and above) alpha values, with the exception of the PAl (Morley, 1991) antisocial attitude scale, the PAl egocentric scale, and Gladue's (1991a,b) impulsivity scale. The internal consistency values found for The Aggression Questionnaire scales were similar to those reported by Buss and Perry (1992; alpha values of 0.85, 0.72, 0.77, and 0.83, for physical aggression, verbal aggression, hostility, and anger, respectively) and M. B. Harris (1996; alpha values of 0.82, 0.69, 0.81, and 0.80, for physical aggression, verbal aggression, hostility, and anger, respectively). Test-retest reliabilities for the scales were found to be moderately high to high (r = 0.47 to 0.88). The test-retest reliability estimates found for The Aggression Questionnaire scales across the seven months were found to be lower in the present study than the values reported by Buss and Perry (1992) for a nine week interval, although the values were still moderately high. In general, the correlations between the second and third test times were higher than the Table 1. Means, standard deviations, and internal and test-retest reliability for each scale Correlations No. of items

Time 1 with Time 3

Time 2 with Time 3

Mean

SD

Physical aggression a Verbal aggression ~ Hostility a Anger a

16.80 13.19 22.33 15.61

5.77 4.00 7.07 5.88

9 5 8 7

0.75 0.70 0.82 0.80

0.78** 0.73** 0.70** 0.77**

0.75** 0.81"* 0.67** 0.82**

0.86** 0.80** 0.83** 0.88**

Affect instability b Verbal aggression b Physical aggression b Antisocial attitude b Self-harm b Aggressive attitude b Negative relations b Egocentric b

7.12 6.89 2.22 6.41 4.28 6.19 6.61 4.69

3.96 3.53 2.41 3.06 2.95 3.86 3.60 2.69

6 6 6 8 6 6 6 8

0.78 0.73 0.62 0.40 0.66 0.72 0.67 0.47

0.77** 0.81"* 0.48* 0.73** 0.81"* 0.81"* -

0.76** 0.82** 0.70** 0.63** 0.77** 0.64** 0.74** 0.47*

0.88** 0.80** 0.54* 0.80** 0.81"* 0.80**

Lack of tolerance¢

3.43

2.37

3

0.82

0.84**

0.64**

0.80**

Verbal aggression a Impulsivity a Avoidance a Impatiencea

17.82 10.20 6.80 8.45

5.17 3.21 2.10 3.61

6 4 2 3

0.78 0.54 0.64 0.82

0.79** 0.67** 0.60** 0.76**

aFrom The Aggression Questionnaire (Buss & Perry, 1992). bFrom the PAI (Morley, 1991). CLack of Frustration Tolerance Scale from Olweus (1986). dFrom The Aggression Inventory (Gladue, 1991a,b). *P < 0.01; **P < 0.001 (two-tailed).

Alpha

Time 1 with Time 2

Scale

0.56* 0.46* 0.56* 0.50* 0.47* 0.54* 0.28* 0.18 0.45* 0.32 0.25 0.38* 0.39* 0.27 -0.26 0.28 -0.50* 0.09

1.

0.51" 0.53* 0.38* 0.67* 0.34* 0.22 0.16 0.41" 0.35* 0.33 0.43* 0.66* 0.25 -0.33* 0.36* 0.37* -0.11

0.77

2.

0.58* 0.59* 0.32* 0.36* 0.34* 0.30* 0.41" 0.59* 0.30 0.50* 0.18 0.27 0.11 0.29 -0.59* -0.08

0.59 0.67

3.

"From The Aggression Questionnaire (Buss & Perry, 1992). bFrom the PAl (Morley, 1991). CLack of Frustration Tolerance Scale from Olweus (1986). dFrom The Aggression Inventory (Gladue, 1991a,b). *P < 0.01 (two-tailed).

1. Physical aggression" 2. Verbal aggression" 3. Hostility a 4. Anger~ 5. Affect instability b 6. Verbal aggressionb 7. Physical aggressionb 8. Antisocial attitude b 9. Self-harmb 10. Aggressive attitude b I1. Negative relations b 12. Egocentricb 13. Lack of tolerancec 14. Verbal aggressiond 15. Impulsivity'l 16. Avoidanced 17. Impatienced 18. Social desirability 19. Age

Scale

0.74* 0.43* 0.54* 0.22 0.34* 0.76* 0.51" 0.36* 0.53* 0.43* 0.45* -0.04 0.59* -0.48* 0.13

0.72 0.71 0.72

4.

0.28* 0.54* 0.26 0.34* 0.67* 0.46* 0.18 0.51" 0.19 0.39* 0.09 0.47* -0.61" 0.09

0.65 0.51 0.74 0.94

5.

0.35* 0.30* 0.24 0.44* 0.28 0.27 0.33 0.64* 0.27 -0.44* 0.32 -0.28 0.14

0.64 0.94 0.41 0.56 0.37

6.

0.36* 0.24 0.67* 0.33 0.28 0.40* 0.40* 0.43 -0.03 0.34* -0.42* 0.06

0.79 0.52 0.50 0.77 0.78 0.52

7.

0.36* 0.36* 0.26 0.35* 0.16 0.19 0.29 -0.12 0.06 -0.35* -0.02

0.51 0.42 0.59 0.39 0.46 0.56 0.72

8.

0.38* 0.33 0.46* 0.29 0.14 0.19 -0.03 0.16 -0.22 0.01

0.26 0.24 0.41 0.47 0.47 0.34 0.38 0.70

9.

0.31 0.33 0.47* 0.44* 0.39* -0.07 0.44* -0.37* 0.04

0.61 0.58 0.53 1.00 0.89 0.61 1.00 0.67 0.55

10.

0.35* 0.42* 0.33 0.36* -0.12 0.32 -0.44* 0.11

0.45 0.51 0.80 0.70 0.64 0.40 0.51 0.50 0.50 0.45

1 I.

0.44* 0.47* 0.21 -0.18 0.28 -0.02 0.05

0.42 0.58 0.48 0.59 0.30 0.46 0.52 0.81 0.82 0.57 0.62

12.

0.40* 0.28 -0.15 0.77* -0.37* -0.08

0.48 0.57 0.61 0.65 0.64 0.43 0.56 0.28 0.39 0.61 0.57 0.71

13.

0.39* -0.31" 0.36* -0.35 0.06

0.51 0.89 0.22 0.54 0.24 0.85 0.58 0.34 0.20 0.59 0.46 0.78 0.50

14.

0.04 0.43* -0.48* 0.06

0.42 0.41 0.40 0.68 0.60 0.43 0.74 0.62 0.32 0.62 0.60 0.42 0.42 0.60

15.

16.

0.08 -0.23 -0.14

-0.38 ~).49 0.15 -0.06 0.13 -0.64 ~).05 ~).24 ~).05 ~).10 ~).18 -0.33 -0.21 -0.44 0.07

Table 2. Observed inter-correlation of the scales (below diagonal), disattenuated scale correlations (above diagonal) and correlations with social desirability and age

-0.38* 0.04

0.36 0.48 0.35 0.73 0.59 0.41 0.48 0.10 0.22 0.57 0.43 0.45 0.94 0.45 0.65 0.11

17.

-0.02

18.

~ o-~" = ~.

o'~" =

,z t~

1052

J.A. Harris

first and second test times (see Table 1). These results are possibly due to the time difference between test sessions. A minimum of 14 weeks separated the first test session from the second, but only a minimum of 8 weeks separated the second test session from the third. The interrelationships among the scales are listed in Table 2. The correlations reported are from the first test session (all participants). Also presented in Table 2 are the disattenuated correlations (corrected for the internal consistency values reported in Table 1). Age was not found to have a significant correlation with any of the scales measured. Within each group of scales, evidence is found of construct validity, or convergence, among the scales designed to measure specific dimensions of aggression. The three verbal aggression scales all have large positive and significant correlations with each other (r = 0.64 to 0.67). The two physical aggression scales (from The Aggression Questionnaire and the PAl) also have a positive and significant relationship (r = 0.54). The PAl (Morley, 1991) scales, which purport to measure borderline disorders (affect instability, self-harm, and negative relations), as well as the PAI aggressive attitude scale, appear to be strongly related to the anger scale from the Aggression Questionnaire (Buss & Perry, 1992). The hostility scale from the Aggression Questionnaire was found to have high positive correlations with The Aggression Questionnaire anger scale, the PAl affect instability scale, and the PAI negative relations scale. Scales measuring lack of frustration tolerance (Olweus, 1986), impulsivity (Gladue, 1991a,b), and impatience (Gladue, 1991a,b) were found to be positively related (r = 0.28 to 0.77), although a stronger relationship was found between impulsivity and lack of frustration control than with the other scale combinations. A final area investigated was the effects of social desirability on the self-report aggression scales. The aggression scales do appear to have some strong correlations with social desirability. The Aggression Questionnaire scales all had significant negative correlations with social desirability (r = - 0 . 3 7 to -0.59). These results suggest that social desirability may be a response bias with respect to measuring self-report aggression, a factor researchers may want to consider for future research, in particular because social desirability has been demonstrated to correlate with probability of item endorsement (see Stricker, Messick & Jackson, 1968). One means of addressing this issue, in particular in clinical or applied settings, is to include a social desirability measure in the test battery and interpret cautiously test scores from individuals with high social desirability scores. Possible limitations to the present study include the participant sample and the choice of measures. Only women were tested in the present study which limits the generalizability of the results. A second issue involves the measures assessed. Scales were chosen such that they were constructed to measure aggression and related facets, that the scales had not been extensively examined previously with The Aggression Questionnaire, as well as to keep the test session to a minimum of an hour. How other measures relate to The Aggression Questionnaire needs further exploration. In conclusion, The Aggression Questionnaire scales were found to have moderate to high reliability, both from internal consistency estimates and test-retest correlations over a period of seven months. The scales also appear to have convergent validity with other self-report measure of aggression. One cautionary note may be that the influence of social desirability responding may influence responses given on The Aggression Questionnaire which may reduce the validity of scale scores in situation such as clinical assessment or may attenuate convergent and discriminant validity estimates of the measures (Neill & Jackson, 1970). The present results suggest that individuals administering The Aggression Questionnaire should consider including a measure of social desirability to further investigate the influence of this response style on scale scores.

REFERENCES Archer, J., Holloway, R., & McLoughlin, K. (1995). Self-reportedphysical aggression among young men. Aggressive Behavior, 21, 325-342. Archer, J., Kilpatrick, G., & Bramwell, R. (1995). Comparison of two aggression inventories. Aggressive Behavior, 21, 371-380. Bernstein, I. H. & Gesn, P. R. (1997). On the dimensionality of the Buss/Perry aggression questionnaire. Behaviour Research and Therapy, 35, 563-568.

The aggression questionnaire

1053

Buss, A. H., & Durkee, A. (1957). An inventory for assessing different kinds of hostility. Journal of Consulting Psychology, 21,343-349. Buss, A. H., & Perry, M. (1992). The aggression questionnaire. Journal of Personality and Social Psychology, 63, 452459. Gladue, B. A. (1991a). Aggressive behavioral characteristics, hormones, and sexual orientation in men and women. Aggressive Behavior, 17, 313-326. Gladue, B. A. (1991b). Qualitative and quantitative sex differences in self-reported aggressive behavioral characteristics. Psychological Reports, 68, 675-684. Harris, J. A. (1995). Confirmatory factor analysis of the aggression questionnaire. Behaviour Research and Therapy, 33, 991-993. Harris, J. A., Rushton, J. P., Hampson, E., & Jackson, D. N. (1996). Salivary testosterone and self-report aggressive and pro-social personality characteristics in men and women. Aggressive Behavior, 22, 321-331. Harris, M. B. (1996). Aggressive experiences and aggressiveness: Relationships to ethnicity, gender, and age. Journal of Applied Social Psychology, 26, 843-870. Jackson, D. N. (1989). Personality researchform manual (3rd ed.). Port Huron, MI: Sigma Assessment Systems. Meesters, C., Muffs, P., Bosma, H., Schouten, E., & Beuving, S. (1996). Psychometric evaluation of the Dutch version of the aggression questionnaire. Behaviour Research and Therapy, 34, 839-843. Morley, L. C. (1991). Personality assessment inventory. Odessa, FL: Psychological Assessment Resources, Inc. Neill, J. A., & Jackson, D. N. (1970). An evaluation of item selection strategies in personality scale construction. Educational and Psychological Measurement, 30, 647-661. Olweus, D. (1986). Aggression and hormones: Behavioral relationships with testosterone and adrenaline. In D. Olweus, J. Block & M. Radke-Yarrow (Eds.), Development of antisocial and prosocial behavior: Research, theories, and issues (pp. 51-72). Orlando, FL: Academic Press. Paunonen, S. V. & Jackson, D. N. (1988). Nonverbal personality questionnaire. London, On: The University of Western Ontario. Russell, G. W., & Arms, R. L. (1995). False consensus effect, physical aggression, anger, and a willingness to escalate a disturbance. Aggressive Behavior, 21, 381-386. Stricker, L. J., Messick, S., & Jackson, D. N. (1968). Desirability judgments and self-reports as predictors of social behavior. Journal of Experimental Research in Personality, 3, 151-167.