- Email: [email protected]
Reliability of assessment of coronary-prone behavior with special reference to the bortner rating scale
0022-3999/79/0201-0045$02.00/0
lournal of Psychosomatic Research, Vol. 23, pp. 45 to 47. 0 Pergamon Press Ltd. 1979. Printed in Great Britain.
RELIABILITY OF ASSESSMENT OF CORONARY-PRONE BEHAVIOR WITH SPECIAL REFERENCE TO THE BORTNER RATING SCALE* t KENNETH P. PRICES (Received 14 August 1978) Abstract-Reliability studies on the Jenkins Activity Survey (JAS) and the Rosenman-Friedman interview for coronary-prone personality typing are reviewed and a study on the reliability of the Bortner Rating Scale for coronary-prone personality is reported. Two-month test-retest reliability on the Bortner is high, and similar to that of the interview and the JAS. Reliabilities on some of the individual items are lower and the implications of this for assessing the coronary-prone personality are discussed. THE ORIGINAL
assessment technique for determining the presence or absence of coronary-prone behavior patterns was a face-to-face interview developed by Friedman and Rosenman [l]. Despite the enormous volume of research involving the interview method of behavioral typing, by comparison relatively little work has been devoted to establishing the reliability of the interview assessment. Data on inter-rater reliabilities have been reported in three studies. Keith et al. [2] reported that the same interviewer re-rated 100 randomly-selected recordings of interviews and agreed with his earlier global rating of Type A or Type B in 74 cases and agreed with his earlier assignment of cases to the four subgroupings Al-B4 in only 57 instances. They concluded that the interview did not have high reliability. Jenkins et al. [3], on the other hand, claimed that their own data were indicative of good reliability for the interview. In their study, Jenkins re-rated 25 tape-recorded interviews originally conducted by Rosenman some years earlier. Results were somewhat better than those of Keith et al [2]. Inter-rater reliability on the global ratings was 84 % and on the specific subgrouping 64%. Some impressively higher inter-rater reliabilities have been recently found. Two studies [4, 51 examined the agreement of multiple raters on specific, objectively-defined voice characteristics in recordings of traditional assessment interviews. Reliability coefficients were found in the nineties in one of the studies [4] and in the sixties in the other 151. While inter-rater reliability of coronary-prone behavior patterns is important, test-retest reliability, or the stability of behavior over time, is crucial for the argument that behavioral traits are related to cardiovascular disease. In one study addressing this issue [3], two interviews of 1064 men were conducted 12-20 months apart. About 80% of the subjects were given the same global rating. But only 66.4% of the subjects were scored the same on the four sub-groupings. If an additional analysis just on the extreme types is performed on the data presented in Table II of Jenkins er al. [3]. one finds the reliability for Type A1 = 63 ‘A and for B4 = 51-%. These are important (and disappointing) statistics, because extreme Type AI and Type Bq are often selected out for studies of behavioral differences 161. Stability of behavioral typing would seem to be required in order for behavioral differences that &z found experimentally to be meaningful, since it is stable, chronic behavioral traits that are believed to be related to coronary heart disease. In a recent review, Rowland and Sokol[7l expressed concern about the interview’s reliability, lack of published scoring criteria, time-consuming administration, cost of large-scale investigations, and the difficulty in adequately training interviewers. Many of these concerns have been acknowledged by Jenkins et al. [3,8]. In an attempt to make assessment of coronary-prone behavior more objective and less timeconsuming and expensive, a paper-and-pencil questionnaire was developed, the Jenkins Activity Survey (JAS) and validated against the interview procedure [S, 91. The one study found [lo] which assessed the test-retest correlation for the JAS reports the re-administration of the questionnaire at l- and 4-year intervals. The correlations ranged between 0.64 and 0.74 for 3 of the JAS sub-scales, and between 0.56 and 0.60 for a fourth. The JAS has been used in numerous studies on coronary heart disease and behavior [6,11]; however, Jenkins et al. [IO] have cautioned that the JAS still misclassifies *From the University of Texas Health Science Center at Dallas, Texas, U.S.A. tThis research was supported in part by a Stress Grant from the RochePsychiatric Service Institute, Nutley, NJ, U.S.A. SRequests for reprints should be sent to: Dr. Kenneth P. Price, UTHSCD, Division of Psychology, 5323 Harry Hines Boulevard, Dallas, TX 75235, U.S.A. 45
46
KENNETH
P. PRICE
too many subjects to be useful clinically or in the evaluation of small groups. They indicate that the Type A scale yields many false positives and false negatives. Another paper-and-pencil, semantic differential questionnaire to measure coronary proneness was developed by Bortner [12]. A commendable aspect of this questionnaire was the establishment of the direction of scoring items empirically, rather than a priori. Two versions of the questionnaire were designed: a 14-item full scale and a 7-item short scale. When scored by a weighted method, the 7-item questionnaire correlates highly with the full scale. The major advantages of the Bortner scale are its brevity and ease of administration and scoring. Surprisingly, only a few studies have reported using it [13-161. In two studies that have used the Bortner scale [14, 161 it was found to correlate only moderately with the JAS. The present study was undertaken to assess the test-retest reliability of the Bortner scale. PROCEDURE AND RESULTS A large number of college undergraduate and graduate students and medical students were given the short form of the Bortner scale and solicited for participation in a study on autonomic nervous system activity and problem solving. Eight weeks later, the same individuals (except those who participated in the study-and were no longer naive) were asked to re-take the Bortner scale. Responses were received from 100 females and 53 males. Pearson product moment correlations calculated on the repeated test scores were, respectively, Y = 0.74 and Y = 0.72. Separate correlations were calculated for each of the 7 items [which related to (1) promptness, (2) competitiveness, (3) feelings of being rushed, (4) number of things done simultaneously, (5) speed of doing things, (6) expressiveness of feelings, and (7) number of interests; see [12] for a full discussion of these items]. The correlations for each item are presented in Table 1. As can be seen, reliability varied from item to item and between the sexes. The most discrepant reliability score between sexes was item number 7. TABLE
I.-TEST-RETEST
CORRELATIONS OF ITEMS OF THE SHORT FORM OF THE BORTNER ASSESSMENTOFCORONARY-PRONE BEHAVIOR
SCALE
FOR
Item 1
Males (fz = 53) Females (n = 100)
0.83 0.74
2 0.82 0.63
3 0.51 0.64
4 0.31 0.58
5 0.64 0.59
6 0.76 0.73
7 0.30 0.75
DISCUSSION The present study found a test-retest reliability for the Bortner scale about as good as that reported for the interview and the JAS. Assuming an equivalent success rate in predicting CHD between the JAS and the Bortner, use of the Bortner has the advantage of extreme brevity and criterion-related scoring over the JAS. One drawback is that the Bortner has not been used as frequently as the JAS and therefore data about its utility in the prediction of CHD is lacking, and another is that it has been found to correlate only moderately with the JAS [14, 161. Present assessment techniques typically find that CHD develops statistically significantly more often in men labeled Type A than in those labeled Type B. But the lack of precision in the prediction of CHD may be, in part, because of the moderate to low test-retest correlations of the assessment tools used. This is illustrated by the scores on the individual items found in the present study. If the Bortner scale reflects traits of coronary proneness or resistance, it is reasonable to expect that the behavior it measures would remain stable over a 2-month period. However, the low correlations found on some of the items reveal a considerable variability in behavior-or at least a variability in the reporting of behavior-over time. This variability may be due to lack of precision in the questionnaire items or to lack of consistency in the behavior of individuals. Questions about the stability of behavior and the relative contributions of person variables and situational variables to the determination of behavior have been debated with great intensity in the recent psychological literature [17, 181. Two suggestions that have been made for improving the prediction of cross-situational behavior are, first, to specify the situations that the investigator believes relevant to the construct he is measuring, and second, to rely on the data of just those individuals who exhibit or report behavior that is cross-situationally consistent with the investigator’s construct [19]. These notions can be applied to the assessment of the coronary-prone personality, first by continuing studies on the specific behaviors that comprise the coronary-prone personality [4, 6, 161 and the situational determinants of those behaviors [6, 161; second, by following, for research purposes, those individuals who show a high consistency or reliability of behavior over time, while considering separately data from individuals who show variability in the behaviors of interest.
Reliability of assessment of coronary-prone
behavior
47
REFERENCES 1. FRIEDMANM. and ROSENMANR. H. Association of specific overt behavior pattern with blood and cardiovascular findings: blood cholesterol level, blood clotting time, incidence of arcus senilis and clinical coronary artery disease. J. Am. Med. Ass. 169, 1286-1296 (1959). 2. KEITHR. A., LOWN B. and STAREF. J. Coronary heart disease and behavior patterns: an examination of method. Psychosom. Med. 27, 424-434.(1965). 3. JENKINSC. D., ROSENMANR. H. and FRIEDMANM. Replicability of rating the coronary-prone behavior pattern. Br. J. Prev. Sot. Med. 22, 16-22 (1968). 4. SCHERWITZL., BERTONK. and LEVENTHALH. Type A assessment and interaction in the behavior pattern interview. Psvchosom. Med. 39, 229-240 (1977). 5. SCHUCKERB. and Jacoss D. R. Assessment of behavior pattern A by voice characteristics. Psychosom. Med. 39,219-228 (1977). 6. GLASS D. C. Stress, behavior patterns, and coronary disease. Am. Scient. 65, 177-187 (1977). 7. ROWLAND K. F. and SOKOL B. A review of research examining the coronary-prone behavior pattern. J. Human Stress 3, 26-33 (1977). 8. JENKINSC. D., ROSENMANR. H. and FRIEDMANM. Development of an objective psychological test for the determination of the coronary-prone behavior pattern in employed men. J. Chron. Dis. 20, 371-379 (1967). 9. JENKINSC. D., ZYZANSKI S. J. and ROSENMANR. H. Progress toward validation of a computerscored test for the Type A coronary-prone behavior pattern. Psychosom. Med. 33,193-202 (1971). 10. JENKINSC. D.. ROSENMANR. H. and ZYZANSKI S. J. Prediction of clinical coronary heart disease by a test for the coronary-prone behavior pattern. New Engl. J. Med. 290, 1271-1275 (1974). 11. JENKINSC. D. Recent evidence supporting psychologic and social risk factors for coronary disease. New Eng. J. Med. 294,987-994, 1033-1038 (1976). 12. BORTNERR. W. A short rating scale as a potential measure of pattern A behavior. J. Chron. Dis. 22, 87-91 (1969). 13. DEFOURXY M. and FRANKIGNOULM. A propos du comuortement predisnosant aux coronaropathies (overt pattern A). J. Psychosom. Res.-17,219-236 (1973). _ _ 14. RUSTINR. M,. DRAMAIX M.. KITTEL F.. DEGREC.. KORNITZERM.. THILLY C. and DEBACKERG. Validation de techniques d’eialuation d; profil comportemental A ;tilisBes dans 1 “Projet Belge de PrCvention des affections cardiovasculaires” (P.B.P.). Rev. Epidemioi. Sante’ Pablique 24,497-507 (1976). 15. COHEND. Sex differences in overt personality patterns in older men and women. Gerontology 23, 262-266 (1977). 16. PRICE K. P. and CLARKE L. K. Behavioral and psychophysiological correlates of the coronaryprone personality: new data and unanswered questions. J. Psychosom. Res. 22,409-417 (1978). 17. MISCHELW. Personality and Assessment. John Wiley, New York (1968). 18. BOWERSK. S. Situationism in psychology: an analysis and a critique. Psychol. Rev. 80, 307-336 (1973). 19. BEM D. J. and ALLEN A. On predicting some of the people some of the time: the search for crosssituational consistencies in behavior. Psycho/. Rev. 81, 506-520 (1974).
lournal of Psychosomatic Research, Vol. 23, pp. 45 to 47. 0 Pergamon Press Ltd. 1979. Printed in Great Britain.
RELIABILITY OF ASSESSMENT OF CORONARY-PRONE BEHAVIOR WITH SPECIAL REFERENCE TO THE BORTNER RATING SCALE* t KENNETH P. PRICES (Received 14 August 1978) Abstract-Reliability studies on the Jenkins Activity Survey (JAS) and the Rosenman-Friedman interview for coronary-prone personality typing are reviewed and a study on the reliability of the Bortner Rating Scale for coronary-prone personality is reported. Two-month test-retest reliability on the Bortner is high, and similar to that of the interview and the JAS. Reliabilities on some of the individual items are lower and the implications of this for assessing the coronary-prone personality are discussed. THE ORIGINAL
assessment technique for determining the presence or absence of coronary-prone behavior patterns was a face-to-face interview developed by Friedman and Rosenman [l]. Despite the enormous volume of research involving the interview method of behavioral typing, by comparison relatively little work has been devoted to establishing the reliability of the interview assessment. Data on inter-rater reliabilities have been reported in three studies. Keith et al. [2] reported that the same interviewer re-rated 100 randomly-selected recordings of interviews and agreed with his earlier global rating of Type A or Type B in 74 cases and agreed with his earlier assignment of cases to the four subgroupings Al-B4 in only 57 instances. They concluded that the interview did not have high reliability. Jenkins et al. [3], on the other hand, claimed that their own data were indicative of good reliability for the interview. In their study, Jenkins re-rated 25 tape-recorded interviews originally conducted by Rosenman some years earlier. Results were somewhat better than those of Keith et al [2]. Inter-rater reliability on the global ratings was 84 % and on the specific subgrouping 64%. Some impressively higher inter-rater reliabilities have been recently found. Two studies [4, 51 examined the agreement of multiple raters on specific, objectively-defined voice characteristics in recordings of traditional assessment interviews. Reliability coefficients were found in the nineties in one of the studies [4] and in the sixties in the other 151. While inter-rater reliability of coronary-prone behavior patterns is important, test-retest reliability, or the stability of behavior over time, is crucial for the argument that behavioral traits are related to cardiovascular disease. In one study addressing this issue [3], two interviews of 1064 men were conducted 12-20 months apart. About 80% of the subjects were given the same global rating. But only 66.4% of the subjects were scored the same on the four sub-groupings. If an additional analysis just on the extreme types is performed on the data presented in Table II of Jenkins er al. [3]. one finds the reliability for Type A1 = 63 ‘A and for B4 = 51-%. These are important (and disappointing) statistics, because extreme Type AI and Type Bq are often selected out for studies of behavioral differences 161. Stability of behavioral typing would seem to be required in order for behavioral differences that &z found experimentally to be meaningful, since it is stable, chronic behavioral traits that are believed to be related to coronary heart disease. In a recent review, Rowland and Sokol[7l expressed concern about the interview’s reliability, lack of published scoring criteria, time-consuming administration, cost of large-scale investigations, and the difficulty in adequately training interviewers. Many of these concerns have been acknowledged by Jenkins et al. [3,8]. In an attempt to make assessment of coronary-prone behavior more objective and less timeconsuming and expensive, a paper-and-pencil questionnaire was developed, the Jenkins Activity Survey (JAS) and validated against the interview procedure [S, 91. The one study found [lo] which assessed the test-retest correlation for the JAS reports the re-administration of the questionnaire at l- and 4-year intervals. The correlations ranged between 0.64 and 0.74 for 3 of the JAS sub-scales, and between 0.56 and 0.60 for a fourth. The JAS has been used in numerous studies on coronary heart disease and behavior [6,11]; however, Jenkins et al. [IO] have cautioned that the JAS still misclassifies *From the University of Texas Health Science Center at Dallas, Texas, U.S.A. tThis research was supported in part by a Stress Grant from the RochePsychiatric Service Institute, Nutley, NJ, U.S.A. SRequests for reprints should be sent to: Dr. Kenneth P. Price, UTHSCD, Division of Psychology, 5323 Harry Hines Boulevard, Dallas, TX 75235, U.S.A. 45
46
KENNETH
P. PRICE
too many subjects to be useful clinically or in the evaluation of small groups. They indicate that the Type A scale yields many false positives and false negatives. Another paper-and-pencil, semantic differential questionnaire to measure coronary proneness was developed by Bortner [12]. A commendable aspect of this questionnaire was the establishment of the direction of scoring items empirically, rather than a priori. Two versions of the questionnaire were designed: a 14-item full scale and a 7-item short scale. When scored by a weighted method, the 7-item questionnaire correlates highly with the full scale. The major advantages of the Bortner scale are its brevity and ease of administration and scoring. Surprisingly, only a few studies have reported using it [13-161. In two studies that have used the Bortner scale [14, 161 it was found to correlate only moderately with the JAS. The present study was undertaken to assess the test-retest reliability of the Bortner scale. PROCEDURE AND RESULTS A large number of college undergraduate and graduate students and medical students were given the short form of the Bortner scale and solicited for participation in a study on autonomic nervous system activity and problem solving. Eight weeks later, the same individuals (except those who participated in the study-and were no longer naive) were asked to re-take the Bortner scale. Responses were received from 100 females and 53 males. Pearson product moment correlations calculated on the repeated test scores were, respectively, Y = 0.74 and Y = 0.72. Separate correlations were calculated for each of the 7 items [which related to (1) promptness, (2) competitiveness, (3) feelings of being rushed, (4) number of things done simultaneously, (5) speed of doing things, (6) expressiveness of feelings, and (7) number of interests; see [12] for a full discussion of these items]. The correlations for each item are presented in Table 1. As can be seen, reliability varied from item to item and between the sexes. The most discrepant reliability score between sexes was item number 7. TABLE
I.-TEST-RETEST
CORRELATIONS OF ITEMS OF THE SHORT FORM OF THE BORTNER ASSESSMENTOFCORONARY-PRONE BEHAVIOR
SCALE
FOR
Item 1
Males (fz = 53) Females (n = 100)
0.83 0.74
2 0.82 0.63
3 0.51 0.64
4 0.31 0.58
5 0.64 0.59
6 0.76 0.73
7 0.30 0.75
DISCUSSION The present study found a test-retest reliability for the Bortner scale about as good as that reported for the interview and the JAS. Assuming an equivalent success rate in predicting CHD between the JAS and the Bortner, use of the Bortner has the advantage of extreme brevity and criterion-related scoring over the JAS. One drawback is that the Bortner has not been used as frequently as the JAS and therefore data about its utility in the prediction of CHD is lacking, and another is that it has been found to correlate only moderately with the JAS [14, 161. Present assessment techniques typically find that CHD develops statistically significantly more often in men labeled Type A than in those labeled Type B. But the lack of precision in the prediction of CHD may be, in part, because of the moderate to low test-retest correlations of the assessment tools used. This is illustrated by the scores on the individual items found in the present study. If the Bortner scale reflects traits of coronary proneness or resistance, it is reasonable to expect that the behavior it measures would remain stable over a 2-month period. However, the low correlations found on some of the items reveal a considerable variability in behavior-or at least a variability in the reporting of behavior-over time. This variability may be due to lack of precision in the questionnaire items or to lack of consistency in the behavior of individuals. Questions about the stability of behavior and the relative contributions of person variables and situational variables to the determination of behavior have been debated with great intensity in the recent psychological literature [17, 181. Two suggestions that have been made for improving the prediction of cross-situational behavior are, first, to specify the situations that the investigator believes relevant to the construct he is measuring, and second, to rely on the data of just those individuals who exhibit or report behavior that is cross-situationally consistent with the investigator’s construct [19]. These notions can be applied to the assessment of the coronary-prone personality, first by continuing studies on the specific behaviors that comprise the coronary-prone personality [4, 6, 161 and the situational determinants of those behaviors [6, 161; second, by following, for research purposes, those individuals who show a high consistency or reliability of behavior over time, while considering separately data from individuals who show variability in the behaviors of interest.
Reliability of assessment of coronary-prone
behavior
47
REFERENCES 1. FRIEDMANM. and ROSENMANR. H. Association of specific overt behavior pattern with blood and cardiovascular findings: blood cholesterol level, blood clotting time, incidence of arcus senilis and clinical coronary artery disease. J. Am. Med. Ass. 169, 1286-1296 (1959). 2. KEITHR. A., LOWN B. and STAREF. J. Coronary heart disease and behavior patterns: an examination of method. Psychosom. Med. 27, 424-434.(1965). 3. JENKINSC. D., ROSENMANR. H. and FRIEDMANM. Replicability of rating the coronary-prone behavior pattern. Br. J. Prev. Sot. Med. 22, 16-22 (1968). 4. SCHERWITZL., BERTONK. and LEVENTHALH. Type A assessment and interaction in the behavior pattern interview. Psvchosom. Med. 39, 229-240 (1977). 5. SCHUCKERB. and Jacoss D. R. Assessment of behavior pattern A by voice characteristics. Psychosom. Med. 39,219-228 (1977). 6. GLASS D. C. Stress, behavior patterns, and coronary disease. Am. Scient. 65, 177-187 (1977). 7. ROWLAND K. F. and SOKOL B. A review of research examining the coronary-prone behavior pattern. J. Human Stress 3, 26-33 (1977). 8. JENKINSC. D., ROSENMANR. H. and FRIEDMANM. Development of an objective psychological test for the determination of the coronary-prone behavior pattern in employed men. J. Chron. Dis. 20, 371-379 (1967). 9. JENKINSC. D., ZYZANSKI S. J. and ROSENMANR. H. Progress toward validation of a computerscored test for the Type A coronary-prone behavior pattern. Psychosom. Med. 33,193-202 (1971). 10. JENKINSC. D.. ROSENMANR. H. and ZYZANSKI S. J. Prediction of clinical coronary heart disease by a test for the coronary-prone behavior pattern. New Engl. J. Med. 290, 1271-1275 (1974). 11. JENKINSC. D. Recent evidence supporting psychologic and social risk factors for coronary disease. New Eng. J. Med. 294,987-994, 1033-1038 (1976). 12. BORTNERR. W. A short rating scale as a potential measure of pattern A behavior. J. Chron. Dis. 22, 87-91 (1969). 13. DEFOURXY M. and FRANKIGNOULM. A propos du comuortement predisnosant aux coronaropathies (overt pattern A). J. Psychosom. Res.-17,219-236 (1973). _ _ 14. RUSTINR. M,. DRAMAIX M.. KITTEL F.. DEGREC.. KORNITZERM.. THILLY C. and DEBACKERG. Validation de techniques d’eialuation d; profil comportemental A ;tilisBes dans 1 “Projet Belge de PrCvention des affections cardiovasculaires” (P.B.P.). Rev. Epidemioi. Sante’ Pablique 24,497-507 (1976). 15. COHEND. Sex differences in overt personality patterns in older men and women. Gerontology 23, 262-266 (1977). 16. PRICE K. P. and CLARKE L. K. Behavioral and psychophysiological correlates of the coronaryprone personality: new data and unanswered questions. J. Psychosom. Res. 22,409-417 (1978). 17. MISCHELW. Personality and Assessment. John Wiley, New York (1968). 18. BOWERSK. S. Situationism in psychology: an analysis and a critique. Psychol. Rev. 80, 307-336 (1973). 19. BEM D. J. and ALLEN A. On predicting some of the people some of the time: the search for crosssituational consistencies in behavior. Psycho/. Rev. 81, 506-520 (1974).