Temporal stability of self-report measures in agoraphobia research

Temporal stability of self-report measures in agoraphobia research

Behau. Res. Thu. 0005.7967183 $3.00+0.00 Vol. 21.No. 6,pp.695-698,1983 Copyright ic‘ 1983Pergamon Press Ltd Printed in Great Britain. All rights r...

381KB Sizes 0 Downloads 32 Views

Behau. Res. Thu.

0005.7967183 $3.00+0.00

Vol. 21.No. 6,pp.695-698,1983

Copyright ic‘ 1983Pergamon Press Ltd

Printed in Great Britain. All rights reserved

Temporal

stability

of self-report

measures in agoraphobia

research

LARRY MICHELSON* and MATIG MAVISSAKAL~AN Unirersit,v of Pittsburgh,

School of Medicine, Department qf Psychiatry. Western Ps_vchiatric Institute Clinic, 3811 O’Hara Street, Pittsburgh, PA 1.5213, U.S.A. (Receioed

5 Februury

und

1983)

subjects, meeting DSM-III criteria for agoraphobia completed, on two occasions, a assessment battery. Measures encompassed anxiety, phobia, depression and general

Summary-Forty

comprehensive

symptomatology dimensions of functioning, and consisted of instruments widely employed as outcome measures in agoraphobia research. The test-retest time periods were divided into 4-, IO- and 16.week intervals to ascertain their temporal stability, an often ignored parameter of reliability. The results indicate that agoraphobia, anxiety, depression and general symptomatology measures were temporally stable. However, test-retest reliabilities of the scales’ subscores were generally inferior to those of the total score. The findings are discussed, with recommendations for more programmatic psychometric research in the assessment of anxiety disorders.

INTRODUCTION Over the past decade there has been a growing

interest in refining assessment strategies and instruments, as reflected in the multitude of books, journals and symposia which are solely focused upon this area (cf. Barlow, 1981; Ciminero, Calhoun and Adams, 1977; Hersen and Bellack, I98 1). Behaviorally-oriented researchers have tended to eschew the use of, or reliance upon, self-report instruments (Borkovec, Weerts and Bernstein, 1977), in favor of more overt motoric and physiological channels. However, as frequently reported, the vast majority of researchers have employed numerous self-report measures of outcome as this is often the most efficient, if not the sole method for “obtaining information regarding the cognitive-phenomenological component of anxiety” (p. 385. Borkovec et al., 1977). Unfortunately, despite the widespread utilization of the various instruments measuring phobia, anxiety, panic and related depression, there are presently no published data regarding the temporal stability of these scales, specifically in regard to agoraphobic populations. As test-retest reliability is an essential and germane psychometric property for anxiety research, it is imperative and requisite that these measures be subjected to more careful scrutiny. Lest perceived outcome effects be misinterpreted, clinical researchers need to consider the temporal patterns of change for each specific disorder. In regard to the present study, we have selected a homogeneous sample of agoraphobics. The sample was selected on the basis of the disorder being chronic, severe and affecting multiple domains of functioning (e.g. phobia. panic, depression etc.). Moreover, given the widespread interest in anxiety disorders research, we felt the present study would provide important data to other investigators and also help stimulate additional psychometric studies of these widely-used outcome measures. Therefore, the primary objective of the present study was to examine, using three different time intervals, the temporal stability of agoraphobia and related assessment devices on a homogeneous population. The time intervals represent ‘traditional’ (4-week), ‘extended’ (IO-week) and ‘no-treatment control’ (16-week) conditions, therefore allowing more careful delineation of the naturalistic temporal changes occurring on both a short- and long-term basis.

METHOD Subjects Forty

Ss were selected

to participate

in the study

on the basis of the following

inclusion

criteria:

(1) DSM-III diagnosis of agoraphobia; (2) age of onset prior to age 40; and (3) duration of present illness of at least 1 yr. Exclusion

criteria

included:

(1) presence of features indicative of and/or previous diagnosis of a major affective disorder. schizophrenia, organic brain syndrome and obsessive%ompulsive disorder; and (2) social phobics and anxiety neurotics without the typical fears of going out alone to open spaces, streets. shops, crowds, cars and using public transport. The 40 Ss had a mean age of 38 yr, with a range of 23-62 yr. Thirty-six were female, and 4 were male. The average duration of illness was Il. 1 yr. All Ss were evaluated by both a clinical psychologist (L.M.) and a psychiatrist (M.M.) Ss were administered their first assessment battery at the time of their initial evaluation, and were not in treatment during the testing period. Procedure Subjects completed the first and second assessments at one of three intervals: (a) 4 weeks; (b) 10 weeks; and (c) 16 weeks, The use of these test-retest periods allowed for examination of temporal stability on a short-, moderateand long-term basis. The sample sizes for these periods were N = 13, N = 14 and N = 13, respectively. *To whom

all reprint

requests

should

be addressed. 695

696

CASE HISTORIES AND SHORWR

COMMUNICATIONS

A.ssrssmen6 Meusures battery consisting of wvidety-usedassessment of phobic functionjng and included:

A comprehensive the diverse domains

instruments

was employed.

The measures

encompassed

Self-Roling ~f.Sevevir~ (S&S’), which is a widely-used q-point analog scale rated by the patient in answer to the rollowing quesiion: “How would ;ou rate the present state of your phobic symptoms on the scale below’.”1 On this scale 0 represents no phobias present; 2--.-slightly disturbing but not really disabling, 4-definitely disturbing, disahlmg: b-markedly disturbing, disabling; and S-very severely disturbing, disabling (Marks and Mathews. 1979).

The total and subscale scores of the Hopkins Symptom Covi, 1974) were used to assess general symptomatology.

Checklisr

(HSCL;

Derogatis.

Lipman.

Rick&.

Uhlenhuth

and

The fiur Sun~ey Schedule (FSS; Wolpe and Lang, 1964) was used as a measure of general fearfulness. %P Four Qu~stimmire (FQ; Marks and Mathews, 1973) is a commonly-used instrument and generates a totaf and several subscores. The s&scores are all of particular relevance to agoraphobics because the scale yields agoraphobia IFQ-AC), blood-injury {FQ-31) and social fFQ-S) phobia factors.

Clinical anxiety was assessed by the use of the Zung Sdf-Ratin,qAnxiet? Scule (SM; Zung. 197 I). an abbreviated version of the Tuybor Manjfeest Anxiety ScaLe (MAS), since the 28 items of this abbreviuted version correlates highly with the remaining 22 items of the original scale (Taylor, 19S3f and the Spielberger (Sltrte-7”Gr Anui<~~~ @t<~s!io,trtuirca (STAI: Spielberger, Gorsuch and Lushene, IYfOf. Parric Panic was measured attacks (Mavissakalian

using a 9-point scale (PANIC) and Michelson, 1982).

instructing

the patient

Thr Beck L?epwssion Irzrenfory jBDk Beck, Ward, Mend&on, Mock Dr~~rt&nn SC& {SDS; Zung, t965f were used as depression measures.

to rate the ‘frequency

and Erbaugh.

t9hII

and intensity’

of their

and the Zunl: .Se[i-Krr,in~q

All data were keypunched and verified prior to performing the analyses. The DEC-IO system was used at the University of Pittsburgh Computer Science Center. SPSS was employed for all statistical analysts. RESULTS The test-retest

data from the 4-, IO- and l6-week intervals were analyzed using Pearson correlations. The results. as presented in Tabk I, reveal that the vast majority of assessment ~nstrun~en~s possess acceptablc temporal stability across the three time intervals, in regard to total scores. Examination of subscore reliability yielded somewhat lower and less consistent indexes of temporal stability, possibly related to their having fewer items. Interestingly. several measures evidenced increased test-retest reliability (e.g. BDI, FQ-Total) over time. whereas others (e.g. MAS and STAI) showed marked diminutions.

The results of tire present study suggest that total scores of measures specifically assessing the phobic-anxiety dimensions of agorapho~ja (e.g. FQ-AG, FSS. SRS, ZAS and the MAS) are temporally stable over both short and longer test retest intervals. Indeed, the FQ-AG averaged 0.83 reliability across the testing intervals. Likewise, total scores for the FSS revealed high temporal stability (e.g. X correlations = 0.87), irrespective of testing period. The stability vf the self-report measure of severity of agoraphobia was also quite stable, averaging 0.82 reliability across the test-retest periods. Thus, in regard it would appear that the severity, intensity, to measures which specifically assess agoraphobic symptomatology, pervasiveness and frequency of the phobic anxiety and avoidance experiences remain relatively constant over a 4- to I&week interval. ‘Therefore, clinicians and researchers who utiIize these measures can probably empioy them as temporally-reliable indices of agoraphobic functioning. However. in regard to the STAI, it’s temporal stab&y was unacceptable at Periods 2 and 3. In regard to concurrent domains of functioning such as those measured by the HSCL. PANIC, BDI and SDS. 1 somewhat different oicture emerges. The HSCL showed increasingly stronger temporal stability over time. with an initially modest correlation of Perind 1being improved by the second- and third-period assessments. Conversely, the PANIC measure which had a marginally acceptable test-retest r 2; 0.69, for Period I, showed decreases in stability over time, A visual inspection of the data revealed marked variations in Period 2. During Period 3, the PANIC test--retest reliability declined to r = 0.14. The reasons fur this phenomena are not entirely clear. Possibly the nature of the index ie.rr. a unitarv &mint scale), may account for the bw reliability. Another explanation of why it’s reliability decreased from &hods t to J m&t be the.wel~~docum~nt~d episodic nature of panic attacks which might produce greater variance over longer periods of time than over a shorter temporal span. Therefore, a PANIC score should prob;lbly be idken as a weekly process measure, rather than as a pm-post outcome scale. The two depression measures, namely the BDI and the SDS, showed acceptable rcliabihty coefficients. On an overall basis, the test-retest reliabilities of the BDI and SDS were 0.80 and 0.80, respectively. In addition, their reliability tended to improve as the test-retest periods increased. These findings suggest that agoraphobic patients’ depression is relatively stable and chronic, mu& like the specific agoraphobic symptoms, and does not dissipate over a 4.month period suggesting that it may be secondary to this severe and chronic anxiety disorder. Lo

CASE HISTORIES

Table 1. Temporal

stability Period

Measure Global Severity Self-Rating of Severity Hop-tom Checklist Obsessive-Compulsive subscore Depression subscore Anxiety subscore Somatic subscore Interpersonal subscore Total score Phobia Fear Survey Schedule A little Not much A fair amount Very much

AND

of agoraphobia

SHORTER

outcome

697

COMMUNICATIONS

measures

14 weeks

at 4-, IO- and 16-week intervals Period 3-16 weeks

Permd 2-10 weeks CON.

Test R

Retest R

5.2

0.91

6.4

5.8

0.76

15.50

15.14

0.89

15.00

14.46

0.84

0.68 0.74 0.76 0.65

21.29 15.50 23.14 14.21

19.86 15.21 23.57 13.07

0.77 0.69 0.92 0.78

20.00 16.71 24.00 14.00

19.77 15.31 21.46 13.69

0.84 0.77 0.86 0.79

0.75

89 64

86.07

0.86

89.69

84.69

0.84

C0rr.

Test R

Retest n

I

5.5

0.60

22.08 17.46 23.31 16.23 94.15

Test R

Retest R

5.5

5.4

0.8

17.00

15.69

22.38 17.69 24.08 15.85 97.00

COK .~

21.62 32.31 33.00 58.23

22.08 34.62 35.69 62. I5

0.63 0.50 0.27 0 x7

27.50 30.2 I 26.57 56.57

25.71 29.79 30.29 46.29

0.82 0.64 0.62 0.95

23.92 35.08 28.62 52.00

24.62 27.77 28.77 43.62

0.74 0.44 0.36 0.75

146.38

154.54

0.82

141.14

132.21

0.89

139.62

120.69

0.90

Blood-mlury subscore Social Phobia subscore

29.46 17.0x 19.69

31.62 19.00 21.77

0.86 0.34 0.61

29.43 13.14 17.50

25.93 IO.93 15.79

0.86 0.73 0.81

33.69 14.69 16.46

31.62 14 I5 14.08

0.85 0.86 0.84

Total ‘A‘ Total ‘B’

66 23 20. I5

72.38 19.08

0.69 0.58

60.07 21.36

51.79 18.07

0.71 0.58

64.85 19.31

59.x5 16.54

0.90 0.90

43.15

41.92

0.x4

42.79

42 71

0.81

44.69

45.23

0.82

19.77

19.92

0.90

23.21

19.93

0.85

I6 69

17.23

0.71

44.46 49.54

44.92 47.33

0.78 0.73

43.29 48.14

42.43 47.86

0 66 0.43

45.23 44.69

42.69 46.00

0.70 0.33

94.00

92.25

0.83

91 43

90.29

0.52

89.92

89.46

0.43

Scale

4.92

5.08

0.69

4.29

4.23

0.41

4.71

5.77

0.14

Depression Beck Depression Inventory -._. Lung Depression Scale

14.23 52.46

13.42 55.83

0.69 0.71

17.64 58.43

15.85 55.71

0.86 0.89

13.38 52.46

13.62 54.3 I

0.84 0.80

Total

Fear Questionnaire Agoraphobia subscore

Anxiety Zung Anxiety Scale Taylor Manifest Anxiety Scale (abbreviated form) Splelberger StatemTralt Anxiety Scale State subscore Trait subscore Total score Panic 7 SubJective

Symptom

In conclusion, it appears that agoraphobia assessment devices analyzed herein, are temporally stable over both short and extended time intervals, Likewise, both depression measures and the HSCL exceed generally-accepted levels of what is considered requisite in regard to temporal stability. Not surprisingly, the test-retest reliability of the various scales’ subscores were generally inferior to those of the total score. Given their increased vulnerability to spurious fluctuations, both clinicians and researchers need to exercise caution in their use. The only possible exception might be the use of the agoraphobia subscale of the Fear Questionnaire, given its high reliability coefficients across all assessment periods. However, in all cases where subscores are reported, total scores should also be routinely included. Often test-retests are performed at 2- to 4-week intervals. Based upon these results, it appears that the psychometric properties of certain instruments might change as a function of the duration of the testing intervals. Clearly, further test-retest, and related psychometric studies are needed to ascertain the reliability and validity of these and related measures. However, despite transient fluctuations, agoraphobia is a chronic disorder, which when assessed at different intervals, reveals temporal stability, thereby suggesting that the minor changes in reported severity do not interfere with outcome measurement. AcknoMled~em~nt-This

study

was supported

by NIMH

Grants

MH34177

and

MH36299.

REFERENCES Barlow D. (Ed.) (1981) Behavioral Assessment of Adult Disorders. Guildford Press, New York. Beck A. T., Ward C. H., Mendelson M. er al. (1961) An inventory for measuring depression. Archs gen. Psychiuf. 4, 53-63. Borkovec T. D., Weerts T. C. and Berstein D. A. (1977) Assessment of anxiety. In Handbook of Behavioral Assessmen (Edited by Ciminero A. R., Calhoun K., S. and Adams H. E.). Wiley, New York. Ciminero A. R., Calhoun K. S. and Adams H. E. (Eds) (1977) Handbook of Behavioral Assessment. Wiley, New York. Derogatis L. R., Lipman R. S., Rickels K.. Uhlenhuth and Covi (1974) The Hopkins Symptom Checklist (HSCL): a self-report symptom inventory. Behav. Sci. 19, 1-15. Hersen M. and Bellack A. S. (Eds] (1981) Behavioral Assessment: A Practical Handbook, 2nd edn. Plenum Press, New York. Marks I. M. and Mathews A. M. (1979) Brief standard self-rating for phobic patients. Behau. Res. Ther. 17, 263-267. Mavissakalian M. and Michelson L. (1983)Agoraphobia: behavioral and pharmacological treatments: preliminary outcome and process findings. Psychopharmac. Bull. 19, 116-l 18.

698

CASE HISTORIES

AND

SHORTER

COMMUNICATIONS

Spielberger C. D., Gorsuch R. L. and Lushene R. E. (1970) Manual,for the Stute-Trait Anxic~ty Inwntor~. Consulting Psychologists Press, Palo Alto, Calif. Taylor J. A. (1953) A personality scale of manifest anxiety. J. ahnorm. sot. Psychol. 48, 285--290. Wolpe J. and Lang P. J. (1974) A fear survey schedule for use in behaviour therapy. Behav. Res. Thu. 2, 27-30. Zung W. W. K. (1965) A self-rating depression scale. Archs gen. Psychiut. 12, 63-70. Zung W. W. K. (1971) A ratmg instrument for anxiety disorders. Psychosomu~ics 12, 371-379.