Validation of the 16-item negative symptom assessment

J.p\gchral. I&., Vol. 21. No 3. ,I,, 2X-258, Prmled ,n Great Bnlam

VALIDATION

1993.

0022.3956/93 $6

on +

.“O

Pergamon Press Lul

OF THE 16-ITEM NEGATIVE ASSESSMENT

SYMPTOM

BRADLEYN. AXELROD,* ROBERT S. GOLDMAN? and LARRY D. ALPHS~ *Department of Veterans Affairs Medical Center, Medical Center, Ann Arbor, Michigan, U.S.A. $Department of Veterans Affairs Medical University,

Allen Park, Michigan, U.S.A.; TDepartment of Veterans Affairs and University of Michigan, Ann Arbor, Michigan, U.S.A.; Center, Allen Park, Michigan, U.S.A. and Wayne State Detroit, Michigan, U.S.A.

(Recriwd 24 February 1993; accqtrd

I I June 1993)

Summary-The dimensional structure of the 16.item Negative Symptom Assessment (NSA-16) was validated in a sample of 223 unmedicated schizophrenic inpatients and cross-validated on an independent sample of 276 patients with schizophrenia. Using a confirmatory factor analytic procedure, a five factor model was found to best characterize the structure of this rating instrument. These factors include: Communication, Emotion/Affect, Social Involvement, Motivation, and Retardation. The latent structure of the NSA-I6 is similar to the larger instrument from which it was derived. The findings provide support for a multidimensional model of negative symptoms in schizophrenia and offer a useful measure for their assessment.

Introduction THE NEGATIVESymptom Assessment (NSA) is a 25item rating scale designed to evaluate the presence, severity, and range of the negative symptoms often associated with schizophrenia. Strengths of the NSA, relative to other negative symptom rating instruments, lie in its standardized structured interview as well as its clearly defined and anchored items (Alphs, Summerfelt, Lann, & Muller, 1989). The NSA was also prospectively designed to provide an instrument that conceptualizes negative symptoms as multidimensional, in an effort to ultimately link dimensions of negative symptoms with aspects of pathophysiology and pharmacological efficacy. A validation study of the multidimensional construct of the NSA was accomplished through confirmatory factor analysis. Results revealed that the NSA is best characterized by six, distinct dimensions: Communication, Emotion/Affect, Social Involvement, Motivation, Gross Cognition, and Retardation (Axelrod, Goldman, Woodard, & Alphs, submitted). The NSA has demonstrated excellent test-retest (.93) and interrater (.85) reliabilities (Alphs et al., 1989) in schizophrenic populations. Furthermore, the NSA is a relatively easy instrument to teach to clinicians. Axelrod and Alphs (1993) demonstrated that individuals who had no prior experience with the NSA accurately assessed negative symptoms following a single 30-minute training session. The 25item, six factor NSA has proven to be a valid and reliable instrument, but more

*Author to whom correspondence should be addressed Affairs Medical Center, Allen Park, MI 48101, U.S.A. 253

at: Psychology

Service (1 l6B), Department

of Veterans

254

B. N. AXUXOII ct al.

widespread use of this scale by clinical researchers may be limited by the perceived length of the NSA. A 16-item, five factor version of the NSA was therefore developed to produce a more parsimonious rating scale without sacrificing either the psychometric properties of the measure or its clinical utility. Nine of the items were removed based on theoretical grounds. The three items comprising the Gross Cognition factor were removed because items on this factor (orientation, memory, abstraction) may be more completely assessed by other measures of cognitive function. Items with redundant face validity were removed from four of the remaining five factors, with the Motivation factor remaining untouched. The purpose of the present study was to systematically determine whether the multidimensional structure of the original (25item) NSA is preserved in the 16-item version. The integrity of the five factor model of the NSA-16 was evaluated in a sample of 223 schizophrenic inpatients and cross-validated in an independent sample of 276 schizophrenic inpatients. Method Subjects The initial sample was composed of 223 schizophrenic inpatients from 12 different psychiatric research settings (6 Veterans Administration Medical Centers; 6 State Hospitals). They were participants in an independent study of a novel psychiatric medication. All but three of the subjects were on single-blinded placebo medication at the time of the structured interview. The remaining three subjects were on anti-psychotic medication for less than 8 days. The sample averaged 40.0 (SD = 11.0) years of age and had an average illness onset of 22.2 years (,SD = 5.9). Subjects were interviewed 7.0 (SD = 1.4) days following hospitalization. Clinical ratings were performed by a total of 27 raters across the treatment sites. Procedure

The factor structure of the NSA-16 was evaluated using confirmatory factor analysis (Bentler, 1989). The five factor model of the NSA was hypothesized to best characterize the dimensional structure of the instrument. Since all factors are moderately intercorrelated, the models were created by progressively combining factors to examine whether fewer factors would result in a more parsimonious dimensional structure. The factors were combined on the basis of similarities in item content and their hypothesized relationship to clinical presentation. As shown in Table 1, Model 1 is the five factor NSA model derived from the NSA-25 (Axelrod et al., submitted). Model 2 is a four factor model which combines the Motivation and Retardation factors into a single factor, with the other factors remaining separate. Model 3 is a three factor model combining Communication and Social Tnvolvement into one factor, Motivation and Retardation into a second factor. and leaving Emotion/Affect as the third factor. A two factor model (Model 4) was created by combining Communication and Social Involvement into one factor with the combination of Emotion Affect, Motivation, and Retardation as the second factor. Model 5 tests the adequacy of a single factor model in which all items are constrained to load on one unitary factor. In the context of performing the confirmatory factor analyses, each of these five models was

VALIDATIONOFTHE

NSA-16

255

statistically compared with each other and to the null model. The null model assumes that each item represents its own factor, yielding a 16-factor model. The null model therefore assumes that a given instrument has no dimensional or latent structure. Table 1 Item Structure

for NSA-16 Models Items

Prolonged time to respond Restricted speech quantity Impoverished speech content Inarticulate speech Emotion: reduced range Affect: reduced modulation Affect: reduced display Reduced social drive Poor rapport with interviewer Reduced sexual interest Poor grooming and hygiene Reduced sense of purpose Reduced hobbies and interests Reduced daily activity Reduced expressive gestures Slow movements

Model

1

Mode1 2 1

1 1 I 2 2 2 3 3 3 4=5 4=5 4=5 4=5 4=5 4=5

Mode1 3

Reliability

Model 5

1=3 I=3 I=3 I=3 2=4=5 2=4=5 2=4=5 I=3 1=3 I=3 2=4=5 2=4=5 2=4=5 2=4=5 2=4=5 2=4=5

1=3 I=3 I=3 I=3 2 2 2 I=3 1=3 I=3 4=5 4=5 4=5 4=5 4=5 4=5

Note. The numbers correspond to the loading ofeach item on the factor. 3 = Social Involvement; 4 = Motivation; 5 = Retardation. Combined

Model 4

1 = Communication; factors

are designated

2 = Emotion/Affect; with an “=‘I sign.

of clinical ratings

An elaborate training procedure was employed to attain consistency in ratings across multiple sites. This procedure was implemented for both the validation and cross validation studies. The training consisted of a 30-minute lecture on the instrument by its creator (LA) to provide administration instructions, item descriptions, and scoring criteria. The training session was followed by the viewing and rating of two videotaped interviews. Using an experienced rater as the criterion, the raters achieved excellent agreement as indexed by minimal deviations from the criterion (see Axelrod & Alphs, 1993, for further description of this reliability procedure). Results Validation of the NSA-16 Table 2 presents the multiple indices evaluating the statistical goodness of fit of each of the models of the NSA-16. “Fit” is defined as the extent to which the covariance matrix of the data is best approximated by a given theoretical model, using maximum likelihood estimation procedures. A number of indices are available to describe the fit of a model (Bentler, 1989). In Table 2, the first row shows the ratio of chi-square to degrees of freedom for each of the models, as well as the null model. Lower ratios are indicative of models with better fit. As can be seen, the poor fit of the null model relative to all other models clearly

256

B. N. AXLLKOI> et al

indicates that the NSA has a defined latent structure. Among the tested models, factor model (Model I) has relatively better fit than the models with fewer factors. Table 2 Goodnrss

of‘

Fit

Indiic~rsfbr

Fuctor

Structure

Index Chi-square T degrees of freedom Bcntler-Bennett Normed Fit Index

Modd~

Model

I

of’~lw NSA-16

Model 2

in Sample

Model 3

the five

I In = 223)

Model 4

Model 5

2.16 .8X

4.57 .78

4.71 .I7

5.82 .71

5.91 .70

.x9 .92

.78 .82

.77 .x1

.70 .75

.70 .74

Null model 17.24

(NW Non-normed Comparative

Fit Index (NNFI) Fit Index (CFI)

Three other fit indices [Bentler-Bonnett Normed Fit Index (NFI), Non-normed Fit Index (NNFI), and Comparative Fit Index (CFI)] compute the fit function of a model directly against the null model (Bentler, 1989). Higher fit indices (> .90) signify superior model lit. The primary difference among the three goodness of fit indices is their sensitivity to degrees of freedom and sample size. The NFT does not include degrees of freedom in the calculation of the index, while the NNFI takes degrees of freedom into account for the theoretical and null models. The CFI better approximates model fit at most sample sizes (Bentler, 1989). In many cases, these three indices will be similar to each other. The present sample size is considered sufficient to appropriately apply these goodness of fit indices. Inspection of Table 2 demonstrates that Model 1 has superior goodness of fit on all indices, relative to the models composed of fewer factors. Since Model 2 had the next best goodness of fit indices, we compared the incremental chi-square differences to determine whether Model I represented a statistically superior model. The results of this comparison revealed Model I to have significantly better goodness of fit in comparison to Model 2 (X2 (4, II = 223) = 189, p < ,001). To determine whether the l6-item sample from a homogenous construct, coefficient Alpha was computed. Alpha coefficients of greater than .90 indicate high internal consistency. This sample yielded an Alpha coefficient of .92, demonstrating that the factors are composed of items which all tap the construct of negative symptoms.

A new sample of 276 schizophrenic inpatients from I9 sites (5 Veterans Administration Medical Centers, 10 state hospitals, 4 private hospitals) participating in a medication trial were evaluated while on single blind placebo to cross-validate the five factor model of the NSA- I6 in a new population. A total of 50 raters across the sites performed the clinical ratings. The sample averaged 36.4 (SD = 8.8) years of age and had an average illness onset of 21.8 years (SD = 6.2). They were interviewed 6.8 (SD = 1.6) days following hospitalization. These demographic data did not differ significantly from the validation sample. Table 3 shows the results of the confirmatory factor analyses for this sample. The same five models that were used in the original sample were tested in the cross-validation sample. The results again revealed that Model 1 is superior to the remaining four models as well as

VALIDATION OF THE NSA- 16

251

the null model, on the basis of the multiple indices of model fit depicted in Table 3. Direct statistical comparison of Model 1 to Model 2 again demonstrates the superior goodness of fit for the five factor model (X2 (4, n = 276) = 236, p < .OOl). A coefficient Alpha of .91 was found for the cross-validation sample, thereby demonstrating equally high internal consistency. Table 3 Goodness

of Fit Indices for Factor Structure Models of the NSA-I6 Index

Chi-square t degrees of freedom Bentler-Bonnett Normed Fit Index (NFI) Non-normed Fit Index (NNFI) Comparative Fit Index (CFI)

Model

1

Model 2

in Sample 2 (n = 276)

Model 3

Model 4

Model 5

2.76 .89

5.06 .80

5.40 .78

7.21 .69

7.83 .66

.91 .93

.I9 .83

.77 .81

.68 .72

.64 .69

Null model 20.20

Clinical comparability of the validation and cross-validation samples was evaluated by comparing mean scores for the five factors in Model 1. The two independent samples were found not to differ on any of the mean factor scores. These results indicate that the samples are similar in terms of severity of the negative symptom dimensions. When considering the findings of the cross validation sample, it should be noted that the NSA-I6 was derived from the interview for the larger NSA-25. The resulting NSA-16 data may have been influenced by this procedure. Discussion The present study used a confirmatory factor analytic methodology to validate the use of an abbreviated form of the NSA with schizophrenic patients. The need for confirmatory factor analysis in the validation of rating instruments that purport to measure dimensional aspects of psychopathology is critically important. All too often, clinical studies apply repeated exploratory factor analysis to test the dimensional structure of an instrument. While useful in the initial development, exploratory factor analysis is not designed to validate a given model. Exploratory factor analysis capitalizes on variability within a data set and provides no means by which to compare a theoretical model with obtained data. In contrast, confirmatory factor analysis imposes theoretically based statistical constraints on the data to validate previously derived models. The results of the present study clearly demonstrate that the NSA-16 is an instrument that is sensitive to the multidimensional aspects of negative symptoms in schizophrenia. The five factors of Communication, Social Involvement, Affect/Emotion, Motivation, and Retardation best describe the latent structure of this rating scale. These five factors were extracted from the larger six factor NSA-25. The NSA-16 retains the same structural integrity found in the original instrument (Axelrod et al., submitted). In fact, the CFI goodness of fit index for the NSA-16 (.92) is relatively greater than that for the NSA-25 (.87) when applied to the same sample of 223 subjects. The improvement of structural integrity seen in the NSA-16 suggests that the addition of the 9 items included in the NSA-25 do not contribute substantially to the multidimensional composition of this rating

258

B. N. AXELKOD

eta1

scale. While the Gross Cognition factor does not add statistically improvement to the model, the NSA-25 may be a preferable version of the NSA when an objective measure of cognition is not otherwise included in an evaluation of a patient. The confirmatory factor analytic methodology employed in the present study represents a useful method for investigating the purported dimensionality of a rating instrument. The utility of the NSA- 16 is dependent on further investigations that establish its dimensionality over the course of illness with populations of varied acuity. It is also necessary to progress beyond the isolated consideration of the adequacy of the NSA in assessing the dimensional structure of negative symptoms. Concurrent validity of the multidimensional nature of this negative symptom rating scale with instruments that are more widely used [e.g., Scale for Assessment of Negative Symptoms (SANS); Positive and Negative Symptoms (PANS)] should be evaluated since similar instruments may yield different results (Fenton & McGlashan, 1992). Further research with the NSA-16 also needs to establish the adequacy of interrater and test-retest reliabilities for this form of the NSA. References Alphs, L. D., Summerfelt, A., Lann, H.. & Muller, R. J. (1989). The Negative Symptom Assessment: A new instrument to assess negative symptoms in schizophrenia. Psychopharmacology Bulletin, 25, 159 163. Axelrod, B. N., & Alphs, L. D. (1993). Training novice raters on the Negative Symptom Assessment scale. Schizophrenia

Research,

9,25-28.

B. N., Goldman, R. S., Woodard, J. L., & Alphs. L. D. (1993). Factor structure of’rhe Ne,gatioe Svmplom Asse.s.vmml. Manuscript submitted for publication. Bcntler, P. M. (1989). E&S: Struclural Equations Program Manual. Los Angeles: BMDP Statistical Software, Inc. Fenton, W. S., & McGlashan, T. H. (1992). Testing systems for assessment of negative symptoms in schizophrenia.

Axelrod,

Archives

qf General

Psychiatry,

49, 119

184.