An empirical method to refine personality disorder classification using stepwise logistic regression modeling to develop diagnostic criteria and thresholds

Comprehensive

Psychiatry

(Official Journal of the American Psychopathological

VOL. 35, NO. 6

Association)

NOVEMBER/DECEMBER

An Empirical Method to Refine Personality Disorder Classification Using Stepwise Logistic Regression Modeling Develop Diagnostic Criteria and Thresholds

1994

to

H. George Nurnberg, Glenn A. Martin, and Simcha Pollack This study of DSM-III-R personality disorder (PD) classification provides an empirical approach to determine (1) the discriminative power of each criterion and (2) the optimal number of criteria needed to diagnose the presence of each PD. A semistructured assessment of 110 outpatients was performed for the 11 PDs and their 104 diagnostic criteria. Sensitivity, specificity, and predictive powers were calculated for each criterion item. Logistic regression was performed to determine (1) univariate weightings of the individual criteria as applied to a given diagnosis, and (2) multivariate measures of the criteria that significantly improved the chi-square value in a stepwise fashion. The significant

T

HE DSM-III-R prototypal system of personahty disorder (PD) classification needs further refinement. It appears that the polythetic criteria sets are not diagnosing distinct PD entities, and 13 disjunctive categories with 120 criteria are too many.’ Proponents of smaller groupings present the extensive overlap between categories as evidence of excessive heterogeneity, which is compounded by the descriptive overlap of the same or similar items being present in more than one diagnosis.2-4 Most studies using semistructured interviews for systematic evaluation of PDs find that more than 50% of patients have two or more coexisting PD diagnosis.5-9Paradoxically, clinicians rarely make multiple PD diagnoses in psychiatric practice.‘O Reliability suffers because a larger number of PD categories require finer distinctions, and many of the diagnostic criteria that are maladaptive traits can only be determined quantitatively as crossing from normal into disorder. The problems in diagnostic accuracy and excessive overlap can be improved with empirical data that weight the individual PD criteria items in

items were then equally weighted to calculate the optimal number needed to diagnose category membership. Of 104 PD criteria, 41 discriminated at a significance level of .05 or less, and each PD could be optimally diagnosed with fewer criteria than currently required. We can empirically reduce the number of criteria combinations comprising individual categories, decrease heterogeneity, and narrow diagnostic boundaries. This increases the likelihood of identifying etiological factors, predictors of clinical course, specific treatments, familial aggregation, and neurobiological correlates for the PD taxa. Copyright 0 1994 by W.B. Saunders Company

diagnostic importance and examine alternative cutoff p0ints.l’ Research efforts into the reliability and clinical description of PDs have been sufficiently productive to justify proceeding to the next level of inquiry into the contents and limits of these constructs. These investigations show that PDs can, to varying degrees, be diagnosed with acceptable reliability by clinical instruments,7JJ2 remain fairly stable over time,l3-15 coexist frequently with axis I disorders to affect outcome,16-18demonstrate disturbances in biological test measures,19-23and respond differentially to psychopharmacological agents.2”-27A number of studies have demonstrated that given differ-

From the Departments of Psychiatty, University of New Mexico School of Medicine, Albuquerque, NM; Albert Einstein College of Medicine, New York; and St. Johns University, New York, NY. Address reprint requests to H. George Numberg, M.D., Medical Director, Department of Psychiatry, UNM Medical School, 2600 Marble Ave NE, Albuquerque, NM 87131. Copyright 0 1994 by WE. Saunders Company OOIO-440X194/3506-0007$03.0010

COmPrehenSive Psychiatry, Vol. 35, No. 6 (November/December). 1994: pp 409-419

409

410

NURNBERG, MARTIN, AND POLLACK

em symptoms, the conditional probabilities of a disorder vary considerably in diagnostic efficiency.8,28-31This suggests that diagnostic precision can be improved by algorithms that assign more importance to some criteria than others. Nurnberg et a1.32showed that a weighted set of six of the eight borderline PD criteria were significantly discriminating, and a less than five of eight algorithm could distinguish such patients at the optimal level of diagnostic efficiencyconsiderations. Therefore, although there are a number of features associated with the various PD diagnoses, what remains to be determined is which criteria represent the core or essential features of the condition and how many of these criteria are needed to make an efficient diagnosis. The present effort seeks to advance the search for more efficient and parsimonious ways to use polythetic criteria in PD diagnosis with the following questions: (1) What are the discriminating and nondiscriminating features of the various PDs? (2) Can the individual criteria be identified according to their diagnostic discriminative power? (3) What are the minimum number of criteria necessary for optimal diagnostic efficiency for the various PDs? The availability of analytic data-reduction techniques that address these questions offers an improved systematic methodology based on empirical data to replace the implicitly defined committee approach of DSM-III.

and V code (5%) disorders (multiple axis I diagnoses total > 100%). Axis II diagnostic criteria ratings were recorded on an inventory checklist containing each of the 104 criteria for each of the current 11 PDs by experienced clinicians with extensive training in the use of the instruments and diagnosis of PDs. Every item was scored as explicitly present or absent on the basis of being characteristic of the person’s recent (past year) and long-term functioning since early adulthood after a sufficient period of treatment (intended to minimize the effects of acute-state conditions on personality measurement), and used all the information available from multiple sources (e.g., patient, chart, significant others, former clinicians). PD diagnosis was rated as present if the minimal required criteria scored at threshold. Multiple diagnoses were recorded. This method for a comprehensive longitudinal clinical assessment provides an accurate and reliable database for making diagnoses.33,34 The Structured Interview for DSM-III Personality Disorders35 was used to establish a measure of reliability for the ratings of both the PD criteria and diagnoses, and was administered by an independent rater trained in the use of the instrument and blind to the diagnoses. Technically, this comparison between instruments provides a measure of concurrent validity rather than interrater reliability. The calculation of diagnostic consistency across instrument measures was a necessary compromise solution, because a second independent clinical inventory checklist rating for each patient was not feasible in this design with patients who were so well known to those who initially rated them. The overall K values, determined by summing all individual criteria (dimensional measure) and diagnoses (categorical measure), were 0.62 for all 104 PD criteria and 0.64 for all 11 PD diagnoses, respectively. The K values for the sum of the criteria for individual PD categories ranged from .35 to .82 (table 1). These values are comparable to those of other studies addressing questions of axis II interrelationships.‘,s

Methods of Data Analysis METHOD The study subjects providing the database for analysis of the PD criteria were 110 outpatients without concurrent “major” DSM-III-R axis I diagnoses. Brief reactive psychosis was included in this group because it is transient, usually secondary to a preexisting PD, and requires a rule-out of the more pervasive disorders considered in the major axis I grouping. Exclusionary criteria were diagnosis of scbizophrenia, schizoaffective disorder, major affective disorder, delusional disorder, organic disorder, mental retardation, and alcohol or drug dependence. Further details regarding recruitment, diagnosis, and procedure are presented elsewhere.32 The age of the subjects ranged from 18 to 60 years, with a mean of 35 and a standard deviation of 9 years. There were 49 (45%) men and 61 (55%) women. All assigned axis I diagnoses were confirmed to meet DSM-III-R criteria and consisted of anxiety (56%), non-major mood spectrum (21%), adjustment (lo%), somatoform (6%), sexual (3%). impulse (1%) eating (1%) brief reactive psychosis (1%)

Determinations of individual category prevalences, the mean number of criteria, and sensitivity, specificity, positive predictive power (PPP), and negative predictive power (NPP) for the individual criteria items in each PD category were performed. Sensitivity, specificity, PPP, and NPP values are particularly useful to examine the internal consistency of the individual PD criterion items.s In the absence of an external validator, this becomes a “boot strap” procedure in which the criteria as a whole are assumed to approximate a real diagnostic entity with patients identified as cases or noncases according to the set taken as a whole. These measures can identify which criteria enhance discrimination from other disorders, which are not strongly related to the syndrome, and how well the components of the criteria set cluster together. To address the overlap or intercorrelation among the criteria measures, a multivariate method was chosen as a general class of modeling technique. This allows an opportunity for measuring interaction among the criteria. Using a parsimonious approach to multivariate regression model-

DIAGNOSTIC CRITERIA AND THRESHOLD FOR PD

411

Table 1. Descriptive and Reliability Statistics for Each DSM-III-R PD (N = 110) No. of Symptoms

Base

% Without

n

Rate

Other PD

K

14

0.13

14

0.78

7

1.15 -c 1.71

SW

4

0.04

50

0.82

7

0.64 + 1.11

STYP

3

0.03

0

0.75

9

0.75 * 1.22

HIST

19

0.17

21

0.47

8

1.38 2 1.95

NARC

18

0.16

11

0.60

9

1.97 f 2.34

ASOC

2

0.02

0

0.61

22

0.82 + 2.10

PD

PAR

Maximum

Mean T SD

BORD

22

0.20

18

0.72

8

2.33 + 2.13

AVOD

26

0.24

31

0.59

7

2.26 +- 1.78

DEP

2.45 + 1.81

18

0.16

24

0.35

9

COMP

6

0.05

17

0.66

9

1.48 2 1.68

PAGG

8

0.07

0

0.43

9

1.38 + 1.93

Abbreviations:

PAR, paranoid PD; SCIZ, schizoid PD; STYP, schizotypal PD; HIST, histrionic PD: NARC, narcissistic PD; ASOC,

antisocial PD; BORD, borderline PD; AVOD, avoidant PD; DEP, dependent PD; COMP, compulsive PD; PAGG, passive-aggressive PD.

ing, a stepwise logistic regression based on a log-likelihood chi-square improvement in the goodness-of-fit test was performed using the Statistical Package for the Social Sciences program. 36 Logistic regression was chosen over discriminant function analysis (and its equivalent, multiple regression) because less assumptions are necessary. For example, the logistic regression model does not require the error term to be normally distributed. The dependent dichotomous variables were the various PD categories, and their corresponding diagnostic criteria provided the independent variables. This technique provides a univariate weighting of the individual criteria as they apply to their corresponding PD diagnosis and a multivariate measure of the criteria that improve chi-square analysis in a stepwise fashion. The total score value (Rao’s efficient score statistic3’), chi-square coefficients for improved fit, and significance of the variable statistics are calculated, as well as a table of misclassification rates. For a variable to enter and remain in the model, the associated chi-square value. before and after entry had to be significant at a minimum .05 level. It is recognized that circularity in data may undermine or constrain analysis. Therefore, where most studies use logistic regression as the final step, we use it as a screen for a data-reduction scheme through which the diagnostic efficiency of significant criteria combinations can be further determined. The optimal diagnostic threshold is determined by the diagnostic error method of Meehl and Rosen.38 This measure quantifies the choice of an optimal cutting score to diagnose category membership. The best criteria items as identified by logistic regression are used equally weighted because differential weighting is impractical for clinical usage, has only a trivial effect on increasing associations with external variables, and more importantly, represents a conceptual logical fallacy. 39 The total rate of diagnostic error for criteria combinations is determined by multiplying the false-positive rate by 1 minus the base rate, multiplying the false-negative rate by the base rate, and then summing the two products for each combination cutoff point. Sensitivity, specificity, PPP, and NPP are also calculated for the criteria combinations based on the proportion of individuals in each group for whom the items were rated present. The

optimal measure

diagnostic threshold of diagnostic error.

is determined

by the

best

RESULTS

Sixty-eight (62%) patients had at least one axis II PD diagnosis. The 42 (38%) patients who did not achieve a specific PD diagnosis also demonstrated a substantial number of PD items. Table 1 presents the base rate, K value, and mean number of symptoms for each of the PDs. Although some of the disorders were infrequent, a substantial proportion of each disorder’s symptoms were often present. The dimensional aspect of PD symptoms is indicated by the mean number of symptoms recorded for each PD in the study sample. The most prevalent PD diagnoses were avoidant, borderline, histrionic, narcissistic, dependent, self-defeating, and paranoid (24% to 13%). The high prevalence of avoidant PD can be expected in this study sample with the substantial base rate (56%) of panic and other anxiety disorders.40 Antisocial, schizotypal, schizoid, obsessivecompulsive, and passive-aggressive PDs were less frequent (2% to 7%). PD was most often not diagnosed alone, with multiple axis II diagnoses present in 37 (54%) PD patients (mean, 2.3 per patient). There were a total of 154 PD diagnoses among the 68 patients with PD. Table 2 shows the covariation or prevalence of overlap among the PD categories. The intercorrelation matrix of the axis II scores demonstrated extensive (n = 31 of 55) significant correlations, with none more than 0.80 (Table 3).

NURNSERG,

412

Table 2. Comorbidity of DSM-III-R PDs in Outpatients

MARTIN,

AND

(N = 110): Prevalence of Overlap by Presence/Absence

POLLACK

of

Co-occurring PD Category PD

%

PAR

.I3

Xl2

.04

SW

STYP

-

.07/.97

.071.98

.50/.88

.25/.88

-

.00/.97

.25/.83

PAR

HIST

NARC

AVOD

DEP

COMP

PAGG

ASOC

BORD

.59/.89

.00/.98

.64/.86

.21/.76

.21/.84

.21 I.97

.36/.97

.00/.83

.00/.98

.25/.80

.50/.77

.00/.83

.251.95

.25/.93

STYP

.03

.33/.88

.00/.96

-

.00/.82

.00/.83

.00/.98

.671.81

.33/.77

.00/.83

.00/.94

.00/.93

HIST

.I7

.37/.92

.05/.97

.00/.97

-

.63/.93

.50/.99

.42/.85

.211.76

.261.86

.21/.98

.321.98

NARC

.16

.391.92

.00/.96

.00/.97

.671.92

-

.06/.99

.56/.87

.22/.76

.331.87

J7l.97

.33/.98

ASOC

.02

.00/.87

.00/.95

.00/.97

.50/.83

.50/.84

-

.00/.80

.00/.76

.00/.83

.00/.94

.00/.93 .27/.98

BORD

.20

.41/.94

.05/.97

.09/.99

.36/.88

.45/.91

.00/.98

-

.321.78

.41/.90

.18/.98

AVOD

.24

.12/.87

.08/.98

.04/.98

.15/.82

.15/.83

.00/.98

.271.82

-

.38/.90

.151.98

.12/.94

DEP

.I6

.17/.88

.00/.96

.00/.97

.28/.85

.33/.87

.00/.98

.50/.86

.56/.83

-

.17/.97

.22/.96

COMP

.05

.50/.89

.171.97

.00/.97

.671.86

.50/.86

.00/.98

.671.83

.67/.79

.50/.86

-

.50/.95

PAGG

.07

.63/.91

.131.97

.00/.97

.75/.87

.?51.88

.00/.98

.75/.84

.38/.77

.50/.86

.38/.97

-

NOTE.

See Table

of nonparanoid

1 for PD abbreviations.

subjects

The table should

be read as follows:

did not have SORD PD; 41% of subjects

64% of individuals

with PAR PD also had SORD PD; 86%

with BORD PD also had PAR PD; 94% of nonborderline

PD subjects

did

not have PAR PD.

Table 4 presents for each PD category the overall K value (total criteria in category), sensitivity, specificity, PPP, NPP, and total score value for the logistic regression that ranks the individual criteria in univariate fashion. The last three columns show the chi-square results of the multivariate stepwise logistic regression for improvement of fit, its significance, and incremental classification accuracy for the criteria that were entered for a threshold set at a level not exceeding .05. This level of significance was chosen to minimize the filtration of criteria. It is stressed that the meaningfulness of determinations dor sensitivities, predictive powers, and diagnostic efficiency of PDs where 5% or less of the 110 patients met criteria for these diagnoses (schizoid, schizotypal, antisocial, compulsive) must be considered extra cautiously. However, the specificity values can still have Table 3. Intercorrelation PD

PAR

PAR

-

SCIZ

SCIZ

STYP

.0088 -

.3681*

HIST

.4233*

.4092*

-.I177

STYP

.0166

HIST NARC ASOC BORD AVOD DEP COMP PAGG NOTE.

Abbreviations

‘P < .Ol. tP < .05.

are as in Table

1.

NARC

.6150* - .0903

precision, since all patients who do not have the PD diagnosis figure into this calculation. From a total of 104 PD criteria for 11 DSMIII-R categories, 24 achieved significance at levels of .OOl or less, 34 at .Ol or less, and 41 at .05 or less. The number of criteria discriminating for the individual PD categories at a level of .05 or less are as follows: paranoid, four; schizoid, one; schizotypal, two; histrionic, five; narcissistic, three; borderline, six; antisocial, one; avoidant, six; dependent, six, compulsive, three; and passive-aggressive, four. Using the best criteria as identified by the logistic regression, the diagnostic efficiency rate for combinations of the significantly discriminating identified criteria is given next and identifies the most efficient cutoff point. These combinations give the approximate optimal values for considerations of sensitivity, specificity, PPP,

Matrix of DSM-III-R PDs (N = 110) ASOC

.3720* -.0431

BORD

AVOD

.5633*

.0990

.0121

.3472’

DEP

.1599 -.0375

COMP

PAGG

.2761 t

.6275*

.3519*

.0160 .2421

.1672

.3004t

.2573t

.4355*

.1164

.2686t

.7643*

.3203*

.4006*

.0262

.2758t

.1867

.6286*

.4687*

.5785*

.1148

.3529*

.3012t

.7028*

.4241* -

- .0004

.I522

.1068

.2921t

.4268*

.I765

.5428*

.1928 -

.4387* -

.3346*

.1863

.0977 -

.3335* .3692* -

DIAGNOSTIC CRITERIA AND THRESHOLD FOR PD

413

Table 4. Logistic Regression Analysis of All DSM-III-R PD Criteria and Diagnostic Efftciency of Discriminating Criteria Combinations Significance

Cluster A Paranoid PD (prevalence, 0.13; K, 0.78) Unjustifably questions loyalty Carries grudges or is unforgiving Expects exploitation or harm Easily slighted, quick to counterattack Reluctance to confide in others Unjustified fidelity concerns Reads hidden threatening meanings

SENS

SPEC

PPP

NPP

SCOW

X’

IPI

0.93 1.00 0.79 0.79 0.71 0.36 0.21

0.97 0.83 0.92 0.85 0.88 0.96 0.98

0.81 0.47 0.58 0.44 0.45 0.56 0.60

0.99 1.oo 0.97 0.96 0.95 0.91 0.90

79.15 42.78 42.18 28.49 26.52 16.19 10.54

57.34 11.83 5.18 5.68

< ,001

FN

FP

TER

1.00 1.oo 1.oo 0.50

0.75 0.84 0.98 1.oo

0.37 0.48 0.88 1.oo

1.00 1.00 1.oo 0.93

Go 0.00 0.00 0.50

035 0.16 0.02 0.00

K2 0.14 0.02 0.06

1.oo 1.oo 0.75 0.75 0.75 0.25 0.25

0.97 0.93 0.94 0.93 0.79 0.97 0.97

0.57 0.36 0.33 0.30 0.12 0.25 0.25

1.oo 1.oo 0.99 0.99 0.99 0.97 0.97

61.08 37.36 24.67 21.82 6.46 5.41 5.41

24.81

< ,001 NS NS NS NS NS NS

FN

FP

1.00

0.97

0.57

1.oo

GO

0<3

TER 0.03

0.67 0.33 1.00 0.67 0.67 0.33 1.oo 0.67 0.67

1.oo 0.99 0.98 0.99 0.98 0.99 0.87 0.79 0.80

1.00

0.50 0.60 0.67 0.50 0.50 0.18 0.08 0.09

0.99 0.98 1.oo 0.99 0.99 0.98 1.00 0.99 0.99

72.66 17.17 64.77 47.53 34.97 17.17 16.87 3.39 3.91

16.17 8.58

< ,001 ,003 NS NS NS NS NS NS NS

FN

FP

TER 0.01 0.03

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 4

Schizoid PD (prevalence, 0.04; K, 0.82) Nearly always chooses solitary activities Little desire for sex with others Claims to rarely experience anger/joy Constricted affect At most, one close friend Indifferent to praise and criticism Little desire for close relationships Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 Schizotypal PD (prevalence, 0.03; K, 0.75) Unusual perceptual experiences Odd speech Ideas of reference Odd beliefs or magical thinking Odd, eccentric behavior or appearance Inappropriate or constricted affect Suspiciousness or paranoid ideation At most, one close friend Excessive social anxiety Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 Cluster B Borderline PD (prevalence, 0.2; K, 0.72) Unstable and intense relationships Chronic emptiness or boredom Self-damaging impulsivity Inappropriate or uncontrolled anger Fear of abandonment Marked chronic identity disturbance Recurrent suicidal threats or behavior Affective instability

1.00 0.00

0.99 1.oo

0.75 IND

1.oo 0.97

Z.00 1.oo

fil 0.00

0.91 0.77 0.73 0.68 0.50 0.82 0.32 1.00

0.88 0.77 0.86 0.85 0.86 0.73 0.99 0.58

0.65 0.46 0.57 0.54 0.48 0.43 0.88 0.37

0.97 0.93 0.93 0.91 0.87 0.94 0.85 1.oo

53.46 23.46 32.39 26.46 14.07 22.18 24.57 23.77

51.11 14.66 16.15 7.18 6.64 8.82

FN

FP

TER

1.oo 1.00

0.41 0.64 0.91 1.oo 1.oo 1.oo

0.30 0.41 0.73 1.00 1.oo 1.oo

1.oo 1.00 1.oo 0.97 0.87 0.82

Z.00 0.00 0.00 0.14 0.59 0.86

rz9 0.36 0.09 0.00 0.00 0.00

0.47 0.29 0.07 0.03 0.12 0.17

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 4 5 6

,001 ,023 ,017 NS NS NS

1

.oo

0.86 0.41 0.14

< .OOl < ,001 < ,001 ,007 ,010 ,003

Classification

96.36 98.18 98.18 99.09

97.27

99.09 99.09

88.18 93.64 95.45 97.27 97.27 98.18

414

NURNBERG, MARTIN, AND POLLACK

Table 4. Logistic Regression Analysis of All DSM-III-R PD Criteria and Diagnostic Efficiency of Discriminating Criteria Combinations (Cont’d) Significance

Histrionic PD (prevalence, 0.17; K, 0.47) Excessively impressionistic speech Self-centered; no frustration tolerance Constantly seeks approval or praise Exaggerated expression of emotions Overly concerned with appearance Needs to be center of attention Shallow, shifting emotional expression Inappropriately sexually seductive

SENS

SPEC

PPP

NPP

SCOW

0.84 0.79 0.95 0.68 0.68 0.47 0.32 0.37

0.90 0.89 0.84 0.96 0.93 0.97 0.96 0.96

0.64 0.60 0.55 0.76 0.68 0.75 0.60 0.64

0.96 0.95 0.99 0.94 0.93 0.90 0.87 0.88

49.44 41.34 45.83 49.32 42.05 31.42 14.06 18.39

FN

FP 073 0.14 0.01 0.00 0.00

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 4 5 Narcissistic PD (prevalence, 0.16; K, 0.6) Entitlement Interpersonal exploitativeness Needs constant attention and admiration Grandiose sense of self-importance Preoccupied with grandiose fantasies Preoccupied with feelings of envy “Unique” problems Reacts to criticism with rage or shame Lack of empathy

(P)

i ,001 < .OOOl < ,001

,003 ,023

0.67 0.86 0.99 1.00 1.oo

0.39 0.59 0.95 1 .oo 1.oo

1.00 1.oo 0.99 0.95 0.87

0.94 0.72

0.93 0.92

0.74 0.65

0.99 0.94

70.37 42.25

60.72 16.14

< .OOl < ,001

0.67 0.89 0.83 0.67 0.50 1.oo 0.22

0.95 0.92 0.90 0.84 0.91 0.54 0.99

0.71 0.70 0.63 0.44 0.53 0.30 0.80

0.94 0.98 0.97 0.93 0.90 1.oo 0.87

43.20 60.14 47.74 20.62 19.66 17.93 15.51

12.19

,005 NS NS NS NS NS NS

FN

FP

1 .oo

0.80 1.oo 1.oo

0.50 1.00 1.oo

1.oo 0.98 0.90

600 0.11 0.56

OYO 0.00 0.00

0.96 0.96 0.93 0.98 0.97 0.96 0.95 0.95 0.94 0.93 0.92 0.94 0.95 0.97 0.99 0.99 0.99 1.oo 1.oo 1.oo 1.oo

0.33 0.33 0.20 0.33 0.25 0.20 0.17 0.17 0.14 0.11 0.10 0.00 0.00 0.00 0.00 0.00 0.00 IND IND IND IND

1.00 1.oo 1 .oo 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98

35.31 35.91 20.37 17.16 12.50 9.70 7.84 7.84 6.51 4.74 4.13 0.14 0.10 0.06 0.02 0.02 0.02 0.00 0.00 0.00 0.00

12.354

FN

FP

0.96

0.33

1.oo

60

OX4

0.89 0.44 1.00 1.oo 1.oo 0.50 0.50 0.50 0.50

1

.oo

Classification

89.09 92.73 93.64 97.27 98.18

TER 0.27 0.12 0.02 0.05 0.13

GO 0.00 0.05 0.26 0.74

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1

42.61 21.60 16.44 8.86 5.14

-

1.00 1.oo 0.95 0.74 0.26

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 Antisocial PD (prevalence, 0.02; K, 0.61) Repeatedly performs unlawful acts Disregard for the truth Regularly uses drugs before age 15 No monogamous relationships > 1 yr Repeatedly violates rules before age 15 Unusually early sexual intercourse Failure to honor debts Often truant before age 15 Lack of remorse Recklessness Often lied before age 15 lnconsistant work behavior > 1 expulsion or suspension before 15 Failure to plan ahead or impulsivity Irresponsible parenting Arrested before age 15 Running away at least twice before 15 Repeated stealing before age 15 Repeated vandalism before age 15 Repeated initiation of fights before 15 Repeated fights or assaults

X2

93.64 94.55 98.18

TER 0.16 0.02 0.09

< ,001 NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS

TER 0.04

98.18

DIAGNOSTIC CRITERIA AND THRESHOLD FOR PD

415

Table 4. Logistic Regression Analysis of All DSM-III-R PD Criteria and Diagnostic Efficiency of Discriminating Criteria Combinations (Cont’d) Significance

Cluster C Avoidant PD (prevalence, 0.24; K, 0.59) Avoids interpersonal activities Unwilling to get involved Afraid of appearing foolish Feelings of easily hurt by criticism Fears being embarrassed by anxiety Exaggerates potential risks No close friends outside of family

SPEC

PPP

NPP

Scare

YZ

0.58 0.81 1.00 0.92 0.77 0.23 0.62

0.99 0.88 0.69 0.40 0.77 0.96 0.86

0.94 0.68 0.50 0.32 0.51 0.67 0.57

0.88 0.94 1.oo 0.94 0.92 0.80 0.88

50.99 46.52 37.98 9.69 25.59 10.06 23.36

44.97 25.89 20.44 8.94 6.12 5.44

FN

FP

TER

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 4 5 6 Dependent PD (prevalence, 0.16; K, 0.35) Preoccupied by fear of ahandonment Allows others to make most decisions Indecisive w/o advice or reassurance Agrees with people even if wrong Devasted when relationship ends Dislikes/avoids being alone Difficulty initiating projects Easily hurt by criticism or disapproval Volunteers so as to be liked

1.00

0.21

1.oo 1.oo 0.92 0.31 0.08

0.62 0.87 1.oo 1.00 1.oo

1.oo 1.00 1.oo 0.98 0.82 0.78

0.00 0.00 0.00 0.08 0.69 0.92

69 0.38 0.13 0.00 0.00 0.00

0.60 0.29 0.10 0.02 0.16 0.22

0.89 0.50 1.oo 0.44 0.56 0.67 0.39 1.oo 0.33

0.84 0.91 0.78 0.85 0.80 0.86 0.90 0.33 0.88

0.52 0.53 0.39 0.36 0.36 0.48 0.44 0.23 0.35

0.97 0.90 0.94 0.89 0.90 0.93 0.88 1.00 0.87

39.19 19.66 18.27 8.04 10.28 23.66 10.26 8.07 5.27

36.45 19.83 7.93 9.29 11.68 5.42

< .OOl < ,001 ,005 ,002 ,001 ,020 NS NS NS

FN

FP

TER

1.00 1.oo 0.83 0.72 0.28 0.06

0.40 0.72 0.88 0.97 1.oo 1.oo

0.25 0.41 0.58 0.81 1.00 1.00

1.oo 1.00 0.96 0.95 0.88 0.84

0.00 0.00 0.17 0.28 0.72 0.94

o.sO 0.28 0.12 0.03 0.00 0.00

G 0.24 0.13 0.07 0.12 0.15

0.83 1.00 0.83 0.67 0.83 0.67 0.67 0.67 0.00

0.94 0.92 0.88 0.88 0.80 0.83 0.83 0.72 0.98

0.45 0.33 0.28 0.24 0.19 0.18 0.18 0.12 0.00

0.99 0.98 0.99 0.98 0.99 0.98 0.98 0.97 0.94

37.92 20.30 20.80 12.74 12.53 8.64 8.64 4.06 0.12

20.23 6.68 5.80

FN

FP

TER

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 Passive-aggressive PD (prevalence, 0.07; K, 0.43) Resents useful suggestions Unjustifiably protects demands made Unrealistic self-evaluation

< ,001 < ,001 < ,001 ,003 ,013 ,020 NS

0.28 0.45 0.70 1.oo 1.oo 1.oo

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 1 2 3 4 5 6 Obsessive-compulsive PD (prevalence, 0.05; K, 0.66) Preoccupation with details/rules Overconscientious and inflexibility Stubborn insistence on “his way” Lack of generosity Indecisiveness Excessive devotion to work Perfectionism hinders task completion Restricted expression of affection Unable to discard worthless things

IPI

SENS

1 .oo

1.00 0.33

0.75 0.88 1 .oo

<.OOl ,010 ,016 NS NS NS NS NS NS

0.81 0.93 1.oo

0.23 0.46 1.00

1.00 1.oo 0.96

0.00 0.00 0.67

fi9 0.07 0.00

9% 0.06 0.04

0.93 0.87 0.84

0.46 0.35 0.33

0.98 0.99 1.oo

33.05 27.87 30.91

19.91 14.18 11.51

< ,001 < ,001 ,001

Classification

89.09 89.09 95.45 97.27 98.18 98.18

84.55 90.00 92.73 95.45 98.18 99.09

94.55 96.36 96.36

92.73 96.36 97.27

NURNBERG, MARTIN, AND POLLACK

416

Table 4. Logistic Regression Analysis of All DSM-III-R PD Criteria and Diagnostic Efficiency of Discriminating Criteria Combinations (Cont’d) Significance

Passive-aggressive PD (prevalence, 0.07; K, 0.43) (Cont’d) Procrastination Unreasonably scornful of authority Deliberately poor or slow work Doesn’t do his share of work Sulky or irritable when asked to work “Forgets” obligations

SENS

SPEC

PPP

NPP

SCOW

X2

0.88 1.oo 0.38 0.25 1.00 0.38

0.87 0.84 0.99 1.oo 0.73 0.94

0.35 0.33 0.75 1.oo 0.22 0.33

0.99 1.oo 0.95 0.94 1.00 0.95

27.87 30.91 28.24 25.98 17.73 9.87

7.24

FN

FP

1.oo 1.oo 0.88 0.63

0.68 0.85 0.99 1.oo

0.20 0.35 0.88 1.oo

1.oo 1.oo 0.99 0.97

zo 0.00 0.13 0.38

0.32 0.15 0.01 0.00

Diagnostic efficiency of discriminating criteria combinations No. of criteria met 2 3 4

(PI

,007 NS NS NS NS NS

Classification

99.09

TER 0% 0.14 0.02 0.03

Abbreviations: SENS, sensitivity; SPEC, specificity; Score, univariate total score; Classification, classification accuracy of stepwise logistic regression; IND, indeterminable; FN , false-negative rate; FP , false-positive rate; TER , total error rate; NS, not significant.

and NPP at the best level of diagnostic error. This assumes that the relative utilities of type I (false-negative) and type II (false-positive) errors are equal. Particular clinical utilities can prevail where the costs of these errors are not equal: e.g., a research design requiring greater sample homogeneity and reduction of falsepositives. In that situation, the cutoff point could be increased to reduce false-positives and increase PPP. The optimal minimum cutoff points using the Meehl and Rosen method are as follows: paranoid, at least three of four; schizoid, one of one; schizotypal, two of two; histrionic, at least four of six; narcissistic, at least two of three; borderline, at least four of six; antisocial, one of one; avoidant, at least four of six, dependent, at least four of six; compulsive, at least two of three; and passive-aggressive, at least three of four. For comparison, the current DSM-III-R algorithms are at least four of seven, four of seven, five of nine, four of eight, five of nine, five of eight, three of 12 plus four of 10, four of seven, five of nine, five of nine, and five of nine, respectively. Such a reduction in diagnostic thresholds reduces the potential number of combinations for each PD category as follows: paranoid, from 64 to five; histrionic, 163 to 16; narcissistic, 255 to four; borderline, 93 to 21; avoidant, 64 to 22; dependent, 255 to 22; compulsive, 255 to four; and passive-aggressive, 255 to five. Low prevalences ( < .05) of schizoid, schizotypal, and anti-

social PDs give less confidence to a reduction to one combination. DISCUSSION

This study shows that among the criteria sets for the individual DSM-III-R PDs, some criteria are more discriminating or possibly more prototypical of the expression of the disorders than others. Their combinations have a greater probability of producing an accurate diagnosis to the extent that they are individually distinctive and occur regularly in those individuals with the disorder and not in those without it.26,41 It also appears that less than the prescribed number of DSM-III-R criteria are necessary to make a particular PD diagnosis with confidence.3Z The additional criteria may reflect additional enduring personality characteristics shared with other PDs. Although shared features can be expected to arise with various longstanding characterological disturbances without precluding distinct syndromes,42 it seems that the inclusion of many nondiscriminating and shared criteria in the criteria sets contributes to the extensive overlap and heterogeneity among PD diagnoses. Although the analysis suggests qualified support for some DSM-III-R diagnoses with methods that may be more descriptively parsimonious, what is derived can only serve a heuristic purpose until a predictive relationship to other clinical domains is established.

DIAGNOSTIC

CRITERIA AND THRESHOLD FOR PD

It must be underscored that the identified significant criteria and cutoff points cannot be taken as definitional. Rather, this investigation represents an additional empirical contribution to the study of the descriptive validity of DSMIII-R axis II criteria sets when applied to patients in a clinical setting. Review of the literature shows a surprising lack of studies with data on the individual diagnostic contributions of all the individual DSM-III-R criteria for all the PDs. To obtain a larger database for the DSM-IV axis II work group, additional data were solicited from a number of unpublished data sets.’ This appears to be the only study that also empirically examines diagnostic criteria thresholds for DSM-III-R PD diagnoses. Although there is extensive agreement over various sets of PD features, exact clusters of symptoms vary among the studies. Substantial variation among findings is to be expected, and is affected by factors of sample size, comparison groups, base rates, ascertainment bias, circularity, diagnostic instruments, reliability, precision of the statistical measures, and capitalization on chance. Given the methodological limitations to comparisons between studies and generalizability, there remains a consistency to the variability across studies, reflecting the protean or inchoate nature of the PD constructs themselves, which may be the most salient issue. The protean nature of the PD constructs and the variable characteristic features of the different comparison groups in various settings will contribute to diagnostic overlap, difficulties in differentiating between particular PD patients, and variability over which features emerge as discriminating. In this context, identification of items in this study as nondiscriminating should not be taken to mean that they do not belong in the criteria listings. Criteria are eliminated by the analysis because they either are highly correlated with others to become redundant, have low predictive power, share high content overlap with others, or are irrelevant in this particular sample. Some of these items might become discriminating in another setting with other comparison groups, whereas other items with which they are correlated could become redundant instead. In

417

that situation, keeping both to catch a case might be advisable. Alternatively, with a different diagnostic decision rule or where a theoretical purpose determines it to be necessary, one or more could be used as a definitional or initial step in diagnosing the PD. The other criteria with better discriminating ability could then be applied to making the particular diagnosis. This emphasizes the importance of distinguishing between core and identifying features of a disorder.43 Although core features represent the essence of the concept, they can be expected to overlap with others within some higher-level category order. Identifying features (which the DSM-III-R criteria are intended to be) are used to assign a given object to a particular category. An ideal discriminating criterion would have a relatively high correlation with the sum of all the criteria in the diagnosis (minus itself), and relatively low average correlations with the other criteria in the set taken individually.44 The use of data-reduction techniques offers a means to improve descriptive validity and determine the optimal number of criteria necessary to diagnose a PD syndrome with confidence. The findings suggest further considerations regarding the setting of operational diagnostic boundaries for PDs, where heterogeneity within the constructs contributes to diagnostic overlap and confusion. Reducing the current DSMIII-R algorithm requirements can decrease the number of possible combinations for individual PD categories. For example, decreasing the required minimum of five of nine criteria to two of three for narcissistic and compulsive, four of six for dependent, or three of four for passiveaggressive would reduce the number of combinations from the present 255 to four, 22, and five, respectively. Similar changes could follow for borderline (93 to 22) paranoid (64 to five), histrionic (163 to 16) and avoidant (64 to 22). The resulting potential increase in homogeneity could narrow the range of diagnostic uncertainty and increase the likelihood of identifying etiological factors, predictors of clinical course, specific treatments, familial aggregation, and neurobiological correlates for PD taxa. However, all such derivations must first be followed

NURNBERG, MARTIN, AND POLLACK

418

by extensive cross-validation to a wide spectrum of criterion groups in other settings. A largescale multicenter collaborative effort dedicated to PDs, modeled after the Epidemiologic Catchment Area study,45 and sampling both patient

and nonpatient populations seems indispensable to advance this field of study. ACKNOWLEDGMENT The authors

thank John Kane, M.D., for his support.

REFERENCES 1. Widiger TA. DSM-IV reviews of the personality disorders: introduction to special series. J Pers Disord 1991;5:122134. 2. Rutter ML. Temperament, personality and personality disorder. Br J Psychiatry 1987;150:443-458. 3. Tyrer P, Casey P, Ferguson B. Personality disorder in perspective. Br J Psychiatry 1991;159:463-471. 4. Livesley WJ. A systematic approach to the delineation of personality disorders. Am J Psychiatry 1987:144:772-777. 5. Nurnberg HG, Raskin M, Levine PE, Pollack S, Siegel 0, Prince R. The comorbidity of borderline personality disorder and other DSM-III-R axis II personality disorders. Am J Psychiatry 1991;148:1371-1377. 6. Fyer MR, Frances AJ, Sullivan T, Hurt SW, Clarkin J. Comorbidity of borderline personality disorder. Arch Gen Psychiatry 1988;45:348-352. 7. Oldham JM, Skodol AE, Kellman HD, Hyler SE, Rosnick L, Davies M. Diagnosis of DSM-III-R personality disorders by two structured interviews: patterns of comorbidity. Am J Psychiatry 1992;149:213-220. 8. Pfohl B, Coryell W, Zimmerman M, Stangl D. DSMIII personality disorders: diagnostic overlap and internal consistency of individual DSM-III criteria. Compr Psychiatry 1986;27:21-34. 9. Morey LC. Personality disorders in DSM-III and DSM-III-R: convergence, coverage, and internal consistency. Am J Psychiatry 1988:145:573-577. 10. Blashfield RK. Structural approaches to classification. In: Millon T, Klerman GL (eds): Contemporary Directions in Psychopathology: Toward the DSM-IV. New York, NY: Guilford, 1986. 11. Widiger TA, Frances A, Spitzer RL, Janet BW, Williams DSW. The DSM-III-R personality disorders: an overview. Am J Psychiatry 1988:145:786-795. 12. Loranger AW, Susman V, Oldham J, Rosakoff L. The Personality Disorder Examination: a preliminary examination report. J Pers Disord 1987;1:1-13. 13. Drake RE, Vaillant GE. A validity study of axis II of DSM-III. Am J Psychiatry 1985;142:553-558. 14. Loranger AW, Lenzenweger MF. Gartner AF. Susman VL, Herzig J, Zammit GK. Trait-state artifacts and the diagnosis of personality disorders. Arch Gen Psychiatry 1991;48:720-728. 15. Tyrer P, Seivewright N, Ferguson B. The Nottingham Study of Neurotic Disorder: relationship between personality status and symptoms. Psycho1 Med 1990;20:423-431. 16. Docherty J, Fiester S, Shea T. Syndrome, diagnosis, and personality disorder. In: Francis A, Hales R (eds): Psychiatric Update: The American Psychiatric Association Annual Review. Vol. 5. Washington, DC: American Psychiatric, 1986. 17. Greer HS. Cawley RH (eds). Some observations on the natural history of neurotic illness. In: Australian Medi-

cal Association. Mervyn Archdall Medical Monograph No. 3. Glebe, Australia: Australasian Medical Publishing, 1966. 18. Zimmerman M, Coryell W, Pfohl B. ECT response in depressed patients with and without a DSM-III personality disorder. Am J Psychiatry 1988;143:1030-1032. 19. Siever LJ, Klar H, Coccaro E. Psychobiological substrates of personality. In: Klar H, Siever LJ (eds): Biologic Response Styles: Clinical Implications. Washington, DC: American Psychiatric, 1985:37-66. 20. Siever LJ, Amin F, Coccaro EF, Bernstein D. Kavoussi RJ, Kalus 0, et al. Plasma homovanillic acid in schizotypal personality disorder. Am J Psychiatry 1991;148: 1246-1248. 21. Southwick SM, Yehuda Altered platelet alphal-adrenergic borderline personality disorder. 1014-1017.

R, Giller EL, Perry BD. receptor binding sites in Am J Psychiatry 1990;147:

22. Raine A, Venables PH, Williams M. Relationships between central and autonomic measures of arousal at age 15 years and criminality at age 24 years. Arch Gen Psychiatry 1990;47:1003-1007. 23. Siever LJ, Davis KL. A psychobiological perspective on the personality disorders. Am J Psychiatry 1991;148:16471658. 24. Liebowitz M, Stone M, Turkat I. Treatment of personality disorders. In Frances A, Hales R (eds): Psychiatric Update: American Psychiatric Association Annual Review. Vol. 5. Washington, DC: American Psychiatric, 1986. 25. Cowdry R, Gardner D. Pharmacotherapy line personality disorder. Arch Gen Psychiatry 119.

of border1988;45:111-

26. Soloff PH, George A. Nathan RS. Progress in pharmacotherapy of borderline disorders: a double blind study of amitriptyline, haloperidol and placebo. Arch Gen Psychiatry 1986;43:691-697. 27. Coccaro EF. Astill JL, Szeeley PJ, Malkowicz DE. Serotonin in personality disorder. Psychiatr Ann 1990;20:587592. 28. Dahl AA. Some aspects of the DSM-III personality disorders illustrated by a consecutive sample of hospitalized patients. Psychiatr Stand 1986;(Supp1)328:61-67. 29. Modestin J. Quality of interpersonal relationships: the most characteristic DSM-III BPD criterion. Compr Psychiatry 1987;28:397-402. 30. Widiger TA, Hurt SW, Frances A, Clarkin JF, Gilmore M. Diagnostic efficiency and DSM-III. Arch Gen Psychiatry 1984;41:1005-1012. 31. Nurnberg HG, Hurt SW, Feldman cient diagnosis of borderline personality Disord 1987;1:307-315. 32. Nurnberg HG, Raskin Siegel 0, Prince R. Hierarchy

A, Suh R. Effidisorder. J Pers

M, Levine PE, Pollack S. of DSM-III-R criteria effi-

DIAGNOSTIC

CRITERIA AND THRESHOLD FOR PD

ciency for the diagnosis of borderline personality disorder. J Pers Disord 1991;5:211-224. 33. Spitzer RL. Psychiatric diagnosis: are clinicians still necessary? Compr Psychiatry 1983;24:399-411. 34. Skodol AE, Rosnick L, Kellman D, Oldham JM, Hyler SE. Validating structured DSM-III-R personality disorder assessments with longitudinal data. Am J Psychiatry 1988;145:1297-1299. 35. Pfohl B, Stangl D, Zimmerman M. The Structured Interview for DSM-III Personality Disorders (SIDP). Iowa City, IA: University of Iowa Press, 1982. 36. Nie NH, Hull CH, Jenkins JG. Statistical Package for the Social Sciences. Ed. 2. New York, NY: McGraw-Hill, 1975. 37. Matthews DE, Farewell VT. Using and Understanding Medical Statistics. New York, NY: Karger, 1985. 38. Meehl P, Rosen A. Antecedent probability and efficiency of psychometric signs, patterns, or cutting scores. Psycho1 Bull 1955;52:194-216. 39. Sokal RR, Sneath PHA. Principles of Numerical Taxonomy. San Francisco, CA: Freeman, 1963.

419

40. Reich J, Noyes R Jr, Troughton E. Dependent personality disorder associated with phobic avoidance in patients with panic disorder. Am J Psychiatry 1987;144:323326. 41. Carpenter WT, Straus JS, Bartko JJ. Flexible system for the diagnosis of schizophrenia: report from WHO international pilot study of schizophrenia. Science 1973;182: 1275-1278. 42. Gunderson JG. Singer MT. Defining borderline patients: an overview. Am J Psychiatry 1975;132:1-10. 43. Blashfield RK, Haymaker D. A prototype analysis of the diagnostic criteria for DSM-III-R personality disorders. J Pers Disord 1988;2:272-280. 44. Epstein S. Aggregation and beyond: some basic issues on the prediction of behavior. J Pers 1983;51:360-392. 45. Regier DA, Myers JK, Kramer M, Robins LN, Blazer DC, Hough RL, et al. The NIMH Epidemiologic Catchment Area Program. Historical context, major objectives and study population characteristics. Arch Gen Psychiatry 1984;41:934-941.