The Expanded Trail Making Test

Archives of Clinical Neuropsychology, Vol. 13, No. 5, pp. 473–487, 1998 Copyright  1998 National Academy of Neuropsychology Printed in the USA. All rights reserved 0887-6177/98 $19.00 1 .00

PII S0887-6177(97)00041-3

The Expanded Trail Making Test: Rationale, Development, and Psychometric Properties Daniel E. Stanczak, Michael D. Lynch, Connie K. McNeil, and Blanca Brown California School of Professional Psychology-Fresno

Many procedures have been employed to determine the specific cognitive components necessary for successful Trail Making Test (TMT) performance. Yet, there is still considerable disagreement in the literature as to what these components might be. The present study explores an alternative methodology to address this problem. By systematically varying the stimuli within the TMT format, it may eventually be possible to isolate the cognitive demands of this test. As the first step toward this goal, two experimental forms of the TMT, forms X and Y, were developed and subjected to empirical validation. The results indicate that this Expanded Trail Making Test possesses adequate concurrent and criterion validity to support the proposed methodology. The results also suggest that the psychometric properties of the TMT format are robust to alterations in test stimuli. Secondary benefits of this methodology, in terms of explaining between-group variance and in terms of cross-cultural assessment, are discussed.  1998 National Academy of Neuropsychology. Published by Elsevier Science Ltd

TRAIL MAKING TEST FORMS X AND Y: RATIONALE, DEVELOPMENT, AND PSYCHOMETRIC PROPERTIES Originally used as one of the six subtests of the Army Individual Test Battery and currently employed as a part of the Halstead-Reitan Neuropsychological Battery, the Trail Making Test (TMT) has long been regarded as a reliable, valid, quick, and sensitive measure of cerebral function (Alvarez, 1962; Armitage, 1946; Cosgrove & Newell, 1991; Lezak, 1995; Reitan, 1955, 1958; Reitan & Wolfson, 1985). In a poll of 133 practitioners to determine which neuropsychological tests were most widely used, Sellers and Nadler (1992) found that the TMT was used by 85% of the respondents, making the TMT the second most widely used neuropsychological measure for patients 60 years and older, second only to the Wechsler Adult Intelligence Scale-Revised (WAIS-R). Correct diagnostic or ‘‘hit’’ rates for the TMT have ranged from 67% (Stanczak, 1984) to 88% (Fitzhugh, Fitzhugh, & Reitan, 1962). Moreover, the TMT is frequently used as a screening device for cerebral dysfunction in many general clinical settings. Despite its documented ability to differentiate brain-damaged (BD) individuals from normal control (NC) subjects, meaningful interpretation of TMT performance is difficult, beThe authors wish to express their gratitude to Dr. Merle Canfield of the California School of Professional PsychologyFresno for his assistance in reviewing and revising this manuscript. The copyright to TMT forms X and Y belongs to the first author. Address correspondence to: Daniel E. Stanczak, California School of Professional Psychology-Fresno, 5130 East Clinton Way, Fresno, CA 93727.

473

474

D. E. Stanczak et al.

cause the specific cognitive components contributing to successful TMT performance are not well-defined. Armitage (1946) speculated that successful performance on the TMT requires the abilities to plan, to perceive a double relationship, and to shift one’s focus of attention. Reitan and Tarshes (1959), however, asserted that successful TMT performance depends upon the capacity to recognize symbols and the ability to organize the task spatially. FalsStewart (1992) stated that the TMT measures visual and spatial skills as well as mental tracking abilities. Some investigators, in an attempt to identify the cognitive skills necessary for successful TMT performance, have examined the correlation between IQ and TMT scores. For example, Corrigan and Hinkeldey (1987) found significant correlations between the TMT and summary IQ’s of the WAIS-R. Specifically, Verbal IQ was negatively correlated to Trails A (rxy 5 2.25) and to Trails B (rxy 5 2.41). Negative correlations were also found between Performance IQ and Trails A (rxy 5 2.51) and Trails B (rxy 5 2.54). It was suggested that such correlations indicate that TMT performance may be a function of perceptuomotor abilities and the capacity to process verbal stimuli. Other investigators have employed factor analytic techniques to more precisely identify the cognitive correlates of successful TMT performance. For instance, Goldstein and Shelley (1975) found the TMT to load heavily on a factor representing motor problem solving. Swiercinsky (1979) found the TMT to load on a factor measuring visuomotor speed and coordination. Groff and Hubble (1981) compared performance scores from the General Aptitude Test Battery with the TMT. Three factors emerged from their analysis: (a) symbolic fluency, (b) visuoperceptual ability, and (c) motor coordination. The TMT loaded only on the visuoperceptual ability factor. However, the three factors generated in this study were able to account for only a small proportion of the total variance. These findings, taken together, seem to suggest that neither symbolic fluency nor motor coordination are significant influences on TMT performance. In a factor analysis of the Halstead-Reitan Neuropsychological Battery, Moehle, Rasmussen, and Fitzhugh-Bell (1990) identified seven factors: (a) verbal, (b) spatial, (c) left-tactile, (d) simple motor, (e) complex motor, (f) basic language, and (g) right-tactile. The TMT loaded only on the ‘‘spatial’’ factor along with measures such as the Tactual Performance Test (total time and location scores), Halstead Category Test, Speech Sounds Perception Test, and Wechsler Adult Intelligence Scale (Digit Symbol, Object Assembly, Block Design, and Picture Arrangement subtests). Shum, McFarland, and Bain (1990) found the TMT to load highest on a factor they called ‘‘visuomotor scanning abilities.’’ Moreover, the obtained factor structure was demonstrated to be stable across both brain-lesioned and control groups. Obviously, the results of exploratory factor analysis will vary as a function of variable inclusion. Thus, it is not surprising that such analyses have failed to yield consistent findings. Fossum, Holberg, and Reinvang (1992) varied the stimuli on the TMT by alternating the configural arrangement of the test stimuli, that is, placing the stimuli of form B in the spatial configuration of form A and visa versa. Using this paradigm they were able to determine that at least a portion of TMT variance could be explained by two stimulus dimensions: symbolic complexity and spatial configuration. They also raised the question of ‘‘. . . whether an expanded version of the TMT should be developed and standardized’’ (p. 74). This particular study suggests that yet another method exists to examine the cognitive demands of the TMT. By systematically varying the stimulus properties of the TMT, one might be able to empirically assess the relative contributions of various cognitive skills. As a first step toward this end, TMT Forms X and Y were developed. Compared to Form A,

The Expanded Trail Making Test

475

FIGURE 1. Sample items from the Expanded Trail Making Test, Forms X and Y.

which requires subjects to connect in order a series of numbered circles, and Form B, which requires subjects to connect numbered and lettered circles in an alternating sequence, TMT Form X consists of a series of 25 clock faces, which the subject is required to connect in order (e.g., 12:00 to 12:15 to 12:30, etc.). TMT Form Y consists of a sequence of 24 solid black dots that the subject is to connect in order of increasing diameter. In terms of spatial configuration, the stimuli of TMT Forms X and Y are arranged in the vertically inverted, mirror-image of Forms A and B, respectively. Because of spatial limitations, only 24 stimulus items are contained in Form Y. The sample items of TMT Forms X and Y are shown in Figure 1. Only one study employing Forms X and Y of the TMT exists. Davis, Adams, Gates, and Cheramie (1989) used the Tactual Performance Test (TPT) and midrange adaptations of Forms A, B, X, and Y of the TMT to differentiate normal from learning disabled children. The performance of these two groups differed significantly on the TPT total time, TPT localization, and TMT Forms B, X, and Y. A follow-up discriminant analysis revealed the TMT

476

D. E. Stanczak et al.

Form X, TPT total time, and TPT localization scores accounted for a significant proportion of the variance between the two groups. Using these three test scores, the authors were able to correctly predict group membership with 80% accuracy. While the work of Davis et al. (1989) provides preliminary evidence for the validity of the mid-range versions of the four forms of the TMT (hereafter referred to as the Expanded Trail Making Test, or ETMT), little is known regarding ETMT performance in adults. Moreover, it is not certain that the ETMT forms X and Y have sufficient validity to support the methodological procedure outlined above. Thus, it is the purpose of the present study to: (a) test the robustness of the TMT format to alterations of test stimuli, (b) establish normative data regarding ETMT performance in an adult population, (c) assess the unique proportion of between-group variance (brain-damaged versus normal subjects) explained by each ETMT form, (d) examine the criterion validity of forms X and Y of the ETMT, (e) determine the factor structure of the ETMT, and (f) determine if the factor structure of the ETMT is stable across clinical and control samples. Once determined, these data will enable implementation of the methodology we have proposed, assuming of course that the ETMT forms X and Y are found to be valid.

ANALYSIS 1 Purpose The purpose of Analysis 1 was to test the robustness of the TMT format to alterations of test stimuli. Robustness, for the purposes of this study, was defined as: (a) significant correlations between the original and altered forms of the ETMT, and (b) significantly different performances by brain-lesioned and control groups across all four ETMT forms.

Subjects Archival data, collected over the past 11 years, served as the basis for analysis. Subjects in this database had been referred for neuropsychological evaluation at one of nine different outpatient or inpatient settings, or were normal volunteers recruited from various university settings in five states (Texas, California, Pennsylvania, Tennessee, or Louisiana). All tests were administered by a licensed neuropsychologist or by neuropsychology doctoral candidates under the supervision of a licensed neuropsychologist (DES). From an initial pool of 803 subjects, 191 subjects were identified as brain-damaged, with lesion confirmation by at least two neurodiagnostic tests, such as electroencephalography, magnetic resonance imaging, computerized tomography, angiography, neurosurgery, etc. Three hundred twenty-three subjects were identified as normal controls, based on a negative history for brain damage or significant psychopathology. Because the sample contained an insufficient number of ethnic minority subjects to permit multivariate analysis, only Caucasian subjects were included in this and subsequent analyses. This subject selection procedure yielded data on 416 individuals: 164 brain-damaged (BD) subjects and 252 normal controls (NC). The mean age for NC subjects was 29.7 years (SD 5 10.51; range 5 15–68), while the mean age for the BD subjects was 43.24 years (SD 5 17.58; range 5 14–88). The mean education for the NC group was 14.95 years (SD 5 2.22; range 5 10–20), while the mean education for the BD group was 13.0 years (SD 5 2.76; range 5 6–20). Of the brain-injured individuals, 81% were right-handed, 8% were lefthanded, and 2% were ambidextrous. Of the normal control subjects, 90% were right-handed, 8% were left-handed, and less than 1% were ambidextrous.

The Expanded Trail Making Test

477

TABLE 1 Etiological Distribution of the Brain-Damaged Group Diagnosis Closed head injury Penetrating head injury Cerebral vascular accident Seizure disorder Neoplasia Degenerative Hydrocephalus Demyelinating disorders Anoxia Lobectomy Infectious processes Total

N

Male

Female

77 3 55 4 8 6 2 1 2 2 4 164

63 3 38 1 2 1 0 0 2 1 3 114

14 0 17 3 6 5 2 1 0 1 1 50

The etiological breakdown of the brain-damaged group is outlined in Table 1. As will be noticed, the brain damaged group is a heterogeneous one, selected so as to optimize the external validity of the study (Gemmell & Stanczak, 1996). Procedure The ETMT was administered in the order A, B, X, Y to all individuals in the sample. Forms A and B were administered using the instructions described by Reitan (1955), with the exception that a 3-minute time limit was imposed for completion of each form. Forms X and Y were administered in the same fashion as Forms A and B. Scores were prorated for subjects not completing a form within the 3-minute time limit. Prorated scores were calculated as follows: (a) 180 seconds was divided by the number of successfully connected stimuli (counting the ‘‘Begin’’ stimulus as number one), and (b) the result was then multiplied by the total number of stimuli on the form (Forms A, B, and X 5 25; Form Y 5 24). The total time, actual or prorated, required for completion served as the dependent measure for each form. Design Because a preliminary review of the raw data revealed leptokurtic, positively skewed distributions, logarithmic transformations were conducted on the raw ETMT scores. This procedure yielded distributions that more closely approximated the normal curve. Pearson product-moment correlations were then obtained to assess the degree of relationship between the original and experimental forms of the TMT. To determine if the brain-damaged group demonstrated significantly longer completion times on the various ETMT forms than did the normal control group, the logarithmically transformed scores were analyzed using a 2 3 4, groups (brain-damaged versus normal) by TMT Form (A, B, X, and Y) factorial design. Preliminary t-tests did not indicate significant gender-specific differences in performance across any of the ETMT forms. Although preliminary t-tests did reveal significant betweengroup differences regarding age (t 5 9.81, df 5 413, p , .001) and education (t 5 6.50, df 5 276, p , .001), information regarding educational level was available on an insufficient number of subjects to permit its use as a covariate. Thus, for the present study, only age was used as a covariate. The statistical significance of results was assessed using multiple analysis of covariance (MANCOVA).

478

D. E. Stanczak et al.

TABLE 2 Correlations: Expanded Trail Making Test, Gender, Age, and Education A Log B Log

X Log

YLog

Gender

Age

Education

B Log

X Log

YLog

Gender

Age

.81 416 ,.001

1.00

.55 416 ,.001

.60 416 ,.001

1.00

.69 416 ,.001

.68 416 ,.001

.54 416 ,.001

1.00

2.18 416 ,.001

2.17 416 .001

2.15 416 .002

2.12 416 .002

1.00

.52 415 ,.001

.54 415 ,.001

.32 415 ,.001

.36 415 ,.001

2.07 415 .160

1.00

2.29 278 ,.001

2.31 278 ,.001

2.29 278 ,.001

2.29 278 ,.001

2.03 278 .610

2.02 278 .804

Cell information: Top line 5 correlation coefficient, middle line 5 N, bottom line 5 probability.

Results The correlation matrix for all variables is presented in Table 2. Inspection of this table indicates moderate but significant correlations between the original and experimental forms of the TMT. The results of the MANCOVA were significant (df 5 4,408, F 5 65.68, p , .001). Follow-up analyses of covariance revealed significant main effects for group (normals versus brain-damaged) and ETMT form (A, B, X, and Y). The effect of the covariate, age, was significant across all four ETMT forms ( p , .001), with increasing age associated with higher ETMT scores in both groups. Follow-up univariate ANCOVAs and Scheffe´ post hoc tests indicated that brain-damaged individuals took significantly more time than did normal subjects to complete all TMT forms (Table 3). Follow-up t-tests, using Bonferroni confidence intervals to correct for the effects of multiple comparisons were performed to assess the significance of within-cells (ETMT Form) variance. These tests (Table 4) revealed significant differences between the mean times to complete each ETMT form with the exception of the difference between TrailsBLog and TrailsYLog. These results indicated that TrailsA took significantly less time to complete than TrailsB and TrailsY, which in turn took significantly less time to complete than TrailsX.

ANALYSIS 2 Purpose The purpose of Analysis 2 was to establish norms for the experimental use of the ETMT with an adult population.

The Expanded Trail Making Test

479

TABLE 3 Analysis of Covariance in the Expanded Trail Making Test Performance by Group with Age as a Covariate ETMT Form

Source of Variation

Degrees of Freedom

Mean Square

F

Probability

A Log

Covariate (Age) Group Explained Residual

1 1 3 411

30.50 25.89 20.03 .14

225.29 191.27 147.94 —

,.001 ,.001 ,.001 —

B Log

Covariate (Age) Group Explained Residual

1 1 3 411

42.36 31.56 26.05 .71

247.44 184.34 152.15 —

,.001 ,.001 ,.001 —

X Log

Covariate (Age) Group Explained Residual

1 1 3 411

21.37 36.95 21.13 .37

57.24 98.97 56.59 —

,.001 ,.001 ,.01 —

YLog

Covariate (Age) Group Explained Residual

1 1 3 411

16.33 27.18 15.12 .22

75.96 126.39 70.29 —

,.001 ,.001 ,.01 —

Subjects Subjects were the same as those in Analysis 1. Procedure Because of the difficulty interpreting logarithmically transformed descriptive data, raw ETMT scores were chosen for analysis. Because of the leptokurtic and positively skewed distributions of those raw scores and because the presence of extreme scores could significantly distort the means and standard deviations of the distributions, outliers were identified

TABLE 4 Differences in Completion Time Across the Expanded Trail Making Test Forms T Values

TrailsA Log

TrailsB Log

TrailsX Log

TrailsB Log

TrailsX Log

TrailsYLog

13.15 511 ,.001

17.65 428 ,.001

12.32 507 ,.001

11.97 523 ,.001

0.042 832 .68 12.14 528 ,.001

Cell information: top line 5 t value; second line 5 degrees of freedom; third line 5 probability.

480

D. E. Stanczak et al.

FIGURE 2. Group Expanded Trail Making Test means: Normal controls, brain damaged, and brain damaged minus outliers groups.

and eliminated from further analysis. For the purposes of this study, an outlier was identified as any case whose prorated score on TrailsA, TrailsB, or TrailsX was greater than 1499 or any case whose prorated score on TrailsY was greater than 1439. This effectively eliminated from consideration any subject who connected fewer than four stimuli on any ETMT form. Thirty such outliers were identified, all but one from the brain-damaged group. To determine if the elimination of outliers significantly altered ETMT means, t-tests were performed, for each ETMT form, between the means obtained by the original brain-damaged group and those obtained by the brain-damaged-minus-outliers group (Figure 2). The results indicated that the elimination of extreme scores did indeed lower the means significantly on TrailsX (t 5 3.83, df 5 231, p , 001) and TrailsY (t 5 2.14, df 5 206, p 5 .03), but failed to do so on TrailsA (t 5 1.68, df 5 269, p 5 .09) and TrailsB (t 5 1.57, df 5 220, p 5 .11). To determine if the elimination of extreme scores significantly affected the differences between the brain-damaged and normal control group, t-tests were performed between the mean scores obtained by the normal control and brain-damaged-minus-outliers groups on each ETMT form. The results (Figure 2) demonstrated that significant between-group differences still obtained on all ETMT forms: TrailsA (t 5 13.42, df 5 206, p , .001), TrailsB (t 5 11.96, df 5 202, p , .001), TrailsX (t 5 9.19, df 5 256, p , .001), and TrailsY (t 5 9.02, df 5 196, p , .001).

The Expanded Trail Making Test

481

TABLE 5 Descriptive Statistics for Brain-Damaged and Normal Subjects Across the Expanded Trail Making Test (ETMT) Forms ETMT Form

M

SD

Minimum

Maximum

Normal Controls (N 5 252)

A B X Y

25.45 55.48 145.82 58.80

8.64 23.99 127.68 20.86

10 19 25 8

65 271 1125 140

Brain-Damaged (N 5 164)

A B X Y

59.94 141.19 331.14 130.40

31.13 95.56 252.27 106.73

16 34 75 33

206 643 1125 864

Group

Results Because the elimination of outliers produced more conservative estimates of the population parameters while preserving the significant differences between brain-damaged and normal subjects, descriptive statistics were obtained for the brain-damaged-minus-outliers group. These statistics are compared to those of the normal control group in Table 5.

ANALYSIS 3 Purpose The purpose of Analysis 3 was to quantify the unique proportion of between-group variance explained by each ETMT form and demographic variables.

Subjects Subjects were the same as those employed in Analysis 1.

Procedure As appropriate, zero-order Pearson product-moment, point biserial, or tetrachoric correlations were calculated between each variable: TrailsALog, TrailsBLog, TrailsXLog, TrailsYLog, age, education, and the dichotomous variables, group and gender. Because of missing data (primarily education level) and in an attempt to maximize the power of the study, pairwise deletion of missing data was chosen. Multiple regression techniques were then employed to decompose the between-group variance.

Results The demographic variables—age, education, and gender—accounted for the largest unique proportions of between-group variance, accounting for 2.35%, 6.95%, and 2.35%, respectively. TrailsXLog, of all the ETMT forms, accounted for the largest proportion of

482

D. E. Stanczak et al.

TABLE 6 Criterion Validity of the Expanded Trail Making Test

Variable(s) a 1 2 3 4 1 and 2 3 and 4 1, 2, 3, and 4 1, 3, and 2 b 1, 2, 3, 4, 5, and 6 1, 6, 2, 3, and 5 b

Hit Rate

Sensitivity

Specificity

Positive Predictive Power

81.11 82.39 74.83 78.64 83.71 79.50 85.65 85.94 87.21 87.05

.76 .76 .73 .75 .77 .77 .81 .82 .83 .84

.88 .91 .79 .87 .93 .85 .95 .95 .95 .94

.88 .93 .89 .94 .94 .92 .97 .97 .97 .96

Negative Predictive Power

Kappa

p

.75 .73 .55 .58 .74 .63 .70 .70 .74 .74

.62 .65 .46 .54 .68 .57 .70 .70 .73 .73

,.0001 ,.0001 ,.0001 ,.0001 ,.0001 ,.0001 ,.0001 ,.0001 ,.0001 ,.0001

1 5 TrailsA Log , 2 5 TrailsB Log , 3 5 TrailsX Log , 4 5 TrailsYLog , 5 5 age, 6 5 gender. Results of stepwise discriminant analysis using variables listed in the immediately preceding direct analysis. Variables are listed in order of step entered. a

b

unique between-group variance (1.42%). TrailsALog, TrailsBLog and TrailsYLog each accounted for a small proportion of unique variance (.96%, .28%, and .25%, respectively). ANALYSIS 4 Purpose The purpose of Analysis 4 was to examine the criterion validity of the ETMT. Subjects Subjects were the same as those employed in Analysis 1. Procedure To assess the relative criterion validity of each ETMT form, separate discriminant analyses were performed, each employing one of the ETMT forms as the discriminating variable with group membership (BD versus NC) as the dependent variable. To compare the relative criterion validity of the original versus newer forms of the ETMT, two additional discriminant analyses were performed: one employing forms A and B as the discriminating variables and another employing forms X and Y as the discriminating variables. Next, to determine the criterion validity of the ETMT as a whole, all four forms were first directly, and then in a step-wise fashion, entered into a discriminant analysis. Finally, to determine the relative criterion validity of the ETMT and the demographic variables, age and gender, a stepwise discriminant analysis was performed. Results The results of these various discriminant analyses are summarized in Tables 6 and 7. As can be seen, the diagnostic ‘‘hit rates’’ for the various ETMT forms were grossly comparable and well above chance levels. The hit rate for forms X and Y combined was slightly lower than that for forms A and B combined. Combining all four forms of the ETMT with the demographic variables produced the highest hit rate, sensitivity, and specificity.

The Expanded Trail Making Test

483

TABLE 7 Prediction Error Rates for the (Expressed in Percentages) Expanded Trail Making Test Variables a 1 2 3 4 1 and 2 3 and 4 1, 2, 3, and 4 1, 3, and 2 b 1, 2, 3, 4, 5, and 6 1, 6, 2, 3, and 5 b

False Positive Errors

False Negative Errors

11.8 7.5 10.6 6.3 5.9 8.3 2.8 2.7 2.8 3.5

25.3 27.1 45.2 41.7 26.0 37.1 30.1 29.6 26.5 25.9

1 5 TrailsA Log , 2 5 TrailsB Log , 3 5 TrailsX Log , 4 5 TrailsYLog , 5 5 gender, 6 5 age. b Results of stepwise discriminant analysis using variables listed in the immediately preceding direct analysis. Variables are listed in order of step entered. a

All ETMT forms produced unacceptably high false negative error rates, with 25% to 45% of brain-damaged patients being misclassified as being neurologically normal. The lowest misclassification rate was produced by combining the ETMT with demographic variables. ETMT forms A and X produced the highest false positive error rates. However, the false positive error rates were substantially smaller than the false negative error rates and were reduced significantly by the simultaneous use of all ETMT forms. ANALYSIS 5 Purpose The purpose of Analysis 5 was to determine the factor structure of the ETMT and to determine whether or not this factor structure is stable across clinical and control samples. Subjects Subjects were the same as those employed in the previous analyses. Procedure To determine whether or not exploratory factor analysis should be based upon pooled group data, separate correlation matrices —including education, age, gender, and all logarithmically transformed ETMT scores—were derived for each group. Structural equation modeling was then used to assess the null hypothesis of no significant difference between the two correlation matrices. The results of this analysis indicated that the two correlation matrices were indeed different ( χ 2 5 102.143, df 5 21, comparative fit index 5 .851, p , .001). Correlations attaining statistically significant differences included: TrailsALog with TrailsYLog, TrailsXLog with education, and TrailsYLog with education. Exploratory factor analyses were thus performed separately for each subject group. For these analyses, principle components factor analyses with orthogonal rotations were used. Factors were selected on the basis of eigenvalues greater than one and the meaningfulness of the factors. A variable loading equal to or greater than .49 was established as the criterion for inclusion.

484

D. E. Stanczak et al.

TABLE 8 Factor Structure of the Expanded Trail Making Test Factor Loadings Group

Variable

Factor 1

Factor 2

Factor 3

Normal Controls

TrailsA Log TrailsB Log TrailsX Log TrailsYLog Age Gender Education

.75 .79 .49 .56 .29 .03 2.19

.14 .21 2.23 2.29 .76 2.10 .73

.11 .01 2.34 2.04 .12 .92 2.18

Brain-Damaged

TrailsA Log TrailsB Log TrailsX Log TrailsYLog Age Gender Education

.90 .89 .76 .85 .59 .19 .15

2.04 .06 .12 .02 2.03 2.71 .79

Results The rotated factor matrices are displayed in Table 8. A three-factor model emerged for the normal control group, while a two-factor model emerged for the BD group. For the NC group, all ETMT forms loaded on the first factor, which accounted for 26.6% of the variance. Age and education loaded highly on Factor 2, which accounted for an additional 18.6% of the variance. Gender, which accounted for 14.5% of the variance, loaded on Factor 3. Within the BD group, all ETMT forms and age loaded on the first factor, which explained 47.3% of the variance. An additional 16.4% of the variance was explained by gender and education, which loaded on Factor 2.

DISCUSSION The purpose of the present study was to establish the basis for a new methodology for identifying the cognitive components of neuropsychological tests, in general, and the Trail Making Test, in particular. The process of systematically varying test stimuli has long been employed, with great success, in cognitive and behavioral research. However, this methodology has rarely, if ever, been employed in evaluating the cognitive substrata of neuropsychological measures, and, until such substrata are clearly identified, it will be very difficult to identify, other than grossly and intuitively, the neural mechanisms underlying neuropsychological test performance. The present study provides support for the assertion that the Trail Making Test format is indeed robust to variations in stimulus dimensions. In particular, it was shown that significant alterations of test stimuli, while affecting mean completion times, did not significantly alter the criterion or factorial validity of the test. Indeed, an interesting clinical finding was the fact that the criterion validity of the original TMT could actually be enhanced through the concomitant use of ETMT forms X and Y. Since the present study focused primarily on the psychometric properties of the ETMT, it is still not yet possible to identify specific cognitive processes underlying successful TMT

The Expanded Trail Making Test

485

performance. Indeed, the alterations in stimulus dimensions, in the present study, were gross, and one can only speculate at this junction as to why a shift from alphanumeric stimuli to clock faces produces such a large increase in mean completion times. While several hypotheses come to mind (e.g., stimulus familiarity, visual discrimination, stimulus complexity, level of processing, sequential versus parallel processing, etc.), the present design does not allow for such hypotheses to be tested. However, the present study does increase our research armamentarium, enabling future systematic testing of such hypotheses. This study also provides norms for ETMT performance within an adult population. However, caution should nevertheless be used when employing these norms clinically or experimentally. First, the sample size is relatively small. It will be interesting to determine how robust these sample statistics are once a larger normative sample is obtained. Furthermore, once an adequate sample size is obtained, it will be possible to establish norms with respect to subject age, gender, and education. Secondly, the sample is limited to Caucasians. Thus, the cross-cultural applicability of these norms is, as of yet, undetermined. Thirdly, prorating of scores was employed in order to mimic the common clinical practice of establishing time limits for completion. Further research is obviously needed to determine the appropriateness of such prorated scores and, if appropriate, the optimal time limit for obtaining such scores. The present study also indicates that ETMT forms X and Y explain a unique proportion of between-group variance, above and beyond that explained by forms A and B and demographic variables. Indeed, it was surprising to find that, once the between-group variance was decomposed, none of the ETMT forms accounted for a large proportion of unique variance. This finding was unanticipated given the literature touting the TMT as one of the most sensitive of the Halstead-Reitan variables (Jarvis & Barth, 1994; Russell, 1986). While the reliability of ETMT forms X and Y have yet to be systematically explored, the present study provides evidence of their validity. First, the observation that ETMT forms X and Y retain significant correlations with forms A and B provides evidence of concurrent validity. Second, the observation that all ETMT forms load on a single factor provides further evidence of concurrent validity. Third, the observations that ETMT forms X and Y discriminate brain-damaged from normal control subjects at statistically and clinically significant rates and that the ETMT produces diagnostic hit rates superior to those produced by the original TMT provide support for the predictive validity of the ETMT. The finding that the ETMT factor structure is not stable across subject groups is surprising and difficult to interpret. The present data are insufficient to offer an explanatory hypothesis. This anomalous finding may be an artifact, unique to the current sample, or may reflect cognitive reorganization as the result of cerebral insult. It may also be the case that such differences in factor structure are not unique to the ETMT and that, indeed, they are relatively common to other neuropsychological measures. It is clear that few studies have examined the stability of neuropsychological factors across subject groups and that alterations in factor structure as a consequence of cerebral insult may prove to be fertile ground for further research into brain-behavior relationships. Because of its empirically demonstrated value in detecting the presence of cerebral dysfunction, it has become common practice, in many clinics, to use the original Trail Making Test as a screening device for ‘‘organicity.’’ However, a reinspection of Tables 7 and 8 reveals the difficulties associated with such screenings. All ETMT forms produce an unacceptably high number of false negative diagnostic errors. These findings are not surprising in that they echo the admonitions of Lezak (1995) and others who argue that no single test is sufficiently sensitive to the heterogeneous effects of brain damage to serve effectively as the ‘‘litmus test’’ for ‘‘organicity,’’ as no single instrument taps into all possible brain functions. Those who routinely use the TMT or ETMT clinically as a screening device for cerebral

486

D. E. Stanczak et al.

dysfunction should be aware of its limitations and should consider using the TMT/ETMT as part of a battery of screening tests rather than as their sole screening instrument. To the authors’ knowledge, the only cross-cultural study to date involving the TMT is that performed by Maj et al. (1993). In this study, Maj et al. constructed the ‘‘Color Trails’’ Test, forms 1 and 2, which were modeled after the TMT forms A and B. In Color Trails 1, the subject connects consecutively numbered pink and yellow circles. In Color Trails 2, the subject is presented with two sets of numbered circles. One set of circles is yellow, and the other is pink. The subject is instructed to connect the numbers in sequence while alternating between pink and yellow stimuli. Although Maj et al. (1993) cite the significant correlations between the Color Trails and the original TMT as evidence of construct validity, the correlations of the Color Trails forms with the original TMT forms are rather modest (.408 for Color Trails 1 and TMT form A and .496 for Color Trails 2 and TMT form B). Since these new tests share only 17% and 25%, respectively, of the original TMT variance, it is unclear to what degree the Color Trails are measuring the same neuropsychological construct(s) as Trails A and B. Similarly, Maj et al. (1993) cite the reduced variance of the Color Trails, across geographic centers, as evidence that the Color Trails are less culturally biased than the original TMT. However, there are several problems with this conclusion. First, the reduction in variance across geographic centers is obtained only on Color Trails 2. Furthermore, since the Color Trails represent such a substantial deviation from the original TMT format, it cannot yet be demonstrated that the difference in variance, between Color Trails 2 and TMT form B, is attributable simply to reduced cultural bias. Indeed, such a difference in variance might be due to a number of factors, such as the introduction of color, the elimination of alphabetic characters, potential differences in stimulus complexity or spatial arrangement, and /or the relative difficulty of the tests. Thus, while the Maj et al. study provides a pioneering and much needed step toward the development of neuropsychological instruments which are valid cross-culturally, the study fails to establish the Color Trails test as a suitable alternative to the TMT. Rather than attempting to construct a single instrument for universal cross-cultural application, an alternative strategy would be to validate existing neuropsychological instruments for use with diverse cultures. Although the present study was conducted using only a Caucasian sample and thus cannot speak directly to cross-cultural issues, the ETMT forms X and Y seem to possess promise for use in cross-cultural applications, because they rely less on alphanumeric stimuli. Indeed, preliminary evidence from our laboratory (Awadalla, Stanczak, & Stanczak, 1995) suggests that an Arabic version of the ETMT, administered to a Sudanese sample, possesses psychometric properties similar to those of the English version. Such work, using diverse cultural samples, is continuing, and we hope to present those results in future reports. Because our brain-damaged and normal control groups differed in terms of age and because a relatively large percentage of cases were missing data regarding subjects’ level of education, further research is warranted to more precisely examine the relationship between age, education, and ETMT performance. Such work is indeed in progress and the results will be the subject of future reports. Meanwhile, it is important to recognize that age, gender, and education each account for a significant percentage of between-group variance. Thus, these demographic variables should be considered when interpreting ETMT performance. In conclusion then, it appears that the TMT format is a robust one. Altering the specific stimuli contained within this format may provide yet another methodology by which to examine the specific cognitive skills contributing to TMT performance. Moreover, such work seems to have secondary benefits in terms of explaining between-group variance and, possibly, in terms of cross-cultural assessment.

The Expanded Trail Making Test

487

REFERENCES Alvarez, R. R. (1962). Comparison of depressive and brain-injured subjects on the Trail Making Test. Perceptual and Motor Skills, 14, 91–96. Armitage, S. G. (1946). An analysis of certain psychological tests used for the evaluation of brain injury. Psychological Monographs, 60, 1–48. Awadalla, A., Stanczak, E. M., & Stanczak, D. E. (1995). Validation of an Arabic version of the Expanded Trail Making Test. Paper presented at the 75th Annual Convention of the Western Psychological Association, Los Angeles, CA. Corrigan, J. D., & Hinkeldey, N. S. (1987). Relationships between parts A & B of the Trail Making Test. Journal of Clinical Psychology, 43, 402–409. Cosgrove, J., & Newell, T. G. (1991). Recovery of neuropsychological functions during reduction in use of phencyclidine. Journal of Clinical Psychology, 47, 159–169. Davis, R. D., Adams, R. E., Gates, D. O., & Cheramie, G. M. (1989). Screening for learning disabilities: A neuropsychological approach. Journal of Clinical Psychology, 45, 423–428. Fals-Stewart, W. (1992). An interrater reliability study of the Trail Making Test (Parts A and B). Perceptual and Motor Skills, 74, 39–42. Fitzhugh, K. B., Fitzhugh, L. C., & Reitan, R. M. (1962). Relation of acuteness of organic brain dysfunction to Trail Making Test performance. Perceptual and Motor Skills, 15, 399–403. Fossum, B., Holmberg, H., & Reinvang, I. (1992). Spatial and symbolic factors in performance on the Trail Making Test. Neuropsychology, 6, 71–75. Gemmell, S. B., & Stanczak, D. E. (1996). Subject selection in neuropsychological research: The use of homogeneous versus heterogeneous samples. Paper presented at the 17th Annual Central California Research Symposium, May 2nd, 1996, California State University, Fresno, CA. Goldstein, G., & Shelly, C. H. (1975). Similarities and differences between psychological deficit in aging and brain damage. Journal of Gerontology, 30, 448–455. Groff, M. G., & Hubble, L. M. (1981). A factor analytic investigation of the Trail Making Test. Clinical Neuropsychology, 3(4), 11–13. Jarvis, P. E., & Barth, J. T. (1994). The Halstead-Reitan Neuropsychological Battery: A guide to interpretations and clinical applications. Odessa, FL: PAR. Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.) New York: Oxford University Press. Maj, M., D’Elia, L., Satz, P., Janssen, R., Zaudig, M., Uchiyama, C., Starace, F., Galderisi, S., & Chervinsky, A. (1993). Evaluation of two new neuropsychological tests designed to minimize cultural bias in the assessment of HIV-1 seropositive persons: A WHO study. Archives of Clinical Neuropsychology, 8, 123–136. Moehle, K. A., Rasmussen, J. L., & Fitzhugh-Bell, K. B. (1990). Factor analysis of neuropsychological tests in an adult sample. International Journal of Clinical Neuropsychology, 12(3–4), 107–115. Reitan, R. M. (1955). The relation of the Trail Making Test to organic brain damage. Journal of Consulting Psychology, 19, 393–394. Reitan, R. M. (1958). Validity of the Trail Making Test as an indicator of organic brain damage. Perceptual and Motor Skills, 8, 271–276. Reitan, R. M., & Tarshes, E. L. (1959). Differential effects of lateralized brain lesions on the Trail Making Test. Journal of Nervous and Mental Disease, 119, 257–261. Reitan, R. M., & Wolfson, D. (1985). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation. South Tucson, AZ: Neuropsychology Press. Russell, E. W. (1986). The psychometric foundation of clinical neuropsychology. In S. B. Filskov & T. J. Boll (Eds.), Handbook of clinical neuropsychology (Vol. 2, pp. 45–80). New York: John Wiley & Sons. Sellers, A. H., & Nadler, J. D. (1992). A survey of current neuropsychological assessment procedures used for different age groups. Psychotherapy in Private Practice, 11, 47–57. Shum, D. H. K., McFarland, K. A., & Bain, J. D. (1990). Construct validity of eight tests of attention: Comparison of normal and closed head injured samples. The Clinical Neuropsychologist, 4, 151–162. Stanczak, D. E. (1984). Assessment of level of performance in neuropsychological diagnosis: A comparison of the discriminant validity of alternative statistical methods. (Doctoral dissertation, Memphis State University, 1984). Dissertation Abstracts International, 45, 3990B. Swiercinsky, D. P. (1979). Factorial pattern description and comparison of functional abilities in neuropsychological assessment. Perceptual and Motor Skills, 48, 231–241.