- Email: [email protected]
An Exploration of the Construct Validity of the Heaton Memory Tests
Archives of Clinical Neuropsychology, Vol. 15, No. 2, pp. 95–103, 2000 Copyright © 2000 National Academy of Neuropsychology Printed in the USA. All rights reserved 0887-6177/00 $–see front matter
PII S0887-6177(98)00152-8
An Exploration of the Construct Validity of the Heaton Memory Tests Raymond K. DiPino, Michael H. Kabat, and Robert L. Kane Veterans Administration Maryland Health Care System
The Heaton techniques for assessing memory differ from most other memory measures by employing the combination of learning trials, repeated exposure to test stimuli, and 4-hour recall measures. The present study was designed to explore the relationship between the Heaton Story and Figure memory procedures, the California Verbal Learning Test (CVLT), and measures of attention, word fluency, and spatial perception. Data from 126 individuals were analyzed. Three separate factor analyses were performed that examined measures of attention, learning, and recall. Each revealed a three-factor solution accounting for 71, 70, and 72% of the variance, respectively. Regression analyses supported the visual components of Figure Memory and the verbal components of Story Memory obtained in the factor analyses. Overall, the findings supported the independence of the Heaton Memory procedures and the role of spatial factors in performance of Figure Memory. Additionally, both Figure and Story Memory shared variance with the CVLT, supporting the validity of both procedures as memory measures. © 2000 National Academy of Neuropsychology. Published by Elsevier Science Ltd
Although most clinical measures of memory maintain a distinction between verbal and visual memory (Smith, Malec, & Ivnik, 1992), factor-analytic studies examining these measures have not consistently found distinct verbal and visual factors (Smith et al., 1992; Larrabee & Curtiss, 1995). In general, verbal memory tests have been found to load on a general or verbal memory factor, with immediate and delayed recall measures producing their strongest contribution to a memory factor (Larrabee & Curtiss, 1995; Larrabee, Kane, Schuck, & Francis, 1985; Loring & Papanicolaou, 1987; Smith et al., 1992). In contrast, examinations of visual memory tests have provided less consistent results. For example, Leonberger, Nicks, Goldfader, and Munz (1991), in an examination of the Wechsler Memory Scale-Revised (WMS-R) (Wechsler, 1987) and the Halstead– Reitan battery (Reitan-Wolfson, 1985), found that Visual Reproduction Subtests I and II both loaded with measures of spatial analysis and reasoning rather than with measures of memory. Other studies have concluded that learning scores from tests of visual memory typically load with measures of visual-spatial processing and intelligence, but delayed
The authors would like to acknowledge the assistance of Glenn J. Larrabee for his comments and suggestions in the preparation of this manuscript. Address correspondence to: Raymond K. DiPino, Veterans Administration Maryland Health Care System, Baltimore, MD.
95
96
R. K. DiPino et al.
recall scores have a stronger loading with memory measures (Larrabee & Curtiss, 1995; Loring & Papanicolaou, 1987). With respect to problems assessing visual memory, two potential confounds have been suggested. First, visual memory is frequently assessed by having a subject recall patterns or duplicate designs. Hence, spatial perception and the ability to produce constructions appear to play a role in the learning and recall phases of these tests (Heilman & Valenstein, 1993. Second, most measures of visual memory also provide the opportunity for verbal encoding of information Lee, Loring & Thompson, 1989; Smith et al., 1992). However, it has been suggested that the use of delayed recall procedures may reduce the effects of verbal processing of visual memory stimuli (DeRenzi & Spinner, 1966; Ellis & Daniel, 1971; Larrabee, Trahan, & Curtiss, 1992). Although recall and reproduction of designs are involved in the Heaton Figure Memory Test (Heaton, Grant, & Matthews, 1991), its unique structure makes it of potential interest in view of the challenges associated with assessing visual memory. The test is described as a measure of nonverbal learning and delayed recall (Heaton et al., 1991). It uses the figures from the Visual Reproduction subtest of the original Wechsler Memory Scale (WMS; Wechsler, 1945). However, unlike the initial or revised WMS, in which immediate recall is tested for one card at a time, in the Heaton procedure the subject is shown each card for 10 seconds and is then asked to reproduce all four figures at the same time. During this acquisition phase, the four figures are presented until the subject reaches a criterion score of 15 points, or until the figures have been presented five times (Heaton et al., 1991), thereby providing an indication of rate of learning. Delayed recall is measured following a 4-hour delay. Thus, in contrast to many tests of visual memory in which the visual stimuli are presented only once, the Figure Memory Test is able to provide immediate recall, learning, and 4-hour delayed recall measures. As a result, the Heaton Figure Memory Test may provide a clearer measure of visual memory than other nonverbal memory procedures. Heaton et al. (1991) also provide a Story Memory Test, which parallels the Figure Memory Test with respect to method of administration and scoring. In both the Figure and Story Memory Tests, the learning score is the number of points obtained on the last administered learning trial divided by the number of trials administered. The memory score is a percentage reflecting the amount of acquired information that was forgotten over the 4-hour delay period (Heaton et al., 1991). Although not a formal part of the scoring, an examination of the number of points obtained on the initial learning trial can be conceptually understood as a measure of immediate memory. Thus, for both procedures, the first trial score can be considered a measure of immediate memory, the learning score a measure of acquisition, and the percent loss score a measure of retention. Scoring the test in this fashion allows for an examination of changes in factor loadings at different points in the learning and recall process. In addition, the use of a 4-hour delayed recall trial may mitigate the impact of the use of verbal cues in the acquisition and recall of visual information (DeRenzi & Spinner, 1966; Ellis & Daniel, 1971; Larrabee et al., 1992). The present study examines the construct validity of the Heaton Figure and Story Memory Tests (Heaton et al., 1991). It also explores the relationship between the Heaton memory procedures, the California Verbal Learning Test (CVLT; Delis, Kramer, Kaplan, & Ober, 1987), and eight measures of attention, word fluency, and spatial perception. In order to examine the factor structure of the Heaton memory measures, a factor analytic approach suggested by Larrabee et al. (1985), was utilized in which marker variables for attention, memory, verbal fluency, and visuospatial intelligence were included in the analyses. Regression analyses were also conducted in order to further examine the interrelationship between measures of immediate memory, learning, and delayed recall obtained from the Heaton techniques.
Construct Validity of the Heaton Memory Tests
97
It was expected that the immediate memory, learning, and delayed recall measures of the Story Memory Test would load with other verbal and memory measures. In contrast, immediate memory and learning measures of the Heaton Figure Memory Test were expected to load with measures of visual-spatial processing, supporting the visual nature of the Figure Memory procedure. Delayed recall measures from the Figure Memory Test were also expected to load with measures of visual-spatial processing, as well as with memory measures, supporting both visual processing and memory components to the procedure. The regression analyses were expected to further support a relationship between Figure Memory, and measures of spatial processing and memory.
METHOD Participants The study was a retrospective analysis of data obtained from 126 outpatients who had been seen for neuropsychological evaluation at the Baltimore division of the Veterans Administration (VA) Maryland Health Care System. Neuropsychological consultation was requested from a variety of specialty areas (e.g., neurology, psychiatry, substance abuse, primary care medicine) throughout the medical center, due to reported or suspected neurocognitive difficulties. All subjects selected for the study were physically capable of participating in a 4- to 5-hour neuropsychological evaluation. The subjects were predominately male (119 males, 7 females). They ranged in age from 20 to 83, with a mean age of 48.25 (614.72). Level of education among the subjects ranged from 5 to 21 years with a mean educational level of 12.40 years (63.05). There were 78 Caucasian, 45 African American, and 5 Hispanic subjects. All subjects were fluent in English. Design and Procedure In accordance with previous studies (Larrabee & Curtiss, 1995; Larrabee et al., 1992) and to satisfy the possibility of imperfect reliability among tests, measures of immediate memory, learning, and recall were submitted to an exploratory principal axis analysis (PAA). Assessment measures examined in the analysis included the Heaton Memory procedures (Heaton et al., 1991), the CVLT, the Digit Span, Arithmetic, Block Design, and Digit Symbol subtests from the Wechsler Adult Intelligence Scale-Revised (WAIS-R; Wechsler, 1981), the Trail Making Test part A (Reitan & Wolfson, 1985), Controlled Oral Word Association Test (COWA; Benton & Hamsher, 1989), Animal Naming (Goodglass & Kaplan, 1983), and Judgement of Line Orientation (Benton, Sivan, Hamsher, Varney, & Spreen, 1994) (see Table 1). An orthogonal rotation (varimax) was utilized to clarify the initial factor structure. Measures were assigned to the factor on which they loaded the highest (based upon the criterion of loadings ..30). The number of factors for extraction was determined by eigenvalue criteria (.1) and scree plot analysis (Gorsuch, 1983; Tinsley & Tinsley, 1987). In order to prevent spurious findings related to “method variance” or high intercorrelations among different scores from the same test (Larrabee & Curtiss, 1995; Smith et al., 1992), only one score from each test was used in each PAA. As a result, no overlapping measures were included in the same analysis. Listwise deletion was employed to handle missing data. To further ascertain the interrelationship among measures of immediate memory, learning, and recall, linear regression analyses were conducted subsequent to the PAA. Regressions were performed utilizing the stepwise (forward) method of model fit. Independent variables were included in the regression equation if the probability of F was
98
R. K. DiPino et al.
TABLE 1 Neurocognitive Measures Included in the Factor Analysis Test
Figure Memory Trial 1a Story Memory Trial 1 CVLT Trial 1 Figure Memory Learning Scoreb Story Memory Learning Score CVLT Trials 1-5 Figure Memory Lossc Story Memory Loss CVLT Percent Lost WAIS-R Backward Digit Spand WAIS-R Arithmetic COWAd Animal Naming Judgment of Line Orientationd WAIS-R Block Design WAIS-R Digit Symbol Trails A
Presumed Neurocognitive Domain Measured
Immediate Visual Memory Immediate Verbal Memory Immediate Verbal Memory Visual Learning Verbal (Narrative) Learning Verbal (List) Learning Visual/Nonverbal Memory Verbal Memory Verbal Memory Working Memory Working Memory Verbal Fluency Verbal Fluency Spatial Analysis Spatial Analysis/Synthesis Perceptual Motor Speed/Accuracy Visual Tracking
CVLT 5 California Verbal Learning Test; WAIS-R 5 Wechsler Adult Intelligence ScaleRevised; COWA 5 Controlled Oral Word Association. aMeasures included only in analyses of Immediate Memory. bMeasures included only in analyses of Acquisition. cMeasures included only in analyses of Memory. dMarker variables included in each analyses.
less than or equal to .05. Independent variables were removed from the equation if the probability of F was .10 or greater. Listwise deletion was again employed to handle missing data. All data were analyzed using SPSS for Windows, Version 7.5 (SPSS, Inc., 1997).
RESULTS Results of the factor analysis of immediate memory measures revealed a three-factor solution accounting for 70.5% of the variance (see Table 2). Factor one consisted of measures of verbal working memory (WAIS-R Backward Digit Span), verbal fluency (Animal Naming, COWA), and immediate verbal memory (the initial learning trials of both the CVLT and Story Memory). Factor two consisted of measures of nonverbal processing (Trial 1 of the Figure Memory Test, Trails A, and WAIS-R Digit Symbol). Factor three was composed of measures of working memory (WAIS-R Arithmetic) and spatial processing (WAIS-R Block Design and Judgement of Line Orientation). Results of the factor analysis of learning measures are presented in Table 3. As can be seen, a three-factor solution accounting for 69.7% of the variance was obtained. Factor one was composed of measures of spatial perception (Judgement of Line Orientation, WAIS-R Block Design) and working memory (WAIS-R Arithmetic and WAIS-R Backward Digit Span), as well as the learning score of the Figure Memory test. Factor two was composed of measures presumed to assess focused attention (Trails A and WAIS-R Digit Symbol). Factor Three consisted of measures of verbal learning (CVLT Trials 1–5, Story Learning) and measures of verbal fluency (Animal Naming, COWA). A three-factor solution, accounting for 71.6% of the variance was also obtained for measures of Delayed Recall (see Table 4). The first factor of the solution was presumed to represent nonverbal processing and was similar to factor two of the analysis of imme-
Construct Validity of the Heaton Memory Tests
99
TABLE 2 Results of Factor Analysis of Immediate Memory Measures Factor Measure
I.
CVLT Trial 1 Animal Naming COWA (CFL) Backward Digit Span Story Memory Trial 1 II. Digit Symbol Trails A Figure Memory Trial 1 III. Block Design JLO Arithmetic Cumulative variance
Loadinga
Variance Accounted
.71 .67 .61 .49 .36 .82 2.82 .54 .75 .70 .60
49.1
12.2 9.2
70.5
aLoadings are reported in the order of extraction and rounded to the nearest 100th place. CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association; JLO 5 Judgment of Line Orientation.
diate memory measures (Figure Memory Loss, Trails A, and WAIS-R Digit Symbol). Factor two included measures of spatial perception (Judgment of Line Orientation, WAIS-R Block Design) and WAIS-R Arithmetic, possibly reflecting a common working memory or focused attention component as in the previous factor analysis. Verbal recall measures (CVLT loss, Story Memory loss) loaded on factor three, along with measures of verbal fluency (COWA, Animal Naming). Interestingly, a component of working memory also loaded on this factor (WAIS-R Backward Digit Span). Results of the regression analysis for immediate memory measures (Table 5) revealed that the best predictor of Trial 1 of Story Memory was performance on the other immediate verbal memory measure, Trial 1 of the CVLT. Story Memory performance was not predicted by measures of working memory or by verbal fluency measures. The best predictors of initial trial recall on Figure Memory were WAIS-R Block Design and CVLT Trial 1, suggesting both a spatial processing and memory component to the task. Interestingly, the best predictor of performance on Trial 1 of the CVLT was performance on Trial 1 of Figure Memory. Performance on other measures of verbal acquisition, verbal fluency, or attention did not significantly predict first trial CVLT performance. Results of the regression analyses for learning measures are presented in Table 6. As can be seen, the best predictor of Story Memory learning was the learning score of the CVLT. Figure Memory learning scores were predicted by two measures with a visuospatial component, WAIS-R Block Design and WAIS-R Digit Symbol, and not by verbal learning measures. As with immediate memory measures, CVLT Learning scores were again predicted by performance on Trial 1 of Figure Memory. However, Story Memory learning and performance of a verbal fluency measure (COWA) also significantly predicted CVLT learning performance. Results of the regression analyses for recall measures are presented in Table 7. Results indicated that the best predictors of Story Memory recall were WAIS-R Backward Digit Span, Judgment of Line Orientation, and recall on the CVLT, reflecting both attention and memory components. Figure Memory recall was also significantly predicted by tests reflecting attention and memory, Trails A and CVLT recall, but not by tests measuring primarily visuospatial abilities. CVLT recall was significantly predicted by COWA scores and, consistent with previous analyses, by Figure Memory scores.
100
R. K. DiPino et al. TABLE 3 Results of Factor Analysis of Learning Measures Factor Measure
I.
Block Design Arithmetic JLO Figure Memory Learning Backward Digit Span II. Trails A Digit Symbol III. CVLT Trials 1–5 COWA (CFL) Animal Naming Story Memory Learning
Loadinga
Variance Accounted
.76 .65 .63 .55 .50 2.83 .82 .82 .63 .61 .36
48.5
12.0 9.2
Cumulative Variance
69.7
aLoadings are reported in the order of extraction and rounded to the nearest 100th place. JLO 5 Judgment of Line Orientation; CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association.
DISCUSSION In the present study, factor analysis of immediate memory measures from the Heaton Memory Tests resulted in Figure Memory loading with measures of visual attention. Story Memory loaded with measures more reflective of verbal attention and fluency (e.g., the first trial performance of the CVLT, Animal Naming). This finding is consistent with previous factor analytic studies (e.g., Larrabee & Curtiss, 1995). However, visuospatial marker variables did not group together to form a separate factor and Figure Memory did not load with measures of spatial analysis such as Judgment of Line Orientation or Block Design. Although initial trial learning of designs seemed related to visual attention, learning over trials had a stronger relationship with measures demanding greater analysis and synthesis of visuospatial information. In the factor analysis of presumed learning measures, Figure Learning clustered with measures such as Judgment of Line Orientation
TABLE 4 Results of Factor Analysis of Delayed Recall Measures Factor Measure
I.
Figure Loss Trails A Digit Symbol II. Block Design JLO Arithmetic III. COWA Animal Naming CVLT Percent Loss Backward Digit Span Story Loss Cumulative Variance aLoadings
Loadinga
.81 .79 2.73 .76 .73 .58 .80 .60 2.52 .49 2.48
Variance Accounted
46.4 16.2 9.1
71.6
are reported in the order of extraction and rounded to the nearest 100th place. JLO 5 Judgment of Line Orientation; COWA 5 Controlled Oral Word Association; CVLT 5 California Verbal Learning Test.
Construct Validity of the Heaton Memory Tests
101
TABLE 5 Variables Predicting Performance on Immediate Memory Measures Dependent Variable
Story Memory Trial 1 Figure Memory Trial 1 CVLT Trial 1
Predictor
b
T
CVLT Trial 1 Block Design CVLT Trial 1 Figure Memory Trial 1
.43 .50 .45 .64
2.94* 4.59** 4.09** 5.20**
CVLT 5 California Verbal Learning Test. *p , .01. **p , .001.
and Block Design rather than Trails A and Digit Symbol. Story Learning measures continued to load with the CVLT and measures of verbal fluency. Examination of the factor analysis of Figure Memory delayed recall measures did not support the concept of separate factors being involved in acquisition and recall of visual information. In fact, the results revealed that Figure Memory delayed recall loaded with the same measures as Figure Memory trial one, Trails A, and Digit Symbol, suggesting that the efficiency of initial encoding of visual information was related to later recall. Percent loss scores for the CVLT and the Heaton story both contributed to a factor associated with verbal recall. However, measures of working memory and verbal fluency also contributed to this factor. Previous research has suggested that visual recall measures load more heavily on a memory factor than do visual learning measures (e.g., Larrabee & Curtiss, 1995; Loring & Papanicolaou, 1987). Yet the results of the current factor analysis do not appear to support a visual memory factor within the Heaton Figure Memory test distinct from measures of visuospatial processing and visual learning. There are several possible reasons why a distinct visual memory factor did not emerge. First, this was a retrospective study based on available test data. As a consequence, the analysis did not contain a second presumed nonverbal memory task to help bolster the formation of an independent factor. Second, most patients used in the sample had generalized rather than focal difficulties. These difficulties typically involved problems with aspects of attention that impacted their performance of other neurocognitive tasks. Hence, a strong influence from tests of attention and performance efficiency may
TABLE 6 Variables Predicting Performance on Learning Measures Dependent Variable
Story Memory Learning Figure Memory Learning CVLT Trial 1–5
Predictor
b
T
CVLT Trial 1–5 Block Design Digit Symbol COWA (CFL) Figure Memory Trial 1 Story Memory Learning
.51 .50 .29 .32 .39 .26
3.65*** 3.55** 2.10* 2.61* 2.88** 2.04*
CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association;. *p , .05. **p , .01. ***p , .001.
102
R. K. DiPino et al. TABLE 7 Variables Predicting Performance on Delayed Recall Measures Dependent Variable
Predictor
b
T
Story Memory Loss
Backward Digit Span JLO CVLT Loss Trails A CVLT Loss COWA Figure Loss
2.44 2.30 .28 .49 .29 2.43 .48
23.73*** 22.61* 2.41 3.70*** 2.15* 23.50** 3.84***
Figure Memory Loss CVLT Loss
JLO 5 Judgment of Line Orientation; CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association. *p , .05. **p , .01. ***p , .001.
have appeared in the factor solutions. Third, in contrast to other studies that have demonstrated a distinct verbal memory factor (Larrabee & Curtiss, 1995; Larrabee et al., 1985; Larrabee et al., 1992; Smith et al., 1992), marker variables in the current study did not clearly define factors of verbal and nonverbal intelligence. Finally, the Heaton procedure may also be vulnerable to the complexities associated with assessing nonverbal memory in general. Results of the regression analyses suggested greater shared variance between measures of visual and verbal attention and learning than was suggested by the factor analyses. The regression analyses also suggested that attention and memory play a role in both visual and verbal recall and that verbal memory factors play a role in predicting performance on a measure of visual memory. Overall, the results of the analysis appear to support the construct validity of the Heaton Story Memory procedure and the independence of Figure and Story Learning. The results suggest that the Heaton Figure and Story Memory tests each assess different components of memory. Figure and Story Memory measures did not load together on any of the factor analyses and did not significantly predict each other on any of the regression analyses. However, the variance each shares with the CVLT suggests that each evaluates a different aspect of memory performance and supports the validity of the Figure Memory test as a measure of memory. The current study was exploratory in nature and involved a retrospective analysis of data collected from 126 subjects who had been seen for evaluation over several months. It included marker variables measuring attention, verbal fluency, and several measures of spatial analysis and synthesis. Aside from the Heaton tests, the only other memory procedure included was a measure of verbal memory, the CVLT. It is possible that including other marker variables in the factor analysis, especially an additional measure of nonverbal memory, may have yielded stronger support for a distinct visual memory component in the factor analysis. Future research designed to examine the factor structure of the Heaton memory procedures may benefit from the inclusion of additional nonverbal memory measures as well as additional measures of verbal and nonverbal intelligence. The inclusion of these additional variables may contribute to greater understanding of the factor structure of the Heaton Memory procedures. Future studies including a wider range of marker variables may be useful not only in further supporting the construct validity of the Heaton Memory Tests, but also in providing support for a distinction between measures of verbal and visual memory.
Construct Validity of the Heaton Memory Tests
103
REFERENCES Benton, A. L., Sivan, A. B., Hamsher, K. D., Varney, N. R., & Spreen, O. (1994). Contributions to neuropsychological assessment (2nd ed.). New York: Oxford University Press. Benton, A. L., & Hamsher, K. D. (1989). Multilingual Aphasia Examination: Manual of instructions (2nd ed.). Iowa City, IO: AJA Associates. Delis, D., Kramer, J., Kaplan, E., & Ober, B. (1987). California Verbal Learning Test manual (research ed.). San Antonio, TX: The Psychological Corporation. DeRenzi, E., & Spinner, H. (1966). Visual recognition in patients with unilateral cerebral disease. Journal of Nervous and Mental Disease, 142, 515–525. Ellis, H., & Daniel, T. (1971). Verbal process in long term stimulus recognition memory. Journal of Experimental Psychology, 90, 18–26. Goodglass, H., & Kaplan, E. (1983). The assessment of aphasia and related disorders (2nd ed.). Philadelphia, PA: Lea & Febiger. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Heaton, R., Grant, I., & Matthews, C. (1991). Comprehensive norms for an expanded Halstead-Reitan Battery: Demographic corrections, research findings, and clinical applications. Odessa, FL.: Psychological Assessment Resources. Heilman, K. M., & Valenstein, E. (1993). Clinical neuropsychology (3rd ed.). New York: Oxford University Press. Larrabee, G. J., & Curtiss, G. (1995). Construct validity of various verbal and visual memory tests. Journal of Clinical and Experimental Neuropsychology, 17, 536–547. Larrabee, G. J., Kane, R., Schuck, J., & Francis, D. (1985). Construct validity of various memory testing procedures. Journal of Clinical and Experimental Neuropsychology, 7, 239–250. Larrabee, G. J., Trahan, D. E., & Curtis, G. (1992). Construct validity of the continuous visual memory test. Archives of Clinical Neuropsychology, 7, 395–405. Lee, G., Loring, D., & Thompson, J. (1989). Construct validity of material-specific memory measures following unilateral temporal lobe ablations. Psychological Assessment, 1, 192–197. Leonberger, T., Nicks, S., Goldfader, P., & Munz, D. (1991). Factor analysis of the Wechsler Memory ScaleRevised and the Halstead-Reitan Neuropsychological Battery. The Clinical Neuropsychologist, 5, 83–88. Loring, D., & Papanicolaou, A. (1987). Memory assessment in neuropsychology: Theoretical considerations and practical utility. Journal of Clinical and Experimental Neuropsychology, 9, 340–358. Reitan, R. M., & Wolfson, D. (1985). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press. Smith, G., Malec, J., & Ivnik, R. (1992). Validity of the construct of nonverbal memory: A factor-analytic study in a normal elderly sample. Journal of Clinical and Experimental Neuropsychology, 14, 211–221. SPSS Inc. (1997). SPSS 7.5 for Windows. Chicago, IL: Author. Tinsley, H. E. A., & Tinsley, D. J. (1987). Uses of factor analysis in counseling psychology research. Journal of Counseling Psychology, 34(4), 14-42. Wechsler, D. (1945). A standardized memory scale for clinical use. The Journal of Psychology, 19, 87–95. Wechsler, D. (1981). WAIS-R manual. New York: Psychological Corporation. Wechsler, D. (1987). Wechsler Memory Scale-Revised Manual. San Antonio, TX: Psychological Corporation.
PII S0887-6177(98)00152-8
An Exploration of the Construct Validity of the Heaton Memory Tests Raymond K. DiPino, Michael H. Kabat, and Robert L. Kane Veterans Administration Maryland Health Care System
The Heaton techniques for assessing memory differ from most other memory measures by employing the combination of learning trials, repeated exposure to test stimuli, and 4-hour recall measures. The present study was designed to explore the relationship between the Heaton Story and Figure memory procedures, the California Verbal Learning Test (CVLT), and measures of attention, word fluency, and spatial perception. Data from 126 individuals were analyzed. Three separate factor analyses were performed that examined measures of attention, learning, and recall. Each revealed a three-factor solution accounting for 71, 70, and 72% of the variance, respectively. Regression analyses supported the visual components of Figure Memory and the verbal components of Story Memory obtained in the factor analyses. Overall, the findings supported the independence of the Heaton Memory procedures and the role of spatial factors in performance of Figure Memory. Additionally, both Figure and Story Memory shared variance with the CVLT, supporting the validity of both procedures as memory measures. © 2000 National Academy of Neuropsychology. Published by Elsevier Science Ltd
Although most clinical measures of memory maintain a distinction between verbal and visual memory (Smith, Malec, & Ivnik, 1992), factor-analytic studies examining these measures have not consistently found distinct verbal and visual factors (Smith et al., 1992; Larrabee & Curtiss, 1995). In general, verbal memory tests have been found to load on a general or verbal memory factor, with immediate and delayed recall measures producing their strongest contribution to a memory factor (Larrabee & Curtiss, 1995; Larrabee, Kane, Schuck, & Francis, 1985; Loring & Papanicolaou, 1987; Smith et al., 1992). In contrast, examinations of visual memory tests have provided less consistent results. For example, Leonberger, Nicks, Goldfader, and Munz (1991), in an examination of the Wechsler Memory Scale-Revised (WMS-R) (Wechsler, 1987) and the Halstead– Reitan battery (Reitan-Wolfson, 1985), found that Visual Reproduction Subtests I and II both loaded with measures of spatial analysis and reasoning rather than with measures of memory. Other studies have concluded that learning scores from tests of visual memory typically load with measures of visual-spatial processing and intelligence, but delayed
The authors would like to acknowledge the assistance of Glenn J. Larrabee for his comments and suggestions in the preparation of this manuscript. Address correspondence to: Raymond K. DiPino, Veterans Administration Maryland Health Care System, Baltimore, MD.
95
96
R. K. DiPino et al.
recall scores have a stronger loading with memory measures (Larrabee & Curtiss, 1995; Loring & Papanicolaou, 1987). With respect to problems assessing visual memory, two potential confounds have been suggested. First, visual memory is frequently assessed by having a subject recall patterns or duplicate designs. Hence, spatial perception and the ability to produce constructions appear to play a role in the learning and recall phases of these tests (Heilman & Valenstein, 1993. Second, most measures of visual memory also provide the opportunity for verbal encoding of information Lee, Loring & Thompson, 1989; Smith et al., 1992). However, it has been suggested that the use of delayed recall procedures may reduce the effects of verbal processing of visual memory stimuli (DeRenzi & Spinner, 1966; Ellis & Daniel, 1971; Larrabee, Trahan, & Curtiss, 1992). Although recall and reproduction of designs are involved in the Heaton Figure Memory Test (Heaton, Grant, & Matthews, 1991), its unique structure makes it of potential interest in view of the challenges associated with assessing visual memory. The test is described as a measure of nonverbal learning and delayed recall (Heaton et al., 1991). It uses the figures from the Visual Reproduction subtest of the original Wechsler Memory Scale (WMS; Wechsler, 1945). However, unlike the initial or revised WMS, in which immediate recall is tested for one card at a time, in the Heaton procedure the subject is shown each card for 10 seconds and is then asked to reproduce all four figures at the same time. During this acquisition phase, the four figures are presented until the subject reaches a criterion score of 15 points, or until the figures have been presented five times (Heaton et al., 1991), thereby providing an indication of rate of learning. Delayed recall is measured following a 4-hour delay. Thus, in contrast to many tests of visual memory in which the visual stimuli are presented only once, the Figure Memory Test is able to provide immediate recall, learning, and 4-hour delayed recall measures. As a result, the Heaton Figure Memory Test may provide a clearer measure of visual memory than other nonverbal memory procedures. Heaton et al. (1991) also provide a Story Memory Test, which parallels the Figure Memory Test with respect to method of administration and scoring. In both the Figure and Story Memory Tests, the learning score is the number of points obtained on the last administered learning trial divided by the number of trials administered. The memory score is a percentage reflecting the amount of acquired information that was forgotten over the 4-hour delay period (Heaton et al., 1991). Although not a formal part of the scoring, an examination of the number of points obtained on the initial learning trial can be conceptually understood as a measure of immediate memory. Thus, for both procedures, the first trial score can be considered a measure of immediate memory, the learning score a measure of acquisition, and the percent loss score a measure of retention. Scoring the test in this fashion allows for an examination of changes in factor loadings at different points in the learning and recall process. In addition, the use of a 4-hour delayed recall trial may mitigate the impact of the use of verbal cues in the acquisition and recall of visual information (DeRenzi & Spinner, 1966; Ellis & Daniel, 1971; Larrabee et al., 1992). The present study examines the construct validity of the Heaton Figure and Story Memory Tests (Heaton et al., 1991). It also explores the relationship between the Heaton memory procedures, the California Verbal Learning Test (CVLT; Delis, Kramer, Kaplan, & Ober, 1987), and eight measures of attention, word fluency, and spatial perception. In order to examine the factor structure of the Heaton memory measures, a factor analytic approach suggested by Larrabee et al. (1985), was utilized in which marker variables for attention, memory, verbal fluency, and visuospatial intelligence were included in the analyses. Regression analyses were also conducted in order to further examine the interrelationship between measures of immediate memory, learning, and delayed recall obtained from the Heaton techniques.
Construct Validity of the Heaton Memory Tests
97
It was expected that the immediate memory, learning, and delayed recall measures of the Story Memory Test would load with other verbal and memory measures. In contrast, immediate memory and learning measures of the Heaton Figure Memory Test were expected to load with measures of visual-spatial processing, supporting the visual nature of the Figure Memory procedure. Delayed recall measures from the Figure Memory Test were also expected to load with measures of visual-spatial processing, as well as with memory measures, supporting both visual processing and memory components to the procedure. The regression analyses were expected to further support a relationship between Figure Memory, and measures of spatial processing and memory.
METHOD Participants The study was a retrospective analysis of data obtained from 126 outpatients who had been seen for neuropsychological evaluation at the Baltimore division of the Veterans Administration (VA) Maryland Health Care System. Neuropsychological consultation was requested from a variety of specialty areas (e.g., neurology, psychiatry, substance abuse, primary care medicine) throughout the medical center, due to reported or suspected neurocognitive difficulties. All subjects selected for the study were physically capable of participating in a 4- to 5-hour neuropsychological evaluation. The subjects were predominately male (119 males, 7 females). They ranged in age from 20 to 83, with a mean age of 48.25 (614.72). Level of education among the subjects ranged from 5 to 21 years with a mean educational level of 12.40 years (63.05). There were 78 Caucasian, 45 African American, and 5 Hispanic subjects. All subjects were fluent in English. Design and Procedure In accordance with previous studies (Larrabee & Curtiss, 1995; Larrabee et al., 1992) and to satisfy the possibility of imperfect reliability among tests, measures of immediate memory, learning, and recall were submitted to an exploratory principal axis analysis (PAA). Assessment measures examined in the analysis included the Heaton Memory procedures (Heaton et al., 1991), the CVLT, the Digit Span, Arithmetic, Block Design, and Digit Symbol subtests from the Wechsler Adult Intelligence Scale-Revised (WAIS-R; Wechsler, 1981), the Trail Making Test part A (Reitan & Wolfson, 1985), Controlled Oral Word Association Test (COWA; Benton & Hamsher, 1989), Animal Naming (Goodglass & Kaplan, 1983), and Judgement of Line Orientation (Benton, Sivan, Hamsher, Varney, & Spreen, 1994) (see Table 1). An orthogonal rotation (varimax) was utilized to clarify the initial factor structure. Measures were assigned to the factor on which they loaded the highest (based upon the criterion of loadings ..30). The number of factors for extraction was determined by eigenvalue criteria (.1) and scree plot analysis (Gorsuch, 1983; Tinsley & Tinsley, 1987). In order to prevent spurious findings related to “method variance” or high intercorrelations among different scores from the same test (Larrabee & Curtiss, 1995; Smith et al., 1992), only one score from each test was used in each PAA. As a result, no overlapping measures were included in the same analysis. Listwise deletion was employed to handle missing data. To further ascertain the interrelationship among measures of immediate memory, learning, and recall, linear regression analyses were conducted subsequent to the PAA. Regressions were performed utilizing the stepwise (forward) method of model fit. Independent variables were included in the regression equation if the probability of F was
98
R. K. DiPino et al.
TABLE 1 Neurocognitive Measures Included in the Factor Analysis Test
Figure Memory Trial 1a Story Memory Trial 1 CVLT Trial 1 Figure Memory Learning Scoreb Story Memory Learning Score CVLT Trials 1-5 Figure Memory Lossc Story Memory Loss CVLT Percent Lost WAIS-R Backward Digit Spand WAIS-R Arithmetic COWAd Animal Naming Judgment of Line Orientationd WAIS-R Block Design WAIS-R Digit Symbol Trails A
Presumed Neurocognitive Domain Measured
Immediate Visual Memory Immediate Verbal Memory Immediate Verbal Memory Visual Learning Verbal (Narrative) Learning Verbal (List) Learning Visual/Nonverbal Memory Verbal Memory Verbal Memory Working Memory Working Memory Verbal Fluency Verbal Fluency Spatial Analysis Spatial Analysis/Synthesis Perceptual Motor Speed/Accuracy Visual Tracking
CVLT 5 California Verbal Learning Test; WAIS-R 5 Wechsler Adult Intelligence ScaleRevised; COWA 5 Controlled Oral Word Association. aMeasures included only in analyses of Immediate Memory. bMeasures included only in analyses of Acquisition. cMeasures included only in analyses of Memory. dMarker variables included in each analyses.
less than or equal to .05. Independent variables were removed from the equation if the probability of F was .10 or greater. Listwise deletion was again employed to handle missing data. All data were analyzed using SPSS for Windows, Version 7.5 (SPSS, Inc., 1997).
RESULTS Results of the factor analysis of immediate memory measures revealed a three-factor solution accounting for 70.5% of the variance (see Table 2). Factor one consisted of measures of verbal working memory (WAIS-R Backward Digit Span), verbal fluency (Animal Naming, COWA), and immediate verbal memory (the initial learning trials of both the CVLT and Story Memory). Factor two consisted of measures of nonverbal processing (Trial 1 of the Figure Memory Test, Trails A, and WAIS-R Digit Symbol). Factor three was composed of measures of working memory (WAIS-R Arithmetic) and spatial processing (WAIS-R Block Design and Judgement of Line Orientation). Results of the factor analysis of learning measures are presented in Table 3. As can be seen, a three-factor solution accounting for 69.7% of the variance was obtained. Factor one was composed of measures of spatial perception (Judgement of Line Orientation, WAIS-R Block Design) and working memory (WAIS-R Arithmetic and WAIS-R Backward Digit Span), as well as the learning score of the Figure Memory test. Factor two was composed of measures presumed to assess focused attention (Trails A and WAIS-R Digit Symbol). Factor Three consisted of measures of verbal learning (CVLT Trials 1–5, Story Learning) and measures of verbal fluency (Animal Naming, COWA). A three-factor solution, accounting for 71.6% of the variance was also obtained for measures of Delayed Recall (see Table 4). The first factor of the solution was presumed to represent nonverbal processing and was similar to factor two of the analysis of imme-
Construct Validity of the Heaton Memory Tests
99
TABLE 2 Results of Factor Analysis of Immediate Memory Measures Factor Measure
I.
CVLT Trial 1 Animal Naming COWA (CFL) Backward Digit Span Story Memory Trial 1 II. Digit Symbol Trails A Figure Memory Trial 1 III. Block Design JLO Arithmetic Cumulative variance
Loadinga
Variance Accounted
.71 .67 .61 .49 .36 .82 2.82 .54 .75 .70 .60
49.1
12.2 9.2
70.5
aLoadings are reported in the order of extraction and rounded to the nearest 100th place. CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association; JLO 5 Judgment of Line Orientation.
diate memory measures (Figure Memory Loss, Trails A, and WAIS-R Digit Symbol). Factor two included measures of spatial perception (Judgment of Line Orientation, WAIS-R Block Design) and WAIS-R Arithmetic, possibly reflecting a common working memory or focused attention component as in the previous factor analysis. Verbal recall measures (CVLT loss, Story Memory loss) loaded on factor three, along with measures of verbal fluency (COWA, Animal Naming). Interestingly, a component of working memory also loaded on this factor (WAIS-R Backward Digit Span). Results of the regression analysis for immediate memory measures (Table 5) revealed that the best predictor of Trial 1 of Story Memory was performance on the other immediate verbal memory measure, Trial 1 of the CVLT. Story Memory performance was not predicted by measures of working memory or by verbal fluency measures. The best predictors of initial trial recall on Figure Memory were WAIS-R Block Design and CVLT Trial 1, suggesting both a spatial processing and memory component to the task. Interestingly, the best predictor of performance on Trial 1 of the CVLT was performance on Trial 1 of Figure Memory. Performance on other measures of verbal acquisition, verbal fluency, or attention did not significantly predict first trial CVLT performance. Results of the regression analyses for learning measures are presented in Table 6. As can be seen, the best predictor of Story Memory learning was the learning score of the CVLT. Figure Memory learning scores were predicted by two measures with a visuospatial component, WAIS-R Block Design and WAIS-R Digit Symbol, and not by verbal learning measures. As with immediate memory measures, CVLT Learning scores were again predicted by performance on Trial 1 of Figure Memory. However, Story Memory learning and performance of a verbal fluency measure (COWA) also significantly predicted CVLT learning performance. Results of the regression analyses for recall measures are presented in Table 7. Results indicated that the best predictors of Story Memory recall were WAIS-R Backward Digit Span, Judgment of Line Orientation, and recall on the CVLT, reflecting both attention and memory components. Figure Memory recall was also significantly predicted by tests reflecting attention and memory, Trails A and CVLT recall, but not by tests measuring primarily visuospatial abilities. CVLT recall was significantly predicted by COWA scores and, consistent with previous analyses, by Figure Memory scores.
100
R. K. DiPino et al. TABLE 3 Results of Factor Analysis of Learning Measures Factor Measure
I.
Block Design Arithmetic JLO Figure Memory Learning Backward Digit Span II. Trails A Digit Symbol III. CVLT Trials 1–5 COWA (CFL) Animal Naming Story Memory Learning
Loadinga
Variance Accounted
.76 .65 .63 .55 .50 2.83 .82 .82 .63 .61 .36
48.5
12.0 9.2
Cumulative Variance
69.7
aLoadings are reported in the order of extraction and rounded to the nearest 100th place. JLO 5 Judgment of Line Orientation; CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association.
DISCUSSION In the present study, factor analysis of immediate memory measures from the Heaton Memory Tests resulted in Figure Memory loading with measures of visual attention. Story Memory loaded with measures more reflective of verbal attention and fluency (e.g., the first trial performance of the CVLT, Animal Naming). This finding is consistent with previous factor analytic studies (e.g., Larrabee & Curtiss, 1995). However, visuospatial marker variables did not group together to form a separate factor and Figure Memory did not load with measures of spatial analysis such as Judgment of Line Orientation or Block Design. Although initial trial learning of designs seemed related to visual attention, learning over trials had a stronger relationship with measures demanding greater analysis and synthesis of visuospatial information. In the factor analysis of presumed learning measures, Figure Learning clustered with measures such as Judgment of Line Orientation
TABLE 4 Results of Factor Analysis of Delayed Recall Measures Factor Measure
I.
Figure Loss Trails A Digit Symbol II. Block Design JLO Arithmetic III. COWA Animal Naming CVLT Percent Loss Backward Digit Span Story Loss Cumulative Variance aLoadings
Loadinga
.81 .79 2.73 .76 .73 .58 .80 .60 2.52 .49 2.48
Variance Accounted
46.4 16.2 9.1
71.6
are reported in the order of extraction and rounded to the nearest 100th place. JLO 5 Judgment of Line Orientation; COWA 5 Controlled Oral Word Association; CVLT 5 California Verbal Learning Test.
Construct Validity of the Heaton Memory Tests
101
TABLE 5 Variables Predicting Performance on Immediate Memory Measures Dependent Variable
Story Memory Trial 1 Figure Memory Trial 1 CVLT Trial 1
Predictor
b
T
CVLT Trial 1 Block Design CVLT Trial 1 Figure Memory Trial 1
.43 .50 .45 .64
2.94* 4.59** 4.09** 5.20**
CVLT 5 California Verbal Learning Test. *p , .01. **p , .001.
and Block Design rather than Trails A and Digit Symbol. Story Learning measures continued to load with the CVLT and measures of verbal fluency. Examination of the factor analysis of Figure Memory delayed recall measures did not support the concept of separate factors being involved in acquisition and recall of visual information. In fact, the results revealed that Figure Memory delayed recall loaded with the same measures as Figure Memory trial one, Trails A, and Digit Symbol, suggesting that the efficiency of initial encoding of visual information was related to later recall. Percent loss scores for the CVLT and the Heaton story both contributed to a factor associated with verbal recall. However, measures of working memory and verbal fluency also contributed to this factor. Previous research has suggested that visual recall measures load more heavily on a memory factor than do visual learning measures (e.g., Larrabee & Curtiss, 1995; Loring & Papanicolaou, 1987). Yet the results of the current factor analysis do not appear to support a visual memory factor within the Heaton Figure Memory test distinct from measures of visuospatial processing and visual learning. There are several possible reasons why a distinct visual memory factor did not emerge. First, this was a retrospective study based on available test data. As a consequence, the analysis did not contain a second presumed nonverbal memory task to help bolster the formation of an independent factor. Second, most patients used in the sample had generalized rather than focal difficulties. These difficulties typically involved problems with aspects of attention that impacted their performance of other neurocognitive tasks. Hence, a strong influence from tests of attention and performance efficiency may
TABLE 6 Variables Predicting Performance on Learning Measures Dependent Variable
Story Memory Learning Figure Memory Learning CVLT Trial 1–5
Predictor
b
T
CVLT Trial 1–5 Block Design Digit Symbol COWA (CFL) Figure Memory Trial 1 Story Memory Learning
.51 .50 .29 .32 .39 .26
3.65*** 3.55** 2.10* 2.61* 2.88** 2.04*
CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association;. *p , .05. **p , .01. ***p , .001.
102
R. K. DiPino et al. TABLE 7 Variables Predicting Performance on Delayed Recall Measures Dependent Variable
Predictor
b
T
Story Memory Loss
Backward Digit Span JLO CVLT Loss Trails A CVLT Loss COWA Figure Loss
2.44 2.30 .28 .49 .29 2.43 .48
23.73*** 22.61* 2.41 3.70*** 2.15* 23.50** 3.84***
Figure Memory Loss CVLT Loss
JLO 5 Judgment of Line Orientation; CVLT 5 California Verbal Learning Test; COWA 5 Controlled Oral Word Association. *p , .05. **p , .01. ***p , .001.
have appeared in the factor solutions. Third, in contrast to other studies that have demonstrated a distinct verbal memory factor (Larrabee & Curtiss, 1995; Larrabee et al., 1985; Larrabee et al., 1992; Smith et al., 1992), marker variables in the current study did not clearly define factors of verbal and nonverbal intelligence. Finally, the Heaton procedure may also be vulnerable to the complexities associated with assessing nonverbal memory in general. Results of the regression analyses suggested greater shared variance between measures of visual and verbal attention and learning than was suggested by the factor analyses. The regression analyses also suggested that attention and memory play a role in both visual and verbal recall and that verbal memory factors play a role in predicting performance on a measure of visual memory. Overall, the results of the analysis appear to support the construct validity of the Heaton Story Memory procedure and the independence of Figure and Story Learning. The results suggest that the Heaton Figure and Story Memory tests each assess different components of memory. Figure and Story Memory measures did not load together on any of the factor analyses and did not significantly predict each other on any of the regression analyses. However, the variance each shares with the CVLT suggests that each evaluates a different aspect of memory performance and supports the validity of the Figure Memory test as a measure of memory. The current study was exploratory in nature and involved a retrospective analysis of data collected from 126 subjects who had been seen for evaluation over several months. It included marker variables measuring attention, verbal fluency, and several measures of spatial analysis and synthesis. Aside from the Heaton tests, the only other memory procedure included was a measure of verbal memory, the CVLT. It is possible that including other marker variables in the factor analysis, especially an additional measure of nonverbal memory, may have yielded stronger support for a distinct visual memory component in the factor analysis. Future research designed to examine the factor structure of the Heaton memory procedures may benefit from the inclusion of additional nonverbal memory measures as well as additional measures of verbal and nonverbal intelligence. The inclusion of these additional variables may contribute to greater understanding of the factor structure of the Heaton Memory procedures. Future studies including a wider range of marker variables may be useful not only in further supporting the construct validity of the Heaton Memory Tests, but also in providing support for a distinction between measures of verbal and visual memory.
Construct Validity of the Heaton Memory Tests
103
REFERENCES Benton, A. L., Sivan, A. B., Hamsher, K. D., Varney, N. R., & Spreen, O. (1994). Contributions to neuropsychological assessment (2nd ed.). New York: Oxford University Press. Benton, A. L., & Hamsher, K. D. (1989). Multilingual Aphasia Examination: Manual of instructions (2nd ed.). Iowa City, IO: AJA Associates. Delis, D., Kramer, J., Kaplan, E., & Ober, B. (1987). California Verbal Learning Test manual (research ed.). San Antonio, TX: The Psychological Corporation. DeRenzi, E., & Spinner, H. (1966). Visual recognition in patients with unilateral cerebral disease. Journal of Nervous and Mental Disease, 142, 515–525. Ellis, H., & Daniel, T. (1971). Verbal process in long term stimulus recognition memory. Journal of Experimental Psychology, 90, 18–26. Goodglass, H., & Kaplan, E. (1983). The assessment of aphasia and related disorders (2nd ed.). Philadelphia, PA: Lea & Febiger. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Heaton, R., Grant, I., & Matthews, C. (1991). Comprehensive norms for an expanded Halstead-Reitan Battery: Demographic corrections, research findings, and clinical applications. Odessa, FL.: Psychological Assessment Resources. Heilman, K. M., & Valenstein, E. (1993). Clinical neuropsychology (3rd ed.). New York: Oxford University Press. Larrabee, G. J., & Curtiss, G. (1995). Construct validity of various verbal and visual memory tests. Journal of Clinical and Experimental Neuropsychology, 17, 536–547. Larrabee, G. J., Kane, R., Schuck, J., & Francis, D. (1985). Construct validity of various memory testing procedures. Journal of Clinical and Experimental Neuropsychology, 7, 239–250. Larrabee, G. J., Trahan, D. E., & Curtis, G. (1992). Construct validity of the continuous visual memory test. Archives of Clinical Neuropsychology, 7, 395–405. Lee, G., Loring, D., & Thompson, J. (1989). Construct validity of material-specific memory measures following unilateral temporal lobe ablations. Psychological Assessment, 1, 192–197. Leonberger, T., Nicks, S., Goldfader, P., & Munz, D. (1991). Factor analysis of the Wechsler Memory ScaleRevised and the Halstead-Reitan Neuropsychological Battery. The Clinical Neuropsychologist, 5, 83–88. Loring, D., & Papanicolaou, A. (1987). Memory assessment in neuropsychology: Theoretical considerations and practical utility. Journal of Clinical and Experimental Neuropsychology, 9, 340–358. Reitan, R. M., & Wolfson, D. (1985). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press. Smith, G., Malec, J., & Ivnik, R. (1992). Validity of the construct of nonverbal memory: A factor-analytic study in a normal elderly sample. Journal of Clinical and Experimental Neuropsychology, 14, 211–221. SPSS Inc. (1997). SPSS 7.5 for Windows. Chicago, IL: Author. Tinsley, H. E. A., & Tinsley, D. J. (1987). Uses of factor analysis in counseling psychology research. Journal of Counseling Psychology, 34(4), 14-42. Wechsler, D. (1945). A standardized memory scale for clinical use. The Journal of Psychology, 19, 87–95. Wechsler, D. (1981). WAIS-R manual. New York: Psychological Corporation. Wechsler, D. (1987). Wechsler Memory Scale-Revised Manual. San Antonio, TX: Psychological Corporation.