- Email: [email protected]
Using Different Memory Cutoffs to Assess Mild Cognitive Impairment
Using Different Memory Cutoffs to Assess Mild Cognitive Impairment David A. Loewenstein, Ph.D., Amarilis Acevedo, Ph.D., Raymond Ownby, M.D., Ph.D., Joscelyn Agron, Psy.D., Warren W. Barker, M.S., Richard Isaacson, M.D., Silvia Strauman, Ph.D., Ranjan Duara, M.D.
Objective: Although mild cognitive impairment (MCI) is characterized by performance on memory and other measures below expected normative values, neither a scientific rationale nor a consensus exists regarding which measures have the most use or the optimal cutoffs to use to establish impairment. Methods: Different memory measures were administered to 80 normal community-dwelling subjects divided into two age groups. This provided conormed data on eight different memory indices by which to compare 23 nondemented clinically diagnosed patients with MCI who met all other criteria for Alzheimer disease (AD). Results: On immediate memory for passages, delayed visual reproduction, object memory, and a measure sensitive to semantic interference, 70%–78% of patients with MCI were identified as impaired at 1.5 standard deviations or greater below expected levels. Conditional logistical regression for age-matched samples indicated that consideration of raw scores for these neuropsychologic tests in combination did not significantly change the odds of MCI diagnosis. When impairment relative to the total normal elderly sample was calculated based on one or more impairments at a 1.5 or greater cutoff, specificity fell below acceptable levels when more than three memory measures were considered. Conclusion: An array of widely used neuropsychologic measures demonstrated utility in distinguishing patients with MCI-AD from cognitively normal communitydwelling elders. The appropriateness of more or less stringent cutoffs was highly influenced by the number of measures considered. These findings have important implications regarding the choice of cut points for impairment used for the diagnosis of MCI in both research and clinical settings. (Am J Geriatr Psychiatry 2006; 14:911– 919) Key Words: MCI, mild cognitive impairment, memory, neuropsychology, Alzheimer disease
Received October 30, 2005; revised March 14, 2006; accepted March 30, 2006. From the Wien Center for Alzheimer’s Disease and Memory Disorders, Mount Sinai Medical Center, Miami Beach, FL (DAL, RO, WWB, RI, SS, RD); the Center on Aging and the Department of Psychiatry and Behavioral Sciences (DAL, AA, RO, JA, SS) and the Departments of Medicine and Psychiatry and Behavioral Sciences (RD), Miller School of Medicine, University of Miami, Miami, FL; and the Johnnie Byrd Alzheimer’s Disease Clinical and Research Center, Tampa, FL (AA, JA). Send correspondence and reprint requests to David A. Loewenstein, 4300 Alton Road, 2nd Floor, MRI Building, Miami Beach, FL 33140. E-mail: [email protected]. © 2006 American Association for Geriatric Psychiatry
Am J Geriatr Psychiatry 14:11, November 2006
911
Different Memory Cutoffs to Assess MCI
M
ild cognitive impairment (MCI) is an intermediate state between a cognitively normal state and dementia1,2 that can pose significant diagnostic challenges to clinicians, especially when the presence of deficits in memory and other cognitive domains is ambiguous.3,4 In this circumstance, objective neuropsychologic assessment can provide useful information for the clinician charged with rendering a clinical diagnosis.5 The diagnosis of early objective cognitive impairment is particularly important given the focus on the early detection of neurodegenerative disease and the future development of newer and more effective pharmacologic treatments.6 Petersen et al.7 note that the median cognitive (presumably memory) scores of the original Mayo Clinic cohort of clinically diagnosed patients with MCI was generally 1.5 standard deviations (SDs) below performance of peers of similar age who were cognitively normal. The 1.5 SD cutoff, which has been adopted in many research studies of MCI, is less stringent than the typical 2.0-SD cutoff used in studies of dementia or the lower fifth percentile cutoff recommended by the NINCDS-ADRDA8 criteria for confirmation of probable Alzheimer disease (AD). However, a review of the literature indicates that there is a lack of consensus as to what type of cognitive tests, how many tests, and what thresholds on these tests should be used to support or corroborate the diagnosis of MCI.5 Many different cognitive measures, particularly those involving memory and new learning, have been used in studies of MCI and the detection of early dementia.9 Given the heterogeneity of the measures used, there is clearly the need to directly evaluate the comparative efficacy of different types of memory indices alone and in combination in the diagnosis of MCI. Furthermore, there is a need to determine the use of specific measures at different cutoffs for impairment. The purpose of the current investigation was to examine the efficacy of eight different memory indices in distinguishing between clinically diagnosed patients with MCI and cognitively normal community-dwelling elders. Derivation of age-related normative data for multiple memory measures relative to a standard reference group allowed for an examination as to whether different cut points (1.5 SD or 2.0 SD below expected values) on single or multiple measures were most effective in correct classification. In addition, the efficacy of the optimal combi-
912
nation of memory measures in distinguishing between age-matched MCI and cognitively normal elderly subjects were also assessed.
METHODS Subjects We recruited 103 English-speaking subjects for this study, of which 80 subjects were cognitively normal and 23 were diagnosed with MCI, as described subsequently. Mild Cognitive Impairment–Alzheimer Disease Twenty-three subjects (12 males and 11 females) had a gradually progressive course of cognitive impairment with predominantly memory-related symptoms. The mean age of this group was 80.65 (SD: 6.1) years, whereas the average level of educational attainment was 13.48 (SD: 2.8) years. The clinical criteria for MCI and its etiologic subtypes have been used in studies at this center previously10,11 and are similar to clinical criteria described for the diagnosis of cognitive impairment, no dementia.12 The diagnosis of MCI-AD required a cognitive complaint from the subject or an informant, judgment of the neurologist of deficits in memory based on a behavioral neurologic evaluation that included multiple delayed recall of the Mini-Mental Status Examination (MMSE),13 the absence of dementia, as determined by Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria for dementia,14 and deficits in memory with or without other cognitive deficits on behavioral neurologic examination. The general neurologic examination revealed no focal signs suggestive of cerebral infarction. In addition, a global Clinical Dementia Rating Scale (CDR)15 of 0.5 and a Folstein MMSE16 score of 23 or greater were required. In addition, magnetic resonance imaging scans were assessed for features thought to be consistent with a diagnosis of incipient AD (i.e., evidence of cortical, hippocampal and entorhinal cortex atrophy, ventricular enlargement, and absence of significant cortical or subcortical infarcts). The presence of periventricular or other white matter hyperintensities on magnetic resonance imaging was not considered material to the diagnosis of MCI-AD as long
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. as frank cerebral infarcts were not present. It is important to note that the cognitive and neurologic impression described here was made without any influence from the neuropsychologic evaluation results. Individuals with MCI-AD met NINCDSADRDA8 clinical criteria for probable AD with the exception that there was not sufficient impairment in social/occupational function to merit a diagnosis of dementia. Previous research has indicated that persons diagnosed in this manner have prodromal AD and that all convert to dementia over a 9.5-year period.17 Cognitively Normal Community-Dwelling Elderly Eighty cognitively normal community-dwelling elderly subjects (26 males and 54 females) were recruited as part of their baseline evaluation for a longitudinal National Institutes of Health study investigating cognitive changes associated with aging. Many of these subjects were spouses of patients who had been evaluated for dementia in our large outpatient memory disorder clinic as well as those who had subsequently participated in a free community memory disorder screening evaluation. All these cognitively normal subjects and their informants, when available, participated in a CDR interview with a clinical neuropsychologist (AA or DL) or a postdoctoral fellow under their supervision. All subjects had MMSE scores of 27 or greater, had a global CDR score of 0, and had no indication of memory or other cognitive impairment. The mean age of this group was 77.15 (SD: 5.6) years, whereas the average level of educational attainment was 14.30 (SD: 2.4) years. This group of cognitively normal community-dwelling older adults was equivalent in gender distribution and education but was slightly younger than the patients with MCI-AD. As such, subjects were subsequently divided into two different age groups so that means and SDs could be derived to determine age-appropriate cutoffs at 1.5 and 2.0 SDs below the mean for each of the nine different memory indices for patients with MCI-AD. The first group of subjects (14 males and 14 females) ranged from 79 years to 93 years of age (mean: 83.68 years; SD: 3.7) with 8 –19 years of educational attainment (mean education: 14.30 years; SD: 2.8). The second group of subjects (14 males and 38 females) ranged from 70 to 78 years of
Am J Geriatr Psychiatry 14:11, November 2006
age (mean: 73.63 years; SD: 2.3) with 8 –19 years of educational attainment (mean: 14.31 years; SD: 2.2).
Neuropsychologic Measures All subjects in this study received a battery of neuropsychologic tests, which included four memory measures that we have used in previous studies of older adults. These include the Fuld Object Memory Evaluation,18 the Semantic Interference Test,9 Logical Memory for Passages from the Wechsler Memory Scale–3rd Edition,19 and Visual Reproduction from the Wechsler Memory Scale–Revised Edition.20
Fuld Object Memory Evaluation This is a modified three-trial version of Fuld Object Memory Evaluation (OME)21 in which the subject is presented with 10 common household objects and is subsequently administered a 60-second verbal fluency distractor task that interferes with the storage and consolidation of the to-be-remembered targets. Selective reminding is provided for objects not recalled followed by another distractor task of 30 seconds’ duration. Subjects are then asked to recall the list of objects for the third trial and are again selectively reminded of the missed objects. The recall scores for the three trials are summed to obtain the total recall score.
Semantic Interference Test The Semantic Interference Test (SIT) is based on the three-trial version of the OME as follows. After the third recall trial of the OME, the subject is presented with another set of 10 objects that are semantically related to the first set of items (e.g., bracelet instead of ring) followed by a 60-second verbal fluency distractor task. Recall of this new set of semantically related items is susceptible to proactive interference (old learning of the previous list interfering with learning of the new list), whereas subsequent recall of the original list is vulnerable to retroactive interference (new learning interferes with retrieval of previously learned information). For this study, we used the combined recall of the new set of targets (bag B) and recall of the old set of targets (bag A).
913
Different Memory Cutoffs to Assess MCI This measure has been shown to be sensitive to early cognitive impairments in MCI.9 Logical Memory for Passages In this test, free recall of a passage is first measured. Free recall is then assessed from a second passage, which is presented twice. The immediate recall score is calculated by summing recall of defined elements of the first passage and the first and second immediate recall of the second passage (total 75 elements). Delayed recall is calculated by summing recall of the first and the second passages (total 50 elements) after a 25- to 35-minute delay. A “savings score” is calculated by dividing the total delayed recall score by the summed immediate recall score.
sensitivities and specificities of neuropsychologic measures in discriminating between the MCI-AD and the 80 cognitively normal elderly participants were assessed using receiver operating characteristic (ROC) curve analyses. The proportion of MCI-AD subjects considered impaired at 1.5 and 2.0 SD cutoffs were compared in this manner. Conditional exact regression models were developed to assess the ability of neuropsychologic measures to distinguish the 23 MCI-AD subjects from a group of 23 agematched normal elderly. Conditional models were used to correct for the relations between variables created by matching, and exact methods were used because the sample was small and normal theorybased methods might have been inappropriate.
Visual Reproduction
RESULTS
This test involves memory recall of four geometric designs. An immediate recall score is calculated by summing recall of all four designs. Delayed recall is calculated by summing recall of the four designs after a 25- to 35-minute delay. A “savings score” is calculated by dividing the total delayed recall score by the total immediate recall score. Statistical Approach Differences among study groups were assessed using one-way analyses of variance for continuous variables and 2 tests for categorical variables. The
TABLE 1.
As depicted in Table 1, the OME correctly identified 78% of the patients with MCI-AD as impaired at the 1.5-SD cutoff relative to 70% of patients with MCI-AD administered the SIT, Immediate Recall for Passages, and Delayed Visual Reproduction. At the 1.5-SD cutoff, this yielded an equivalent area under the ROC curve for the OME relative to these other memory indices. In contrast, the savings score for Logical Memory for Passages and Immediate Visual Reproduction only correctly identified 44% of patients with MCI-AD, which yielded significantly less areas under the ROC curve than the OME. As ex-
Classification of Patients With Mild Cognitive Impairment Considered Impaired at 1.5 Standard Deviations and 2.0 Standard Deviations Below Expected Levels Compared with Impairment based on Age-Related Conormed Data on 80 Normal Elderly Subjects Sensitivity/Specificity at 1.5 Standard Deviations
Immediate Passages Delayed Passages Immediate Visual Reproduction Delayed Visual Reproduction Three-trial Fuld Object Memory Evaluationa Total Semantic Interference score Percent forgetting passages Percent forgetting visual reproduction
0.70/0.92 0.61/0.91 0.44/0.91 0.70/0.90 0.78/0.93 0.70/0.91 0.44/0.90 0.70/0.87
Area Under Receiver Operating Characteristic 0.810 (SE: 0.759 (SE: 0.673 (SE: 0.797 (SE: 0.854 (SE: 0.804 (SE: 0.666 (SE: 0.784 (SE:
0.06) 0.06) 0.07) 0.06) 0.05) 0.06) 0.07) 0.06)
Sensitivity/Specificity at 2.0 Standard Deviations 0.52/0.98 0.44/0.96 0.26/0.96 0.39/0.99 0.65/0.98 0.44/0.99 0.39/0.96 0.48/0.99
Area Under Receiver Operating Characteristic 0.748 (SE: 0.698 (SE: 0.611 (SE: 0.689 (SE: 0.814 (SE: 0.711 (SE: 0.676 (SE: 0.733 (SE:
0.06) 0.07) 0.07) 0.07) 0.06) 0.07) 0.07) 0.07)
a Subjects had a greater area under the receiver operating characteristic curve relative to immediate visual reproduction and the forgetting index for passages at both the 1.5-standard deviation and 2.0-standard deviation cutoffs. SE: standard error.
914
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. pected, the area under the ROC curve did not differ at a 1.5- versus a 2.0-SD cutoff for a particular memory measure, although the sensitivity for a single measure in classifying patients with AD was considerably less with the highest number of patients with MCI-AD classified at the 2.0-SD cutoff for the OME (65%) and Immediate Memory for Passages (52%). Although cutoffs for impairment for patients with MCI-AD and cognitively normal elderly subjects were derived using age-appropriate data for 70 –78 years and 79 years or above, and groups were equivalent with regard to education, some classification analyses required raw scores of the memory measures. Because patients with MCI-AD were slightly younger than the normal elderly subjects, an equivalent number of MCI-AD and cognitively normal elderly subjects were matched on age. This resulted in groups that were equivalent in terms of age (F[1,44]⫽ 0.002; p⫽0.96), years of education (F[1,44]⫽ 1.91; p⫽0.17), and gender (2 ⫽0; p⫽1.0). The threetrial Fuld OME total score yielded a statistically significant area under the ROC curve resulting in 87.0% sensitivity and 95.7% specificity with the exact conditional probability for matched samples calculated by logistic regression being .5026 (confidence interval: 0.1834 –1.779) at p ⬍0.0001. However, a comparison of ROC curves did not indicate any greater area under the ROC curve for the three-trial OME versus
TABLE 2.
Sensitivity and Specificity of Multiple Measures in the Classification of Mild Cognitive Impairment at ⴚ1.5 Standard Deviation and ⴚ2.0 Standard Deviation Relative to Normal Elderly Subjects Test Order A
1 2 3 4 5 6 7 8
Test Tests Tests Tests Tests Tests Tests Tests
the Total Semantic Interference Score or the Immediate and Delayed Recall of Passages and Visual Reproduction. Rate of forgetting for passages evidenced less area under the ROC curve than the threetrial OME score and Delayed Visual Reproduction. Rate of forgetting for Visual Reproduction also evidenced less area under the ROC curve than Delayed Visual Reproduction. The results of conditional logistical regression for combinations of neuropsychologic measures did not change the odds of MCI-AD diagnosis or resulted in distributions that could not be estimated using conditional logistic regression. Neuropsychologists and other clinicians are frequently encouraged to examine multiple memory indices to arrive at a judgment of impairment. However, as the number of tests considered as impaired at a particular cut point increases, so does the possibility of errors of classification. To examine the potential tradeoff between sensitivity and specificity at the 1.5-SD cutoff, classification rates for patients with MCI-AD were compared with error rates of classification based on the means and SDs derived for the two age-appropriate groups of the original normative sample. We first considered the subtests of the Wechsler Memory Scales, because they are among the most widely used memory measures among neuropsychologists.21 As depicted in Table 2, using test Order A when the Delayed and Immediate Recall for
Test Order B
Test Order C
1.5 SD
2.0 SD
1.5 SD
2.0 SD
1.5 SD
2.0 SD
0.70/0.90 0.83/0.83 0.91/0.79 0.91/0.75 0.91/0.73 0.91/0.70 0.96/0.69 0.96/0.65
0.48/0.99 0.65/0.97 0.78/0.95 0.83/0.92 0.83/0.90 0.83/0.89 0.96/0.87 0.96/0.86
0.70/0.93 0.70/0.88 0.91/0.81 0.91/0.75 0.91/0.74 0.91/0.70 0.96/0.69 0.96/0.65
0.52/0.96 0.57/0.95 0.70/0.92 0.83/0.92 0.83/0.91 0.83/0.89 0.96/0.87 0.96/0.86
0.78/0.93 0.87/0.88 0.87/0.81 0.91/0.74 0.96/0.71 0.96/0.68 0.96/0.66 0.96/0.65
0.65/0.97 0.70/0.96 0.78/0.95 0.83/0.94 0.91/0.91 0.96/0.89 0.96/0.88 0.96/0.86
Note: Each ordered set includes 1) a specific memory measure; 2) the original memory measure plus an additional measure; 3) the two original measures plus an additional measure; and 4) the three original memory measures plus an additional measure up to cumulative total of nine different measures Order A: 1) Delayed Visual Reproduction; 2) Delayed Passages; 3) Immediate Visual Reproduction; 4) Immediate Passages; 5) percent forgetting visual reproduction; 6) percent forgetting passages; 7) Fuld OME; and 8) SIT Total Interference. Order B: 1) Immediate Passages; 2) Delayed Passages; 3) Immediate Visual Reproduction; 4) Delayed Visual Reproduction; 5) percent forgetting passages; 6) percent forgetting visual reproduction; 7) Fuld OME; and 8) SIT Total Interference. Order C: 1) Fuld OME; 2) Semantic Interference Score; 3) Delayed Visual Reproduction; 4) Delayed Passages; 5) Immediate Visual Reproduction; 6) Immediate WMS-R Passages; 7) percent forgetting visual reproduction; and 8) percent forgetting passages. SD: standard deviation; OME: Object Memory Evaluation; SIT: Semantic Interference Test; WMS-R: Wechsler Memory Scale–Revised Edition.
Am J Geriatr Psychiatry 14:11, November 2006
915
Different Memory Cutoffs to Assess MCI Passages and Visual Reproduction were examined in conjunction with the savings scores for these indices (six measures), impairment in at least one or more of these indices resulted in 91% sensitivity but an unacceptable specificity of 0.70. Using test Order A when only Delayed and Immediate Memory for Passages and Visual Reproduction (four memory indices) was considered, 25% of normal elderly subjects still remained incorrectly classified. When test Order B was considered, an equivalent number of cognitively normal elderly subjects were misclassified using Immediate Recall and the savings scores for passages and Visual Reproduction (four memory indices). In fact, post hoc analyses revealed that all combinations of Delayed Recall of Passages and Visual Reproduction with any two other memory indices in the study (for a total of four measures) invariably correctly classified less than 80% of normal subjects at the 1.5-SD cutoff. In contrast, when a 2.0-SD cutoff was used, all eight memory measures resulted in correct classification of 96% of patients with MCI-AD and 86% correct classification of cognitively normal elderly subjects.
DISCUSSION In the current study, conormed data were available on eight different memory indices for cognitively normal elderly subjects 70 –78 years and 79 years and above. This enabled classification of patients with MCI-AD of equivalent educational attainment at both a 1.5- and 2.0-SD cutoffs based on age-related comparison data. At the 1.5-SD cutoff, 78% of patients with MCI-AD were classified using an OME as compared with 70% patients with MCI-AD administered the SIT and the Immediate Logical Memory for Passages and Delayed Visual Reproduction of the Wechsler Memory Scales. As expected, a reduction of sensitivity was noted for all of these measures when a 2.0-SD cutoff was used, but there were no differences in area under the ROC curve at either the 1.5-SD and 2.0-SD cutoff suggesting that the threetrial Fuld OME, SIT, Immediate and Delayed Logical Memory for Passages and Delayed Visual Reproduction were of similar efficacy in the classification of MCI-AD and cognitively normal elderly subjects. In contrast, Immediate Visual Reproduction and the
916
percentage of passages forgotten on Logical Memory evidenced low sensitivities and significantly less areas under the ROC curve than the Fuld OME. These results lend support for the notion that the type of cognitive measures used by the clinician may affect the accuracy of detection of MCI. Although deficits in delayed recall and rate of forgetting have been traditionally associated with early AD,23–25 the findings of substantial deficits on the Fuld OME, as well as immediate recall of passages, is consistent with emerging evidence that the greatest deficits in AD may actually occur in the initial storage and acquisition of new information.26,27 The OME is unique from many list learning tasks in that distractor trials are interspersed with learning trials, which greatly interferes with the initial acquisition of to-beremembered information. The finding that the savings score for Logical Memory for Passages was inferior to the OME in overall classification of subjects and that immediate recall of this information did not appreciably discriminate subjects any better than delayed recall at 25–30 minutes lends support to the notion that the large amount of information that the subject must acquire on this task may exceed the demands of working memory and results in primary acquisition and consolidation deficits. Similar findings have been demonstrated in studies that failed to find delayed recall deficits when controlling for immediate memory.28 In contrast, immediate visual reproduction may involve less stringent demands on working memory and thus lends itself to greater acquisition and retention of to-be-remembered material, although the visual information that is encoded may be more sensitive to retrieval deficits over time. Clearly, this is an area worthy of further exploration. In the current study, conormed data were available on eight different memory indices for cognitively normal elderly subjects 70 –78 years and 79 years and above. This enabled classification of patients with MCI-AD of equivalent educational attainment at both 1.5- and 2.0-SD cutoffs based on agerelated comparison data. Logical Memory for Passages, Visual Reproduction, and other subtests of the Wechsler Memory Scales are among the most widely tests used by neuropsychologists22 so that scores on these measures were first considered alone and in combination with other measures. As depicted in test Order A and test Order B in
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. Table 2, consideration of impairment in one or more subtests of the traditional immediate and delayed Logical Memory for Passages and Delayed Visual Reproduction indices in determining impairment resulted in misclassification of 25% of subjects in our cognitively normal elderly sample at the 1.5-SD cutoff below their own age-appropriate group mean. As described in the “Results” section, post hoc analyses of every possible combination delayed recall of story passages and delayed visual memory with any two other measures in our battery (including the OME and the SIT as indicated in test Order C) all indicated misclassification of more than 20% of our community-dwelling normal elderly subjects. If impairment on all eight memory indices were considered at a 1.5-SD cutoff, 35% of our cognitively normal group would have been classified as impaired. In contrast, using a more conservative cutoff of 2.0 SD of four measures would only misclassify from 6%– 8% normal subjects using test Orders A, B, and C. Furthermore, even when all eight tests are considered regardless of order, misclassification of normal elderly subjects would never be lower than 14%. Palmer et al.29 have reported similar findings of increased classification errors when multiple neuropsychologic measures are administered to cognitively normal subjects. Thus, it appears that among commonly used measures such as subtests of the Wechsler Scales and the OME, the use of two memory indices at the 1.5-SD cutoff will not result in excessive misclassification of normal elderly subjects. Although the acceptability of the 1.5-SD cutoff for three memory indices may be arguable, it is clear that misclassification of over 20% of cognitively normal elderly subjects using four or more memory indices is unacceptable and this difficulty can only be overcome using a 2.0-SD cutoff. A strength of the current study is that the memory performance of a clinically characterized group of patients with MCI-AD was contrasted with a cognitively normal community-dwelling elderly sample that scored in the normal range on both extensive clinical evaluation and mental status examination. The importance of the present study for clinicians doing research in the field of MCI is that we have compared routinely used memory indices, independently and in combination, as they relate to the diagnosis of MCI-AD in a clinic population. Age-re-
Am J Geriatr Psychiatry 14:11, November 2006
lated normative values were used derived from two age groups of community-dwelling subjects who were classified based on the MMSE and a comprehensive clinical diagnostic interview. Although it is always possible that certain subjects will be diagnostically misclassified, the independence of the cognitive measures and the clinical diagnosis of MCI-AD, as well as cognitive normality, is critical to avoid potential circularity that may occur when measures used for initial diagnoses are then used as outcome measures. Although a final diagnosis of AD requires neuropathologic verification at autopsy, clinical criteria similar to that used in the present study for the diagnosis of MCI-AD has been shown in other studies to demonstrate conversion rates to AD of 100% over a 9.5-year period. The vast majority of these cases evidenced AD pathology on postmortem examination.17 Similarly, our method of recruiting normal elderly subjects on the basis of both a mental status evaluation, CDR, and an extensive clinical interview is more rigorous than the recruitment of elderly subjects in the majority of studies conducted to date. Our finding that the three-trial OME was as effective in the classification of MCI-AD as other traditional memory measures may be a reflection of the utility of using distractor trials to interfere with the acquisition and retrieval of new information on listlearning tasks. Future studies should examine the performance of the OME with similar tasks that do not use distractor trials during the acquisition stages of learning (e.g., California Verbal Learning Test,30 Rey Auditory Verbal Learning Test,31 Hopkins Verbal Learning Test,32 CERAD List Learning Task33). Composite memory scores, developed by correspondence analyses, principal component analysis, or and other factor analytic approaches, should also be examined to determine whether increased parsimony can be achieved without sacrificing the sensitivity or specificity of a particular measure or set of measures as described in this study. It has been reported that deficits in global cognitive abilities and perceptual speed in MCI may be significant and even similar in magnitude to episodic memory deficits.34 Thus, these nonmemory domains should be considered in future studies of MCI.35 Furthermore, it will be important to investigate other etiologic subtypes
917
Different Memory Cutoffs to Assess MCI of MCI (e.g., vascular, diffuse Lewy body disease) to determine the generalizability of the present findings to other diagnostic groups. A consensus is required regarding the instruments that provide the best methods to identify MCI cases in the clinic, in epidemiologic studies, and in clinical trials. There is also a need for consensus regarding the best instruments for classifying MCI into various cognitive or etiologic subtypes, especially for research purposes. The results presented in this study provide guidelines for the use of specific cognitive tests, either alone or in combination, for this purpose. Our results also indicate that factors such as the number of measures used, the type of measures
used, and the selected cutoffs for impairment may all influence classification of MCI-AD cases. It is generally acknowledged that heterogeneity in methods and case definition of MCI has hampered the field by making it difficult to compare studies and to design new studies. Unmistakably, this is an area that deserves greater attention. This research was supported by 5R01AG020094-03 from the National Institute on Aging to David A. Loewenstein, Ph.D., Principal Investigator; and NIA Alzheimer’s Disease Research Center 1P50AG025711-01. This work was also supported by the Johnnie Byrd Alzheimer’s Disease Clinical and Research Center.
References 1. Petersen RC, Smith GE, Waring SC, et al: Aging, memory, and mild cognitive impairment. Int Psychogeriatr 1997; 9:65–69 2. Petersen R, Doody R, Kurz A, et al: Current concepts in mild cognitive impairment. Arch Neurol 2001; 58:1985–1992 3. Celsis P: Age-related cognitive decline, mild cognitive impairment or preclinical Alzheimer’s disease? Ann Med 2000; 32:6 –14 4. Sherwin BB: Mild cognitive impairment: potential pharmacologic treatment options. J Am Geriatr Soc 2000; 48:431–441 5. Luis CA, Loewenstein DA, Acevedo A, et al: Mild cognitive impairment: directions for future research. Neurology 2003; 61: 438 –444 6. Ritchie K, Touchon RM: Mild cognitive impairment in conceptual basis and current nosological status. Lancet 2000; 355:225–228 7. Petersen RC: Mild cognitive impairment as a diagnostic entity. J Intern Med 2004; 256:183–194 8. McKhann G, Drachman D, Folstein M, et al: Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA work group under the auspices of Department of Health and Human Services task force on Alzheimer’s disease. Neurology 1984; 34:939 –944 9. Loewenstein DA, Acevedo A, Luis C, et al: Semantic interference deficits and the detection of mild Alzheimer’s disease and mild cognitive impairment without dementia. J Int Neuropsychol Soc 2004; 10:91–100 10. Luis CA, Barker WW, Loewenstein DA, et al: Conversion to dementia among two groups with cognitive impairment. A preliminary report. Dement Geriatr Cogn Disord 2004; 18:307–313 11. Loewenstein DA, Acevedo Agron J, Issacson R, et al: Cognitive profiles in Alzheimer’s disease and in mild cognitive impairment of different etiologies. Dement Geriatr Cogn Disord. 2006; 21: 309-315 12. Graham JE, Rockwood K, Beattie BL, et al: Standardization of the diagnosis of dementia in the Canadian Study of Health and Aging. Neuroepidemiology 1996; 15:246 –256 13. Loewenstein DA, Barker WW, Harwood DG, et al: Utility of a modified Mini-Mental State Examination with extended delayed recall in screening for mild cognitive impairment and dementia among community dwelling elders: a preliminary study. Int J Geriatr Psychiatry 2000; 15:434 –440 14. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC, American Psychiatric Association, 1994
918
15. Morris JC: The Clinical Dementia Rating (CDR): current versions and scoring rules. Neurology 1993; 43:2412–2414 16. Folstein M, Folstein S, McHugh P: Mini-mental state. A practical method for grading the cognitive state of patients for the physician. J Psychiatr Res 1975; 12:189 –198 17. Morris JC, Storandt M, Miller JP, et al: Mild cognitive impairment represents early stage Alzheimer’s disease. Arch Neurol 2001; 58:397–405 18. Fuld PA: Fuld Object-Memory Evaluation. Woodale, IL, Stoelting Company, 1981 19. Wechsler D: The Wechsler Memory Scale–Third Edition. San Antonio, Psychological Corporation, 1997 20. Wechsler D: The Wechsler Memory Scale–Revised. San Antonio, Psychological Corporation, 1987 21. Loewenstein DA, Argu ¨ elles T, Barker WW, et al: The utility of a modified object memory test in distinguishing between three different age groups of Alzheimer’s disease patients and normal controls. J Ment Health Aging 2001; 7:317–324 22. Rabin L, Barr WB, Burton LA: Assessment practices of clinical neuropsychologists in the United States and Canada: a survey of INS, NAN and APA members. Arch Clin Neuropsychol 2005; 20:33–65 23. Locasio JJ, Growden JH, Corkin S: Cognitive test performance in detecting, staging and tracking Alzheimer’s disease. Arch Neurol 1995; 52:1087–1099 24. Troster AI, Butters N, Salmon D, et al: The diagnostic utility of savings scores: differentiating Alzheimer’s and Huntington’s disease with logical memory and visual reproduction tasks. J Clin Exp Neuropsychol 1993; 5:773–788 25. Welsh K, Butters N, Hughes J, et al: Detection of abnormal memory decline in cases of Alzheimer’s disease using CERAD neurological measures. Arch Neurol 1991; 48:278 –281 26. Germano C, Kinsella GJ: Working memory and learning in early Alzheimer’s disease. Neuropsychol Rev 2005; 15:1–10 27. Grober E, Kawas C: Learning and retention in preclinical and early Alzheimer’s disease. Psychol Aging 1997; 12:183–188 28. Greene JD, Baddeley AD, Hodges JR: Analysis of the episodic memory deficit in early Alzheimer’s disease: evidence from the doors and people test. Neuropsychologia 1996; 34:537–551 29. Palmer BW, Boone KB, Lesser IM, et al: Base rates of ‘impaired’
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. neuropsychological test performance among healthy older adults. Arch Clin Neuropsychol 1998; 13:503–511 30. Delis DC, Kramer JH, Kaplan E, et al: California Verbal Learning Test. San Antonio, Psychological Corporation, 1987 31. Schmidt M: Rey Auditory Verbal Learning Test. Los Angeles, Western Psychological Services, 1996 32. Brant J, Benedict R: Hopkins Verbal Learning Test–Revised. Odessa, FL, Psychological Assessment Resources, 2001 33. Morris JC, Heyman A, Mohs RC, et al: The Consortium to Establish
Am J Geriatr Psychiatry 14:11, November 2006
a Registry for Alzheimer’s Disease: part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology 1989; 39:1159 –1165 34. Backman L, Jones S, Berger AK, et al: Cognitive impairment in preclinical Alzheimer’s disease: a meta-analysis. Neuropsychology 2005; 19:520 –531 35. Backman L, Jones S, Berger AK, et al: Multiple cognitive deficits during the transition to Alzheimer’s disease. J Intern Med 2004; 256:195–204
919
Objective: Although mild cognitive impairment (MCI) is characterized by performance on memory and other measures below expected normative values, neither a scientific rationale nor a consensus exists regarding which measures have the most use or the optimal cutoffs to use to establish impairment. Methods: Different memory measures were administered to 80 normal community-dwelling subjects divided into two age groups. This provided conormed data on eight different memory indices by which to compare 23 nondemented clinically diagnosed patients with MCI who met all other criteria for Alzheimer disease (AD). Results: On immediate memory for passages, delayed visual reproduction, object memory, and a measure sensitive to semantic interference, 70%–78% of patients with MCI were identified as impaired at 1.5 standard deviations or greater below expected levels. Conditional logistical regression for age-matched samples indicated that consideration of raw scores for these neuropsychologic tests in combination did not significantly change the odds of MCI diagnosis. When impairment relative to the total normal elderly sample was calculated based on one or more impairments at a 1.5 or greater cutoff, specificity fell below acceptable levels when more than three memory measures were considered. Conclusion: An array of widely used neuropsychologic measures demonstrated utility in distinguishing patients with MCI-AD from cognitively normal communitydwelling elders. The appropriateness of more or less stringent cutoffs was highly influenced by the number of measures considered. These findings have important implications regarding the choice of cut points for impairment used for the diagnosis of MCI in both research and clinical settings. (Am J Geriatr Psychiatry 2006; 14:911– 919) Key Words: MCI, mild cognitive impairment, memory, neuropsychology, Alzheimer disease
Received October 30, 2005; revised March 14, 2006; accepted March 30, 2006. From the Wien Center for Alzheimer’s Disease and Memory Disorders, Mount Sinai Medical Center, Miami Beach, FL (DAL, RO, WWB, RI, SS, RD); the Center on Aging and the Department of Psychiatry and Behavioral Sciences (DAL, AA, RO, JA, SS) and the Departments of Medicine and Psychiatry and Behavioral Sciences (RD), Miller School of Medicine, University of Miami, Miami, FL; and the Johnnie Byrd Alzheimer’s Disease Clinical and Research Center, Tampa, FL (AA, JA). Send correspondence and reprint requests to David A. Loewenstein, 4300 Alton Road, 2nd Floor, MRI Building, Miami Beach, FL 33140. E-mail: [email protected]. © 2006 American Association for Geriatric Psychiatry
Am J Geriatr Psychiatry 14:11, November 2006
911
Different Memory Cutoffs to Assess MCI
M
ild cognitive impairment (MCI) is an intermediate state between a cognitively normal state and dementia1,2 that can pose significant diagnostic challenges to clinicians, especially when the presence of deficits in memory and other cognitive domains is ambiguous.3,4 In this circumstance, objective neuropsychologic assessment can provide useful information for the clinician charged with rendering a clinical diagnosis.5 The diagnosis of early objective cognitive impairment is particularly important given the focus on the early detection of neurodegenerative disease and the future development of newer and more effective pharmacologic treatments.6 Petersen et al.7 note that the median cognitive (presumably memory) scores of the original Mayo Clinic cohort of clinically diagnosed patients with MCI was generally 1.5 standard deviations (SDs) below performance of peers of similar age who were cognitively normal. The 1.5 SD cutoff, which has been adopted in many research studies of MCI, is less stringent than the typical 2.0-SD cutoff used in studies of dementia or the lower fifth percentile cutoff recommended by the NINCDS-ADRDA8 criteria for confirmation of probable Alzheimer disease (AD). However, a review of the literature indicates that there is a lack of consensus as to what type of cognitive tests, how many tests, and what thresholds on these tests should be used to support or corroborate the diagnosis of MCI.5 Many different cognitive measures, particularly those involving memory and new learning, have been used in studies of MCI and the detection of early dementia.9 Given the heterogeneity of the measures used, there is clearly the need to directly evaluate the comparative efficacy of different types of memory indices alone and in combination in the diagnosis of MCI. Furthermore, there is a need to determine the use of specific measures at different cutoffs for impairment. The purpose of the current investigation was to examine the efficacy of eight different memory indices in distinguishing between clinically diagnosed patients with MCI and cognitively normal community-dwelling elders. Derivation of age-related normative data for multiple memory measures relative to a standard reference group allowed for an examination as to whether different cut points (1.5 SD or 2.0 SD below expected values) on single or multiple measures were most effective in correct classification. In addition, the efficacy of the optimal combi-
912
nation of memory measures in distinguishing between age-matched MCI and cognitively normal elderly subjects were also assessed.
METHODS Subjects We recruited 103 English-speaking subjects for this study, of which 80 subjects were cognitively normal and 23 were diagnosed with MCI, as described subsequently. Mild Cognitive Impairment–Alzheimer Disease Twenty-three subjects (12 males and 11 females) had a gradually progressive course of cognitive impairment with predominantly memory-related symptoms. The mean age of this group was 80.65 (SD: 6.1) years, whereas the average level of educational attainment was 13.48 (SD: 2.8) years. The clinical criteria for MCI and its etiologic subtypes have been used in studies at this center previously10,11 and are similar to clinical criteria described for the diagnosis of cognitive impairment, no dementia.12 The diagnosis of MCI-AD required a cognitive complaint from the subject or an informant, judgment of the neurologist of deficits in memory based on a behavioral neurologic evaluation that included multiple delayed recall of the Mini-Mental Status Examination (MMSE),13 the absence of dementia, as determined by Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria for dementia,14 and deficits in memory with or without other cognitive deficits on behavioral neurologic examination. The general neurologic examination revealed no focal signs suggestive of cerebral infarction. In addition, a global Clinical Dementia Rating Scale (CDR)15 of 0.5 and a Folstein MMSE16 score of 23 or greater were required. In addition, magnetic resonance imaging scans were assessed for features thought to be consistent with a diagnosis of incipient AD (i.e., evidence of cortical, hippocampal and entorhinal cortex atrophy, ventricular enlargement, and absence of significant cortical or subcortical infarcts). The presence of periventricular or other white matter hyperintensities on magnetic resonance imaging was not considered material to the diagnosis of MCI-AD as long
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. as frank cerebral infarcts were not present. It is important to note that the cognitive and neurologic impression described here was made without any influence from the neuropsychologic evaluation results. Individuals with MCI-AD met NINCDSADRDA8 clinical criteria for probable AD with the exception that there was not sufficient impairment in social/occupational function to merit a diagnosis of dementia. Previous research has indicated that persons diagnosed in this manner have prodromal AD and that all convert to dementia over a 9.5-year period.17 Cognitively Normal Community-Dwelling Elderly Eighty cognitively normal community-dwelling elderly subjects (26 males and 54 females) were recruited as part of their baseline evaluation for a longitudinal National Institutes of Health study investigating cognitive changes associated with aging. Many of these subjects were spouses of patients who had been evaluated for dementia in our large outpatient memory disorder clinic as well as those who had subsequently participated in a free community memory disorder screening evaluation. All these cognitively normal subjects and their informants, when available, participated in a CDR interview with a clinical neuropsychologist (AA or DL) or a postdoctoral fellow under their supervision. All subjects had MMSE scores of 27 or greater, had a global CDR score of 0, and had no indication of memory or other cognitive impairment. The mean age of this group was 77.15 (SD: 5.6) years, whereas the average level of educational attainment was 14.30 (SD: 2.4) years. This group of cognitively normal community-dwelling older adults was equivalent in gender distribution and education but was slightly younger than the patients with MCI-AD. As such, subjects were subsequently divided into two different age groups so that means and SDs could be derived to determine age-appropriate cutoffs at 1.5 and 2.0 SDs below the mean for each of the nine different memory indices for patients with MCI-AD. The first group of subjects (14 males and 14 females) ranged from 79 years to 93 years of age (mean: 83.68 years; SD: 3.7) with 8 –19 years of educational attainment (mean education: 14.30 years; SD: 2.8). The second group of subjects (14 males and 38 females) ranged from 70 to 78 years of
Am J Geriatr Psychiatry 14:11, November 2006
age (mean: 73.63 years; SD: 2.3) with 8 –19 years of educational attainment (mean: 14.31 years; SD: 2.2).
Neuropsychologic Measures All subjects in this study received a battery of neuropsychologic tests, which included four memory measures that we have used in previous studies of older adults. These include the Fuld Object Memory Evaluation,18 the Semantic Interference Test,9 Logical Memory for Passages from the Wechsler Memory Scale–3rd Edition,19 and Visual Reproduction from the Wechsler Memory Scale–Revised Edition.20
Fuld Object Memory Evaluation This is a modified three-trial version of Fuld Object Memory Evaluation (OME)21 in which the subject is presented with 10 common household objects and is subsequently administered a 60-second verbal fluency distractor task that interferes with the storage and consolidation of the to-be-remembered targets. Selective reminding is provided for objects not recalled followed by another distractor task of 30 seconds’ duration. Subjects are then asked to recall the list of objects for the third trial and are again selectively reminded of the missed objects. The recall scores for the three trials are summed to obtain the total recall score.
Semantic Interference Test The Semantic Interference Test (SIT) is based on the three-trial version of the OME as follows. After the third recall trial of the OME, the subject is presented with another set of 10 objects that are semantically related to the first set of items (e.g., bracelet instead of ring) followed by a 60-second verbal fluency distractor task. Recall of this new set of semantically related items is susceptible to proactive interference (old learning of the previous list interfering with learning of the new list), whereas subsequent recall of the original list is vulnerable to retroactive interference (new learning interferes with retrieval of previously learned information). For this study, we used the combined recall of the new set of targets (bag B) and recall of the old set of targets (bag A).
913
Different Memory Cutoffs to Assess MCI This measure has been shown to be sensitive to early cognitive impairments in MCI.9 Logical Memory for Passages In this test, free recall of a passage is first measured. Free recall is then assessed from a second passage, which is presented twice. The immediate recall score is calculated by summing recall of defined elements of the first passage and the first and second immediate recall of the second passage (total 75 elements). Delayed recall is calculated by summing recall of the first and the second passages (total 50 elements) after a 25- to 35-minute delay. A “savings score” is calculated by dividing the total delayed recall score by the summed immediate recall score.
sensitivities and specificities of neuropsychologic measures in discriminating between the MCI-AD and the 80 cognitively normal elderly participants were assessed using receiver operating characteristic (ROC) curve analyses. The proportion of MCI-AD subjects considered impaired at 1.5 and 2.0 SD cutoffs were compared in this manner. Conditional exact regression models were developed to assess the ability of neuropsychologic measures to distinguish the 23 MCI-AD subjects from a group of 23 agematched normal elderly. Conditional models were used to correct for the relations between variables created by matching, and exact methods were used because the sample was small and normal theorybased methods might have been inappropriate.
Visual Reproduction
RESULTS
This test involves memory recall of four geometric designs. An immediate recall score is calculated by summing recall of all four designs. Delayed recall is calculated by summing recall of the four designs after a 25- to 35-minute delay. A “savings score” is calculated by dividing the total delayed recall score by the total immediate recall score. Statistical Approach Differences among study groups were assessed using one-way analyses of variance for continuous variables and 2 tests for categorical variables. The
TABLE 1.
As depicted in Table 1, the OME correctly identified 78% of the patients with MCI-AD as impaired at the 1.5-SD cutoff relative to 70% of patients with MCI-AD administered the SIT, Immediate Recall for Passages, and Delayed Visual Reproduction. At the 1.5-SD cutoff, this yielded an equivalent area under the ROC curve for the OME relative to these other memory indices. In contrast, the savings score for Logical Memory for Passages and Immediate Visual Reproduction only correctly identified 44% of patients with MCI-AD, which yielded significantly less areas under the ROC curve than the OME. As ex-
Classification of Patients With Mild Cognitive Impairment Considered Impaired at 1.5 Standard Deviations and 2.0 Standard Deviations Below Expected Levels Compared with Impairment based on Age-Related Conormed Data on 80 Normal Elderly Subjects Sensitivity/Specificity at 1.5 Standard Deviations
Immediate Passages Delayed Passages Immediate Visual Reproduction Delayed Visual Reproduction Three-trial Fuld Object Memory Evaluationa Total Semantic Interference score Percent forgetting passages Percent forgetting visual reproduction
0.70/0.92 0.61/0.91 0.44/0.91 0.70/0.90 0.78/0.93 0.70/0.91 0.44/0.90 0.70/0.87
Area Under Receiver Operating Characteristic 0.810 (SE: 0.759 (SE: 0.673 (SE: 0.797 (SE: 0.854 (SE: 0.804 (SE: 0.666 (SE: 0.784 (SE:
0.06) 0.06) 0.07) 0.06) 0.05) 0.06) 0.07) 0.06)
Sensitivity/Specificity at 2.0 Standard Deviations 0.52/0.98 0.44/0.96 0.26/0.96 0.39/0.99 0.65/0.98 0.44/0.99 0.39/0.96 0.48/0.99
Area Under Receiver Operating Characteristic 0.748 (SE: 0.698 (SE: 0.611 (SE: 0.689 (SE: 0.814 (SE: 0.711 (SE: 0.676 (SE: 0.733 (SE:
0.06) 0.07) 0.07) 0.07) 0.06) 0.07) 0.07) 0.07)
a Subjects had a greater area under the receiver operating characteristic curve relative to immediate visual reproduction and the forgetting index for passages at both the 1.5-standard deviation and 2.0-standard deviation cutoffs. SE: standard error.
914
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. pected, the area under the ROC curve did not differ at a 1.5- versus a 2.0-SD cutoff for a particular memory measure, although the sensitivity for a single measure in classifying patients with AD was considerably less with the highest number of patients with MCI-AD classified at the 2.0-SD cutoff for the OME (65%) and Immediate Memory for Passages (52%). Although cutoffs for impairment for patients with MCI-AD and cognitively normal elderly subjects were derived using age-appropriate data for 70 –78 years and 79 years or above, and groups were equivalent with regard to education, some classification analyses required raw scores of the memory measures. Because patients with MCI-AD were slightly younger than the normal elderly subjects, an equivalent number of MCI-AD and cognitively normal elderly subjects were matched on age. This resulted in groups that were equivalent in terms of age (F[1,44]⫽ 0.002; p⫽0.96), years of education (F[1,44]⫽ 1.91; p⫽0.17), and gender (2 ⫽0; p⫽1.0). The threetrial Fuld OME total score yielded a statistically significant area under the ROC curve resulting in 87.0% sensitivity and 95.7% specificity with the exact conditional probability for matched samples calculated by logistic regression being .5026 (confidence interval: 0.1834 –1.779) at p ⬍0.0001. However, a comparison of ROC curves did not indicate any greater area under the ROC curve for the three-trial OME versus
TABLE 2.
Sensitivity and Specificity of Multiple Measures in the Classification of Mild Cognitive Impairment at ⴚ1.5 Standard Deviation and ⴚ2.0 Standard Deviation Relative to Normal Elderly Subjects Test Order A
1 2 3 4 5 6 7 8
Test Tests Tests Tests Tests Tests Tests Tests
the Total Semantic Interference Score or the Immediate and Delayed Recall of Passages and Visual Reproduction. Rate of forgetting for passages evidenced less area under the ROC curve than the threetrial OME score and Delayed Visual Reproduction. Rate of forgetting for Visual Reproduction also evidenced less area under the ROC curve than Delayed Visual Reproduction. The results of conditional logistical regression for combinations of neuropsychologic measures did not change the odds of MCI-AD diagnosis or resulted in distributions that could not be estimated using conditional logistic regression. Neuropsychologists and other clinicians are frequently encouraged to examine multiple memory indices to arrive at a judgment of impairment. However, as the number of tests considered as impaired at a particular cut point increases, so does the possibility of errors of classification. To examine the potential tradeoff between sensitivity and specificity at the 1.5-SD cutoff, classification rates for patients with MCI-AD were compared with error rates of classification based on the means and SDs derived for the two age-appropriate groups of the original normative sample. We first considered the subtests of the Wechsler Memory Scales, because they are among the most widely used memory measures among neuropsychologists.21 As depicted in Table 2, using test Order A when the Delayed and Immediate Recall for
Test Order B
Test Order C
1.5 SD
2.0 SD
1.5 SD
2.0 SD
1.5 SD
2.0 SD
0.70/0.90 0.83/0.83 0.91/0.79 0.91/0.75 0.91/0.73 0.91/0.70 0.96/0.69 0.96/0.65
0.48/0.99 0.65/0.97 0.78/0.95 0.83/0.92 0.83/0.90 0.83/0.89 0.96/0.87 0.96/0.86
0.70/0.93 0.70/0.88 0.91/0.81 0.91/0.75 0.91/0.74 0.91/0.70 0.96/0.69 0.96/0.65
0.52/0.96 0.57/0.95 0.70/0.92 0.83/0.92 0.83/0.91 0.83/0.89 0.96/0.87 0.96/0.86
0.78/0.93 0.87/0.88 0.87/0.81 0.91/0.74 0.96/0.71 0.96/0.68 0.96/0.66 0.96/0.65
0.65/0.97 0.70/0.96 0.78/0.95 0.83/0.94 0.91/0.91 0.96/0.89 0.96/0.88 0.96/0.86
Note: Each ordered set includes 1) a specific memory measure; 2) the original memory measure plus an additional measure; 3) the two original measures plus an additional measure; and 4) the three original memory measures plus an additional measure up to cumulative total of nine different measures Order A: 1) Delayed Visual Reproduction; 2) Delayed Passages; 3) Immediate Visual Reproduction; 4) Immediate Passages; 5) percent forgetting visual reproduction; 6) percent forgetting passages; 7) Fuld OME; and 8) SIT Total Interference. Order B: 1) Immediate Passages; 2) Delayed Passages; 3) Immediate Visual Reproduction; 4) Delayed Visual Reproduction; 5) percent forgetting passages; 6) percent forgetting visual reproduction; 7) Fuld OME; and 8) SIT Total Interference. Order C: 1) Fuld OME; 2) Semantic Interference Score; 3) Delayed Visual Reproduction; 4) Delayed Passages; 5) Immediate Visual Reproduction; 6) Immediate WMS-R Passages; 7) percent forgetting visual reproduction; and 8) percent forgetting passages. SD: standard deviation; OME: Object Memory Evaluation; SIT: Semantic Interference Test; WMS-R: Wechsler Memory Scale–Revised Edition.
Am J Geriatr Psychiatry 14:11, November 2006
915
Different Memory Cutoffs to Assess MCI Passages and Visual Reproduction were examined in conjunction with the savings scores for these indices (six measures), impairment in at least one or more of these indices resulted in 91% sensitivity but an unacceptable specificity of 0.70. Using test Order A when only Delayed and Immediate Memory for Passages and Visual Reproduction (four memory indices) was considered, 25% of normal elderly subjects still remained incorrectly classified. When test Order B was considered, an equivalent number of cognitively normal elderly subjects were misclassified using Immediate Recall and the savings scores for passages and Visual Reproduction (four memory indices). In fact, post hoc analyses revealed that all combinations of Delayed Recall of Passages and Visual Reproduction with any two other memory indices in the study (for a total of four measures) invariably correctly classified less than 80% of normal subjects at the 1.5-SD cutoff. In contrast, when a 2.0-SD cutoff was used, all eight memory measures resulted in correct classification of 96% of patients with MCI-AD and 86% correct classification of cognitively normal elderly subjects.
DISCUSSION In the current study, conormed data were available on eight different memory indices for cognitively normal elderly subjects 70 –78 years and 79 years and above. This enabled classification of patients with MCI-AD of equivalent educational attainment at both a 1.5- and 2.0-SD cutoffs based on age-related comparison data. At the 1.5-SD cutoff, 78% of patients with MCI-AD were classified using an OME as compared with 70% patients with MCI-AD administered the SIT and the Immediate Logical Memory for Passages and Delayed Visual Reproduction of the Wechsler Memory Scales. As expected, a reduction of sensitivity was noted for all of these measures when a 2.0-SD cutoff was used, but there were no differences in area under the ROC curve at either the 1.5-SD and 2.0-SD cutoff suggesting that the threetrial Fuld OME, SIT, Immediate and Delayed Logical Memory for Passages and Delayed Visual Reproduction were of similar efficacy in the classification of MCI-AD and cognitively normal elderly subjects. In contrast, Immediate Visual Reproduction and the
916
percentage of passages forgotten on Logical Memory evidenced low sensitivities and significantly less areas under the ROC curve than the Fuld OME. These results lend support for the notion that the type of cognitive measures used by the clinician may affect the accuracy of detection of MCI. Although deficits in delayed recall and rate of forgetting have been traditionally associated with early AD,23–25 the findings of substantial deficits on the Fuld OME, as well as immediate recall of passages, is consistent with emerging evidence that the greatest deficits in AD may actually occur in the initial storage and acquisition of new information.26,27 The OME is unique from many list learning tasks in that distractor trials are interspersed with learning trials, which greatly interferes with the initial acquisition of to-beremembered information. The finding that the savings score for Logical Memory for Passages was inferior to the OME in overall classification of subjects and that immediate recall of this information did not appreciably discriminate subjects any better than delayed recall at 25–30 minutes lends support to the notion that the large amount of information that the subject must acquire on this task may exceed the demands of working memory and results in primary acquisition and consolidation deficits. Similar findings have been demonstrated in studies that failed to find delayed recall deficits when controlling for immediate memory.28 In contrast, immediate visual reproduction may involve less stringent demands on working memory and thus lends itself to greater acquisition and retention of to-be-remembered material, although the visual information that is encoded may be more sensitive to retrieval deficits over time. Clearly, this is an area worthy of further exploration. In the current study, conormed data were available on eight different memory indices for cognitively normal elderly subjects 70 –78 years and 79 years and above. This enabled classification of patients with MCI-AD of equivalent educational attainment at both 1.5- and 2.0-SD cutoffs based on agerelated comparison data. Logical Memory for Passages, Visual Reproduction, and other subtests of the Wechsler Memory Scales are among the most widely tests used by neuropsychologists22 so that scores on these measures were first considered alone and in combination with other measures. As depicted in test Order A and test Order B in
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. Table 2, consideration of impairment in one or more subtests of the traditional immediate and delayed Logical Memory for Passages and Delayed Visual Reproduction indices in determining impairment resulted in misclassification of 25% of subjects in our cognitively normal elderly sample at the 1.5-SD cutoff below their own age-appropriate group mean. As described in the “Results” section, post hoc analyses of every possible combination delayed recall of story passages and delayed visual memory with any two other measures in our battery (including the OME and the SIT as indicated in test Order C) all indicated misclassification of more than 20% of our community-dwelling normal elderly subjects. If impairment on all eight memory indices were considered at a 1.5-SD cutoff, 35% of our cognitively normal group would have been classified as impaired. In contrast, using a more conservative cutoff of 2.0 SD of four measures would only misclassify from 6%– 8% normal subjects using test Orders A, B, and C. Furthermore, even when all eight tests are considered regardless of order, misclassification of normal elderly subjects would never be lower than 14%. Palmer et al.29 have reported similar findings of increased classification errors when multiple neuropsychologic measures are administered to cognitively normal subjects. Thus, it appears that among commonly used measures such as subtests of the Wechsler Scales and the OME, the use of two memory indices at the 1.5-SD cutoff will not result in excessive misclassification of normal elderly subjects. Although the acceptability of the 1.5-SD cutoff for three memory indices may be arguable, it is clear that misclassification of over 20% of cognitively normal elderly subjects using four or more memory indices is unacceptable and this difficulty can only be overcome using a 2.0-SD cutoff. A strength of the current study is that the memory performance of a clinically characterized group of patients with MCI-AD was contrasted with a cognitively normal community-dwelling elderly sample that scored in the normal range on both extensive clinical evaluation and mental status examination. The importance of the present study for clinicians doing research in the field of MCI is that we have compared routinely used memory indices, independently and in combination, as they relate to the diagnosis of MCI-AD in a clinic population. Age-re-
Am J Geriatr Psychiatry 14:11, November 2006
lated normative values were used derived from two age groups of community-dwelling subjects who were classified based on the MMSE and a comprehensive clinical diagnostic interview. Although it is always possible that certain subjects will be diagnostically misclassified, the independence of the cognitive measures and the clinical diagnosis of MCI-AD, as well as cognitive normality, is critical to avoid potential circularity that may occur when measures used for initial diagnoses are then used as outcome measures. Although a final diagnosis of AD requires neuropathologic verification at autopsy, clinical criteria similar to that used in the present study for the diagnosis of MCI-AD has been shown in other studies to demonstrate conversion rates to AD of 100% over a 9.5-year period. The vast majority of these cases evidenced AD pathology on postmortem examination.17 Similarly, our method of recruiting normal elderly subjects on the basis of both a mental status evaluation, CDR, and an extensive clinical interview is more rigorous than the recruitment of elderly subjects in the majority of studies conducted to date. Our finding that the three-trial OME was as effective in the classification of MCI-AD as other traditional memory measures may be a reflection of the utility of using distractor trials to interfere with the acquisition and retrieval of new information on listlearning tasks. Future studies should examine the performance of the OME with similar tasks that do not use distractor trials during the acquisition stages of learning (e.g., California Verbal Learning Test,30 Rey Auditory Verbal Learning Test,31 Hopkins Verbal Learning Test,32 CERAD List Learning Task33). Composite memory scores, developed by correspondence analyses, principal component analysis, or and other factor analytic approaches, should also be examined to determine whether increased parsimony can be achieved without sacrificing the sensitivity or specificity of a particular measure or set of measures as described in this study. It has been reported that deficits in global cognitive abilities and perceptual speed in MCI may be significant and even similar in magnitude to episodic memory deficits.34 Thus, these nonmemory domains should be considered in future studies of MCI.35 Furthermore, it will be important to investigate other etiologic subtypes
917
Different Memory Cutoffs to Assess MCI of MCI (e.g., vascular, diffuse Lewy body disease) to determine the generalizability of the present findings to other diagnostic groups. A consensus is required regarding the instruments that provide the best methods to identify MCI cases in the clinic, in epidemiologic studies, and in clinical trials. There is also a need for consensus regarding the best instruments for classifying MCI into various cognitive or etiologic subtypes, especially for research purposes. The results presented in this study provide guidelines for the use of specific cognitive tests, either alone or in combination, for this purpose. Our results also indicate that factors such as the number of measures used, the type of measures
used, and the selected cutoffs for impairment may all influence classification of MCI-AD cases. It is generally acknowledged that heterogeneity in methods and case definition of MCI has hampered the field by making it difficult to compare studies and to design new studies. Unmistakably, this is an area that deserves greater attention. This research was supported by 5R01AG020094-03 from the National Institute on Aging to David A. Loewenstein, Ph.D., Principal Investigator; and NIA Alzheimer’s Disease Research Center 1P50AG025711-01. This work was also supported by the Johnnie Byrd Alzheimer’s Disease Clinical and Research Center.
References 1. Petersen RC, Smith GE, Waring SC, et al: Aging, memory, and mild cognitive impairment. Int Psychogeriatr 1997; 9:65–69 2. Petersen R, Doody R, Kurz A, et al: Current concepts in mild cognitive impairment. Arch Neurol 2001; 58:1985–1992 3. Celsis P: Age-related cognitive decline, mild cognitive impairment or preclinical Alzheimer’s disease? Ann Med 2000; 32:6 –14 4. Sherwin BB: Mild cognitive impairment: potential pharmacologic treatment options. J Am Geriatr Soc 2000; 48:431–441 5. Luis CA, Loewenstein DA, Acevedo A, et al: Mild cognitive impairment: directions for future research. Neurology 2003; 61: 438 –444 6. Ritchie K, Touchon RM: Mild cognitive impairment in conceptual basis and current nosological status. Lancet 2000; 355:225–228 7. Petersen RC: Mild cognitive impairment as a diagnostic entity. J Intern Med 2004; 256:183–194 8. McKhann G, Drachman D, Folstein M, et al: Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA work group under the auspices of Department of Health and Human Services task force on Alzheimer’s disease. Neurology 1984; 34:939 –944 9. Loewenstein DA, Acevedo A, Luis C, et al: Semantic interference deficits and the detection of mild Alzheimer’s disease and mild cognitive impairment without dementia. J Int Neuropsychol Soc 2004; 10:91–100 10. Luis CA, Barker WW, Loewenstein DA, et al: Conversion to dementia among two groups with cognitive impairment. A preliminary report. Dement Geriatr Cogn Disord 2004; 18:307–313 11. Loewenstein DA, Acevedo Agron J, Issacson R, et al: Cognitive profiles in Alzheimer’s disease and in mild cognitive impairment of different etiologies. Dement Geriatr Cogn Disord. 2006; 21: 309-315 12. Graham JE, Rockwood K, Beattie BL, et al: Standardization of the diagnosis of dementia in the Canadian Study of Health and Aging. Neuroepidemiology 1996; 15:246 –256 13. Loewenstein DA, Barker WW, Harwood DG, et al: Utility of a modified Mini-Mental State Examination with extended delayed recall in screening for mild cognitive impairment and dementia among community dwelling elders: a preliminary study. Int J Geriatr Psychiatry 2000; 15:434 –440 14. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC, American Psychiatric Association, 1994
918
15. Morris JC: The Clinical Dementia Rating (CDR): current versions and scoring rules. Neurology 1993; 43:2412–2414 16. Folstein M, Folstein S, McHugh P: Mini-mental state. A practical method for grading the cognitive state of patients for the physician. J Psychiatr Res 1975; 12:189 –198 17. Morris JC, Storandt M, Miller JP, et al: Mild cognitive impairment represents early stage Alzheimer’s disease. Arch Neurol 2001; 58:397–405 18. Fuld PA: Fuld Object-Memory Evaluation. Woodale, IL, Stoelting Company, 1981 19. Wechsler D: The Wechsler Memory Scale–Third Edition. San Antonio, Psychological Corporation, 1997 20. Wechsler D: The Wechsler Memory Scale–Revised. San Antonio, Psychological Corporation, 1987 21. Loewenstein DA, Argu ¨ elles T, Barker WW, et al: The utility of a modified object memory test in distinguishing between three different age groups of Alzheimer’s disease patients and normal controls. J Ment Health Aging 2001; 7:317–324 22. Rabin L, Barr WB, Burton LA: Assessment practices of clinical neuropsychologists in the United States and Canada: a survey of INS, NAN and APA members. Arch Clin Neuropsychol 2005; 20:33–65 23. Locasio JJ, Growden JH, Corkin S: Cognitive test performance in detecting, staging and tracking Alzheimer’s disease. Arch Neurol 1995; 52:1087–1099 24. Troster AI, Butters N, Salmon D, et al: The diagnostic utility of savings scores: differentiating Alzheimer’s and Huntington’s disease with logical memory and visual reproduction tasks. J Clin Exp Neuropsychol 1993; 5:773–788 25. Welsh K, Butters N, Hughes J, et al: Detection of abnormal memory decline in cases of Alzheimer’s disease using CERAD neurological measures. Arch Neurol 1991; 48:278 –281 26. Germano C, Kinsella GJ: Working memory and learning in early Alzheimer’s disease. Neuropsychol Rev 2005; 15:1–10 27. Grober E, Kawas C: Learning and retention in preclinical and early Alzheimer’s disease. Psychol Aging 1997; 12:183–188 28. Greene JD, Baddeley AD, Hodges JR: Analysis of the episodic memory deficit in early Alzheimer’s disease: evidence from the doors and people test. Neuropsychologia 1996; 34:537–551 29. Palmer BW, Boone KB, Lesser IM, et al: Base rates of ‘impaired’
Am J Geriatr Psychiatry 14:11, November 2006
Loewenstein et al. neuropsychological test performance among healthy older adults. Arch Clin Neuropsychol 1998; 13:503–511 30. Delis DC, Kramer JH, Kaplan E, et al: California Verbal Learning Test. San Antonio, Psychological Corporation, 1987 31. Schmidt M: Rey Auditory Verbal Learning Test. Los Angeles, Western Psychological Services, 1996 32. Brant J, Benedict R: Hopkins Verbal Learning Test–Revised. Odessa, FL, Psychological Assessment Resources, 2001 33. Morris JC, Heyman A, Mohs RC, et al: The Consortium to Establish
Am J Geriatr Psychiatry 14:11, November 2006
a Registry for Alzheimer’s Disease: part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology 1989; 39:1159 –1165 34. Backman L, Jones S, Berger AK, et al: Cognitive impairment in preclinical Alzheimer’s disease: a meta-analysis. Neuropsychology 2005; 19:520 –531 35. Backman L, Jones S, Berger AK, et al: Multiple cognitive deficits during the transition to Alzheimer’s disease. J Intern Med 2004; 256:195–204
919