Traditional neuropsychological correlates and reliability of the Automated Neuropsychological Assessment Metrics-4 battery for Parkinson’s disease

Parkinsonism and Related Disorders 18 (2012) 864e870

Contents lists available at SciVerse ScienceDirect

Parkinsonism and Related Disorders journal homepage: www.elsevier.com/locate/parkreldis

Traditional neuropsychological correlates and reliability of the Automated Neuropsychological Assessment Metrics-4 battery for Parkinson’s disease Keith A. Hawkins a, *, Danna Jennings b, Andrea S. Vincent c, Kirby Gilliland c, Adrienne West a, Kenneth Marek b a b c

Yale University School of Medicine, USA Institute for Neurodegenerative Disorders, USA University of Oklahoma, USA

a r t i c l e i n f o

a b s t r a c t

Article history: Received 28 December 2010 Received in revised form 19 March 2012 Accepted 17 April 2012

Background: The Automated Neuropsychological Assessment Metrics Battery-4 for PD offers the promise of a computerized approach to cognitive assessment. Methods: To assess its utility, the ANAM4-PD was administered to 72 PD patients and 24 controls along with a traditional battery. Reliability was assessed by retesting 26 patients. Results: The Cognitive Efficiency Score (CES; a global score) exhibited high reliability (r ¼ 0.86). Constituent variables exhibited lower reliability. The CES correlated strongly with the traditional battery global score, but displayed weaker relationships to UPDRS scores than the traditional score. Multivariate analysis of variance revealed a significant difference between the patient and control groups in ANAM4PD performance, with three ANAM4-PD tests, Math, Tower, and Pursuit Tracking, displaying sizeable differences. In discriminant analyses these variables were as effective as the total ANAM4-PD in classifying cases designated as impaired based on traditional variables. Principal components analyses uncovered fewer factors in the ANAM4-PD relative to the traditional battery. ANAM4-PD variables correlated at higher levels with traditional motor and processing speed variables than with untimed executive, intellectual or memory variables. Conclusions: The ANAM4-PD displays high global reliability, but variable subtest reliability. The battery assesses a narrower range of cognitive functions than traditional tests, and discriminates between patients and controls less effectively. Three ANAM4-PD tests, Pursuit Tracking, Math, and Tower performed as well as the total ANAM4-PD in classifying patients as cognitively impaired. These findings could guide the refinement of the ANAM4-PD as an efficient method of screening for mild to moderate cognitive deficits in PD patients. Ó 2012 Elsevier Ltd. All rights reserved.

Keywords: Parkinson’s disease Cognition Neuropsychological assessment ANAM4-PD

1. Introduction Parkinson’s disease (PD) is accompanied by cognitive declines that can begin prior to the emergence of motor symptoms, and ends in dementia for many patients [1e5]. The deficits observed range from impairment in motor and attention functions through to severe declines in higher-order intellectual abilities, with variations in onset and course that may reflect pathophysiological differences underlying disease subtypes [5e7]. A PD form of mild cognitive impairment is relatively common and is a risk state for progression

* Corresponding author. Yale University School of Medicine, Room 530, CMHC, 34 Park St., New Haven, Connecticut 06519, USA. Tel.: þ1 203 974 7831. E-mail address: [email protected] (K.A. Hawkins). 1353-8020/$ e see front matter Ó 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.parkreldis.2012.04.021

to dementia [8,9]. The cognitive features of PD significantly impact patient management and long term planning for patients and caregivers. Given their relevance to everyday functioning and quality of life, therapies for cognitive deficits are a key unmet need for PD patients. Treatment trials require a comprehensive and easily applicable approach to assessment of cognitive function. Cognitive deficits are typically assessed with neuropsychological tests administered subject by subject. In recognition of the potential advantages of computerized testing e including utility in the field, automated scoring, and the simultaneous assessment of numerous subjects e the U.S. Department of Defense developed a computerized testing battery to assess the effects of various insults and stressors on cognition [10]. This library, known as the Automated Neuropsychological Assessment Metric (ANAM) was designed to be a sensitive and repeatable measure of processing speed and

K.A. Hawkins et al. / Parkinsonism and Related Disorders 18 (2012) 864e870

cognitive efficiency [10e12]. The ANAM has been increasingly employed for assessing cognitive functioning across a broad spectrum of clinical disorders and neurological insults [13]. The Automated Neuropsychological Assessment Metrics Parkinson’s Disease battery (ANAM4-PD) is a selection of tests from the larger ANAM library based on neuropsychological findings in PD [8,13]. Deficits are commonly found in attention, working memory, executive functioning (e.g., response inhibition, set switching), and visuo-spatial orientation [8,9]. As detailed in Appendix 1, the tests forming ANAM4-PD were selected based on their capacity to assess these domains. The battery is designed to detect cognitive changes of mild to moderate severity (i.e., before indicative of dementia), allowing for the monitoring of disease progression or treatment effects on cognition via serial administrations. Since batteries specifically designed to detect cognitive deficits in PD already exist [14], the need for a further instrument can be questioned. Each of the existing batteries have weaknesses [14], and none have been validated to the point where the investigation of newer batteries is redundant. The Mini-Mental Parkinson [15] lacks test-retest reliability [14], and its brevity precludes adequate probing of specific functional domains. The Scales for Outcomes of Parkinson’s Disease-Cognition [16], though possibly limited by a heavy focus on “frontal-subcortical” functions [14], is empirically well supported although brief [14]. The Parkinson Neuropsychometric Dementia Assessment [17] is even briefer, but as yet lacks extensive psychometric data [14]. The Parkinson’s Disease-Cognitive Rating Scale [18] displays good psychometric properties, and is also relatively brief [14]. All the above differ from the ANAM4-PD in that none are computer administered, a potential disadvantage in circumstances where large-scale screenings are contemplated, where use of a human examiner is not feasible, or when serial assessment is required. Additionally, the relative brevity of each works against the assessment in depth of specific functional domains necessary for the detection of early cognitive decline, or the identification of domain-specific strengths and weaknesses. A further strength of the ANAM4-PD is the inclusion of subtests designed to detect (and monitor) subtle motor deficits in addition to cognitive tests.

865

To determine the utility of the ANAM4-PD we administered both the ANAM4-PD and a battery of traditional neuropsychological tests to idiopathic PD patients and healthy controls. The ANAM4-PD was re-administered one to three weeks after the first administration to determine test-retest reliability. 2. Methods 2.1. Participants Seventy-two PD subjects and 24 healthy controls were recruited from the Institute for Neurodegenerative disorders (IND) clinical database and from advertising. Study IRB approval was obtained and all subjects provided written informed consent. Inclusion criteria required PD subjects to have a clinical diagnosis of idiopathic PD and age >40 years. Healthy subjects had no motor or cognitive disorder, no family history of Parkinsonism, and MMSE  28. PD subjects were allowed to be on anti-Parkinson’s medications, and were assessed during the “on” phase of treatment. Forty-seven percent of the patients were treated with Levodopa, 43% a dopamine agonist, and 21% a Mao B inhibitor at time of testing. Thirty-five percent of patients were rated as having Hoehn and Yahr scores of less than 2. Fifty percent were rated at 2, and 15% greater than 2. The PD patients had a total UPDRS mean of 29.8 (SD ¼ 13.6), range 5e67, and a Motor UPDRS mean of 18.7 (SD ¼ 9.4), range 2e42. The PD patients (time since diagnosis M ¼ 4.2 years, SD ¼ 3.9; 65% male) did not differ from the controls (42% male) on age M ¼ 63.8, SD ¼ 8.4 vs. M 64.5, SD ¼ 8.4 (p ¼ 0.31), or education, M ¼ 16.0, SD ¼ 2.8 vs. M ¼ 16.4, SD ¼ 2.8, p ¼ 0.64.

2.2. ANAM4-PD battery The ANAM4-PD battery includes eleven tests (Appendix 1) designed to detect cognitive impairment across a variety of domains. The battery can be administered on a standard personal computer hardware configuration (laptop or desktop with external USB mouse). Each test begins with practice items to assist with learning the procedures and instructions before data collection. The battery takes approximately 1 h to complete. A number of summary variables are automatically computed for each of the ANAM tests. For the current analyses, a subset of 9 core variables plus one global variable (Table 1) were examined. Most variables reflect “throughput”, considered a measure of effectiveness or cognitive efficiency and reflects both accuracy and speed of responding [19]. Single variables consisting of the addition of highly correlated scores were created for the domains of pursuit tracking (sine wave and horizontal line) and concentration (derived from the procedural reaction time test and a “same versus different” continuous performance test). The overall composite score (the Cognitive Efficiency Score; CES) was formed by averaging the primary variables z scores, based on the control group means and standard deviations for each variable.

Table 1 ANAM4-PD and neuropsychological variables. Variable

ANAM4-PD source variables

Description

Pursuit Tracking

SQPdist þ HZPdist

Simple Reaction Time Choice Reaction Time Concentration

SRTtp 2CHtp PROtp þ CPTtp

Matching Math Manikin Switch Reaction Tower CES

M2Stp MTHtp MKNtp SWTtp ATPscore The above

Mean distance from target for Pursuit Tracking tests (square wave and horizontal line). Higher scores denote poorer performance. Number of correct responses per minute (Throughputa) for Simple Reaction Time test. Number of correct responses per minute (Throughputa) for 2-Choice Reaction Time test. Number of correct responses per minute (Throughputa) for Procedural Reaction Time and Running Memory Continuous Performance Tests. Number of correct responses per minute (Throughputa) for Matching to Sample test. Number of correct responses per minute (Throughputa) for Math test. Number of correct responses per minute (Throughputa) for Manikin test. Number of correct responses per minute (Throughputa) for Switching test. Performance score for Tower Puzzle. Cognitive Efficiency Score: average z score (based on the control group mean and SD) of the above nine variables.

Neuropsychological variables Finger Tapping Grooved Pegboard Processing Speed Graphic Sequencing Fluency Hooper VOT Picture Completion Drawings Verbal Memory Fluency Switching Wisconsin Card Sorting Global Neuropsych

L þ R hand L þ R hand WAIS-III PSI Correct in 45 s. FAS þ Animals Total correct WAIS-III test Clock, cube HVLT-R Vegetables & “p” Perseverations The above

a

Manual speed measured by index finger tapping on keyboard (left plus right hand totals) Manual dexterity measured by rapid insertion of key-like pegs into holes with different orientations. Information processing speed (Wechsler Adult Intelligence Test-III Digit Symbol þ Symbol Search) Psychomotor flexibility: drawing alternating peaks and plateaus (correct productivity per time unit.) Verbal generativity: Phonemic fluency (words beginning with F, A, S) þ semantic fluency (animals). Visual analysis & gestalt formation: Mental rearrangement of visual fragments to identify object. Visual analysis & reasoning: Wechsler Adult Intelligence Test-III Picture Completion test. Construction/praxis: Clock drawing with time setting, plus a cube copy and free drawing. Word list learning over three trials (total recall on the Hopkins Verbal Learning Test). Flexible generativity: Alternating vegetable names and other words beginning with “p” Mental flexibility: Deducing and applying sorting principals that shift. Higher scores are better. Average of the above neuropsychological variables (standardized to ensure equal weighting).

Throughput is considered a measure of effectiveness or cognitive efficiency and reflects both accuracy and speed of responding (Thorne, [19]).

866

K.A. Hawkins et al. / Parkinsonism and Related Disorders 18 (2012) 864e870

Test-retest reliability coefficients were obtained for the ANAM-PD tests by retesting 26 PD patients one to three weeks after their initial testing.

Table 3 Factor loadings (varimax rotation) for the ANAM4-PD variables. Test

2.3. Neuropsychological variables

1

Eleven primary variables were derived from a neuropsychological battery, with composite score constituents standardized (based on the control group means and standard deviations) prior to addition to ensure equal weightings. A 12th variable, Global Neuropsych, is the average of the 11 primary variables. Table 1 provides descriptions of the variables used for both batteries.

Switch Reaction Manikin Math Matching Pursuit Tracking Tower Choice Reaction Time Simple Reaction Time Concentration

2.4. Data analysis plan Individual variables will be examined for marked skew and transformed where necessary to normalize distributions. Test-retest reliability for the ANAM4-PD will be determined by Pearson correlation coefficients between baseline and repeated administration scores obtained 1e3 weeks later. The relationship of ANAM4-PD to conventional neuropsychological data will be determined by correlation coefficients. Principal components analysis (PCA) with varimax rotation will shed further light on the diversity of cognitive domains covered by the respective batteries. Multivariate analysis of variance (MANOVA) will be used to assess whether PD patients differed from controls on the ANAM, with follow-up univariate testing to determine the significance of differences on individual variables. Magnitude of group differences (effect size) will be assessed by Cohen’s d for each variable. Discriminant analysis will be used to assess the extent to which the patient and control groups could be separated based on the ANAM4-PD and traditional batteries. Discriminant analyses will also determine the extent to which the ANAM4-PD variables correctly identify patients identified as cognitively impaired by the traditional battery.

3. Results 3.1. Normalization of score distributions Four variables (one traditional, three ANAM4-PD) exhibited skewed distributions and were transformed. Negative skew evident in both groups for the Hooper VOT scores was successfully transformed by cubing the raw scores. Choice Reaction Time exhibited a pronounced negative skew in the control group, successfully transformed by score squaring. Positive skew (both groups) for Pursuit Tracking was corrected by log10 (variable score), as was positive skew in the control group for Manikin. 3.2. ANAM4-PD reliability The ANAM4-PD was well tolerated. Test-retest correlations for 26 retested patients ranged from a low, non-significant correlation of 0.23 for simple reaction time to 0.86 for the CES, a high reliability level [20]. Test-retest coefficients are presented in Table 2. 3.3. ANAM4-PD comparison to neuropsychological measures: bivariate correlations Within the patient group the ANAM4-PD CES correlated at 0.82 with the traditional battery global score, and significantly with each

Factor loading 2

0.90 0.80 0.68 0.62 0.55 0.54

0.54

0.84 0.75 0.61

of the 11 traditional variables. Table 2 reveals that of the 11 neuropsychological variables, two feature prominently as the strongest correlates of ANAM4-PD scores: processing speed and the grooved pegboard. Appendix 2 presents a full matrix of correlations for the primary traditional and ANAM4-PD variables. 3.4. Principal components analysis 3.4.1. ANAM-PD Principal components analysis (PCA) of the ANAM4-PD variables using the patients’ data revealed two components with eigenvalues exceeding 1, explaining 49.7% and 11.8% of the variance respectively for a total of 61.5%. Following varimax rotation, variables loading most heavily on component 1 (Switch Reaction, Manikin, and Math) suggest that this component in part reflects higher-order cognition. Choice Reaction and Simple Reaction loaded substantially on component 2, suggesting this reflects the more fundamental cognitive process of rapid responding. Table 3 displays the rotated components loadings, with loadings less than 0.5 omitted for clarity. 3.4.2. Neuropsychological battery PCA of the 11 neuropsychological variables revealed the presence of three components with eigenvalues exceeding 1, explaining 45.2%, 11.7%, and 9.5% of the variance respectively for a total of 66.5%. Following varimax rotation, the variables loading most heavily on component 1, Graphic Sequencing, Verbal Fluency, Fluency Switching, and Processing Speed, suggest that this component reflects rapid and flexible information processing and productivity. Picture Completion and Hooper Visual Organization loaded on component 2, reflecting visuo-spatial higher-order cognition. The Wisconsin Card Sorting Test loaded heavily on the third factor, suggesting it taps abstraction and mental set switching executive functioning. A lesser loading for Drawing on the component likely reflects the reasoning involved in the drawing

Table 2 ANAM-PD test-retest reliability and strongest neuropsychological correlates (patients only). ANAM-PD variable

Test-retest r (n ¼ 26)

Reliability level (Strauss, Sherman & Spreen, [20])

Strongest neuropsychological correlates (Pearson r in brackets)

Manikin Matching Tower Switch Reaction Pursuit Tracking Choice Reaction Concentration Math Simple Reaction Time CES

0.79** 0.77** 0.74** 0.73** 0.71** 0.69** 0.67** 0.60* 0.23 ns 0.86**

Adequate Adequate Adequate Adequate Adequate Marginal Marginal Marginal Low High

Finger Tapping (0.54)** Processing Speed (0.58)** Grooved Pegboard (0.61)** Processing Speed (0.66)** Graphic Sequencing (0.48)** Fluency Switching (0.36)* Processing Speed (0.61)** Processing Speed (0.58)** Grooved Pegboard (0.44)** Processing Speed (0.75)**

*p < 0.01. **p < 0.001.

Processing Speed (0.50)** Grooved Pegboard (0.54)** Processing Speed (0.58)** Graphic Sequencing (0.57)** Processing Speed (0.48)** Processing Speed (0.35)* Grooved Pegboard (0.58)** Fluency Switching (0.49)** Finger Tapping (0.43)** Grooved Pegboard (0.67)**

K.A. Hawkins et al. / Parkinsonism and Related Disorders 18 (2012) 864e870

3.5.2. Neuropsychological variables A multivariate analysis of variance (MANOVA) demonstrated that the patient and control groups differed on a linear combination of the neuropsychological variables, Wilks’ L ¼ 0.63, F (11, 84) ¼ 4.41, p < 0.001, multivariate h2 ¼ 0.37. Follow-up univariate ANOVAs indicated that six variables, when examined alone, significantly differed between the groups (Table 5). Discriminant analysis correctly classified 83.3% of cases (80.6% of patients; 91.7% of controls). After removing the motor variables (Finger Tapping and Grooved Pegboard) discriminant analysis correctly identified 74% of cases (70.8% patients; 83.3% controls).

Table 4 Factor loadings (varimax rotation) for the traditional neuropsychological variables. Test

Factor loading 1

Graphic Sequencing Fluency Fluency Switching Processing Speed Grooved Pegboard Finger Tapping Verbal Memory Picture Completion Hooper VOT Wisconsin Card Sorting Drawings

2

3

0.82 0.78 0.75 0.73 0.69 0.51 0.81 0.62 0.51

867

0.89 0.59

3.6. ANAM-PD identification of cognitively impaired vs nonimpaired PD patients Discriminant analysis was performed to test whether the ANAM4-PD battery identified patients as impaired based on performance on the traditional tests. Patients were labeled as impaired if they attained a mean z score of minus 1 (or worse) on the eleven neuropsychological variables. Twenty-three patients exceeded this threshold (compared to only 1 control). Discriminant analysis using the 9 ANAM-PD variables correctly classified 83.3% of the participants as impaired (20 of the 24), and labeled 13.9% (10 of 72 participants) as impaired when, per the traditional battery threshold, they were not. Conversely, 86.1% of unimpaired participants were correctly classified as such by the ANAM4-PD. To shed light the sensitivity of the ANAM4-PD to the more cognitive/less motoric aspects of the neuropsychological deficiencies associated with PD, participants were reclassified using the mean z score of the traditional tests after removal of finger tapping, the grooved pegboard, and graphic sequencing. Of the 16 cases (15 patients, 1 control) that averaged 1 SD or worse on the remaining neuropsychological variables, the ANAM4-PD correctly classified 81.3% as impaired (13 of 16), and 85% as normal (68 of 80). Rerunning the analysis with just the three most sensitive ANAM4-

tasks, e.g. time setting for the clock, and the higher-order demands of drawing a cube. Table 4 displays the rotated components loadings, with loadings less than 0.5 omitted for clarity. 3.5. Discrimination of PD patients from controls based on cognitive performance 3.5.1. ANAM4-PD variables A multivariate analysis of variance (MANOVA) found that the patient and control groups differed on a linear combination of ANAM4-PD variables, Wilks’ L ¼ 0.77, F (9, 84) ¼ 2.8, p ¼ 0.006, multivariate h2 ¼ 0.23. Follow-up univariate ANOVAs indicated that three variables (Pursuit Tracking, Math, and Tower) differed significantly between the groups (Table 5). Discriminant analyses demonstrated that ANAM4-PD variables could correctly identify 74% of cases (73.6% of the patients; 75% of the controls). Re-running the analysis after the deletion of two significantly motoric variables (Pursuit Tracking and Reaction Time) correctly classified 70.8% of cases (69.4% of patients; 75% of controls).

Table 5 Patient versus control groups on ANAM-PD and neuropsychological variables. Global scores include transformed scores as indicated by italics. Variable Grooved Pegboard (pegs placed in 45 s L þ R trials) Processing Speed (sum of WAIS-III Symbol Search and Digit Symbol scaled scores) ANAM4-PD Pursuit Tracking (low scores are better) Pursuit Tracking (transformed to reduce positive skew) Wisconsin Card Sorting Picture Completion (scaled score) HVLT Total Recall Graphic Sequencing (correct units per second) ANAM4-PD Math ANAM4-PD Tower Finger Tapping (sum of 5 L þ 5 R 10 s trials) ANAM4-PD Concentration Drawing Hooper (raw score) Hooper (transformed to reduce negative skew) Verbal Fluency Switching (raw correct) ANAM4-PD Matching ANAM4-PD Reaction Time ANAM4-PD Choice Reaction Choice Reaction (transformed to reduce negative skew) ANAM4-PD Switch Reaction Verbal Fluency (mean z score of F, A, S, and Animals based on control sample mean & SD) ANAM4-PD Manikin Manikin (transformed to reduce positive skew) ANAM4-PD CES (mean of ANAM4-PD z scores) Neuropsychological Global (mean of neuropsych z scores)

Control Mean and (Standard Deviation) 27.21 (4.1) 23.42 (5.5) 26.62 1.42 99.00 12.83 26.04 0.89 20.77 1160.8 374.0 164.0 0.0 25.35 16797 14.5 24.56 181.09 115.79 14372 21.26 0.0 21.86 1.28 0.0 0.0

Patient Mean and (Standard Deviation) 17.53 (7.4) 19.00 (5.5)

t test p <0.001 0.001

Effect size 1.63 0.81

(5.80) (0.08) (10.3) (3.0) (5.9) (0.26) (6.56) (475.4) (46.7) (27.1) (1.0) (2.7) (4425) (4.9) (7.45) (37.7) (31.7) (5886) (7.3) (0.87)

32.59 (8.68) 1.49 (0.11) 91.6 (10.9) 10.76 (2.9) 22.36 (5.4) 0.71 (0.36) 17.09 (7.05) 905.5 (497.7) 342.4 (79.3) 146.4 (45.0) 0.48(1.3) 23.9 (3.6) 14582 (5296) 12.38 (5.1) 21.60 (8.33) 164.85 (48.3) 108.61 (29.4) 12649 (5879) 19.26 (7.4) 0.09 (0.90)

0.002 0.001 0.004 0.004 0.006 0.02 0.03 0.03 0.02 0.03 0.07 0.08 0.07 0.09 ns ns ns ns ns ns

0.81 0.73 0.71 0.69 0.66 0.59 0.54 0.52 0.49 0.47 0.45 0.44 0.45 0.41 0.38 0.37 0.24 0.29 0.27 0.1

(11.7) (0.25) (0.61) (0.59)

25.67 (12.8) 1.35 (0.24) 0.381(0.77) 0.718(0.78)

ns ns 0.031 <0.001

0.31 0.29 0.55 1.04

868

K.A. Hawkins et al. / Parkinsonism and Related Disorders 18 (2012) 864e870

PD variables (per control group-patient difference effect sizes; Pursuit Tracking, Math, and Tower) as independent variables again correctly identified 81.3% as impaired (13 of 16), and 86.3% as normal (69 of 80). In all the preceding analyses, the single control case identified as impaired by the traditional tests was also classified as impaired by the ANAM4-PD. 3.7. Relationship to symptoms The traditional neuropsychological battery global score correlated significantly with UPDRS Total Score and UPDRS Motor Score, r ¼ 0.37 (p < 0.005) and r ¼ 0.32 (p < 0.01), respectively. The ANAM4-PD CES correlated at r ¼ 0.29 (p < 0.02) with UPDRS Total Score, and r ¼ 0.25 (p < 0.05) with UPDRS Motor Score. 4. Conclusions This study examined the ANAM4-PD computerized test battery as a tool to assess cognitive function in PD patients. Constituent ANAM4-PD tests demonstrated variable but mostly adequate testretest reliability coefficients over a 1e3 week period. Simple reaction time displayed poor test-retest reliability, but global score (CES) displayed high reliability per widely accepted criteria [20]. Unless test-retest data from other sources indicate higher reliability, consideration could be given to removal of the simple reaction time subtest from the ANAM4-PD. The utility of the ANAM4-PD in assessing neuropsychological deficiencies commonly associated with PD was evaluated in several ways. The CES correlated at a moderately high level with a global score derived from 11 traditional neuropsychological tests, suggesting reasonable concurrent validity. However, the ANAM4PD yielded fewer components than the traditional tests in principal components analyses (PCA), and the ANAM4-PD tests correlated most strongly with the processing speed and motor tests of the traditional battery. Though this reveals sensitivity to cardinal PD features, it also suggests that the ANAM4-PD may lack sufficient breadth of coverage to detect some of the broader, higher level cognitive deficiencies associated with PD. Notably, three of the traditional tests showing greatest separation between the patients and controls do not place an emphasis on rapid responding, but rather rely on intellectual analysis (Picture Completion), new declarative learning/memory (Hopkins Verbal Learning Test-Revised total recall), or executive functioning with a focus on abstract reasoning and mental flexibility (Wisconsin Card Sorting Test). Consistent with this concern, in discriminant function analyses the ANAM4-PD did not distinguishes patients from controls as effectively as the standard neuropsychological tests included in this study. Univariate comparisons between the patient and control groups also predominately favored the traditional tests in terms of statistical significance and effect sizes. Since not all PD patients display cognitive deficits, we did not expect discriminant analysis to result in perfect classification rates. Additional analyses were undertaken to determine the capacity of the ANAM4-PD to correctly classify patients designated as impaired based on their traditional battery scores. Participants were considered cognitively impaired if they averaged minus one standard deviation (or worse) from the control group mean on the 11 traditional tests. Per the Movement Disorder Society task force guidelines for diagnostic criteria for PD-related mild cognitive impairment (MCI), MCI-PD exists if two neuropsychological test performances fall between one to two standard deviations below a demographically relevant mean (8). On that basis, the threshold adopted in our study of an average of minus 1 standard deviation is conservative; all participants

designated as impaired within our study would meet criteria for PD-related MCI or worse. The ANAM-PD correctly classified 83.3% of the participants judged neuropsychologically impaired, and correctly classified 86.1% of the unimpaired. After reclassification of participants post removal of tests with a significant motor component from the traditional battery (so that those meeting impairment criteria did so on more purely cognitive grounds), the ANAM4-PD correctly classified 81.3% as impaired and 85% of the unimpaired. Three ANAM4-PD tests (Pursuit Tracking, Math, and Tower) performed as well as the entire ANAM4-PD when the analysis was re-run with just those three entered as independent variables. In summary, the tendency of the ANAM4-PD tests to correlate primarily with the processing speed and motor tests of the neuropsychological battery, coupled with a limited coverage of cognitive domains as indicated by PCA, suggests that the ANAM4-PD does not fully assess the diverse cognitive functions affected by PD. Computerized testing for cognition in general is limited by the confound of motor slowing in PD. On the other hand, the ANAM4PD performed well at the global level, as evidenced by the moderately strong correlation coefficient between its global score and that of the neuropsychological battery, and by the moderately high level of agreement between the batteries in the identification of the most cognitively impaired participants. Pending further development, the test-retest reliability of the ANAM4-PD CES (global score) suggests a potential for use in PD clinical trials, where cognitive testing has become increasingly predominant. Computerized testing incorporates advantages that in some circumstances may offset disadvantages, such as potential use in mass screenings, the need for minimal examiner input, automatic scoring and results, and suitability for serial evaluations (e.g., in determining treatment response or delineation of course). Our findings suggest ways in which the ANAM4-PD could be improved. Three tests, Pursuit Tracking, Math, and Tower, identified impaired participants as effectively as the full ANAM4-PD. Pending cross validation, this finding could allow for a progressive refinement and shortening of the battery. The reliability of each of these tests would likely be improved by lengthening. Whether the inclusion of other tests from the broader ANAM library e for example, tests are more dependent on higher-order cognition and executive functions, and less depend on speeded responding e would broaden the sensitivity of the ANAM-PD to the diverse deficits of PD is a matter for future research. Financial/conflict of interest disclosure Keith A. Hawkins, Danna Jennings, Adrienne West, and Kenneth Marek have no financial interests relative to this research to declare. The University of Oklahoma has had contracts with the US Department of Defense Army Medical Research Acquisition Activity: US Army Telemedicine and Advanced Technology Center contracts that have partially supported the positions of Kirby Gilliland and Andrea S. Vincent. The University of Oklahoma (OU) holds the license for the Automated Neuropsychological Assessment Metrics (ANAM). The Cognitive Science Research Center (formerly C-SHOP) at OU is responsible for ANAM development. Author K. Gilliland has a standard university royalty agreement for the sale of ANAM. Acknowledgements Funding was provided by the U.S. Department of Defense Cooperative Agreement Contract # W81XWH-06-1-0680; K. Marek & D. Jennings, Principal Investigators.

K.A. Hawkins et al. / Parkinsonism and Related Disorders 18 (2012) 864e870

869

Appendix 1. ANAM4-PD battery test descriptions.

Test name

Description

2-Choice Reaction Time

Measures attention and processing speed by having the user respond as quickly as possible to a stimulus (of 2) as soon as the stimulus appears (e.g., press left mouse button if you see “*” and right mouse button if you see “o”). Measures attention, concentration, and working memory. Single characters are displayed on the screen in rapid sequence. The user presses designated buttons to indicate if the displayed character matches or does not match the preceding character. Measures visual spatial discrimination and working memory by presenting the user with a visual pattern for a specified period of time and then, following a brief delay, asking the user to select the previously seen pattern from two choices. Measures visuo-spatial problem solving, as well as left-right orientation by asking the user to indicate which of a man’s hands is holding a target object. The man may be standing upright or upside down and either facing toward the user or facing away. Measures attention, basic computational skills, and working memory by asking the user to solve a single-digit arithmetic problem (e.g., "52 þ 3¼") involving two operations. Measures attention and processing speed by having the user respond as quickly as possible to different sets of stimuli based on simple rules (e.g., press left mouse button if you see a 2 or 3 and right mouse button if you see a 4 or 5). Measures simple motor reaction time by having the user respond as quickly as possible to a target stimulus. This test is a combination of the Manikin and Mathematical Processing tests and measures directed attention and executive function in addition to the abilities evaluated by the individual tests. One problem from each test appears simultaneously on the screen. The user is instructed to respond to one of the problems as directed by an arrow on the screen. Measures visuo-spatial ability, motor control, rule adherence, spatial planning, and strategy development and execution (similar to the Tower of Hanoi test). Measures visuo-motor control by having the user track an object moving on the screen. In this variant, the object is traveling in a horizontal line. The user completes this test once using the preferred hand and once using the non-preferred hand. Measures visuo-motor control by having the user track an object moving on the screen. In this variant, the object is traveling in a square wave pattern. The user completes this test once using the preferred hand and once using the non-preferred hand.

Running Memory Continuous Performance Test Matching to Sample

Manikin

Mathematical Processing Procedural Reaction Time

Simple Reaction Time Switching

Tower Puzzle Pursuit Tracking e Horizontal

Pursuit Tracking e Square Wave

Appendix 2. Correlations between UPDRS total score, UPDRS motor score, ANAM4-PD and traditional neuropsychological variables.

UPDRS total UPDRS Motor Neuro Global CES Fluency Switch Graphic Seq Hooper WCST HVLT Fluency Picture Comp Process Spd Tapping Pegboard Drawing Pursuit Concentration Reaction Time Switch Reaction Choicer Reaction Matching Math Manikin Tower

UPDRS motor

Neuro global

CES

Fluency Switch

Graphic Seq

Hooper

WCST

HVLT

Fluency

Picture Comp

0.96*** 0.37**

0.32**

0.29* 0.11

0.25* 0.04

0.82*** 0.73***

0.6***

0.17

0.13

0.74***

0.62***

0.53***

0.11 0.34** 0.18 0.21 0.06

0.09 0.27* 0.1 0.19 0.08

0.64*** 0.45*** 0.63*** 0.67*** 0.49***

0.48*** 0.34** 0.55*** 0.53*** 0.40**

0.39** 0.18 0.61*** 0.61*** 0.29*

0.43*** 0.11 0.41*** 0.53*** 0.19

0.21 0.28* 0.43*** 0.57***

0.26* 0.14 0.21

0.30* 0.35**

0.21

0.22

0.16

0.86***

0.75***

0.68***

0.62***

0.49***

0.40***

0.61***

0.58***

0.38** 0.45*** 0.35** 0.22* 0.26* 0.17

0.41*** 0.41*** 0.30** 0.23 0.24* 0.21

0.5*** 0.82*** 0.69*** 0.53*** 0.71*** 0.46***

0.50*** 0.67*** 0.48*** 0.51*** 0.79*** 0.66***

0.16 0.52*** 0.49*** 0.35** 0.58*** 0.29*

0.31** 0.60*** 0.48*** 0.48*** 0.52*** 0.34**

0.10 0.50*** 0.30* 0.32** 0.44*** 0.27*

0.16 0.28* 0.40*** 0.30* 0.25* 0.15

0.09 0.52*** 0.38** 0.23 0.50*** 0.30*

0.34** 0.42*** 0.39** 0.32** 0.52*** 0.18

0.20

0.15

0.73***

0.80***

0.57***

0.57***

0.39**

0.29*

0.47***

0.55***

0.28*

0.12

0.13

0.42***

0.52***

0.36**

0.30*

0.17

0.13

0.26*

0.36**

0.19

0.19 0.15 0.14 0.20

0.18 0.09 0.14 0.16

0.61*** 0.57*** 0.58*** 0.64***

0.77*** 0.75*** 0.64*** 0.74***

0.38** 0.49*** 0.47*** 0.40**

0.51*** 0.44*** 0.43*** 0.47***

0.41*** 0.27* 0.20 0.43***

0.13 0.23* 0.13 0.20

0.39** 0.41*** 0.31** 0.55***

0.33** 0.45*** 0.48*** 0.39**

0.34** 0.28* 0.17 0.35**

***p < 0.001; **p < 0.01; *p < 0.05 Neuro Global ¼ Global Neuropsychological Score (traditional battery). Graphic Seq ¼ Graphic Sequencing. Picture Comp ¼ WAIS-III Picture Completion.

0.42*** 0.05 0.34** 0.26* 0.13 0.32** 0.21

870

K.A. Hawkins et al. / Parkinsonism and Related Disorders 18 (2012) 864e870

References [1] Williams-Gray CH, Foltynie T, Brayne CE, Robbins TW, Barker RA. Evolution of cognitive dysfunction in an incident Parkinson’s disease cohort. Brain 2007; 130:1787e98. [2] Kandiah N, Narasimhalu K, Lau PN, Seah SH, Au WL, Tan LC. Cognitive decline in early Parkinson’s disease. Mov Disord 2009;24:605e8. [3] Levin BE, Katzen HL. Early cognitive changes and nondementing behavioral abnormalities in Parkinson’s disease. Adv Neurol 2005;96:84e94. [4] Riedel O, Klotsche J, Spottke A, Deuschl G, Forstl H, Henn F, et al. Cognitive impairment in 873 patients with idiopathic Parkinson’s disease. Results from the German Study on Epidemiology of Parkinson’s Disease with Dementia (GEPAD). J Neurol 2008;255:255e64. [5] Marder K. Cognitive impairment and dementia in Parkinson’s disease. Mov Disord 2010;25(Suppl. 1):S106e10. [6] Kehagia AA, Barker RA, Robbins TW. Neuropsychological and clinical heterogeneity of cognitive impairment and dementia in patients with Parkinson’s disease. Lancet Neurol 2010;9:1200e13. [7] Muslimovic D, Schmand B, Speelman JD, de Haan RJ. Course of cognitive decline in Parkinson’s disease: a meta-analysis. J Int Neuropsychol Soc 2007; 13:920e32. [8] Litvan I, Goldman JG, Troster AI, Schmand BA, Weintraub D, Petersen RC, et al. Diagnostic criteria for mild cognitive impairment in Parkinson’s disease: Movement Disorder Society Task Force guidelines. Mov Disord, published online Jan 26 2012, doi:10.1002/mds.24893. [9] Litvan I, Aarsland D, Adler CH, Goldman JG, Kulisevsky J, Mollenhauer B, et al. MDS task force on mild cognitive impairment in Parkinson’s disease: critical review of PD-MCI. Mov Disord 2011;26:1814e24. [10] Friedl KE, Grate SJ, Proctor SP, Ness JW, Lukey BJ, Kane RL. Army research needs for automated neuropsychological tests: monitoring soldier health and performance status. Arch Clin Neuropsychol 2007;22(S):S7e14.

[11] Reeves DL, Winter K, LaCour S, Raynsford K, Elsmore T. Automated neuropsychological assessment metrics documentation. In: Test administration guide, vol. 1. Silver Springs, Maryland: Office of Military Performance Assessment Technology; 1992. [12] (C-SHOP) CftSoHOM. Automated neuropsychological assessment metrics (version 4). Norman, OK: University of Oklahoma; 2007. [13] Kane RL, Roebuck-Spencer T, Short P, Kabat M, Wilken J. Identifying and monitoring cognitive deficits in clinical populations using Automated Neuropsychological Assessment Metrics (ANAM) tests. Arch Clin Neuropsychol 2007;22(Suppl. 1):S115e26. [14] Kulisevsky J, Pagonabarraga J. Cognitive impairment in Parkinson’s disease: tools for diagnosis and assessment. Mov Disord 2009;24:1103e10. [15] Mahieux F, Boller F, Fermanian J, Guillard G. Mini-mental Parkinson: first validation study of a new bedside test constructed for Parkinson’s disease. Behav Neurol 1995;8:15e22. [16] Marinus J, Visser M, Verwey NA, Verhey FR, Middelkoop HA, Stiggelbout AM, et al. Assessment of cognition in Parkinson’s disease. Neurology 2003;61: 1222e8. [17] Kalbe E, Calabrese P, Kohn N, Hilker R, Riedel O, Wittchen HU, et al. Screening for cognitive deficits in Parkinson’s disease with the Parkinson neuropsychometric dementia assessment (PANDA) instrument. Parkinsonism Relat Disord 2008;14:93e101. [18] Pagonabarraga J, Kulisevsky J, Llebaria G, Garcia-Sanchez C, Pascual-Sedano B, Gironell A. Parkinson’s disease-cognitive rating scale: a new cognitive scale specific for Parkinson’s disease. Mov Disord 2008;23:998e1005. [19] Thorne DR. Throughput: a simple performance index with desirable characteristics. Behav Res Meth 2006;38:569e73. [20] Strauss E, Sherman EMS, Spreen O. A compendium of neuropsychological tests: administration, norms, and commentary. 3rd ed. New York: Oxford University Press; 2006.

Appendix 2 (continued) Process Spd

Tapping

Pegboard

Drawing

Pursuit

Concentration

Reaction Time

Switch

0.31** 0.70*** 0.48*** 0.48*** 0.61*** 0.33**

0.37** 0.37** 0.34** 0.40** 0.43***

0.43*** 0.37** 0.58*** 0.44***

0.41*** 0.37** 0.18

0.37** 0.43***

0.46***

0.66***

0.41***

0.57***

0.49***

0.38**

0.61***

0.30**

0.35**

0.29*

0.35**

0.23

0.10

0.51***

0.44***

0.30*

0.58*** 0.58*** 0.50*** 0.58***

0.30** 0.24* 0.54*** 0.32**

0.54*** 0.39** 0.42*** 0.61***

0.41*** 0.41*** 0.41*** 0.27*

0.36** 0.34** 0.36** 0.30*

0.46*** 0.60*** 0.42*** 0.58***

0.51*** 0.35** 0.29* 0.49***

0.64*** 0.76*** 0.69*** 0.52***

Choice

Match

Math

Manikin

0.35** 0.32** 0.18 0.28*

0.53*** 0.40** 0.62***

0.35** 0.39**

0.41***