Further Investigation of Traumatic Brain Injury Versus Insufficient Effort with the California Verbal Learning Test

Archives of Clinical Neuropsychology, Vol. 15, No. 2, pp. 105–113, 2000 Copyright © 2000 National Academy of Neuropsychology Printed in the USA. All rights reserved 0887-6177/00 $–see front matter

PII S0887-6177(98)00153-X

Further Investigation of Traumatic Brain Injury Versus Insufficient Effort with the California Verbal Learning Test Jerry J. Sweet Evanston Hospital/Northwestern University and Medical School

Penny Wolfe Evanston Hospital/Illinois Institute of Technology

Elizabeth Sattlberger, Bobbi Numan, and J. Peter Rosenfeld Northwestern University

Steven Clingerman Cook County Hospital

Kristie J. Nies Blue Ridge Neuroscience Center

The present study replicates and attempts to extend previous research using the California Verbal Learning Test (CVLT) to identify malingerers. Documented moderate and severe traumatic brain injury patients (n 5 42) were compared with clinical malingerers identified by criteria other than the CVLT (n 5 21), malingering simulators instructed in realistic potential injury sequelae (n 5 25), and normal controls (n 5 21). Results of discriminant function analyses for high and low base rates are reported, showing similar results. Also, the four individual cutoff scores (Recognition Hits, Discriminability, Total Words Recalled, Long Delay Cued Recall) from Millis, Putnam, Adams, and Ricker (1995) were evaluated with these groups. Similar specificity rates were found with all four variables, while sensitivity rates were slightly lower than that of Millis. Adjusted cutoffs derived from the new samples resulted in slightly improved overall classification rates. Overall, present findings support those of Millis et al. (1995) with regard to the use of the CVLT in detection of malingering. Exploratory use of Total Intrusions and Recognition Hits Compared to Long Delay Free Recall was not promising. Simulators were

Portions of this research were presented at the Annual Meeting of the National Academy of Neuropsychology, November 1997, Las Vegas, Nevada. The authors gratefully acknowledge Scott Millis, PhD, for comments on an earlier draft of this manuscript. Address correspondence to: Jerry Sweet, Neuropsychology Service, Evanston Hospital, 2650 Ridge Avenue, Evanston, IL 60201.

105

106

J. J. Sweet et al. found to be fairly comparable in performance to actual malingerers, affirming their use in malingering research. © 2000 National Academy of Neuropsychology. Published by Elsevier Science Ltd

As an increasing need to detect malingering has been recognized within the field of clinical neuropsychology, increased attention to development of new assessment strategies has been evident. By definition, malingering is not a diagnostic conclusion that rests solely on test performance. In this regard, the present article distinguishes between insufficient effort1, operationally defined as test performance not associated with brain dysfunction and not attributable to moderator variables (e.g., age, depression, education, etc.) and significantly worse than performance standards known to reflect genuine neurological disorder, and malingering, operationally defined as a specific instance of purposeful insufficient effort to obtain external gain. Not all insufficient effort represents malingering (e.g., pseudodementia). As neuropsychologists have become more interested in detection of malingering, efforts to identify or devise improved methods and strategies have resulted in greater sophistication and effectiveness in arriving at this diagnostic conclusion (see reviews by Franzen, Iverson, & McCracken, 1990; Nies & Sweet, 1994; Rogers, Harrell, & Liff, 1993). Among other developments, there has been an increased interest in examining possible contributions of traditional measures already in use by practitioners. For example, researchers have investigated frequently used instruments, such as the Halstead-Reitan Neuropsychological Battery (Mittenberg, Rotholc, Russell, & Heilbronner, 1996), Wechsler Adult Intelligence Scale-Revised (Mittenberg, Theroux-Fichera, Zielinski, & Heilbronner, 1995), Wechsler Memory Scale-Revised (Mittenberg, Azrin, Millsaps, & Heilbronner, 1993), Warrington Recognition Memory Test (Millis, 1992; Millis & Putnam, 1994), and the Booklet Category Test (Tenhula & Sweet, 1996), with regard to malingering. Numerous other traditional tests of ability have also been explored (see reviews in recent texts by Reynolds, 1998; Sweet, 1999). Because the California Verbal Learning Test (CVLT; Delis, Kramer, Kaplan, & Ober, 1987) produces numerous indices of immediate learning and delayed memory, and includes both recall and recognition formats, this instrument has been useful to clinicians in evaluating serial verbal learning capabilities. For a variety of reasons, the CVLT appears to have particular promise with regard to detection of malingering. Specifically, the measure is, in fact, a challenging learning and memory task that produces more than 25 quantitative age-corrected parameters that have been normed by the test authors. Given the complexity of the task format and the multitude of resulting scores, it is very unlikely that an individual intending to malinger will be successful in producing credible scores (i.e., performances that make sense neuropsychologically with regard to the disorder in question). Trueblood and Schmidt (1993) appear to have been the first to report the use of CVLT-derived scores relevant to malingering. Using small samples of actual clinical re-

1While

respecting the usefulness of the terms incomplete effort (Millis et al., 1995) as an improvement over previous delineations, this terminology could be interpreted literally to mean any effort below 100%, even 99%. The present authors prefer insufficient effort as a means of conveying that an explicit value judgement has been made regarding a performance that is incomplete and reaches a meaningful clinical threshold of incomplete or partial effort (e.g., as established on the Multi-Digit Memory Test for performance below the 90% correct response level; Guilmette, Hart, & Giuliano, 1993; Prigatano & Amin, 1993).

Performance in Malingering

107

ferrals of mild head injury who were given a variety of neuropsychological measures, these investigators found Recognition Hits to be sensitive to the presence of malingering and “questionable validity” groups when compared to well-motivated gender, age, and education matched mild head injury groups. Using similar groupings of larger samples of mild head injured clinical referrals, Trueblood (1994) found CVLT Recognition Hits and Total (List A Trials 1–5) words recalled to be significantly lower among those subjects demonstrating insufficient effort on independent indices of motivation. In this latter study, Trueblood also reported success in identifying insufficient effort using clinical cutoffs for Recognition Hits. These first two studies appear to indicate that mild head-injured individuals exhibiting insufficient effort can be discriminated on the CVLT from mild head-injured patients exhibiting motivated performances. In the first extensive evaluation of the CVLT in detecting possible malingering, Millis, Putnam, Adams, and Ricker (1995) found four CVLT variables (Total (List A Trials 1–5), Discriminability, Recognition Hits, Long-Delayed Cued Recall) able to differentiate mild head injury patients who demonstrated insufficient effort on independent indices of motivation from moderate to severe traumatic brain injury (TBI) patients who had demonstrated good motivation. Importantly, this study established that insufficient effort among mild head-injured patients is discriminable on the CVLT from motivated moderate and severe head injury patients. In other words, patients with more severe degrees of head injury were not incorrectly classified as exhibiting insufficient effort, which if it occurred in some circumstances (i.e., litigation), could mistakenly be construed as malingering. The methodology used in all three of these initial studies (i.e., Trueblood & Schmidt, 1993; Trueblood, 1994; Millis et al., 1995) involving the CVLT lends itself to evaluating the type of thresholds of performance used frequently by clinicians. Although Millis and Putnam (1997) have more recently explored sophisticated multivariate procedures based on the CVLT, which may have great potential in the future, the present study attempts to replicate and extend the procedures likely to be used by practitioners from the initial Millis et al. (1995) research. Because of the importance of demonstrating diagnostic discrimination between legitimate injuries that, because of greater than mild severity, might be expected to have greater impairment than mild cases, and thus may potentially be confused with malingerers who often show exaggerated deficits, the present study also used a TBI comparison group with moderate and severe injuries.

METHOD The CVLT was administered to 109 individuals, including moderate and severe TBI patients (n 5 42), clinical malingerers identified by consensual criteria other than the CVLT (n 5 21), malingering simulators instructed in realistic potential injury sequelae (n 5 25), and normal controls (n 5 21). Ages of the participants were: normals 5 mean 19.0 (SD 5 1.1), malingering simulators 5 18.4 (SD 5 .64), moderate/severe TBI 5 35.8 (SD 5 12.9), insufficient effort/clinical malingerers 5 40.7 (SD 5 8.2). Educational attainment of the participants was: normals 5 mean 13.0 (SD 5 1.1), malingering simulators 5 13.2 (SD 5 .55), moderate/ severe TBI 5 14.2 (SD 5 2.8), insufficient effort/clinical malingerers 5 13.0 (SD 5 2.8). The TBI and malingering groups represented clinical referrals for which extensive independent documentation of presence or absence of brain injury was available, as well as independent measures of insufficient effort (e.g., forced-choice digit recognition, Rey Memory for 15 Item). As in the Millis et al. (1995) study, well-established independent

108

J. J. Sweet et al.

criteria for determination of head injury severity were used to determine moderate and severe TBI. Among the TBI subjects, 22 received their injury in car or truck accidents, 3 in motorcycle accidents, 6 as pedestrians hit by vehicles, 4 in bicycle accidents, and 7 fell from heights. Of these, 41 brain injuries were complicated (e.g., with documented intracranial lesion and/or fractured skull) and 1 brain injury involved confirmed loss of consciousness without further complication. Confirmed by independent and documented observations of the immediate consequences of the brain injury, duration of loss of consciousness (LOC) was established in most cases, as follows: none 5 3; ,30 minutes 5 3; 30 to 60 minutes 5 1; .1 hour to 24 hours 5 3; .24 hours to 1 week 5 7; .1 week 5 7; and present, but indeterminate 5 4. Confirmed by independent and documented observations, posttraumatic amnesia (PTA) was established in most cases, as follows: none 5 3; ,30 minutes 5 1; 30 to 60 minutes 5 4; 24 hours to 1 week 5 12; and .1 week 5 12. In those cases for which LOC or PTA was not present or could not be established, independent evidence of brain injury was available (e.g., neuroimaging). Litigation was present in 13 cases at the time of evaluation. Performance on the Multi-Digit Memory Test (MDMT), a 72-item forced-choice digit recognition procedure used to evaluate motivation, for the TBI group was mean correct 5 70.3 (SD 5 2.5) and range of 63 to 72 correct. Rey Memory for 15 Items performance for the TBI group was mean rows 5 4.2 (SD 5 .9) with a range of 2 to 5 rows and mean items 5 13.0 (SD 5 2.1) with a range of 8 to 15 items. Of the 21 clinical referrals deemed to be malingerers, all were involved in either personal injury litigation or were in the process of being evaluated for disability benefits, and 17 claimed TBI as the cause of their symptoms and disability. MDMT performance for the malingering group was mean correct 5 50.8 (SD 5 12.7), with a range of 29 to 71 correct. Rey Memory for 15 Items performance for the malingering group was mean rows 5 2.6 (SD 5 1.2) with a range of 1 to 5 rows and mean items 5 9.5 (SD 5 2.7) with a range of 3 to 15 items. Given that malingering cannot be determined solely by a single test score and requires context, multiple criteria (poor effortful performance on MDMT and/or Rey 15 Item, evidence of insufficient effort on one or more traditional neuropsychological measures for which valid criteria have been established (e.g., Warrington RMT, Category Test), plus absence of credible history of neurotrauma, blatant discrepancy between potential injury and patient complaints, blatant discrepancy between type of potential disorder and neuropsychological presentation, and exaggerated patient presentation within a context of litigation or disability application) were investigated with all clinical referrals to identify or rule out malingering, as suggested by Nies and Sweet (1994). Patients were determined to be malingerers only when presenting with multiple of the above criteria and lacking a plausible alternative explanation for their behavior. From computer scoring results, the CVLT variables explored by Millis et al. (1995) were evaluated, using both discriminant function analysis and clinical cutoff scores. The utility of two additional measures was also investigated.

RESULTS Means and standard deviations of key CVLT measures are presented in Table 1. Results of discriminant function analysis for a two group classification (normal effort vs. insufficient effort) using the same three variables used by Millis et al. (1995) (Discriminability, Total (List A Trials 1–5), Long Delay Cued Recall) are presented in Table 2. There is a reduction in overall classification accuracy compared to Millis et al. (1995) of 91% to the present study of 78.9%. Adding a fourth variable of Recognition Hits to the

Performance in Malingering

109

TABLE 1 Means and Standard Deviations of Key Indicators Group Normal Effort CVLT Variable

Total M SD LDCR M SD Recognition M SD Discriminability M SD Intrusions M SD Recognition/Recall M SD

Insufficient Effort

Normals

Moderate/Severe TBI

Simulators

Malingerers

54.3 6.9

46.2 10.8

29.2 16.0

36.0 12.6

12.8 2.6

9.8 3.2

6.5 3.7

6.5 4.0

15.1 1.0

13.8 2.1

8.2 3.9

10.3 3.8

97.4 2.9

90.1 7.6

72.9 13.6

78.0 13.3

1.4 2.1

6.0 8.0

11.8 17.9

11.1 12.8

22.6 22.9

76.3 83.5

135.2 232.6

139.7 176.4

CVLT 5 California Verbal Learning Test; TBI 5 traumatic brain injury. Total 5 List A Trials 1–5 total words recalled; LDCR 5 Long Delay Cued Recall; Recognition 5 Recognition Hits; Discriminability 5 [(1 2 ((False Positives 1 Misses)/44)] 3 100; Intrusions 5 (Free and Cued Recall Total); Recognition/Recall 5 Recognition Hits Compared to Long Delay Free Recall.

discriminant function resulted in an improved classification of 85.3% in the present sample. Given the artificially high base rate of insufficient effort created by group composition (approximately 42%) in the present study, which is higher than generally reported in the malingering literature (Sweet, 1999) there was a need to evaluate the effect of a lower base rate on classification rates. Therefore, a second discriminant function analysis, using the three CVLT variables selected by Millis et al. (1995), was run using a lower malingering base rate of approximately 15% (11 of the actual malingerers randomly selected from the original group and all 63 of the normally motivated participants were included). Results of this analysis were essentially the same as that with the higher base rate, with an overall classification rate of 81.1%. Unstandardized discriminant function coefficients from these analyses are presented in Table 3. Results of clinical cutoffs from Millis et al. (1995) and results of adjusted cutoffs that

TABLE 2 Results of Discriminant Function Analysis for Two Group Classification (Normal Effort Versus Insufficient Effort)

CVLT Variables in Function

Total, LDCR, and Discrim Total, LDCR, Discrim, and Recognition Hits

Sensitivity

Specificity

Overall Accuracy (%)

73.9 78.3

82.5 90.5

78.9 85.3

CVLT 5 California Verbal Learning Test; Total 5 List A Trials 1–5 total words recalled; LDCR 5 Long Delay Cued Recall; Recognition 5 Recognition Hits; Discrim 5 [(1 2 ((False Positives 1 Misses/44)] 3 100.

110

J. J. Sweet et al.

TABLE 3 Unstandardized Discriminant Function Coefficients 42% Base Rate

25% Base Rate

CVLT Variable

Three Variables

Four Variables

Three Variables

Four Variables

Total LDCR Discriminability Recognition (Constant)

.021 .024 .073 – 27.359

.010 2.005 .040 .197 26.135

.016 2.018 .111 – 210.609

.007 2.020 .059 .252 28.940

CVLT 5 California Verbal Learning Test; Total 5 List A Trials 1–5 total words recalled; LDCR 5 Long Delay Cued Recall; Recognition 5 Recognition Hits; Discriminability 5 [(1 2 ((False Positives 1 Misses)/44)] 3 100.

maximize correct classification from the present study are presented in Table 4, as are results of cutoffs for the CVLT variables Intrusions and Recognition (Hits) Compared to Long Delay Free Recall. Given the similarity in performance of simulators and malingerers, these groups were combined into an “insufficient effort” group and contrasted to a combined group of “normal effort” subjects (TBIs plus controls), as shown in Table 5.

DISCUSSION Based only on present findings, without comparative reference to the earlier findings of Millis et al. (1995), it appears that some of the major scoring indices of the CVLT are able to distinguish between individuals who appear well-motivated (i.e., clinical TBI reTABLE 4 Percentage of Correct Classification, by Group, Using Cutoffsa Group CVLT Variable

Total

Cutoff

Original New LDCR Original New Recognition Original New Discriminability Original New Intrusions New Recognition/Recall New

Malingerer

Simulator

Moderate/Severe TBI

Normals

52 62 48 62 48 71 57 62 29 30

64 80 56 56 76 88 60 68 32 30

83 76 83 74 90 83 86 81 88 80

100 100 95 95 100 95 100 100 100 100

CVLT 5 California Verbal Learning test; TBI 5 traumatic brain injury; Total 5 List A Trials 1–5 total words recalled; Original 5 Recommended cutoff by Millis, Putnam, Adams, and Ricker (1995); New 5 Adjusted cutoffs based on present data; LDCR 5 Long Delay Cued Recall; Recognition 5 Recognition Hits; Discriminability 5 [(1 2 ((False Positives 1 Misses/44)] 3 100; Intrusions 5 (Free and Cued Recall Total); Recognition/ Recall 5 Recognition Hits Compared to Long Delay Free Recall. aThere is an inherent conflict between the need to pursue and report research on malingering, and ethical and professional considerations regarding the publication of such research findings (Ben-Porath, 1994; Berry, Lamb, Wetter, Baer, & Widiger, 1994). Fundamentally, allowing cutoff scores to be released into the public domain could lead to coaching, by nonpsychologists, of strategies to avoid detection of malingering. In keeping with current publishing precedent meant to be responsive to this concern, psychologists can obtain specific cutoff scores from the authors upon request.

Performance in Malingering

111

TABLE 5 Percent Correctly Classified, by Group, Using Cutoffsa Group CVLT Variable

Total LDCR Recognition Discriminability Intrusions Recognition/Recall

Cutoff

Insufficient Effort

Normal Effort

Overall Hit Rate (%)

Original New Original New Original New Original New New New

59 72 52 59 63 80 59 65 30 30

89 84 87 81 94 87 90 87 90 87

76 79 72 72 81 84 77 78 66 64

CVLT 5 California Verbal Learning Test; Total 5 List A Trials 1–5 total words recalled; Original 5 Recommended cutoff by Millis, Putnam, Adams, and Ricker (1995); New 5 Adjusted cutoffs based on present data; LDCR 5 Long Delay Cued Recall; Recognition 5 Recognition Hits; Discriminability 5 [(1 2 ((False Positives 1 Misses)/44)] 3 100; Intrusions 5 (Free and Cued Recall Total); Recognition/Recall 5 Recognition Hits Compared to Long Delay Free Recall. aThere is an inherent conflict between the need to pursue and report research on malingering, and ethical and professional considerations regarding the publication of such research findings (Ben-Porath, 1994; Berry, Lamb, Wetter, Baer, & Widiger, 1994). Fundamentally, allowing cutoff scores to be released into the public domain could lead to coaching, by nonpsychologists, of strategies to avoid detection of malingering. In keeping with current publishing precedent meant to be responsive to this concern, psychologists can obtain specific cutoff scores from the authors upon request.

ferrals and normal controls in the present study) and those who display independent evidence of insufficient effort (i.e., clinical referrals with independent evidence of insufficient effort within a context of external gain, and simulating malingerers in the present study). As a legitimate neuropsychological measure of learning and memory capacities, the CVLT lends further utility in this decision-making process with regard to possible malingering, as the measure has excellent face validity and obvious relevance to the frequent domain (i.e., memory) chosen by malingerers. These latter two test characteristics are thought to be important in development of better techniques for detection of malingering (see review by Nies & Sweet, 1994). Discriminant function analysis and clinical cutoff results in the present study were not as favorable as Millis et al. (1995), but still within acceptable ranges. Application of clinical cutoffs for the four variables suggested by Millis was reasonably effective. These clinical cutoffs could be adjusted slightly in the present sample to improve overall accuracy by reducing specificity slightly to increase sensitivity. Although the CVLT variables Intrusions and Recognition (Hits) Compared to Long Delay Free Recall seem conceptually relevant to the identification of insufficient effort on a rational basis, present results were not promising. The four variables of original interest appear to provide better group discrimination. A meaningful difference between the Millis et al. (1995) and present studies is education of TBI patients, with the present study having a meaningfully higher educational level (11.6 vs. 14.2). Education effects on the CVLT would be expected to result in better performance, perhaps making more educated participants appear less impaired. As education can exert a powerful moderating influence on some neuropsychological measures, and may have impacted both studies, clinicians are encouraged to consider their own clinical samples in this regard before applying clinical cutoffs.

112

J. J. Sweet et al.

Further comparison of the cutoffs of Trueblood and Schmidt (1993) and Trueblood (1994) can also be made, with regard to Recognition Hits. The adjusted cutoff used in the present study is higher than that of Millis et al. (1995), and is identical to that of the two Trueblood studies. Given that education in the Millis et al. (1995) and two Trueblood studies is comparable and below that of the present investigation, the effectiveness of this variable, at a relatively high cutoff setting, is apparently independent of education. Also noteworthy is the observation, supported within the present results and becoming generally evident in numerous relevant publications, that recognition memory techniques provide an excellent source of information with regard to detection of insufficient effort, which will in some cases reflect malingering. The similarity in performance between simulators and malingerers in the present study affirms the use of individuals engaging in insufficient effort in research pertaining to the more specific real world instance of malingering. Other investigators have also found comparable results between simulators and genuine malingerers when compared within the same study or across studies (e.g., Inman, Vickery, Berry, Lamb, Edwards, & Smith, 1998; Mittenberg et al., 1995;). Finally, local base rates will affect diagnostic accuracy of all clinicians. Although somewhat comforting to have found very similar classification rates using discriminant analysis with base rates set at approximately 42% and approximately 15% within the present study, at the low base-rate level these results could be viewed as unfavorable. Clinicians must examine the impact of their own local base rates on diagnostic effectiveness. A commonly accepted resolution of the “low base rate dilemma” (for base rates less than 50%, when diagnostic error rates exceed base rates) at present, with regard to malingering, is to create clinical decision rules that maximize specificity, knowing full well that the price for doing so is a reduction of sensitivity. Present results are positive enough to be taken as supporting the use of the CVLT in detection of insufficient effort and malingering, but given the diagnostic accuracy rates found and the significant real life implications of ruling out or diagnosing malingering, should not be viewed as supporting the use of CVLT scores in isolation to detect malingering. The authors view the present findings as supporting previous recommendations of a broader multidimensional diagnostic approach toward identifying insufficient effort and the subset of these patients who are truly malingering. For example, a litigating individual with a history of mild TBI and only one CVLT indicator of insufficient effort, with no additional evidence, might not arouse strong concern regarding malingering. However, if the same individual demonstrates three CVLT indicators, as well as evidence of insufficient effort on other traditional measures of ability and a forced-choice digit recognition procedure, excessive inconsistency within and across serial evaluations, serious discrepancy between test behavior and real life activities, and/or other suggested multidimensional criteria (Nies & Sweet, 1994), there should be a much stronger suspicion of malingering as the cause of the insufficient effort.

REFERENCES Ben-Porath, Y. (1994). The ethical dilemma of coached malingering research. Psychological Assessment, 6, 14–15. Berry, D., Lamb, D., Wetter, M., Baer, R., & Widiger, T. (1994). Ethical considerations in research on coached malingering. Psychological Assessment, 6, 16–17. Delis, D., Kramer, J., Kaplan, E., & Ober, B. (1987). California Verbal Learning Test-Adult version: Manual. San Antonio, TX: Psychological Corporation.

Performance in Malingering

113

Franzen, M., Iverson, G., & McCracken, L. (1990). The detection of malingering in neuropsychological assessment. Neuropsychology Review, 1, 247–279. Guilmette, T., Hart, K., & Giuliano, A. (1993). Malingering detection: The use of a forced-choice method in identifying organic versus simulated memory impairment. The Clinical Neuropsychologist, 7, 59–69. Inman, T., Vickery, C., Berry, D., Lamb, D., Edwards, C., & Smith, G. (1998). Development and initial validation of a new procedure for evaluating adequacy of effort given during neuropsychological testing: The Letter Memory Test. Psychological Assessment, 10, 128–139. Millis, S. (1992). The Recognition Memory Test in the detection of malingered and exaggerated memory deficits. The Clinical Neuropsychologist, 6, 406–414. Millis, S., & Putnam, S. (1994). The Recognition Memory Test in the assessment of memory impairment after financially compensable mild head injury: A replication. Perceptual and Motor Skills, 79, 384–386. Millis, S., & Putnam, S. (1997). The California Verbal Learning Test in the assessment of financially compensable mild head injury: Further developments [Abstract]. Journal of the International Neuropsychological Society, 3, 225. Millis, S., Putnam, S., Adams, K., & Ricker, J. (1995). The California Verbal Learning Test in the detection of incomplete effort in neuropsychological evaluation. Psychological Assessment, 7, 463–471. Mittenberg, W., Azrin, R., Millsaps, C., & Heilbronner, R. (1993). Identification of malingered head injury on the Wechsler Memory Scale-Revised. Psychological Assessment, 5, 34–40. Mittenberg, W., Rotholc, A., Russell, E., & Heilbronner, R. (1996). Identification of malingered head injury on the Halstead-Reitan Battery. Archives of Clinical Neuropsychology, 11, 271–281. Mittenberg, W., Theroux-Fichera, S., Zielinski, R., & Heilbronner, R. (1995). Identification of malingered head injury on the Wechsler Adult Intelligence Scale-Revised. Professional Psychology: Research and Practice, 26, 491–498. Nies, K., & Sweet, J. (1994). Neuropsychological assessment and malingering: A critical review of past and present strategies. Archives of Clinical Neuropsychology, 9, 501–552. Prigatano, G., & Amin, K. (1993). Digit Memory Test: Unequivocal cerebral dysfunction and suspected malingering. Journal of Clinical and Experimental Neuropsychology, 15, 537–546. Reynolds, C. (Ed.). (1998). Detection of malingered head injury during litigation. New York: Plenum Press. Rogers, R., Harrell, E., & Liff, C. (1993). Feigning neuropsychological impairment: A critical review of methodological and clinical considerations. Clinical Psychology Review, 13, 255–274. Sweet, J. (Ed.). (1999). Forensic neuropsychology: Fundamentals and practice. Lisse, Netherlands: Swets & Zeitlinger. Tenhula, W., & Sweet, J. (1996). Double cross validation of the Booklet Category Test in detecting malingered traumatic brain injury. The Clinical Neuropsychologist, 10, 104–116. Trueblood, W. (1994). Qualitative and quantitative characteristics of malingered and other invalid WAIS-R and clinical memory data. Journal of Clinical and Experimental Neuropsychology, 16, 597–607. Trueblood, W., & Schmidt, M. (1993). Malingering and other validity considerations in the neuropsychological evaluation of mild head injury. Journal of Clinical and Experimental Neuropsychology, 15, 578–590.