Suboptimal Performance on Neuropsychological Tests in Patients with Suspected Chronic Toxic Encephalopathy

NeuroToxicology 24 (2003) 547–551

Suboptimal Performance on Neuropsychological Tests in Patients with Suspected Chronic Toxic Encephalopathy Moniek S.E. van Hout1,*, Ben Schmand2, Ellie M. Wekking3, Gerard Hageman1, Betto G. Deelman4 1

Department of Psychology, Medical Spectrum Twente Hospital, Enschede, The Netherlands Department of Neurology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands 3 Multifunctional Unit Psychiatry, Academic Medical Center/De Meren, Amsterdam, The Netherlands 4 Department of Neuropsychology, University of Groningen, Groningen, The Netherlands 2

Received 18 July 2002; accepted 4 April 2003

Abstract Suboptimal performance during neuropsychological testing can seriously complicate assessment in behavioral neurotoxicology. We present data on the prevalence of suboptimal performance in a group of Dutch patients with suspected chronic toxic encephalopathy (CTE) after long-term occupational exposure to solvents. One hundred and fortyfive subjects referred to one of two Dutch national assessment centers for CTE were administered the Amsterdam ShortTerm Memory Test (ASTM) and the Test of Memory Malingering (TOMM), two tests specifically developed for the detection of suboptimal performance. For both tests, very cautious cut-off scores were chosen with a specificity of 99%. Results indicated that suboptimal performance appears to be a substantial problem in this group of patients with suspected CTE after long-term exposure to organic solvents. Only 54% of our subjects obtained normal scores on both tests of malingering, i.e. at or above cut-off score. The two tests seemed to measure the same concept in that nearly all the subjects with low TOMM scores also had low ASTM scores. However, a higher proportion of subjects scored below the cut-off on the ASTM than on the TOMM. # 2003 Elsevier Science Inc. All rights reserved.

Keywords: Chronic toxic encephalopathy; Malingering; Neuropsychological tests; Solvents’ toxicity; Occupational exposure

INTRODUCTION Long-term exposure to organic solvents may lead to symptoms of impaired concentration and memory, irritability, fatigue, instability of affect and difficulties in impulse control. This syndrome is called chronic toxic encephalopathy (CTE), also referred to as toxicant-induced organic brain syndrome. The symptoms of CTE are difficult to objectify. Therefore, the World *

Corresponding author. Tel.: þ31-53-4872850; fax: þ31-53-4873072. E-mail address: [email protected] (M.S.E. van Hout).

Health Organization formulated criteria for the diagnostic classification (World Health Organization, 1985) and advised a core battery of neuropsychological tests for use in diagnosing CTE (World Health Organization, 1986). This battery focuses on the neuropsychological domains of psychomotor functioning, memory and attention. Neuropsychological testing is the keystone in substantiating cognitive complaints associated with solvent exposure. The results of neuropsychological testing, however, may be invalidated by intentional or unintentional production of mental symptoms or by aggravation of cognitive problems. Fabrication or aggravation of symptoms is often found in other syndromes

0161-813X/$ – see front matter # 2003 Elsevier Science Inc. All rights reserved. doi:10.1016/S0161-813X(03)00054-8

548

M.S.E. van Hout et al. / NeuroToxicology 24 (2003) 547–551

characterized by subjective complaints, such as chronic whiplash (Kessels et al., 2000; Schmand et al., 1998), chronic fatigue (Van der Werf et al., 2000), or postconcussional syndromes (Binder and Rohling, 1996). It should also be considered in patients who are engaged in litigation or financial compensation procedures (Binder and Rohling, 1996; Schmand et al., 1998). There may be a variety of reasons for fabrication or aggravation of symptoms in these populations, ranging from subconscious sick-role playing to hysterical presentations of various types to intentional faking. In this paper, we will use the neutral term ‘‘suboptimal performance’’ which has no motivational signification. In cases of suspected CTE, suboptimal performance may be a problem, because CTE symptoms are subjective and patients are often involved in litigation procedures. White and Proctor (1997), therefore, advocated the use of specific tests for motivation and malingering in patients with suspected CTE. Surprisingly, there are no data on the prevalence of suboptimal performance in patients with suspected CTE. In a survey of diagnostic protocols for CTE used in 18 assessment centers for occupational diseases throughout the world, Van der Hoek et al. found that only 3 centers used specific tests to detect suboptimal performance. Five centers used a lack of consistency across tests as a technique to detect suboptimal performance, whereas 10 centers did not explicitly operationalize the concept of suboptimal performance in their test interpretation (Van der Hoek et al., 2001). In this report, we present data on the prevalence of suboptimal performance in a large group of Dutch patients with suspected CTE. This study is part of a nation-wide research program concerning chronic toxic encephalopathy.

MATERIALS AND METHODS Participants Included were 184 subjects with suspected CTE who were referred by general practitioners or medical officers to one of the two locations of The Netherlands Center of Occupational Diseases in the years 1998– 2000, and completed the entire diagnostic protocol (Van der Hoek et al., 2000). This diagnostic protocol includes three stages. All 184 subjects reached the third stage. In the first stage, the symptoms and the medical and occupational history are assessed by interview, and routine blood tests are done. In order to pass on to the second stage, the following criteria should be met: long

and/or heavy exposure to organic solvents, relevant symptoms, a relation in time between exposure and development of symptoms and signs, and no obvious other cause for the disease. In the second stage, the computer-based neurobehavioral evaluation system (NES) is administered (Baker et al., 1985; Emmen et al., 1988). If NES testing reveals abnormal scores on two or more of six selected subtests, the patient enters the third stage. The third stage consists of a clinical neurological examination, the assessment of an exposure index, and a comprehensive standardized neuropsychological evaluation (Wekking et al., 2000). The occupational hygienist uses a retrospective exposure index to assess the level of exposure. This index consists of the sum of scores on four variables, namely, duration of exposure, level of exposure, symptoms of acute intoxication, and use of personal protection equipment. The exposure index ranges from 0 (no exposure) to 9 (high exposure). An exposure from 0 to 4 is classified as ‘‘low’’, 5 and 6 is classified as ‘‘intermediate’’ and 7–9 is classified as ‘‘high’’ exposure. Measures Two tests specifically devised for the detection of suboptimal performance were included in the neuropsychological battery, the Amsterdam Short-Term Memory Test (ASTM) (Schmand et al., 1999) and the Test of Memory Malingering (TOMM) (Tombaugh, 1996). Both tests have been constructed according to the symptom validity testing paradigm and have been validated in several studies (Bolan et al., 2002; Schagen et al., 1997; Schmand et al., 1998, 1999; Tombaugh, 1996). In a symptom validity test, a set of memory stimuli is presented, followed by a forced choice recognition task. The TOMM is a visual and the ASTM is a verbal memory test. Both tests appear to be difficult memory tasks, but even people with brain damage can perform the tests well. Norm-based cutoff scores have been established below which a subject is considered to be suboptimally performing (Schmand et al., 1999; Tombaugh, 1996). Analyses Percentages of subnormal TOMM and the ASTM scores were calculated. The advocated cut-off score of the TOMM was used. The specificity at this point is 99%. This implies that less than 1% of a reference group of neurological patients with various diagnoses, all of them cognitively impaired, score below cut-off. Using these cut-off scores, there is less than a 1%

M.S.E. van Hout et al. / NeuroToxicology 24 (2003) 547–551

chance that a person will incorrectly be considered to have a suboptimal performance (Tombaugh, 1996). For the ASTM, we used a more stringent cut-off score than the advised cut-off score. This was done to ensure an equally high specificity of 99% (Schmand et al., 1999). To assess possible relationships between the outcome measures and relevant subject characteristics like age, education, and exposure, we performed logistic regression analysis, with the TOMM and the ASTM as dependent, and the subject characteristics as independent variables.

RESULTS One hundred and eighty-four subjects took part in the diagnostic procedure. Tables 1 and 2 summarize the characteristics of the study group. In short, most subjects were males in their 40s with a lower occupational education. Mean exposure duration was 23 years, and modal level of exposure was intermediate. All subjects completed the TOMM and 145 the ASTM. For 39 subjects, the ASTM was not administered in the clinical evaluation, because these subjects exceeded the time limit of the neuropsychological assessment procedure. However, the ASTM was only skipped when the TOMM score gave no indication for suboptimal performance. The mean TOMM score for those subjects who did not complete the ASTM was 47.7 (S:D: ¼ 5:6) compared with 46.7 (S:D: ¼ 5:4) for those who completed both tests. The difference was not significant (Mann–Whitney U-test, P ¼ 0:18). The mean TOMM score for 184 subjects was 46.9 (S:D: ¼ 5:4; range ¼ 22–50); 34 out of 184 subjects (18.5%) performed below the TOMM cut-off score on either trial 2 or the retention trial. The mean ASTM score for N ¼ 145 subjects was 80.5 (S:D: ¼ 6:1; range ¼ 61–90); 63 subjects (43.4%) had scores below the ASTM cut-off score. One hundred and nine subjects (75.2%) had scores below the recommended ASTM cut-off score in the manual. No subjects significantly performed below chance level on either the TOMM or the ASTM. There was agreement between the TOMM and ASTM scores in 106 out of 145 cases (73%): 79 Table 1 Group characteristics

Age (years) Exposure duration (years)

N

Mean

S.D.

Range

184 184

46.7 22.7

8.3 10.0

25.5–67.2 1–43

549

Table 2 Group characteristics

Gender Male Female Educationa Primary education Lower occupational (no diploma) Lower occupational (diploma) Mid-level College Level of exposure Low Intermediate High Not assessed Occupation Painters/paint industry Spray-painters Printers Chemical/paint industry Upholsterers Others Missing

Frequency

Percentage

177 7

96.0 4.0

13 32 92 35 12

7.1 17.4 50.0 19.0 6.5

18 136 27 3

9.7 74.0 14.7 1.6

64 34 24 10 6 37 9

34.8 18.4 13.1 5.4 3.2 20.1 4.9

a Education is classified into five categories: primary education, lower occupational education without diploma, lower occupational education with diploma, mid-level occupational education, and college or university. This classification system is not based on years of education, but on intellectual level of education.

subjects (54.5%) had normal scores on both TOMM and ASTM (i.e. scores higher than the cut-off value), 27 subjects (18.6%) had low scores on both tests (i.e. scores lower than the cut-off value) (Table 3). Thirtysix subjects (24.8%) had normal TOMM scores, but low ASTM scores and only three subjects (2.1%) had low TOMM scores and normal ASTM scores. Spearman’s rank correlation between ASTM and TOMM scores was 0.54 (P < 0:01). Logistic regression analysis showed that TOMM and ASTM scores

Table 3 Number and percentage of CTE patients obtaining normal or low scores on the TOMM and ASTM ASTM

Total

Normal

Subnormal

TOMM Normal Subnormal

79 (54.5) 3 (2.1)

36 (24.8) 27 (18.6)

115 (79.3) 30 (20.7)

Total

82 (56.6)

63 (43.4)

145 (100.0)

The data are expressed as number of patients (percentage of total).

550

M.S.E. van Hout et al. / NeuroToxicology 24 (2003) 547–551

were independent of age, education, exposure duration and severity.

DISCUSSION The results of this nation-wide study indicate that suboptimal performance is indeed a substantial problem in patients with suspected CTE. Only 54% of our subjects performed normally on both tests of malingering. Nineteen percent had low scores on both tests, and 45% had low scores on at least one of the tests, although we used very cautious cut-off scores with a specificity of 99%. With the advised cut-off score for the ASTM, as many of 75% of the subjects could be considered to have a suboptimal performance. While this study group was a highly selected one, being referred by medical officers and general practitioners, and although it is not representative for the population working with solvents, it is representative of the small subpopulation with lasting cognitive complaints. The referral process probably even reduced the number of subjects with suboptimal performance, because subjects without significant exposure were not referred for further diagnosis. Viewed from this point, our results are even more striking. Our findings need not imply that a large proportion of patients with suspected CTE is consciously malingering during the neuropsychological evaluation. The behavior of these patients (in the test situation as well as in a broader context) may be due to their focussing on a somatic explanation for their complaints or to their feeling that medical practitioners or their social environment do not take their problems seriously. Whatever the reason for the suboptimal performance, it undermines the ability to accurately interpret the results of other neuropsychological tests. Thus, when specific tests for the detection of malingering give indication for suboptimal performance, we cannot conclude that the low scores of these subjects on other cognitive tests are due to neurological disorders. The two detection methods used gave rather different results for the proportion of poor performance. This difference reflects once more the problems of developing methods for detecting suboptimal performance. There is no ‘‘gold standard’’ to determine with certainty whether subjects are producing or aggravating symptoms other than the clinical tests or experimental procedures being investigated, so we cannot directly determine the criterion validity of these tests (Cercy et al., 1997). Nevertheless, the two tests seemed to measure the same concept in that nearly all the subjects

with low TOMM scores also had low ASTM scores. But low ASTM scores were not necessarily seen in association with low TOMM scores. The ASTM gave a higher proportion of scores consistent with suboptimal performance. These results are similar to those of Green et al. (2000), who compared the TOMM and another detection test for malingering, the Word Memory Test (WMT). They found that nearly all the subjects who failed the TOMM also failed the WMT, but many subjects passed the TOMM and failed the WMT. There are several possible explanations for this difference in results. Firstly, our test procedure of not administering the ASTM in case of lack of time, provided that TOMM scores were normal, may have resulted in a slight overestimation of suboptimal performance on the ASTM. However, this is not very probable because differences between the subgroup administered only the TOMM and the subgroup administered both the TOMM and the ASTM were not significant. So, the test procedure alone cannot account for the discrepancy. Secondly, differences in the objective difficulty of both tests may have influenced our results. Validation studies show that the TOMM score may be normal in patients with Alzheimer’s disease, whereas the ASTM test is considered to be too difficult for these patients (Schmand et al., 1999). One could argue that the poor ASTM scores of our subjects thus reflect severe cognitive problems. This, however, is not very probable, because the symptoms of our patient group were not nearly as severe as those of demented patients and did not fit the diagnostic criteria for dementia. A third explanation is that subjects may perceive the distraction element in the ASTM as very difficult, and in this sense the test result may be a reflection of their attention problems. However, even subjects with attention problems as a long-term result of traumatic brain injury have normal scores on the ASTM. A fourth possible explanation is that subjects perceive the ASTM, a verbal task, as more relevant to their subjective memory complaints than the TOMM, a visual task. If this is the case the ASTM might be more sensitive than non-verbal tasks in detecting suboptimal performance. Gervais et al. (1999) illustrated this possibility. They administered a memory complaint questionnaire to a group of patients undergoing compensation-related neuropsychological evaluation. Subjects reported more verbal memory complaints than numerical and visual spatial memory complaints. A study by Bolan et al. (2002) confirms that subjects perceive the ASTM as more difficult than the TOMM, which renders the ASTM test more sensitive to suboptimal performance.

M.S.E. van Hout et al. / NeuroToxicology 24 (2003) 547–551

In conclusion, the results of this study underscore the notion that clinicians evaluating patients with cognitive complaints after long-term exposure to organic solvents must be alert to the possibility of malingering and other kinds of suboptimal performance and that specific detection methods should be part of the diagnostic protocol. However, it should be borne in mind that the detection tests currently in use do not give direct evidence for a deliberate malingering intention of the subject. Suboptimal performance can be the result of a range of motivations, from a deliberate, fraudulent intention to an unconscious aggravation of symptoms.

ACKNOWLEDGEMENTS The Solvent Team Project is funded by the Health Care Insurance Board, Amstelveen, The Netherlands. We are grateful to J. van der Palen, Ph.D., for his advise on statistical issues and express our gratitude to I.J. Berg, J.A.F. van der Hoek and J. Ba¨ r-Sykes for their suggestions on improving the text of this article. We also thank M. van Bruggen and I. Oppers for their careful data management.

REFERENCES Baker EL, Letz RE, Fidler A. A computer-administered neurobehavioral evaluation system for occupational and environmental epidemiology. J Occup Med 1985;27:206–12. Binder LM, Rohling ML. Money matters: a meta-analytic review of the effects of financial incentives on recovery after closedhead injury. Am J Psychiatry 1996;153(1):7–10. Bolan B, Foster JK, Schmand B, Bolan S. A comparison of three tests to detect feigned amnesia: the effects of feedback and the measurement of response latency. J Clin Exp Neuropsychol 2002;24(2):154–67. Cercy SP, Schretlen DJ, Brandt J. Simulated amnesia and the pseudo-memory phenomena. In: Rogers R, editor. Clinical assessment of malingering and deception. New York: Guilford Press; 1997. p. 85–107. Emmen HH, Hoogendijk EMG, Hooisma JT, Orlebeke JF, Uitdehage SHJ. Adaptation of two standardized international test

551

batteries for use in The Netherlands for detection of exposure to neurotoxic compounds. Internal Report 1988-18. Rijswijk, The Netherlands: TNO, Medical Biological Laboratory; 1988. Gervais R, Green P, Allen L. Differential sensitivity to symptom exaggeration of verbal visual and numerical symptom validity tests. Arch Clin Neuropsychol 1999;14(8):746–7. Green P, Allen L, Gervais R. Patterns of memory complaints in two consecutive series of compensation claimants passing or failing symptom validity tests. In: Proceedings of the Presentation of Poster at the Annual Meeting of the National Academy of Neuropsychology, Orlando, FL, 22 November 2000. Kessels RP, Aleman A, Verhagen WI, van Luijtelaar EL. Cognitive functioning after whiplash injury: a meta-analysis. J Int Neuropsychol Soc 2000;6(3):271–8. Schagen S, Schmand B, de Sterke S, Lindeboom J. Amsterdam Short-Term Memory Test: a new procedure for the detection of feigned memory deficits. J Clin Exp Neuropsychol 1997;19:43– 51. Schmand B, Lindeboom J, Schagen S, Heijt R, Koene T, Hamburger HL. Cognitive complaints in patients after whiplash injury. J Neurol Neurosurg Psychiatry 1998;64:339–43. Schmand B, de Sterke S, Lindeboom J. The Amsterdam ShortTerm Memory Test: manual. Lisse: Swets and Zeitlinger; 1999 Tombaugh TM. Test of Memory Malingering (TOMM): manual. Toronto: Multi Health Systems; 1996 Van der Hoek JAF, Verberk MM, Van der Laan G, Hageman G. A protocol for the diagnosis of solvent-induced CTE, 2 years of experience. Neurotoxicology 2000;21(5):887 [abstract]. Van der Hoek JAF, Verberk MM, van der Laan G, Hageman G. Routine diagnostic procedures for solvent-induced chronic toxic encephalopathy: survey of experts. Occup Environ Med 2001;58(6):382–5. Van der Werf SP, Prins JP, Jongen PJ, van der Meer JW, Bleijenberg G. Abnormal neuropsychological findings are not necessarily a sign of cerebral impairment: a matched comparison between chronic fatigue syndrome and multiple sclerosis. Neuropsychiatry Neuropsychol Behav Neurol 2000;13(3):199–203. Wekking EM, van Hout MSE, Emmen HH. The Dutch neuropsychological test battery for diagnosing CTE. Neurotoxicology 2000;21(5):887–8 [abstract]. White RF, Proctor SP. Solvents and neurotoxicity. Lancet 1997; 349:1239–43. World Health Organization. Chronic effects of organic solvents on the central nervous system and diagnostic criteria, vol. 5. Copenhagen: WHO Office of Environmental Health; 1985. World Health Organization. Operational guide for the WHO neurobehavioral core test battery. Geneva: WHO Office of Occupational Health; 1986.