Clinical Note
The NIH Stroke Scale Is Unreliable in Untrained Hands Charles Andre´,
MD, PhD
Background and Purpose: The NIH (National Institutes of Health) Stroke Scale helps to define candidates for thrombolytic treatment and other important clinical decisions. We evaluated its use by last-year medical students who had no specific training in use of this scale. Methods: The scale (principles, utilization, methods for scoring, main pitfalls) was presented between 2 stroke lectures. Following a 30-minute study period, a case from the NIH training videotape (case 1, tape 2, total score 4) was presented, and the 13 items were scored by 42 voluntary students. The analysis focused on possible errors of potential clinical relevance. Results: Only 15 students (36%) scored all items correctly. Twenty (48%) gave results at least 2 points apart from the correct total score, and 14 students had scores ⱖ6, which could lead to the inappropriate use of recombinant tissue-type plasminogen activator (rt-PA). At least 10% of the students made mistakes in most (8 of 13) items of the scale. Conclusions: Graduating medical students are not ready to use the NIH Stroke Scale without specific training. The magnitude of the mistakes found in this study is enough to inappropriately include or exclude patients from rt-PA treatment. Training programs should be considered a sine qua non for qualification for rt-PA use in daily clinical practice. Key Words: Cerebral ischemia—Health education— stroke, acute—Thrombolytic therapy. Copyright © 2002 by National Stroke Association
The use of recombinant tissue-type plasminogen activator (rt-PA) following cerebral infarct (CI) depends on the correct and rapid interpretation of various neurological findings. Risks from rt-PA administration are increased if physicians responsible for the management of patients with hyperacute CI are not trained on this task. The NIH (National Institutes of Health) Stroke Scale (NIHSS) was conceived to guide physicians in this decision and in the follow-up of patients in the acute phase of
From the Department of Neurology, Hospital Universita´rio Clementino Fraga Filho, Federal University of Rio de Janeiro, Rio de Janeiro, Brasil. Received June 29, 2000; accepted February 12, 2002. Address reprint requests to Charles Andre´, MD, PhD, Servic¸o de Neurologia, Hospital Universita´rio Clementino Fraga Filho, Av. Brigadeiro Trompowsky s/n°, sala 10E36, Ilha do Funda o CEP.21941590, Rio de Janeiro Brasil. Copyright © 2002 by National Stroke Association 1052-3057/02/1101-0007$35.00/0 doi:10.1053/jscd.2002.123974
CI. A 3-part videotape was created to prepare and certify physicians involved in the NIH rt-PA trial. The untrained use of this scale could lead to a number of potentially serious problems. Specifically, rt-PA is usually contraindicated in patients with severe strokes (i.e., NIHSS ⬎22) and especially in those with very mild or rapidly recovering strokes.1 Also, the incorrect interpretation of neurological findings may lead to inappropriate upholding of rt-PA in eligible patients. This study evaluates the performance of graduating medical students of a large South American city, with no formal training, in the use of the NIHSS. This group may well represent the untrained general physicians and nonNeurology specialists that will probably first encounter most CI patients in the emergency room.
Subjects and Methods On May 22, 1999, the author gave 2 90-minute sequential lectures on the management of acute cerebral infarct
Journal of Stroke and Cerebrovascular Diseases, Vol. 11, No. 1 (January-February), 2002: pp 43-46
43
C. ANDRE´
44
Table 1. NIH Stroke Scale Scores (Tape 2, Case No. 1): Correct Values and Results From 42 Interns
Item
Patient (Correct values)
ⱖ1 Error
Ia (0-3) Ib (0-2) Ic (0-2) II (0-2) III (0-3) IV (0-3) V (0-4: each UL) VI (0-4: each LL) VII (0-2) VIII (0-2) IX (0-3) X (0-2) XI (0-2) Total (0-42)
0 0 0 0 0 0 0 0 0 1 1 1 1 4
3 (7%) 2 (5%) 2 (5%) 2 (5%) 3 (7%) 9 (21%) 7 (17%) 8 (19%) 6 (14%) 4 (10%) 6 (14%) 6 (14%) 8 (19%) 26 (62%)
ⱖ2 Errors*
Total Score ⱖ6†
1 (2%) 1 (2%)
5 (12%) 7 (17%) 2 (5%)
20 (48%)
14 (33%)
Abbreviations: LL, lower limb; UL, upper limb. *ⱖ2 errors: difference between the correct value and the answer of 2 or more points. †Total ⱖ6: total score obtained by observer equal to or greater than 6.
in a special course on medical emergencies directed to 42 last-year medical students preparing for residence selection exams. At the end of the first part of this activity, the NIHSS (objectives, development, and principles of utilization) was presented in 10 minutes and the present study was proposed to all students. A Portuguese version of the scale was distributed, and the audience was advised to study the NIHSS and discuss it with the author for the next 30 minutes. Following that, the first case of the videotape no. 2 (first of 2 certification tapes) was presented, and the medical students signalled the scores for each step as the corresponding semiotic maneuvres were sequentially shown in the original videotape (English language). At the end of the session, the author briefly discussed the results (total scores immediately available) and highlighted the importance of training for adequate use of the NIHSS. The patient shown exhibited normal level of conscience, verbal comprehension, visual fields, ocular movements, and upper and lower limb strength. The total score for this patient was 4, corresponding to mild to moderate disturbances in pain perception and simultaneous touch stimulation, language, and word articulation (1 point each). The main hypothesis of this study was that there would be a number of errors that would be enough to lead some individuals to make wrong clinical decisions regarding the use of rt-PA, specifically its indication as recommended in published criteria. For each item and total score, the percentage of wrong answers was calculated.
Results The correct item values for the studied patient and the scores found by the 42 medical students are shown in Table 1. Sixteen students (38%) found the correct total score for the patient, but only 15 (36%) answered all the items correctly (1 student arrived at the correct value with 2 errors in opposite directions). Twenty individuals (48%) found a total score at least 2 points apart from the correct value. A clear tendency to overestimate the severity of the neurological deficit was detected. Four students found values of 10 or more (maximum of 13). Except for items VIII, X, and XI, students always tended to overestimate the deficits. Also, 14 students (33%) arrived at scores of 6 or more, which theoretically would lead them to indicate rt-PA use inappropriately in actual practice. The items with a larger number of errors were those assessing strength in either the face (IV) or limbs (V and VI) and sensory extinction (XI). Items addressing pain sensation, language, and word articulation (VIII, IX, and X) also exhibited error percentages of at least 10% (Table 1).
Discussion Treatment with rt-PA in the first 3 hours following CI onset does not decrease mortality, but significantly increases the number of patients who will exhibit a complete or near total recovery after 3 months.2 Current contraindications for rt-PA use in CI include conditions in which the risk of clinically significant cerebral bleeding complications would outweigh its possible benefits.1 Se-
NIH STROKE SCALE IS UNRELIABLE IN UNTRAINED HANDS
verely ill patients with very large strokes may constitute a group with relative contraindications to rt-PA treatment.3 Also, 1 of the main concerns is the withholding of treatment in patients with isolated or very mild signs (e.g., mild and isolated paresis or isolated ataxia, disartria, or disturbances in skin sensation) or with rapidly and spontaneously resolving symptoms. As a general rule, NIHSS scores of 6 or less are found in patients with mild strokes and a good potential for complete spontaneous recovery. The main finding of the present study is the large number of errors with use of the NIHSS; only about one third of last-year medical students correctly assessed all items in the context of a relatively simple acute stroke case with a high probability of spontaneous recovery. At least one third of these future physicians could inappropriately expose a similar patient to rt-PA treatment. Difficulties in the interpretation of basic aspects of the neurological examination by these students are hence suggested. A large number of errors were detected in the various evaluation items. The few exceptions (level of consciousness, verbal comprehension, conjugate gaze, and visual fields [items I to III]) could simply reflect the absence of disturbances in the case studied. Problems in the identification and grading of skin perception disturbances (items VIII and XI) could possibly be justified by the absence of formal training. On the other hand, the tendency to detect paresis where it is not apparent is somewhat surprising. Small limb oscillation (without any drift) may have contributed to these errors, which can be of great importance, as the NIHSS is heavily influenced by strength analysis of the face and both upper and lower limbs (items IV, V, and VI: 19 of 42 points). The use of rt-PA in CI is gradually expanding, even in developing countries.4 The present study highlights 1 of the many problems of a medical system excluding neurologists from the evaluation of acute stroke patients. Incorrect indication or contraindication of rt-PA use or other antithrombotic drugs is an obvious concern, but other potentially serious errors could result from inadequate evaluation of the clinical course of the disease or interpretation of diagnostic tests. Clinicians, critical care physicians, and other emergency physicians are welcome in the stroke field. They should be adequately trained, however, in the subtle aspects of diagnosis (e.g., early CT signs of large infarcts) and clinical management of acute stroke patients. The clinical diagnosis of acute stroke by nonneurologists is not difficult.5,6 Even nurses and paramedics can be trained to adequately recognize the acute stroke syndrome7-9 or to assess parts of the NIHSS.10 A number of potentially serious problems make it imperative to any involved physician to become extensively familiar with the disease before assuming risky clinical decisions.11
45
As to rt-PA, it has been demonstrated that emergency physicians have marked difficulties in the interpretation of subtle computed tomography (CT) signs of acute CI, with up to 0% of correct identification in difficult cases and only 46% in moderately difficult cases.12 The good news is that increasing experience, and hence training, in CT analysis of acute stroke positively influences performance (from 59% to 71% in the identification of ischemic or hemorrhagic lesions).13 Training protocols may lead to high reliability in the early evaluation of ischemic changes.13 However, recent studies suggest that intraobserver and interobserver concordance problems may appear even among experienced researchers,14 and this could make it more difficult to safely introduce rt-PA treatment in routine clinical practice. Expertise in the use of the NIHSS is not innate. The National Institute of Neurological Disorders and Stroke (NINDS) rt-PA trial participants2 had to be trained and show proficiency in its use via the training videotapes. The author can only speculate on the level of training necessary for adequate use of the NIHSS, but this use can be learned by nonneurologists with audiovisual material.15 Until simpler methods of teaching prove dependable, this approach should probably be recommended. This small study had a number of limitations. It addressed only students and not physicians already working in emergency rooms. Only 1 single case was studied, and this suggested that some patients in routine clinical practice could inappropriately receive rt-PA. Nothing can be said about other settings, such as withholding treatment from eligible patients or giving it routinely to severely ill patients (i.e., those with NIHSS ⬎22). Finally, Brazilian students were tested with the original English version of the videotape, and this could have induced them to unnecessary errors in the areas of sensation and language. However, translation problems would hardly explain errors in items, such as limb antigravity power. In conclusion, there is a great potential for therapeutically relevant errors in the use of the NIHSS by untrained interns. A minimum training program is strongly recommended for any physician who will be involved in rt-PA treatment of acute CI patients. Acknowledgment: Training videotapes for the use of the NIHSS—National Institute of Neurological Disorders and Stroke (NINDS) rt-PA Stroke Study Investigators. National Institutes of Health Stroke Scale (NIHSS) Educational Tapes: Revised Version, 1999-05-29 —were recently updated and can be ordered. (Barbara C. Tilley, Ph.D, Medical University of South Carolina, Telephone: 001-843-876-1327, Fax: 002843-876-1127, E-mail:
[email protected]; or American Academy of Neurology (AAN) Member Services, Telephone; 1-800-879-1960 [only in USA], http://www.aan.com).
C. ANDRE´
46
References 1. Adams Jr HP, Brott TG, Furlan AJ, et al. Guidelines for thrombolytic therapy for acute stroke: A supplement to the guidelines for the management of patients with acute ischemic stroke. Stroke 1996;27:1711-1718. 2. The National Institute of Neurological Disorders and Stroke r-tPA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995; 333:1581-1587. 3. Brott T, Broderick J, Kothari R, et al. Generalized efficacy of t-PA for acute stroke—Subgroup analysis of the NINDS t-PA stroke trial. Stroke 1997;28:2119-2125. 4. Andre´ C, Moraes-Neto JBM, Novis SAP. Experience with t-PA treatment in a large South American city. J Stroke Cerebrovasc Dis 1998;7:255-258. 5. Gomes MM, Costa MF, Andre´ C, et al. Emergency physician’s diagnosis of stroke: An accuracy study. Arq Neuropsiquiatr 1998;56(3-B):523-527. 6. Ferro JM, Pinto AN, Falca o I, et al. Diagnosis of stroke by the nonneurologist: A validation study. Stroke 1998;29: 1106-1109. 7. Kidwell CS, Saver JL, Eckstein M, et al. High accuracy of emergency medical technician identification of stroke using the Los Angeles Prehospital Stroke Screen (LAPSS). Stroke 1998;29:313 (abstr)
8. Kidwell CS, Starkman S, Eckstein M, et al. Identifying stroke in the field: Prospective validation of the Los Angeles Prehospital Stroke Screen (LAPSS). Stroke 1999; 30:248 (abstr) 9. Gordon DL, Issenberg SB, LaCombe DM, et al. Prehospital provider training in the management of acute stroke. Stroke 1999;30:265 (abstr) 10. Kothari RU, Pancioli A, Liu T, et al. Cincinnati prehospital stroke scale: Validity and reproducibility. Stroke 1998;29:313 (abstr) 11. Chung SS, Hemphill JC. Factitious stroke presenting for acute treatment. Neurology 1998;50:A115 (abstr, suppl 4) 12. Kalafut M, Starkman S, Saver J, et al. Early infarct and hemorrhage CT interpretation by emergency physicians: Implications for thrombolytic stroke therapy. Stroke 1997;28:271 (abstr) 13. VonKummer R. Effect of training in reading CT scans on patient selection for ECASS II. Neurology 1998;51:S50S52. 14. Besson G, Moulin Garnier P, Crepin-Leblond, The Mast Group. Intraobserver concordance of the Neuroradiologis Reviewing Committee in CT scan reviewing in MAST-E. Acta Neurol Scand 1998;98:292-293. 15. Goldstein LB, Samsa GP. Extension of the reliability of the NIH Stroke Scale to non-neurologists. Stroke 1997; 28:236 (abstr)