- Email: [email protected]
Cincinnati Prehospital Stroke Scale: Reproducibility and Validity
ORIGINAL CONTRIBUTION
Cincinnati Prehospital Stroke Scale: Reproducibility and Validity
From the Department of Emergency Medicine* and the Department of Neurology,‡ University of Cincinnati Medical Center, Cincinnati, OH. Received for publication July 27, 1998. Revision received October 27, 1998. Accepted for publication November 2, 1998.
Rashmi U Kothari, MD* Arthur Pancioli, MD* Tiepu Liu, MD, DPH* Thomas Brott, MD‡ Joseph Broderick, MD‡
Supported by the Emergency Medicine Foundation through a grant from Genentech. Address for reprints: Rashmi Kothari, MD, Department of Emergency Medicine, University of Cincinnati, Post Office Box 670769, Cincinnati, OH 45267-0769; 513-558-5281, fax 513-558-5791; E-mail [email protected]. Copyright © 1999 by the American College of Emergency Physicians. 0196-0644/99/$8.00 + 0 47/1/96801
See editorial, p. 450. Study objective: The Cincinnati Prehospital Stroke Scale (CPSS) is a 3-item scale based on a simplification of the National Institutes of Health (NIH) Stroke Scale. When performed by a physician, it has a high sensitivity and specificity in identifying patients with stroke who are candidates for thrombolysis. The objective of this study was to validate and verify the reproducibility of the CPSS when used by prehospital providers. Methods: The CPSS was performed and scored by a physician certified in the use of the NIH Stroke Scale (gold standard). Simultaneously, a group of 4 paramedics and EMTs scored the same patient. Results: A total of 860 scales were completed on a convenience sample of 171 patients from the emergency department and neurology inpatient service. Of these patients, 49 had a diagnosis of stroke or transient ischemic attack. High reproducibility was observed among prehospital providers for total score (intraclass correlation coefficient [rI], .89; 95% confidence interval [CI], .87 to .92) and for each scale item: arm weakness, speech, and facial droop (.91, .84, and .75, respectively). There was excellent intraclass correlation between the physician and the prehospital providers for total score (rI, .92; 95% CI, .89 to .93) and for the specific items of the scale (.91, .87, and .78, respectively). Observation by the physician of an abnormality in any 1 of the 3 stroke scale items had a sensitivity of 66% and specificity of 87% in identifying a stroke patient. The sensitivity was 88% for identification of patients with anterior circulation strokes. Conclusion: The CPSS has excellent reproducibility among prehospital personnel and physicians. It has good validity in
APRIL 1999
33:4
ANNALS OF EMERGENCY MEDICINE
3 7 3
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
identifying patients with stroke who are candidates for thrombolytic therapy, especially those with anterior circulation stroke. [Kothari RU, Pancioli A, Liu T, Brott T, Broderick J: Cincinnati Prehospital Stroke Scale: Reproducibility and validity. Ann Emerg Med April 1999;33:373-378.] INTRODUCTION
Early recognition and prompt medical evaluation is critical for the use of thrombolytic therapy for patients with acute ischemic stroke. Patients must be treated with tissue plasminogen activator (t-PA) within 180 minutes of symptom onset for the treatment to be effective.1 To accomplish this, clinical centers have emphasized “prehospital education” and “en-route notification by EMS personnel.”2 Anecdotal experience at the University of Cincinnati indicated that early notification by paramedics via medical command may reduce time to treatment by allowing early mobilization of appropriate personnel.2,3 Based on this experience, we developed a 3-item stroke scale that could be used by prehospital providers to identify patients with stroke who are candidates for thrombolytic therapy.4 The Cincinnati Prehospital Stroke Scale (CPSS) was derived from a simplification of the 15-item National
Figure.
The CPSS evaluates for facial palsy, arm weakness, and speech abnormalities. Items are scored as either normal or abnormal. Facial Droop (The patient shows teeth or smiles) Normal: Both sides of face move equally. Abnormal: One side of face does not move as well as the other. Arm Drift (The patient closes their eyes and extends both arms straight out for 10 seconds) Normal: Both arms move the same, or both arms do not move at all. Abnormal: One arm either does not move, or one arm drifts down compared to the other. Speech (The patient repeats “The sky is blue in Cincinnati”) Normal: The patient says correct words with no slurring of words. Abnormal: The patient slurs words, says the wrong words, or is unable to speak.
3 7 4
Institutes of Health (NIH) Stroke Scale.5,6 The CPSS evaluates the presence or absence of facial palsy, asymmetric arm weakness, and speech abnormalities in potential stroke patients (Figure). This prehospital scale has been shown to have high sensitivity and specificity in identifying patients with stroke when performed by a physician on an emergency department population.4 The goal of this study was to verify the reproducibility of the CPSS scale when scored by prehospital care providers and to validate its ability to identify patients with stroke. M AT E R I A L S A N D M E T H O D S
A total of 24 prehospital care providers (17 paramedics and 7 EMTs) from University of Cincinnati Mobile Care Unit were evaluated during 23 different sessions. Groups of 4 to 11 patients with or without a final discharge diagnosis of stroke were identified from the ED and the inpatient neurology service for each of these 23 different sessions. The testing physician identified a convenience sample of patients from the ED. An attempt was made to identify patients with chief complaints that were suggestive of stroke or of other diseases that could be mistaken for stroke. To further increase the proportion of stroke patients and patients with stroke-mimicking conditions, patients were recruited from the inpatient neurology service as well as the ED. The inpatient physician on the neurology service identified patients with stroke, transient ischemic attack (TIA), a stroke-mimicking condition, or a combination of these conditions, as well as patients with other neurologic disorders, and gave a list of these patients to the testing physicians. The testing physicians were aware of the patients’ chief complaints but not of their clinical findings or final diagnoses. Paramedics and EMTs were blinded to all patient information, including the patients’ medical histories, clinical findings, and diagnoses.
Table 1.
Patient demographics. Variable No. patients Mean age (y) No. (%) male No. (%) black No. (%) white
All Patients
Stroke/TIA
171 57.8 72 (42) 102 (60) 69 (40)
49 62.5 17 (35) 29 (59) 20 (41)
ANNALS OF EMERGENCY MEDICINE
Nonstroke 122 55.8 55 (45) 73 (60) 49 (40)
33:4 APRIL 1999
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
Sensitivity, specificity, and positive and negative predictive values were determined by using the physicians’ and prehospital care providers’ scores and comparing them with the final hospital discharge stroke diagnoses. Patients with a diagnosis of TIA were excluded from this analysis since it was unknown when the neurologic deficits resolved. All statistical analyses were performed with the use of programs from SAS Institute Inc (Cary, NC). An SAS macro program (INTRACC) was used to calculate intraclass correlation coefficients.
The CPSS was performed and scored by 1 of 2 physicians (gold standard) certified in the use of the NIH Stroke Scale during each session. Simultaneously, a group of 4 paramedics and EMTs scored the same patient. To avoid fatiguing the patient, the scale was performed only by the physician while the prehospital personnel scored the patient’s response. Before each evaluation session, the physician conducted a 10-minute review on how to perform and score the CPSS with the 4 paramedics and EMTs. Only verbal instructions were given. This protocol was approved by the University of Cincinnati Institutional Review Board, and verbal consent was obtained from all patients. The intraclass correlation coefficient (rI) was calculated as the variance component among patients divided by all variance components (patients, raters, and residuals). The 95% confidence intervals (CIs) were calculated from F distribution.7 For reproducibility, rI was calculated for prehospital providers. For validity, raters were grouped as physicians versus prehospital personnel (EMTs and paramedics). Agreement between these 2 groups of raters was evaluated by r I for the total score and for each of the 3 stroke scale item scores. The rI has the same interpretation as κ statistics for measuring agreement and reliability.8 Because the total score was treated as a quantitative rating, there are multiple raters (EMTs and paramedics) for each patient, and the raters were different for each group of patients; therefore, we used rI instead of κ statistics. According to Fleiss’8 criteria, values greater than .75 represent excellent agreement, those between .4 and .75 represent fair to good agreement, and those lower than .4 represent poor agreement.
R E S U LT S
A total of 860 scales were completed on 171 patients. Of these patients, 38 had a final diagnosis of stroke and 11 a final diagnosis of TIA. There was no difference in terms of race or sex between stroke/TIA and nonstroke patients; however, nonstroke patients were significantly younger (mean difference, 6.7 years, 95% CI, 11.7 to 1.7 years), as shown in Table 1. Of the 38 patients with stroke, 14 (37%) had deficits involving the posterior circulation. Thirty-two patients (18.7%) had nonstroke neurologic disorders or altered mental status; 7 (21%) of these 32 patients had at least 1 abnormality on the CPSS (Table 2). Excellent reproducibility was observed among prehospital care providers for total score (rI , .89; 95% CI, .87 to .92) and for each scale item: arm weakness (rI, .91; 95% CI, .88 to .93); speech (rI, .84; 95% CI, .80 to .87); and facial droop (rI, .75; 95% CI, .69 to .80). In addition, there was excellent correlation for total score between the physician (gold standard) and the prehospital providers
Table 2.
Percentage of stroke and nonstroke patients with abnormalities detected on the CPSS (n = number of CPSS ratings performed).
Abnormality Type Face Arm Speech No. None 1 2 3
APRIL 1999
All Patients MD EMS (n=171) (n=689)
Stroke
TIA
MD (n=38)
EMS (n=154)
MD (n=11)
EMS (n=45)
Neurologic Nonstroke MD EMS (n=32) (n=129)
Nonneurologic MD EMS (n=90) (n=361)
7 15 15
9 14 13
18 47 37
25 37 38
0 0 9
0 0 0
9 16 13
11 18 12
2 3 6
3 4 4
73 15 6 3
78 13 5 4
34 39 16 11
41 32 14 13
91 9 0 0
100 0 0 0
78 9 9 3
77 13 5 5
91 7 2 0
92 6 1 1
33:4
ANNALS OF EMERGENCY MEDICINE
3 7 5
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
(rI, .92; 95% CI, .89 to .93), with no difference related to level of training (rI for paramedics, .88, with 95% CI, .85 to .91; rI for EMTs, .85, with 95% CI, .81 to .89). Agreement on scoring of specific items between physicians and prehospital personnel was excellent for all 3 items—arm weakness (rI, .91), speech (rI, .87), and facial droop (rI, .78). Again, there was no difference in terms of level of training (Table 3). Presence of a single abnormality on the CPSS had a sensitivity of 66% and a specificity of 87% in identifying a patient with stroke when scored by a physician, and 59% and 89%, respectively, when scored by prehospital providers (Table 4). Of the 13 patients with stroke who were not identified by an abnormality on the prehospital stroke scale, 10 had posterior circulation stroke (Table 5). The CPSS correctly identified 21 of the 24 patients with anterior circulation stroke (sensitivity, 87.5%; 95% CI, 67% to 97%). The 3 patients with anterior circulation strokes who were missed had minimal or atypical symptoms and would not have been candidates for thrombolysis. One of these patient presented with altered mental status and no focal deficits and was noted to have a subacute infarct in her left caudate. Another patient had a small subcortical infarct with only mild leg weakness. The third patient had a small lacunar infarct with only minimal symptoms. DISCUSSION
The rapid identification of potential stroke patients and early ED notification are important components of the prehospital management of stroke patients.9 The CPSS is a 3-item neurologic examination that was developed to assist paramedics and EMTs in identifying patients with stroke who are candidates for thrombolysis. This scale has been shown to be effective in identifying such stroke patients when it is performed by a trained physician.4 It can be taught in approximately 10 minutes and per-
formed in less than 1 minute.4 In this study, we found that the CPSS was easily taught, was reproducible, and was a valid tool when performed by paramedics and EMTs in identifying stroke patients who may be candidates for thrombolysis. Interobserver reproducibility measures the agreement among different persons using the same assessment tool. Poor reproducibility (variation among separate observers) increases error. Reproducibility among prehospital care providers using the CPSS was excellent both for total score (rI, .89) and for the individual stroke scale items (rI ranging from .75 to .91). Brott et al5 reported similar results using the 15-item NIH Stroke Scale, from which our stroke scale was derived. The NIH Stroke Scale uses a graded scoring system (eg, 0 to 2 or 0 to 4 points) for each item rather than the binary (normal/abnormal) score used by our CPSS. Brott et al5 found good interrater reliability for all items of the NIH Stroke Scale (mean κ, .69) among neurologists, emergency physicians, residents, and nurses. Interrater reliability ranged from excellent for motor arm (κ, .85), to moderate for dysarthria and best language (κ, .64 and .55, respectively), to poor for facial palsy (κ, .39).5 Similar results have been noted by other investigators.6,10,11 Reproducibility for other prehospital stroke scales has not been reported. A measurement tool is valid if it correctly describes the underlying phenomenon (eg, a specific neurologic deficit) or disease (eg, stroke).12 The CPSS accurately identifies specific neurologic abnormalities (ie, facial palsy, arm weakness, and speech abnormality) when performed by prehospital care providers. Correlation for the total score between prehospital care providers and the physician was excellent (rI, .92). Similarly, agreement between the physician and the prehospital providers on individual scale items was good (rI ranging from .78 to .91), with arm weakness being the most consistent. The
Table 4.
Sensitivity and specificity of the CPSS in identifying patients with stroke.
Table 3.
Agreement (rI and 95% CI) between physicians and prehospital care providers in scoring of individual CPSS items. Item
Paramedics
Facial palsy Arm Speech Total
.72 (.67–.78) .87 (.84–.90) .82 (.78–.86) .88 (.85–.91)
3 7 6
EMTs .64 (.55–.71) .86 (.81–.89) .77 (.71–.82) .85 (.81–.89)
Combined .78 (.74–.83) .91 (.89–.93) .87 (.34–.90) .92 (.89–.93)
Physicians No. of Sensitivity Specificity Abnormalities (95% CI) (95% CI)
Prehospital Care Providers Sensitivity Specificity (95% CI) (95% CI)
1 2 3
59 (51–67) 27 (21–35) 13 (8–20)
66 (49–80) 26 (14–43) 11 (3–26)
87 (80–92) 95 (90–98) 99 (95–100)
ANNALS OF EMERGENCY MEDICINE
88 (86–91) 96 (94–97) 98 (96–99)
33:4 APRIL 1999
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
CPSS is also valid in identifying patients with stroke (sensitivity, 66%; specificity, 87%), especially anterior circulation stroke (sensitivity, 88%). Of the 13 stroke patients who were missed by the CPSS, 10 had posterior circulation strokes and the other 3 remaining had minimal or atypical symptoms of their anterior circulation strokes and would not have been candidates for thrombolysis based on our current protocols. Presence of a single abnormality on the CPSS identified all patients with anterior circulation stroke who would have been candidates for thrombolytic therapy. The addition of a test for ataxia would have identified 6 of the 10 missed patients with posterior circulation stroke. However, posterior circulation strokes are difficult to diagnosis even by physicians13 and in our experience have been infrequently treated with intravenous thrombolytic agents. Furthermore, ataxia is one of the most poorly reproducible items on the NIH Stroke Scale.5,6,10 We previously reported that, when performed by a physician on an ED population, a single abnormality on the CPSS had a sensitivity of 100% and a specificity of almost 90% in identifying stroke patients who are candidates for thrombolysis. The difference between those results and the findings of the current study may relate to differences in the populations being studied or differences in the severity of the strokes being evaluated. In the previous study, patients were recruited only from the ED. In contrast, the patients in this study were recruited from both the ED and the inpatient neurology service (to increase the number of patients with strokes or strokemimicking conditions). This led to a greater proportion of patients (14 of 38) with posterior circulation stroke and a
Table 5.
Stroke patients not identified by the CPSS. Patient Sex Age (y) 1 2 3 4 5 6 7 8 9 10 11 12 13
M F M F F F F F F F M F M
APRIL 1999
42 70 72 73 75 72 72 54 70 59 50 79 40
33:4
Symptoms Ataxia and numbness Vertigo/dizziness Vertigo/dizziness Diploplia and vertigo Ataxia Mental status changes Vertigo/dizziness Visual disturbances Minimal unilateral weakness Minimal unilateral weakness Ataxia Mild unilateral leg weakness Minimal unilateral weakness
Final Diagnosis Cerebellar infarct Cerebellar infarct Pontine infarct Pontine infarct Cerebellar infarct Subacute caudate infarct Cerebellar infarct Occipital infarct Occipital infarct Brainstem infarct Cerebellar infarct Subcortical infarct Lacuanar infarct
ANNALS OF EMERGENCY MEDICINE
greater proportion of nonstroke patients with neurologic disorders and deficits, compared with the previous study or what would be expected in a prehospital population. In addition, in the previous study, the “final diagnosis” was that of a patient with stroke who was a candidate for thrombolysis (all patients scored 4 or higher on the NIH Stroke Scale, and all were treated with t-PA). In the current study, the “final diagnosis” combined all patients with a final hospital discharge diagnosis of stroke regardless of the severity of their deficit. The CPSS in this study could have missed a stroke patient with minimal symptoms, and in fact, the 3 patients with anterior circulation stroke who were missed all had minimal or atypical symptoms and would not have been candidates for thrombolysis by our current protocol. The only other prehospital stroke scale described in the medical literature is the Los Angeles Prehospital Stroke Scale. It evaluates arm strength, hand grip strength, and facial droop. Kidwell et al 14 reported that prehospital providers could effectively learn this scale and increase their stroke knowledge base after watching a brief training video. However, the validity and reproducibility of this scale have not been reported. There are a number of limitations to our study. First, the patient mix was not representative of the prehospital patient population. Almost a half of the study population had a stroke or another neurologic diagnosis. We purposefully tried to increase the number of patients with stroke or stroke-mimicking conditions to test this tool. The proportion of patients with posterior circulation stroke was also much higher than would be expected in the general population. The sensitivity of this tool may be higher when it is used in a population with a higher proportion of patients with anterior circulation stroke. The mode of testing was also somewhat artificial in that prehospital personnel were instructed on how to use the tool immediately before the exercise and were asked only to score the findings and not to try to elicit them. Ultimately this tool needs to be studied in the prehospital setting with paramedics and EMTs performing the examination under routine work conditions. The CPSS has excellent reproducibility among prehospital care providers and can be taught in less than 10 minutes. It can accurately identify patients with stroke, especially those with anterior circulation stroke, who are candidates for thrombolysis. We thank the men and women of the University of Cincinnati Mobile Care Unit for their assistance in this project.
3 7 7
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
REFERENCES 1. National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group: Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995;333:1581-1587. 2. Barsan WG, Brott TG, Broderick JP, et al: Time of hospital presentation in patients with acute stroke. Arch Intern Med 1993;153:2558-2561. 3. Brott TG, Haley EC, Levy DE, et al: Urgent therapy for stroke: Part I. Pilot study of tissue plasminogen activator administered within 90 minutes. Stroke 1992;23:632-640. 4. Kothari RU, Hall K, Brott T, et al: Early stroke recognition: Developing an out-of-Hospital NIH Stroke Scale. Acad Emerg Med 1997;4:986-990. 5. Brott T, Adams HP Jr, Olinger CP, et al: Measurements of acute cerebral infarction: A clinical examination scale. Stroke 1989;20:864-870. 6. Lyden P, Brott T, Tilley B, et al: Improved reliability of the NIH Stroke Scale using video training. Stroke 1994;25:2220-2226. 7. Snedecor GW, Cochran WG: Statistical Methods, ed 7. Ames, IA: The Iowa State University Press, 1980:243-246. 8. Fleiss JL: Statistical Methods for Rates and Proportions, ed 2. New York: John Wiley & Sons, 1981:212-236. 9. Emergency Stroke Care Task Force: Acute stroke, in Cummins RO (ed): Advanced Cardiac Life Support. Dallas, TX: American Heart Association, 1997:10-1,10-28. 10. Schmulling S, Grond M, Rudolf J: Training as a prerequisite for reliable use of NIH Stroke Scale [letter]. Stroke 1998;29:1258-1259. 11. Goldstein LB, Bartels C, Davis JN: Interrater reliability of the NIH Stroke Scale. Arch Neurol 1989;46:660-662. 12. Lyden PD, Lau GT: A critical appraisal of stroke evaluation and rating scales. Stroke 1991;22:1345-1352. 13. Ferro JM, Pinto AN, Falcao I, et al: Diagnosis of stroke by the non-neurologist: A validation study. Stroke 1998;29:1106-1109. 14. Kidwell CS, Saver JL, Eckstein M, et al: High accuracy of emergency medical technician identification of stroke using the Los Angeles Prehospital Stroke Screen (LAPSS). Stroke 1998;29:313.
3 7 8
ANNALS OF EMERGENCY MEDICINE
33:4 APRIL 1999
Cincinnati Prehospital Stroke Scale: Reproducibility and Validity
From the Department of Emergency Medicine* and the Department of Neurology,‡ University of Cincinnati Medical Center, Cincinnati, OH. Received for publication July 27, 1998. Revision received October 27, 1998. Accepted for publication November 2, 1998.
Rashmi U Kothari, MD* Arthur Pancioli, MD* Tiepu Liu, MD, DPH* Thomas Brott, MD‡ Joseph Broderick, MD‡
Supported by the Emergency Medicine Foundation through a grant from Genentech. Address for reprints: Rashmi Kothari, MD, Department of Emergency Medicine, University of Cincinnati, Post Office Box 670769, Cincinnati, OH 45267-0769; 513-558-5281, fax 513-558-5791; E-mail [email protected]. Copyright © 1999 by the American College of Emergency Physicians. 0196-0644/99/$8.00 + 0 47/1/96801
See editorial, p. 450. Study objective: The Cincinnati Prehospital Stroke Scale (CPSS) is a 3-item scale based on a simplification of the National Institutes of Health (NIH) Stroke Scale. When performed by a physician, it has a high sensitivity and specificity in identifying patients with stroke who are candidates for thrombolysis. The objective of this study was to validate and verify the reproducibility of the CPSS when used by prehospital providers. Methods: The CPSS was performed and scored by a physician certified in the use of the NIH Stroke Scale (gold standard). Simultaneously, a group of 4 paramedics and EMTs scored the same patient. Results: A total of 860 scales were completed on a convenience sample of 171 patients from the emergency department and neurology inpatient service. Of these patients, 49 had a diagnosis of stroke or transient ischemic attack. High reproducibility was observed among prehospital providers for total score (intraclass correlation coefficient [rI], .89; 95% confidence interval [CI], .87 to .92) and for each scale item: arm weakness, speech, and facial droop (.91, .84, and .75, respectively). There was excellent intraclass correlation between the physician and the prehospital providers for total score (rI, .92; 95% CI, .89 to .93) and for the specific items of the scale (.91, .87, and .78, respectively). Observation by the physician of an abnormality in any 1 of the 3 stroke scale items had a sensitivity of 66% and specificity of 87% in identifying a stroke patient. The sensitivity was 88% for identification of patients with anterior circulation strokes. Conclusion: The CPSS has excellent reproducibility among prehospital personnel and physicians. It has good validity in
APRIL 1999
33:4
ANNALS OF EMERGENCY MEDICINE
3 7 3
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
identifying patients with stroke who are candidates for thrombolytic therapy, especially those with anterior circulation stroke. [Kothari RU, Pancioli A, Liu T, Brott T, Broderick J: Cincinnati Prehospital Stroke Scale: Reproducibility and validity. Ann Emerg Med April 1999;33:373-378.] INTRODUCTION
Early recognition and prompt medical evaluation is critical for the use of thrombolytic therapy for patients with acute ischemic stroke. Patients must be treated with tissue plasminogen activator (t-PA) within 180 minutes of symptom onset for the treatment to be effective.1 To accomplish this, clinical centers have emphasized “prehospital education” and “en-route notification by EMS personnel.”2 Anecdotal experience at the University of Cincinnati indicated that early notification by paramedics via medical command may reduce time to treatment by allowing early mobilization of appropriate personnel.2,3 Based on this experience, we developed a 3-item stroke scale that could be used by prehospital providers to identify patients with stroke who are candidates for thrombolytic therapy.4 The Cincinnati Prehospital Stroke Scale (CPSS) was derived from a simplification of the 15-item National
Figure.
The CPSS evaluates for facial palsy, arm weakness, and speech abnormalities. Items are scored as either normal or abnormal. Facial Droop (The patient shows teeth or smiles) Normal: Both sides of face move equally. Abnormal: One side of face does not move as well as the other. Arm Drift (The patient closes their eyes and extends both arms straight out for 10 seconds) Normal: Both arms move the same, or both arms do not move at all. Abnormal: One arm either does not move, or one arm drifts down compared to the other. Speech (The patient repeats “The sky is blue in Cincinnati”) Normal: The patient says correct words with no slurring of words. Abnormal: The patient slurs words, says the wrong words, or is unable to speak.
3 7 4
Institutes of Health (NIH) Stroke Scale.5,6 The CPSS evaluates the presence or absence of facial palsy, asymmetric arm weakness, and speech abnormalities in potential stroke patients (Figure). This prehospital scale has been shown to have high sensitivity and specificity in identifying patients with stroke when performed by a physician on an emergency department population.4 The goal of this study was to verify the reproducibility of the CPSS scale when scored by prehospital care providers and to validate its ability to identify patients with stroke. M AT E R I A L S A N D M E T H O D S
A total of 24 prehospital care providers (17 paramedics and 7 EMTs) from University of Cincinnati Mobile Care Unit were evaluated during 23 different sessions. Groups of 4 to 11 patients with or without a final discharge diagnosis of stroke were identified from the ED and the inpatient neurology service for each of these 23 different sessions. The testing physician identified a convenience sample of patients from the ED. An attempt was made to identify patients with chief complaints that were suggestive of stroke or of other diseases that could be mistaken for stroke. To further increase the proportion of stroke patients and patients with stroke-mimicking conditions, patients were recruited from the inpatient neurology service as well as the ED. The inpatient physician on the neurology service identified patients with stroke, transient ischemic attack (TIA), a stroke-mimicking condition, or a combination of these conditions, as well as patients with other neurologic disorders, and gave a list of these patients to the testing physicians. The testing physicians were aware of the patients’ chief complaints but not of their clinical findings or final diagnoses. Paramedics and EMTs were blinded to all patient information, including the patients’ medical histories, clinical findings, and diagnoses.
Table 1.
Patient demographics. Variable No. patients Mean age (y) No. (%) male No. (%) black No. (%) white
All Patients
Stroke/TIA
171 57.8 72 (42) 102 (60) 69 (40)
49 62.5 17 (35) 29 (59) 20 (41)
ANNALS OF EMERGENCY MEDICINE
Nonstroke 122 55.8 55 (45) 73 (60) 49 (40)
33:4 APRIL 1999
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
Sensitivity, specificity, and positive and negative predictive values were determined by using the physicians’ and prehospital care providers’ scores and comparing them with the final hospital discharge stroke diagnoses. Patients with a diagnosis of TIA were excluded from this analysis since it was unknown when the neurologic deficits resolved. All statistical analyses were performed with the use of programs from SAS Institute Inc (Cary, NC). An SAS macro program (INTRACC) was used to calculate intraclass correlation coefficients.
The CPSS was performed and scored by 1 of 2 physicians (gold standard) certified in the use of the NIH Stroke Scale during each session. Simultaneously, a group of 4 paramedics and EMTs scored the same patient. To avoid fatiguing the patient, the scale was performed only by the physician while the prehospital personnel scored the patient’s response. Before each evaluation session, the physician conducted a 10-minute review on how to perform and score the CPSS with the 4 paramedics and EMTs. Only verbal instructions were given. This protocol was approved by the University of Cincinnati Institutional Review Board, and verbal consent was obtained from all patients. The intraclass correlation coefficient (rI) was calculated as the variance component among patients divided by all variance components (patients, raters, and residuals). The 95% confidence intervals (CIs) were calculated from F distribution.7 For reproducibility, rI was calculated for prehospital providers. For validity, raters were grouped as physicians versus prehospital personnel (EMTs and paramedics). Agreement between these 2 groups of raters was evaluated by r I for the total score and for each of the 3 stroke scale item scores. The rI has the same interpretation as κ statistics for measuring agreement and reliability.8 Because the total score was treated as a quantitative rating, there are multiple raters (EMTs and paramedics) for each patient, and the raters were different for each group of patients; therefore, we used rI instead of κ statistics. According to Fleiss’8 criteria, values greater than .75 represent excellent agreement, those between .4 and .75 represent fair to good agreement, and those lower than .4 represent poor agreement.
R E S U LT S
A total of 860 scales were completed on 171 patients. Of these patients, 38 had a final diagnosis of stroke and 11 a final diagnosis of TIA. There was no difference in terms of race or sex between stroke/TIA and nonstroke patients; however, nonstroke patients were significantly younger (mean difference, 6.7 years, 95% CI, 11.7 to 1.7 years), as shown in Table 1. Of the 38 patients with stroke, 14 (37%) had deficits involving the posterior circulation. Thirty-two patients (18.7%) had nonstroke neurologic disorders or altered mental status; 7 (21%) of these 32 patients had at least 1 abnormality on the CPSS (Table 2). Excellent reproducibility was observed among prehospital care providers for total score (rI , .89; 95% CI, .87 to .92) and for each scale item: arm weakness (rI, .91; 95% CI, .88 to .93); speech (rI, .84; 95% CI, .80 to .87); and facial droop (rI, .75; 95% CI, .69 to .80). In addition, there was excellent correlation for total score between the physician (gold standard) and the prehospital providers
Table 2.
Percentage of stroke and nonstroke patients with abnormalities detected on the CPSS (n = number of CPSS ratings performed).
Abnormality Type Face Arm Speech No. None 1 2 3
APRIL 1999
All Patients MD EMS (n=171) (n=689)
Stroke
TIA
MD (n=38)
EMS (n=154)
MD (n=11)
EMS (n=45)
Neurologic Nonstroke MD EMS (n=32) (n=129)
Nonneurologic MD EMS (n=90) (n=361)
7 15 15
9 14 13
18 47 37
25 37 38
0 0 9
0 0 0
9 16 13
11 18 12
2 3 6
3 4 4
73 15 6 3
78 13 5 4
34 39 16 11
41 32 14 13
91 9 0 0
100 0 0 0
78 9 9 3
77 13 5 5
91 7 2 0
92 6 1 1
33:4
ANNALS OF EMERGENCY MEDICINE
3 7 5
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
(rI, .92; 95% CI, .89 to .93), with no difference related to level of training (rI for paramedics, .88, with 95% CI, .85 to .91; rI for EMTs, .85, with 95% CI, .81 to .89). Agreement on scoring of specific items between physicians and prehospital personnel was excellent for all 3 items—arm weakness (rI, .91), speech (rI, .87), and facial droop (rI, .78). Again, there was no difference in terms of level of training (Table 3). Presence of a single abnormality on the CPSS had a sensitivity of 66% and a specificity of 87% in identifying a patient with stroke when scored by a physician, and 59% and 89%, respectively, when scored by prehospital providers (Table 4). Of the 13 patients with stroke who were not identified by an abnormality on the prehospital stroke scale, 10 had posterior circulation stroke (Table 5). The CPSS correctly identified 21 of the 24 patients with anterior circulation stroke (sensitivity, 87.5%; 95% CI, 67% to 97%). The 3 patients with anterior circulation strokes who were missed had minimal or atypical symptoms and would not have been candidates for thrombolysis. One of these patient presented with altered mental status and no focal deficits and was noted to have a subacute infarct in her left caudate. Another patient had a small subcortical infarct with only mild leg weakness. The third patient had a small lacunar infarct with only minimal symptoms. DISCUSSION
The rapid identification of potential stroke patients and early ED notification are important components of the prehospital management of stroke patients.9 The CPSS is a 3-item neurologic examination that was developed to assist paramedics and EMTs in identifying patients with stroke who are candidates for thrombolysis. This scale has been shown to be effective in identifying such stroke patients when it is performed by a trained physician.4 It can be taught in approximately 10 minutes and per-
formed in less than 1 minute.4 In this study, we found that the CPSS was easily taught, was reproducible, and was a valid tool when performed by paramedics and EMTs in identifying stroke patients who may be candidates for thrombolysis. Interobserver reproducibility measures the agreement among different persons using the same assessment tool. Poor reproducibility (variation among separate observers) increases error. Reproducibility among prehospital care providers using the CPSS was excellent both for total score (rI, .89) and for the individual stroke scale items (rI ranging from .75 to .91). Brott et al5 reported similar results using the 15-item NIH Stroke Scale, from which our stroke scale was derived. The NIH Stroke Scale uses a graded scoring system (eg, 0 to 2 or 0 to 4 points) for each item rather than the binary (normal/abnormal) score used by our CPSS. Brott et al5 found good interrater reliability for all items of the NIH Stroke Scale (mean κ, .69) among neurologists, emergency physicians, residents, and nurses. Interrater reliability ranged from excellent for motor arm (κ, .85), to moderate for dysarthria and best language (κ, .64 and .55, respectively), to poor for facial palsy (κ, .39).5 Similar results have been noted by other investigators.6,10,11 Reproducibility for other prehospital stroke scales has not been reported. A measurement tool is valid if it correctly describes the underlying phenomenon (eg, a specific neurologic deficit) or disease (eg, stroke).12 The CPSS accurately identifies specific neurologic abnormalities (ie, facial palsy, arm weakness, and speech abnormality) when performed by prehospital care providers. Correlation for the total score between prehospital care providers and the physician was excellent (rI, .92). Similarly, agreement between the physician and the prehospital providers on individual scale items was good (rI ranging from .78 to .91), with arm weakness being the most consistent. The
Table 4.
Sensitivity and specificity of the CPSS in identifying patients with stroke.
Table 3.
Agreement (rI and 95% CI) between physicians and prehospital care providers in scoring of individual CPSS items. Item
Paramedics
Facial palsy Arm Speech Total
.72 (.67–.78) .87 (.84–.90) .82 (.78–.86) .88 (.85–.91)
3 7 6
EMTs .64 (.55–.71) .86 (.81–.89) .77 (.71–.82) .85 (.81–.89)
Combined .78 (.74–.83) .91 (.89–.93) .87 (.34–.90) .92 (.89–.93)
Physicians No. of Sensitivity Specificity Abnormalities (95% CI) (95% CI)
Prehospital Care Providers Sensitivity Specificity (95% CI) (95% CI)
1 2 3
59 (51–67) 27 (21–35) 13 (8–20)
66 (49–80) 26 (14–43) 11 (3–26)
87 (80–92) 95 (90–98) 99 (95–100)
ANNALS OF EMERGENCY MEDICINE
88 (86–91) 96 (94–97) 98 (96–99)
33:4 APRIL 1999
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
CPSS is also valid in identifying patients with stroke (sensitivity, 66%; specificity, 87%), especially anterior circulation stroke (sensitivity, 88%). Of the 13 stroke patients who were missed by the CPSS, 10 had posterior circulation strokes and the other 3 remaining had minimal or atypical symptoms of their anterior circulation strokes and would not have been candidates for thrombolysis based on our current protocols. Presence of a single abnormality on the CPSS identified all patients with anterior circulation stroke who would have been candidates for thrombolytic therapy. The addition of a test for ataxia would have identified 6 of the 10 missed patients with posterior circulation stroke. However, posterior circulation strokes are difficult to diagnosis even by physicians13 and in our experience have been infrequently treated with intravenous thrombolytic agents. Furthermore, ataxia is one of the most poorly reproducible items on the NIH Stroke Scale.5,6,10 We previously reported that, when performed by a physician on an ED population, a single abnormality on the CPSS had a sensitivity of 100% and a specificity of almost 90% in identifying stroke patients who are candidates for thrombolysis. The difference between those results and the findings of the current study may relate to differences in the populations being studied or differences in the severity of the strokes being evaluated. In the previous study, patients were recruited only from the ED. In contrast, the patients in this study were recruited from both the ED and the inpatient neurology service (to increase the number of patients with strokes or strokemimicking conditions). This led to a greater proportion of patients (14 of 38) with posterior circulation stroke and a
Table 5.
Stroke patients not identified by the CPSS. Patient Sex Age (y) 1 2 3 4 5 6 7 8 9 10 11 12 13
M F M F F F F F F F M F M
APRIL 1999
42 70 72 73 75 72 72 54 70 59 50 79 40
33:4
Symptoms Ataxia and numbness Vertigo/dizziness Vertigo/dizziness Diploplia and vertigo Ataxia Mental status changes Vertigo/dizziness Visual disturbances Minimal unilateral weakness Minimal unilateral weakness Ataxia Mild unilateral leg weakness Minimal unilateral weakness
Final Diagnosis Cerebellar infarct Cerebellar infarct Pontine infarct Pontine infarct Cerebellar infarct Subacute caudate infarct Cerebellar infarct Occipital infarct Occipital infarct Brainstem infarct Cerebellar infarct Subcortical infarct Lacuanar infarct
ANNALS OF EMERGENCY MEDICINE
greater proportion of nonstroke patients with neurologic disorders and deficits, compared with the previous study or what would be expected in a prehospital population. In addition, in the previous study, the “final diagnosis” was that of a patient with stroke who was a candidate for thrombolysis (all patients scored 4 or higher on the NIH Stroke Scale, and all were treated with t-PA). In the current study, the “final diagnosis” combined all patients with a final hospital discharge diagnosis of stroke regardless of the severity of their deficit. The CPSS in this study could have missed a stroke patient with minimal symptoms, and in fact, the 3 patients with anterior circulation stroke who were missed all had minimal or atypical symptoms and would not have been candidates for thrombolysis by our current protocol. The only other prehospital stroke scale described in the medical literature is the Los Angeles Prehospital Stroke Scale. It evaluates arm strength, hand grip strength, and facial droop. Kidwell et al 14 reported that prehospital providers could effectively learn this scale and increase their stroke knowledge base after watching a brief training video. However, the validity and reproducibility of this scale have not been reported. There are a number of limitations to our study. First, the patient mix was not representative of the prehospital patient population. Almost a half of the study population had a stroke or another neurologic diagnosis. We purposefully tried to increase the number of patients with stroke or stroke-mimicking conditions to test this tool. The proportion of patients with posterior circulation stroke was also much higher than would be expected in the general population. The sensitivity of this tool may be higher when it is used in a population with a higher proportion of patients with anterior circulation stroke. The mode of testing was also somewhat artificial in that prehospital personnel were instructed on how to use the tool immediately before the exercise and were asked only to score the findings and not to try to elicit them. Ultimately this tool needs to be studied in the prehospital setting with paramedics and EMTs performing the examination under routine work conditions. The CPSS has excellent reproducibility among prehospital care providers and can be taught in less than 10 minutes. It can accurately identify patients with stroke, especially those with anterior circulation stroke, who are candidates for thrombolysis. We thank the men and women of the University of Cincinnati Mobile Care Unit for their assistance in this project.
3 7 7
CINCINNATI PREHOSPITAL STROKE SCALE Kothari et al
REFERENCES 1. National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group: Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995;333:1581-1587. 2. Barsan WG, Brott TG, Broderick JP, et al: Time of hospital presentation in patients with acute stroke. Arch Intern Med 1993;153:2558-2561. 3. Brott TG, Haley EC, Levy DE, et al: Urgent therapy for stroke: Part I. Pilot study of tissue plasminogen activator administered within 90 minutes. Stroke 1992;23:632-640. 4. Kothari RU, Hall K, Brott T, et al: Early stroke recognition: Developing an out-of-Hospital NIH Stroke Scale. Acad Emerg Med 1997;4:986-990. 5. Brott T, Adams HP Jr, Olinger CP, et al: Measurements of acute cerebral infarction: A clinical examination scale. Stroke 1989;20:864-870. 6. Lyden P, Brott T, Tilley B, et al: Improved reliability of the NIH Stroke Scale using video training. Stroke 1994;25:2220-2226. 7. Snedecor GW, Cochran WG: Statistical Methods, ed 7. Ames, IA: The Iowa State University Press, 1980:243-246. 8. Fleiss JL: Statistical Methods for Rates and Proportions, ed 2. New York: John Wiley & Sons, 1981:212-236. 9. Emergency Stroke Care Task Force: Acute stroke, in Cummins RO (ed): Advanced Cardiac Life Support. Dallas, TX: American Heart Association, 1997:10-1,10-28. 10. Schmulling S, Grond M, Rudolf J: Training as a prerequisite for reliable use of NIH Stroke Scale [letter]. Stroke 1998;29:1258-1259. 11. Goldstein LB, Bartels C, Davis JN: Interrater reliability of the NIH Stroke Scale. Arch Neurol 1989;46:660-662. 12. Lyden PD, Lau GT: A critical appraisal of stroke evaluation and rating scales. Stroke 1991;22:1345-1352. 13. Ferro JM, Pinto AN, Falcao I, et al: Diagnosis of stroke by the non-neurologist: A validation study. Stroke 1998;29:1106-1109. 14. Kidwell CS, Saver JL, Eckstein M, et al: High accuracy of emergency medical technician identification of stroke using the Los Angeles Prehospital Stroke Screen (LAPSS). Stroke 1998;29:313.
3 7 8
ANNALS OF EMERGENCY MEDICINE
33:4 APRIL 1999