- Email: [email protected]
Do we need a scoring system in the ICU?
D O WE N E E D A S C O R I N G S Y S T E M I N T H E I C U ?
.. V I N C E N T
It is certainly important to quantify as precisely as possible the degree of severity of a disease process and the prognosis of a critically ill patient. Potential applications are: -- Comparison of groups of patients included in clinical studies: to assess the severity of the disease process and the comparability of the treated and the control groups [1]; -- Improvement in ICU management: better assessment of the type of patients, global triage decisions, costs containment; -- Quality assurance: by comparing different patient groups in different hospitals; --Applications to individual decisions: assistance in the decisions of admission in the ICU or discharge from the ICU, decisions to forego life support measures, etc. Unfortunately, despite significant improvements, all scoring systems have their limitations and will keep some limitations regardless of future improvements. The problems become increasingly serious as we go from the first to the fourth potential application listed above.
•
T h e s e p r o b l e m s c a n be s u m m a r i z e d in 12 i t e m s
Several objectives are often confused There is a number of specific scoring systems available for a series of particular situations: examples are the Apgar score, _the Killip or the Norris score for myocardial infarction, the Glasgow coma scale developed in 1974 by Teasdale and Jennett, the Ranson
Departmentof Intensive Care, ErasmeHospital, Free University of Brussels, Belgium.
score for pancreatitis, and the AIDS score. Although each of these scoring systems has its limitations, it also has its own value. However, the prediction of outcome has a broader scope, which takes into account the type of acute disease, the severity of the disease and also the patient's physiological reserve [2].
There is no scoring system which is valid for all groups of patients For instance, the Apache score specifically eliminates children, burned patients, or patients after coronary artery bypass graft [3]. In addition, the Apache score has been found less reliable in patients after trauma [4], after cardiac arrest, with drug intoxications or with coronary care problems. In one study [5], an inverse relationship was even found between the Apache II score and the outcome in patients with cardiogenic pulmonary edema, and so on.
The admission diagnosis remains a predominant element in any scoring system All prognostic indexes are very sensitive to the primary diagnosis. A patient with epiglottitis may have a low degree of physiological abnormalities but a high mortality rate while a patient with decompensated diabetic ketoacidosis may have a high severity score but a low probability of death.
Some prognostic indicators are established only 24 hours after ICU admission (except MPMO) To extend the period of observation over some time after ICU admission can make sense, but it also has several problems: -- The scoring system can hardly be applied for triage decisions before ICU admission; R6an. Urg., 1994, 3 (2 bis), 169-171
-
170
-
Do we need a scoring system in the ICU?
- - The scoring system will depend on the number of measurements obtained during the first 24 hour period; -- Patients who die within the first 24 hours may have lower scores than patients who survive; -- The scoring system will be influenced by therapy during these first 24 hours.
All scoring systems remain subjective In particular, the choice of a primary diagnosis and the assessment of chronic health status are influenced by the physician's opinion.
All scoring systems primarily focus on the risk of death The reason is evident: mortality is well defined, is relevant for everyone and represents an objective endpoint. However, other parameters such as complications, length of ICU stay and especially quality of life are other important determinants. Although the Apache III score gives some estimate of some these parameters (i.e., length of ICU stay, resources required), the quality of life in particular cannot be taken into account by objective criteria.
All scoring systems are subject to selection bias and lead-time bias There can always be differences in selection criteria and variations in the evolution of the illness prior to ICU admission. In addition, all prognostic indicators must be regularly updated in view of changes in influential factors and especially progress in therapy.
All scoring systems are influenced by therapy This is obvious for the scoring systems established in the 24 hours following admission but the influence of treatment can already be present on admission. For instance, profound hypotension may have been corrected by high doses of vasopressor agents.
Day by day assessment of prognosis has not been properly validated Although some attempts have been made to repeat the evaluation of prognosis at regular time intervals [6], there is not enough evidence that the repeated use of a scoring system can be used reliably. The Apache III system proposes such an evaluation of the time course, but this has not been properly validated.
Scoring systems cannot be applied to decisions concerning individual patients None of the scoring systems is sensitive and specific enough to dictate such individual decisions. It may be argued that every decision process is based on previous experiences with similar groups of R#an. Urg., 1 9 9 4 , 3 ( 2 bis), 1 6 9 - 1 7 1
patients. Nevertheless, doctors do feel secure with some individual decisions, especially those to admit in the ICU or to forego life support. Hence, there is a risk of abusive applications of scoring systems in ethical decisions.
Scoring systems may not be very superior to the assessment by doctors and nurses A study by Kruse et al. [7] on an ICU population having a 40 p. 100 mortality rate showed that doctors and nurses may be better at discriminating the survivors from the non-survivors than an established prognostic indicator (Apache II). Interestingly, the interns, residents and fellows had a comparable assessment, indicating that the clinical experience did not significantly improve this judgment. Nurses could also discriminate patients accurately. Admittedly, more sophisticated scoring systems allow a better calibration of the prognosis.
Simplicity and accuracy of scoring systems do not go together Optimally, a scoring system should include easily measurable and widely available parameters that could be combined in a simple calculation resulting in sound applications. Most of the scoring systems are relatively simple but have relatively limited accuracy. The Apache III score is more sophisticated but is also less easily manageable: such a system has become protected and access to it quite expensive.
In conclusion, scoring systems may be very helpful to assess the severity of disease and even the prognosis of specific groups of patients. However, no system is, or will be, entirely accurate. Therefore, there is a risk of abusive applications of these systems especially by administrators who would use it as a tool to assess the performance of ICU. In particular, the Apache III system proposes to objectively compare the performance of one ICU with others having a similar type of activity and environment. Boyd and Grounds [8] have rightly stressed the problems associated with such "standardised mortality rates". Since the Apache III score takes into account the entire 24 hour period following ICU admission, the initial treatment may have a significant influence on this assessment. Specifically, a sub-optimal treatment will result in more profound abnormalities and thus in a higher score. Hence, the mortality rate may be higher but the severity score will also be higher than in another unit. There is also a risk of abusive applications of a scoring system in important ethical decisions. It is very important that a scoring system remains as objective as possible and that it constitutes an addition to, rather than a substitute for clinical judgement. Even the most sophisticated equipment will not replace good clinical judgment.
Do we need a scoring system in the ICU?
References [5] [1] KNAUSW.A., HARRELL F.E., FISHER C.J. et aL - - The clinical evaluation of new drugs for sepsis: A prospective study design based on survival analysis. JAMA, 1993, 270, 1233-1241. [2] SCHUSTER D.P. - - Predicting outcome after ICU admission: The art and science of assessing risk. Chest., 1992, 102, 1861-1870. [3] KNAUS W.A., WAGNER D.P., DRAPER E.A. et aL - - The Apache III prognosis system: Risk prediction of hospital mortality for critically ill hospitalized adults. Chest., 1991, 100, 1619-1636. [4] MCANENA O.J., MOORE F.A., MOORE E.E., MAl-rox K.L., MARX J.A., PEPE P. - - Invalidation of the Apache II scoring
[6]
[7]
[8]
-
171
-
system for patients with acute trauma. J. Trauma, 1992, 33, 504-507. FEDULLOA.J., SWINBURNEA.J., WAH G.W. Apache II score and mortality in respiratory failure due to cardiogenic pulmonary edema. Crit. Care Med., 1988, 16, 1218-1223. CHANG R.W.S., JACOBS S., LEE B. - - Predicting outcome among intensive care unit patients using computerized trend analysis of daily Apache II scores corrected for organ system failure. Intens. Care Med., 1988, 14, 558-566. KRUSE J.A., THILL-BAHAROZIAN M.C., CARLSON R.W. - - Comparison of clinical assessment with Apache II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA, 1988, 260, 1739-1742. BOYD O., GROUNDS R.M. - - Physiological scoring systems and audit. Lancet, 1993, 341, 1573-1574. m
mmm
R#an. Urg., 1994, 3 (2 bis), 169-171
.. V I N C E N T
It is certainly important to quantify as precisely as possible the degree of severity of a disease process and the prognosis of a critically ill patient. Potential applications are: -- Comparison of groups of patients included in clinical studies: to assess the severity of the disease process and the comparability of the treated and the control groups [1]; -- Improvement in ICU management: better assessment of the type of patients, global triage decisions, costs containment; -- Quality assurance: by comparing different patient groups in different hospitals; --Applications to individual decisions: assistance in the decisions of admission in the ICU or discharge from the ICU, decisions to forego life support measures, etc. Unfortunately, despite significant improvements, all scoring systems have their limitations and will keep some limitations regardless of future improvements. The problems become increasingly serious as we go from the first to the fourth potential application listed above.
•
T h e s e p r o b l e m s c a n be s u m m a r i z e d in 12 i t e m s
Several objectives are often confused There is a number of specific scoring systems available for a series of particular situations: examples are the Apgar score, _the Killip or the Norris score for myocardial infarction, the Glasgow coma scale developed in 1974 by Teasdale and Jennett, the Ranson
Departmentof Intensive Care, ErasmeHospital, Free University of Brussels, Belgium.
score for pancreatitis, and the AIDS score. Although each of these scoring systems has its limitations, it also has its own value. However, the prediction of outcome has a broader scope, which takes into account the type of acute disease, the severity of the disease and also the patient's physiological reserve [2].
There is no scoring system which is valid for all groups of patients For instance, the Apache score specifically eliminates children, burned patients, or patients after coronary artery bypass graft [3]. In addition, the Apache score has been found less reliable in patients after trauma [4], after cardiac arrest, with drug intoxications or with coronary care problems. In one study [5], an inverse relationship was even found between the Apache II score and the outcome in patients with cardiogenic pulmonary edema, and so on.
The admission diagnosis remains a predominant element in any scoring system All prognostic indexes are very sensitive to the primary diagnosis. A patient with epiglottitis may have a low degree of physiological abnormalities but a high mortality rate while a patient with decompensated diabetic ketoacidosis may have a high severity score but a low probability of death.
Some prognostic indicators are established only 24 hours after ICU admission (except MPMO) To extend the period of observation over some time after ICU admission can make sense, but it also has several problems: -- The scoring system can hardly be applied for triage decisions before ICU admission; R6an. Urg., 1994, 3 (2 bis), 169-171
-
170
-
Do we need a scoring system in the ICU?
- - The scoring system will depend on the number of measurements obtained during the first 24 hour period; -- Patients who die within the first 24 hours may have lower scores than patients who survive; -- The scoring system will be influenced by therapy during these first 24 hours.
All scoring systems remain subjective In particular, the choice of a primary diagnosis and the assessment of chronic health status are influenced by the physician's opinion.
All scoring systems primarily focus on the risk of death The reason is evident: mortality is well defined, is relevant for everyone and represents an objective endpoint. However, other parameters such as complications, length of ICU stay and especially quality of life are other important determinants. Although the Apache III score gives some estimate of some these parameters (i.e., length of ICU stay, resources required), the quality of life in particular cannot be taken into account by objective criteria.
All scoring systems are subject to selection bias and lead-time bias There can always be differences in selection criteria and variations in the evolution of the illness prior to ICU admission. In addition, all prognostic indicators must be regularly updated in view of changes in influential factors and especially progress in therapy.
All scoring systems are influenced by therapy This is obvious for the scoring systems established in the 24 hours following admission but the influence of treatment can already be present on admission. For instance, profound hypotension may have been corrected by high doses of vasopressor agents.
Day by day assessment of prognosis has not been properly validated Although some attempts have been made to repeat the evaluation of prognosis at regular time intervals [6], there is not enough evidence that the repeated use of a scoring system can be used reliably. The Apache III system proposes such an evaluation of the time course, but this has not been properly validated.
Scoring systems cannot be applied to decisions concerning individual patients None of the scoring systems is sensitive and specific enough to dictate such individual decisions. It may be argued that every decision process is based on previous experiences with similar groups of R#an. Urg., 1 9 9 4 , 3 ( 2 bis), 1 6 9 - 1 7 1
patients. Nevertheless, doctors do feel secure with some individual decisions, especially those to admit in the ICU or to forego life support. Hence, there is a risk of abusive applications of scoring systems in ethical decisions.
Scoring systems may not be very superior to the assessment by doctors and nurses A study by Kruse et al. [7] on an ICU population having a 40 p. 100 mortality rate showed that doctors and nurses may be better at discriminating the survivors from the non-survivors than an established prognostic indicator (Apache II). Interestingly, the interns, residents and fellows had a comparable assessment, indicating that the clinical experience did not significantly improve this judgment. Nurses could also discriminate patients accurately. Admittedly, more sophisticated scoring systems allow a better calibration of the prognosis.
Simplicity and accuracy of scoring systems do not go together Optimally, a scoring system should include easily measurable and widely available parameters that could be combined in a simple calculation resulting in sound applications. Most of the scoring systems are relatively simple but have relatively limited accuracy. The Apache III score is more sophisticated but is also less easily manageable: such a system has become protected and access to it quite expensive.
In conclusion, scoring systems may be very helpful to assess the severity of disease and even the prognosis of specific groups of patients. However, no system is, or will be, entirely accurate. Therefore, there is a risk of abusive applications of these systems especially by administrators who would use it as a tool to assess the performance of ICU. In particular, the Apache III system proposes to objectively compare the performance of one ICU with others having a similar type of activity and environment. Boyd and Grounds [8] have rightly stressed the problems associated with such "standardised mortality rates". Since the Apache III score takes into account the entire 24 hour period following ICU admission, the initial treatment may have a significant influence on this assessment. Specifically, a sub-optimal treatment will result in more profound abnormalities and thus in a higher score. Hence, the mortality rate may be higher but the severity score will also be higher than in another unit. There is also a risk of abusive applications of a scoring system in important ethical decisions. It is very important that a scoring system remains as objective as possible and that it constitutes an addition to, rather than a substitute for clinical judgement. Even the most sophisticated equipment will not replace good clinical judgment.
Do we need a scoring system in the ICU?
References [5] [1] KNAUSW.A., HARRELL F.E., FISHER C.J. et aL - - The clinical evaluation of new drugs for sepsis: A prospective study design based on survival analysis. JAMA, 1993, 270, 1233-1241. [2] SCHUSTER D.P. - - Predicting outcome after ICU admission: The art and science of assessing risk. Chest., 1992, 102, 1861-1870. [3] KNAUS W.A., WAGNER D.P., DRAPER E.A. et aL - - The Apache III prognosis system: Risk prediction of hospital mortality for critically ill hospitalized adults. Chest., 1991, 100, 1619-1636. [4] MCANENA O.J., MOORE F.A., MOORE E.E., MAl-rox K.L., MARX J.A., PEPE P. - - Invalidation of the Apache II scoring
[6]
[7]
[8]
-
171
-
system for patients with acute trauma. J. Trauma, 1992, 33, 504-507. FEDULLOA.J., SWINBURNEA.J., WAH G.W. Apache II score and mortality in respiratory failure due to cardiogenic pulmonary edema. Crit. Care Med., 1988, 16, 1218-1223. CHANG R.W.S., JACOBS S., LEE B. - - Predicting outcome among intensive care unit patients using computerized trend analysis of daily Apache II scores corrected for organ system failure. Intens. Care Med., 1988, 14, 558-566. KRUSE J.A., THILL-BAHAROZIAN M.C., CARLSON R.W. - - Comparison of clinical assessment with Apache II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA, 1988, 260, 1739-1742. BOYD O., GROUNDS R.M. - - Physiological scoring systems and audit. Lancet, 1993, 341, 1573-1574. m
mmm
R#an. Urg., 1994, 3 (2 bis), 169-171