- Email: [email protected]
Screening early renal failure: Cut-off values for serum creatinine as an indicator of renal impairment
Kidney International, Vol. 55 (1999), pp. 1878–1884
CLINICAL NEPHROLOGY – EPIDEMIOLOGY – CLINICAL TRIALS
Screening early renal failure: Cut-off values for serum creatinine as an indicator of renal impairment CE´CILE COUCHOUD, NICOLE POZET, MICHEL LABEEUW, and CLAIRE POUTEIL-NOBLE Nephrology-Dialysis-Transplantation Unit, Lyon-Sud Hospital, Pierre-Benite, and Renal Physiology Laboratory, Edouard Herriot Hospital, Lyon, France
Screening early renal failure: Cut-off values for serum creatinine as indicator of renal impairment. Background. The aim of this study was to define cut-off values for serum creatinine as an indicator of several levels of renal impairment. Methods. To identify the suitable values, receiver operating characteristic curves were constructed based on the data of 984 laboratory assessments of renal function. The glomerular filtration rate was measured with inulin clearance. Three levels of renal impairment were analyzed. An index that gave the same weight to false positive and false negative results was used to determine the thresholds. Robustness of the results was tested using a “bootstrap” technique. Results. Considering an inulin clearance of less than 80 ml/ min/1.73 m2, the cut-off value for serum creatinine was 115 mmol/liter for men and 90 mmol/liter for women. The cut-off value for a clearance of less than 60 ml/min/1.73 m2 was 137 mmol/liter for men and 104 mmol/liter for women. For a clearance of less than 30 ml/min/1.73 m2, the cut-off value was 177 mmol/liter for men and 146 mmol/liter for women. Conclusion. This method is useful to determine a cut-off value for serum creatinine in epidemiological studies concerning early chronic renal failure screening. The value of the glomerular filtration rate of reference and the weight of false positive and false negative results have to be adapted to the aim of the individual study design.
Because of the lack of pertinent and early clinical criteria, the diagnosis of renal failure is made on biological criteria. Since standard methods for clinical measurement of glomerular filtration such as inulin or iothalamate clearances can be performed only in specialized units, the most widely used marker for monitoring glomerular filtration rate remains the quantitative measure of serum
Key words: chronic renal failure, serum creatinine, receiver operating characteristic curves, glomerular filtration rate, diagnosis. Received for publication June 3, 1998 and in revised form October 22, 1998 Accepted for publication November 30, 1998
1999 by the International Society of Nephrology
creatinine. Given the effort required to obtain accurately timed urine specimens, the glomerular filtration rate is often estimated from serum creatinine concentration alone or by standard formulas estimating creatinine clearance. The diagnosis of renal failure is usually suspected when serum creatinine is greater than the upper limit of the “normal” interval. These limits are defined as the 97.5th percentile established by the distribution of the value of serum creatinine in a large sample of population considered healthy. One study, based on 18,000 serum creatinine measurements (Jaffe´ reaction), provided a normal value in men of 63 to 112 mmol/liter and in women the value of 53 to 102 mmol/liter [1]. To our knowledge there is no study estimating the diagnostic value of these upper limits. Moreover, these extreme values are not synonymous with renal failure as defined by the reduction of glomerular filtration. In the context of renal failure screening, the problem was to transform a quantitative continuous variable (serum creatinine) into a qualitative binary variable (pathological value yes/no) and then to define a cut-off value for serum creatinine with the best diagnostic value of renal failure. To identify the suitable value, receiver operating characteristic curves (ROC curves) were constructed [2–4]. The aim was to use these cut-off values in screening surveys. The choice of a good cut-off value is important because it should not be too high or too low. A low cutoff value gives many false-positive results and leads to an excessive diagnosis of renal failure, especially as the prognosis value of an early renal failure is not clearly known. On the other hand, a high cut-off value will give many false-negative results and leads to an underestimated prevalence of early renal failure, which in turn will decrease the benefits of an early intervention in those patients whose diagnosis is missed [5, 6]. The advantage of constructing a ROC curve is to adjust the cut-off value for serum creatinine to a specific diagnostic purpose.
1878
Couchoud et al: Screening early renal failure
METHODS Population Calculations were made with the data of 984 laboratory assessments of renal function in a patient population stemming from various medical units. It included patients suspected of having a nephropathy, patients with a pathology that could potentially have renal consequences (such as hypertension, urinary lithiasis, and cirrhosis waiting for a liver transplant), and also healthy patients (healthy volunteers for therapeutic studies, living-related kidney donors). Four hundred and sixty-four women and 520 men were included in this analysis, with a mean age of 52.8 years. Serum creatinine was determined with the Jaffe´ reaction in continuous flow after dialysis (automated method, Autoanalyzer Technicone, AAI-11). Choice of the reference values for the glomerular filtration rate Glomerular filtration was measured with inulin clearance (Inuteste; 25% by Fresenius, standard colorimetric method on an Autoanalyzer Technicone, AAI). After overnight fasting, the protocol included three periods of 20 minutes duration during which blood and urine were collected. After a loading dose, inulin was continuously infused during all three periods. All of the results were expressed according to the body surface area in ml/min/ 1.73 m2. According to the aim of the screening, a value of glomerular filtration had to be chosen, and three references values were analyzed. The first value was 80 ml/ min/1.73 m2, which corresponds to the usual value defining the onset of renal failure. The second value was 60 ml/ min/1.73 m2, which corresponds to an established renal failure whatever the sex or age. The third value chosen was 30 ml/min/1.73 m2, which defines a clinically pertinent renal failure. Construction of the receiver operating characteristic curves To transform the quantitative variable (serum creatinine) into a qualitative variable, a cutting line was applied to the scale of its variations. This line defined the pathological or normal character of the result on both sides. The choice of the cutting line determined the discriminating power of the separator. Among the possible values of the discriminant variable (serum creatinine), each value chosen as a cut-off point defined a sensitivity and a specificity. On the ROC curve, the rate of false positives (1 – specificity) was represented on the abscissa, and the rate of the true positives (sensitivity) on the ordinate [2–4]. This curve was used to visualize the effects of changing the cut-off points. Choice of the cutoff value: The Younden index There was no “natural” cut-off value because there was some overlap between the distribution of creatinine
1879
values for affected and unaffected individuals for each reference value of glomerular filtration rate. As the cutoff point changed, the sensitivity and specificity of the test changed in opposite directions. The upper left-hand corner of the ROC graph denoted a perfect diagnostic test: a true positive rate of 100% and a false positive rate of 0%. Thus, the point where the ROC curve was close to the left and top boundaries of the ROC graph corresponded to the value in which the sum of the specificity and sensitivity was the highest. The Younden index, defined as (sensitivity 1 specificity) 2 1, gave the same weight to false positives as it did to the false negatives. To test the robustness of the results, confidence intervals were obtained by using the “bootstrap” technique [7]. For each threshold, after randomly sampling one third of the study group, drawn with a replacement 100 times, a mean and a confidence interval of the 100 bootstrapped cut-off estimates were calculated. For each chosen cut-off value, the sensitivity, specificity, and positive and negative likelihood ratios of the test (serum creatinine) were calculated, as well as the corresponding standard error. The sensitivity of a test corresponded to the rate of true positives and the specificity to the rate of the true negatives. The positive likelihood ratio was the ratio of the rate of the true positives and the rate of the false positives. The negative likelihood ratio was the ratio of the rate of the false negatives and the rate of the true negatives. A positive likelihood ratio equal to 10 means that a positive response of the test (serum creatinine higher than the cut-off value) is 10-fold more frequent among patients with renal failure than among the patients without renal failure. A negative likelihood ratio equal to 0.10 means that a negative response of the test (serum creatinine lower than the cutoff value) is 10-fold less frequent among patients with renal failure than among the patients without renal failure. The predictive values of this test could not be calculated because they depend on the prevalence of the disease. In this population, measuring the prevalence makes no sense, and the prevalence of early renal failure in the general population is not yet known. Thus, projections of predictive values for several plausible values of prevalence were calculated. Calculation of the serum creatinine cut-off value With the Younden index, a cut-off value for serum creatinine for each sex and for three levels of renal function (inulin clearance , 80 ml/min/1.73 m2, , 60 ml/min/ 1.73 m2, or , 30 ml/min/1.73 m2) was determined. Calculation of the cut-off value for the creatinine clearance estimated with Cockroft-Gault’s formula The Cockroft-Gault’s formula predicts the creatinine clearance, taking into account age and sex [8]. It is the
1880
Couchoud et al: Screening early renal failure
Table 1. Characteristics of the patients included in the study Men N 520 Age years 49.7 6 16.7 (range) (15–87) Weight kg 72.8 6 13.2 mean 6 sd (range) (36–125) Height cm 170.8 6 7.4 mean 6 sd, (range) (145–190) 1.84 6 0.17 Body surface m2 mean 6 sd, (range) (1.22–2.41) Serum creatinine lmol /liter 168.1 6 145.5 mean 6 sd, (range) (53–1309) Inulin clearance ml/min/1.73 m2 77.1 6 41.7 mean 6 sd, (range) (4–172) Indications of renal function assessment N (%) Glomerular disease 79 (15) Diabetic nephropathy 62 (12) Tubulointerstitial disease 52 (10) Arteriolar nephrosclerosis 36 (7) Polycystic kidney disease 13 (3) Multisystemic disease 18 (3) Systemic lupus erythematosus 3 (1) Toxemia gravidis 0 Urinary abnormalities 28 (5) Hypertension 19 (4) Chronic renal insufficiency 22 (4) Single kidney 13 (3) “Normal control” 16 (3) Living donor before kidney donation 28 (5) Before liver transplantation 96 (18) Other 35 (7)
Women 464 45.3 6 16.5 (17–95) 62.0 6 12.5 30–130) 159.2 6 7.2 (136–180) 1.63 6 0.16 (1.22–2.26) 114.8 6 93.8 (44–841) 85.5 6 38.4 (2–233) 54 32 42 34 18 46 58 21 37 15 7 8 11 28 42 11
(12) (7) (9) (7) (4) (10) (13) (5) (8) (3) (2) (2) (2) (6) (9) (2)
Fig. 1. Receiver operating characteristic (ROC) curve for serum creatinine in men corresponding to an inulin clearance at 60 ml/min/1.73 m2. The small square indicates the point where the Younden index is maximum (serum creatinine 5 137 mmol/liter).
This AUC calculation is equal to the probability that the test (serum creatinine) is higher in “pathological” patients than in “normal” patients. Thus, an AUC varies from 0.5 (noninformative test) to 1 (most discriminating test). All of the calculations and the graphs were made with the Excele software. RESULTS
most widely employed and the best validated formula for use in adults. The formula is as follows: Creatinine clearance 5 K 3 (140 2 age) 3 weight/serum creatinine where age is in years, weight in kg, and serum creatinine is in mmol/liter. The K equals 1.23 in men and 1.04 in women. With the Younden index, a cut-off value for estimated creatinine clearance for three levels of renal function (inulin clearance ,80 ml/min/1.73 m2, ,60 ml/min/1.73 m2, or ,30 ml/min/1.73 m2) was determined. Glomerular filtration was measured the same way for inulin clearance as for serum creatinine alone. Calculation of the area under the curve To reduce an entire ROC curve to a single quantitative index of diagnostic accuracy, the area under the curve (AUC) was calculated with the method described by Hanley and McNeil [9]. AUC 5 [S 2 p 3 (p 1 1)/2]/(p 3 np) where S equals the sum of the rank of pathological patients; p equals the number of pathological patients, and np 5 number of non-pathological patients.
The characteristics of the 984 patients are presented in Table 1. As an example, one ROC curve for men is presented in the Figure 1. Renal failure was defined by an inulin clearance of less than 60 ml/min/1.73 m2. The graph indicates the point where the Younden index is maximum. This point does not correspond to the intersection with the bisectrix because of sample fluctuations. Tables 2, 3, and 4 summarize the results. The serum creatinine value for a cut-off value at 100% sensitivity, a cut-off value at 95%, 98%, and 100% specificity, and a cut-off value corresponding to maximum Younden index [(sensitivity 1 specificity) 2 1] are given for each glomerular filtration. The estimations of the parameters of accuracy (sensitivity, specificity, and positive and negative likelihood ratios) at the chosen cut-off value and the AUCs are also given in these tables. The projections of a positive predictive value for several plausible values of prevalence are presented in Figure 2, considering the specificity and sensitivity calculated in men corresponding to an inulin clearance of less than 60 ml/min/1.73 m2 (sensitivity 90.0% and specificity 93.3%). Figure 3 represents the trade-off between sensitivity and specificity according to serum creatinine value and the corresponding Younden index. The cut-off values for serum creatinine in various sub-
1881
Couchoud et al: Screening early renal failure Table 2. Cut-off values of serum creatinine in men, for each glomerular filtration rate and at different trade-offs between sensitivity and specificity Inulin clearance ml /min /1.73 m2
Men N 5 520 Number of pathological values Number of non-pathological values Creatininemia at 100% sensitivity lmol /liter Creatininemia at 95% specificity lmol /liter Creatininemia at 98% specificity lmol /liter Creatininemia at 100% specificity lmol /liter Creatininemia at threshold lmol /liter Sensitivity % 6 standard error Specificity % 6 standard error Positive likelihood 6 standard error Negative likelihood 6 standard error Area under the ROC curve
, 80
, 60
, 30
259 261 66 124 133 150 115 86.87 6 1.48 90.80 6 1.27 9.44 6 2.11 0.14 6 0.16 0.941
190 330 80 142 150 208 137 90.0 6 1.32 93.33 6 1.09 13.49 6 2.19 0.11 6 0.22 0.972
97 423 150 208 239 292 177 98.97 6 0.44 90.78 6 1.27 10.73 6 1.04 0.01 6 1.0 0.991
At the cut-off value finally retained, which maximizes the Younden index, are given the estimations of the accuracy markers of the test.
Table 3. Cut-off values of serum creatinine in women, for each glomerular filtration rate and at different trade-offs between sensitivity and specificity Inulin clearance ml /min /1.73 m2
Women N 5 464 Number of pathological values Number of non-pathological values Creatininemia at 100% sensitivity lmol /liter Creatininemia at 95% specificity lmol /liter Creatininemia at 98% specificity lmol /liter Creatininemia at 100% specificity lmol /liter Creatininemia at threshold lmol /liter Estimation of sensitivity % 6 standard error Estimation of specificity % 6 standard error Positive likelihood 6 standard error Negative likelihood 6 standard error Area under the ROC curve
, 80
, 60
, 30
188 276 60 106 113 153 90 85.64 6 1.63 85.51 6 1.63 5.91 6 2.56 0.17 6 0.18 0.914
123 341 75 113 131 153 104 87.80 6 1.52 90.32 6 1.37 9.07 6 2.96 0.14 6 0.24 0.957
50 414 96 148 168 226 146 98.00 6 0.65 94.93 6 1.02 19.33 6 1.99 0.02 6 0.99 0.988
At the cut-off value finally retained, which maximizes the Younden index, are given the estimations of the accuracy markers of the test.
groups are given in Table 5. Bootstrapped cut-off value estimates and their standard deviations are given in Table 6. DISCUSSION Cut-off values of serum creatinine obtained with bootstrapped ROC curves at 80 ml/min/1.73 m2 compared with the results corresponding to the 97.5th percentile of a healthy population [1] are lower in women [92 mmol/ liter (95% CI, 85 to 98) instead of 102 mmol/liter] but not statistically different in men [118 mmol/liter (95% CI, 107 to 131) instead of 112 mmol/liter]. A recent study has confirmed the predictive value of high serum creatinine for the risk of end-stage renal failure [10]. The adjusted relative risk (adjusted on age, proteinuria, hematuria, and hypertension) increased significantly beyond a cutoff value for serum creatinine at 1.2 mg/dl (105 mmol/liter) for women and 1.4 mg/dl (125 mmol/liter) for men. These cut-off values are not different from results calculated at a glomerular filtration of
60 ml/min/1.73 m2: 104 mmol/liter (95% CI, 91 to 117) for women and 139 mmol/liter (120 to 158) for men. The diagnostic function of a biological test is to evaluate, according to its result, the probability for a given disease or the probability of its absence. This function of the test can be expressed by the intrinsic qualities of the test, which are sensitivity and specificity. In fact, the accuracy of a test depends on the frequency of the disease in the studied population [11]. Our analysis was based on a population of patients who underwent a renal function assessment. Although some of them could be considered to be healthy (healthy volunteers for therapeutics studies and living-related kidney donors), the majority of patients were suspected of having a nephropathy. Forty-five percent of the patients had a renal failure defined by an inulin clearance lower than 80 ml/min/1.73 m2. This population was selected in order to reach the needed sample size and not to be representative of the general population. The cost and the complexity of a laboratory evaluation of inulin clearance prevented us from proposing it
1882
Couchoud et al: Screening early renal failure
Table 4. Cut-off values of estimated creatinine clearance (Cockroft-Gault), for each glomerular filtration rate and at different trade-offs between sensitivity and specificity Inulin clearance ml /min /1.73 m2
Cockroft-Gault N 5 900 Number of pathological values Number of non-pathological values Creatine clearance at 100% sensitivity ml /min Creatine clearance at 95% specificity ml /min Creatine clearance at 98% specificity ml /min Creatine clearance at 100% specificity ml /min Creatine clearance at threshold ml /min Estimation of sensitivity % 6 standard error Estimation of specificity % 6 standard error Positive likelihood 6 standard error Negative likelihood 6 standard error Area under the ROC curve
, 80
, 60
, 30
419 481 130 98 95 42 56 74.94 6 1.44 95.43 6 0.70 16.40 6 2.13 0.26 6 0.09 0.936
297 603 82 47 41 26 55 87.88 6 1.09 88.56 6 1.06 7.68 6 4.62 0.14 6 0.38 0.958
140 760 55 33 27 14 37 95.00 6 0.73 92.24 6 0.89 12.24 6 3.08 0.05 6 0.62 0.983
At the cut-off value finally retained, which maximizes the Younden index, are given the estimations of the accuracy markers of the test.
Fig. 2. Projections of predictive values for several plausible values of prevalence, with sensitivity at 90% (men at 60 ml/min/1.73 m2). Symbols are: (d) positive predictive value; (h) negative predictive value.
as a screening test in a large-sized healthy population. That is why we accept dealing with this biased population in which the prevalence of renal failure is higher than in the general population. In our population, the calculation of the predictive values of this test made no sense. In the general population, the prevalence of early renal failure is unknown, but certainly is very low. Figure 2 shows that for a low prevalence, the serum creatinine factor does not have a good positive predictive value. On the contrary, the negative predictive value is always at 100%. This means that by using this test, no patients with renal impairment will be “lost,” but that many healthy patients will be “caught” by the screening study. The advantage of constructing a ROC curve is to adjust the sensitivity or the specificity. Because we were looking for a compromise between high sensitivity and high specificity, the Younden index was chosen because it gives the same weight to the two types of error (false positive and false negative results). A false positive result
can have a psychological impact on the patient and leads to complementary tests that will at least include the search for proteinuria, urinary cytological abnormality and hypertension; they entail a more in-depth renal function assessment and, in the worst case scenario, a renal biopsy. A false negative result leads to the lack of a diagnosis of a disease that could have been curtailed or even treated before the evolution to end-stage renal disease. However, maximizing the Younden index is not necessarily an optimal strategy for screening studies. In the low-prevalence situation, improving the specificity will decrease the number of false positives without greatly increasing the false negatives. One might choose an arbitrary specificity of 98%, to the detriment of sensitivity. That is why for each study, the trade-off between sensitivity and specificity has to be determined by a group of experts according to the aim(s) of the study and the use of these serum creatinine cut-off values. The second advantage of constructing a ROC curve is to adjust the cut-off value for serum creatinine for a specific diagnostic purpose. These cut-off values are calculated according to inulin clearance. Thus, according to the study aims, each serum creatinine cut-off value can be chosen according to a specific glomerular filtration rate. Only the results corresponding to an inulin clearance of 80, 60, and 30 ml/min/1.73 m2 have been presented here, but this method can be generalized to all desired clearances. The few studies concerning the epidemiology of renal failure have taken their cut-off values for serum creatinine in an apparently arbitrary manner, often a whole value. For example, a French study used a cut-off value of 200 mmol/liter [12], a British study used a value of 300 mmol/liter [13], and an American study used 177 mmol/liter as the cut-off value, which corresponds to 2 mg/dl [14]. These studies did not distinguish the gender of the patients and chose high cut-off values that correspond to advanced renal failure.
1883
Couchoud et al: Screening early renal failure
Fig. 3. Trade-off between sensitivity and specificity for each level of serum creatinine in men at 60 ml/min/1.73 m2. The vertical line indicates the point where the Younden index is maximum (serum creatinine 5 137 mmol/liter). The lightly dotted line is the Younden index, the medium line is the sensitivity, and the heavy line is the specificity.
Table 5. Cut-off values of serum creatinine in various subgroups of patients, for the three levels of renal impairment
Subgroups Systemic lupus Diabetes #40 years old $60 years old
Inulin clearance ml /min 80 60 30 80 60 30 80 60 30 80 60 30
Serum creatine lmol /liter Men
Women
— — — 124 128 261 115 144 196 119 137 186
89 111 190 113 113 150 92 124 164 82 106 173
In older patients, one might choose lower ranges for the glomerular filtration target to take into consideration the “physiological” aging process. That is why a glomerular filtration rate at 60 ml/min/1.73 m2, which corresponds to an established renal failure whatever the sex or age, was chosen. Cut-off values according to age have been calculated for smaller populations, with a lower precision than for the entire group. These results have shown some small differences between patients older than 60 years and patients less than 40 years old. These differences were not always caused by a lower cut-off value for older patients, which is expected if one considers that older patients have a lower muscle mass. The difference in serum creatinine between men and women was even higher in older patients. Results for very old patients
Table 6. Robustness of the optimal cut-off values tested by bootstrap methods (100 subsamples of 1/3 of the population) Inulin clearance ml /min/1.73 m2
Serum creatinine lmol /liter
80 60 30
118.6 6 6.15 92.1 6 3.16 139.0 6 9.5 104.1 6 6.83 193.7 6 20.62 156.4 6 13.4
Men
Women
Creatinine clearance ml /min estimated by COCKROFT formula 61.9 6 6.02 53.2 6 5.36 36.9 6 3.31
Data are mean 6 sd.
could not be provided because of the low number of patients concerned in this population. Note that the cutoff values at 60 ml/min/1.73 m2 for patients older than 60 years are higher than cut-off values at 80 ml/min/1.73 m2 for patients less than 40 years old. The area under the curve (AUC) increased as the cutoff value for serum creatinine increased. This means that the diagnostic accuracy of serum creatinine was better when the renal failure was worsening. As the relationship between creatinine concentration and glomerular filtration rate is an hyperbolic function, serum creatinine is often considered to be an inadequate estimate of glomerular filtration, especially for slight reductions in renal function. All AUCs were higher than 0.9, which means that at a population level, plasma creatinine is not that poor of an index of the glomerular filtration rate [15]. Individually, the interpretation of the serum creatinine value is more difficult. For example, one does not know exactly if there is a modification in the relationship between creatinine clearance and inulin clearance according to the underlying disease. A diabetic status might modify
1884
Couchoud et al: Screening early renal failure
this relationship. As well, patients with glomerular disease or systemic lupus who have potentially higher rates of tubular creatinine secretion might have lower cut-off values for serum creatinine. This was not observed in this study, but the population was too small to explore this question precisely. Cut-off values presented for these subgroups are given for information only. Also, the different treatments or the protein intaken, which can also modify the relationship, were not taken into account. The effect of heavy proteinuria on the cut-off values for serum creatinine was not analyzed because of the lack of information on 24-hour proteinuria. The relative heterogeneity of the study population should be considered as a benefit for a later generalization. If a very precisely defined population had been selected, the results possibly would have been more “exact,” however, they would be useless for screening the general population in which the treatment, diet, and comorbidity are not controlled. The Cockroft-Gault’s formula is based on the serum creatinine concentration and has been developed to predict creatinine clearance. The quality criteria of this “test” gave good results. However, there was no clinical difference between the cut-off value at 80 ml/min/1.73 m2 inulin clearance (calculated clearance 56 ml/min) and the cut-off value at 60 ml/min/1.73 m2 inulin clearance (calculated clearance 55 ml/min). Thus, the practical use of the highest cut-off value seemed difficult. The results given by the bootstrap method showed a greater difference between 80 and 60 ml/min/1.73 m2 (62 and 53 ml/ min, respectively), but confirm that this formula underestimates glomerular filtration in the high values. Is this a specific problem related to the population of the study (French specificity, wide range of renal function, etc.)? The impression was that this formula was not well adapted to estimate subnormal glomerular function rate. Under 60 ml/min/1.73 m2 of inulin clearance, the results were good but not better than serum creatinine alone. However, it has the advantage of taking into account the sex of the individual, and thus, there is only one cutoff value to memorize at a given glomerular filtration, but it depends on four different variables (serum creatinine, sex, age, weight) and on the hypothesis of stable creatinine urinary excretion. Another problem is that the “gold standard” definition of renal failure is a biological test (inulin clearance) that also has its limits. Other methods could have been used with the same difficulties. Because of the technical difficulties and high costs, these tests simply estimate the glomerular filtration rate. To thus limit the diagnosis of renal failure to a decrease in the glomerular filtration rate is debatable. In conclusion, ROC curve methods are useful to determine a cut-off value for serum creatinine in epidemiological studies concerning early chronic renal failure screening. The value of the glomerular filtration rate of reference
and the weight of false positive and false negative results have to be adapted to the individual study goals. A renal failure screening in our general population is planned. Our descriptive study should permit us to measure the prevalence of early renal failure and to identify high-risk subgroups. Thus, we will be able to better understand the natural history of nephropathies, plan intervention studies, and organize health care networks. ACKNOWLEDGMENTS The help of Monica Lombardo, Laurent Heyer, Rene´ Ecochard, and all of the patients and nephrologists who are involved in the database is acknowledged. Reprint requests to Docteur Ce´cile Couchoud-Heyer, Service de Ne´phrologie-Hemodialyse-Transplantation Renale, Centre Hospitalier De´partemental Fe´lix Guyon, 97405 Saint-Denis Ile de la Reunion Cedex, France. E-mail: [email protected]
REFERENCES 1. Lauture de H, Caces E, Dubost P: Concentrations of cholesterol, uric acid, urea, glucose and creatinine in a population of 50,000 active individuals, in Reference Values in Human Chemistry, edited by Siest G, Basel, Karger, 1973, pp 141–152 2. Green DM, Swets JA: Signal Detection Theory and Psychophysics. New York, Wiley, 1966 (reprinted Los Altos Hills, CA, Peninsula Publishing, 1988) 3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P: Clinical Epidemiology: A Basic Science for Clinical Medicine (2nd ed). Boston, Little, Brown and Company, 1991, pp 69–152 4. Griner PF, Mayewski RJ, Mushlin AI, Greenland P: Selection and interpretation of diagnostic tests and procedures: Principles and applications. Ann Intern Med 94:553–600, 1981 5. Striker GE: Report on a workshop to develop management recommendations for the prevention of progression in chronic renal disease, Bethesda (MD) April 1994. Nephrol Dial Transplant 10: 290–292, 1995 6. Klahr S: Chronic renal failure: Management. Lancet 338:423–427, 1991 7. Efron B: Bootstrap methods: Another look at the jacknife. Ann Stat 7:1–26, 1979 8. Cockroft DW, Gault MH: Prediction of creatinine clearance from serum creatinine. Nephron 16:31–41, 1976 9. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic curve. Radiology 143:29–36, 1982 10. Iseki K, Ikemiya Y, Fukiyama K: Risk factors of end-stage renal disease and serum creatinine in a community-based mass screening. Kidney Int 51:850–854, 1997 11. Ransohoff DF, Feinstein AR: Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 299:926– 930, 1978 12. Jungers P, Chauveau P, Descamps-Latscha B, Labrunie M, Giraud E, Man NK, Gru¨nfeld JP, Jacobs C: Age and genderrelated incidence of chronic renal failure in a French urban area: A prospective epidemiologic study. Nephrol Dial Transplant 11: 1542–1546, 1996 13. Khan IH, Catto GRD, Edward N, MacLeod AM: Chronic renal failure: Factors influencing nephrology referral. Q J Med 87:559– 564, 1994 14. Strauss MJ, Port FK, Somen C, Wolfe RA: An estimate of the size of the US predialysis population with renal insufficiency and anemia. Am J Kidney Dis 21:264–269, 1993 15. Perneger ThV, Brancati FL, Whelton PK, Klag MJ: Studying the causes of kidney disease in humans: A review of methodologic obstacles and possible solutions. Am J Kidney Dis 25:722–731, 1995
CLINICAL NEPHROLOGY – EPIDEMIOLOGY – CLINICAL TRIALS
Screening early renal failure: Cut-off values for serum creatinine as an indicator of renal impairment CE´CILE COUCHOUD, NICOLE POZET, MICHEL LABEEUW, and CLAIRE POUTEIL-NOBLE Nephrology-Dialysis-Transplantation Unit, Lyon-Sud Hospital, Pierre-Benite, and Renal Physiology Laboratory, Edouard Herriot Hospital, Lyon, France
Screening early renal failure: Cut-off values for serum creatinine as indicator of renal impairment. Background. The aim of this study was to define cut-off values for serum creatinine as an indicator of several levels of renal impairment. Methods. To identify the suitable values, receiver operating characteristic curves were constructed based on the data of 984 laboratory assessments of renal function. The glomerular filtration rate was measured with inulin clearance. Three levels of renal impairment were analyzed. An index that gave the same weight to false positive and false negative results was used to determine the thresholds. Robustness of the results was tested using a “bootstrap” technique. Results. Considering an inulin clearance of less than 80 ml/ min/1.73 m2, the cut-off value for serum creatinine was 115 mmol/liter for men and 90 mmol/liter for women. The cut-off value for a clearance of less than 60 ml/min/1.73 m2 was 137 mmol/liter for men and 104 mmol/liter for women. For a clearance of less than 30 ml/min/1.73 m2, the cut-off value was 177 mmol/liter for men and 146 mmol/liter for women. Conclusion. This method is useful to determine a cut-off value for serum creatinine in epidemiological studies concerning early chronic renal failure screening. The value of the glomerular filtration rate of reference and the weight of false positive and false negative results have to be adapted to the aim of the individual study design.
Because of the lack of pertinent and early clinical criteria, the diagnosis of renal failure is made on biological criteria. Since standard methods for clinical measurement of glomerular filtration such as inulin or iothalamate clearances can be performed only in specialized units, the most widely used marker for monitoring glomerular filtration rate remains the quantitative measure of serum
Key words: chronic renal failure, serum creatinine, receiver operating characteristic curves, glomerular filtration rate, diagnosis. Received for publication June 3, 1998 and in revised form October 22, 1998 Accepted for publication November 30, 1998
1999 by the International Society of Nephrology
creatinine. Given the effort required to obtain accurately timed urine specimens, the glomerular filtration rate is often estimated from serum creatinine concentration alone or by standard formulas estimating creatinine clearance. The diagnosis of renal failure is usually suspected when serum creatinine is greater than the upper limit of the “normal” interval. These limits are defined as the 97.5th percentile established by the distribution of the value of serum creatinine in a large sample of population considered healthy. One study, based on 18,000 serum creatinine measurements (Jaffe´ reaction), provided a normal value in men of 63 to 112 mmol/liter and in women the value of 53 to 102 mmol/liter [1]. To our knowledge there is no study estimating the diagnostic value of these upper limits. Moreover, these extreme values are not synonymous with renal failure as defined by the reduction of glomerular filtration. In the context of renal failure screening, the problem was to transform a quantitative continuous variable (serum creatinine) into a qualitative binary variable (pathological value yes/no) and then to define a cut-off value for serum creatinine with the best diagnostic value of renal failure. To identify the suitable value, receiver operating characteristic curves (ROC curves) were constructed [2–4]. The aim was to use these cut-off values in screening surveys. The choice of a good cut-off value is important because it should not be too high or too low. A low cutoff value gives many false-positive results and leads to an excessive diagnosis of renal failure, especially as the prognosis value of an early renal failure is not clearly known. On the other hand, a high cut-off value will give many false-negative results and leads to an underestimated prevalence of early renal failure, which in turn will decrease the benefits of an early intervention in those patients whose diagnosis is missed [5, 6]. The advantage of constructing a ROC curve is to adjust the cut-off value for serum creatinine to a specific diagnostic purpose.
1878
Couchoud et al: Screening early renal failure
METHODS Population Calculations were made with the data of 984 laboratory assessments of renal function in a patient population stemming from various medical units. It included patients suspected of having a nephropathy, patients with a pathology that could potentially have renal consequences (such as hypertension, urinary lithiasis, and cirrhosis waiting for a liver transplant), and also healthy patients (healthy volunteers for therapeutic studies, living-related kidney donors). Four hundred and sixty-four women and 520 men were included in this analysis, with a mean age of 52.8 years. Serum creatinine was determined with the Jaffe´ reaction in continuous flow after dialysis (automated method, Autoanalyzer Technicone, AAI-11). Choice of the reference values for the glomerular filtration rate Glomerular filtration was measured with inulin clearance (Inuteste; 25% by Fresenius, standard colorimetric method on an Autoanalyzer Technicone, AAI). After overnight fasting, the protocol included three periods of 20 minutes duration during which blood and urine were collected. After a loading dose, inulin was continuously infused during all three periods. All of the results were expressed according to the body surface area in ml/min/ 1.73 m2. According to the aim of the screening, a value of glomerular filtration had to be chosen, and three references values were analyzed. The first value was 80 ml/ min/1.73 m2, which corresponds to the usual value defining the onset of renal failure. The second value was 60 ml/ min/1.73 m2, which corresponds to an established renal failure whatever the sex or age. The third value chosen was 30 ml/min/1.73 m2, which defines a clinically pertinent renal failure. Construction of the receiver operating characteristic curves To transform the quantitative variable (serum creatinine) into a qualitative variable, a cutting line was applied to the scale of its variations. This line defined the pathological or normal character of the result on both sides. The choice of the cutting line determined the discriminating power of the separator. Among the possible values of the discriminant variable (serum creatinine), each value chosen as a cut-off point defined a sensitivity and a specificity. On the ROC curve, the rate of false positives (1 – specificity) was represented on the abscissa, and the rate of the true positives (sensitivity) on the ordinate [2–4]. This curve was used to visualize the effects of changing the cut-off points. Choice of the cutoff value: The Younden index There was no “natural” cut-off value because there was some overlap between the distribution of creatinine
1879
values for affected and unaffected individuals for each reference value of glomerular filtration rate. As the cutoff point changed, the sensitivity and specificity of the test changed in opposite directions. The upper left-hand corner of the ROC graph denoted a perfect diagnostic test: a true positive rate of 100% and a false positive rate of 0%. Thus, the point where the ROC curve was close to the left and top boundaries of the ROC graph corresponded to the value in which the sum of the specificity and sensitivity was the highest. The Younden index, defined as (sensitivity 1 specificity) 2 1, gave the same weight to false positives as it did to the false negatives. To test the robustness of the results, confidence intervals were obtained by using the “bootstrap” technique [7]. For each threshold, after randomly sampling one third of the study group, drawn with a replacement 100 times, a mean and a confidence interval of the 100 bootstrapped cut-off estimates were calculated. For each chosen cut-off value, the sensitivity, specificity, and positive and negative likelihood ratios of the test (serum creatinine) were calculated, as well as the corresponding standard error. The sensitivity of a test corresponded to the rate of true positives and the specificity to the rate of the true negatives. The positive likelihood ratio was the ratio of the rate of the true positives and the rate of the false positives. The negative likelihood ratio was the ratio of the rate of the false negatives and the rate of the true negatives. A positive likelihood ratio equal to 10 means that a positive response of the test (serum creatinine higher than the cut-off value) is 10-fold more frequent among patients with renal failure than among the patients without renal failure. A negative likelihood ratio equal to 0.10 means that a negative response of the test (serum creatinine lower than the cutoff value) is 10-fold less frequent among patients with renal failure than among the patients without renal failure. The predictive values of this test could not be calculated because they depend on the prevalence of the disease. In this population, measuring the prevalence makes no sense, and the prevalence of early renal failure in the general population is not yet known. Thus, projections of predictive values for several plausible values of prevalence were calculated. Calculation of the serum creatinine cut-off value With the Younden index, a cut-off value for serum creatinine for each sex and for three levels of renal function (inulin clearance , 80 ml/min/1.73 m2, , 60 ml/min/ 1.73 m2, or , 30 ml/min/1.73 m2) was determined. Calculation of the cut-off value for the creatinine clearance estimated with Cockroft-Gault’s formula The Cockroft-Gault’s formula predicts the creatinine clearance, taking into account age and sex [8]. It is the
1880
Couchoud et al: Screening early renal failure
Table 1. Characteristics of the patients included in the study Men N 520 Age years 49.7 6 16.7 (range) (15–87) Weight kg 72.8 6 13.2 mean 6 sd (range) (36–125) Height cm 170.8 6 7.4 mean 6 sd, (range) (145–190) 1.84 6 0.17 Body surface m2 mean 6 sd, (range) (1.22–2.41) Serum creatinine lmol /liter 168.1 6 145.5 mean 6 sd, (range) (53–1309) Inulin clearance ml/min/1.73 m2 77.1 6 41.7 mean 6 sd, (range) (4–172) Indications of renal function assessment N (%) Glomerular disease 79 (15) Diabetic nephropathy 62 (12) Tubulointerstitial disease 52 (10) Arteriolar nephrosclerosis 36 (7) Polycystic kidney disease 13 (3) Multisystemic disease 18 (3) Systemic lupus erythematosus 3 (1) Toxemia gravidis 0 Urinary abnormalities 28 (5) Hypertension 19 (4) Chronic renal insufficiency 22 (4) Single kidney 13 (3) “Normal control” 16 (3) Living donor before kidney donation 28 (5) Before liver transplantation 96 (18) Other 35 (7)
Women 464 45.3 6 16.5 (17–95) 62.0 6 12.5 30–130) 159.2 6 7.2 (136–180) 1.63 6 0.16 (1.22–2.26) 114.8 6 93.8 (44–841) 85.5 6 38.4 (2–233) 54 32 42 34 18 46 58 21 37 15 7 8 11 28 42 11
(12) (7) (9) (7) (4) (10) (13) (5) (8) (3) (2) (2) (2) (6) (9) (2)
Fig. 1. Receiver operating characteristic (ROC) curve for serum creatinine in men corresponding to an inulin clearance at 60 ml/min/1.73 m2. The small square indicates the point where the Younden index is maximum (serum creatinine 5 137 mmol/liter).
This AUC calculation is equal to the probability that the test (serum creatinine) is higher in “pathological” patients than in “normal” patients. Thus, an AUC varies from 0.5 (noninformative test) to 1 (most discriminating test). All of the calculations and the graphs were made with the Excele software. RESULTS
most widely employed and the best validated formula for use in adults. The formula is as follows: Creatinine clearance 5 K 3 (140 2 age) 3 weight/serum creatinine where age is in years, weight in kg, and serum creatinine is in mmol/liter. The K equals 1.23 in men and 1.04 in women. With the Younden index, a cut-off value for estimated creatinine clearance for three levels of renal function (inulin clearance ,80 ml/min/1.73 m2, ,60 ml/min/1.73 m2, or ,30 ml/min/1.73 m2) was determined. Glomerular filtration was measured the same way for inulin clearance as for serum creatinine alone. Calculation of the area under the curve To reduce an entire ROC curve to a single quantitative index of diagnostic accuracy, the area under the curve (AUC) was calculated with the method described by Hanley and McNeil [9]. AUC 5 [S 2 p 3 (p 1 1)/2]/(p 3 np) where S equals the sum of the rank of pathological patients; p equals the number of pathological patients, and np 5 number of non-pathological patients.
The characteristics of the 984 patients are presented in Table 1. As an example, one ROC curve for men is presented in the Figure 1. Renal failure was defined by an inulin clearance of less than 60 ml/min/1.73 m2. The graph indicates the point where the Younden index is maximum. This point does not correspond to the intersection with the bisectrix because of sample fluctuations. Tables 2, 3, and 4 summarize the results. The serum creatinine value for a cut-off value at 100% sensitivity, a cut-off value at 95%, 98%, and 100% specificity, and a cut-off value corresponding to maximum Younden index [(sensitivity 1 specificity) 2 1] are given for each glomerular filtration. The estimations of the parameters of accuracy (sensitivity, specificity, and positive and negative likelihood ratios) at the chosen cut-off value and the AUCs are also given in these tables. The projections of a positive predictive value for several plausible values of prevalence are presented in Figure 2, considering the specificity and sensitivity calculated in men corresponding to an inulin clearance of less than 60 ml/min/1.73 m2 (sensitivity 90.0% and specificity 93.3%). Figure 3 represents the trade-off between sensitivity and specificity according to serum creatinine value and the corresponding Younden index. The cut-off values for serum creatinine in various sub-
1881
Couchoud et al: Screening early renal failure Table 2. Cut-off values of serum creatinine in men, for each glomerular filtration rate and at different trade-offs between sensitivity and specificity Inulin clearance ml /min /1.73 m2
Men N 5 520 Number of pathological values Number of non-pathological values Creatininemia at 100% sensitivity lmol /liter Creatininemia at 95% specificity lmol /liter Creatininemia at 98% specificity lmol /liter Creatininemia at 100% specificity lmol /liter Creatininemia at threshold lmol /liter Sensitivity % 6 standard error Specificity % 6 standard error Positive likelihood 6 standard error Negative likelihood 6 standard error Area under the ROC curve
, 80
, 60
, 30
259 261 66 124 133 150 115 86.87 6 1.48 90.80 6 1.27 9.44 6 2.11 0.14 6 0.16 0.941
190 330 80 142 150 208 137 90.0 6 1.32 93.33 6 1.09 13.49 6 2.19 0.11 6 0.22 0.972
97 423 150 208 239 292 177 98.97 6 0.44 90.78 6 1.27 10.73 6 1.04 0.01 6 1.0 0.991
At the cut-off value finally retained, which maximizes the Younden index, are given the estimations of the accuracy markers of the test.
Table 3. Cut-off values of serum creatinine in women, for each glomerular filtration rate and at different trade-offs between sensitivity and specificity Inulin clearance ml /min /1.73 m2
Women N 5 464 Number of pathological values Number of non-pathological values Creatininemia at 100% sensitivity lmol /liter Creatininemia at 95% specificity lmol /liter Creatininemia at 98% specificity lmol /liter Creatininemia at 100% specificity lmol /liter Creatininemia at threshold lmol /liter Estimation of sensitivity % 6 standard error Estimation of specificity % 6 standard error Positive likelihood 6 standard error Negative likelihood 6 standard error Area under the ROC curve
, 80
, 60
, 30
188 276 60 106 113 153 90 85.64 6 1.63 85.51 6 1.63 5.91 6 2.56 0.17 6 0.18 0.914
123 341 75 113 131 153 104 87.80 6 1.52 90.32 6 1.37 9.07 6 2.96 0.14 6 0.24 0.957
50 414 96 148 168 226 146 98.00 6 0.65 94.93 6 1.02 19.33 6 1.99 0.02 6 0.99 0.988
At the cut-off value finally retained, which maximizes the Younden index, are given the estimations of the accuracy markers of the test.
groups are given in Table 5. Bootstrapped cut-off value estimates and their standard deviations are given in Table 6. DISCUSSION Cut-off values of serum creatinine obtained with bootstrapped ROC curves at 80 ml/min/1.73 m2 compared with the results corresponding to the 97.5th percentile of a healthy population [1] are lower in women [92 mmol/ liter (95% CI, 85 to 98) instead of 102 mmol/liter] but not statistically different in men [118 mmol/liter (95% CI, 107 to 131) instead of 112 mmol/liter]. A recent study has confirmed the predictive value of high serum creatinine for the risk of end-stage renal failure [10]. The adjusted relative risk (adjusted on age, proteinuria, hematuria, and hypertension) increased significantly beyond a cutoff value for serum creatinine at 1.2 mg/dl (105 mmol/liter) for women and 1.4 mg/dl (125 mmol/liter) for men. These cut-off values are not different from results calculated at a glomerular filtration of
60 ml/min/1.73 m2: 104 mmol/liter (95% CI, 91 to 117) for women and 139 mmol/liter (120 to 158) for men. The diagnostic function of a biological test is to evaluate, according to its result, the probability for a given disease or the probability of its absence. This function of the test can be expressed by the intrinsic qualities of the test, which are sensitivity and specificity. In fact, the accuracy of a test depends on the frequency of the disease in the studied population [11]. Our analysis was based on a population of patients who underwent a renal function assessment. Although some of them could be considered to be healthy (healthy volunteers for therapeutics studies and living-related kidney donors), the majority of patients were suspected of having a nephropathy. Forty-five percent of the patients had a renal failure defined by an inulin clearance lower than 80 ml/min/1.73 m2. This population was selected in order to reach the needed sample size and not to be representative of the general population. The cost and the complexity of a laboratory evaluation of inulin clearance prevented us from proposing it
1882
Couchoud et al: Screening early renal failure
Table 4. Cut-off values of estimated creatinine clearance (Cockroft-Gault), for each glomerular filtration rate and at different trade-offs between sensitivity and specificity Inulin clearance ml /min /1.73 m2
Cockroft-Gault N 5 900 Number of pathological values Number of non-pathological values Creatine clearance at 100% sensitivity ml /min Creatine clearance at 95% specificity ml /min Creatine clearance at 98% specificity ml /min Creatine clearance at 100% specificity ml /min Creatine clearance at threshold ml /min Estimation of sensitivity % 6 standard error Estimation of specificity % 6 standard error Positive likelihood 6 standard error Negative likelihood 6 standard error Area under the ROC curve
, 80
, 60
, 30
419 481 130 98 95 42 56 74.94 6 1.44 95.43 6 0.70 16.40 6 2.13 0.26 6 0.09 0.936
297 603 82 47 41 26 55 87.88 6 1.09 88.56 6 1.06 7.68 6 4.62 0.14 6 0.38 0.958
140 760 55 33 27 14 37 95.00 6 0.73 92.24 6 0.89 12.24 6 3.08 0.05 6 0.62 0.983
At the cut-off value finally retained, which maximizes the Younden index, are given the estimations of the accuracy markers of the test.
Fig. 2. Projections of predictive values for several plausible values of prevalence, with sensitivity at 90% (men at 60 ml/min/1.73 m2). Symbols are: (d) positive predictive value; (h) negative predictive value.
as a screening test in a large-sized healthy population. That is why we accept dealing with this biased population in which the prevalence of renal failure is higher than in the general population. In our population, the calculation of the predictive values of this test made no sense. In the general population, the prevalence of early renal failure is unknown, but certainly is very low. Figure 2 shows that for a low prevalence, the serum creatinine factor does not have a good positive predictive value. On the contrary, the negative predictive value is always at 100%. This means that by using this test, no patients with renal impairment will be “lost,” but that many healthy patients will be “caught” by the screening study. The advantage of constructing a ROC curve is to adjust the sensitivity or the specificity. Because we were looking for a compromise between high sensitivity and high specificity, the Younden index was chosen because it gives the same weight to the two types of error (false positive and false negative results). A false positive result
can have a psychological impact on the patient and leads to complementary tests that will at least include the search for proteinuria, urinary cytological abnormality and hypertension; they entail a more in-depth renal function assessment and, in the worst case scenario, a renal biopsy. A false negative result leads to the lack of a diagnosis of a disease that could have been curtailed or even treated before the evolution to end-stage renal disease. However, maximizing the Younden index is not necessarily an optimal strategy for screening studies. In the low-prevalence situation, improving the specificity will decrease the number of false positives without greatly increasing the false negatives. One might choose an arbitrary specificity of 98%, to the detriment of sensitivity. That is why for each study, the trade-off between sensitivity and specificity has to be determined by a group of experts according to the aim(s) of the study and the use of these serum creatinine cut-off values. The second advantage of constructing a ROC curve is to adjust the cut-off value for serum creatinine for a specific diagnostic purpose. These cut-off values are calculated according to inulin clearance. Thus, according to the study aims, each serum creatinine cut-off value can be chosen according to a specific glomerular filtration rate. Only the results corresponding to an inulin clearance of 80, 60, and 30 ml/min/1.73 m2 have been presented here, but this method can be generalized to all desired clearances. The few studies concerning the epidemiology of renal failure have taken their cut-off values for serum creatinine in an apparently arbitrary manner, often a whole value. For example, a French study used a cut-off value of 200 mmol/liter [12], a British study used a value of 300 mmol/liter [13], and an American study used 177 mmol/liter as the cut-off value, which corresponds to 2 mg/dl [14]. These studies did not distinguish the gender of the patients and chose high cut-off values that correspond to advanced renal failure.
1883
Couchoud et al: Screening early renal failure
Fig. 3. Trade-off between sensitivity and specificity for each level of serum creatinine in men at 60 ml/min/1.73 m2. The vertical line indicates the point where the Younden index is maximum (serum creatinine 5 137 mmol/liter). The lightly dotted line is the Younden index, the medium line is the sensitivity, and the heavy line is the specificity.
Table 5. Cut-off values of serum creatinine in various subgroups of patients, for the three levels of renal impairment
Subgroups Systemic lupus Diabetes #40 years old $60 years old
Inulin clearance ml /min 80 60 30 80 60 30 80 60 30 80 60 30
Serum creatine lmol /liter Men
Women
— — — 124 128 261 115 144 196 119 137 186
89 111 190 113 113 150 92 124 164 82 106 173
In older patients, one might choose lower ranges for the glomerular filtration target to take into consideration the “physiological” aging process. That is why a glomerular filtration rate at 60 ml/min/1.73 m2, which corresponds to an established renal failure whatever the sex or age, was chosen. Cut-off values according to age have been calculated for smaller populations, with a lower precision than for the entire group. These results have shown some small differences between patients older than 60 years and patients less than 40 years old. These differences were not always caused by a lower cut-off value for older patients, which is expected if one considers that older patients have a lower muscle mass. The difference in serum creatinine between men and women was even higher in older patients. Results for very old patients
Table 6. Robustness of the optimal cut-off values tested by bootstrap methods (100 subsamples of 1/3 of the population) Inulin clearance ml /min/1.73 m2
Serum creatinine lmol /liter
80 60 30
118.6 6 6.15 92.1 6 3.16 139.0 6 9.5 104.1 6 6.83 193.7 6 20.62 156.4 6 13.4
Men
Women
Creatinine clearance ml /min estimated by COCKROFT formula 61.9 6 6.02 53.2 6 5.36 36.9 6 3.31
Data are mean 6 sd.
could not be provided because of the low number of patients concerned in this population. Note that the cutoff values at 60 ml/min/1.73 m2 for patients older than 60 years are higher than cut-off values at 80 ml/min/1.73 m2 for patients less than 40 years old. The area under the curve (AUC) increased as the cutoff value for serum creatinine increased. This means that the diagnostic accuracy of serum creatinine was better when the renal failure was worsening. As the relationship between creatinine concentration and glomerular filtration rate is an hyperbolic function, serum creatinine is often considered to be an inadequate estimate of glomerular filtration, especially for slight reductions in renal function. All AUCs were higher than 0.9, which means that at a population level, plasma creatinine is not that poor of an index of the glomerular filtration rate [15]. Individually, the interpretation of the serum creatinine value is more difficult. For example, one does not know exactly if there is a modification in the relationship between creatinine clearance and inulin clearance according to the underlying disease. A diabetic status might modify
1884
Couchoud et al: Screening early renal failure
this relationship. As well, patients with glomerular disease or systemic lupus who have potentially higher rates of tubular creatinine secretion might have lower cut-off values for serum creatinine. This was not observed in this study, but the population was too small to explore this question precisely. Cut-off values presented for these subgroups are given for information only. Also, the different treatments or the protein intaken, which can also modify the relationship, were not taken into account. The effect of heavy proteinuria on the cut-off values for serum creatinine was not analyzed because of the lack of information on 24-hour proteinuria. The relative heterogeneity of the study population should be considered as a benefit for a later generalization. If a very precisely defined population had been selected, the results possibly would have been more “exact,” however, they would be useless for screening the general population in which the treatment, diet, and comorbidity are not controlled. The Cockroft-Gault’s formula is based on the serum creatinine concentration and has been developed to predict creatinine clearance. The quality criteria of this “test” gave good results. However, there was no clinical difference between the cut-off value at 80 ml/min/1.73 m2 inulin clearance (calculated clearance 56 ml/min) and the cut-off value at 60 ml/min/1.73 m2 inulin clearance (calculated clearance 55 ml/min). Thus, the practical use of the highest cut-off value seemed difficult. The results given by the bootstrap method showed a greater difference between 80 and 60 ml/min/1.73 m2 (62 and 53 ml/ min, respectively), but confirm that this formula underestimates glomerular filtration in the high values. Is this a specific problem related to the population of the study (French specificity, wide range of renal function, etc.)? The impression was that this formula was not well adapted to estimate subnormal glomerular function rate. Under 60 ml/min/1.73 m2 of inulin clearance, the results were good but not better than serum creatinine alone. However, it has the advantage of taking into account the sex of the individual, and thus, there is only one cutoff value to memorize at a given glomerular filtration, but it depends on four different variables (serum creatinine, sex, age, weight) and on the hypothesis of stable creatinine urinary excretion. Another problem is that the “gold standard” definition of renal failure is a biological test (inulin clearance) that also has its limits. Other methods could have been used with the same difficulties. Because of the technical difficulties and high costs, these tests simply estimate the glomerular filtration rate. To thus limit the diagnosis of renal failure to a decrease in the glomerular filtration rate is debatable. In conclusion, ROC curve methods are useful to determine a cut-off value for serum creatinine in epidemiological studies concerning early chronic renal failure screening. The value of the glomerular filtration rate of reference
and the weight of false positive and false negative results have to be adapted to the individual study goals. A renal failure screening in our general population is planned. Our descriptive study should permit us to measure the prevalence of early renal failure and to identify high-risk subgroups. Thus, we will be able to better understand the natural history of nephropathies, plan intervention studies, and organize health care networks. ACKNOWLEDGMENTS The help of Monica Lombardo, Laurent Heyer, Rene´ Ecochard, and all of the patients and nephrologists who are involved in the database is acknowledged. Reprint requests to Docteur Ce´cile Couchoud-Heyer, Service de Ne´phrologie-Hemodialyse-Transplantation Renale, Centre Hospitalier De´partemental Fe´lix Guyon, 97405 Saint-Denis Ile de la Reunion Cedex, France. E-mail: [email protected]
REFERENCES 1. Lauture de H, Caces E, Dubost P: Concentrations of cholesterol, uric acid, urea, glucose and creatinine in a population of 50,000 active individuals, in Reference Values in Human Chemistry, edited by Siest G, Basel, Karger, 1973, pp 141–152 2. Green DM, Swets JA: Signal Detection Theory and Psychophysics. New York, Wiley, 1966 (reprinted Los Altos Hills, CA, Peninsula Publishing, 1988) 3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P: Clinical Epidemiology: A Basic Science for Clinical Medicine (2nd ed). Boston, Little, Brown and Company, 1991, pp 69–152 4. Griner PF, Mayewski RJ, Mushlin AI, Greenland P: Selection and interpretation of diagnostic tests and procedures: Principles and applications. Ann Intern Med 94:553–600, 1981 5. Striker GE: Report on a workshop to develop management recommendations for the prevention of progression in chronic renal disease, Bethesda (MD) April 1994. Nephrol Dial Transplant 10: 290–292, 1995 6. Klahr S: Chronic renal failure: Management. Lancet 338:423–427, 1991 7. Efron B: Bootstrap methods: Another look at the jacknife. Ann Stat 7:1–26, 1979 8. Cockroft DW, Gault MH: Prediction of creatinine clearance from serum creatinine. Nephron 16:31–41, 1976 9. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic curve. Radiology 143:29–36, 1982 10. Iseki K, Ikemiya Y, Fukiyama K: Risk factors of end-stage renal disease and serum creatinine in a community-based mass screening. Kidney Int 51:850–854, 1997 11. Ransohoff DF, Feinstein AR: Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 299:926– 930, 1978 12. Jungers P, Chauveau P, Descamps-Latscha B, Labrunie M, Giraud E, Man NK, Gru¨nfeld JP, Jacobs C: Age and genderrelated incidence of chronic renal failure in a French urban area: A prospective epidemiologic study. Nephrol Dial Transplant 11: 1542–1546, 1996 13. Khan IH, Catto GRD, Edward N, MacLeod AM: Chronic renal failure: Factors influencing nephrology referral. Q J Med 87:559– 564, 1994 14. Strauss MJ, Port FK, Somen C, Wolfe RA: An estimate of the size of the US predialysis population with renal insufficiency and anemia. Am J Kidney Dis 21:264–269, 1993 15. Perneger ThV, Brancati FL, Whelton PK, Klag MJ: Studying the causes of kidney disease in humans: A review of methodologic obstacles and possible solutions. Am J Kidney Dis 25:722–731, 1995