- Email: [email protected]
Reporting standards and statistics for evaluating intervention
Cardiovascular
Vol. 3, No. 5, pp. 455-461, 1995 Copyright 0 1995 Eisevier ScienceLtd Printed in Great Britain. At! rights reserved Surgery,
VASCULAR REVIEW
~-
Reporting standardsand statistics for evahmting intervention K. A.
Myers
i3epartrnent of Vascular Surges, Monash Medical Centre and Epworth Hospital. Melbourne, Ai~~t&i~
Keywords: statistics, life-table analysis. arterial grafting, endovascular
Peer groups now encourage doctors to avoid presenting their experience by anecdotal reminiscences. This is an era where outcome is assessed by statistical techniques. Most studies of treatment for vascular disease use life-table analysis or some variation”2. The technique was first used to determine survival rates after treatment f or cancer3T4 so that purists prefer the term ‘survival analysis’. The most comprehensive description by Lee1 details methods that determine the hazard function, univariate life-table analyses, formulae for standard errors and six techniques to compare results for two groups including the log-rank rest which is a form of x2 analysis. Multivariate Cox proportional hazards regression analysis based on sequential x2 tests compares the independent influence of factors that can affect outcome i,6 , These are satisfying techniques for they squeeze as much information as possible from all patients in the study including those who have just been treated, as well as those who have died or are no longer being followed up. However, statistical methods are just like operations. Some are so good that no further discussion is required like, removing a suppurating appendix, whereas others require careful assessment - perhaps life-table analysis can be compared to a femoropopliteal bypass. Adaptation to study outcomes for vascular disease other than survival such as patency rates, limb salvage or clinical improvement requires assumptions, the validity of which can be disputed. Strict rules need to be defined. Scrutiny of the literature suggests that there are frequent indiscretions (undoubtedly inadvertent) where a 90% lo-year patency rate appears to depend on the lasting good fortune of two or three surviving patients, or a mean 3-month follow-up is entitled ‘. . . long-term results. . .‘. Examples will not be quoted -there are too
Correspondence to: Dr K. A. Myers, 182 Lennox St, Richmond, 3121, Melbourne. Victoria, Australia
CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO 5
many. A cynic would deduce that the way to get the best results is to operate on patients appreciating that they may not live for long and to discharge them early just in case they survive, for provided they are quickly lost then life-table analysis will show that good results at 1-2 months will be almost identical to those calcuiated many years later. Underwood and colleagues’ have described how these techniques can be abused. In response to these concerns, North American vascular surgical and radiology committees have made important recommendations for re orting results after treatin lower-limb arterial disease‘- 11( carotid artey diseaseg2 , aneurysms13 and chronic venous disease’ . However, it is a personal impression that many authors who use life-table analysis for vascular disease state that it was performed in accordance with recommendations by the committees and then proceed to ignore the rules. Accordingly, original articles in four major vascular surgical journals from January 1992 to December 1993 inclusive were reviewed to assess bow well they followed the recommendations. These were the Journal of Cardiovascular Surgery (Torinoj in 1992 and Cardiovascular Surgery in 1993, the journal of V;zscular Strrgery, the European Journal of Vasc&r Surgery and Annuls of Vascular Surgery. Original articles which reported survival alone, presidential addresses, reviews, technical notes and case reports were not analysed. A summary of the techniques for presenting long-term results, how well they complied with the committees’ recommendations, and reasons why the results may be unsatisfactory is presented.
Principles of life-table analysis Patients enter a study for treatment on different dates and these are adjusted back to a common starting point. Each patient, limb or procedure is then followed to an endpoint. This may be the date that treatment fails, or patients may be withdrawn because they complete the study, are lost to follow-up ot die withour the treatment
455
Standards and stat&t&
for evaluating vascular’inten/enthn: h’l A. Myers
having failed, which is termed ‘censored data’. All consecutive patients treated should be included. The actuarial technique of Cutler and Ederer2 reports results at regular intervals and is the most convenient way to analyse larger samples. The hazard function is the probability that a procedure will fail during an interval and allows each interval failure or success rate to be calculated. Interval success rates are progressively multiplied to provide cumulative success rates. It is assumed that censored procedures are withdrawn half-way through the interval and that they would have continued to behave the same as those that remain in the study. Standard errors for the cumulative success rates calculated by either Peto’s or Greenwood’s formulae’ are a function of the number at risk entering each interval. The product-limit technique of Kaplan and Meier’ differs only in that it shows a change at the time of each event, which is useful for smaller samples. The results are identical at the intervals chosen for the actuarial technique. A formula for calculating standard errors gives results similar to those for the actuarial technique’. ‘Covariates’ such as smoking, diabetes, hypertension or myocardial ischaemia, presenting symptoms, extent of disease and technique for intervention have been defined*, and their influence can be studied, comparing subgroups by the log-rank test or all covariates by multivariate analysis.
Recommendationsfrom the committees for reportlng results The following discussion relates to treating lower-limb ischaemia but the principles are the same wherever the site. The ‘surgical committee” described the actuarial technique and Peto’s formula for standard errors and defined most covariates as well as criteria for success and failure for operations. The ‘radiology committee” subsequently presented their guidelines, attempts were made to reconcile differences between the two by representatives from each*‘, and an ‘endovascular committee’l r then presented recommendations. The surgical committee’ recommended that: convariates that could influence outcome should be carefully described; the committee’s definitions for success or failure should be used; a life-table graph or table should show the numbers at risk, the standard error for each interval or both; the numbers that fail or are withdrawn with censored data should be listed for each interval in a table; observations beyond the interval where the standard error exceeds an arbitary 10% should be omitted or at least represented in a manner which indicates poor reliability for estimates beyond that point; and significance of differences between groups should be tested by a log-rank or equivalent test. Presumably, these recommendations apply to the actuarial and the product-limit techniques. 456
Success after open surgery Technical success or patency once was easy to define, namely that the artery treated had not blocked. However, the concept of a ‘failing graft’ that requires intervention to prevent thrombosis led to the surgical committee’ defining ‘primary patency’ as ‘. . uninterrupted patency with either no procedure performed on it or . . procedures performed for disease beyond the graft and its two anastomoses.‘; and ‘secondary patency’ as ‘ by . . .graft patency. . restored after occlusion thrombectomy, thrombolysis or transluminal angioplasty, and/or problems with the graft itself or one of its anastomoses (that) require revision or reconstruction. . .‘. Criteria for patency included the pulse status, vascular laboratory tests, arteriography, or direct observation at operation or post-mortem examination, but not symptoms. Loss of primary patency or technical failure of the operation was not defined by these criteria - only by further intervention. The surgical committee also described categories for clinical severity linked to arbitrary levels of ankle/ brachial pressure indices so as to allow reports for clinical success. Success after endovascular intervention Introduction of endovascular procedures made it even more difficult to present results. Patency alone is not sufficient to define long-term success, for the initial pathology may have been a stenosis which persists or recurs. The radiology committee’ defined outcome by ‘technical success’ - ‘. . . substantial relief of stenosis or occlusion with residual narrowing of 20% or less, significant hemodynamic improvement, and no major and ‘clinical success’ - ‘. . . complete or morbidity.‘; substantial relief of symptoms on baseline . . .’ associated in the aortoliliac segment with ‘. . . alleviation of the systolic pressure gradient across the lesion or the normalisation of femoral pulses’ and in the femoropopliteal segment as ‘. . . increase in the ankle/brachial index of at least 0.15 and/or normalisation of the popliteal pulse.’ They then defined the patency rate as ‘. . . the percentage of patients who have undergone an initially successful procedure for whom flow at the treated site and symptomatic improvement are uninterrupted.’ This combines both technical and clinical success. The definitions are different to those from the surgical committee so that it would not be possible to use them to compare groups treated by either method (even if the groups were similar). The endovascular committee” described ‘clinical success’, ‘hemodynamic improvement’ and ‘anatomic success’. They defined ‘continued success’ in terms of persisting clinical, haemodynamic and anatomical success, and this is similar but not identical to the CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO 5
Standardsand statistics for evaluatihg vascularinten/entjon: K. A. Myers recommendation from the radiology committee. They also defined ‘anatomic failure’ as ‘restenosis to 50% or greater of normal diameter by angiography or other similar imaging assessment’. Thus ‘anatomic success’ for endovascular intervention does not correspond to ‘primary patency’ for surgery, for endovascular failure occurs if there is a stenosis whereas surgical failure occurs only if the stenosis leads to further intervention. The endovascular committeerl recommended that definitions for primary and secondary patency or clinical success should be the same as those recommended by the surgical committees but they did not describe how this was achieved. As the surgical committee stated that secondary patency was maintained if there was ‘proximal or distal extension to the graft’, that is by bypass, it could be argued that a similar local bypass across a recurrence after endovascular intervention should also maintain secondary patency. However, it seems that the definition IS limited to reintervention by endovascular methods only, for example by balloon dilatation for stenosis or fibrinolysis for thrombosis. A further
Table 1 Number of articles using life-table analysis in four vascular surgical journals from January 1992 to December 1993 inclusive Total number of original vascular articles
.Journal
115
ZO(17)
374 185 111
68(18) 28(18) 14(13)
Total
785
130(17)
The findings from the four journals reviewed are shown in Tables 1 and 2. There were 78.5 original articles and 130 (17%) used some form of life-table analysis to present results for vascular disease. The articles related to open surgery for lower-limb arterial disease in X3, endovascular intervention in 13, carorid artery surgery in 21 and treatment at other sites in 13. The actuarial technique was used in 108 and the product-limit technique in 22. Some form of statistical comparison between subgroups was performed in 90 i69%i. of covariates
It was extremely difficult to assess the degree to which articles defined covariates. Most referred to the covariates but apparently assumed rhat their definitions were obvious. Ten articles (8%) failed to refer to any definitions of covariates in the text, I2 (9%) clearly indicated that covariates were as defined by the committees, and only three (2%) provided their own detailed descriptions. Reports that did not adequately define subgroups could hardly then compare them. This is unfortunate for
Values in parentheses are percentages
Table 2
CompIiancewith the committees” recommendations
Description
Number using life-table analysis
J Cardivvasc Surg (Tormo!/ cardovasc sllry J Vast Surg Eur J Vast Surg Ann Vast Surg
category of ‘assisted primary patency’ has been introduced for both open surgery and endovascular intervention in which the graft or artery remains patent and continued atency is assisted by further endovascular procedures ‘r, l5 but how this differs from secondary patency for endovascular intervention is not described. The endovascular committee” recommended that the point at which standard error exceeds 10% be used to define results as ‘short-term’ (valid data to a 6-month follow-up), ‘intermediate-term’ (6-24 months) or ‘longterm’ (minimum 2-year follow-up).
Failure of articles to follow committee recommendations
Failure of recommendation
Covarrates not adequately described Success and failure not defined Numbers at risk or standard error not shown for each interval Failures and censored data not listed for each interval Graph extended well beyond standard error > 10% Log-rank test not used to compare groups
J Card/ovasc Surg (Torino)/ J VaseSurg (n= 68) CardiovascSurg (n=ZO) 5(25) 6(3D) 1 l(55)
2~3) w3
8(12)
EurJ VascSurg (n=28)
Ann VascSurg(rwl4
Total (n= 130)
~(7)
l(7)
IO@!
1 l(39) 3(11i
l(7) l(7)
27(21) 23( 18)
15(75)
37(54)
24(86)
6(43!
82(63)
8(4D)
18(26)
1W4)
6(43\
50(38)
l(5)
14(21)
2~7)
2(14)
19(15)
Values !n parentheses are percentages
CARDlOVASClJLAR SURGERY OCTOBER 1995 VOL 3 NO 5
457
Standards andstatktics for evaluating vascular intervention: K A. Myen
many covariates have been carefully studied. For example, a European consensus has defined critical ischaemia 16, admitted1 without universal acceptancei’, ” Strandness and Carter Y* clarified several clinical definitions, and the surgical committee* carefully described associated general disease, symptoms, operations and arteriographic grading. It should be the authors’ responsibility to succinctly but precisely define what they think they are talking about. However, accurate definition can be difficult - is there any less threat from symptomless extensive coronary artery disease than disease that has caused a long past myocardial infarct, what is the significance of an abnormal glucose tolerance test without symptoms of diabetes, can smoking habits be accurately quantitated, can critical ischaemia be defined, does arteriographic grading really correlate with outflow resistance, and where is the vein graft diameter measured? Further, as all practitioners perform each operation differently, do conclusions relate to their technical skills rather than patient characteristics? Definition
of success and failure
There were 27 articles (21%) in which it appeared that no attempt was made to define what was meant by success or failure so that the precise endpoints were obscure. Several authors appeared to consider that the were too restrictive. Some committees’ definitions simply recorded crude patency rates” which is reasonable as long as this is clearly stated. Compliance with the endovascular committee’s recommendationsl’ has yet to be adequately tested. Ahn and colleagues2’ used the surgical committee’s recommendations’ to present patency rates for endovascular intervention by atherectomy, but a report on atherectomy subsequent to the period studied used criteria which required clinical, haemodynamic and anatomical success with predictably bad results21, a technique which may well lead to honest reporting but which was not observed in any other article during the period of study. The surgical committee’s definitions are strict, perhaps too strict for many. For example, a patient with critical ischaemia treated by bypass for an occlusion may subsequently return with claudication due to stenosis at the treated site; the bypass is considered to have failed at the date for reintervention by balloon dilatation even though the patient is much better than when bypass was performed. The endovascular committee’s preferred definition relating to clinical success‘* is even more strict and it will be interesting to see if it is accepted. For example, a patient may have a patent artery and complete symptom relief yet is judged to have anatomical or haemodynamic failure on the basis of an abnormal vascular laboratory test, using criteria which have yet to be validated in these patients. It can be difficult to be sure that an artery is patent.
Many studies failed to note that there was any objective evaluation to confirm clinical judgement. Some patients report clinical benefit even though objective studies show that the intervention has failed; conservative amputations can remain healed long after a graft has become blocked. Many patients are not prepared to complain after failure, perhaps because they fear that the operation will be repeated, yet it appeared that several studies accepted patient satisfaction to indicate patency. Pulses can be misleading, as many patients have occluded crural arteries and never regain ankle pulses, and detecting a weakening pulse is unreliable. A low or falling ankle/brachial pressure index should be viewed with great suspicion22 but may be the result of disease progression above and below rather than at the intervention site. The intervention may fail long before this becomes apparent23 so that regular duplex ultrasonography allows honest appraisal, but ultrasonographic criteria to predict failure after endovascular intervention have not yet been adequately defined. It can be difficult to date failure. Patients are reviewed at increasing intervals, they may not have been told to represent immediately if symptoms recur, and many are vague about when this occurred. The surgical committee recommended that the date for failure should be the last date that the patient was seen with a patent graft, but it is likely that many reports recorded the outcome at the date that the patient represented with symptoms. No study was noted to specifically refer to this recommendation. The ongoing status cannot be adequately assessed by the patients or inexperienced general practitioners so that it is best not to rely on phone or mail reports from either. Most papers did not state how the information was obtained. Strictly, the intervention should be considered to be successful only if this is assessed within the unit, and then preferably by an independent observer. However, Harris24 cautions that a vascular unit that performs ten bypass and five endovascular procedures per week and follows the patients four times/year will be besieged by 400 outpatients/week within 5 years, even allowing for an annual 20% attrition from death or geographical relocation. Patency rates were reported more frequently than clinical success. This is reasonable given that technical outcome is the aspect over which a surgeon has most control. Patients are, however, more likely to be interested in knowing whether intervention will relieve their disability. Rates for symptom relief are usually less than patency rates, particularly if there are multiple levels of disease as with aortofemoral bypass25,26. Limb salvage rates were frequently presented for critical ischaemia but no report was found describing rates for relief from claudication. The patient may even wish to know the more general picture, namely the prospects of surviving with an intact leg and a patent artery with symptom relieP’ but this was not reported, perhaps because it is too depressing. CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO S
Standardsandstatistks
Numbers at risk and standard error for each interval There was no attempt to show the numbers at risk or the standard error for any interval in 23 articles (18%) making it impossible to determine whether results were valid. This was frequently associated with small numbers being collected over a long time with life-table analysis applied for the entire duration, so that conclusions drawn for late results were undoubtedly invalid. There are no strict rules as to how many patients should be studied but the techniques can be abused if the original sample size is small, particularly if the numbers lost as failures or censored data are large. The standard error is a function of the number entering each interval - a report should be ignored if this does not progressively increase. The committees recommend that data be ignored or at least treated with caution when the standard error exceeds 10%8,11. More than 30-40 procedures in a group would usually be required for data to be valid for medium-term analysis at 12 months. Short-term results from life-table analysis can be useful as small numbers followed for a limited time can be analysed to show which procedures appear to be promising and which are not likely to be effective. However, reliance on life-table analysis to determine true results demands careful follow-up of relatively large numbers. Uncommon diseases or unusual operations need to be reported, but not by life-table analysis if this involves only a few patients; it is better to use some other technique - perhaps by anecdotal descriptions. Numbers with censored data The relative numbers that failed or were withdrawn patent with censored data were not recorded in 82 articles (63%). This was almost invariable in papers that used the product-limit technique. There were differences between journals which may in part be the result of editorial policy. This made it difficult to determine whether there was possible bias due to a high level of censored data, An assumption that patients withdrawn with censored data would have behaved just like those who remain in the study may not be valid. It is as well to be suspicious of a report if the numbers of censored data are not shown by a table or histogram. Completing the study It is rarely possible to review every patient close to the date chosen to finish the study so that it can be difficult to know whether or not they are still under review. An arbitrary decision could be made that a patient is under review if seen within 12 months of the date for completing the study. Lost to follow-up Many patients are lost to follow-up if they are discharged back to their referring practitioners or move CARLllOVASCULAR SURGERY OCTOBER 199s VOL 3 NO S
forevaluating vascularinte~er~ii'i~~n:h: A. Myers
far away. They may, however, simply elect not to return, either becausethey are completely satisfied with the result, but perhaps more often because they have had a bad result and no longer wish to expose themselves to further ineffectual treatment. An excess who withdraw with bad results will lead ro life-table analysis calculating artificially high successrates. In a study of femorodistal bypass grafts”, the author found that withdrawal rates were 2-6%/annum for loss to follow-up. It is best to doubt long-term conclusions if the proportion lost to follow-up is too high - perhaps more than lo-lS%lannum. Death Death with a patent artery is a frequenr reason for censored data. Patients who die are probably different to those who survive - in particular, they are likely to have more severe atherosclerosis. Patency rates for femorodistal vein bypass grafts performed in the author’s unit were calculated before and after removing patients who died with patent grafts”. Patients in the censored group who died were significantly more likely to have had a past clinical history of myocardial ischaemia and to have presented with critical ischaemia. Their removal from the analysis resulted in calculated late patency rates being almost 10% lower than the initial calculation for all grafts. Amputation This is a difficult problem for any form of intervention. How is the endpoint defined in a patient with gangrene due to superficial femoral artery diseasewho is noted to have spurting tibia1 arteries distal to a technically successful femoral bypass or balloon dilatation at the time of below-knee amputation? Obvioudy this is a clinical failure, but some authors2’ reasonably considered that this should be censored data for reports of technical outcome. Standard errors less than 10% The graphs failed to show the point at which the standard error exceeded 10% in 50 articles (38%). There were at least five studies in which subgroups of less than 25 were analysed for periods of 5-10 years in spite of the standard error exceeding 10% within a few months; log-rank testing then demonstrated ‘significant’ differences between groups which probably would not have been found if the follow-up had been terminated earlier, raising the possibility of a type I error. This recommendation was commonly ignored so that results for a small favoured few were drawn out for many years, which is either naive or mischievous. It should be editorial policy to insist that the numbers be shown in a table or that standard errors be shown in a graph, and that prolonged follow-up of small numbers should be discouraged. A long flat distal segment or a 459
Standards andstatkti~s for evaluating vascular intervention: K A. Myers
sudden terminal dip in the graph indicates that this part at least should be ignored. Comparison of groups by a log-rank test There were several papers in which subgroups were compared but with no explanation as to how this was performed. Subgroups appeared to have been compared by statistical techniques other than those described by the committees*‘l’ in 19 articles (15%), usually by separate univariate analysis performed at each interval which is not valid. However, using life-table analysis to compare subgroups can lead to problems. Even if the total group is large, one subgroup is likely to be appreciably smaller than the other so that a test to compare the two can be unreliable. A significant difference between groups does not mean that it is directly caused by the covariate; late patency rates may be lower in diabetic patients than in non-diabetics but this could be becausethey have more severe ischaemia and more extensive distal disease requiring more difficult operations. Confounding as a result of interdependence between multiple covariates is the reason why univariate techniques should be applied to simply describe outcome rather than to interpret reasons for results, and why multivariate techniques are useful.
Follow-up data from the hospital records, general practitioners, or the patients or their relatives are usually unreliable. Even if good long-term data are obtained, they may be passed on to a statistician who usually does not comprehend the implications of the results. Life-table calculations are tedious so that they may be imposed on inexperienced junior staff who do not understand the methods. Demands for a new paper for the ever-impending next meeting may lead to dredging out another group for hasty analysis before the inexorable deadline. Preparing graphs may require talents from lay staff who may not fully appreciate the statistical requirements - this can usually be detected by presentations using gaudy multicoloured slides. The definitions can be confusing but the techniques for survival analysis and their illustration are not difficult and should always be the personal responsibility of the presenting author. Whatever the reason, authors, reviewers and editors should look to limit life-table analysis to studies in which there are enough patients followed up adequately for a sufficient length of time. If possible, the results should be presented in one of the recommended fashions which must be fully explained and adhered. Authors should use other techniques to present their results if this is not possible.
References Conclusions This review independently mirrors anxieties expressed by Underwood and colleagues’ more than 10 years ago. Many of the vascular committees’*‘rr subsequent recommendations flowed from the use and abuse of life-table analysis. However, problems clearly persist. Many practitioners have become cynical about reported results which do not appear to reflect their own experience. There were reports with a modest outcome in keeping with the experience of many in which all recommendations by the committees were followed perfectly15F29.There were other reports where life-table analysis showed excellent results for various methods used to manage vascular disease. Many appeared to reflect the expertise of the doctors and efficacy of the chosen management. Unfortunately, others appeared to disregard the recommendations casting doubts on their conclusions. There are many possible reasons why performance is poor. The larger the unit, the more patients there are to follow up and the more difficult this becomes. The better the unit, the more likely it is to attract patients from far away who will go home after treatment and be lost to follow-up. Few units have developed good audit systems. Many contemporary prospective cardiovascular trials spend considerable funds developing systemsto track down patients, even employing private detective agencies - no vascular unit could justify such expense. 460
1. 2.
3.
4.
5. 6. 7. 8.
9.
10.
Lee ET. Statistical Methods for Survival Data Analysis. Belmont, California: Lifetime Learnine Publications. 1980. Benedetti J, Yuen K, Young L”. Life tables and survivor functions. In: Dixon WJ, ed. BMDP Statistical Software. Los Angeles: University of California Press, 1990: 739-68. Peto R, Pike MC, Armitage P et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I Introduction and design. BY ] Cancer 1976; 34: 585-612. Peco R, Pike MC, Armitage P et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II Analysis and examples. BY J Cancer 1977; 35: l-39. Cox DR. Regression models and life-tables. /ournal ofthe Royal Statistical Society 1972; 34: 187-220. Hopkins A. Survival analysis with covariates. In: Dixon WJ, ed. BMDP Statistical Software. Los Aneeles: Universitv of California Press, 1990: 769’-806. ” Underwood CJ, Faragher EB, Charlesworth D. The uses and abuses of life-table methods in vascular surgery. BY] Surg 1984; 71: 495-8. Rutherford RB, Flanigan DP, Gupta SK et al. Suggested standards for reports dealing with lower extremity ischaemia. Prepared by the Ad Hoc Committee on Reporting Standards, Society for Vascular Surgery/North American Chapter, International Society for Cardiovascular Surgery. ] Vast Surg 1986; 4: 80-94. Spies JB, Bakal CW, Burke DR et al. Guidelines for percutaneous transluminal angioplasty. Standards of Practice Committee of the Society of Cardiovacular and Interventional Radiology. Journal of Vascular and Interventional Radiology 1990; 1: S-IS. Rutherford RB, Becker GJ. Standards for evaluating and reporting the results of surgical and percutaneous therapy for peripheral arterial disease. Journal of Vascular and Inter-
CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO S
Standardsandstatfforevaluating
11.
12. 13.
14. 15.
16. 17.
18. 19.
20.
21.
ventional Radiology 1991; 2: 169-74. Ahn SS, Rutherford RB, Becker GJ et al. Reporting standards for lower extremity arterial endovascular procedures. 1 Vusc Surg 1993; 17: 1103-7. Moore WS, Mohr JP, Najafi H et al. Carotid endarterectomy: practice guidelines. ] Vusc Surg 1992; 15: 469-79. Johnston KW, Rutherford RB, Tilson MD et al. Suggested standards for reporting on arterial aneurysms. J Vusc Surg 1991; 13: 452-8. Porter JM, Rutherford RB, Clagett GP et al. Reporting standards m venous disease. I Vusc Surg 1988; 8: 172-8 1. Donaldson MC, Whittemore AD, Mannick JA. Further experience with an all-autogenous tissue policy for infrainguinal reconstruction. J Vusc Surg 1993; 18: 41-8. European Working Group on critical leg ischaemia. Second European consensus document on chronic critical leg ischaemia. Eur J Vusc Surg 1992; b(Suppl A): l-28. Thompson MM, Sayers RD, Varty K, Reid A, London NJM, Bell PRF. Chrome critical leg ischaemia must be redefined. Eur J Vast Surg 1993; 7: 420-6. Strandness DE, Carter SA. Outcome criteria in patients with peripheral arterial disease. Ann Vusc Surg 1993; 7: 491-6. Myers KA, Fuller JA, Scott DF, Devine TJ, Denton MJ, Chan A. Multivariate Cox regression analysis of covariates for patency rates after femorodistal vein bypass grafting. Ann Vusc Surg 1993; 7: 262-L). Ahn SS, Eton D, Yeatman LR, Deutsch L-S, Moore WS. Intraoperative peripheral rotary atherectomy: early and late clinical results. Ann Vast Sure 1992: 6: 272-80. The Collaboranve Rotablat& Atherectomy Group (CRAG).
CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO 5
22.
23.
24. 25.
26.
27.
28.
29.
vascular'i~ten/pntirin: K. .hMyefi
Peripheral atherectomy with the rotablator: a rnult~cenrrr report. J Vusc Surg 1994; 19: 509-15. Prime HT, Myers KA, Matthews PG, Zeng C. Early postoperative ankie pressure indices help predict late patency rates after femoro-distal vein bypass grafting. ! Gzrdiorww Surg (Tormo) 1992; 33: 426-31. Grigg MJ, Wolfe JHN, Tovar A, Nicolaides AN. The reliability of duplex derived haemodynamic measurements m the assessment of femoro-distal grafts. EurJ VNSCSurg 1988; 2: 177-81. Harris Pi.. Follow-up after reconstructwe Jrtcrial surgery. EUY I Vast Surg 1991; 5: 369-7.3. Hill DA, McGrath MA, Lord RSA. The effect of superficial femoral artery occlusion on the outcome of aortofemoral bypass for intermittent claudication. Surgery 1980: 87: 133- 6. King RB, Myers KA, Scott DF, Devine 1’1. ‘The choice ot operation in aortoiliac reconstructions for intcrmment claudication. World J Surg 1983; 7: 334-9. Myers KA, Scott DF, Devine TJ. Reversed autogenous vein grafts for occlusive arterial disease in the lower limb. In: Greenhalgh RM, ed. Indications in Vascular Surgery Philadelphia: WB Saunders, 1988: 225-48. Wagner WH, Levin PM, Treiman RL, Cossmdn DV, Foran RF, Cophen JL. Early results of infrainguinal arterial reconstruction with a modified biological conduit. Am C’&rsr Burg 1992; 6: 325-33. Harrington ME, Harrington EB, Schanzer H. Jacobson JH, Haimov M. The dorsalis pedis bypass - moderate success in difficult situations. J Vssc Surg 1992, 1’7, 409.. 16.
Paper accepted 30 January 199.5
461
Vol. 3, No. 5, pp. 455-461, 1995 Copyright 0 1995 Eisevier ScienceLtd Printed in Great Britain. At! rights reserved Surgery,
VASCULAR REVIEW
~-
Reporting standardsand statistics for evahmting intervention K. A.
Myers
i3epartrnent of Vascular Surges, Monash Medical Centre and Epworth Hospital. Melbourne, Ai~~t&i~
Keywords: statistics, life-table analysis. arterial grafting, endovascular
Peer groups now encourage doctors to avoid presenting their experience by anecdotal reminiscences. This is an era where outcome is assessed by statistical techniques. Most studies of treatment for vascular disease use life-table analysis or some variation”2. The technique was first used to determine survival rates after treatment f or cancer3T4 so that purists prefer the term ‘survival analysis’. The most comprehensive description by Lee1 details methods that determine the hazard function, univariate life-table analyses, formulae for standard errors and six techniques to compare results for two groups including the log-rank rest which is a form of x2 analysis. Multivariate Cox proportional hazards regression analysis based on sequential x2 tests compares the independent influence of factors that can affect outcome i,6 , These are satisfying techniques for they squeeze as much information as possible from all patients in the study including those who have just been treated, as well as those who have died or are no longer being followed up. However, statistical methods are just like operations. Some are so good that no further discussion is required like, removing a suppurating appendix, whereas others require careful assessment - perhaps life-table analysis can be compared to a femoropopliteal bypass. Adaptation to study outcomes for vascular disease other than survival such as patency rates, limb salvage or clinical improvement requires assumptions, the validity of which can be disputed. Strict rules need to be defined. Scrutiny of the literature suggests that there are frequent indiscretions (undoubtedly inadvertent) where a 90% lo-year patency rate appears to depend on the lasting good fortune of two or three surviving patients, or a mean 3-month follow-up is entitled ‘. . . long-term results. . .‘. Examples will not be quoted -there are too
Correspondence to: Dr K. A. Myers, 182 Lennox St, Richmond, 3121, Melbourne. Victoria, Australia
CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO 5
many. A cynic would deduce that the way to get the best results is to operate on patients appreciating that they may not live for long and to discharge them early just in case they survive, for provided they are quickly lost then life-table analysis will show that good results at 1-2 months will be almost identical to those calcuiated many years later. Underwood and colleagues’ have described how these techniques can be abused. In response to these concerns, North American vascular surgical and radiology committees have made important recommendations for re orting results after treatin lower-limb arterial disease‘- 11( carotid artey diseaseg2 , aneurysms13 and chronic venous disease’ . However, it is a personal impression that many authors who use life-table analysis for vascular disease state that it was performed in accordance with recommendations by the committees and then proceed to ignore the rules. Accordingly, original articles in four major vascular surgical journals from January 1992 to December 1993 inclusive were reviewed to assess bow well they followed the recommendations. These were the Journal of Cardiovascular Surgery (Torinoj in 1992 and Cardiovascular Surgery in 1993, the journal of V;zscular Strrgery, the European Journal of Vasc&r Surgery and Annuls of Vascular Surgery. Original articles which reported survival alone, presidential addresses, reviews, technical notes and case reports were not analysed. A summary of the techniques for presenting long-term results, how well they complied with the committees’ recommendations, and reasons why the results may be unsatisfactory is presented.
Principles of life-table analysis Patients enter a study for treatment on different dates and these are adjusted back to a common starting point. Each patient, limb or procedure is then followed to an endpoint. This may be the date that treatment fails, or patients may be withdrawn because they complete the study, are lost to follow-up ot die withour the treatment
455
Standards and stat&t&
for evaluating vascular’inten/enthn: h’l A. Myers
having failed, which is termed ‘censored data’. All consecutive patients treated should be included. The actuarial technique of Cutler and Ederer2 reports results at regular intervals and is the most convenient way to analyse larger samples. The hazard function is the probability that a procedure will fail during an interval and allows each interval failure or success rate to be calculated. Interval success rates are progressively multiplied to provide cumulative success rates. It is assumed that censored procedures are withdrawn half-way through the interval and that they would have continued to behave the same as those that remain in the study. Standard errors for the cumulative success rates calculated by either Peto’s or Greenwood’s formulae’ are a function of the number at risk entering each interval. The product-limit technique of Kaplan and Meier’ differs only in that it shows a change at the time of each event, which is useful for smaller samples. The results are identical at the intervals chosen for the actuarial technique. A formula for calculating standard errors gives results similar to those for the actuarial technique’. ‘Covariates’ such as smoking, diabetes, hypertension or myocardial ischaemia, presenting symptoms, extent of disease and technique for intervention have been defined*, and their influence can be studied, comparing subgroups by the log-rank test or all covariates by multivariate analysis.
Recommendationsfrom the committees for reportlng results The following discussion relates to treating lower-limb ischaemia but the principles are the same wherever the site. The ‘surgical committee” described the actuarial technique and Peto’s formula for standard errors and defined most covariates as well as criteria for success and failure for operations. The ‘radiology committee” subsequently presented their guidelines, attempts were made to reconcile differences between the two by representatives from each*‘, and an ‘endovascular committee’l r then presented recommendations. The surgical committee’ recommended that: convariates that could influence outcome should be carefully described; the committee’s definitions for success or failure should be used; a life-table graph or table should show the numbers at risk, the standard error for each interval or both; the numbers that fail or are withdrawn with censored data should be listed for each interval in a table; observations beyond the interval where the standard error exceeds an arbitary 10% should be omitted or at least represented in a manner which indicates poor reliability for estimates beyond that point; and significance of differences between groups should be tested by a log-rank or equivalent test. Presumably, these recommendations apply to the actuarial and the product-limit techniques. 456
Success after open surgery Technical success or patency once was easy to define, namely that the artery treated had not blocked. However, the concept of a ‘failing graft’ that requires intervention to prevent thrombosis led to the surgical committee’ defining ‘primary patency’ as ‘. . uninterrupted patency with either no procedure performed on it or . . procedures performed for disease beyond the graft and its two anastomoses.‘; and ‘secondary patency’ as ‘ by . . .graft patency. . restored after occlusion thrombectomy, thrombolysis or transluminal angioplasty, and/or problems with the graft itself or one of its anastomoses (that) require revision or reconstruction. . .‘. Criteria for patency included the pulse status, vascular laboratory tests, arteriography, or direct observation at operation or post-mortem examination, but not symptoms. Loss of primary patency or technical failure of the operation was not defined by these criteria - only by further intervention. The surgical committee also described categories for clinical severity linked to arbitrary levels of ankle/ brachial pressure indices so as to allow reports for clinical success. Success after endovascular intervention Introduction of endovascular procedures made it even more difficult to present results. Patency alone is not sufficient to define long-term success, for the initial pathology may have been a stenosis which persists or recurs. The radiology committee’ defined outcome by ‘technical success’ - ‘. . . substantial relief of stenosis or occlusion with residual narrowing of 20% or less, significant hemodynamic improvement, and no major and ‘clinical success’ - ‘. . . complete or morbidity.‘; substantial relief of symptoms on baseline . . .’ associated in the aortoliliac segment with ‘. . . alleviation of the systolic pressure gradient across the lesion or the normalisation of femoral pulses’ and in the femoropopliteal segment as ‘. . . increase in the ankle/brachial index of at least 0.15 and/or normalisation of the popliteal pulse.’ They then defined the patency rate as ‘. . . the percentage of patients who have undergone an initially successful procedure for whom flow at the treated site and symptomatic improvement are uninterrupted.’ This combines both technical and clinical success. The definitions are different to those from the surgical committee so that it would not be possible to use them to compare groups treated by either method (even if the groups were similar). The endovascular committee” described ‘clinical success’, ‘hemodynamic improvement’ and ‘anatomic success’. They defined ‘continued success’ in terms of persisting clinical, haemodynamic and anatomical success, and this is similar but not identical to the CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO 5
Standardsand statistics for evaluatihg vascularinten/entjon: K. A. Myers recommendation from the radiology committee. They also defined ‘anatomic failure’ as ‘restenosis to 50% or greater of normal diameter by angiography or other similar imaging assessment’. Thus ‘anatomic success’ for endovascular intervention does not correspond to ‘primary patency’ for surgery, for endovascular failure occurs if there is a stenosis whereas surgical failure occurs only if the stenosis leads to further intervention. The endovascular committeerl recommended that definitions for primary and secondary patency or clinical success should be the same as those recommended by the surgical committees but they did not describe how this was achieved. As the surgical committee stated that secondary patency was maintained if there was ‘proximal or distal extension to the graft’, that is by bypass, it could be argued that a similar local bypass across a recurrence after endovascular intervention should also maintain secondary patency. However, it seems that the definition IS limited to reintervention by endovascular methods only, for example by balloon dilatation for stenosis or fibrinolysis for thrombosis. A further
Table 1 Number of articles using life-table analysis in four vascular surgical journals from January 1992 to December 1993 inclusive Total number of original vascular articles
.Journal
115
ZO(17)
374 185 111
68(18) 28(18) 14(13)
Total
785
130(17)
The findings from the four journals reviewed are shown in Tables 1 and 2. There were 78.5 original articles and 130 (17%) used some form of life-table analysis to present results for vascular disease. The articles related to open surgery for lower-limb arterial disease in X3, endovascular intervention in 13, carorid artery surgery in 21 and treatment at other sites in 13. The actuarial technique was used in 108 and the product-limit technique in 22. Some form of statistical comparison between subgroups was performed in 90 i69%i. of covariates
It was extremely difficult to assess the degree to which articles defined covariates. Most referred to the covariates but apparently assumed rhat their definitions were obvious. Ten articles (8%) failed to refer to any definitions of covariates in the text, I2 (9%) clearly indicated that covariates were as defined by the committees, and only three (2%) provided their own detailed descriptions. Reports that did not adequately define subgroups could hardly then compare them. This is unfortunate for
Values in parentheses are percentages
Table 2
CompIiancewith the committees” recommendations
Description
Number using life-table analysis
J Cardivvasc Surg (Tormo!/ cardovasc sllry J Vast Surg Eur J Vast Surg Ann Vast Surg
category of ‘assisted primary patency’ has been introduced for both open surgery and endovascular intervention in which the graft or artery remains patent and continued atency is assisted by further endovascular procedures ‘r, l5 but how this differs from secondary patency for endovascular intervention is not described. The endovascular committee” recommended that the point at which standard error exceeds 10% be used to define results as ‘short-term’ (valid data to a 6-month follow-up), ‘intermediate-term’ (6-24 months) or ‘longterm’ (minimum 2-year follow-up).
Failure of articles to follow committee recommendations
Failure of recommendation
Covarrates not adequately described Success and failure not defined Numbers at risk or standard error not shown for each interval Failures and censored data not listed for each interval Graph extended well beyond standard error > 10% Log-rank test not used to compare groups
J Card/ovasc Surg (Torino)/ J VaseSurg (n= 68) CardiovascSurg (n=ZO) 5(25) 6(3D) 1 l(55)
2~3) w3
8(12)
EurJ VascSurg (n=28)
Ann VascSurg(rwl4
Total (n= 130)
~(7)
l(7)
IO@!
1 l(39) 3(11i
l(7) l(7)
27(21) 23( 18)
15(75)
37(54)
24(86)
6(43!
82(63)
8(4D)
18(26)
1W4)
6(43\
50(38)
l(5)
14(21)
2~7)
2(14)
19(15)
Values !n parentheses are percentages
CARDlOVASClJLAR SURGERY OCTOBER 1995 VOL 3 NO 5
457
Standards andstatktics for evaluating vascular intervention: K A. Myen
many covariates have been carefully studied. For example, a European consensus has defined critical ischaemia 16, admitted1 without universal acceptancei’, ” Strandness and Carter Y* clarified several clinical definitions, and the surgical committee* carefully described associated general disease, symptoms, operations and arteriographic grading. It should be the authors’ responsibility to succinctly but precisely define what they think they are talking about. However, accurate definition can be difficult - is there any less threat from symptomless extensive coronary artery disease than disease that has caused a long past myocardial infarct, what is the significance of an abnormal glucose tolerance test without symptoms of diabetes, can smoking habits be accurately quantitated, can critical ischaemia be defined, does arteriographic grading really correlate with outflow resistance, and where is the vein graft diameter measured? Further, as all practitioners perform each operation differently, do conclusions relate to their technical skills rather than patient characteristics? Definition
of success and failure
There were 27 articles (21%) in which it appeared that no attempt was made to define what was meant by success or failure so that the precise endpoints were obscure. Several authors appeared to consider that the were too restrictive. Some committees’ definitions simply recorded crude patency rates” which is reasonable as long as this is clearly stated. Compliance with the endovascular committee’s recommendationsl’ has yet to be adequately tested. Ahn and colleagues2’ used the surgical committee’s recommendations’ to present patency rates for endovascular intervention by atherectomy, but a report on atherectomy subsequent to the period studied used criteria which required clinical, haemodynamic and anatomical success with predictably bad results21, a technique which may well lead to honest reporting but which was not observed in any other article during the period of study. The surgical committee’s definitions are strict, perhaps too strict for many. For example, a patient with critical ischaemia treated by bypass for an occlusion may subsequently return with claudication due to stenosis at the treated site; the bypass is considered to have failed at the date for reintervention by balloon dilatation even though the patient is much better than when bypass was performed. The endovascular committee’s preferred definition relating to clinical success‘* is even more strict and it will be interesting to see if it is accepted. For example, a patient may have a patent artery and complete symptom relief yet is judged to have anatomical or haemodynamic failure on the basis of an abnormal vascular laboratory test, using criteria which have yet to be validated in these patients. It can be difficult to be sure that an artery is patent.
Many studies failed to note that there was any objective evaluation to confirm clinical judgement. Some patients report clinical benefit even though objective studies show that the intervention has failed; conservative amputations can remain healed long after a graft has become blocked. Many patients are not prepared to complain after failure, perhaps because they fear that the operation will be repeated, yet it appeared that several studies accepted patient satisfaction to indicate patency. Pulses can be misleading, as many patients have occluded crural arteries and never regain ankle pulses, and detecting a weakening pulse is unreliable. A low or falling ankle/brachial pressure index should be viewed with great suspicion22 but may be the result of disease progression above and below rather than at the intervention site. The intervention may fail long before this becomes apparent23 so that regular duplex ultrasonography allows honest appraisal, but ultrasonographic criteria to predict failure after endovascular intervention have not yet been adequately defined. It can be difficult to date failure. Patients are reviewed at increasing intervals, they may not have been told to represent immediately if symptoms recur, and many are vague about when this occurred. The surgical committee recommended that the date for failure should be the last date that the patient was seen with a patent graft, but it is likely that many reports recorded the outcome at the date that the patient represented with symptoms. No study was noted to specifically refer to this recommendation. The ongoing status cannot be adequately assessed by the patients or inexperienced general practitioners so that it is best not to rely on phone or mail reports from either. Most papers did not state how the information was obtained. Strictly, the intervention should be considered to be successful only if this is assessed within the unit, and then preferably by an independent observer. However, Harris24 cautions that a vascular unit that performs ten bypass and five endovascular procedures per week and follows the patients four times/year will be besieged by 400 outpatients/week within 5 years, even allowing for an annual 20% attrition from death or geographical relocation. Patency rates were reported more frequently than clinical success. This is reasonable given that technical outcome is the aspect over which a surgeon has most control. Patients are, however, more likely to be interested in knowing whether intervention will relieve their disability. Rates for symptom relief are usually less than patency rates, particularly if there are multiple levels of disease as with aortofemoral bypass25,26. Limb salvage rates were frequently presented for critical ischaemia but no report was found describing rates for relief from claudication. The patient may even wish to know the more general picture, namely the prospects of surviving with an intact leg and a patent artery with symptom relieP’ but this was not reported, perhaps because it is too depressing. CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO S
Standardsandstatistks
Numbers at risk and standard error for each interval There was no attempt to show the numbers at risk or the standard error for any interval in 23 articles (18%) making it impossible to determine whether results were valid. This was frequently associated with small numbers being collected over a long time with life-table analysis applied for the entire duration, so that conclusions drawn for late results were undoubtedly invalid. There are no strict rules as to how many patients should be studied but the techniques can be abused if the original sample size is small, particularly if the numbers lost as failures or censored data are large. The standard error is a function of the number entering each interval - a report should be ignored if this does not progressively increase. The committees recommend that data be ignored or at least treated with caution when the standard error exceeds 10%8,11. More than 30-40 procedures in a group would usually be required for data to be valid for medium-term analysis at 12 months. Short-term results from life-table analysis can be useful as small numbers followed for a limited time can be analysed to show which procedures appear to be promising and which are not likely to be effective. However, reliance on life-table analysis to determine true results demands careful follow-up of relatively large numbers. Uncommon diseases or unusual operations need to be reported, but not by life-table analysis if this involves only a few patients; it is better to use some other technique - perhaps by anecdotal descriptions. Numbers with censored data The relative numbers that failed or were withdrawn patent with censored data were not recorded in 82 articles (63%). This was almost invariable in papers that used the product-limit technique. There were differences between journals which may in part be the result of editorial policy. This made it difficult to determine whether there was possible bias due to a high level of censored data, An assumption that patients withdrawn with censored data would have behaved just like those who remain in the study may not be valid. It is as well to be suspicious of a report if the numbers of censored data are not shown by a table or histogram. Completing the study It is rarely possible to review every patient close to the date chosen to finish the study so that it can be difficult to know whether or not they are still under review. An arbitrary decision could be made that a patient is under review if seen within 12 months of the date for completing the study. Lost to follow-up Many patients are lost to follow-up if they are discharged back to their referring practitioners or move CARLllOVASCULAR SURGERY OCTOBER 199s VOL 3 NO S
forevaluating vascularinte~er~ii'i~~n:h: A. Myers
far away. They may, however, simply elect not to return, either becausethey are completely satisfied with the result, but perhaps more often because they have had a bad result and no longer wish to expose themselves to further ineffectual treatment. An excess who withdraw with bad results will lead ro life-table analysis calculating artificially high successrates. In a study of femorodistal bypass grafts”, the author found that withdrawal rates were 2-6%/annum for loss to follow-up. It is best to doubt long-term conclusions if the proportion lost to follow-up is too high - perhaps more than lo-lS%lannum. Death Death with a patent artery is a frequenr reason for censored data. Patients who die are probably different to those who survive - in particular, they are likely to have more severe atherosclerosis. Patency rates for femorodistal vein bypass grafts performed in the author’s unit were calculated before and after removing patients who died with patent grafts”. Patients in the censored group who died were significantly more likely to have had a past clinical history of myocardial ischaemia and to have presented with critical ischaemia. Their removal from the analysis resulted in calculated late patency rates being almost 10% lower than the initial calculation for all grafts. Amputation This is a difficult problem for any form of intervention. How is the endpoint defined in a patient with gangrene due to superficial femoral artery diseasewho is noted to have spurting tibia1 arteries distal to a technically successful femoral bypass or balloon dilatation at the time of below-knee amputation? Obvioudy this is a clinical failure, but some authors2’ reasonably considered that this should be censored data for reports of technical outcome. Standard errors less than 10% The graphs failed to show the point at which the standard error exceeded 10% in 50 articles (38%). There were at least five studies in which subgroups of less than 25 were analysed for periods of 5-10 years in spite of the standard error exceeding 10% within a few months; log-rank testing then demonstrated ‘significant’ differences between groups which probably would not have been found if the follow-up had been terminated earlier, raising the possibility of a type I error. This recommendation was commonly ignored so that results for a small favoured few were drawn out for many years, which is either naive or mischievous. It should be editorial policy to insist that the numbers be shown in a table or that standard errors be shown in a graph, and that prolonged follow-up of small numbers should be discouraged. A long flat distal segment or a 459
Standards andstatkti~s for evaluating vascular intervention: K A. Myers
sudden terminal dip in the graph indicates that this part at least should be ignored. Comparison of groups by a log-rank test There were several papers in which subgroups were compared but with no explanation as to how this was performed. Subgroups appeared to have been compared by statistical techniques other than those described by the committees*‘l’ in 19 articles (15%), usually by separate univariate analysis performed at each interval which is not valid. However, using life-table analysis to compare subgroups can lead to problems. Even if the total group is large, one subgroup is likely to be appreciably smaller than the other so that a test to compare the two can be unreliable. A significant difference between groups does not mean that it is directly caused by the covariate; late patency rates may be lower in diabetic patients than in non-diabetics but this could be becausethey have more severe ischaemia and more extensive distal disease requiring more difficult operations. Confounding as a result of interdependence between multiple covariates is the reason why univariate techniques should be applied to simply describe outcome rather than to interpret reasons for results, and why multivariate techniques are useful.
Follow-up data from the hospital records, general practitioners, or the patients or their relatives are usually unreliable. Even if good long-term data are obtained, they may be passed on to a statistician who usually does not comprehend the implications of the results. Life-table calculations are tedious so that they may be imposed on inexperienced junior staff who do not understand the methods. Demands for a new paper for the ever-impending next meeting may lead to dredging out another group for hasty analysis before the inexorable deadline. Preparing graphs may require talents from lay staff who may not fully appreciate the statistical requirements - this can usually be detected by presentations using gaudy multicoloured slides. The definitions can be confusing but the techniques for survival analysis and their illustration are not difficult and should always be the personal responsibility of the presenting author. Whatever the reason, authors, reviewers and editors should look to limit life-table analysis to studies in which there are enough patients followed up adequately for a sufficient length of time. If possible, the results should be presented in one of the recommended fashions which must be fully explained and adhered. Authors should use other techniques to present their results if this is not possible.
References Conclusions This review independently mirrors anxieties expressed by Underwood and colleagues’ more than 10 years ago. Many of the vascular committees’*‘rr subsequent recommendations flowed from the use and abuse of life-table analysis. However, problems clearly persist. Many practitioners have become cynical about reported results which do not appear to reflect their own experience. There were reports with a modest outcome in keeping with the experience of many in which all recommendations by the committees were followed perfectly15F29.There were other reports where life-table analysis showed excellent results for various methods used to manage vascular disease. Many appeared to reflect the expertise of the doctors and efficacy of the chosen management. Unfortunately, others appeared to disregard the recommendations casting doubts on their conclusions. There are many possible reasons why performance is poor. The larger the unit, the more patients there are to follow up and the more difficult this becomes. The better the unit, the more likely it is to attract patients from far away who will go home after treatment and be lost to follow-up. Few units have developed good audit systems. Many contemporary prospective cardiovascular trials spend considerable funds developing systemsto track down patients, even employing private detective agencies - no vascular unit could justify such expense. 460
1. 2.
3.
4.
5. 6. 7. 8.
9.
10.
Lee ET. Statistical Methods for Survival Data Analysis. Belmont, California: Lifetime Learnine Publications. 1980. Benedetti J, Yuen K, Young L”. Life tables and survivor functions. In: Dixon WJ, ed. BMDP Statistical Software. Los Angeles: University of California Press, 1990: 739-68. Peto R, Pike MC, Armitage P et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I Introduction and design. BY ] Cancer 1976; 34: 585-612. Peco R, Pike MC, Armitage P et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II Analysis and examples. BY J Cancer 1977; 35: l-39. Cox DR. Regression models and life-tables. /ournal ofthe Royal Statistical Society 1972; 34: 187-220. Hopkins A. Survival analysis with covariates. In: Dixon WJ, ed. BMDP Statistical Software. Los Aneeles: Universitv of California Press, 1990: 769’-806. ” Underwood CJ, Faragher EB, Charlesworth D. The uses and abuses of life-table methods in vascular surgery. BY] Surg 1984; 71: 495-8. Rutherford RB, Flanigan DP, Gupta SK et al. Suggested standards for reports dealing with lower extremity ischaemia. Prepared by the Ad Hoc Committee on Reporting Standards, Society for Vascular Surgery/North American Chapter, International Society for Cardiovascular Surgery. ] Vast Surg 1986; 4: 80-94. Spies JB, Bakal CW, Burke DR et al. Guidelines for percutaneous transluminal angioplasty. Standards of Practice Committee of the Society of Cardiovacular and Interventional Radiology. Journal of Vascular and Interventional Radiology 1990; 1: S-IS. Rutherford RB, Becker GJ. Standards for evaluating and reporting the results of surgical and percutaneous therapy for peripheral arterial disease. Journal of Vascular and Inter-
CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO S
Standardsandstatfforevaluating
11.
12. 13.
14. 15.
16. 17.
18. 19.
20.
21.
ventional Radiology 1991; 2: 169-74. Ahn SS, Rutherford RB, Becker GJ et al. Reporting standards for lower extremity arterial endovascular procedures. 1 Vusc Surg 1993; 17: 1103-7. Moore WS, Mohr JP, Najafi H et al. Carotid endarterectomy: practice guidelines. ] Vusc Surg 1992; 15: 469-79. Johnston KW, Rutherford RB, Tilson MD et al. Suggested standards for reporting on arterial aneurysms. J Vusc Surg 1991; 13: 452-8. Porter JM, Rutherford RB, Clagett GP et al. Reporting standards m venous disease. I Vusc Surg 1988; 8: 172-8 1. Donaldson MC, Whittemore AD, Mannick JA. Further experience with an all-autogenous tissue policy for infrainguinal reconstruction. J Vusc Surg 1993; 18: 41-8. European Working Group on critical leg ischaemia. Second European consensus document on chronic critical leg ischaemia. Eur J Vusc Surg 1992; b(Suppl A): l-28. Thompson MM, Sayers RD, Varty K, Reid A, London NJM, Bell PRF. Chrome critical leg ischaemia must be redefined. Eur J Vast Surg 1993; 7: 420-6. Strandness DE, Carter SA. Outcome criteria in patients with peripheral arterial disease. Ann Vusc Surg 1993; 7: 491-6. Myers KA, Fuller JA, Scott DF, Devine TJ, Denton MJ, Chan A. Multivariate Cox regression analysis of covariates for patency rates after femorodistal vein bypass grafting. Ann Vusc Surg 1993; 7: 262-L). Ahn SS, Eton D, Yeatman LR, Deutsch L-S, Moore WS. Intraoperative peripheral rotary atherectomy: early and late clinical results. Ann Vast Sure 1992: 6: 272-80. The Collaboranve Rotablat& Atherectomy Group (CRAG).
CARDIOVASCULAR SURGERY OCTOBER 1995 VOL 3 NO 5
22.
23.
24. 25.
26.
27.
28.
29.
vascular'i~ten/pntirin: K. .hMyefi
Peripheral atherectomy with the rotablator: a rnult~cenrrr report. J Vusc Surg 1994; 19: 509-15. Prime HT, Myers KA, Matthews PG, Zeng C. Early postoperative ankie pressure indices help predict late patency rates after femoro-distal vein bypass grafting. ! Gzrdiorww Surg (Tormo) 1992; 33: 426-31. Grigg MJ, Wolfe JHN, Tovar A, Nicolaides AN. The reliability of duplex derived haemodynamic measurements m the assessment of femoro-distal grafts. EurJ VNSCSurg 1988; 2: 177-81. Harris Pi.. Follow-up after reconstructwe Jrtcrial surgery. EUY I Vast Surg 1991; 5: 369-7.3. Hill DA, McGrath MA, Lord RSA. The effect of superficial femoral artery occlusion on the outcome of aortofemoral bypass for intermittent claudication. Surgery 1980: 87: 133- 6. King RB, Myers KA, Scott DF, Devine 1’1. ‘The choice ot operation in aortoiliac reconstructions for intcrmment claudication. World J Surg 1983; 7: 334-9. Myers KA, Scott DF, Devine TJ. Reversed autogenous vein grafts for occlusive arterial disease in the lower limb. In: Greenhalgh RM, ed. Indications in Vascular Surgery Philadelphia: WB Saunders, 1988: 225-48. Wagner WH, Levin PM, Treiman RL, Cossmdn DV, Foran RF, Cophen JL. Early results of infrainguinal arterial reconstruction with a modified biological conduit. Am C’&rsr Burg 1992; 6: 325-33. Harrington ME, Harrington EB, Schanzer H. Jacobson JH, Haimov M. The dorsalis pedis bypass - moderate success in difficult situations. J Vssc Surg 1992, 1’7, 409.. 16.
Paper accepted 30 January 199.5
461