- Email: [email protected]
Ruptured aneurysms and the International Subarachnoid Aneurysm Trial (ISAT): What is known and what remains to be questioned
Neurochirurgie 58 (2012) 103–108
Disponible en ligne sur
www.sciencedirect.com
Report 2012: Intracranial aneurysms: clips or coils
Ruptured aneurysms and the International Subarachnoid Aneurysm Trial (ISAT): What is known and what remains to be questioned J. Raymond a,∗ , M. Kotowski b , T.E. Darsaut a,c , A.J. Molyneux d , R.S. Kerr d a
Centre hospitalier de l’université de Montréal, Notre-Dame Hospital, Department of Radiology, 1560, Sherbrooke est, Pavillon Simard, suite Z12909 Montreal, Quebec, Canada Département de neurochirurgie, centre hospitalier universitaire Vaudois, Lausanne, Switzerland Department of Surgery, Division of Neurosurgery, Mackenzie Health Sciences Centre, University of Alberta Hospital, Edmonton, Alberta, Canada d Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom b c
a r t i c l e
i n f o
Article history: Received 27 February 2012 Accepted 27 February 2012 Keyword: Intracranial aneurysm
a b s t r a c t Background and purpose. – The International Subarachnoid Aneurysm Trial (ISAT) was a major study comparing clipping and coiling of ruptured intracranial aneurysms. The trial provided answers that were the subject of multiple letters and editorial comments. Methods. – We review the most common critiques that were published in major neurosurgical journals, trying to identify what aspects of ISAT could have been improved, and what questions remain to be studied. Results. – Frequent critiques include imprecise selection criteria, lack of requirement for the technical proficiency of surgical participants, lack of angiographic controls of surgically treated patients, and poorly chosen primary endpoints. Other issues that are often raised are the poor recruitment rate which endangered generalization of results, the excessive delays in performing clipping which led to rebleeding episodes, lack of blinding in the assessment of outcomes, and subjective questionnaires. Nonetheless, the design of the study as a pragmatic trial was appropriate, as was the choice of the primary endpoint. The overall trial results could not be verified for all subgroups of interest, and currently many patients who would not have been included in ISAT are treated by coiling, while other patients are treated by clipping despite ISAT results. Hence a new trial, complementary to the original ISAT, may now be indicated. Conclusion. – ISAT was well-designed and conducted, but many questions remain. They should be addressed by a new trial. © 2012 Published by Elsevier Masson SAS.
A man cannot learn what he thinks he already knows Epictetus With the introduction of detachable coils for the treatment of aneurysms, approved by the US Food and Drug Administration in 1995, selective coil detachment gave the operator unprecedented control, beyond what could be achieved with previous endovascular tools such as detachable balloons and free coils (Higashida et al., 1990; Casasco et al., 1993). Endovascular coiling became widely used in patients with ruptured and unruptured intracranial aneurysms, without evidence of superiority over the accepted standard treatment (Zacharia et al., 2011). The International Subarachnoid Aneurysm Trial (ISAT) was a turning point in modern neurosurgical history (Molyneux et al., 2002). The trial showed that for 2143 aneurismal subarachnoid hemorrhage (SAH)
Abbreviations: mRS, Modified Rankin score; WFNS, World Federation of Neurological Surgeons; MCA, Middle cerebral artery; EVT, Endovascular therapy; PICA, Posterior inferior cerebellar artery; SCA, Superior cerebellar artery. ∗ Corresponding author. E-mail address: [email protected] (J. Raymond). 0028-3770/$ – see front matter © 2012 Published by Elsevier Masson SAS. doi:10.1016/j.neuchi.2012.02.020
patients eligible for both surgery and endovascular coiling, randomised allocation to coiling was associated with better 1-year clinical outcomes, defined as survival without dependency. In 2005 (Molyneux et al., 2005), ISAT reported the 1-year outcomes for 1063 of 1073 patients allocated to endovascular coiling, and 1055 of 1070 patients allocated to surgical clipping: 250 patients (23.5%) allocated to endovascular treatment were dead or dependent at 1 year, compared with 326 patients (30.9%) allocated to neurosurgery, an absolute risk reduction of 7.4% (95% CI 3.6–11.2, p = 0.0001). Long-term results of ISAT patients were published in 2009 (Molyneux et al., 2009): The risk of death at 5 years was significantly lower in the coiling group than in the clipping group (relative risk 0.77, 95% CI 0.61–0.98; p = 0.03), but the proportion of survivors at 5 years who were independent did not differ between the two groups: endovascular 83% (626 of 755) and neurosurgical 82% (584 of 713). Unsurprising for a trial that significantly impacted clinical practice, each new publication of results from ISAT was followed by numerous editorials and comments, summarized in Table 1. We will review the most common criticisms of ISAT, trying to identify what could have been done better, and what remains unknown,
104
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
Table 1 Common critiques of ISAT. Design Selection criteria too wide and imprecise Deficient selection criteria for centres Lack of requirements for the proficiency of surgical participants
Lack of angiographic control of clipped patients Inappropriate primary endpoint Un-blinded adjudication of outcome and subjective questionnaires Implementation Cherry picking of participants
Imbalance of study populationa Surgical delays Poor overall outcomes Poor management of potential confoundersb Large number of ISAT participants entered by small-volume hospitals Analyses Intent to treat analyses Timing of endpoints Subgroup analyses Results Limited generalizabilityc
Inappropriate or missing conclusionsd Lack of evidence for efficacy of EVT
(Ausman, 2003; Britz et al., 2003; Harbaugh, 2003; Barrow, 2004; Sade and Mohr, 2004; Britz, 2005; Heros, 2008a,b; Figueiredo, 2010) (Ausman, 2003; Britz et al., 2003; Harbaugh et al., 2003; Kirkpatrick et al., 2003; Lindsay, 2003; Barrow, 2004; Sade and Mohr, 2004; Britz, 2005) (Heros et al., 2002; Ausman, 2003; Britz et al., 2003; Harbaugh, 2003; Batjer, 2003; Kirkpatrick et al., 2003; Solomon, 2003; ter Brugge, 2003; Barrow, 2004; Hernesniemi and Koivisto, 2004; Maurice-Williams, 2004; Sade and Mohr, 2004; Britz, 2005; Mitha and Ogilvy, 2005; Ausman, 2008; Heros, 2008a,b) (Ausman, 2003; Britz et al., 2003; Debrun, 2003; Maurice-Williams, 2004) (Harbaugh, 2003; Batjer, 2003; Leung et al., 2003; Lindsay, 2003) (Kirkpatrick et al., 2003; Lindsay, 2003; Sade and Mohr, 2004)
(Heros et al., 2002; Britz et al., 2003; Derdeyn et al., 2003; Harbaugh, 2003; Batjer, 2003; Kirkpatrick et al., 2003; Kobayashi, 2003; Lindsay, 2003; Raabe et al., 2003; Barrow, 2004; Maurice-Williams, 2004; Sade and Mohr, 2004; Britz, 2005; Mitha and Ogilvy, 2005; Laitt, 2007; Ausman, 2008; Heros, 2008a,b; Mocco and Hopkins, 2008) (Ausman, 2003; Mitha and Ogilvy, 2005) (Ausman, 2003; Lindsay, 2003; Britz, 2005; Tait et al., 2007; Figueiredo, 2010) (Barrow, 2004; Sade and Mohr, 2004) (Ausman, 2003; Debrun, 2003) (Britz et al., 2003; Barrow, 2004)
(Javadpour, 2007; Tait et al., 2007; Heros, 2008a,b) (Harbaugh et al., 2003; Mohr, 2003; Barrow, 2004; Tait et al., 2007) (Mitha and Ogilvy, 2005; Carter, 2008) (Heros et al., 2002; Ausman, 2003; Britz et al., 2003; Derdeyn et al., 2003; Harbaugh, 2003; Harbaugh et al., 2003; Batjer, 2003; Kobayashi, 2003; Lindsay, 2003; Raabe et al., 2003; Solomon, 2003; ter Brugge, 2003; Barrow, 2004; Maurice-Williams, 2004; Britz, 2005; Laitt, 2007; Ausman, 2008; Heros, 2008a,b) (Heros et al., 2002; Britz et al., 2003; Hernesniemi and Koivisto, 2004; Britz, 2005) (Heros et al., 2002; Debrun, 2003; Harbaugh, 2003; Batjer, 2003; Kobayashi, 2003; Leung et al., 2003; Raabe et al., 2003; Maurice-Williams, 2004; Sade and Mohr, 2004; Tait et al., 2007)
a
e.g. due to location of aneurysm (50% ACA RIAs). e.g. Vasospasm. c e.g. non-US study. d e.g. coiling is not superior if clipping would have been performed in high-volume centres, clipping and coiling are complementary techniques with particular strengths and weaknesses for individual patients. b
almost 10 years after the trial was completed. We will then propose a rationale for a new trial. 1. The design of ISAT ISAT was designed as a pragmatic trial. Design issues frequently mentioned in editorial comments and letters include imprecise selection criteria, lack of requirements for the proficiency of surgical participants, lack of angiographic controls of surgically treated patients, and poorly chosen primary endpoints. Design issues can be discussed from two different viewpoints. 2. The scientific point of view The complaints regarding vague selection criteria, wide inclusion of interventionists and surgeons of varying proficiency, absence of angiographic controls for clipped patients, and the choice of the mRS outcome scale stem from a failure to distinguish the respective roles of explanatory and pragmatic trials in clinical research (Sackett, 2006). On one hand, explanatory trials are most appropriate to answer the question: Can this treatment work in ideal circumstances? Ideal here means strict and precise selection criteria, closely monitored compliance to a precisely defined protocol, a strict selection of expert surgeons, tests and visits that may
exceed normal care, and that may be done exclusively for research purposes, with end-points that are causally related to the biological mechanism that is being explored. When an explanatory trial shows no clear benefit from treatment, it is sensible to abandon this treatment. However, an explanatory trial showing a clear benefit from the tightly controlled delivery of a new treatment cannot be used to infer that it will benefit patients in general; hence it remains ambiguous whether an explanatory trial with a positive result will apply to the real world. On the other hand, a ‘management’ or ‘pragmatic trial’ is the most appropriate design to answer the question: Does this therapy work under usual circumstances? This type of trial calls for wide selection criteria, normal or typical patient compliance and technical proficiency, routine patient care, with tests and visits that are part of established clinical routine, and outcomes that are general and clinically pertinent. A positive pragmatic trial result means that it is clearly worthwhile to adopt the treatment, while a negative trial is ambiguous: might treatment still have been found valuable if patients had been more carefully selected, if physicians had been better trained, or protocols more precisely defined? Given that endovascular treatment was already beginning to replace surgical clipping in many centres, an explanatory trial would not have properly addressed the right question. The question ISAT proposed was: is coiling superior to clipping in usual
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
circumstances, with normal patient care? This was the question most relevant to practising clinicians, and hence, from a scientific point of view, the choice of a pragmatic trial was appropriate. 3. The ethical, societal and patient care point of view According to some, clinical research experiments are foreign to patient care; research should be limited to a select small group of participants; maximum information will be extracted from this altruistic group to infer conclusions regarding how best to manage with reasonable certainty the remaining patients that can then avoid being “used” as research subjects. This approach cannot be appropriate because even though care is delivered by well-intentioned physicians, it leaves the majority of patients vulnerable to biased opinions, learning curves, and conflicts of interests. In this paradigm, while a small percentage of research subjects are told the truth: that an alternative, potentially better treatment exists, and that both treatments could cause benefit or harm, the great majority of patients are led to believe that we physicians already know which treatment is best for them. Confronted with this type of clinical dilemma, the honest, most ethical way to act in the absence of reliable evidence of superiority of either treatment, is to offer participation in a trial: the best treatment is whatever the trial allocates (Ashcroft, 2000), at least until the trial provides a clear answer. The scientific and ethical points of view converge when we try to interpret results from ISAT: if coiling is shown to be better than clipping, to whom do results apply? The quick answer is to patients, centres and professionals that were included in the trial. We conclude that the pragmatic design of ISAT, with wide inclusion criteria, normal care and expertise for centres and surgeons, lack of extraneous tests and visits (such as requiring angiograms for clipped patients in centres where these were not part of standard of care), was appropriate from ethical and scientific viewpoints. Another commonly voiced criticism was the choice of the primary endpoint: functional status at one year. This judiciously chosen endpoint allowed sufficient time for rebleeds from ineffective coiling; past observational studies had shown very low rate of rebleeding after one year even for untreated ruptured aneurysms (Nishioka, 1966). This choice also permits time for surgical patients to recover from transient disabilities, a fact demonstrated by the decreased morbidity when 1 year poor outcomes (30.9%) were compared to those found at 2 months (36.9%) (ISAT, 2002). A primary endpoint collected at a later time would run the risk of missing the true benefit of one treatment over another (illustrated by the truism that with a long enough period of observation, all treatments give the same outcome: everyone dies). In addition, with a primary endpoint collected at a later time, the number of patients lost to follow-up can only increases, and no one knows how to manage missing data (Sackett, 2006). Finally, the fact that the p value was not significant for each mRS category, or for various combinations of categories (Harbaugh et al., 2003), is not surprising, and cannot be required of any trial, for the same reasons that benefits cannot usually be shown for all imaginable subgroups, a problem we will examine later. 4. The implementation of ISAT If we accept that ISAT was well designed, the immediate next question is: was it well-conducted and implemented? In the real, difficult world of practicalities, are the demographics of participants ‘representative’ of the population of patients to whom we wish to apply results; is the way the protocol was followed or the data collected appropriate to provide meaningful results and reliable conclusions?
105
4.1. Cherry picking of trial participants A frequent criticism was that ISAT recruited only a small portion of the ‘universe’ of patients with ruptured aneurysms, that many patients were excluded from randomisation, and that exclusions varied from one centre to another, suggesting that surgeons and interventionists carefully ‘cherry-picked’ patients to include in order to portray coiling in a good light. Paradoxically, the critics wanting more precise, restrictive inclusion criteria are the same ones that bemoan that participants were selected, and not representative of the ‘universe’ of patients. One inevitable reality of both patient care and clinical research is that it is impossible (and ill-advised) to coerce clinicians to treat a patient when they think they cannot offer safe, and hopefully effective treatment. In ISAT, patient recruitment was conditional, requiring both interventionist and surgeon to feel they could offer a ‘good’ alternative treatment to the patient. Of course, this evaluation has to take local expertise, beliefs and circumstances into account. Since variety is an essential part of the real world, there is no solution to this problem, except for repeatedly insisting that we ought to admit the uncertainty concerns far more patients in our daily clinical practice and enrol more patients into those trials, to decrease this kind of selection. Heterogeneity has a bright side: we must remember that diversity provides strength to a positive pragmatic trial like ISAT: one treatment was shown to be superior despite the uncontrollable variety of patients, beliefs and expertise that were included in the final results. 5. Surgical delays and expertise Other practical aspects that were the objects of criticism were the longer delay to surgical treatment after randomisation, that neurosurgeons in the trial were general neurosurgeons while interventionists were specialists, and that overall results show that patient care was suboptimal in all participating centres. Surgical delays in ISAT were associated with more frequent re-bleeding events between randomisation and treatment in the clipping group (28 events resulting in 19 deaths) as compared to the coiling group (17 events with seven deaths) (Molyneux et al., 2005). Ideally, the time between randomisation and treatment for either arm should always be minimized. Excluding these patients from analysis, as proposed by some (Bakker et al., 2010), could be considered a ‘sensitivity analysis’, but once the events have occurred, no one knows what might have happened had these patients not re-bled. Excluding these patients from analysis is an exercise in speculation, no more justified than if we now excluded rebleeds after coiling on the speculative basis that newer catheters and coils ‘would have produced better immediate occlusions, had they been available at the time’. Furthermore, in the real world, it is possible that the average delay for surgery really is slightly longer than for endovascular treatment. The claim that neurosurgeons involved in ISAT were less expert than interventionists is without foundation. This contention that the ‘losing arm’ of the study had worse technicians is: • controlled by using multi-centre studies; • can always be argued either way, depending on overall study results; • remains untestable and thus irrelevant. 6. Analyses and interpretation Multiple secondary analyses were performed, discussed, and some even published using the ISAT data. The ISAT investigators
106
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
chose to provide the community with enough data to form their own impressions regarding the reliability of the evidence. This praiseworthy gesture allowed people to engage in all sorts of speculation regarding what the results could be if this or that particular subgroup was considered separately, or if non-significant trends were projected over a lifetime (Mitchell et al., 2008; Ryttlefors et al., 2008). We will restrict the discussion to the published primary intent-to-treat analyses, and comment on both the importance of adhering to pre-defined hypotheses and endpoints, and the dangers of subgroup analyses (Naggara et al., 2011). Given ISAT’s pragmatic research question, the intent-to-treat analysis of the primary hypothesis regarding the pre-defined outcome measure (mRS > two at 1 year) was not only appropriate, but mandatory. Had the mRS scale been divided in some other way (Harbaugh et al., 2003), or had outcomes been compared at some other time point (Bakker et al., 2010) are interesting speculative explorations, which promise to yield different, non-significant, or even contrary conclusions. Had the ISAT authors used such a procedure to claim superiority of coiling, they would have received the appropriate indignation. These explorations only serve to illustrate the need to rigorously adhere to pre-defined definitions of endpoints and hypotheses in the interpretation of trial results: the statistical tools we use cannot tolerate the multiplicity and variety of questions and reformulations; if this rule is transgressed, anything goes. Recruitment in ISAT was stopped in early 2002 because an independent monitoring committee judged that differences in the primary clinical outcome between patients treated with clip ligation and those treated with coil embolization were too large to continue with the uncertainty compatible with randomisation. After trial completion, we now have a generalization, which is not necessarily applicable to all individuals: we now know that when coiling and clipping are both valid options for a particular patient, coiling leads to better outcomes at 1 year. Now was this verified for all patients, aneurysms, and centre characteristics, such as sex, age, race, location, size, experience? Of course not. Clinical diversity makes sure this could never be achieved, and one has to expect heterogeneous results, some apparently increasing, others decreasing the effects of treatments, when dissecting the data post hoc. A study powered to provide evidence for so many “kinds of patients” would lead to an eternal trial, the evidence becoming non-convincing each time the data is split into other “interesting” subgroups. We must be realistic regarding what kind of knowledge is obtainable. At best, unplanned uncontrolled subgroup findings can only be used as hypotheses for future trials. Until such trials are performed, “the answer to a RCT that does not confirm one’s beliefs is not to conduct several sub-analyses until one can see what one believes. Rather the answer is to re-examine one’s beliefs carefully.” (Altman and Bland, 1995; Brookes et al., 2001; Collins and MacMahon, 2001; Molyneux et al., 2005; Schulz and Grimes, 2005; Cowan et al., 2007). A reasonable interpretation of ISAT is that patients with small, ruptured anterior circulation aneurysm should be treated with endovascular coiling.
7. What remains to be questioned and hypotheses for a new trial In real world practice, coiling is now being offered to types of patients that were not studied in ISAT (Gnanalingham et al., 2006). Clinicians often claim that catheter and coil technology has significantly improved make coiling applicable to many more patients than what was originally possible in ISAT. It is also true, however, that many so-called advances have not been rigorously tested, and some have significant drawbacks (i.e.: stents requiring antiplatelet agents). No one knows if these patients would not be better served
Table 2 Patients potentially appropriate for recruitment in ISAT II. Potential higher risks of coiling Lesser efficacy of coiling Certain locations
Weak subgroup evidence
Potential advantage of surgery
Very small aneurysms (< 3 mm) Wide necked aneurysms (> 4 mm) Large aneurysms (≥ 10 mm) MCA aneurysms; pericallosal Posterior circulation (not basilar bifurcation) Patients older than 70 years old Patients younger than 40 years old WFNS 4–5 Non-life threatening cerebral hematoma Multiple accessible aneurysms
with surgical clipping. We also know that many patients continue to be clipped in spite of ISAT, which may or may not be appropriate. For example, no one has proven that large, wide necked aneurysms should be clipped, but given the ‘smaller’ benefit of coiling over clipping in this subgroup, and given the greater risks of angiographic recurrences after coiling large aneurysms (Raymond et al., 2003), many believe that clipping is preferable. Needless to say this line of reasoning is hypothetical and deserves more rigorous evaluation. Therapeutic dilemmas persist for many patients, which cannot be resolved with cases series or epidemiological surveys. Many authors have proposed, in a reconciliatory tone, that ‘Each patient and their aneurysm is different and the decision has to consider the best interests for each patient’, . . . ‘No one treatment option is superior, but rather each has strengths and weaknesses that must be considered when choosing what is best for each patient.’ Such statements are at first glance respectable, but upon further examination, remain empty or even imprudent. With such reasoning, anything goes; no one can ever test a claim or assess treatment results. What is neglected by authors who claim that optimal results are achieved using a case-by-case approach, is that the easiest way to obtain an excellent series is to carefully select patients, leaving all difficult cases to the rival intervention. The goal for physician should not be a perfect personal case series, but better patient outcomes.
8. ISAT II: a new trial Many questions remain regarding choice of coiling or clipping, and many patients are still treated surgically despite ISAT, and all without convincing evidence. The essence of scientific medicine is to question the hypotheses behind our as yet not validated actions. Another trial is indicated. But how would a new trial differ from the initial ISAT? Again we would need a pragmatic trial, which is the best way to offer the best possible care in the presence of uncertainty. A functional primary endpoint at one year would be appropriate. Follow-up tests and visits would be routine. Selection criteria would be few to include a diversity of patients and clinicians, to study the results in the very context in which real care is delivered. To whom would this trial be offered? We can organize our thoughts by looking at the subgroup analyses published in ISAT, 2005, realizing that many correspond to intuitive justifications for our current medical decisions that may not follow general ISAT conclusions, and still need validation. Now is an appropriate time to look at subgroups, as we are looking for hypotheses for another trial! Patients recruited in ISAT II would include those for whom we are tempted to resort to clipping despite ISAT results, or patients we are tempted to coil, but that we could not or did not include in the original formally tested population. These patients and circumstances are summarized in Table 2. They may include circumstances where:
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
• subgroup analyses point to a potential benefit from surgery (patients older than 70 or younger than 40 years old, or patients with WFNS grades 4–5); • cases under-represented in ISAT, perhaps because surgery was felt to be better (i.e.: MCA aneurysms); • when EVT may be more risky (as judged from case series, such as) very small (<3 mm) aneurysms (Nguyen et al., 2008) or less effective (i.e.: large or wide-necked aneurysms); anticipated use of stents (Piotin et al., 2010) or flow diverter; • cases where surgery could offer an immediate advantage (nonlife threatening but large hematomas; multiple aneurysms treatable through the same craniotomy); or certain posterior circulation aneurysms that may not entail the same level of risks as a basilar bifurcation lesion (i.e.: PICA or SCA aneurysms). The conduct of ISAT-II should be integrated into normal patient care. Anytime we wish to practise an unproven intervention or an intervention not proven to be beneficial in certain circumstances such as treating a patient with a ruptured aneurysm for whom the results of ISAT may not apply, recruitment into a trial is indicated (Raymond et al., 2011). When such trials become a part of routine clinical care, we will not only be able to offer impeccable scientific and ethical care to current patients, but we will also begin producing reliable knowledge to guide the management of all patients with ruptured aneurysms. 9. Conclusion The ISAT study was a well-designed and conducted trial, given real-world limitations. Many of the subsequent interpretations, subgroup analyses, and extrapolations of results do not stand up to rigorous scientific evaluation. ISAT only answered one question, and many patients continue to have ruptured aneurysms treated by coiling or clipping without a basis in reliable evidence. We are ready for the next pragmatic trial, complementary to ISAT, which would address the best management of patients with ruptured aneurysms for whom uncertainty remains. Disclosure of interest J.R. and T.E.D. are co-principal investigators of the CURES trial, funded by the Canadian Institute of Health Research (CIHR MOP119554) and the STAT and FIAT trials (non-funded). A.J.M. and R.S.K. were principal investigators of the ISAT trial. M.K. has no conflict of interest to declare. References Altman, D.G., Bland, J.M., 1995. Absence of evidence is not evidence of absence. BMJ 311 (7003), 485. Ashcroft, R., 2000. Giving medicine a fair trial. Trials should not second guess what patients want. BMJ 320 (7251), 1686. Ausman, J.I., 2003. ISAT study: is coiling better than clipping? Surg Neurol 59 (3), 162–165 [discussion 165–173; author reply 173–185]. Ausman, J.I., 2008. The International Subarachnoid Aneurysm Trial II: comparison of clipping vs coiling: key questions. Are the results of the study generalizable? Should clipping be done for patients less than 40 years of age? Surg Neurol 70 (1), 104–107. Bakker, N.A., Metzemaekers, J.D., et al., 2010. International subarachnoid aneurysm trial 2009: endovascular coiling of ruptured intracranial aneurysms has no significant advantage over neurosurgical clipping. Neurosurgery 66 (5), 961–962. Barrow, D.L., 2004. Bad science ISAT: the impact on neurosurgical practice. Clin Neurosurg 51, 126–131. Batjer, H.H., 2003. Surg Neurol 59 (3), 168–169. Britz, G.W., 2005. ISAT trial: coiling or clipping for intracranial aneurysms? Lancet 366 (9488), 783–785. Britz, G.W., Newell, D.W., et al., 2003. The ISAT trial. Lancet 361 (9355), 431–432. Brookes, S.T., Whitley, E., et al., 2001. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess 5 (33), 1–56.
107
Carter, B.S., 2008. ISAT subgroups: is aneurysm clipping better for some patients? Neurosurgery 63 (6), 14. Casasco, A.E., Aymard, A., et al., 1993. Selective endovascular treatment of 71 intracranial aneurysms with platinum coils. J Neurosurg 79 (1), 3–10. Collins, R., MacMahon, S., 2001. Reliable assessment of the effects of treatment on mortality and major morbidity, I: clinical trials. Lancet 357 (9253), 373–380. Cowan J.A.Jr., J., Ziewacz, et al., 2007. Use of endovascular coil embolization and surgical clip occlusion for cerebral artery aneurysms. J Neurosurg 107 (3), 530–535. Debrun, G., 2003. Commentary. Surg Neurol 59 (3), 168. Derdeyn, C.P., Barr, J.D., et al., 2003. The International Subarachnoid Aneurysm Trial (ISAT): a position statement from the Executive Committee of the American Society of Interventional and Therapeutic Neuroradiology and the American Society of Neuroradiology. AJNR Am J Neuroradiol 24 (7), 1404–1408. Figueiredo, E.G., 2010. The ISAT. Again. World Neurosurg 73 (6), 601–603. Gnanalingham, K.K., Apostolopoulos, V., et al., 2006. The impact of the international subarachnoid aneurysm trial (ISAT) on the management of aneurysmal subarachnoid haemorrhage in a neurosurgical unit in the UK. Clin Neurol Neurosurg 108 (2), 117–123. Harbaugh, R.E., 2003. Surg Neurol 59 (3), 165–167. Harbaugh, R.E., Heros, R.C., et al., 2003. More on ISAT. Lancet 361 (9359), 783–784. Hernesniemi, J., Koivisto, T., 2004. Comments on The impact of the International Subarachnoid Aneurysm Treatment Trial (ISAT) on neurosurgical practice. Acta Neurochir (Wien) 146 (2), 203–208. Heros, R.C., Hadley, M.N., Harbaugh, R.E., 2002. Position statement by the AANS, CNS and AANS/CNS section on cerebrovascular surgery. AANS Bull 11 (4.). Heros, R.C., 2008a. International Subarachnoid Aneurysm Trial analysis. J Neurosurg 108 (3), 433–435. Heros, R.C., 2008b. Vascular neurosurgery since the International Subarachnoid Aneurysm Trial. J Neurosurg 109 (6), 989–991 [author reply 990–991]. Higashida, R.T., Halbach, V.V., et al., 1990. Treatment of intracranial aneurysms with preservation of the parent vessel: results of percutaneous balloon embolization in 84 patients. AJNR Am J Neuroradiol 11 (4), 633–640. Javadpour, M., 2007. ISAT: Will the results stand the test of time? Br J Neurosurg 21 (1), 7–8. Kirkpatrick, P.J., Kirollos, R.W., et al., 2003. Lessons to be learnt from the International Subarachnoid Haemorrhage Trial (ISAT). Br J Neurosurg 17 (1), 5–7. Kobayashi, S., 2003. Commentary. Surg Neurol 59 (3), 167–168. Laitt, R.D., 2007. How much can be concluded from ISAT? Br J Neurosurg 21 (1), 9–10. Leung, C.H., Poon, W.S., et al., 2003. The ISAT trial. Lancet 361 (9355), 430–431 [author reply 432]. Lindsay, K.W., 2003. The impact of the International Subarachnoid Aneurysm Treatment Trial (ISAT) on neurosurgical practice. Acta Neurochir (Wien) 145 (2), 97–99. Maurice-Williams, R.S., 2004. Aneurysm surgery after the International Subarachnoid Aneurysm Trial (ISAT). J Neurol Neurosurg Psychiatry 75 (6), 807–808. Mitchell, P., Kerr, R., et al., 2008. Could late rebleeding overturn the superiority of cranial aneurysm coil embolization over clip ligation seen in the International Subarachnoid Aneurysm Trial? J Neurosurg 108 (3), 437–442. Mitha, A.P., Ogilvy, C.S., 2005. ISAT: coiling or clipping for ruptured intracranial aneurysms? Lancet Neurol 4 (12), 791–792. Mocco, J., Hopkins, L.N., 2008. International Subarachnoid Aneurysm Trial analysis. J Neurosurg 108 (3), 436. Mohr, J.P., 2003. The ISAT trial. Lancet 361 (9355), 431 [author reply 432]. Molyneux, A., Kerr, R., et al., 2002. International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet 360 (9342), 1267–1274. Molyneux, A.J., Kerr, R.S., et al., 2009. Risk of recurrent subarachnoid haemorrhage, death, or dependence and standardised mortality ratios after clipping or coiling of an intracranial aneurysm in the International Subarachnoid Aneurysm Trial (ISAT): long-term follow-up. Lancet Neurol 8 (5), 427–433. Molyneux, A.J., Kerr, R.S., et al., 2005. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet 366 (9488), 809–817. Naggara, O., Raymond, J., et al., 2011. The problem of subgroup analyses: an example from a trial on ruptured intracranial aneurysms. AJNR Am J Neuroradiol 32 (4), 633–636. Nguyen, T.N., Raymond, J., et al., 2008. Association of endovascular therapy of very small ruptured aneurysms with higher rates of procedure-related rupture. J Neurosurg 108 (6), 1088–1092. Nishioka, H., 1966. Report on the cooperative study of intracranial aneurysms and subarachnoid hemorrhage. Section VII. I. Evaluation of the conservative management of ruptured intracranial aneurysms. J Neurosurg 25 (5), 574–592. Piotin, M., Blanc, R., et al., 2010. Stent-assisted coiling of intracranial aneurysms: clinical and angiographic results in 216 consecutive aneurysms. Stroke 41 (1), 110–115. Raabe, A., Schmiedek, P., et al., 2003. German Society of Neurosurgery Section on Vascular Neurosurgery: Position Statement on the International Subarachnoid Hemorrhage Trial (ISAT). Zentralblatt fur Neurochirurgie 64 (3), 99–103. Raymond, J., Darsaut, T.E., et al., 2011. A trial on unruptured intracranial aneurysms (the TEAM trial): results, lessons from a failure and the necessity for clinical care trials. Trials 12, 64.
108
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
Raymond, J., Guilbert, F., et al., 2003. Long-term angiographic recurrences after selective endovascular treatment of aneurysms with detachable coils. Stroke 34 (6), 1398–1403. Ryttlefors, M., Enblad, P., et al., 2008. International subarachnoid aneurysm trial of neurosurgical clipping versus endovascular coiling: subgroup analysis of 278 elderly patients. Stroke 39 (10), 2720–2726. Sackett, D. (2006). The Tactics of performing therapeutic trials. Cilical epidemiology: How to do clinical practice research. R. Haynes, Sackett, DR, Guyatt, GH, Tugwell, P. Philadelphia, Lippincott, Williams & Wilkins: p. 66–172. Sade, B., Mohr, G., 2004. Critical appraisal of the International Subarachnoid Aneurysm Trial (ISAT). Neurol India 52 (1), 32–35.
Schulz, K.F., Grimes, D.A., 2005. Multiplicity in randomised trials II: subgroup and interim analyses. Lancet 365 (9471), 1657–1661. Solomon, R., 2003. Commentary. Surg Neurol 59 (3), 173. Tait, M.J., Critchley, G.R., et al., 2007. How much can be concluded from the International Subarachnoid Aneurysm Trial (ISAT)? Br J Neurosurg 21 (1), 3–6. ter Brugge, K.G., 2003. Commentary. Surg Neurol 59 (3), 172–173. Zacharia, B.E., Ducruet, A.F., et al., 2011. Technological advances in the management of unruptured intracranial aneurysms fail to improve outcome in New York state. Stroke 42 (10), 2844–2849.
Disponible en ligne sur
www.sciencedirect.com
Report 2012: Intracranial aneurysms: clips or coils
Ruptured aneurysms and the International Subarachnoid Aneurysm Trial (ISAT): What is known and what remains to be questioned J. Raymond a,∗ , M. Kotowski b , T.E. Darsaut a,c , A.J. Molyneux d , R.S. Kerr d a
Centre hospitalier de l’université de Montréal, Notre-Dame Hospital, Department of Radiology, 1560, Sherbrooke est, Pavillon Simard, suite Z12909 Montreal, Quebec, Canada Département de neurochirurgie, centre hospitalier universitaire Vaudois, Lausanne, Switzerland Department of Surgery, Division of Neurosurgery, Mackenzie Health Sciences Centre, University of Alberta Hospital, Edmonton, Alberta, Canada d Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom b c
a r t i c l e
i n f o
Article history: Received 27 February 2012 Accepted 27 February 2012 Keyword: Intracranial aneurysm
a b s t r a c t Background and purpose. – The International Subarachnoid Aneurysm Trial (ISAT) was a major study comparing clipping and coiling of ruptured intracranial aneurysms. The trial provided answers that were the subject of multiple letters and editorial comments. Methods. – We review the most common critiques that were published in major neurosurgical journals, trying to identify what aspects of ISAT could have been improved, and what questions remain to be studied. Results. – Frequent critiques include imprecise selection criteria, lack of requirement for the technical proficiency of surgical participants, lack of angiographic controls of surgically treated patients, and poorly chosen primary endpoints. Other issues that are often raised are the poor recruitment rate which endangered generalization of results, the excessive delays in performing clipping which led to rebleeding episodes, lack of blinding in the assessment of outcomes, and subjective questionnaires. Nonetheless, the design of the study as a pragmatic trial was appropriate, as was the choice of the primary endpoint. The overall trial results could not be verified for all subgroups of interest, and currently many patients who would not have been included in ISAT are treated by coiling, while other patients are treated by clipping despite ISAT results. Hence a new trial, complementary to the original ISAT, may now be indicated. Conclusion. – ISAT was well-designed and conducted, but many questions remain. They should be addressed by a new trial. © 2012 Published by Elsevier Masson SAS.
A man cannot learn what he thinks he already knows Epictetus With the introduction of detachable coils for the treatment of aneurysms, approved by the US Food and Drug Administration in 1995, selective coil detachment gave the operator unprecedented control, beyond what could be achieved with previous endovascular tools such as detachable balloons and free coils (Higashida et al., 1990; Casasco et al., 1993). Endovascular coiling became widely used in patients with ruptured and unruptured intracranial aneurysms, without evidence of superiority over the accepted standard treatment (Zacharia et al., 2011). The International Subarachnoid Aneurysm Trial (ISAT) was a turning point in modern neurosurgical history (Molyneux et al., 2002). The trial showed that for 2143 aneurismal subarachnoid hemorrhage (SAH)
Abbreviations: mRS, Modified Rankin score; WFNS, World Federation of Neurological Surgeons; MCA, Middle cerebral artery; EVT, Endovascular therapy; PICA, Posterior inferior cerebellar artery; SCA, Superior cerebellar artery. ∗ Corresponding author. E-mail address: [email protected] (J. Raymond). 0028-3770/$ – see front matter © 2012 Published by Elsevier Masson SAS. doi:10.1016/j.neuchi.2012.02.020
patients eligible for both surgery and endovascular coiling, randomised allocation to coiling was associated with better 1-year clinical outcomes, defined as survival without dependency. In 2005 (Molyneux et al., 2005), ISAT reported the 1-year outcomes for 1063 of 1073 patients allocated to endovascular coiling, and 1055 of 1070 patients allocated to surgical clipping: 250 patients (23.5%) allocated to endovascular treatment were dead or dependent at 1 year, compared with 326 patients (30.9%) allocated to neurosurgery, an absolute risk reduction of 7.4% (95% CI 3.6–11.2, p = 0.0001). Long-term results of ISAT patients were published in 2009 (Molyneux et al., 2009): The risk of death at 5 years was significantly lower in the coiling group than in the clipping group (relative risk 0.77, 95% CI 0.61–0.98; p = 0.03), but the proportion of survivors at 5 years who were independent did not differ between the two groups: endovascular 83% (626 of 755) and neurosurgical 82% (584 of 713). Unsurprising for a trial that significantly impacted clinical practice, each new publication of results from ISAT was followed by numerous editorials and comments, summarized in Table 1. We will review the most common criticisms of ISAT, trying to identify what could have been done better, and what remains unknown,
104
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
Table 1 Common critiques of ISAT. Design Selection criteria too wide and imprecise Deficient selection criteria for centres Lack of requirements for the proficiency of surgical participants
Lack of angiographic control of clipped patients Inappropriate primary endpoint Un-blinded adjudication of outcome and subjective questionnaires Implementation Cherry picking of participants
Imbalance of study populationa Surgical delays Poor overall outcomes Poor management of potential confoundersb Large number of ISAT participants entered by small-volume hospitals Analyses Intent to treat analyses Timing of endpoints Subgroup analyses Results Limited generalizabilityc
Inappropriate or missing conclusionsd Lack of evidence for efficacy of EVT
(Ausman, 2003; Britz et al., 2003; Harbaugh, 2003; Barrow, 2004; Sade and Mohr, 2004; Britz, 2005; Heros, 2008a,b; Figueiredo, 2010) (Ausman, 2003; Britz et al., 2003; Harbaugh et al., 2003; Kirkpatrick et al., 2003; Lindsay, 2003; Barrow, 2004; Sade and Mohr, 2004; Britz, 2005) (Heros et al., 2002; Ausman, 2003; Britz et al., 2003; Harbaugh, 2003; Batjer, 2003; Kirkpatrick et al., 2003; Solomon, 2003; ter Brugge, 2003; Barrow, 2004; Hernesniemi and Koivisto, 2004; Maurice-Williams, 2004; Sade and Mohr, 2004; Britz, 2005; Mitha and Ogilvy, 2005; Ausman, 2008; Heros, 2008a,b) (Ausman, 2003; Britz et al., 2003; Debrun, 2003; Maurice-Williams, 2004) (Harbaugh, 2003; Batjer, 2003; Leung et al., 2003; Lindsay, 2003) (Kirkpatrick et al., 2003; Lindsay, 2003; Sade and Mohr, 2004)
(Heros et al., 2002; Britz et al., 2003; Derdeyn et al., 2003; Harbaugh, 2003; Batjer, 2003; Kirkpatrick et al., 2003; Kobayashi, 2003; Lindsay, 2003; Raabe et al., 2003; Barrow, 2004; Maurice-Williams, 2004; Sade and Mohr, 2004; Britz, 2005; Mitha and Ogilvy, 2005; Laitt, 2007; Ausman, 2008; Heros, 2008a,b; Mocco and Hopkins, 2008) (Ausman, 2003; Mitha and Ogilvy, 2005) (Ausman, 2003; Lindsay, 2003; Britz, 2005; Tait et al., 2007; Figueiredo, 2010) (Barrow, 2004; Sade and Mohr, 2004) (Ausman, 2003; Debrun, 2003) (Britz et al., 2003; Barrow, 2004)
(Javadpour, 2007; Tait et al., 2007; Heros, 2008a,b) (Harbaugh et al., 2003; Mohr, 2003; Barrow, 2004; Tait et al., 2007) (Mitha and Ogilvy, 2005; Carter, 2008) (Heros et al., 2002; Ausman, 2003; Britz et al., 2003; Derdeyn et al., 2003; Harbaugh, 2003; Harbaugh et al., 2003; Batjer, 2003; Kobayashi, 2003; Lindsay, 2003; Raabe et al., 2003; Solomon, 2003; ter Brugge, 2003; Barrow, 2004; Maurice-Williams, 2004; Britz, 2005; Laitt, 2007; Ausman, 2008; Heros, 2008a,b) (Heros et al., 2002; Britz et al., 2003; Hernesniemi and Koivisto, 2004; Britz, 2005) (Heros et al., 2002; Debrun, 2003; Harbaugh, 2003; Batjer, 2003; Kobayashi, 2003; Leung et al., 2003; Raabe et al., 2003; Maurice-Williams, 2004; Sade and Mohr, 2004; Tait et al., 2007)
a
e.g. due to location of aneurysm (50% ACA RIAs). e.g. Vasospasm. c e.g. non-US study. d e.g. coiling is not superior if clipping would have been performed in high-volume centres, clipping and coiling are complementary techniques with particular strengths and weaknesses for individual patients. b
almost 10 years after the trial was completed. We will then propose a rationale for a new trial. 1. The design of ISAT ISAT was designed as a pragmatic trial. Design issues frequently mentioned in editorial comments and letters include imprecise selection criteria, lack of requirements for the proficiency of surgical participants, lack of angiographic controls of surgically treated patients, and poorly chosen primary endpoints. Design issues can be discussed from two different viewpoints. 2. The scientific point of view The complaints regarding vague selection criteria, wide inclusion of interventionists and surgeons of varying proficiency, absence of angiographic controls for clipped patients, and the choice of the mRS outcome scale stem from a failure to distinguish the respective roles of explanatory and pragmatic trials in clinical research (Sackett, 2006). On one hand, explanatory trials are most appropriate to answer the question: Can this treatment work in ideal circumstances? Ideal here means strict and precise selection criteria, closely monitored compliance to a precisely defined protocol, a strict selection of expert surgeons, tests and visits that may
exceed normal care, and that may be done exclusively for research purposes, with end-points that are causally related to the biological mechanism that is being explored. When an explanatory trial shows no clear benefit from treatment, it is sensible to abandon this treatment. However, an explanatory trial showing a clear benefit from the tightly controlled delivery of a new treatment cannot be used to infer that it will benefit patients in general; hence it remains ambiguous whether an explanatory trial with a positive result will apply to the real world. On the other hand, a ‘management’ or ‘pragmatic trial’ is the most appropriate design to answer the question: Does this therapy work under usual circumstances? This type of trial calls for wide selection criteria, normal or typical patient compliance and technical proficiency, routine patient care, with tests and visits that are part of established clinical routine, and outcomes that are general and clinically pertinent. A positive pragmatic trial result means that it is clearly worthwhile to adopt the treatment, while a negative trial is ambiguous: might treatment still have been found valuable if patients had been more carefully selected, if physicians had been better trained, or protocols more precisely defined? Given that endovascular treatment was already beginning to replace surgical clipping in many centres, an explanatory trial would not have properly addressed the right question. The question ISAT proposed was: is coiling superior to clipping in usual
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
circumstances, with normal patient care? This was the question most relevant to practising clinicians, and hence, from a scientific point of view, the choice of a pragmatic trial was appropriate. 3. The ethical, societal and patient care point of view According to some, clinical research experiments are foreign to patient care; research should be limited to a select small group of participants; maximum information will be extracted from this altruistic group to infer conclusions regarding how best to manage with reasonable certainty the remaining patients that can then avoid being “used” as research subjects. This approach cannot be appropriate because even though care is delivered by well-intentioned physicians, it leaves the majority of patients vulnerable to biased opinions, learning curves, and conflicts of interests. In this paradigm, while a small percentage of research subjects are told the truth: that an alternative, potentially better treatment exists, and that both treatments could cause benefit or harm, the great majority of patients are led to believe that we physicians already know which treatment is best for them. Confronted with this type of clinical dilemma, the honest, most ethical way to act in the absence of reliable evidence of superiority of either treatment, is to offer participation in a trial: the best treatment is whatever the trial allocates (Ashcroft, 2000), at least until the trial provides a clear answer. The scientific and ethical points of view converge when we try to interpret results from ISAT: if coiling is shown to be better than clipping, to whom do results apply? The quick answer is to patients, centres and professionals that were included in the trial. We conclude that the pragmatic design of ISAT, with wide inclusion criteria, normal care and expertise for centres and surgeons, lack of extraneous tests and visits (such as requiring angiograms for clipped patients in centres where these were not part of standard of care), was appropriate from ethical and scientific viewpoints. Another commonly voiced criticism was the choice of the primary endpoint: functional status at one year. This judiciously chosen endpoint allowed sufficient time for rebleeds from ineffective coiling; past observational studies had shown very low rate of rebleeding after one year even for untreated ruptured aneurysms (Nishioka, 1966). This choice also permits time for surgical patients to recover from transient disabilities, a fact demonstrated by the decreased morbidity when 1 year poor outcomes (30.9%) were compared to those found at 2 months (36.9%) (ISAT, 2002). A primary endpoint collected at a later time would run the risk of missing the true benefit of one treatment over another (illustrated by the truism that with a long enough period of observation, all treatments give the same outcome: everyone dies). In addition, with a primary endpoint collected at a later time, the number of patients lost to follow-up can only increases, and no one knows how to manage missing data (Sackett, 2006). Finally, the fact that the p value was not significant for each mRS category, or for various combinations of categories (Harbaugh et al., 2003), is not surprising, and cannot be required of any trial, for the same reasons that benefits cannot usually be shown for all imaginable subgroups, a problem we will examine later. 4. The implementation of ISAT If we accept that ISAT was well designed, the immediate next question is: was it well-conducted and implemented? In the real, difficult world of practicalities, are the demographics of participants ‘representative’ of the population of patients to whom we wish to apply results; is the way the protocol was followed or the data collected appropriate to provide meaningful results and reliable conclusions?
105
4.1. Cherry picking of trial participants A frequent criticism was that ISAT recruited only a small portion of the ‘universe’ of patients with ruptured aneurysms, that many patients were excluded from randomisation, and that exclusions varied from one centre to another, suggesting that surgeons and interventionists carefully ‘cherry-picked’ patients to include in order to portray coiling in a good light. Paradoxically, the critics wanting more precise, restrictive inclusion criteria are the same ones that bemoan that participants were selected, and not representative of the ‘universe’ of patients. One inevitable reality of both patient care and clinical research is that it is impossible (and ill-advised) to coerce clinicians to treat a patient when they think they cannot offer safe, and hopefully effective treatment. In ISAT, patient recruitment was conditional, requiring both interventionist and surgeon to feel they could offer a ‘good’ alternative treatment to the patient. Of course, this evaluation has to take local expertise, beliefs and circumstances into account. Since variety is an essential part of the real world, there is no solution to this problem, except for repeatedly insisting that we ought to admit the uncertainty concerns far more patients in our daily clinical practice and enrol more patients into those trials, to decrease this kind of selection. Heterogeneity has a bright side: we must remember that diversity provides strength to a positive pragmatic trial like ISAT: one treatment was shown to be superior despite the uncontrollable variety of patients, beliefs and expertise that were included in the final results. 5. Surgical delays and expertise Other practical aspects that were the objects of criticism were the longer delay to surgical treatment after randomisation, that neurosurgeons in the trial were general neurosurgeons while interventionists were specialists, and that overall results show that patient care was suboptimal in all participating centres. Surgical delays in ISAT were associated with more frequent re-bleeding events between randomisation and treatment in the clipping group (28 events resulting in 19 deaths) as compared to the coiling group (17 events with seven deaths) (Molyneux et al., 2005). Ideally, the time between randomisation and treatment for either arm should always be minimized. Excluding these patients from analysis, as proposed by some (Bakker et al., 2010), could be considered a ‘sensitivity analysis’, but once the events have occurred, no one knows what might have happened had these patients not re-bled. Excluding these patients from analysis is an exercise in speculation, no more justified than if we now excluded rebleeds after coiling on the speculative basis that newer catheters and coils ‘would have produced better immediate occlusions, had they been available at the time’. Furthermore, in the real world, it is possible that the average delay for surgery really is slightly longer than for endovascular treatment. The claim that neurosurgeons involved in ISAT were less expert than interventionists is without foundation. This contention that the ‘losing arm’ of the study had worse technicians is: • controlled by using multi-centre studies; • can always be argued either way, depending on overall study results; • remains untestable and thus irrelevant. 6. Analyses and interpretation Multiple secondary analyses were performed, discussed, and some even published using the ISAT data. The ISAT investigators
106
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
chose to provide the community with enough data to form their own impressions regarding the reliability of the evidence. This praiseworthy gesture allowed people to engage in all sorts of speculation regarding what the results could be if this or that particular subgroup was considered separately, or if non-significant trends were projected over a lifetime (Mitchell et al., 2008; Ryttlefors et al., 2008). We will restrict the discussion to the published primary intent-to-treat analyses, and comment on both the importance of adhering to pre-defined hypotheses and endpoints, and the dangers of subgroup analyses (Naggara et al., 2011). Given ISAT’s pragmatic research question, the intent-to-treat analysis of the primary hypothesis regarding the pre-defined outcome measure (mRS > two at 1 year) was not only appropriate, but mandatory. Had the mRS scale been divided in some other way (Harbaugh et al., 2003), or had outcomes been compared at some other time point (Bakker et al., 2010) are interesting speculative explorations, which promise to yield different, non-significant, or even contrary conclusions. Had the ISAT authors used such a procedure to claim superiority of coiling, they would have received the appropriate indignation. These explorations only serve to illustrate the need to rigorously adhere to pre-defined definitions of endpoints and hypotheses in the interpretation of trial results: the statistical tools we use cannot tolerate the multiplicity and variety of questions and reformulations; if this rule is transgressed, anything goes. Recruitment in ISAT was stopped in early 2002 because an independent monitoring committee judged that differences in the primary clinical outcome between patients treated with clip ligation and those treated with coil embolization were too large to continue with the uncertainty compatible with randomisation. After trial completion, we now have a generalization, which is not necessarily applicable to all individuals: we now know that when coiling and clipping are both valid options for a particular patient, coiling leads to better outcomes at 1 year. Now was this verified for all patients, aneurysms, and centre characteristics, such as sex, age, race, location, size, experience? Of course not. Clinical diversity makes sure this could never be achieved, and one has to expect heterogeneous results, some apparently increasing, others decreasing the effects of treatments, when dissecting the data post hoc. A study powered to provide evidence for so many “kinds of patients” would lead to an eternal trial, the evidence becoming non-convincing each time the data is split into other “interesting” subgroups. We must be realistic regarding what kind of knowledge is obtainable. At best, unplanned uncontrolled subgroup findings can only be used as hypotheses for future trials. Until such trials are performed, “the answer to a RCT that does not confirm one’s beliefs is not to conduct several sub-analyses until one can see what one believes. Rather the answer is to re-examine one’s beliefs carefully.” (Altman and Bland, 1995; Brookes et al., 2001; Collins and MacMahon, 2001; Molyneux et al., 2005; Schulz and Grimes, 2005; Cowan et al., 2007). A reasonable interpretation of ISAT is that patients with small, ruptured anterior circulation aneurysm should be treated with endovascular coiling.
7. What remains to be questioned and hypotheses for a new trial In real world practice, coiling is now being offered to types of patients that were not studied in ISAT (Gnanalingham et al., 2006). Clinicians often claim that catheter and coil technology has significantly improved make coiling applicable to many more patients than what was originally possible in ISAT. It is also true, however, that many so-called advances have not been rigorously tested, and some have significant drawbacks (i.e.: stents requiring antiplatelet agents). No one knows if these patients would not be better served
Table 2 Patients potentially appropriate for recruitment in ISAT II. Potential higher risks of coiling Lesser efficacy of coiling Certain locations
Weak subgroup evidence
Potential advantage of surgery
Very small aneurysms (< 3 mm) Wide necked aneurysms (> 4 mm) Large aneurysms (≥ 10 mm) MCA aneurysms; pericallosal Posterior circulation (not basilar bifurcation) Patients older than 70 years old Patients younger than 40 years old WFNS 4–5 Non-life threatening cerebral hematoma Multiple accessible aneurysms
with surgical clipping. We also know that many patients continue to be clipped in spite of ISAT, which may or may not be appropriate. For example, no one has proven that large, wide necked aneurysms should be clipped, but given the ‘smaller’ benefit of coiling over clipping in this subgroup, and given the greater risks of angiographic recurrences after coiling large aneurysms (Raymond et al., 2003), many believe that clipping is preferable. Needless to say this line of reasoning is hypothetical and deserves more rigorous evaluation. Therapeutic dilemmas persist for many patients, which cannot be resolved with cases series or epidemiological surveys. Many authors have proposed, in a reconciliatory tone, that ‘Each patient and their aneurysm is different and the decision has to consider the best interests for each patient’, . . . ‘No one treatment option is superior, but rather each has strengths and weaknesses that must be considered when choosing what is best for each patient.’ Such statements are at first glance respectable, but upon further examination, remain empty or even imprudent. With such reasoning, anything goes; no one can ever test a claim or assess treatment results. What is neglected by authors who claim that optimal results are achieved using a case-by-case approach, is that the easiest way to obtain an excellent series is to carefully select patients, leaving all difficult cases to the rival intervention. The goal for physician should not be a perfect personal case series, but better patient outcomes.
8. ISAT II: a new trial Many questions remain regarding choice of coiling or clipping, and many patients are still treated surgically despite ISAT, and all without convincing evidence. The essence of scientific medicine is to question the hypotheses behind our as yet not validated actions. Another trial is indicated. But how would a new trial differ from the initial ISAT? Again we would need a pragmatic trial, which is the best way to offer the best possible care in the presence of uncertainty. A functional primary endpoint at one year would be appropriate. Follow-up tests and visits would be routine. Selection criteria would be few to include a diversity of patients and clinicians, to study the results in the very context in which real care is delivered. To whom would this trial be offered? We can organize our thoughts by looking at the subgroup analyses published in ISAT, 2005, realizing that many correspond to intuitive justifications for our current medical decisions that may not follow general ISAT conclusions, and still need validation. Now is an appropriate time to look at subgroups, as we are looking for hypotheses for another trial! Patients recruited in ISAT II would include those for whom we are tempted to resort to clipping despite ISAT results, or patients we are tempted to coil, but that we could not or did not include in the original formally tested population. These patients and circumstances are summarized in Table 2. They may include circumstances where:
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
• subgroup analyses point to a potential benefit from surgery (patients older than 70 or younger than 40 years old, or patients with WFNS grades 4–5); • cases under-represented in ISAT, perhaps because surgery was felt to be better (i.e.: MCA aneurysms); • when EVT may be more risky (as judged from case series, such as) very small (<3 mm) aneurysms (Nguyen et al., 2008) or less effective (i.e.: large or wide-necked aneurysms); anticipated use of stents (Piotin et al., 2010) or flow diverter; • cases where surgery could offer an immediate advantage (nonlife threatening but large hematomas; multiple aneurysms treatable through the same craniotomy); or certain posterior circulation aneurysms that may not entail the same level of risks as a basilar bifurcation lesion (i.e.: PICA or SCA aneurysms). The conduct of ISAT-II should be integrated into normal patient care. Anytime we wish to practise an unproven intervention or an intervention not proven to be beneficial in certain circumstances such as treating a patient with a ruptured aneurysm for whom the results of ISAT may not apply, recruitment into a trial is indicated (Raymond et al., 2011). When such trials become a part of routine clinical care, we will not only be able to offer impeccable scientific and ethical care to current patients, but we will also begin producing reliable knowledge to guide the management of all patients with ruptured aneurysms. 9. Conclusion The ISAT study was a well-designed and conducted trial, given real-world limitations. Many of the subsequent interpretations, subgroup analyses, and extrapolations of results do not stand up to rigorous scientific evaluation. ISAT only answered one question, and many patients continue to have ruptured aneurysms treated by coiling or clipping without a basis in reliable evidence. We are ready for the next pragmatic trial, complementary to ISAT, which would address the best management of patients with ruptured aneurysms for whom uncertainty remains. Disclosure of interest J.R. and T.E.D. are co-principal investigators of the CURES trial, funded by the Canadian Institute of Health Research (CIHR MOP119554) and the STAT and FIAT trials (non-funded). A.J.M. and R.S.K. were principal investigators of the ISAT trial. M.K. has no conflict of interest to declare. References Altman, D.G., Bland, J.M., 1995. Absence of evidence is not evidence of absence. BMJ 311 (7003), 485. Ashcroft, R., 2000. Giving medicine a fair trial. Trials should not second guess what patients want. BMJ 320 (7251), 1686. Ausman, J.I., 2003. ISAT study: is coiling better than clipping? Surg Neurol 59 (3), 162–165 [discussion 165–173; author reply 173–185]. Ausman, J.I., 2008. The International Subarachnoid Aneurysm Trial II: comparison of clipping vs coiling: key questions. Are the results of the study generalizable? Should clipping be done for patients less than 40 years of age? Surg Neurol 70 (1), 104–107. Bakker, N.A., Metzemaekers, J.D., et al., 2010. International subarachnoid aneurysm trial 2009: endovascular coiling of ruptured intracranial aneurysms has no significant advantage over neurosurgical clipping. Neurosurgery 66 (5), 961–962. Barrow, D.L., 2004. Bad science ISAT: the impact on neurosurgical practice. Clin Neurosurg 51, 126–131. Batjer, H.H., 2003. Surg Neurol 59 (3), 168–169. Britz, G.W., 2005. ISAT trial: coiling or clipping for intracranial aneurysms? Lancet 366 (9488), 783–785. Britz, G.W., Newell, D.W., et al., 2003. The ISAT trial. Lancet 361 (9355), 431–432. Brookes, S.T., Whitley, E., et al., 2001. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess 5 (33), 1–56.
107
Carter, B.S., 2008. ISAT subgroups: is aneurysm clipping better for some patients? Neurosurgery 63 (6), 14. Casasco, A.E., Aymard, A., et al., 1993. Selective endovascular treatment of 71 intracranial aneurysms with platinum coils. J Neurosurg 79 (1), 3–10. Collins, R., MacMahon, S., 2001. Reliable assessment of the effects of treatment on mortality and major morbidity, I: clinical trials. Lancet 357 (9253), 373–380. Cowan J.A.Jr., J., Ziewacz, et al., 2007. Use of endovascular coil embolization and surgical clip occlusion for cerebral artery aneurysms. J Neurosurg 107 (3), 530–535. Debrun, G., 2003. Commentary. Surg Neurol 59 (3), 168. Derdeyn, C.P., Barr, J.D., et al., 2003. The International Subarachnoid Aneurysm Trial (ISAT): a position statement from the Executive Committee of the American Society of Interventional and Therapeutic Neuroradiology and the American Society of Neuroradiology. AJNR Am J Neuroradiol 24 (7), 1404–1408. Figueiredo, E.G., 2010. The ISAT. Again. World Neurosurg 73 (6), 601–603. Gnanalingham, K.K., Apostolopoulos, V., et al., 2006. The impact of the international subarachnoid aneurysm trial (ISAT) on the management of aneurysmal subarachnoid haemorrhage in a neurosurgical unit in the UK. Clin Neurol Neurosurg 108 (2), 117–123. Harbaugh, R.E., 2003. Surg Neurol 59 (3), 165–167. Harbaugh, R.E., Heros, R.C., et al., 2003. More on ISAT. Lancet 361 (9359), 783–784. Hernesniemi, J., Koivisto, T., 2004. Comments on The impact of the International Subarachnoid Aneurysm Treatment Trial (ISAT) on neurosurgical practice. Acta Neurochir (Wien) 146 (2), 203–208. Heros, R.C., Hadley, M.N., Harbaugh, R.E., 2002. Position statement by the AANS, CNS and AANS/CNS section on cerebrovascular surgery. AANS Bull 11 (4.). Heros, R.C., 2008a. International Subarachnoid Aneurysm Trial analysis. J Neurosurg 108 (3), 433–435. Heros, R.C., 2008b. Vascular neurosurgery since the International Subarachnoid Aneurysm Trial. J Neurosurg 109 (6), 989–991 [author reply 990–991]. Higashida, R.T., Halbach, V.V., et al., 1990. Treatment of intracranial aneurysms with preservation of the parent vessel: results of percutaneous balloon embolization in 84 patients. AJNR Am J Neuroradiol 11 (4), 633–640. Javadpour, M., 2007. ISAT: Will the results stand the test of time? Br J Neurosurg 21 (1), 7–8. Kirkpatrick, P.J., Kirollos, R.W., et al., 2003. Lessons to be learnt from the International Subarachnoid Haemorrhage Trial (ISAT). Br J Neurosurg 17 (1), 5–7. Kobayashi, S., 2003. Commentary. Surg Neurol 59 (3), 167–168. Laitt, R.D., 2007. How much can be concluded from ISAT? Br J Neurosurg 21 (1), 9–10. Leung, C.H., Poon, W.S., et al., 2003. The ISAT trial. Lancet 361 (9355), 430–431 [author reply 432]. Lindsay, K.W., 2003. The impact of the International Subarachnoid Aneurysm Treatment Trial (ISAT) on neurosurgical practice. Acta Neurochir (Wien) 145 (2), 97–99. Maurice-Williams, R.S., 2004. Aneurysm surgery after the International Subarachnoid Aneurysm Trial (ISAT). J Neurol Neurosurg Psychiatry 75 (6), 807–808. Mitchell, P., Kerr, R., et al., 2008. Could late rebleeding overturn the superiority of cranial aneurysm coil embolization over clip ligation seen in the International Subarachnoid Aneurysm Trial? J Neurosurg 108 (3), 437–442. Mitha, A.P., Ogilvy, C.S., 2005. ISAT: coiling or clipping for ruptured intracranial aneurysms? Lancet Neurol 4 (12), 791–792. Mocco, J., Hopkins, L.N., 2008. International Subarachnoid Aneurysm Trial analysis. J Neurosurg 108 (3), 436. Mohr, J.P., 2003. The ISAT trial. Lancet 361 (9355), 431 [author reply 432]. Molyneux, A., Kerr, R., et al., 2002. International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet 360 (9342), 1267–1274. Molyneux, A.J., Kerr, R.S., et al., 2009. Risk of recurrent subarachnoid haemorrhage, death, or dependence and standardised mortality ratios after clipping or coiling of an intracranial aneurysm in the International Subarachnoid Aneurysm Trial (ISAT): long-term follow-up. Lancet Neurol 8 (5), 427–433. Molyneux, A.J., Kerr, R.S., et al., 2005. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet 366 (9488), 809–817. Naggara, O., Raymond, J., et al., 2011. The problem of subgroup analyses: an example from a trial on ruptured intracranial aneurysms. AJNR Am J Neuroradiol 32 (4), 633–636. Nguyen, T.N., Raymond, J., et al., 2008. Association of endovascular therapy of very small ruptured aneurysms with higher rates of procedure-related rupture. J Neurosurg 108 (6), 1088–1092. Nishioka, H., 1966. Report on the cooperative study of intracranial aneurysms and subarachnoid hemorrhage. Section VII. I. Evaluation of the conservative management of ruptured intracranial aneurysms. J Neurosurg 25 (5), 574–592. Piotin, M., Blanc, R., et al., 2010. Stent-assisted coiling of intracranial aneurysms: clinical and angiographic results in 216 consecutive aneurysms. Stroke 41 (1), 110–115. Raabe, A., Schmiedek, P., et al., 2003. German Society of Neurosurgery Section on Vascular Neurosurgery: Position Statement on the International Subarachnoid Hemorrhage Trial (ISAT). Zentralblatt fur Neurochirurgie 64 (3), 99–103. Raymond, J., Darsaut, T.E., et al., 2011. A trial on unruptured intracranial aneurysms (the TEAM trial): results, lessons from a failure and the necessity for clinical care trials. Trials 12, 64.
108
J. Raymond et al. / Neurochirurgie 58 (2012) 103–108
Raymond, J., Guilbert, F., et al., 2003. Long-term angiographic recurrences after selective endovascular treatment of aneurysms with detachable coils. Stroke 34 (6), 1398–1403. Ryttlefors, M., Enblad, P., et al., 2008. International subarachnoid aneurysm trial of neurosurgical clipping versus endovascular coiling: subgroup analysis of 278 elderly patients. Stroke 39 (10), 2720–2726. Sackett, D. (2006). The Tactics of performing therapeutic trials. Cilical epidemiology: How to do clinical practice research. R. Haynes, Sackett, DR, Guyatt, GH, Tugwell, P. Philadelphia, Lippincott, Williams & Wilkins: p. 66–172. Sade, B., Mohr, G., 2004. Critical appraisal of the International Subarachnoid Aneurysm Trial (ISAT). Neurol India 52 (1), 32–35.
Schulz, K.F., Grimes, D.A., 2005. Multiplicity in randomised trials II: subgroup and interim analyses. Lancet 365 (9471), 1657–1661. Solomon, R., 2003. Commentary. Surg Neurol 59 (3), 173. Tait, M.J., Critchley, G.R., et al., 2007. How much can be concluded from the International Subarachnoid Aneurysm Trial (ISAT)? Br J Neurosurg 21 (1), 3–6. ter Brugge, K.G., 2003. Commentary. Surg Neurol 59 (3), 172–173. Zacharia, B.E., Ducruet, A.F., et al., 2011. Technological advances in the management of unruptured intracranial aneurysms fail to improve outcome in New York state. Stroke 42 (10), 2844–2849.