- Email: [email protected]
Cost-effectiveness analysis of CTA and LP for evaluation of suspected SAH after negative non-contrast CT
Clinical Neurology and Neurosurgery 142 (2016) 104–111
Contents lists available at ScienceDirect
Clinical Neurology and Neurosurgery journal homepage: www.elsevier.com/locate/clineuro
Cost-effectiveness analysis of CTA and LP for evaluation of suspected SAH after negative non-contrast CT Xiao Wu, Vivek B. Kalra, Howard P. Forman, Ajay Malhotra ∗ Department of Diagnostic Radiology, Yale School of Medicine, United States
a r t i c l e
i n f o
Article history: Received 18 September 2015 Received in revised form 24 December 2015 Accepted 27 December 2015 Available online 4 January 2016 Keywords: Thunderclap headache Subarachnoid hemorrhage Lumbar puncture CT angiography Cost-effectiveness analysis
a b s t r a c t Objectives: Diagnostic workup of patients presenting with thunderclap headache and negative initial head CT remains a challenge, with most commonly employed strategies being lumbar puncture (LP) and CT angiography (CTA). The objective of this study was to determine the cost-effectiveness of these options. Patients and methods: A decision model was designed using clinical probabilities, costs, and utilities from published values in the literature. Base case analysis and Monte Carlo simulation were performed using the model to determine the cost-effectiveness of both options. Results: CTA was associated with an expected cost of $747 and an expected utility of 0.798603029. In comparison, LP was associated with a cost of $504 and an expected utility of 0.799259526, making it the optimal strategy from both the cost and the utility perspectives. LP was also the more cost-effective strategy in all iterations in the Monte Carlo simulation. A sensitivity analysis showed that with the 2014 US Medicare reimbursement values, LP would remain the more cost-effective strategy unless its cost exceeded 4 times its current value. Conclusion: LP should remain the preferred strategy for evaluation of SAH in patients presenting with thunderclap headache and negative non-contrast head CT. CTA is not an effective replacement, from either a utility or cost perspective. © 2015 Elsevier B.V. All rights reserved.
1. Introduction The diagnosis of nontraumatic SAH presenting acutely with thunderclap headache is challenging but critical. Misdiagnosis is associated with a greater risk of death, which increases with time [1]. The reported sensitivity of non-contrast CT for diagnosis of acute SAH has been 94–100% [2–5]. The dilemma arises when the initial CT is negative for acute blood, as even a low false negative rate may be unacceptable. LP has generally been accepted as the next step for evaluation of acute SAH by the American Heart Association, the American Stroke Association and the American College of Radiology [6,7]. However, LPs are invasive and painful, take time and skill to perform, and traumatic taps are difficult to interpret. CTA has thus been proposed in the Emergency Medicine literature as an alternative to LP, to exclude aneurysmal SAH after a negative non-contrast CT [8]. Both CTA and LP have inherent limitations,
costs and risks of complications. Although cost-effectiveness is not the primary consideration at the point of care, it is important for a system-wide perspective on different strategies for this condition.
2. Materials and methods 2.1. Study design The study was designed as a decision tree model based on probabilities and sensitivities derived from literature. Institutional review board approval was not necessary as no human subject research or patient-specific evaluation was involved. The model was based on a patient presenting with thunderclap headache to the Emergency Department (ED) with an intact neurological exam. 2.2. Model design
∗ Corresponding author at: Department of Diagnostic Radiology, Yale School of Medicine, Box 208042, Tompkins East 2, 333 Cedar St, New Haven, CT 06520-8042, United States. Fax: +203 785 3024. E-mail addresses: [email protected] (X. Wu), [email protected] (V.B. Kalra), [email protected] (H.P. Forman), [email protected] (A. Malhotra). http://dx.doi.org/10.1016/j.clineuro.2015.12.021 0303-8467/© 2015 Elsevier B.V. All rights reserved.
Our decision tree was built using TreeAge Pro Suite 2014 (TreeAge Software, Inc, Williamstown, Massachusetts) with starting point of a patient presenting thunderclap headache but having a negative non-contrast CT scan for SAH in the ED. The two followup strategies considered were CTA and LP. The model was built to estimate costs, quality-adjusted life years (QALYs), and incremental
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
Fig. 1. Decision tree with selected nodes.
105
106
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
Fig. 2. Cost-effectiveness analysis. An optimal strategy should have a high effectiveness and a low cost (bottom right corner).
cost-effectiveness ratios (ICERs), using a standard WTP threshold of $50,000 per QALY to compare the two strategies. The initial negative CT results for SAH could be true or false negative. The arm labeled “True Negative” included patients without SAH, and subsequently performed CTA or LP could return true negative or false positive results. The arm labeled “False Negative” included patients with SAH missed on initial non-contrast CT, and CTA or LP could be considered for follow-up. Either of the two strategies could give true positive or false negative results. Failed cases of LP were included, with the assumption that the patient would subsequently undergo a CTA if LP could not be performed successfully. From true negative results of both CTA and LP, patients without SAH might still experience complications of their respective follow-up procedure. In true negatives of LP (without SAH), the possibility of an undetected aneurysms rupturing in the future, was discussed. The false positive results of CTA represented findings of incidental aneurysms that had not ruptured and were not responsible for acute SAH. These aneurysms could then be managed conservatively or surgically, with risk of rupturing associated with conservative management, and complications associated with both. Conservative treatment of incidental aneurysms was defined as 5 follow-up CTAs [9]. Surgical management was defined as coiling of the unruptured aneurysm, as it is more commonly performed than clipping and associated with lower cost in the short term and complication rate [10]. The complications from coiling were hydrocephalus, postoperative stroke, intracerebral hemorrhage, and death. The false positive results of LP represented traumatic taps and these patients would normally undergo a further CTA
as LP results cannot be relied upon [11]. These patients were further grouped according to the presence and absence of incidental aneurysms, and sensitivity and specificity of CTA were considered respectively. The sensitivity and specificity of two CTAs combined were assumed to be 100% since the probability of two consecutive false positives or false negatives would be on a scale of 10−5 . The true positive results were considered as treated aneurysms for both strategies (CTA and LP) and false negatives as untreated aneurysms for CTA, with the untreated case having a significantly higher cost and lower utility. Terminal nodes were assigned to each arm, with specified cost and effectiveness. The complete model of this cost-effectiveness is provided in Fig. 1and Fig. S1. 2.3. Model parameters 2.3.1. Diagnostic test parameters Initial CT scan was assigned a negative predictive value for SAH of 99.82%, based on a average sensitivity of 98%, a specificity of 100%, and an SAH prevalence of 8.41% [2–4,12]. CTA was assigned a sensitivity for aneurysm detection of 99.2%, based on a metaanalysis of 16 and 64 row scanners with more than 3600 patients [13]. LP was assigned a specificity of 86.7%, reflecting an incidence of 13.3% traumatic taps, and a sensitivity of 100% [11,14,15]. 2.3.2. Clinical probabilities Each probability in this model was based on published literature values. The probability of reaching each terminal node was calculated along the path from the starting decision node, with Bayesian
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
107
Fig. 3. Monte Carlo simulation scatter plot of 10,000 iterations. In reference to LP, CTA has negative incremental effectiveness and a positive incremental cost in all iterations, rendering it the suboptimal choice.
updating when necessary. If “Aneurysm” was preceded by a “False Negative” node, the outcome probabilities were constructed to reflect the poor natural history of an untreated ruptured aneurysm. If aneurysms were detected from the “True Positive” nodes, probabilities were constructed to reflect the outcome of patients with treated ruptured aneurysms. Complications of CTA are related to iodinated contrast administration, specifically allergic-like reaction and contrast-induced nephropathy. Radiation-associated stochastic risks were not included in the model [16,17]. Complications of LP include post-procedural headache, meningitis, and spinal canal hematoma (subdural as well as epidural). The incidence of postprocedural headache was based on the data reported for lumbar punctures performed using non-cutting needles [18]. The incidence of meningitis following LP was assigned a 0 probability, as it is very rare and only few cases are reported in literature [18,19]. Most of these were reported before the use of face masks from Streptoccocus salivarius. The incidence has dropped significantly in the last few years with the routine use of face masks. Similarly, epidural and subdural hematomas, although possible complications of LP, are quite uncommon, usually small and only require conservative follow up [20]. For the purpose of this study, the incidence of hematoma was also assigned a probability of 0. The incidences of other complications were all derived from literature values.
Medicare reimbursement values when possible. Costs incurred from complications were derived from the literature. The cost of post-procedural headache treatment was a weighted average based on the proportion of these patients being treated with intravenous caffeine infusion, blood patch, and/or hospitalization. To minimize bias and subjectivity, each cost was assigned to have a normal distribution with a standard deviation of 10% in the Monte Carlo simulation.
2.4. Costs
Analyses were performed using TreeAge Pro Suite 2014 (TreeAge, Inc., Williamstown, Massachusetts). The costeffectiveness of each follow-up strategy was evaluated at the corresponding decision node and Monte Carlo simulation was
Total cost at each terminal node was derived by totaling all the costs along its path, which were determined using 2014 regional
2.5. Utilities Each terminal node was assigned a utility value according to the modified Rankin Scale (mRS) score and a value was based on Quality Adjusted Life Year (QALY). A mRS of 0 to 2 corresponded to a good outcome, mRS of 3–5 corresponded to a poor outcome, and mRS of 6 corresponded to death. The exact value for each outcome was obtained from Samsa et al. with a variation range for Monte Carlo simulation. A detailed list of all parameters, their values, distributions (when applicable), and references is presented in Table S1. 2.6. Statistical analyses
108
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
Fig. 4. One-way sensitivity analysis varying cost of LP.
performed over the range of each distributional variable with 10,000 iterations. 3. Results
as the more cost-effective strategy until its cost exceeds $364.45 (Fig. 4). A two-way sensitivity analysis was performed varying the costs of LP and CTA. The shaded regions correspond to the more costeffective strategy given the costs shown on the coordinates. (Fig. 5).
3.1. Base case scenario 4. Discussion In the base case scenario, the expected cost and utility of each strategy were used to evaluate the cost-effectiveness of each strategy. CTA was associated with a cost of $747 and an expected utility of 0.798603029; LP was associated with a cost of $504 and an expected utility of 0.799259526. LP was shown to be the optimal strategy from both the cost and the utility perspectives. The respective cost and expected utility of each strategy are presented in Fig. 2. 3.2. Monte Carlo simulation A Monte Carlo simulation was carried out with 10,000-iteration probabilistic sampling. Results demonstrate that LP is the more cost-effective strategy in 100% of the cases at the $50,000/QALY WTP threshold. A scatter plot of CTA against LP is presented in Fig. 3. 3.2.1. Sensitivity analysis A one-way sensitivity analysis was performed by varying the cost of LP assuming a cost of $498.02 for CTA, as it might differ from institution to institution. The result showed that LP remains
Lumbar puncture has long been regarded as the standard practice for thunderclap headaches in patients with negative noncontrast CT. However, CTA is being increasingly recommended to replace LP in headache patients with negative non-contrast CT scan. The supporting arguments favoring CTA over LP include the high sensitivity and specificity of CTA, and the low complication rate due to its non-invasive nature [8]. Our economic analysis shows that LP is more cost-effective than CTA, with a slightly higher utility and a lower cost. However, the difference in outcome is minimal, suggesting comparable clinical performance of the two tests. In the paper by McCormack et al., mathematical calculations based on probabilities were carried out to demonstrate a high sensitivity of CTA in detecting aneurysm and subsequently to conclude that CTA would be a better follow-up strategy as compared to LP [8]. However, a significant limitation in the calculations is the high probability of aneurysm detected by CTA being incidental, rather than causal, given a negative CT scan. Unlike LP, which is designed specifically to detect blood in cerebrospinal fluid, CTA is performed primarily to detect aneurysms, incidental or causal. With the parameters provided, the probability of the aneurysm
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
109
Fig. 5. Two-way sensitivity analysis varying costs of LP and CTA.
detected by CTA being the true cause of undetected SAH on CT is only approximately 35%, given a negative initial CT scan. Since follow-up is only performed after negative non-contrast CT scans, this result would call the utility of CTA follow-up into question. The detection of incidental aneurysms also raises the question of extensive follow-up against treatment of unruptured aneurysms, both of which are not cost or risk free. Another decision analysis on similar topic was performed by Ward et al., evaluating the cost-effectiveness of CT only, CT followed by CTA, CT followed by LP, and CT followed by MRA as imaging strategies for patients presenting with headaches [15]. However, the mathematical model assigned inaccurate complication rates to follow-up strategies, with 0 complication rate from LP and 11% contrast-induced nephropathy from CTA. In previous literature, the complication rate of LP can be as high as 4% for headache that requires hospitalization and blood patch [18]. On the other hand, the rate of contrast-induced nephropathy from CTA is low (reported as 0.02% in a study with nearly 20,000 patients)[21]. The discrepancy between the assigned values and the reported literature values renders the conclusion of questionable significance. In the model, LP was ascribed a 100% sensitivity in SAH detection, as LP has been regarded as a reference for SAH detection. However, LP is an imperfect reference standard with a 13.3% occurrence of traumatic taps performed at the bedside [14]. These also add to the dilemma as no clear-cut thresholds have been established in distinguishing a traumatic tap from a true positive result for SAH [11]. From our clinical experience, patients with traumatic taps are usually managed with a follow-up CTA by most ED physicians,
due to the high risk and poor outcome of undetected SAH. Some institutions might opt to perform digital subtraction angiography (DSA) rather than CTA, as it is regarded as the gold standard for aneurysms. However, neither DSA and CTA are designed to detect for blood, and CTA now with the current 16 and 64 row scanners has a submillimeter resolution of 0.5–0.7 mm [13]. In the context where perimesencephalic SAH is a concern, our past research has also shown CTA alone to be sufficient for diagnosis [22]. The traumatic taps should be considered in the context of extensive follow-up procedures and associated complications and costs to make the model more fair. Similar to McCormack et al., Ward et al. also did not discuss the possibility of detecting incidental aneurysms in patients without SAH, which is a significant limitation of just angiographic (CTA or MRA) follow-up. A wide range of failed LP rates have been reported in literature, being more common in obese and older patients [23,24]. A failed LP attempt at bedside prompts imaging guided LP using fluoroscopy which is associated with a much lower failure rate. Failed fluoroscopy-guided LP, though unusual, are followed by CTA. The complication rate with LP remains the same, whether failed or successful. Another significant factor in clinical decision-making between LP and CTA would be the time required to complete the tests, especially in EDs. One might argue that LP would be a more time-consuming procedure and that CTA may be preferred by the emergency room physician. However, time is a subjective measure, and the amount of time required may differ due to various institutional practices. Getting a CTA might require less time than
110
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
performing an LP for an ED physician, but in our clinical experience, LP and CTA take approximately the same amount of time overall for making final management decisions. This is because the lab analysis results of spinal fluid may take less time than the final CTA read by radiologists. Thus, both strategies require roughly the same amount of time to produce the official report. One limitation of this model would be the costs assigned to LP and CTA, as these values usually differ in hospitals. Since our paper puts the emphasis on the overall cost-effectiveness of the two approaches to the healthcare system, the values were derived from the 2014 Medicare reimbursement values. It has also been a standard practice in cost-effectiveness analyses to make use of the Medicare reimbursement value, which might represent a lower bound of the actual cost incurred to the payer [15,25,26]. The actual costs incurred to patients might different significantly from the reimbursement value, but this study was not conducted from an individual’s perspective. The cost result would be more variable depending on the nature of the hospital, the local cost of living, and so on. The cost difference is likely to be enlarged in scale when out-of-pocket costs are used. In addition, sensitivity analyses were carried out to assess the robustness of the conclusion against the costs of LP and CTA. With an addition of $99.74 from fluoroscopy guidance, the cost of LP is still lower than the $364.45 threshold. Even when the actual cost of LP exceeds the threshold, LP would still be a better strategy when considering the expected effectiveness (in QALYs) for patients. MRA is not discussed in this model as a possible follow-up strategy, although it avoids complications related to iodinated contrast. MRA has limited availability in ERs, safety concerns, lower sensitivity and specificity and higher costs than CTA [27]. Although the cost and utility of each follow-up strategy were fully accounted for by including all possible scenarios and respective outcomes, the analysis does not discuss the different amounts of time required to perform LP or CTA. Another limitation of the study is that opportunity costs are not factored into the strategies. Indirect costs, such as lost time from work due to post LP headache, are difficult to assess and incorporate. Also, Medicare reimbursement may not be the most accurate indicator of actual costs for a middle-aged patient. Cross-sectional imaging can also reveal other etiologies for the presenting headaches, something that the LP strategy would not be able to identify. We did not consider surgical clipping in the management of detected aneurysms. The long term costs of clipping might be lower than coiling. However, increasingly aneurysms are being treated by coiling and the current modelling was done with a one year time horizon [28].
5. Conclusions LP following a negative initial non-contrast CT is the dominant strategy for diagnosing SAH in patients with thunderclap headache in the ED. In this clinical scenario CT/CTA is not an effective replacement from a utility or cost perspective.
Funding None.
Acknowledgement None.
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.clineuro.2015.12. 021.
References [1] R.G. Kowalski, J. Claassen, K.T. Kreiter, J.E. Bates, N.D. Ostapkovich, E.S. Connolly, S.A. Mayer, Initial misdiagnosis and outcome after subarachnoid hemorrhage, Jama 291 (2004) 866–869. [2] B.M. Boesiger, J.R. Shiber, Subarachnoid hemorrhage diagnosis by computed tomography and lumbar puncture: are fifth generation CT scanners better at identifying subarachnoid hemorrhage? J. Emerg. Med. 29 (2005) 23–27. [3] R.L. Byyny, W.R. Mower, N. Shum, G.Z. Gabayan, S. Fang, L.J. Baraff, Sensitivity of noncontrast cranial computed tomography for the emergency department diagnosis of subarachnoid hemorrhage, Ann. Emerg. Med. 51 (2008) 697–703. [4] J.J. Perry, I.G. Stiell, M.L.A. Sivilotti, M.J. Bullard, M. Émond, C. Symington, J. Sutherland, A. Worster, C. Hohl, J.S. Lee, M.A. Eisenhauer, M. Mortensen, D. Mackey, M. Pauls, H. Lesiuk, G.A. Wells, Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study, BMJ 343 (2011) d4277. [5] N. van der Wee, G.J. Rinkel, D. Hasan, J. van Gijn, Detection of subarachnoid haemorrhage on early CT: is lumbar puncture still needed after a negative scan? J. Neurol. Neurosurg. Psychiatry 58 (1995) 357–359. [6] J.B. Bederson, E.S. Connolly, H.H. Batjer, R.G. Dacey, J.E. Dion, M.N. Diringer, J.E. Duldner, R.E. Harbaugh, A.B. Patel, R.H. Rosenwasser, Guidelines for the management of aneurysmal subarachnoid hemorrhage: a statement for healthcare professionals from a special writing group of the stroke council, American heart association, Stroke 40 (2009) 994–1025. [7] Douglas AC, Wippold II FJ, Broderick DF, et al. ACR Appropriateness Criteria® Acute Trauma to Headache. Available at https://acsearch.acr.org/docs/69482/ Narrative/. American College of Radiology. Accessed Nov 29, 2015. [8] R.F. McCormack, A. Hutson, Can computed tomography angiography of the brain replace lumbar puncture in the evaluation of acute-onset headache after a negative noncontrast cranial computed tomography scan? Acad. Emerg. Med. 17 (2010) 444–451. [9] R.D. Brown Jr., J.P. Broderick, Unruptured intracranial aneurysms: epidemiology, natural history, management options, and familial screening, Lancet Neurol. 13 (2014) 393–404. [10] I.H. Wadd, A. Haroon, S. Habibullah Ansari, S. Mukhtar, U. Rashid, A.H. Vohra, Aneurysmal subarachnoid hemorrhage: outcome of aneurysm clipping versus coiling in anterior circulation aneurysm, J. Coll Phys. Surg. Pak. 25 (2015) 798–801. [11] J. Gorchynski, J. Oman, T. Newton, Interpretation of traumatic lumbar punctures in the setting of possible subarachnoid hemorrhage: who can be safely discharged, Cal. J. Emerg. Med. 8 (2007) 3–7. [12] J.J. Perry, I.G. Stiell, M.L.A. Sivilotti, M.J. Bullard, J.S. Lee, M. Eisenhauer, C. Symington, M. Mortensen, J. Sutherland, H. Lesiuk, G.A. Wells, High risk clinical characteristics for subarachnoid haemorrhage in patients with acute headache: prospective cohort study, BMJ 341 (2010) c5204. [13] J. Menke, J. Larsen, K. Kallenberg, Diagnosing cerebral aneurysms by computed tomographic angiography: meta-analysis, Ann. Neurol. 69 (2011) 646–654. [14] K.H. Shah, K.M. Richard, S. Nicholas, J.A. Edlow, Incidence of traumatic lumbar puncture, Acad. Emerg. Med. 10 (2003) 151–154. [15] M.J. Ward, J.B. Bonomo, O. Adeoye, A.S. Raja, J.M. Pines, Cost-effectiveness of diagnostic strategies for evaluation of suspected subarachnoid hemorrhage in the emergency department, Acad. Emerg. Med. 19 (2012) 1134–1144. [16] W. Hendee, Risk of medical imaging, Med. Phys. 40 (2013) 040401. [17] A. Sabarudin, M.Z. Yusof, M. Mohamad, Z. Sun, Radiation dose associated with cerebral CT angiography and CT perfusion: an experimental phantom study, Radiat. Prot. Dosim. (2013). [18] Y. Dakka, N. Warra, R.J. Albadareen, M. Jankowski, B. Silver, Headache rate and cost of care following lumbar puncture at a single tertiary care hospital, Neurology 77 (2011) 71–74. [19] H. Zetterberg, K. Tullhög, O. Hansson, L. Minthon, E. Londos, K. Blennow, Low incidence of post-lumbar puncture headache in 1089 consecutive memory clinic patients, Eur. Neurol. 63 (2010) 326–330. [20] D.J. Gaucher Jr., J.A. Perez Jr., Subdural hematoma following lumbar puncture, Arch. Intern. Med. 162 (2002) 1904–1905. [21] T.J. Kaufmann, J. Huston, J.N. Mandrekar, C.D. Schleck, K.R. Thielen, D.F. Kallmes, Complications of diagnostic cerebral angiography: evaluation of 19,826 consecutive patients, Radiology 243 (2007) 812–819. [22] V.B. Kalra, X. Wu, H.P. Forman, A. Malhotra, Cost-effectiveness of angiographic imaging in isolated perimesencephalic subarachnoid hemorrhage, Stroke 45 (2014) 3576–3582. [23] S.D. Yu, M.Y. Chen, A.J. Johnson, Factors associated with traumatic fluoroscopy-guided lumbar punctures: a retrospective review, Am. J. Neuroradiol. 30 (2009) 512–515. [24] J.T. Nomura, S.J. Leech, S. Shenbagamurthi, P.R. Sierzenski, R.E. O’Connor, M. Bollinger, M. Humphrey, J.A. Gukhool, A randomized controlled trial of ultrasound-assisted lumbar puncture, J. Ultrasound Med. 26 (2007) 1341–1348.
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111 [25] P.C. Sanelli, A. Pandya, A.Z. Segal, A. Gupta, S. Hurtado-Rua, J. Ivanidze, K. Kesavabhotla, D. Mir, A.I. Mushlin, M.G. Hunink, Cost-effectiveness of CT angiography and perfusion imaging for delayed cerebral ischemia and vasospasm in aneurysmal subarachnoid hemorrhage, Am. J. Neuroradiol. (2014). [26] C.E. Tung, Y.T. So, M.G. Lansberg, Cost comparison between the atraumatic and cutting lumbar puncture needles, Neurology 78 (2012) 109–113. [27] M. Okahara, H. Kiyosue, M. Yamashita, H. Nagatomi, H. Hata, T. Saginoya, Y. Sagara, H. Mori, Diagnostic accuracy of magnetic resonance angiography for
111
cerebral aneurysms in correlation with 3D–digital subtraction angiographic images: a study of 133 aneurysms. Stroke, J. Cereb. Circ. 33 (2002) 1803–1808. [28] N. Andaluz, M. Zuccarello, Recent trends in the treatment of cerebral aneurysms: analysis of a nationwide inpatient database, J. Neurosurg. 108 (2008) 1163–1169.
Contents lists available at ScienceDirect
Clinical Neurology and Neurosurgery journal homepage: www.elsevier.com/locate/clineuro
Cost-effectiveness analysis of CTA and LP for evaluation of suspected SAH after negative non-contrast CT Xiao Wu, Vivek B. Kalra, Howard P. Forman, Ajay Malhotra ∗ Department of Diagnostic Radiology, Yale School of Medicine, United States
a r t i c l e
i n f o
Article history: Received 18 September 2015 Received in revised form 24 December 2015 Accepted 27 December 2015 Available online 4 January 2016 Keywords: Thunderclap headache Subarachnoid hemorrhage Lumbar puncture CT angiography Cost-effectiveness analysis
a b s t r a c t Objectives: Diagnostic workup of patients presenting with thunderclap headache and negative initial head CT remains a challenge, with most commonly employed strategies being lumbar puncture (LP) and CT angiography (CTA). The objective of this study was to determine the cost-effectiveness of these options. Patients and methods: A decision model was designed using clinical probabilities, costs, and utilities from published values in the literature. Base case analysis and Monte Carlo simulation were performed using the model to determine the cost-effectiveness of both options. Results: CTA was associated with an expected cost of $747 and an expected utility of 0.798603029. In comparison, LP was associated with a cost of $504 and an expected utility of 0.799259526, making it the optimal strategy from both the cost and the utility perspectives. LP was also the more cost-effective strategy in all iterations in the Monte Carlo simulation. A sensitivity analysis showed that with the 2014 US Medicare reimbursement values, LP would remain the more cost-effective strategy unless its cost exceeded 4 times its current value. Conclusion: LP should remain the preferred strategy for evaluation of SAH in patients presenting with thunderclap headache and negative non-contrast head CT. CTA is not an effective replacement, from either a utility or cost perspective. © 2015 Elsevier B.V. All rights reserved.
1. Introduction The diagnosis of nontraumatic SAH presenting acutely with thunderclap headache is challenging but critical. Misdiagnosis is associated with a greater risk of death, which increases with time [1]. The reported sensitivity of non-contrast CT for diagnosis of acute SAH has been 94–100% [2–5]. The dilemma arises when the initial CT is negative for acute blood, as even a low false negative rate may be unacceptable. LP has generally been accepted as the next step for evaluation of acute SAH by the American Heart Association, the American Stroke Association and the American College of Radiology [6,7]. However, LPs are invasive and painful, take time and skill to perform, and traumatic taps are difficult to interpret. CTA has thus been proposed in the Emergency Medicine literature as an alternative to LP, to exclude aneurysmal SAH after a negative non-contrast CT [8]. Both CTA and LP have inherent limitations,
costs and risks of complications. Although cost-effectiveness is not the primary consideration at the point of care, it is important for a system-wide perspective on different strategies for this condition.
2. Materials and methods 2.1. Study design The study was designed as a decision tree model based on probabilities and sensitivities derived from literature. Institutional review board approval was not necessary as no human subject research or patient-specific evaluation was involved. The model was based on a patient presenting with thunderclap headache to the Emergency Department (ED) with an intact neurological exam. 2.2. Model design
∗ Corresponding author at: Department of Diagnostic Radiology, Yale School of Medicine, Box 208042, Tompkins East 2, 333 Cedar St, New Haven, CT 06520-8042, United States. Fax: +203 785 3024. E-mail addresses: [email protected] (X. Wu), [email protected] (V.B. Kalra), [email protected] (H.P. Forman), [email protected] (A. Malhotra). http://dx.doi.org/10.1016/j.clineuro.2015.12.021 0303-8467/© 2015 Elsevier B.V. All rights reserved.
Our decision tree was built using TreeAge Pro Suite 2014 (TreeAge Software, Inc, Williamstown, Massachusetts) with starting point of a patient presenting thunderclap headache but having a negative non-contrast CT scan for SAH in the ED. The two followup strategies considered were CTA and LP. The model was built to estimate costs, quality-adjusted life years (QALYs), and incremental
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
Fig. 1. Decision tree with selected nodes.
105
106
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
Fig. 2. Cost-effectiveness analysis. An optimal strategy should have a high effectiveness and a low cost (bottom right corner).
cost-effectiveness ratios (ICERs), using a standard WTP threshold of $50,000 per QALY to compare the two strategies. The initial negative CT results for SAH could be true or false negative. The arm labeled “True Negative” included patients without SAH, and subsequently performed CTA or LP could return true negative or false positive results. The arm labeled “False Negative” included patients with SAH missed on initial non-contrast CT, and CTA or LP could be considered for follow-up. Either of the two strategies could give true positive or false negative results. Failed cases of LP were included, with the assumption that the patient would subsequently undergo a CTA if LP could not be performed successfully. From true negative results of both CTA and LP, patients without SAH might still experience complications of their respective follow-up procedure. In true negatives of LP (without SAH), the possibility of an undetected aneurysms rupturing in the future, was discussed. The false positive results of CTA represented findings of incidental aneurysms that had not ruptured and were not responsible for acute SAH. These aneurysms could then be managed conservatively or surgically, with risk of rupturing associated with conservative management, and complications associated with both. Conservative treatment of incidental aneurysms was defined as 5 follow-up CTAs [9]. Surgical management was defined as coiling of the unruptured aneurysm, as it is more commonly performed than clipping and associated with lower cost in the short term and complication rate [10]. The complications from coiling were hydrocephalus, postoperative stroke, intracerebral hemorrhage, and death. The false positive results of LP represented traumatic taps and these patients would normally undergo a further CTA
as LP results cannot be relied upon [11]. These patients were further grouped according to the presence and absence of incidental aneurysms, and sensitivity and specificity of CTA were considered respectively. The sensitivity and specificity of two CTAs combined were assumed to be 100% since the probability of two consecutive false positives or false negatives would be on a scale of 10−5 . The true positive results were considered as treated aneurysms for both strategies (CTA and LP) and false negatives as untreated aneurysms for CTA, with the untreated case having a significantly higher cost and lower utility. Terminal nodes were assigned to each arm, with specified cost and effectiveness. The complete model of this cost-effectiveness is provided in Fig. 1and Fig. S1. 2.3. Model parameters 2.3.1. Diagnostic test parameters Initial CT scan was assigned a negative predictive value for SAH of 99.82%, based on a average sensitivity of 98%, a specificity of 100%, and an SAH prevalence of 8.41% [2–4,12]. CTA was assigned a sensitivity for aneurysm detection of 99.2%, based on a metaanalysis of 16 and 64 row scanners with more than 3600 patients [13]. LP was assigned a specificity of 86.7%, reflecting an incidence of 13.3% traumatic taps, and a sensitivity of 100% [11,14,15]. 2.3.2. Clinical probabilities Each probability in this model was based on published literature values. The probability of reaching each terminal node was calculated along the path from the starting decision node, with Bayesian
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
107
Fig. 3. Monte Carlo simulation scatter plot of 10,000 iterations. In reference to LP, CTA has negative incremental effectiveness and a positive incremental cost in all iterations, rendering it the suboptimal choice.
updating when necessary. If “Aneurysm” was preceded by a “False Negative” node, the outcome probabilities were constructed to reflect the poor natural history of an untreated ruptured aneurysm. If aneurysms were detected from the “True Positive” nodes, probabilities were constructed to reflect the outcome of patients with treated ruptured aneurysms. Complications of CTA are related to iodinated contrast administration, specifically allergic-like reaction and contrast-induced nephropathy. Radiation-associated stochastic risks were not included in the model [16,17]. Complications of LP include post-procedural headache, meningitis, and spinal canal hematoma (subdural as well as epidural). The incidence of postprocedural headache was based on the data reported for lumbar punctures performed using non-cutting needles [18]. The incidence of meningitis following LP was assigned a 0 probability, as it is very rare and only few cases are reported in literature [18,19]. Most of these were reported before the use of face masks from Streptoccocus salivarius. The incidence has dropped significantly in the last few years with the routine use of face masks. Similarly, epidural and subdural hematomas, although possible complications of LP, are quite uncommon, usually small and only require conservative follow up [20]. For the purpose of this study, the incidence of hematoma was also assigned a probability of 0. The incidences of other complications were all derived from literature values.
Medicare reimbursement values when possible. Costs incurred from complications were derived from the literature. The cost of post-procedural headache treatment was a weighted average based on the proportion of these patients being treated with intravenous caffeine infusion, blood patch, and/or hospitalization. To minimize bias and subjectivity, each cost was assigned to have a normal distribution with a standard deviation of 10% in the Monte Carlo simulation.
2.4. Costs
Analyses were performed using TreeAge Pro Suite 2014 (TreeAge, Inc., Williamstown, Massachusetts). The costeffectiveness of each follow-up strategy was evaluated at the corresponding decision node and Monte Carlo simulation was
Total cost at each terminal node was derived by totaling all the costs along its path, which were determined using 2014 regional
2.5. Utilities Each terminal node was assigned a utility value according to the modified Rankin Scale (mRS) score and a value was based on Quality Adjusted Life Year (QALY). A mRS of 0 to 2 corresponded to a good outcome, mRS of 3–5 corresponded to a poor outcome, and mRS of 6 corresponded to death. The exact value for each outcome was obtained from Samsa et al. with a variation range for Monte Carlo simulation. A detailed list of all parameters, their values, distributions (when applicable), and references is presented in Table S1. 2.6. Statistical analyses
108
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
Fig. 4. One-way sensitivity analysis varying cost of LP.
performed over the range of each distributional variable with 10,000 iterations. 3. Results
as the more cost-effective strategy until its cost exceeds $364.45 (Fig. 4). A two-way sensitivity analysis was performed varying the costs of LP and CTA. The shaded regions correspond to the more costeffective strategy given the costs shown on the coordinates. (Fig. 5).
3.1. Base case scenario 4. Discussion In the base case scenario, the expected cost and utility of each strategy were used to evaluate the cost-effectiveness of each strategy. CTA was associated with a cost of $747 and an expected utility of 0.798603029; LP was associated with a cost of $504 and an expected utility of 0.799259526. LP was shown to be the optimal strategy from both the cost and the utility perspectives. The respective cost and expected utility of each strategy are presented in Fig. 2. 3.2. Monte Carlo simulation A Monte Carlo simulation was carried out with 10,000-iteration probabilistic sampling. Results demonstrate that LP is the more cost-effective strategy in 100% of the cases at the $50,000/QALY WTP threshold. A scatter plot of CTA against LP is presented in Fig. 3. 3.2.1. Sensitivity analysis A one-way sensitivity analysis was performed by varying the cost of LP assuming a cost of $498.02 for CTA, as it might differ from institution to institution. The result showed that LP remains
Lumbar puncture has long been regarded as the standard practice for thunderclap headaches in patients with negative noncontrast CT. However, CTA is being increasingly recommended to replace LP in headache patients with negative non-contrast CT scan. The supporting arguments favoring CTA over LP include the high sensitivity and specificity of CTA, and the low complication rate due to its non-invasive nature [8]. Our economic analysis shows that LP is more cost-effective than CTA, with a slightly higher utility and a lower cost. However, the difference in outcome is minimal, suggesting comparable clinical performance of the two tests. In the paper by McCormack et al., mathematical calculations based on probabilities were carried out to demonstrate a high sensitivity of CTA in detecting aneurysm and subsequently to conclude that CTA would be a better follow-up strategy as compared to LP [8]. However, a significant limitation in the calculations is the high probability of aneurysm detected by CTA being incidental, rather than causal, given a negative CT scan. Unlike LP, which is designed specifically to detect blood in cerebrospinal fluid, CTA is performed primarily to detect aneurysms, incidental or causal. With the parameters provided, the probability of the aneurysm
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
109
Fig. 5. Two-way sensitivity analysis varying costs of LP and CTA.
detected by CTA being the true cause of undetected SAH on CT is only approximately 35%, given a negative initial CT scan. Since follow-up is only performed after negative non-contrast CT scans, this result would call the utility of CTA follow-up into question. The detection of incidental aneurysms also raises the question of extensive follow-up against treatment of unruptured aneurysms, both of which are not cost or risk free. Another decision analysis on similar topic was performed by Ward et al., evaluating the cost-effectiveness of CT only, CT followed by CTA, CT followed by LP, and CT followed by MRA as imaging strategies for patients presenting with headaches [15]. However, the mathematical model assigned inaccurate complication rates to follow-up strategies, with 0 complication rate from LP and 11% contrast-induced nephropathy from CTA. In previous literature, the complication rate of LP can be as high as 4% for headache that requires hospitalization and blood patch [18]. On the other hand, the rate of contrast-induced nephropathy from CTA is low (reported as 0.02% in a study with nearly 20,000 patients)[21]. The discrepancy between the assigned values and the reported literature values renders the conclusion of questionable significance. In the model, LP was ascribed a 100% sensitivity in SAH detection, as LP has been regarded as a reference for SAH detection. However, LP is an imperfect reference standard with a 13.3% occurrence of traumatic taps performed at the bedside [14]. These also add to the dilemma as no clear-cut thresholds have been established in distinguishing a traumatic tap from a true positive result for SAH [11]. From our clinical experience, patients with traumatic taps are usually managed with a follow-up CTA by most ED physicians,
due to the high risk and poor outcome of undetected SAH. Some institutions might opt to perform digital subtraction angiography (DSA) rather than CTA, as it is regarded as the gold standard for aneurysms. However, neither DSA and CTA are designed to detect for blood, and CTA now with the current 16 and 64 row scanners has a submillimeter resolution of 0.5–0.7 mm [13]. In the context where perimesencephalic SAH is a concern, our past research has also shown CTA alone to be sufficient for diagnosis [22]. The traumatic taps should be considered in the context of extensive follow-up procedures and associated complications and costs to make the model more fair. Similar to McCormack et al., Ward et al. also did not discuss the possibility of detecting incidental aneurysms in patients without SAH, which is a significant limitation of just angiographic (CTA or MRA) follow-up. A wide range of failed LP rates have been reported in literature, being more common in obese and older patients [23,24]. A failed LP attempt at bedside prompts imaging guided LP using fluoroscopy which is associated with a much lower failure rate. Failed fluoroscopy-guided LP, though unusual, are followed by CTA. The complication rate with LP remains the same, whether failed or successful. Another significant factor in clinical decision-making between LP and CTA would be the time required to complete the tests, especially in EDs. One might argue that LP would be a more time-consuming procedure and that CTA may be preferred by the emergency room physician. However, time is a subjective measure, and the amount of time required may differ due to various institutional practices. Getting a CTA might require less time than
110
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111
performing an LP for an ED physician, but in our clinical experience, LP and CTA take approximately the same amount of time overall for making final management decisions. This is because the lab analysis results of spinal fluid may take less time than the final CTA read by radiologists. Thus, both strategies require roughly the same amount of time to produce the official report. One limitation of this model would be the costs assigned to LP and CTA, as these values usually differ in hospitals. Since our paper puts the emphasis on the overall cost-effectiveness of the two approaches to the healthcare system, the values were derived from the 2014 Medicare reimbursement values. It has also been a standard practice in cost-effectiveness analyses to make use of the Medicare reimbursement value, which might represent a lower bound of the actual cost incurred to the payer [15,25,26]. The actual costs incurred to patients might different significantly from the reimbursement value, but this study was not conducted from an individual’s perspective. The cost result would be more variable depending on the nature of the hospital, the local cost of living, and so on. The cost difference is likely to be enlarged in scale when out-of-pocket costs are used. In addition, sensitivity analyses were carried out to assess the robustness of the conclusion against the costs of LP and CTA. With an addition of $99.74 from fluoroscopy guidance, the cost of LP is still lower than the $364.45 threshold. Even when the actual cost of LP exceeds the threshold, LP would still be a better strategy when considering the expected effectiveness (in QALYs) for patients. MRA is not discussed in this model as a possible follow-up strategy, although it avoids complications related to iodinated contrast. MRA has limited availability in ERs, safety concerns, lower sensitivity and specificity and higher costs than CTA [27]. Although the cost and utility of each follow-up strategy were fully accounted for by including all possible scenarios and respective outcomes, the analysis does not discuss the different amounts of time required to perform LP or CTA. Another limitation of the study is that opportunity costs are not factored into the strategies. Indirect costs, such as lost time from work due to post LP headache, are difficult to assess and incorporate. Also, Medicare reimbursement may not be the most accurate indicator of actual costs for a middle-aged patient. Cross-sectional imaging can also reveal other etiologies for the presenting headaches, something that the LP strategy would not be able to identify. We did not consider surgical clipping in the management of detected aneurysms. The long term costs of clipping might be lower than coiling. However, increasingly aneurysms are being treated by coiling and the current modelling was done with a one year time horizon [28].
5. Conclusions LP following a negative initial non-contrast CT is the dominant strategy for diagnosing SAH in patients with thunderclap headache in the ED. In this clinical scenario CT/CTA is not an effective replacement from a utility or cost perspective.
Funding None.
Acknowledgement None.
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.clineuro.2015.12. 021.
References [1] R.G. Kowalski, J. Claassen, K.T. Kreiter, J.E. Bates, N.D. Ostapkovich, E.S. Connolly, S.A. Mayer, Initial misdiagnosis and outcome after subarachnoid hemorrhage, Jama 291 (2004) 866–869. [2] B.M. Boesiger, J.R. Shiber, Subarachnoid hemorrhage diagnosis by computed tomography and lumbar puncture: are fifth generation CT scanners better at identifying subarachnoid hemorrhage? J. Emerg. Med. 29 (2005) 23–27. [3] R.L. Byyny, W.R. Mower, N. Shum, G.Z. Gabayan, S. Fang, L.J. Baraff, Sensitivity of noncontrast cranial computed tomography for the emergency department diagnosis of subarachnoid hemorrhage, Ann. Emerg. Med. 51 (2008) 697–703. [4] J.J. Perry, I.G. Stiell, M.L.A. Sivilotti, M.J. Bullard, M. Émond, C. Symington, J. Sutherland, A. Worster, C. Hohl, J.S. Lee, M.A. Eisenhauer, M. Mortensen, D. Mackey, M. Pauls, H. Lesiuk, G.A. Wells, Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study, BMJ 343 (2011) d4277. [5] N. van der Wee, G.J. Rinkel, D. Hasan, J. van Gijn, Detection of subarachnoid haemorrhage on early CT: is lumbar puncture still needed after a negative scan? J. Neurol. Neurosurg. Psychiatry 58 (1995) 357–359. [6] J.B. Bederson, E.S. Connolly, H.H. Batjer, R.G. Dacey, J.E. Dion, M.N. Diringer, J.E. Duldner, R.E. Harbaugh, A.B. Patel, R.H. Rosenwasser, Guidelines for the management of aneurysmal subarachnoid hemorrhage: a statement for healthcare professionals from a special writing group of the stroke council, American heart association, Stroke 40 (2009) 994–1025. [7] Douglas AC, Wippold II FJ, Broderick DF, et al. ACR Appropriateness Criteria® Acute Trauma to Headache. Available at https://acsearch.acr.org/docs/69482/ Narrative/. American College of Radiology. Accessed Nov 29, 2015. [8] R.F. McCormack, A. Hutson, Can computed tomography angiography of the brain replace lumbar puncture in the evaluation of acute-onset headache after a negative noncontrast cranial computed tomography scan? Acad. Emerg. Med. 17 (2010) 444–451. [9] R.D. Brown Jr., J.P. Broderick, Unruptured intracranial aneurysms: epidemiology, natural history, management options, and familial screening, Lancet Neurol. 13 (2014) 393–404. [10] I.H. Wadd, A. Haroon, S. Habibullah Ansari, S. Mukhtar, U. Rashid, A.H. Vohra, Aneurysmal subarachnoid hemorrhage: outcome of aneurysm clipping versus coiling in anterior circulation aneurysm, J. Coll Phys. Surg. Pak. 25 (2015) 798–801. [11] J. Gorchynski, J. Oman, T. Newton, Interpretation of traumatic lumbar punctures in the setting of possible subarachnoid hemorrhage: who can be safely discharged, Cal. J. Emerg. Med. 8 (2007) 3–7. [12] J.J. Perry, I.G. Stiell, M.L.A. Sivilotti, M.J. Bullard, J.S. Lee, M. Eisenhauer, C. Symington, M. Mortensen, J. Sutherland, H. Lesiuk, G.A. Wells, High risk clinical characteristics for subarachnoid haemorrhage in patients with acute headache: prospective cohort study, BMJ 341 (2010) c5204. [13] J. Menke, J. Larsen, K. Kallenberg, Diagnosing cerebral aneurysms by computed tomographic angiography: meta-analysis, Ann. Neurol. 69 (2011) 646–654. [14] K.H. Shah, K.M. Richard, S. Nicholas, J.A. Edlow, Incidence of traumatic lumbar puncture, Acad. Emerg. Med. 10 (2003) 151–154. [15] M.J. Ward, J.B. Bonomo, O. Adeoye, A.S. Raja, J.M. Pines, Cost-effectiveness of diagnostic strategies for evaluation of suspected subarachnoid hemorrhage in the emergency department, Acad. Emerg. Med. 19 (2012) 1134–1144. [16] W. Hendee, Risk of medical imaging, Med. Phys. 40 (2013) 040401. [17] A. Sabarudin, M.Z. Yusof, M. Mohamad, Z. Sun, Radiation dose associated with cerebral CT angiography and CT perfusion: an experimental phantom study, Radiat. Prot. Dosim. (2013). [18] Y. Dakka, N. Warra, R.J. Albadareen, M. Jankowski, B. Silver, Headache rate and cost of care following lumbar puncture at a single tertiary care hospital, Neurology 77 (2011) 71–74. [19] H. Zetterberg, K. Tullhög, O. Hansson, L. Minthon, E. Londos, K. Blennow, Low incidence of post-lumbar puncture headache in 1089 consecutive memory clinic patients, Eur. Neurol. 63 (2010) 326–330. [20] D.J. Gaucher Jr., J.A. Perez Jr., Subdural hematoma following lumbar puncture, Arch. Intern. Med. 162 (2002) 1904–1905. [21] T.J. Kaufmann, J. Huston, J.N. Mandrekar, C.D. Schleck, K.R. Thielen, D.F. Kallmes, Complications of diagnostic cerebral angiography: evaluation of 19,826 consecutive patients, Radiology 243 (2007) 812–819. [22] V.B. Kalra, X. Wu, H.P. Forman, A. Malhotra, Cost-effectiveness of angiographic imaging in isolated perimesencephalic subarachnoid hemorrhage, Stroke 45 (2014) 3576–3582. [23] S.D. Yu, M.Y. Chen, A.J. Johnson, Factors associated with traumatic fluoroscopy-guided lumbar punctures: a retrospective review, Am. J. Neuroradiol. 30 (2009) 512–515. [24] J.T. Nomura, S.J. Leech, S. Shenbagamurthi, P.R. Sierzenski, R.E. O’Connor, M. Bollinger, M. Humphrey, J.A. Gukhool, A randomized controlled trial of ultrasound-assisted lumbar puncture, J. Ultrasound Med. 26 (2007) 1341–1348.
X. Wu et al. / Clinical Neurology and Neurosurgery 142 (2016) 104–111 [25] P.C. Sanelli, A. Pandya, A.Z. Segal, A. Gupta, S. Hurtado-Rua, J. Ivanidze, K. Kesavabhotla, D. Mir, A.I. Mushlin, M.G. Hunink, Cost-effectiveness of CT angiography and perfusion imaging for delayed cerebral ischemia and vasospasm in aneurysmal subarachnoid hemorrhage, Am. J. Neuroradiol. (2014). [26] C.E. Tung, Y.T. So, M.G. Lansberg, Cost comparison between the atraumatic and cutting lumbar puncture needles, Neurology 78 (2012) 109–113. [27] M. Okahara, H. Kiyosue, M. Yamashita, H. Nagatomi, H. Hata, T. Saginoya, Y. Sagara, H. Mori, Diagnostic accuracy of magnetic resonance angiography for
111
cerebral aneurysms in correlation with 3D–digital subtraction angiographic images: a study of 133 aneurysms. Stroke, J. Cereb. Circ. 33 (2002) 1803–1808. [28] N. Andaluz, M. Zuccarello, Recent trends in the treatment of cerebral aneurysms: analysis of a nationwide inpatient database, J. Neurosurg. 108 (2008) 1163–1169.