- Email: [email protected]
Commentary on ‘Predictors for Recanalization of the Great Saphenous Vein in RCTs 1 Year After Endovenous Thermal Ablation’: The Dark Side of Systematic Reviews
Eur J Vasc Endovasc Surg (2016)
-,
1
INVITED COMMENTARY
Commentary on ‘Predictors for Recanalization of the Great Saphenous Vein in RCTs 1 Year After Endovenous Thermal Ablation’: The Dark Side of Systematic Reviews M. Cairols Department of Vascular Surgery, Hospital Universitari de Bellvitge, Barcelona, Spain “Natura inest in mentibus nostris insatiabilis quaedam cupiditas very vivendi” (Cicero, Tusculanes, 1, 19) “Nature has put in our minds an insatiable desire to know the truth”
The paper by van der Velden et al., authors with extensive experience of chronic superficial venous insufficiency (CVI) therapy, published in this issue, is excellent.1 Beyond congratulating the editors of EJVES for its publication, the world of venous publications gives us the opportunity to comment on systematic review papers. It is obvious that endovenous procedures have been introduced for treating varicose veins in order to reduce post-operative complications. Their main advantages are quicker recovery, fewer complications, and improvement in quality of life compared with conventional surgery.2 As has been the rule, many physicians around the world have enthusiastically adopted those new methods. Nevertheless, as is clear in the paper by van der Velden et al., we do not know much about the prognostic factors for success.1,3 Moreover, is there an appropriate patient for these technologies? Despite this obvious limited knowledge, almost all doctors, sometimes not specialists, have dared to manage these patients. The previous comments support the fact that this widespread use of endovascular procedures produces a huge variety of papers, probably to back their practice. Unfortunately, numerous papers are published, but papers with adequate methodology are few, as proven in many systematic reviews.2,4,5 As previously mentioned, the majority of these reviews have favoured endovenous procedures; however, none of them gives information regarding how many patients are not appropriate for endovenous treatment.This lack of information introduces bias, as pooling effects are limited to long saphenous vein (LSV) outcome. The paper by van Der Velden et al. also proves what has already been stated: there are too many venous publications under the heading of endothermal ablation with methodological flaws.1 Moreover, not all published papers have enough information with which to assess results properly and, surprisingly, these low quality papers are accepted in many reputable medical journals. Lack of evidence is particularly common in venous publications for almost all comparisons among different procedures (radiofrequency ablation [RFA], endovenous laser ablation, surgery, and ultrasound venous ablation).5 van Der Velden et al.’s paper looks at a relevant problem: prognostic factors for recanalisation of the LSV after endothermal ablation.1 However, should we consider LSV recanalisation as the only parameter for success or failure? Many readers will agree that it is a relevant parameter with which to explain the recurrence of varicose veins, but recurrence of varicose veins is a complex phenomenon in which many other factors contribute, among them below knee LSV reflux, reflux in the tributaries, neovascularisation, saphenopopliteal reflux, and perforator insufficiency. Neovascularisation might differ between the procedures, which may influence long-term effectiveness between endovenous techniques and surgery, in addition to primary failure. By only pooling primary failure, and not reflux in the tributaries and neovascularisation, as composite outcomes of clinical recurrence, the result is dilution of the efficacy of EVLA and RFA for the LSV.5 Therefore, primary failure of the treated LSV is a surrogate outcome of clinical recurrence and not the complete picture. Another source of bias is the fact that the medical literature is more likely to contain trials showing beneficial effects of treatments. Should it not be equally valid to publish trials showing no effect? Indeed, this is a main concern of meta-analyses. Why are negative effect papers not published? Are they a consequence of subtle suggestion from industry? Perhaps journal editors do DOI of original article: http://dx.doi.org/10.1016/j.ejvs.2016.01.021 E-mail address: [email protected] (M. Cairols). 1078-5884/Ó 2016 European Society for Vascular Surgery. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ejvs.2016.04.012
not like their journals to be identified as a negative publication? Again, the results of “positive” trials are sometimes reported more than once, increasing the probability that they will be included in meta-analyses (multiple publication bias). These biases are likely to affect smaller studies to a greater degree than large trials. The potentially serious consequences of such publication bias may have deleterious consequences on clinical practice. Perhaps a worldwide registration of clinical trials at inception may reduce these undesired effects. Unfortunately, I am pessimistic that this type of registry will be widely instituted in the foreseeable future. There is no question that a failure to publish negative results will inappropriately influence the evolution of clinical practice and so it falls on all of us to make a change.6 Publication bias has long been associated with funnel plot asymmetry. A funnel plot is a graph designed to check for the existence of publication bias.7 Besides the influence of results, another source of bias may be English language, as many publications with negative findings are likely published in languages other than English.Therefore, negative studies are quoted less frequently and are likely to be missed in the search for relevant trials. Another source of asymmetry in funnel plots arises from differences in methodological quality. Smaller studies are frequently performed and evaluated with less methodological robustness than larger studies. However, in the absence of large, conclusive randomised controlled trials for most medical interventions, systematic reviews based on the available randomised controlled trials are clearly the best strategy. Egger’s technique provides a reproducible measure for the likely presence, or apparent absence, of such biases. Although the basic conclusion is that there are patients who have certain characteristics that may result in an unsuccessful procedure, the study by van der Velden has several limitations.1 The first is a lack of external validation of the final models, and therefore the generalisability of the models remains uncertain. Second, the generalisability of the developed models might be affected by some exclusion criteria of this randomised controlled trial cohort. Several studies did not represent the entire varicose vein population because legs with C3eC6 disease or a large diameter saphenous vein (>12 mm) were excluded. As stated, although there is a dark side of the systematic review, nature wants us to continue the search of truth. REFERENCES 1 van der Velden S, Lawaetz M, De Maeseneer M, Hollestein L, Nijsten T, van den Bos R. Predictors for recanalization of the great saphenous vein in RCTs one year after endovenous thermal ablation. Eur J Vasc Endovasc Surg 2016. 2 Nesbitt C, Bedenis R, Bhattacharya V, Stansby G. Endovenous ablation (radiofrequency and laser) and foam sclerotherapy versus open surgery for great saphenous vein varices (review). Cochrane Database Syst Rev 2014;30:CD005624. 3 Theivacumar NS, Dellagrammaticas D, Beale RJ, Mavor AID, Gough MJ. Factors influencing the effectiveness of endovenous laser ablation (EVLA) in the treatment of great saphenous vein reflux. Eur J Vasc Endovasc Surg 2008;35:119e23. 4 van den Bos RR, Kockaert MA, Neumann HAM, Nijsten T. Technical review of endovenous laser therapy for varicose veins. Eur J Vasc Endovasc Surg 2008;35:88e95. 5 Siribumrungwong PB, Noorit C, Wilasrusmee J, Attia A, Thakkinstian A. Systematic review and meta-analysis of randomised controlled trials comparing endovenous ablation and surgical intervention in patients with varicose vein. Eur J Vasc Endovasc Surg 2012;44:214e23. 6 Mastracci TTM. Scientific methods and the reporting of negative results: critically important for patients safety. Eur J Vasc Endovasc Surg 2016;51:165e6. 7 Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta analysis detected by a simple, graphical test. BMJ 1997;13:629e34.
Please cite this article in press as: Cairols M, Commentary on ‘Predictors for Recanalization of the Great Saphenous Vein in RCTs 1 Year After Endovenous Thermal Ablation’: The Dark Side of Systematic Reviews, European Journal of Vascular and Endovascular Surgery (2016), http://dx.doi.org/10.1016/ j.ejvs.2016.04.012
-,
1
INVITED COMMENTARY
Commentary on ‘Predictors for Recanalization of the Great Saphenous Vein in RCTs 1 Year After Endovenous Thermal Ablation’: The Dark Side of Systematic Reviews M. Cairols Department of Vascular Surgery, Hospital Universitari de Bellvitge, Barcelona, Spain “Natura inest in mentibus nostris insatiabilis quaedam cupiditas very vivendi” (Cicero, Tusculanes, 1, 19) “Nature has put in our minds an insatiable desire to know the truth”
The paper by van der Velden et al., authors with extensive experience of chronic superficial venous insufficiency (CVI) therapy, published in this issue, is excellent.1 Beyond congratulating the editors of EJVES for its publication, the world of venous publications gives us the opportunity to comment on systematic review papers. It is obvious that endovenous procedures have been introduced for treating varicose veins in order to reduce post-operative complications. Their main advantages are quicker recovery, fewer complications, and improvement in quality of life compared with conventional surgery.2 As has been the rule, many physicians around the world have enthusiastically adopted those new methods. Nevertheless, as is clear in the paper by van der Velden et al., we do not know much about the prognostic factors for success.1,3 Moreover, is there an appropriate patient for these technologies? Despite this obvious limited knowledge, almost all doctors, sometimes not specialists, have dared to manage these patients. The previous comments support the fact that this widespread use of endovascular procedures produces a huge variety of papers, probably to back their practice. Unfortunately, numerous papers are published, but papers with adequate methodology are few, as proven in many systematic reviews.2,4,5 As previously mentioned, the majority of these reviews have favoured endovenous procedures; however, none of them gives information regarding how many patients are not appropriate for endovenous treatment.This lack of information introduces bias, as pooling effects are limited to long saphenous vein (LSV) outcome. The paper by van Der Velden et al. also proves what has already been stated: there are too many venous publications under the heading of endothermal ablation with methodological flaws.1 Moreover, not all published papers have enough information with which to assess results properly and, surprisingly, these low quality papers are accepted in many reputable medical journals. Lack of evidence is particularly common in venous publications for almost all comparisons among different procedures (radiofrequency ablation [RFA], endovenous laser ablation, surgery, and ultrasound venous ablation).5 van Der Velden et al.’s paper looks at a relevant problem: prognostic factors for recanalisation of the LSV after endothermal ablation.1 However, should we consider LSV recanalisation as the only parameter for success or failure? Many readers will agree that it is a relevant parameter with which to explain the recurrence of varicose veins, but recurrence of varicose veins is a complex phenomenon in which many other factors contribute, among them below knee LSV reflux, reflux in the tributaries, neovascularisation, saphenopopliteal reflux, and perforator insufficiency. Neovascularisation might differ between the procedures, which may influence long-term effectiveness between endovenous techniques and surgery, in addition to primary failure. By only pooling primary failure, and not reflux in the tributaries and neovascularisation, as composite outcomes of clinical recurrence, the result is dilution of the efficacy of EVLA and RFA for the LSV.5 Therefore, primary failure of the treated LSV is a surrogate outcome of clinical recurrence and not the complete picture. Another source of bias is the fact that the medical literature is more likely to contain trials showing beneficial effects of treatments. Should it not be equally valid to publish trials showing no effect? Indeed, this is a main concern of meta-analyses. Why are negative effect papers not published? Are they a consequence of subtle suggestion from industry? Perhaps journal editors do DOI of original article: http://dx.doi.org/10.1016/j.ejvs.2016.01.021 E-mail address: [email protected] (M. Cairols). 1078-5884/Ó 2016 European Society for Vascular Surgery. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ejvs.2016.04.012
not like their journals to be identified as a negative publication? Again, the results of “positive” trials are sometimes reported more than once, increasing the probability that they will be included in meta-analyses (multiple publication bias). These biases are likely to affect smaller studies to a greater degree than large trials. The potentially serious consequences of such publication bias may have deleterious consequences on clinical practice. Perhaps a worldwide registration of clinical trials at inception may reduce these undesired effects. Unfortunately, I am pessimistic that this type of registry will be widely instituted in the foreseeable future. There is no question that a failure to publish negative results will inappropriately influence the evolution of clinical practice and so it falls on all of us to make a change.6 Publication bias has long been associated with funnel plot asymmetry. A funnel plot is a graph designed to check for the existence of publication bias.7 Besides the influence of results, another source of bias may be English language, as many publications with negative findings are likely published in languages other than English.Therefore, negative studies are quoted less frequently and are likely to be missed in the search for relevant trials. Another source of asymmetry in funnel plots arises from differences in methodological quality. Smaller studies are frequently performed and evaluated with less methodological robustness than larger studies. However, in the absence of large, conclusive randomised controlled trials for most medical interventions, systematic reviews based on the available randomised controlled trials are clearly the best strategy. Egger’s technique provides a reproducible measure for the likely presence, or apparent absence, of such biases. Although the basic conclusion is that there are patients who have certain characteristics that may result in an unsuccessful procedure, the study by van der Velden has several limitations.1 The first is a lack of external validation of the final models, and therefore the generalisability of the models remains uncertain. Second, the generalisability of the developed models might be affected by some exclusion criteria of this randomised controlled trial cohort. Several studies did not represent the entire varicose vein population because legs with C3eC6 disease or a large diameter saphenous vein (>12 mm) were excluded. As stated, although there is a dark side of the systematic review, nature wants us to continue the search of truth. REFERENCES 1 van der Velden S, Lawaetz M, De Maeseneer M, Hollestein L, Nijsten T, van den Bos R. Predictors for recanalization of the great saphenous vein in RCTs one year after endovenous thermal ablation. Eur J Vasc Endovasc Surg 2016. 2 Nesbitt C, Bedenis R, Bhattacharya V, Stansby G. Endovenous ablation (radiofrequency and laser) and foam sclerotherapy versus open surgery for great saphenous vein varices (review). Cochrane Database Syst Rev 2014;30:CD005624. 3 Theivacumar NS, Dellagrammaticas D, Beale RJ, Mavor AID, Gough MJ. Factors influencing the effectiveness of endovenous laser ablation (EVLA) in the treatment of great saphenous vein reflux. Eur J Vasc Endovasc Surg 2008;35:119e23. 4 van den Bos RR, Kockaert MA, Neumann HAM, Nijsten T. Technical review of endovenous laser therapy for varicose veins. Eur J Vasc Endovasc Surg 2008;35:88e95. 5 Siribumrungwong PB, Noorit C, Wilasrusmee J, Attia A, Thakkinstian A. Systematic review and meta-analysis of randomised controlled trials comparing endovenous ablation and surgical intervention in patients with varicose vein. Eur J Vasc Endovasc Surg 2012;44:214e23. 6 Mastracci TTM. Scientific methods and the reporting of negative results: critically important for patients safety. Eur J Vasc Endovasc Surg 2016;51:165e6. 7 Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta analysis detected by a simple, graphical test. BMJ 1997;13:629e34.
Please cite this article in press as: Cairols M, Commentary on ‘Predictors for Recanalization of the Great Saphenous Vein in RCTs 1 Year After Endovenous Thermal Ablation’: The Dark Side of Systematic Reviews, European Journal of Vascular and Endovascular Surgery (2016), http://dx.doi.org/10.1016/ j.ejvs.2016.04.012