- Email: [email protected]
Further medical experience will be required to validate these results: How experience -based medicine shapes the validity of medical evidence
Journal Pre-proof Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence Darius BAGLI, Kathrine HERBST, Luke HARPER, Goedele BECKERS, Martin KAEFER, Nicolas KALFA, Magdalena FOSSUM, On behalf of the ESPU research committee PII:
S1477-5131(19)30342-0
DOI:
https://doi.org/10.1016/j.jpurol.2019.10.022
Reference:
JPUROL 3306
To appear in:
Journal of Pediatric Urology
Received Date: 2 September 2019 Accepted Date: 23 October 2019
Please cite this article as: BAGLI D, HERBST K, HARPER L, BECKERS G, KAEFER M, KALFA N, FOSSUM M, On behalf of the ESPU research committee, Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence Journal of Pediatric Urology, https://doi.org/10.1016/j.jpurol.2019.10.022. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier Ltd on behalf of Journal of Pediatric Urology Company.
Title: Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence 1. Darius BAGLI Division of Urology, Departments of Surgery and Physiology, University of Toronto, Developmental and Stem Cell Biology, The Hospital for Sick Children and Research Institute, Toronto, Ontario, Canada. 2. Kathrine HERBST Division of Urology, Department of Research, Connecticut Children's Medical Center, Hartford, CT, United States 3. Luke HARPER Service de Chirurgie Pédiatrique, Hôpital Pellegrin-Enfants, CHU de Bordeaux, France. 4. Goedele BECKERS Department of Urology, Pediatric Urology Section, Amsterdam University Medical Centre Location VUmc, Amsterdam, the Netherlands. 5. Martin KAEFER Riley Hospital for Children, Indiana University, Indianapolis, IN, United States. 6. Nicolas KALFA Service de Chirurgie Urologique Pédiatrique, Hôpital Lapeyronie, CHU de Montpellier, Université, de Montpellier, France. 7. Magdalena FOSSUM Department of Pediatric Surgery, Surgical Clinic C, Copenhagen University Hospital Rigshospitalet, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark and Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden Corresponding author: Magdalena Fossum Bioclinicum floor 11, Department of Women's and Children's Health, Karolinska Institutet, 171 76 Stockholm, Sweden. Electronic address: [email protected]. On behalf of the ESPU research committee.
Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence
In 1992 there appeared in the Journal of The American Medical Association a landmark article entitled Evidence-Based Medicine (EvBM): A New Approach To Teaching The Practice Of Medicine [1]. This marked the soft launch of an EvBM movement. In the ensuing years, every corner of medicine has been charged by payers, administrators, quality leaders, department heads, and academia with justifying medical practice and its value to patients through the use of evidence. However, the well-known limitations of EvBM and the very literature on which it is ‘based’ eventually spawned a backlash from practitioners defending the importance of individual and collective experience in patient care. This latter position has come to be known as experience-based medicine (ExBM). Ironically, ExBM was the only medicine to exist before EvBM, and arguably remains the basis of the profession worldwide. So, if neither is superior, how should ExBM and EvBM co-exist? A first EvBM concept to emphasize is that evidence is never absolute, nor absolutely good, or bad. It is important to remember the context under which evidence is generated. For example, a novel testicular de-torsion technique may show a highly improved testis salvage rate. However, if the subjects were all in an urban center and presented to surgery quickly, the same evidence may not prove valid if the technique is performed on a rural population with much longer transport times to the surgical center. Evidence thus must always be considered in the context of the patients seen in the experience of the practitioner. Indeed even in the most rigorously performed trials—the gold-standard Double Blind- RCTs, each and every individual patient or subject is handled by someone with a given degree of experience. Every study will possess unrecognized confounding variables and effect modifiers, even if these have been identified during study design. It is impossible to account for and control everything that may affect a study patient in the real-world, or affect later patients to whom the evidence is meant to apply. That is the role of experience. This is not to suggest, however, that the stories from experience necessarily take priority over all evidence. EvBM and ExBM are both essential to medicine and it is important to understand ways they complement each other.
ExBM should not only critique evidence in the usual terms such as sample size, intervention timing, subgroups, stratifications, confounders, and the like. ExBM should motivate a thoughtful consideration of the real world to which the EvBM is hoped to apply. Consider evidence touting a superior surgical technique. The general evidentiary reaction is that a single surgeon series limits variability so such evidence must be stronger than a multiple surgeon design. Again, context is critical when considering evidence, and context derives from experience. If the context is the technique itself: ie, does it work? then single surgeon evidence is logical. If the context is real-world robustness, then whether the same technique lives or dies in the hands of a variety of operators is the evidence we seek. ExBM is also all we have when there is poor or no evidence. The intoxication of the ‘significant p value’ is now being tempered by another parameter: the fragility index (FI), the number of subjects with the opposite outcome needed to lose statistical significance. Commonly FI’s can yield as few as 1 or 2 subjects to flip significance to non-significance. A related value for studies with large numbers of subjects, the Fragility Quotient (FI / total number of subjects), similarly unpacks “significance”. Indeed, if much of our current statistically significant evidence were cockroaches, the FI will almost certainly become the light which sends significant p-values racing for the cover of darkness. A career built on listening and learning from patients, while carefully applying whatever evidence published research can provide is the basis of modern medicine. In some ways, the term itself ‘EvBM’ creates unrealistic expectations, the same way ‘autonomous driving’ systems in a physician’s car are truly only ‘driver-assist’ systems. EvBM should more aptly be applied as Evidence-Informed Care. In both the car and care, the physician’s judgment must maintain primal control over all information and all decisions. REFERENCES: 1. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992 Nov 4;268(17):2420-5.
S1477-5131(19)30342-0
DOI:
https://doi.org/10.1016/j.jpurol.2019.10.022
Reference:
JPUROL 3306
To appear in:
Journal of Pediatric Urology
Received Date: 2 September 2019 Accepted Date: 23 October 2019
Please cite this article as: BAGLI D, HERBST K, HARPER L, BECKERS G, KAEFER M, KALFA N, FOSSUM M, On behalf of the ESPU research committee, Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence Journal of Pediatric Urology, https://doi.org/10.1016/j.jpurol.2019.10.022. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier Ltd on behalf of Journal of Pediatric Urology Company.
Title: Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence 1. Darius BAGLI Division of Urology, Departments of Surgery and Physiology, University of Toronto, Developmental and Stem Cell Biology, The Hospital for Sick Children and Research Institute, Toronto, Ontario, Canada. 2. Kathrine HERBST Division of Urology, Department of Research, Connecticut Children's Medical Center, Hartford, CT, United States 3. Luke HARPER Service de Chirurgie Pédiatrique, Hôpital Pellegrin-Enfants, CHU de Bordeaux, France. 4. Goedele BECKERS Department of Urology, Pediatric Urology Section, Amsterdam University Medical Centre Location VUmc, Amsterdam, the Netherlands. 5. Martin KAEFER Riley Hospital for Children, Indiana University, Indianapolis, IN, United States. 6. Nicolas KALFA Service de Chirurgie Urologique Pédiatrique, Hôpital Lapeyronie, CHU de Montpellier, Université, de Montpellier, France. 7. Magdalena FOSSUM Department of Pediatric Surgery, Surgical Clinic C, Copenhagen University Hospital Rigshospitalet, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark and Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden Corresponding author: Magdalena Fossum Bioclinicum floor 11, Department of Women's and Children's Health, Karolinska Institutet, 171 76 Stockholm, Sweden. Electronic address: [email protected]. On behalf of the ESPU research committee.
Further medical experience will be required to validate these results: How Experience Based Medicine Shapes the Validity of Medical Evidence
In 1992 there appeared in the Journal of The American Medical Association a landmark article entitled Evidence-Based Medicine (EvBM): A New Approach To Teaching The Practice Of Medicine [1]. This marked the soft launch of an EvBM movement. In the ensuing years, every corner of medicine has been charged by payers, administrators, quality leaders, department heads, and academia with justifying medical practice and its value to patients through the use of evidence. However, the well-known limitations of EvBM and the very literature on which it is ‘based’ eventually spawned a backlash from practitioners defending the importance of individual and collective experience in patient care. This latter position has come to be known as experience-based medicine (ExBM). Ironically, ExBM was the only medicine to exist before EvBM, and arguably remains the basis of the profession worldwide. So, if neither is superior, how should ExBM and EvBM co-exist? A first EvBM concept to emphasize is that evidence is never absolute, nor absolutely good, or bad. It is important to remember the context under which evidence is generated. For example, a novel testicular de-torsion technique may show a highly improved testis salvage rate. However, if the subjects were all in an urban center and presented to surgery quickly, the same evidence may not prove valid if the technique is performed on a rural population with much longer transport times to the surgical center. Evidence thus must always be considered in the context of the patients seen in the experience of the practitioner. Indeed even in the most rigorously performed trials—the gold-standard Double Blind- RCTs, each and every individual patient or subject is handled by someone with a given degree of experience. Every study will possess unrecognized confounding variables and effect modifiers, even if these have been identified during study design. It is impossible to account for and control everything that may affect a study patient in the real-world, or affect later patients to whom the evidence is meant to apply. That is the role of experience. This is not to suggest, however, that the stories from experience necessarily take priority over all evidence. EvBM and ExBM are both essential to medicine and it is important to understand ways they complement each other.
ExBM should not only critique evidence in the usual terms such as sample size, intervention timing, subgroups, stratifications, confounders, and the like. ExBM should motivate a thoughtful consideration of the real world to which the EvBM is hoped to apply. Consider evidence touting a superior surgical technique. The general evidentiary reaction is that a single surgeon series limits variability so such evidence must be stronger than a multiple surgeon design. Again, context is critical when considering evidence, and context derives from experience. If the context is the technique itself: ie, does it work? then single surgeon evidence is logical. If the context is real-world robustness, then whether the same technique lives or dies in the hands of a variety of operators is the evidence we seek. ExBM is also all we have when there is poor or no evidence. The intoxication of the ‘significant p value’ is now being tempered by another parameter: the fragility index (FI), the number of subjects with the opposite outcome needed to lose statistical significance. Commonly FI’s can yield as few as 1 or 2 subjects to flip significance to non-significance. A related value for studies with large numbers of subjects, the Fragility Quotient (FI / total number of subjects), similarly unpacks “significance”. Indeed, if much of our current statistically significant evidence were cockroaches, the FI will almost certainly become the light which sends significant p-values racing for the cover of darkness. A career built on listening and learning from patients, while carefully applying whatever evidence published research can provide is the basis of modern medicine. In some ways, the term itself ‘EvBM’ creates unrealistic expectations, the same way ‘autonomous driving’ systems in a physician’s car are truly only ‘driver-assist’ systems. EvBM should more aptly be applied as Evidence-Informed Care. In both the car and care, the physician’s judgment must maintain primal control over all information and all decisions. REFERENCES: 1. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992 Nov 4;268(17):2420-5.