- Email: [email protected]
Distant healing: an unlikely hypothesis
Distant Healing: An Unlikely Hypothesis Warren S. Browner, MD, MPH, Lee Goldman, MD
I
n this issue of The Green Journal, Harkness and colleagues (1) report that distant healing was not effective at healing cutaneous warts. In the study, a group of self-identified healers were provided with the names of patients who had warts; other patients with warts served as controls. At the end of the study, there was no evidence that the patients who had received distant healing fared any better than those who had not. Perhaps not surprisingly, the manuscript led to considerable discussion at our editors’ meeting, as we debated the pros and cons of publishing a study of a therapy that none of the editors believed could have worked. Part of our job as editors, assisted by the peer reviewers, is to determine the “information content” of a manuscript. Not every study contains useful information, either because the research question is not interesting or relevant to our readers, or because the study was poorly executed, analyzed, or presented. Agreement with the editors’ beliefs, however, is not one of our criteria. How did we decide to publish the manuscript? One question that we addressed was whether we would have published the study had the results suggested that distant healing was effective. We also had concerns about publication bias—the likelihood that a positive study (one with a statistically significant result) is more likely to be published than a negative study (2). If there is publication bias, small positive studies are more likely to be published than small negative studies, thus biasing the estimated effect of a treatment in favor of benefit. Based on the assumption that smaller studies have greater variance in their results, graphic techniques have been developed to determine whether publication bias is likely (3). In the absence of publication bias, there should be an inverse correlation between the size of study and the likelihood that its results are similar to those of a meta-analysis. For studies of unusual therapies such as distant healing, the opposite problem—what might be called “reverse publication bias”—would occur if journal editors were less likely to accept positive studies. Ultimately, we decided that we would have published the study had it shown a
Am J Med. 2000;108:507–508. From the Department of Medicine, University of California, San Francisco, San Francisco, California. Correspondence should be addressed to Warren S. Browner, MD, Chief, General Internal Medicine, Veterans Affairs Medical Center, 4150 Clement Street, San Francisco, California 94121. 䉷2000 by Excerpta Medica, Inc. All rights reserved.
beneficial effect. Granted, such a decision was easier to make in the abstract than if we had been dealing with a positive study that made little sense scientifically, at least in the opinion of the editors. We did not ignore the question of whether a positive study would have made sense. Bayesian reasoning reminds us that just as a positive diagnostic test in a patient with a low prior probability of a disease does not prove that the patient has that disease, so too a single positive study of an unlikely hypothesis cannot establish that the therapy actually works (4). Nevertheless, if a series of well-done studies shows that distant healing or other seemingly “miraculous” types of treatments are effective, we would have to reassess currently accepted ideas about pathophysiology and therapy. Medical science would face a dilemma comparable with the one that occurred in the 16th century, when Copernicus developed the idea that the Earth revolves around the Sun. The challenge of explaining the Copernican solar system led to the discovery of gravity, a force that was previously unappreciated. A conclusive demonstration that distant healing works would likely require a similar revolution in our understanding of biology. It is our belief, however, that when the same standards required for trials of conventional medical treatments, such as thrombolysis or cholesterol lowering, are also applied to studies of distant healing and similar therapies, no revolution will be needed. Strict enrollment criteria, rigorous methods of randomization, adequate follow-up, intention-to-treat analyses, maintenance of blinding, and, perhaps most important of all, meaningful outcome variables are essential. For warts, cure is clearly the objective of treatment. For other conditions, “softer” endpoints, such as functional status or quality of life, may be more relevant; in these situations, the challenges of scientific rigor will be even more complex. Scientists and journal editors have a responsibility to be receptive to new ideas. Humility about the state of medical knowledge is essential. Only a few years ago, peptic ulcer was thought of as a stress-related disease, and many academic careers were built upon studies of acid secretion and hypersecretion in response to various stimuli. Still, skepticism and rigor must accompany the willingness to consider unlikely hypotheses. We promise to keep an open mind, but not so open that our brains fall out. 0002-9343/00/$–see front matter 507 PII S0002-9343(00)00325-9
Distant Healing: An Unlikely Hypothesis/Browner and Goldman
REFERENCES 1. Harkness EF, Abbot NC, Ernst E. A randomized trial of distant healing for warts. Am J Med. 2000;108:448 – 452. 2. Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data. J Royal Stat Soc A. 1988;151:419 – 463.
508
April 15, 2000
THE AMERICAN JOURNAL OF MEDICINE威
Volume 108
3. Light RJ, Pillemer DB. Summing Up: The Science of Reviewing Research. Cambridge, Mass: Harvard University Press; 1984:68 – 69. 4. Browner WS, Newman TB. Are all significant P values created equal? The analogy between diagnostic tests and clinical research. JAMA. 1987;257:2459 –2463.
I
n this issue of The Green Journal, Harkness and colleagues (1) report that distant healing was not effective at healing cutaneous warts. In the study, a group of self-identified healers were provided with the names of patients who had warts; other patients with warts served as controls. At the end of the study, there was no evidence that the patients who had received distant healing fared any better than those who had not. Perhaps not surprisingly, the manuscript led to considerable discussion at our editors’ meeting, as we debated the pros and cons of publishing a study of a therapy that none of the editors believed could have worked. Part of our job as editors, assisted by the peer reviewers, is to determine the “information content” of a manuscript. Not every study contains useful information, either because the research question is not interesting or relevant to our readers, or because the study was poorly executed, analyzed, or presented. Agreement with the editors’ beliefs, however, is not one of our criteria. How did we decide to publish the manuscript? One question that we addressed was whether we would have published the study had the results suggested that distant healing was effective. We also had concerns about publication bias—the likelihood that a positive study (one with a statistically significant result) is more likely to be published than a negative study (2). If there is publication bias, small positive studies are more likely to be published than small negative studies, thus biasing the estimated effect of a treatment in favor of benefit. Based on the assumption that smaller studies have greater variance in their results, graphic techniques have been developed to determine whether publication bias is likely (3). In the absence of publication bias, there should be an inverse correlation between the size of study and the likelihood that its results are similar to those of a meta-analysis. For studies of unusual therapies such as distant healing, the opposite problem—what might be called “reverse publication bias”—would occur if journal editors were less likely to accept positive studies. Ultimately, we decided that we would have published the study had it shown a
Am J Med. 2000;108:507–508. From the Department of Medicine, University of California, San Francisco, San Francisco, California. Correspondence should be addressed to Warren S. Browner, MD, Chief, General Internal Medicine, Veterans Affairs Medical Center, 4150 Clement Street, San Francisco, California 94121. 䉷2000 by Excerpta Medica, Inc. All rights reserved.
beneficial effect. Granted, such a decision was easier to make in the abstract than if we had been dealing with a positive study that made little sense scientifically, at least in the opinion of the editors. We did not ignore the question of whether a positive study would have made sense. Bayesian reasoning reminds us that just as a positive diagnostic test in a patient with a low prior probability of a disease does not prove that the patient has that disease, so too a single positive study of an unlikely hypothesis cannot establish that the therapy actually works (4). Nevertheless, if a series of well-done studies shows that distant healing or other seemingly “miraculous” types of treatments are effective, we would have to reassess currently accepted ideas about pathophysiology and therapy. Medical science would face a dilemma comparable with the one that occurred in the 16th century, when Copernicus developed the idea that the Earth revolves around the Sun. The challenge of explaining the Copernican solar system led to the discovery of gravity, a force that was previously unappreciated. A conclusive demonstration that distant healing works would likely require a similar revolution in our understanding of biology. It is our belief, however, that when the same standards required for trials of conventional medical treatments, such as thrombolysis or cholesterol lowering, are also applied to studies of distant healing and similar therapies, no revolution will be needed. Strict enrollment criteria, rigorous methods of randomization, adequate follow-up, intention-to-treat analyses, maintenance of blinding, and, perhaps most important of all, meaningful outcome variables are essential. For warts, cure is clearly the objective of treatment. For other conditions, “softer” endpoints, such as functional status or quality of life, may be more relevant; in these situations, the challenges of scientific rigor will be even more complex. Scientists and journal editors have a responsibility to be receptive to new ideas. Humility about the state of medical knowledge is essential. Only a few years ago, peptic ulcer was thought of as a stress-related disease, and many academic careers were built upon studies of acid secretion and hypersecretion in response to various stimuli. Still, skepticism and rigor must accompany the willingness to consider unlikely hypotheses. We promise to keep an open mind, but not so open that our brains fall out. 0002-9343/00/$–see front matter 507 PII S0002-9343(00)00325-9
Distant Healing: An Unlikely Hypothesis/Browner and Goldman
REFERENCES 1. Harkness EF, Abbot NC, Ernst E. A randomized trial of distant healing for warts. Am J Med. 2000;108:448 – 452. 2. Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data. J Royal Stat Soc A. 1988;151:419 – 463.
508
April 15, 2000
THE AMERICAN JOURNAL OF MEDICINE威
Volume 108
3. Light RJ, Pillemer DB. Summing Up: The Science of Reviewing Research. Cambridge, Mass: Harvard University Press; 1984:68 – 69. 4. Browner WS, Newman TB. Are all significant P values created equal? The analogy between diagnostic tests and clinical research. JAMA. 1987;257:2459 –2463.