- Email: [email protected]
Methodological errors in Medical Research
TIPS- February 1992 /Vol. 121
Lies, damned lies and statistics; Methodological ‘Errors in Medical Research by B. Andersen, Blackwell Scientific Publications, 19%. W5.00 foiii + 270 pages) lSBN 0 632 02137 3 If you take a malicious pleasure in read@ of the flaws, obvious or otherwise, which invalidate the papers of your colleagues and competitors, then this is a book for bedside reading. On the other hand, if you have too tender a conscience, this book could give you nightmares in which a reproachful anay of past papers return to castigate you for the methodo!ogical errors, venal or otherwise, with which you have sent them to their eternal resting place in the literature. This volume is a casebook of errorstobefoundinreportsof clinical trials; primarily errors of statistics, logic and commonsense, rather than errors of interpretation or outright fraud. The author hastens to add, however, that this is not a textbook of statistics. Some of the errors are obvious, but are common in one disguise or another. Thus, the report that symphony conductors lived almost four years longer tl,;n the average lifespan of the entire male population ignores the fact that life expectancy from birth of males is being comparr?d to life expectancy of conductom who survive the rigors of infancy and the trauma of their teens, joining their pmfeaston at some later time. Another exampk involves a long-term study of anticoagulants in angina pectoris. No control patients were used, but the paper compared mortality with a group of patients reported on by others in which anticoagu&nts were not used. However, the other study had been performed five years ago in another country. Examples such as these allow a sigh of relief. We would never commit errors so egregious. But read on, gentle reader, and discover that conscience can make cowards of the best of us. How often do we do a study, measuring a number of vatiables, and then look for comelations between the variables? We fishers of numbers run afoul of the paradox that ‘the improbable is extremely probable’. As Andersen
77
phrases it, ‘since we do not know in advance what pattern we are looking for, it is a good bet that we can discover some kind of unusual pattern’. The problem is that we will be unable to replicate the pattern. Mea culpa. Well-controlled trials are difficult to conduct. There may be ethical constraints, difficulties in finding patients, and other problems. However, a good reason for doing a bad drug trial does not improve the validity of the trial. The author points out that some errors are so gross as to provide a license to write condusions at will. Some of the examples are hilarious, as with the patient in a two-week trial who preferred the placebo. He was ‘punished’ by being put through the trial again. This time he chose the drug perhaps an example of regression towards the mean, which is also discussed in the book. The reader is forcibly struck by the frequency with which misused logic, language and statistics coincide The organization of the book is loose. It would benefit from a closer discussion of methodological errors to counterpoint
Establishing new principles principles of Drug Action, The Basis of pharmacology (3rd edn) edited by Willirm B. Pratt and Palmer Taylor, Churchill Liuingstone, 1990. E37.00(816 pages) ISBN 0 443 08676 1 The appearance of the third edition of Principles of Drug 4ction will be welcomed by all pharmacologists and, not least, by students and their teachers. It is 16 yean since the publication of the excellent second edition, by Avram Goldstein, Lewis Aronow and Sumner Kalman. This was the first textbook to be devoted entirely to the basic principles of pharmacology and to its growing points. The new edition has the same general aims and is a thoroughly worthy successor, though it differs in several ways. There are now 11 contributors, all new, from three centres in the USA - Bethesda, La Jolla and Ann Arbor. This multi-authorship could have led to disparities in style and in depth of treatment,
and summarize the ‘:,..o studies. Examples are divided into design errors and data errors, but the author admits that some errors defy classification. Many of the cases are complex and require more thinking about than this unpaid reviewer was willing to give. But this is a book to slip in your pocket, and pull out at odd intervals in airports, over a cup of coffee, or during the sempitemal delay in your doctor’s waiting room. Although the author is Danish, he writes in English-real English, not the mangled and discombobulated assembly of indigestible text that masquerades as language in much scientific publishing. The book is aimed at physicians those who write or conduct clinical trials - but any researcher could benefit f’roma perusal. Even the most puritanical and Bunyanesque will not escape a sense of ‘there but for the grace of God go I’, in the ramble through this collection of dbsurdity and error. RYAN j. HUXTABLE Drprrtmrnt
o\
Pkmnrcdogy, colkge of Mtdicine, Uniotity of Arizimr Health Sciencn Center, Tucson,AZ 85724. USA.
but the two editors have either written or co-authored more than half the text, and there are no noticeable differences in presentation or approach. Though most of the chapter headings are similar to those of the earlier editions, their content is often very different, and the editors estimate that 70% of the book is entirely new. Three chapters, on drug toxicity, drug development and drug evaluation, have been deleted, though some of the material appears elsewhere. The extent of the other changes varies fmm topic to topic, depending on the advances since the second edition. The first two chapters, on the molecular ‘basis of pharma4ogical selectivity and drug action, now form almost a quarter of the book, and have been completely rewritten. Together, they give an excellent account of many of the most important and exciting molecular developments in pharmacology. The main emphasis is on the application of chemical and biochemical pdnciples and methodology to the study of drug moIecules and their
Lies, damned lies and statistics; Methodological ‘Errors in Medical Research by B. Andersen, Blackwell Scientific Publications, 19%. W5.00 foiii + 270 pages) lSBN 0 632 02137 3 If you take a malicious pleasure in read@ of the flaws, obvious or otherwise, which invalidate the papers of your colleagues and competitors, then this is a book for bedside reading. On the other hand, if you have too tender a conscience, this book could give you nightmares in which a reproachful anay of past papers return to castigate you for the methodo!ogical errors, venal or otherwise, with which you have sent them to their eternal resting place in the literature. This volume is a casebook of errorstobefoundinreportsof clinical trials; primarily errors of statistics, logic and commonsense, rather than errors of interpretation or outright fraud. The author hastens to add, however, that this is not a textbook of statistics. Some of the errors are obvious, but are common in one disguise or another. Thus, the report that symphony conductors lived almost four years longer tl,;n the average lifespan of the entire male population ignores the fact that life expectancy from birth of males is being comparr?d to life expectancy of conductom who survive the rigors of infancy and the trauma of their teens, joining their pmfeaston at some later time. Another exampk involves a long-term study of anticoagulants in angina pectoris. No control patients were used, but the paper compared mortality with a group of patients reported on by others in which anticoagu&nts were not used. However, the other study had been performed five years ago in another country. Examples such as these allow a sigh of relief. We would never commit errors so egregious. But read on, gentle reader, and discover that conscience can make cowards of the best of us. How often do we do a study, measuring a number of vatiables, and then look for comelations between the variables? We fishers of numbers run afoul of the paradox that ‘the improbable is extremely probable’. As Andersen
77
phrases it, ‘since we do not know in advance what pattern we are looking for, it is a good bet that we can discover some kind of unusual pattern’. The problem is that we will be unable to replicate the pattern. Mea culpa. Well-controlled trials are difficult to conduct. There may be ethical constraints, difficulties in finding patients, and other problems. However, a good reason for doing a bad drug trial does not improve the validity of the trial. The author points out that some errors are so gross as to provide a license to write condusions at will. Some of the examples are hilarious, as with the patient in a two-week trial who preferred the placebo. He was ‘punished’ by being put through the trial again. This time he chose the drug perhaps an example of regression towards the mean, which is also discussed in the book. The reader is forcibly struck by the frequency with which misused logic, language and statistics coincide The organization of the book is loose. It would benefit from a closer discussion of methodological errors to counterpoint
Establishing new principles principles of Drug Action, The Basis of pharmacology (3rd edn) edited by Willirm B. Pratt and Palmer Taylor, Churchill Liuingstone, 1990. E37.00(816 pages) ISBN 0 443 08676 1 The appearance of the third edition of Principles of Drug 4ction will be welcomed by all pharmacologists and, not least, by students and their teachers. It is 16 yean since the publication of the excellent second edition, by Avram Goldstein, Lewis Aronow and Sumner Kalman. This was the first textbook to be devoted entirely to the basic principles of pharmacology and to its growing points. The new edition has the same general aims and is a thoroughly worthy successor, though it differs in several ways. There are now 11 contributors, all new, from three centres in the USA - Bethesda, La Jolla and Ann Arbor. This multi-authorship could have led to disparities in style and in depth of treatment,
and summarize the ‘:,..o studies. Examples are divided into design errors and data errors, but the author admits that some errors defy classification. Many of the cases are complex and require more thinking about than this unpaid reviewer was willing to give. But this is a book to slip in your pocket, and pull out at odd intervals in airports, over a cup of coffee, or during the sempitemal delay in your doctor’s waiting room. Although the author is Danish, he writes in English-real English, not the mangled and discombobulated assembly of indigestible text that masquerades as language in much scientific publishing. The book is aimed at physicians those who write or conduct clinical trials - but any researcher could benefit f’roma perusal. Even the most puritanical and Bunyanesque will not escape a sense of ‘there but for the grace of God go I’, in the ramble through this collection of dbsurdity and error. RYAN j. HUXTABLE Drprrtmrnt
o\
Pkmnrcdogy, colkge of Mtdicine, Uniotity of Arizimr Health Sciencn Center, Tucson,AZ 85724. USA.
but the two editors have either written or co-authored more than half the text, and there are no noticeable differences in presentation or approach. Though most of the chapter headings are similar to those of the earlier editions, their content is often very different, and the editors estimate that 70% of the book is entirely new. Three chapters, on drug toxicity, drug development and drug evaluation, have been deleted, though some of the material appears elsewhere. The extent of the other changes varies fmm topic to topic, depending on the advances since the second edition. The first two chapters, on the molecular ‘basis of pharma4ogical selectivity and drug action, now form almost a quarter of the book, and have been completely rewritten. Together, they give an excellent account of many of the most important and exciting molecular developments in pharmacology. The main emphasis is on the application of chemical and biochemical pdnciples and methodology to the study of drug moIecules and their