- Email: [email protected]
Is peer-reviewing fair?
Pergamon
J Clin Epidemiol Vol. 47, No. 7, Pp. X19-820. 1994 Elsevier Science Ltd. Printed in Great Britain
0895_4356(94)EOO12-E
Editorial
Comment
IS PEER-REVIEWING WALTER McGill-Potsdam
0.
FAIR?
SPITZER
Institute of Pharmacoepidemiology and Technology Otto-Erich-Strasse 7, 14482 Potsdam, Germany
Assessment,
(Received for publication 31 January 1994)
Vuriance and Dissent. It permits diffusion of data and perspectives that merit wide public discussion among many peers, not just a few referees and editors of a journal. The charge of chauvinism is hard to substantiate. Do editors and reviewers favour material from their own country or continent or do authors tend to submit to editors who are compatriots? I suspect much of the latter goes on. I have been getting a lot more manuscripts in my office from Canadians than Feinstein (proportionately and absolutely). I deal with the potential leniency bias by selecting nonCanadian reviewers when I can, often three or more. Now that I live and work in Europe I have become acutely aware of another tendency which is susceptible to be misinterpreted as chauvinism. My colleagues in neighbouring universities cherish the “endorsement” of the well-known American and British biomedical journals. I noticed that propensity even in Scandinavia, where excellent, intenationallyacknowledged journals have flourished for a long time. Beleaguered editors of the target publications are forced into high rejection rates because of space restrictions; the perception of the authors understandably is of unfavourable bias or chauvinism even if the rejection rate is similar for American and British authors. Ernst reasonably refrains from any suggestion to do away with peer-review. With all the imperfections he enumerates it is the best we have. But some of his specific advice on remedial measures is not practical. Only journals
I concur with much of what Ernst has written as he “criticizes the critics”. I would like to show how the Journal seeks to deal with some of his valid concerns and to offer a different perspective on a few of his points. After 4 years as Associate Editor and 11 years as Editor, I have observed very little systematic preference by reviewers of manuscripts from well-established groups. If anything, our workload as editors is regularly expanded as referees urge us to have authors “revise and resubmit”, with many constructive specific suggestions. In the case of newcomers, the tough but positive interaction with the reviewers often takes us into third and fourth rounds. We also try to help authors whose native language is not English when their science seems to be good and the text unclear, awkward or incorrect linguistically. Feinstein and I find this time-consuming but very worthwhile. We are rewarded by a persistent stream of submissions from less-established investigators and from many countries outside the English-speaking world. It has been our explicit policy not to favour positive findings. Even if we must occasionally disagree with our reviewers, “negative” papers get equal opportunity. Moreover, we welcome reports and opinions that question mainstream thinking. (Such papers are sometimes sent by the so-called “newcomers”.) To avoid misplaced or premature endorsement of controversial findings or of unconventional views which, nevertheless, should see the light of day we introduced a new section some years ago, 819
820
WALTER0. SPITZER
with very large circulation, with sponsorship of societies and large editorial staffs can do effective blinding, for instance. Effective blinding requires blocking of the references as well as the text and thus becomes a labour-intensive, tedious and delaying procedure. I doubt that the reliability and accuracy Ernst calls for can be achieved in the foreseeable future. I also doubt the standards that he implies are necessary could prevent fraud, or the occasional publication of damaging error or disinformation. Only audits of raw data in the laboraories could even attempt to do so. Imperfect as they also are, I prefer to rely on the checks and balances of responsible well-managed academic, industrial and governmental laboratories.
While pursuing constant improvement in peer-review, I believe the potential adverse consequences of a flawed process should also be contained, concurrently, with another strategy. The key element of that strategy is greater editorial risk-taking. It implies a positive attitude about the work of unproven younger scientists, rejection of obsolescence and rigidity when apparent in established peers and authors, and harbouring an assertive forum for legitimate controversy (not polemicism). The ensuing open discussion in all the media available to the scientific community of published variance and dissent creates further checks and balances, fostering an orderly advancement of science and preventing distortions and errors that could harm patients or society.
J Clin Epidemiol Vol. 47, No. 7, Pp. X19-820. 1994 Elsevier Science Ltd. Printed in Great Britain
0895_4356(94)EOO12-E
Editorial
Comment
IS PEER-REVIEWING WALTER McGill-Potsdam
0.
FAIR?
SPITZER
Institute of Pharmacoepidemiology and Technology Otto-Erich-Strasse 7, 14482 Potsdam, Germany
Assessment,
(Received for publication 31 January 1994)
Vuriance and Dissent. It permits diffusion of data and perspectives that merit wide public discussion among many peers, not just a few referees and editors of a journal. The charge of chauvinism is hard to substantiate. Do editors and reviewers favour material from their own country or continent or do authors tend to submit to editors who are compatriots? I suspect much of the latter goes on. I have been getting a lot more manuscripts in my office from Canadians than Feinstein (proportionately and absolutely). I deal with the potential leniency bias by selecting nonCanadian reviewers when I can, often three or more. Now that I live and work in Europe I have become acutely aware of another tendency which is susceptible to be misinterpreted as chauvinism. My colleagues in neighbouring universities cherish the “endorsement” of the well-known American and British biomedical journals. I noticed that propensity even in Scandinavia, where excellent, intenationallyacknowledged journals have flourished for a long time. Beleaguered editors of the target publications are forced into high rejection rates because of space restrictions; the perception of the authors understandably is of unfavourable bias or chauvinism even if the rejection rate is similar for American and British authors. Ernst reasonably refrains from any suggestion to do away with peer-review. With all the imperfections he enumerates it is the best we have. But some of his specific advice on remedial measures is not practical. Only journals
I concur with much of what Ernst has written as he “criticizes the critics”. I would like to show how the Journal seeks to deal with some of his valid concerns and to offer a different perspective on a few of his points. After 4 years as Associate Editor and 11 years as Editor, I have observed very little systematic preference by reviewers of manuscripts from well-established groups. If anything, our workload as editors is regularly expanded as referees urge us to have authors “revise and resubmit”, with many constructive specific suggestions. In the case of newcomers, the tough but positive interaction with the reviewers often takes us into third and fourth rounds. We also try to help authors whose native language is not English when their science seems to be good and the text unclear, awkward or incorrect linguistically. Feinstein and I find this time-consuming but very worthwhile. We are rewarded by a persistent stream of submissions from less-established investigators and from many countries outside the English-speaking world. It has been our explicit policy not to favour positive findings. Even if we must occasionally disagree with our reviewers, “negative” papers get equal opportunity. Moreover, we welcome reports and opinions that question mainstream thinking. (Such papers are sometimes sent by the so-called “newcomers”.) To avoid misplaced or premature endorsement of controversial findings or of unconventional views which, nevertheless, should see the light of day we introduced a new section some years ago, 819
820
WALTER0. SPITZER
with very large circulation, with sponsorship of societies and large editorial staffs can do effective blinding, for instance. Effective blinding requires blocking of the references as well as the text and thus becomes a labour-intensive, tedious and delaying procedure. I doubt that the reliability and accuracy Ernst calls for can be achieved in the foreseeable future. I also doubt the standards that he implies are necessary could prevent fraud, or the occasional publication of damaging error or disinformation. Only audits of raw data in the laboraories could even attempt to do so. Imperfect as they also are, I prefer to rely on the checks and balances of responsible well-managed academic, industrial and governmental laboratories.
While pursuing constant improvement in peer-review, I believe the potential adverse consequences of a flawed process should also be contained, concurrently, with another strategy. The key element of that strategy is greater editorial risk-taking. It implies a positive attitude about the work of unproven younger scientists, rejection of obsolescence and rigidity when apparent in established peers and authors, and harbouring an assertive forum for legitimate controversy (not polemicism). The ensuing open discussion in all the media available to the scientific community of published variance and dissent creates further checks and balances, fostering an orderly advancement of science and preventing distortions and errors that could harm patients or society.