- Email: [email protected]
Modification of peer review?
Library & Information Science Research 33 (2011) 1–2
Contents lists available at ScienceDirect
Library & Information Science Research
Editorial
Modification of peer review?
For decades, peer review has been the subject of close inspection. It is widely acknowledged as imperfect and various commenters have raised the question, “What is the alternative, especially since peer review might become a subject of research misconduct?” The proponents of peer review stress that, because science is always self-correcting, we need to adopt a historical perspective. They might point to the coverage in the national press of a noted psychologist from Harvard University who has recently had some of his works retracted and some of his federally funded research questioned. The number of occurrences of such instances has been a matter of much scrutiny and debate (e.g., Altman & Hernon, 1997). Today, with the Internet presenting both challenges and opportunities and with so many places where one could place research and scholarly writings, debate has not abated. Recently, and owing in part to the crowd-sourcing capabilities afforded by the Internet, it has been suggested that perhaps the peerreview process might be expanded; after all why should only a small number of people be engaged in the process? Proponents of this approach encourage journal editors to place recently received manuscripts on the Internet. “Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience” (Cohen, 2010, p. A1). Presumably the editor either bypasses traditional reviewers or complements them with an opportunity for open comment. For a preset time period, the editor places the paper on the Internet for comment, affixes some statement of copyright to protect the author, and sweeps in all commentary. There are a number of interesting questions that come out of such a scenario. What role does the editor play in peer review? Does he or she merely compile all comments and pass them on to the author to accept or reject (with a defensible argument), or does the editor become another peer reviewer, shifting through comments and deciding which are valid and which are not? Are the authors and reviewers identified or not? If individuals know that their papers are available for comment and that those comments form a basis for acceptance or rejection, might the peer-review process itself become subject to a form of misconduct and create some hostility depending on how comments are written and whether they are signed? Does the narrowly cast short-lived dialogue that might then develop between the authors and the commenters replace the more formally and possibly more thoughtfully written letter to the editor? On another note, from our many years of teaching research at the master's and doctoral levels and reading (and writing) about the widespread lack of understanding of research among librarians, we would expect that a number of the open reviewer comments might be incorrect. Does the editor have a sufficient understanding of research to separate appropriate form inappropriate comments? We are not 0740-8188/$ – see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.lisr.2010.10.002
convinced the answer is always “yes.” Finally, it is inevitable that some (probably not all) of the comments and discussions will move from the reviewing platform to the public Web, making a permanent public record. 1. Shakespeare Quarterly Cohen (2010) announced that the fall 2010 issue of the prestigious Shakespeare Quarterly became “the first traditional humanities journal to open its reviewing to the World Wide Web” (p. A1). Apparently four papers were posted for commentary prior making an editorial decision “and a core group of experts … were invited to post their signed comments on the Web site …. Others could add their thoughts as well, after registering with their own names.” (p. A3). Presumably based on the comments, the authors can revise the papers, which the editors then review for a final determination. We are sure that everyone will monitor this situation as the issue was published in September 2010.1 According to Cohen (2010), the journal Postmedieval will conduct a similar peer review test in 2011. 2. Conclusion Cohen (2010) also discusses traditional peer review, noting that an editorial decision can take months or even years. To us, this is less a fault of peer review than the editorial policy that the journal practices. Apparently those that Cohen interviews charge that anonymous peer reviewing promotes “chubby exclusiveness, sloppy editing and fraud.” Further, Anonymity can help prevent personal bias, but it can also make reviewers less accountable; exclusiveness can help ensure quality control but can also narrow the range of feedback and participants. Open review more closely resembles Wikipedia behind the scenes, where anyone with an interest can post a comment. This open-door policy has made Wikipedia, on balance, a crucial reference resource” (p. A. 3). We do not see LISR adopting this approach. The present system works well; readers will note that in the final issue of a volume, we identify a number of supplementary peer reviewers who ensure a fresh set of responses. We set term limits for members of the editorial board and periodically poll the board about the peer-review process (e.g., Hernon & Schwartz, 2010). If an underlying reason for adopting this alternative approach is to provide the public with more timely 1 For more discussion of the Web as an alternative to traditional peer review and related issues of scholarly communication, see MediaCommons, http://mediacommons.futureofthebook.org/.
2
Editorial
access to publications, we reiterate that ScienceDirect provides access to articles in press, our peer-review process is normally completed within 3 to 4 weeks, and manuscripts are normally published within 3 to 6 months of acceptance. Nonetheless, it will be interesting to see how open peer review develops. References Altman, E., & Hernon, P. (1997). Research misconduct: Issues, implications, and strategies. Norwood, NJ: Ablex. Cohen, P. (2010, August 24). Scholars test Web alternative to the venerable peer review. The New York Times CLIX (55142), A1, A3.
Hernon, P., & Schwartz, C. (2010). Some thoughts on peer review. Library & Information Science Research, 32, 13−15.
Peter Hernon⁎ Candy Schwartz Graduate School of Library & Information Science, Simmons College, 300 The Fenway, Boston, MA 0215, USA ⁎Corresponding author. E-mail addresses: [email protected] (P. Hernon), [email protected] (C. Schwartz).
Contents lists available at ScienceDirect
Library & Information Science Research
Editorial
Modification of peer review?
For decades, peer review has been the subject of close inspection. It is widely acknowledged as imperfect and various commenters have raised the question, “What is the alternative, especially since peer review might become a subject of research misconduct?” The proponents of peer review stress that, because science is always self-correcting, we need to adopt a historical perspective. They might point to the coverage in the national press of a noted psychologist from Harvard University who has recently had some of his works retracted and some of his federally funded research questioned. The number of occurrences of such instances has been a matter of much scrutiny and debate (e.g., Altman & Hernon, 1997). Today, with the Internet presenting both challenges and opportunities and with so many places where one could place research and scholarly writings, debate has not abated. Recently, and owing in part to the crowd-sourcing capabilities afforded by the Internet, it has been suggested that perhaps the peerreview process might be expanded; after all why should only a small number of people be engaged in the process? Proponents of this approach encourage journal editors to place recently received manuscripts on the Internet. “Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience” (Cohen, 2010, p. A1). Presumably the editor either bypasses traditional reviewers or complements them with an opportunity for open comment. For a preset time period, the editor places the paper on the Internet for comment, affixes some statement of copyright to protect the author, and sweeps in all commentary. There are a number of interesting questions that come out of such a scenario. What role does the editor play in peer review? Does he or she merely compile all comments and pass them on to the author to accept or reject (with a defensible argument), or does the editor become another peer reviewer, shifting through comments and deciding which are valid and which are not? Are the authors and reviewers identified or not? If individuals know that their papers are available for comment and that those comments form a basis for acceptance or rejection, might the peer-review process itself become subject to a form of misconduct and create some hostility depending on how comments are written and whether they are signed? Does the narrowly cast short-lived dialogue that might then develop between the authors and the commenters replace the more formally and possibly more thoughtfully written letter to the editor? On another note, from our many years of teaching research at the master's and doctoral levels and reading (and writing) about the widespread lack of understanding of research among librarians, we would expect that a number of the open reviewer comments might be incorrect. Does the editor have a sufficient understanding of research to separate appropriate form inappropriate comments? We are not 0740-8188/$ – see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.lisr.2010.10.002
convinced the answer is always “yes.” Finally, it is inevitable that some (probably not all) of the comments and discussions will move from the reviewing platform to the public Web, making a permanent public record. 1. Shakespeare Quarterly Cohen (2010) announced that the fall 2010 issue of the prestigious Shakespeare Quarterly became “the first traditional humanities journal to open its reviewing to the World Wide Web” (p. A1). Apparently four papers were posted for commentary prior making an editorial decision “and a core group of experts … were invited to post their signed comments on the Web site …. Others could add their thoughts as well, after registering with their own names.” (p. A3). Presumably based on the comments, the authors can revise the papers, which the editors then review for a final determination. We are sure that everyone will monitor this situation as the issue was published in September 2010.1 According to Cohen (2010), the journal Postmedieval will conduct a similar peer review test in 2011. 2. Conclusion Cohen (2010) also discusses traditional peer review, noting that an editorial decision can take months or even years. To us, this is less a fault of peer review than the editorial policy that the journal practices. Apparently those that Cohen interviews charge that anonymous peer reviewing promotes “chubby exclusiveness, sloppy editing and fraud.” Further, Anonymity can help prevent personal bias, but it can also make reviewers less accountable; exclusiveness can help ensure quality control but can also narrow the range of feedback and participants. Open review more closely resembles Wikipedia behind the scenes, where anyone with an interest can post a comment. This open-door policy has made Wikipedia, on balance, a crucial reference resource” (p. A. 3). We do not see LISR adopting this approach. The present system works well; readers will note that in the final issue of a volume, we identify a number of supplementary peer reviewers who ensure a fresh set of responses. We set term limits for members of the editorial board and periodically poll the board about the peer-review process (e.g., Hernon & Schwartz, 2010). If an underlying reason for adopting this alternative approach is to provide the public with more timely 1 For more discussion of the Web as an alternative to traditional peer review and related issues of scholarly communication, see MediaCommons, http://mediacommons.futureofthebook.org/.
2
Editorial
access to publications, we reiterate that ScienceDirect provides access to articles in press, our peer-review process is normally completed within 3 to 4 weeks, and manuscripts are normally published within 3 to 6 months of acceptance. Nonetheless, it will be interesting to see how open peer review develops. References Altman, E., & Hernon, P. (1997). Research misconduct: Issues, implications, and strategies. Norwood, NJ: Ablex. Cohen, P. (2010, August 24). Scholars test Web alternative to the venerable peer review. The New York Times CLIX (55142), A1, A3.
Hernon, P., & Schwartz, C. (2010). Some thoughts on peer review. Library & Information Science Research, 32, 13−15.
Peter Hernon⁎ Candy Schwartz Graduate School of Library & Information Science, Simmons College, 300 The Fenway, Boston, MA 0215, USA ⁎Corresponding author. E-mail addresses: [email protected] (P. Hernon), [email protected] (C. Schwartz).