- Email: [email protected]
Peer review and scientific revolutions
BIOL PSYCHIA~Y 1986;21:1-2
1
EDITORIAL
Peer Review and Scientific Revolutions
The miracles of science do not come from the many, but from the very few. The very nature of revolutionary research makes it impossible to predict whether a problem has a solution; the experimental course leading to the solution; the timetable for its completion; or whether an unexpected finding will divert the effort in a new direction. Thus there is an inherent conflict between the ‘*peerreview system” as currently managed and the ability to promote scientific revolutions. This system requires an investigator to spell out detailed plans of investigation for an extended period ranging from 3 to 7 years. There are many deficiencies inherent in such a system. First of all it presupposes that an investigator can predict the results of studies not yet performed. If he can do so he does not expect to make a discovery; in fact that mind-set may keep him from recognizing a discovery. In addition, is it realistic to expect an investigator to reveal to his peers a highly novel original idea that he plans to work on a year or more in the future, without fear that his priority might thereby be dissipated? For instance, when Hahn and Strassman’s observations and the subsequent theoretical considerations on fission were revealed to the scientific community, within days confirmatory experiments were performed in laboratories around the world. Furthermore it would be impossible to assure that knowledge of presumably confidential information supplied in grant requests does not confer on the reviewers an unfair advantage not given to the general scientific community. Is there a better way? For established investigators funding should be based, not on specific detailed proposals, but rather on a retrospective review. Funding at a constant level could be renewable, perhaps at 3-year intervals, by submission of a satisfactory progress report demons~ating that effective use has been made of the funds. If there is evidence for increased or decreased productivity, funding could be increased or decreased appropriately. The entire research effort of the individual or team should be examined at one time. There should not be multiple grant requests that make it impossible to evaluate the degree of overlapping in requesting funding from different agencies for presumably different projects. The review system suggested here simplifies the preparation of a grant renewal request, as much of the progress report would be papers already written. It simplifies the work of the reviewers in that papers published or in press have already had peer review and the reviewers can judge independently the quality of papers. Ongoing retrospective review is essential if we are to assure optimal utilization of resources. There is some concern that retrospective review does not permit funding of an imaginative investigator with a highly original idea that requires a long-time investment and which may, at the end of that period, prove to be either a great breakthrough or perhaps a complete loss. I think that no peer review system that requires accountability can completely avoid this difficulty. It is more likely to be met, if at all, by insti~tional or private research funds. Nonetheless, investing in a person does permit him to develop
2
BIOLPSYCHIATRY 1986;21:1-2
Editorial
unorthodox concepts while carrying on a body of more traditional research. For the most part original ideas develop out of a body of sound investigations rather than as a bolt from the blue from one without a track record in research. For continuity of research we must be concerned with the development of new investigators and their access to funding. Few are the people entering biomedical investigation at present without a period of postdoctoral training. Their accomplishments during this period and the recommendations of their preceptors should be sufficient to assure a small investment to permit young investigators to take their first steps toward independence. This does put a heavy responsibility on the preceptors. We must provide an opportunity for the growth and development of our trainees, identify among them those with a true flair for discovery, and divert the others to different paths. Funding for new investigators should be on the basis of their accomplishments during the training period and the proven track record of the preceptors in identifying talented scientists. As scientists we must accept that the world has limited resources. In all fields we must be alert to cost-effectiveness and maximization of efficiency. The major disadvantage in the present system is that it diverts scientists from investigation to feats of grantsmanship. Leo Szilard recognized the problem a quarter-century ago when he wrote (in The Voice of the Dolphins and Other Stories, 1961, p 100) that progress in research could be brought to a halt by the total commitment of the time of the research community to writing, reviewing, and supervising a peer review grant system very much like the one currently in force. We are approaching that day. If we are to continue to hope for revolutions in science, the time has come to consider revolutionizing the mechanisms for funding science and scientists. Rosul_+wS. YaloKt
1
EDITORIAL
Peer Review and Scientific Revolutions
The miracles of science do not come from the many, but from the very few. The very nature of revolutionary research makes it impossible to predict whether a problem has a solution; the experimental course leading to the solution; the timetable for its completion; or whether an unexpected finding will divert the effort in a new direction. Thus there is an inherent conflict between the ‘*peerreview system” as currently managed and the ability to promote scientific revolutions. This system requires an investigator to spell out detailed plans of investigation for an extended period ranging from 3 to 7 years. There are many deficiencies inherent in such a system. First of all it presupposes that an investigator can predict the results of studies not yet performed. If he can do so he does not expect to make a discovery; in fact that mind-set may keep him from recognizing a discovery. In addition, is it realistic to expect an investigator to reveal to his peers a highly novel original idea that he plans to work on a year or more in the future, without fear that his priority might thereby be dissipated? For instance, when Hahn and Strassman’s observations and the subsequent theoretical considerations on fission were revealed to the scientific community, within days confirmatory experiments were performed in laboratories around the world. Furthermore it would be impossible to assure that knowledge of presumably confidential information supplied in grant requests does not confer on the reviewers an unfair advantage not given to the general scientific community. Is there a better way? For established investigators funding should be based, not on specific detailed proposals, but rather on a retrospective review. Funding at a constant level could be renewable, perhaps at 3-year intervals, by submission of a satisfactory progress report demons~ating that effective use has been made of the funds. If there is evidence for increased or decreased productivity, funding could be increased or decreased appropriately. The entire research effort of the individual or team should be examined at one time. There should not be multiple grant requests that make it impossible to evaluate the degree of overlapping in requesting funding from different agencies for presumably different projects. The review system suggested here simplifies the preparation of a grant renewal request, as much of the progress report would be papers already written. It simplifies the work of the reviewers in that papers published or in press have already had peer review and the reviewers can judge independently the quality of papers. Ongoing retrospective review is essential if we are to assure optimal utilization of resources. There is some concern that retrospective review does not permit funding of an imaginative investigator with a highly original idea that requires a long-time investment and which may, at the end of that period, prove to be either a great breakthrough or perhaps a complete loss. I think that no peer review system that requires accountability can completely avoid this difficulty. It is more likely to be met, if at all, by insti~tional or private research funds. Nonetheless, investing in a person does permit him to develop
2
BIOLPSYCHIATRY 1986;21:1-2
Editorial
unorthodox concepts while carrying on a body of more traditional research. For the most part original ideas develop out of a body of sound investigations rather than as a bolt from the blue from one without a track record in research. For continuity of research we must be concerned with the development of new investigators and their access to funding. Few are the people entering biomedical investigation at present without a period of postdoctoral training. Their accomplishments during this period and the recommendations of their preceptors should be sufficient to assure a small investment to permit young investigators to take their first steps toward independence. This does put a heavy responsibility on the preceptors. We must provide an opportunity for the growth and development of our trainees, identify among them those with a true flair for discovery, and divert the others to different paths. Funding for new investigators should be on the basis of their accomplishments during the training period and the proven track record of the preceptors in identifying talented scientists. As scientists we must accept that the world has limited resources. In all fields we must be alert to cost-effectiveness and maximization of efficiency. The major disadvantage in the present system is that it diverts scientists from investigation to feats of grantsmanship. Leo Szilard recognized the problem a quarter-century ago when he wrote (in The Voice of the Dolphins and Other Stories, 1961, p 100) that progress in research could be brought to a halt by the total commitment of the time of the research community to writing, reviewing, and supervising a peer review grant system very much like the one currently in force. We are approaching that day. If we are to continue to hope for revolutions in science, the time has come to consider revolutionizing the mechanisms for funding science and scientists. Rosul_+wS. YaloKt