- Email: [email protected]
Henry Burger
DISSECTING ROOM
JABS & JIBES
LIFELINE Henry Burger Henry Burger is director of Prince Henry’s Institute of Medical Research, in Victoria, Australia. By training a clinical endocrinologist, his interests have progressed much like the ageing process—growth hormone physiology and treatment, reproduction, and latterly women’s health, particularly the menopause. Inhibin physiology has been a fundamental preoccupation. Who was your most influential teacher? The late John Gerald Hayden was the first professor of medicine at Melbourne’s St Vincent’s Hospital, a remarkable clinician and diagnostician. Which event has had most effect on your work? Attendance at the US “Young Turks” (meeting of the Society for Clinical Investigation) and hearing the early debates on radioimmunoassay, with Sol Berson in particular—debates about scientific achievements that I aimed to emulate. What would be your advice to a newly qualified doctor? Keep your sights high and try to get some training in research. How do you relax? By listening to music of all kinds but particularly baroque and early classical, by playing tennis, by skiing downhill and by avid watching of cricket and Australian Rules football. What is your greatest regret? Not being as creative as I would like. What complementary therapies have you tried? I have tried diets high in phyto-oestrogens for some of my menopausal patients with symptomatic benefit. How would you like to die? Just as so many others, in my sleep but after a day where I had had a faultless run down a steep, double black diamond slope. What is your favourite play? King Lear—based on the wonderful appreciation given to me by my school English literature teacher. What is your greatest fear? Being found out for all my real weaknesses. What is your worst habit? Falling asleep in meetings.
686
Chance and grants fter hearing yet another researcher denounce the scientifically illiterate apparatchiks who rejected his grant proposal, I’m willing to consider that time is up for the hallowed institution of peer review. Ad nauseam, we hear that it’s the worst possible system, except for all the others. So say the mandarins of science, echoing Churchill on democracy. About the superiority of democracy, no quibble. But when it comes to providing research money, there’s got to be a better way than the cumbersome, snail-paced, expensive, and unproven peer-review derby long in effect in the USA and elsewhere. Why not try a lottery among qualified researchers? The knee-jerk response: it would be foolish to substitute chance for the rational deliberate decision-making that is the essence of peer review. However, these claimed virtues are rarely subjected to examination. And they frequently are confounded by evidence seeping out of the closed chambers where applications are reviewed. With most proposals rated meritorious, and budgets insufficient for financing even one third of them, peer-reviewing often comes down to a crapshoot. From the front lines of reviewing, it’s reported that the chosen are frequently no better than the rejected. Rather, they are the beneficiaries of a system that belts out winners purely by chance—like a lottery—up to the amount of money available. The lack of distinction between winners and losers is cited as an impediment to recruiting for the heavy, unpaid burdens of peer review. Why bother, if merit is not the decisive factor? Even so, it will be argued, peer review provides some degree of selectivity— doesn’t it? Actually, retrospection is sparse, and we don’t know how well peer review really works. Those who seek renewal of expiring grants
A
naturally spruce up their applications with whatever bibliographical evidence they can muster to show that the money was well spent. But their accomplishments, if any, cast no light on the overall wisdom of the selection process. What is to be concluded about those who accomplished little or nothing with their grants? Did the system err in giving them the nod, while neglecting worthier applicants? Given a grant, some of the losers might have excelled. Occasionally, some of them scrape together money and achieve distinction, irrespective of the verdict rendered by the peer-reviewers. Did the system err in rejecting them? No-one knows, and little effort is invested in trying to find out. But it would be worth a try, because peer review not only is unproven for picking winners, it also is extremely costly, in direct outlays of money for staff at the granting agencies and travel by reviewers. And it’s slow. So, as a first step towards either verifying peer review or moving on to a better system, the powers that be should slice off some respectable percentage of the research funds—say, 15–20% over 5 years—and set them aside. These funds would be awarded by lottery to applicant scientists whose qualifications and projects have been certified as respectable, ratings easily determined at a small fraction of the cost of peer review. Details aside, the basic principle is clear; instead of dodging the fact that chance plays a big part in awarding money, the system will sanctify chance as the determining factor. After a few years, let’s look back and evaluate the science that came out of this system. If it’s no worse than what we’re getting now, let’s chuck peer review, and thereby save a lot of needless effort and money. A lottery might turn out to be the worst possible system, except for all others. Daniel S Greenberg
THE LANCET • Vol 351 • February 28, 1998
JABS & JIBES
LIFELINE Henry Burger Henry Burger is director of Prince Henry’s Institute of Medical Research, in Victoria, Australia. By training a clinical endocrinologist, his interests have progressed much like the ageing process—growth hormone physiology and treatment, reproduction, and latterly women’s health, particularly the menopause. Inhibin physiology has been a fundamental preoccupation. Who was your most influential teacher? The late John Gerald Hayden was the first professor of medicine at Melbourne’s St Vincent’s Hospital, a remarkable clinician and diagnostician. Which event has had most effect on your work? Attendance at the US “Young Turks” (meeting of the Society for Clinical Investigation) and hearing the early debates on radioimmunoassay, with Sol Berson in particular—debates about scientific achievements that I aimed to emulate. What would be your advice to a newly qualified doctor? Keep your sights high and try to get some training in research. How do you relax? By listening to music of all kinds but particularly baroque and early classical, by playing tennis, by skiing downhill and by avid watching of cricket and Australian Rules football. What is your greatest regret? Not being as creative as I would like. What complementary therapies have you tried? I have tried diets high in phyto-oestrogens for some of my menopausal patients with symptomatic benefit. How would you like to die? Just as so many others, in my sleep but after a day where I had had a faultless run down a steep, double black diamond slope. What is your favourite play? King Lear—based on the wonderful appreciation given to me by my school English literature teacher. What is your greatest fear? Being found out for all my real weaknesses. What is your worst habit? Falling asleep in meetings.
686
Chance and grants fter hearing yet another researcher denounce the scientifically illiterate apparatchiks who rejected his grant proposal, I’m willing to consider that time is up for the hallowed institution of peer review. Ad nauseam, we hear that it’s the worst possible system, except for all the others. So say the mandarins of science, echoing Churchill on democracy. About the superiority of democracy, no quibble. But when it comes to providing research money, there’s got to be a better way than the cumbersome, snail-paced, expensive, and unproven peer-review derby long in effect in the USA and elsewhere. Why not try a lottery among qualified researchers? The knee-jerk response: it would be foolish to substitute chance for the rational deliberate decision-making that is the essence of peer review. However, these claimed virtues are rarely subjected to examination. And they frequently are confounded by evidence seeping out of the closed chambers where applications are reviewed. With most proposals rated meritorious, and budgets insufficient for financing even one third of them, peer-reviewing often comes down to a crapshoot. From the front lines of reviewing, it’s reported that the chosen are frequently no better than the rejected. Rather, they are the beneficiaries of a system that belts out winners purely by chance—like a lottery—up to the amount of money available. The lack of distinction between winners and losers is cited as an impediment to recruiting for the heavy, unpaid burdens of peer review. Why bother, if merit is not the decisive factor? Even so, it will be argued, peer review provides some degree of selectivity— doesn’t it? Actually, retrospection is sparse, and we don’t know how well peer review really works. Those who seek renewal of expiring grants
A
naturally spruce up their applications with whatever bibliographical evidence they can muster to show that the money was well spent. But their accomplishments, if any, cast no light on the overall wisdom of the selection process. What is to be concluded about those who accomplished little or nothing with their grants? Did the system err in giving them the nod, while neglecting worthier applicants? Given a grant, some of the losers might have excelled. Occasionally, some of them scrape together money and achieve distinction, irrespective of the verdict rendered by the peer-reviewers. Did the system err in rejecting them? No-one knows, and little effort is invested in trying to find out. But it would be worth a try, because peer review not only is unproven for picking winners, it also is extremely costly, in direct outlays of money for staff at the granting agencies and travel by reviewers. And it’s slow. So, as a first step towards either verifying peer review or moving on to a better system, the powers that be should slice off some respectable percentage of the research funds—say, 15–20% over 5 years—and set them aside. These funds would be awarded by lottery to applicant scientists whose qualifications and projects have been certified as respectable, ratings easily determined at a small fraction of the cost of peer review. Details aside, the basic principle is clear; instead of dodging the fact that chance plays a big part in awarding money, the system will sanctify chance as the determining factor. After a few years, let’s look back and evaluate the science that came out of this system. If it’s no worse than what we’re getting now, let’s chuck peer review, and thereby save a lot of needless effort and money. A lottery might turn out to be the worst possible system, except for all others. Daniel S Greenberg
THE LANCET • Vol 351 • February 28, 1998