Nursing research and the 2001 UK research assessment exercise

Nursing research and the 2001 UK research assessment exercise Departments of nursing, in keeping with other academic departments in UK universities, will be gearing up to prepare submissions to have their research performance judged in the 2001 Higher Education Funding Council Research Assessment Exercise (RAE). Rating methods and league tables of performance are now common place in the UK public sector. They are formally applied in universities for both of their primary functions - teaching through the Teaching Quality Assessment (TQA) which is also placing demands on nursing departments over the coming year, and research through the RAE. Both quality ratings serve as public indicators and benchmarks of academic performance. Both are taken very seriously indeed by university departments which regard themselves as full players in higher education. At present there arc no direct funding implications for universities of teaching quality ratings, but the scores gained are important signals to prospective students and those who advise them when considering where to study. Research ratings are also important public summaries of standing in the research world, featuring prominently in prospectuses, brochures and job advertisements when ratings are good. The grade attained by departments also serves as a reference point for those funding research and research training. However, unlike teaching assessments, RAE ratings are used by the university funding councils as a means of distributing funding to universities specifically to support research. In 1999-2000 this amounts to s m distributed to universities by the Higher Education Funding Council for England, 1999. Of this sum, s m (97.7%) is distributed according the to quality criteria (QR) reflecting the ratings obtained in the 1996 RAE. This investment, and the additional monies provided by similar bodies in Scotland, Wales and Northern Ireland, is not insubstantial and can represent an important fraction of the income of individual universities and, subsequently, those departments which do well in the RAE. Doing well depends on both of the rating gained and the number of staff returned as research active. Equally, departments which perform poorly or drop grades between successive RAEs are likely to be penalized Clinical Effectiveness in Nursing (I 999) 3, 149--I 50 9 1999 Harcourt Publishers Ltd

in direct income and/or in other types of investment with knock-on effects into the next RAE. But it is not only at individual university and departmental level that the RAE is important, comparisons are also drawn between subjects. Observers have not been slow to point out that nursing has been rooted to bottom position in the league table for RAE grades obtained among all subjects entered. Several commentators have advanced reasons to explain nursing's lowly position. These fall broadly into two camps. The first group accepts nursing's poor showing (Tiemey 1994, Kitson 1997, Traynor & Rafferty 1999) as the inevitable consequence of its newcomer status in higher education with excessive competing demands, particularly in the 'new' universities, which lack a strong research tradition. Lacking sophistication in knowing the rules of the game and how best to play it, without strategic direction in research and an adequate research infl'astructure, the competing teaching ethos resulted in many departments achieving a low grade. The second group of commentators attribute nnrsing's poor outcome to aspects of the RAE process itself. These include a lack of explicitness in the criteria by which judgements were made leading to second guessing how to respond, in-built biases within the panel toward maintaining the status quo in terms of failing to reward emerging new centres of excellence (Robinson 1994). Changing criteria by which research is judged (Watson 1997, Traynor & Rafferty 1999) baffled some departments about the criteria for selecting individuals and research to be entered. Also castigated was the view that quality judgements were based on the medium of dissemination rather than the quality of the research itself (Tierney 1994). The opacity of the judgements meant that there was the possibility of the panel being unrealistically biased and downgrading nursing (Watson 1997). So will nursing fare better this time round? The outcome of the 2001 RAE is likely to follow the same logic governing selective resource allocation as on previous occasions. That is, a meritocratic system with cash following research quality and volume with a skewed distribution towards the higher grades. Voices against the absence of

150 Clinical Effectiveness in Nursing investment funding from HEFCE to raise standards in poorly performing subjects or in the 'new' universities with less of a research tradition remain unheard. However, critics of the assessment process have been heard, including the need to ensure that interdisciplinary research is judged fairly (RAE 2001 1999a), and changes have been made. First, principles governing the process have been spelled out in some detail (RAE 2001 1999b). Once again the process is dependent on peer review. The inevitable subjectivity of this process is moderated by adherence to the principles of clarity of information; consistency across subjects and calibration of ratings; continuity with previous RAEs, but changes made to accommodate improvements that outweigh their costs; credibility to those being assessed; efficiency consistent with a robust process; neutrality with respect to encouraging only activity or behaviour consistent with improvements in research quality; parity across types of research; and transparency about the decision-making process itself. These principles were made available early in the development period when specific assessment criteria were being created. A second innovation is to provide the research community with the draft criteria by which ratings will be made by all of the panels and invite their comments prior to the individual panels producing final versions. These criteria can be judged against the principles outlined above and enable judgements to be made about their appropriateness for individual subjects. Improved explicitness should serve to clarify the process by which ratings are assigned. This is carried through to a third major innovation - providing feedback to institutions about the reasons why particular grades are assigned to individual submissions: This should improve understanding of how the data provided in each submission were treated (also providing grounds for challenges if this is felt to be warranted) and provide assistance in preparation of future rounds (so long as the goal posts remain in the same place). The fourth innovation is to make public the details provided in each submission, apart from those that would prejudice individuals or the strategic plans of the departments entered, so that the research community can then perform its own benchmarking and assess the consistency and credibility of the assessment panels' decisions. These procedural changes may go some way to appeasing those who justifiably criticized the opacity of the process and its liability to different forms of abuse, and may serve nursing and other subjects well by improving it. Providing improved information about the criteria by which submissions will be judged and the methods used by the panels to arrive at the ratings, should also lead to more considered and appropriate submissions and enable those less familiar with the process to obtain assistance. Entering a submission to the RAE is the right of every academic department, but, unlike the TQA, there is no obligation to enter. The RAE process

cannot legislate for those departments which choose to enter when the quality of research outputs and prevailing research culture is such that even a modest rating cannot be achieved. Achieving a low rating, while providing a benchmark of performance, offers no financial advantage to the institution nor prestige to the department or the subject. Neither can improving the RAE assessment process compensate for the overall quality of the research which is submitted. Ultimately research quality depends on nursing putting its own research house in order. Doing so begins with attracting a sufficient number of entrants of the right calibre into nursing. The dual system of diploma and degree level entry inevitably reduces the available research talent. Wide entry portals are essential to produce the sheer numbers of nurses required, of which no more than a tiny fraction will ever make a career in research or academic life. Undergraduate and post-graduate education needs to be of sufficient quality to sustain the best brains within that tiny fraction, while providing a culture which encourages them to remain in nursing and find research an attractive option. Post-doctoral research-training opportunities have to be available to those showing evidence of research aptitude. Career structures that offer nurses the opportunity to do clinically relevant research need to be developed. Nursing needs different kinds of research. While it is feasible to do research that develops from a thorough textbook knowledge of the subject, we also need research than stems from hypotheses generated from nursing practice. We lack career structures that enable nurses to keep abreast of, and be practised in, nursing, and do high quality research about nursing. Developing them will depend on partnerships between the NHS and the universities, and on forming strategic alliances and initiatives for nursing research which maximize payoff for health care as well as for nursing. Increasing the amount of research based on sound theory, which is practice relevant and addresses the effectiveness of nursing, is fundamental to this agenda. REFERENCE

Higher Education Funding Council for England 1999 Recurrent grants for 1999-2000. Report 99/13 Kitson A 1997 Lessons from the 1996 ResearchAssessment Exercise. Nurse Researcher 4(3): 81-93 RAE 2001 1999a Interdisciplinaryresearch and the Research Assessment Exercise. RAE 1/99 RAE 2001 1999b Research Assessment Exercise 2001: guidance on submissions. RAE 2/99 Robinson J 1993 Nursing and the Research Assessment Exercise: what counts. Nurse Researcher 1(1): 84-93 Tierney A 1994 An analysis of nursing's performance in the 1992 assessment of research in British universities. Journal of Advanced Nursing 19(4): 593-602 Traynor M, Rafferty AM 1999 Nursing and the research assessment exercise: past, present and future. Journal of Advanced Nursing 30(1): 186-192 Watson R 1997 United Kingdom universities Research Assessment Exercise 1996: critique, comment and concern. Journal of Advanced Nursing 26(4): 641-642

Senga Bond