- Email: [email protected]
The Research Assessment Exercise (RAE): Criteria and issues
The Research Assessment Exercise (RAE): criteria and issues BACKGROUND A N D C O N T E X T The Higher Education Funding Council, hereafter The Council, assumed its funding role on 1 April 1993. It had been created on 6 May 1992 under the terms of the Further and Higher Education act of 1992. It could be said that The Council has two main functions namely to: • advise the secretary of state for education and employment on the funding needs of institutions • distribute funds to support teaching activities to the tune of £2,270 million (Figure for England for 1995/96), distribute funds to support research to the tune o f £ 6 3 6 million (figure for England for 1995/96)
RESEARCH FUNDING BY THE COUNCIL The Council is a major contributor to the United Kingdom's research base (HEFCE 1995). It is highly committed to promoting excellence in research. It achieves this by selectively distributing funds to institutions that have demonstrated their strength and ability based on national and international standards. Funding for research provided by the Council for 1995/96 amounts to £636 million which is further distributed as follows: • generic research (GtK) funding of about £ 2 0 million. G1K funding was created in response to the White Paper 1Kealising our Potential published in 1993. It is essentially there to support collaborative research especially research that paves the way towards improved applicability. Nurse Education Today (1995) t 5, 395-396
© 1995 Pearson Professional Ltd
• development research (DevR) funding of about £ 1 6 million. D e v i l funding was created to encourage the development of research in institutions without substantial research funds before 1993. • quality-related research (Q1K) funding of about £600 million. QtK funding is distributed to institutions on the basis of the RAE quality ratings.
RESEARCH ASSESSMENT EXERCISE PANELS KAE takes place every three or four years. It is carried out through peer review in discrete units of assessments (UOA). For instance, for purposes of the 1996 RAE, Nursing will be U O A 10. So far the 1996 RAE appears to have 69 U O A that reflect the specialist nature of their subject. One seeming question is how to conduct it especially if they have to observe an academic commonality as well as U O A distinctiveness. Indeed panels themselves are addressing this issue as part of the criteria and method of working. Panels are expected to work colIaboratively in order to reach a reasonable level of uniformity and comparability. Following consultations (see 1KAE96 2/94) panel chairs were appointed by the four funding bodies, by the end of 1994. The four funding bodies are The Higher Education Funding Council for England (HEFCE), The Department of Education for Northern Ireland (DENI), Higher Education Funding Council for Wales (HEFCW) and Scottish Higher Education Funding Council (SHEFC). Panel chairs (all of them professors except U O A 9 Pharmacy) were in turn asked to recommend names for membership of their own panel taking into account. # nominations already received through consultations # panel member's perceived knowledge of the pattern of research in the U O A covered by the panel and researcher active in that area • the desirability for continuity (about one third of panel members are from the 1992 RAE). However no member should serve in more than three consecutive exercises • the desirability of securing, intra and inter panels, membership with shared knowledge of current research not only in the UK but within institutions of different histories • the need to involve, as panel assessors, representatives of research commissioning and user communities in commerce, industry, government and public sector agencies. The
UOA
panel members have been
396 NurseEducationToday selected on the strength of their experience and perceived standing within the research comrriunity. There, however, appear to be some issues related to equal opportunity and representativeness. O n 4 August 1995 The Times Higher Education Supplement carried an article stating that out of the 542 panel members covering the 69 U O A only 51 members came from the new universities. The article also highlighted that only 5 out of the 56 chairs announced were females. Perhaps the preponderance of panel membership from the old universities is a benign reflection of the perceived expertise o f the individual. What will be most important is that each U O A declares its criteria and method o f working in advance.
A S S E S S M E N T CRITERIA & M E T H O D OF W O R K I N G The four funding bodies have collectively asked panel chairs and their members to produce written criteria that clarify: I. the nature of evidence a panel will regard as significant within its U O A and the weighting given to such evidence II. how the panel intends to deal with quantitative approach to evidence amenable to such analysis. O n the other hand there is the question of qualitative evidence which from time to time is variously interpreted. III. how the panel will assess the quality of work or research which falls into two subareas of the U O A or those submissions that clearly cross over boundaries between two or more Units of Assessments. This last consideration will be o f particular interest to U O A 19 (Nursing) as its knowledge base and indeed its research if often informed by other scientific paradigms. At the time o f writing no assessment criteria had been published therefore could not state what will be inadmissible as nursing research.
situations where new staff are recruited to replace research active staff. Perhaps this also raises a key issue about perceived potential o f research units ahead o f any evidence of assessable output © the balance between permanent and fixed term staff and the importance o f their respective contribution to the R A E the ratio o f research active staff against the total number of staff within the U O A (critical mass) and how this may affect the research quality rating o f submissions. O the criteria for the assessment of the conduct and integration o f research into the teaching and learning process as the 1996 R A E is based on quality and not necessarily quantity panels need to be clear about the treatment of the 'up to four' research works that individuals/institutions will cite as evidence o f the quality o f their research the relationship between quality and volume of the output needs further clarification as the funding bodies are likely to use the 'quality multiplied by volume' formula to determine subsequent funding. In addition to the above issues which are yet to be clarified, there is the suggestion that outputs in the form o f publications in certain refereed journals may be granted automatic quality recognition and that obtaining funding from certain organisations will also imply quality that may not require further verification. The problem with all this is that the U O A panels are advised against naming the refereed journals that may quality for this quality assessment exercise. It is anticipated the KAE criteria will be available on the Joint Academic Network (JANET) on 6 November 1995. It can be accessed at http.//www.niss.ac.uk/education/ hefc.html. Lovemore Nyatanga MSc, MA, Dip N, Cert Ed, R G N , R M N Network Editor
REFERENCES O T H E R ISSUES Panels will need to form a view in relation to the following issues: staff turnover and its implication for the assessment of research quality. This includes
HEFCE 1995 PromotingExcellence. HEFCE Armual Report, 1994-95 HEFCE 1995 1996RAE Membership of Assessment Panels. Ref: RAE 1/95 HMSO 1993Realising our Potential: A Strategyfor Science, Engineeringand TechnologyCm 2250. HMSO, London
RESEARCH FUNDING BY THE COUNCIL The Council is a major contributor to the United Kingdom's research base (HEFCE 1995). It is highly committed to promoting excellence in research. It achieves this by selectively distributing funds to institutions that have demonstrated their strength and ability based on national and international standards. Funding for research provided by the Council for 1995/96 amounts to £636 million which is further distributed as follows: • generic research (GtK) funding of about £ 2 0 million. G1K funding was created in response to the White Paper 1Kealising our Potential published in 1993. It is essentially there to support collaborative research especially research that paves the way towards improved applicability. Nurse Education Today (1995) t 5, 395-396
© 1995 Pearson Professional Ltd
• development research (DevR) funding of about £ 1 6 million. D e v i l funding was created to encourage the development of research in institutions without substantial research funds before 1993. • quality-related research (Q1K) funding of about £600 million. QtK funding is distributed to institutions on the basis of the RAE quality ratings.
RESEARCH ASSESSMENT EXERCISE PANELS KAE takes place every three or four years. It is carried out through peer review in discrete units of assessments (UOA). For instance, for purposes of the 1996 RAE, Nursing will be U O A 10. So far the 1996 RAE appears to have 69 U O A that reflect the specialist nature of their subject. One seeming question is how to conduct it especially if they have to observe an academic commonality as well as U O A distinctiveness. Indeed panels themselves are addressing this issue as part of the criteria and method of working. Panels are expected to work colIaboratively in order to reach a reasonable level of uniformity and comparability. Following consultations (see 1KAE96 2/94) panel chairs were appointed by the four funding bodies, by the end of 1994. The four funding bodies are The Higher Education Funding Council for England (HEFCE), The Department of Education for Northern Ireland (DENI), Higher Education Funding Council for Wales (HEFCW) and Scottish Higher Education Funding Council (SHEFC). Panel chairs (all of them professors except U O A 9 Pharmacy) were in turn asked to recommend names for membership of their own panel taking into account. # nominations already received through consultations # panel member's perceived knowledge of the pattern of research in the U O A covered by the panel and researcher active in that area • the desirability for continuity (about one third of panel members are from the 1992 RAE). However no member should serve in more than three consecutive exercises • the desirability of securing, intra and inter panels, membership with shared knowledge of current research not only in the UK but within institutions of different histories • the need to involve, as panel assessors, representatives of research commissioning and user communities in commerce, industry, government and public sector agencies. The
UOA
panel members have been
396 NurseEducationToday selected on the strength of their experience and perceived standing within the research comrriunity. There, however, appear to be some issues related to equal opportunity and representativeness. O n 4 August 1995 The Times Higher Education Supplement carried an article stating that out of the 542 panel members covering the 69 U O A only 51 members came from the new universities. The article also highlighted that only 5 out of the 56 chairs announced were females. Perhaps the preponderance of panel membership from the old universities is a benign reflection of the perceived expertise o f the individual. What will be most important is that each U O A declares its criteria and method o f working in advance.
A S S E S S M E N T CRITERIA & M E T H O D OF W O R K I N G The four funding bodies have collectively asked panel chairs and their members to produce written criteria that clarify: I. the nature of evidence a panel will regard as significant within its U O A and the weighting given to such evidence II. how the panel intends to deal with quantitative approach to evidence amenable to such analysis. O n the other hand there is the question of qualitative evidence which from time to time is variously interpreted. III. how the panel will assess the quality of work or research which falls into two subareas of the U O A or those submissions that clearly cross over boundaries between two or more Units of Assessments. This last consideration will be o f particular interest to U O A 19 (Nursing) as its knowledge base and indeed its research if often informed by other scientific paradigms. At the time o f writing no assessment criteria had been published therefore could not state what will be inadmissible as nursing research.
situations where new staff are recruited to replace research active staff. Perhaps this also raises a key issue about perceived potential o f research units ahead o f any evidence of assessable output © the balance between permanent and fixed term staff and the importance o f their respective contribution to the R A E the ratio o f research active staff against the total number of staff within the U O A (critical mass) and how this may affect the research quality rating o f submissions. O the criteria for the assessment of the conduct and integration o f research into the teaching and learning process as the 1996 R A E is based on quality and not necessarily quantity panels need to be clear about the treatment of the 'up to four' research works that individuals/institutions will cite as evidence o f the quality o f their research the relationship between quality and volume of the output needs further clarification as the funding bodies are likely to use the 'quality multiplied by volume' formula to determine subsequent funding. In addition to the above issues which are yet to be clarified, there is the suggestion that outputs in the form o f publications in certain refereed journals may be granted automatic quality recognition and that obtaining funding from certain organisations will also imply quality that may not require further verification. The problem with all this is that the U O A panels are advised against naming the refereed journals that may quality for this quality assessment exercise. It is anticipated the KAE criteria will be available on the Joint Academic Network (JANET) on 6 November 1995. It can be accessed at http.//www.niss.ac.uk/education/ hefc.html. Lovemore Nyatanga MSc, MA, Dip N, Cert Ed, R G N , R M N Network Editor
REFERENCES O T H E R ISSUES Panels will need to form a view in relation to the following issues: staff turnover and its implication for the assessment of research quality. This includes
HEFCE 1995 PromotingExcellence. HEFCE Armual Report, 1994-95 HEFCE 1995 1996RAE Membership of Assessment Panels. Ref: RAE 1/95 HMSO 1993Realising our Potential: A Strategyfor Science, Engineeringand TechnologyCm 2250. HMSO, London