- Email: [email protected]
Is complete autonomy necessarily desirable?
•
Pergamon
S0952-8733(96)00009-8
Higher EducationPolicy, Vol. 9, No. 3, pp. 211-214, 1996 Copyright © 1996International Association of Universities Published by Elsevier Science Ltd. Printed in Great Britain 0952-8733/96 $15.00+0.00
Is complete autonomy necessarily desirable?* Anne C. Brook Head of Quality Assurance, The University of Liverpool, Liverpool L69 3BX, England The audit of systems for academic quality assurance and the assessment of the quality of education provided to students in individual subject areas are both relatively recent developments in the U.K., which derive directly or indirectly from govermnent pressure to make universities more accountable. This paper examines some aspects of the impact of those developments on universities and expresses the view that some of the effects have been both positive and of a kind which would not necessarily have occurred without such internal pressures. In particular it is argued that self-evaluation of teaching requires a well-informed view of other approaches to teaching elsewhere but that the networks to facilitate that rounded view of different approaches to teaching are poorly developed; assessment has benefited those networks. Copyright © 1996 International Association of Universities
INTRODUCTION Most people will be familiar with the two systems currently operating in the United K i n g d o m in relation to Higher Education--audit and assessment. Audit looks at the institutional-level systems that are in place to enable an individual university or college to know that it is achieving what it sets out to achieve. Assessment looks at how students in a particular subject area in a university or college are taught and what they learn as a result. Audit proceeds by asking a university to document the means by which it controls the quality of a whole range of academic activities, such as research degree supervision, library and computing support, and then talks to a wide range of staff and students to assess what happens in reality. Assessment requests from an academic department information about what it aims to achieve in the design and implementation of its various degree courses and then tests that information by observation of teaching sessions, scrutiny of coursework and examination scripts, discussion with students, etc. Both audit and assessment aim to judge an institution within its own definition o f mission and not in relation to some national n o r m or gold standard. I intend to regard both systems as exemplifying developments driven by the state with a view to achieving a greater degree of accountability over the traditionally a u t o n o m o u s higher education institutions. To do so m a y be regarded as somewhat misleading for two reasons. First, assessment is undertaken by government agencies,
*Part of this paper is based on a brief article published in 1994 in HigherEducation Digest, 20 (Autumn) by the Open University's Quality Support Centre. 211
212
Anne C. Brook
the Higher Education Funding Councils, under statutory powers. Audit, however, is undertaken by a body, the Higher Education Quality Council, entirely owned by the universities and colleges themselves. Second, many of today's universities traditionally enjoyed no autonomous tradition in that they were, until very recently, integral parts of local government with no separate legal status, little financial or managerial independence and no right to award their own degrees. Nevertheless, those institutions aspired to university status as delivering managerial, financial and academic autonomy. Similarly, audit was not developed by universities as an altruistic exercise but as a calculated political attempt to avoid direct inspections by government agencies-and, as such, it failed because it did not prevent the creation of Funding Council assessment. I believe it is therefore legitimate to treat audit and assessment as equivalent examples of the tension between autonomy and accountability, even though their detailed historical contexts are different and complex. This paper will not trace in detail how audit and assessment have been working nor the full range of effects on individuals, groups, institutions and the higher education system nationally. Indeed, neither system has been operating long enough to sustain a comprehensive analysis. Both systems are changing in response to reports on their initial phases of operation.* I wish to highlight a few of the effects that I regard as positive and to indicate why I think it is unlikely that those effects would have occurred without the external, state-driven impetus. In doing so I should also sound a note of caution. Those effects described are, I believe, already there at least in embryo and, if they continue to exist and take root as integral parts of university life, I believe they would be beneficial. It is, however, equally possible that those effects may be only temporary and the benefits transitory and never fully realized. AUDIT AND A S S E S S M E N T At the most general level, audit and assessment have legitimated, or at least partially so, the allocation of considerable time and effort to identifying, disseminating and questioning the purposes of each university's basic academic activities, the means by which those purposes are carried through and the nature of the actual predicted outcomes. For example, a university must decide to what extent it should have a formal system for monitoring the academic progress of individual students, whether the system should be the same for all students regardless of discipline, how students are to know what to expect of the system, by what criteria the success or otherwise of the system should be judged, and how the system is to be revised if it does not work as intended. By asking similar questions of each of the academic activities a university undertakes, staff, students, employers and others stand to benefit from a much greater degree of clarity about what is intended. Thought has had to be given to identifying legitimate forms of evidence relating to the achievement of those intentions. Systems have not only had to be designed or existing ones revised, but their actual reality in operation has also to be examined. Perhaps ironically, given the political ideologies of the current United Kingdom government, a potentially valuable counterbalance
*The initial phase of audit was reviewed in 1993, and the resulting report--Review of Quality Audit Report by Coopers and Lybrand, published by HEQC (PublicationsCode MIS 100)--contains a useful summaryof the nature of audit. The equivalentreviewreport for assessment(in Englandand Wales)-Assessment of the Quality of Higher Education: a Review and Evaluation, published by HEFCE (Publications Code M5/94)--was produced in 1994 and forms a similarlyuseful introduction.
Is completeautonomy desirable?
213
has been created that compels institutional managers to divert their attention on a regular basis from income generation and other resource issues to academic matters-or even to recognize that the two are inextricably and intricately linked. As part of this process the perceptions of students and of employers about the education received are also achieving greater prominence. It is not implied here that the senior members of institutions have not always regarded academic matters as important, but that it has tended to be assumed that such affairs could be safely left to academics individually or in departmental groupings rather than requiring active management in a corporate sense. Arguably, any well-run organization committed to total quality management principles would have arrived at such a position without any external intervention. There are, indeed, some United Kingdom universities that may have had little to learn in this context. There is, however, a not insignificant number, particularly those who have traditionally been able to rely on selecting their students from a large, wellqualified pool rather than needing to recruit actively, for whom these changes have been significant. The importance of the student as consumer does not have the same impact on the University of Oxford as it does on the University of Wolverhampton, for example. The benefits described so far could be regarded as attainable without external inputs or, once such an external impetus has had its effects, capable of self-perpetuation without any further external intervention. However, the effect with potentially the most far-reaching benefit depends for its efficacy on continuing external inputs. One of the most difficult balancing acts inherent in both audit and assessment is that of respecting the individual and distinctive mission of each university at both institutional and subject level and the means by which each chooses to achieve that mission, while continuing to question whether all definitions of mission and all means of implementation are a priori legitimate--at least in terms of publicly funded higher education. Both those within the institution, determining the choice of mission and implementation, and those from outside, coming to the institution as auditors or assessors, need a framework of reference, if only to make a considered rejection of it. Whether it is at individual, subject or institutional level, the clarity of the picture of self is developed by reference to entities that are not self. However, the evidential base for that process of self-referencing is very different for research and for teaching.
NETWORKS
The networks for the discussion of research are worldwide and built on widely available artefacts in the form of books, articles and conferences. Almost all academic staff will regard it as important to be at least familiar with the main trends in the relevant research network, even if they are not themselves active contributors. It is not by accident that the first state intervention was in the area of research assessment and that when we speak of the academic reputation of Professor Bloggs or of Department X we mean its research reputation. The self-referencing framework is clearly in place and of long standing. However, at least in the older universities of the United Kingdom, the teaching activity is a very different matter. Although the formal curriculum may not have been quite such a "secret garden" as it was until recently in schools, the activity of teaching and learning has nevertheless been largely one between consenting adults in private.
214
Anne C. Brook
The inter-institutional and inter-departmental networks focused on teaching and learning, unlike those for research, are very patchy in both geographical and subject terms. Indeed, many staff at present see no need for such networks. In an institution such as my own, many academic staff will spend a lifetime in the profession without anyone else observing the means by which they teach students. Similarly, whether or not such staff discuss with each other the process of teaching and learning varies considerably, and may be effectively almost nil. Taken together with the relative immobility of the academic population, the evidential and experiential base for selfassessment thereby becomes a matter of concern, particularly if the professionalization of teaching, and thus its prestige, is ever to reach parity with that of research. Audit and, particularly in this context, assessment are systems that have the potential to ensure regular interaction between staff in one institution and ones drawn from other institutions of different types and cultures. Such interactions are invaluable--~ven more so if the results of those individual interactions are regularly analysed to provide "state of the nation" syntheses for the subject area involved. Such a network could, of course, be operated by the institutions themselves, entirely independent of the state. However, it is not as obvious as perhaps it ought to be that those who influence decisions, both in individual universities or in national fora such as meetings of vicechancellors, regard the health and status o f teaching as of sufficient importance for them to agree to sustain such a network of interactions for its own sake. CONCLUSION I am, therefore, forced to conclude not only that the state-driven systems of audit and assessment have had positive benefits with considerable future potential, but that, were those external forces to be removed, I am not confident that those benefits would be maintained. By putting forward such an argument, I am not seeking to gloss over the defects of audit and assessment as currently operated, or to defend direct state intervention in higher education p e r se. It would, however, be less than honest to avoid identifying those areas where positive effects may be being achieved. [Postscript. This paper was written in late I994, before the plans for a single national agency to run audit and assessment were developed. However the general argument remains relevant.]
Pergamon
S0952-8733(96)00009-8
Higher EducationPolicy, Vol. 9, No. 3, pp. 211-214, 1996 Copyright © 1996International Association of Universities Published by Elsevier Science Ltd. Printed in Great Britain 0952-8733/96 $15.00+0.00
Is complete autonomy necessarily desirable?* Anne C. Brook Head of Quality Assurance, The University of Liverpool, Liverpool L69 3BX, England The audit of systems for academic quality assurance and the assessment of the quality of education provided to students in individual subject areas are both relatively recent developments in the U.K., which derive directly or indirectly from govermnent pressure to make universities more accountable. This paper examines some aspects of the impact of those developments on universities and expresses the view that some of the effects have been both positive and of a kind which would not necessarily have occurred without such internal pressures. In particular it is argued that self-evaluation of teaching requires a well-informed view of other approaches to teaching elsewhere but that the networks to facilitate that rounded view of different approaches to teaching are poorly developed; assessment has benefited those networks. Copyright © 1996 International Association of Universities
INTRODUCTION Most people will be familiar with the two systems currently operating in the United K i n g d o m in relation to Higher Education--audit and assessment. Audit looks at the institutional-level systems that are in place to enable an individual university or college to know that it is achieving what it sets out to achieve. Assessment looks at how students in a particular subject area in a university or college are taught and what they learn as a result. Audit proceeds by asking a university to document the means by which it controls the quality of a whole range of academic activities, such as research degree supervision, library and computing support, and then talks to a wide range of staff and students to assess what happens in reality. Assessment requests from an academic department information about what it aims to achieve in the design and implementation of its various degree courses and then tests that information by observation of teaching sessions, scrutiny of coursework and examination scripts, discussion with students, etc. Both audit and assessment aim to judge an institution within its own definition o f mission and not in relation to some national n o r m or gold standard. I intend to regard both systems as exemplifying developments driven by the state with a view to achieving a greater degree of accountability over the traditionally a u t o n o m o u s higher education institutions. To do so m a y be regarded as somewhat misleading for two reasons. First, assessment is undertaken by government agencies,
*Part of this paper is based on a brief article published in 1994 in HigherEducation Digest, 20 (Autumn) by the Open University's Quality Support Centre. 211
212
Anne C. Brook
the Higher Education Funding Councils, under statutory powers. Audit, however, is undertaken by a body, the Higher Education Quality Council, entirely owned by the universities and colleges themselves. Second, many of today's universities traditionally enjoyed no autonomous tradition in that they were, until very recently, integral parts of local government with no separate legal status, little financial or managerial independence and no right to award their own degrees. Nevertheless, those institutions aspired to university status as delivering managerial, financial and academic autonomy. Similarly, audit was not developed by universities as an altruistic exercise but as a calculated political attempt to avoid direct inspections by government agencies-and, as such, it failed because it did not prevent the creation of Funding Council assessment. I believe it is therefore legitimate to treat audit and assessment as equivalent examples of the tension between autonomy and accountability, even though their detailed historical contexts are different and complex. This paper will not trace in detail how audit and assessment have been working nor the full range of effects on individuals, groups, institutions and the higher education system nationally. Indeed, neither system has been operating long enough to sustain a comprehensive analysis. Both systems are changing in response to reports on their initial phases of operation.* I wish to highlight a few of the effects that I regard as positive and to indicate why I think it is unlikely that those effects would have occurred without the external, state-driven impetus. In doing so I should also sound a note of caution. Those effects described are, I believe, already there at least in embryo and, if they continue to exist and take root as integral parts of university life, I believe they would be beneficial. It is, however, equally possible that those effects may be only temporary and the benefits transitory and never fully realized. AUDIT AND A S S E S S M E N T At the most general level, audit and assessment have legitimated, or at least partially so, the allocation of considerable time and effort to identifying, disseminating and questioning the purposes of each university's basic academic activities, the means by which those purposes are carried through and the nature of the actual predicted outcomes. For example, a university must decide to what extent it should have a formal system for monitoring the academic progress of individual students, whether the system should be the same for all students regardless of discipline, how students are to know what to expect of the system, by what criteria the success or otherwise of the system should be judged, and how the system is to be revised if it does not work as intended. By asking similar questions of each of the academic activities a university undertakes, staff, students, employers and others stand to benefit from a much greater degree of clarity about what is intended. Thought has had to be given to identifying legitimate forms of evidence relating to the achievement of those intentions. Systems have not only had to be designed or existing ones revised, but their actual reality in operation has also to be examined. Perhaps ironically, given the political ideologies of the current United Kingdom government, a potentially valuable counterbalance
*The initial phase of audit was reviewed in 1993, and the resulting report--Review of Quality Audit Report by Coopers and Lybrand, published by HEQC (PublicationsCode MIS 100)--contains a useful summaryof the nature of audit. The equivalentreviewreport for assessment(in Englandand Wales)-Assessment of the Quality of Higher Education: a Review and Evaluation, published by HEFCE (Publications Code M5/94)--was produced in 1994 and forms a similarlyuseful introduction.
Is completeautonomy desirable?
213
has been created that compels institutional managers to divert their attention on a regular basis from income generation and other resource issues to academic matters-or even to recognize that the two are inextricably and intricately linked. As part of this process the perceptions of students and of employers about the education received are also achieving greater prominence. It is not implied here that the senior members of institutions have not always regarded academic matters as important, but that it has tended to be assumed that such affairs could be safely left to academics individually or in departmental groupings rather than requiring active management in a corporate sense. Arguably, any well-run organization committed to total quality management principles would have arrived at such a position without any external intervention. There are, indeed, some United Kingdom universities that may have had little to learn in this context. There is, however, a not insignificant number, particularly those who have traditionally been able to rely on selecting their students from a large, wellqualified pool rather than needing to recruit actively, for whom these changes have been significant. The importance of the student as consumer does not have the same impact on the University of Oxford as it does on the University of Wolverhampton, for example. The benefits described so far could be regarded as attainable without external inputs or, once such an external impetus has had its effects, capable of self-perpetuation without any further external intervention. However, the effect with potentially the most far-reaching benefit depends for its efficacy on continuing external inputs. One of the most difficult balancing acts inherent in both audit and assessment is that of respecting the individual and distinctive mission of each university at both institutional and subject level and the means by which each chooses to achieve that mission, while continuing to question whether all definitions of mission and all means of implementation are a priori legitimate--at least in terms of publicly funded higher education. Both those within the institution, determining the choice of mission and implementation, and those from outside, coming to the institution as auditors or assessors, need a framework of reference, if only to make a considered rejection of it. Whether it is at individual, subject or institutional level, the clarity of the picture of self is developed by reference to entities that are not self. However, the evidential base for that process of self-referencing is very different for research and for teaching.
NETWORKS
The networks for the discussion of research are worldwide and built on widely available artefacts in the form of books, articles and conferences. Almost all academic staff will regard it as important to be at least familiar with the main trends in the relevant research network, even if they are not themselves active contributors. It is not by accident that the first state intervention was in the area of research assessment and that when we speak of the academic reputation of Professor Bloggs or of Department X we mean its research reputation. The self-referencing framework is clearly in place and of long standing. However, at least in the older universities of the United Kingdom, the teaching activity is a very different matter. Although the formal curriculum may not have been quite such a "secret garden" as it was until recently in schools, the activity of teaching and learning has nevertheless been largely one between consenting adults in private.
214
Anne C. Brook
The inter-institutional and inter-departmental networks focused on teaching and learning, unlike those for research, are very patchy in both geographical and subject terms. Indeed, many staff at present see no need for such networks. In an institution such as my own, many academic staff will spend a lifetime in the profession without anyone else observing the means by which they teach students. Similarly, whether or not such staff discuss with each other the process of teaching and learning varies considerably, and may be effectively almost nil. Taken together with the relative immobility of the academic population, the evidential and experiential base for selfassessment thereby becomes a matter of concern, particularly if the professionalization of teaching, and thus its prestige, is ever to reach parity with that of research. Audit and, particularly in this context, assessment are systems that have the potential to ensure regular interaction between staff in one institution and ones drawn from other institutions of different types and cultures. Such interactions are invaluable--~ven more so if the results of those individual interactions are regularly analysed to provide "state of the nation" syntheses for the subject area involved. Such a network could, of course, be operated by the institutions themselves, entirely independent of the state. However, it is not as obvious as perhaps it ought to be that those who influence decisions, both in individual universities or in national fora such as meetings of vicechancellors, regard the health and status o f teaching as of sufficient importance for them to agree to sustain such a network of interactions for its own sake. CONCLUSION I am, therefore, forced to conclude not only that the state-driven systems of audit and assessment have had positive benefits with considerable future potential, but that, were those external forces to be removed, I am not confident that those benefits would be maintained. By putting forward such an argument, I am not seeking to gloss over the defects of audit and assessment as currently operated, or to defend direct state intervention in higher education p e r se. It would, however, be less than honest to avoid identifying those areas where positive effects may be being achieved. [Postscript. This paper was written in late I994, before the plans for a single national agency to run audit and assessment were developed. However the general argument remains relevant.]