- Email: [email protected]
Uncertainties in medical audit
Public Health (1997) 111,67-69 8 The Society of Public Health, 1997
Leading article Uncertainties in medical audit DP Forster 11 Ashdale, Ponteland, Northumberland,
NE20 9DR
Keywords: Medical audit; uncertainty; organisational theory
Introduction
The fundamental objective of medical audit, namely that of improving the quality of care which doctors and other professionals provide for patients, is widely accepted. The task of medical audit is to change the clinical approach of professionals, in order to provide better care for patients. The key features of the process of medical audit are that it should be structured, systematic and preferably quantified. ’ There are relatively few studies with a rigorous research design which have evaluated the effectiveness of audit.’ Moreover, few studies relate effectiveness to the costs of the audit exercise or investigate the opportunity costs.3 The informal modification of practice occurs continuously as practitioners respond, according to their perceptions, to scientific advances or discussion about their procedures. The critical question, therefore, is whether formalising audit produces appropriate change, and if so, how and at what cost? The difficulty of demonstrating effectiveness, in the absence of randomised controlled trials, is present even for well respected external audit procedures. For many years, maternal mortality has been considered to be a condition which may be potentially avoidable by the use of timely, appropriate medical care.4*5 The Confidential Enquiry investigates these events and reports the findings in a collective epidemiological format. However, scientific advances and changes in health or social policy act as serious confounding factors when attempting to interpret the value of audit in data trends. Even if improvements in outcome or process measures are a result of audit procedures like a Confidential Enquiry, it is uncertain whether these are through the lessons learned and subsequent changes in clinical approach by the individual clinicians involved in the untoward event, and perhaps by their immediate colleagues; or through the dissemination of the epidemiological information from the enquiry at a collective level to all clinicians concerned with the care of pregnant women. Contrary to the theory of medical audit, the actual evaluation of the effectiveness of audit has been sporadic rather than systematic. In particular, even if effectiveness is proven, it is uncertain which aspects of audit lead to a change to a more appropriate clinical approach. The difference between research and medical audit is not in the depth or quality of the investigation and measurements, Correspondence: Dr DP Forster. Accepted 20 December 1996
nor indeed in the desire to intluence practice. It lies in the fact that good medical audit should be continuous, cyclical and therefore repetitive. Medical audit’s very strength, therefore, is its weakness since it involves maintaining the long term motivation of health professionals, and often the setting up of complex, expensive information systems. A view from organisational theory
The approach to audit has been rather mechanistic and the motivation of individuals to involve themselves in audit and subsequently to modify their clinical approach, if necessary, has been little considered. The readiness, and hence potential, for active participation and change is governed mainly by the degree of dissatisfaction felt by an individual professional when the current clinical situation is compared with the desired future.6*7 The presence or otherwise of dissatisfaction as a motivating factor is influenced to a significant extent by the values of the individuals concerned. Given the ethos of medicine, these values are likely to be derived from the interaction of doctors with patients and relatives. Indeed, successful commercial companies learn from the people they serve and pride themselves in being ‘close to the customer’.* The most valuable task of medical audit, therefore, is to capture the motivating influence of the personal feelings in health professionals of dissatisfaction with the clinical service arising from doctor-patient interactions, yet translate this into the objective recordings of an evaluation. This is potentially the most productive route for assembling information which can produce agreed policies and services for the improvement of care. The difficulties of implementing medical audit at this sensitive level should not be underestimated. For instance, in medical education teaching on topics such as the care of the dying, bereavement and communication were introduced into the curriculum at a relatively late stage.g Moreover, in current practice, for example in general practice and community child health, a plea is still being made for doctors to listen carefully to the parents when there is childhood illness.” A view from the natural sciences
In quantum physics, the simultaneous measurement of the position and momentum of atomic matter has an inherent indeterminacy (Heisenberg’s uncertainty principle).” The uncertainty is present regardless of the accuracy of the measuring instruments, technique or the observer. The
Uncertaintiesin medical audit DP Forster
68
uncertainties in measurement are only important when the ~~entals of matter are being investigated, for example atoms and subatomic particles. The act of trying to measure the position and moment of particles such as electrons disturbs the system in a manner which is unpredictable. That is to say, measurement itself creates change at this level. The similarity here with human behaviour is that, when the actions of humans at work are closely observed in experimental (or audit) situations, there is a change in work behaviour.i2 The important principle common to physical and human systems is that these discoveries concerning change refer to the most detailed levels of measurement. In human systems of course, as a prelude to decisions about change in behaviour, there is the added complexity of varying perceptions about the observers, particularly about the values they hold and about the merits or otherwise of the observations and measurements being made. The success or otherwise of medical audit in stimulating appropriate change depends, therefore, on the perceptions of those who are being audited. It is a small step, therefore, to see the advantages from professionals intemalising the concept of audit with an acceptable peer group and of self audit. The role of management
If the objective of medical audit is to improve the care of patients, then the context in which this is carried out should be one which encourages all aspects of quality. The concept of audit for quality sits uneasily in a National Health Service in which there are restrictive clauses in employment contracts concerning the right to speak freely on matters of public importance.‘3 The views expressed by individual caring professionals may not in themselves amount to audit since they do not futfil the criterion of being systematic, but they usually arise from genuine concerns. Since the essence of medical audit is that it is systematic, there has been a readiness by management to provide information system s~pport.‘~ If audit is successful, good practice with a favourable outcome for patients or ways of reducing risks in medical care are identified. Some risks will be remediable at little or no cost by appropriate changes in medical practice; some will require expenditure to reduce or eliminate them whilst others may prove to be unavoidable. The managerial role in audit is to specify the cost of reducing a particular risk to patients, identified from the medical audit. A doctor’s commitment to investigate systematically should be complemented by a manager’s concomitant and systematic response in indicating the threshold level of identified risk at which the provider would intervene with further expenditure. When funds for medical audit used to be hypothecated from the centre, there was, crucially, no comparable pledge about money for risk reduction measures successfully revealed through the medial audit process.”
professionals. Models from the seemingly disparate field of organisational theory and the natural sciences suggest that, in order to have the best chance of influencing clinical behaviour, medical audit should be focussed on the actual interactions between doctor and patient; the ways in which clinical teams work together; and how the lessons of audit are intemalised. The corollary is that effective medical audit may not be necessarily helped by the proliferation of distant audit committees, long lines of coruscation and remote information systems. The dangers lie in a displacement of the goals of the NBS away from curing and caring to an undue emphasis on management control systems and activities becoming focused on meeting the procedures of the organisation rather than on its true goals.‘4*‘6*‘7 It has been argued that, if medical audit is to be effective, it should be concentrated on those areas of practice in which clinical values are principally located. In addition, the objective data that are really required are often not obtainable or cannot always be captured in a sufficiently sensitive form by the use of questionnaires or surveys. Indeed, Powell has argued that in the changed NHS, attempts are being made to measure the unmeasurable.ls More likely, it is a matter of achieving the right balance between the objective and the subjective. In a different but relevant context, Charles Booth in the late nineteenth century was the pioneer of scientific surveys, the origins of which arose from his personal preception of and concern for the poverty and living conditions of the working class. The balance in his approach would not be amiss in considering medical audit today: ‘In intensity of feeling, and not in statistics lies the power to move the world. But by statistics must this power be guided if it would move the world aright’.” Decisions in medicine must remain an art as well as a science. Acknowledgement I am very grateful to Dr Christine Forster for discussions about Heisenberg’s uncertainty principle. References Department of Health. The Quality of Medical Care. Report of the Standing Medical Advisory Committee. HMSO: London, 1990. Committee for North of England Study of Standards and Performance in General P&tice. Medical audit in general practice. I: effects of doctors’ clinical behaviour for common childhood conditions. 3hQ 1992; 384: 1480-1484. Maynard A. Casefor auditing audit. Health ServJ 1991; 101: 26, Rutstein DD, Berenberg W, ChalmersTC, Child CG, Fishman AP, Perrin EB. Measuring the quality of medical care. N Engl JMed 1976; 11: 582-588. Mackenbach JP, Bouvier-Colle MI-I, Jougla E. ‘Avoidable’ mortality and health services: a review of aggregate data studies.J Epidemiol Comm Health 1990; 44: 106-l Il. 6 Kolb DA, Rubin IM, MacIntyre JM. Organizational Psychologv: an Experienrial Approach to Organizational Behaviour.
A balanced approach The increasing importance of the scientific approach to medicine is inevitable and necessary. An extension of this is the measurement approach of medical audit. In the fhtal analysis, however, medicine is about the care and treatment of an individual either by an individual or a small group of
Prentice Hall: Englewood Cliffs, New Jersey, 1984. 7 Forster DP, Hadley R. The NHS Reforms: conditions for successfulchange?Health Serv h@mt 1989; 85: 215-218. 8 PetersTJ, Waterman RH. In Search of Excellence.Harper and Row: London, 1982. 9 General Medical Council: Education Committee. Recommendations on basic medical education. GeneralMedical Council: London, 1980.
Uncertainties DP Forster
10 Roberts H. Listen to the parents. BMJ 1996; 313: 954-955.
11 Heisenberg W. ijber den anschaulichen Inhalt der QuantentheoretischenKinematik und Mechanik. ZPhysik 1927; 43: 172-l 98. 12 Mayo E. The Human Problems of an Industrial Civilisation. Macmillan: New York, 1933. 13 Greene D, Cooper J. Whistle blowers. BMJ 1992; 305: 13431344. 14 Bowden D, Walshe K. When medical audit starts to count. BMJ 1991; 303: 101-103.
in medical audit
15 NHS Management Executive. Medical audit--allocation of finds 1992-93 Executive Letter EL(92)21, Department of Health, 1992. 16 Hadley R, Forster DP (eds). Docfors as Managers: Experiences in the Front-Line of the NHS. Longmans: Harlow, 1993. 17 Etzioni A. Modem Organizations. Prentice Hall: Englewood Cliffs, New Jersey, 1964. 18 Powell E. Lost in a maze of hopelesschange. BMA News Rev 1992; 18: 11. 19 Booth C (ed). Lubour and Life of the People of London. Macmillan: London, 1889-1902.
69
Leading article Uncertainties in medical audit DP Forster 11 Ashdale, Ponteland, Northumberland,
NE20 9DR
Keywords: Medical audit; uncertainty; organisational theory
Introduction
The fundamental objective of medical audit, namely that of improving the quality of care which doctors and other professionals provide for patients, is widely accepted. The task of medical audit is to change the clinical approach of professionals, in order to provide better care for patients. The key features of the process of medical audit are that it should be structured, systematic and preferably quantified. ’ There are relatively few studies with a rigorous research design which have evaluated the effectiveness of audit.’ Moreover, few studies relate effectiveness to the costs of the audit exercise or investigate the opportunity costs.3 The informal modification of practice occurs continuously as practitioners respond, according to their perceptions, to scientific advances or discussion about their procedures. The critical question, therefore, is whether formalising audit produces appropriate change, and if so, how and at what cost? The difficulty of demonstrating effectiveness, in the absence of randomised controlled trials, is present even for well respected external audit procedures. For many years, maternal mortality has been considered to be a condition which may be potentially avoidable by the use of timely, appropriate medical care.4*5 The Confidential Enquiry investigates these events and reports the findings in a collective epidemiological format. However, scientific advances and changes in health or social policy act as serious confounding factors when attempting to interpret the value of audit in data trends. Even if improvements in outcome or process measures are a result of audit procedures like a Confidential Enquiry, it is uncertain whether these are through the lessons learned and subsequent changes in clinical approach by the individual clinicians involved in the untoward event, and perhaps by their immediate colleagues; or through the dissemination of the epidemiological information from the enquiry at a collective level to all clinicians concerned with the care of pregnant women. Contrary to the theory of medical audit, the actual evaluation of the effectiveness of audit has been sporadic rather than systematic. In particular, even if effectiveness is proven, it is uncertain which aspects of audit lead to a change to a more appropriate clinical approach. The difference between research and medical audit is not in the depth or quality of the investigation and measurements, Correspondence: Dr DP Forster. Accepted 20 December 1996
nor indeed in the desire to intluence practice. It lies in the fact that good medical audit should be continuous, cyclical and therefore repetitive. Medical audit’s very strength, therefore, is its weakness since it involves maintaining the long term motivation of health professionals, and often the setting up of complex, expensive information systems. A view from organisational theory
The approach to audit has been rather mechanistic and the motivation of individuals to involve themselves in audit and subsequently to modify their clinical approach, if necessary, has been little considered. The readiness, and hence potential, for active participation and change is governed mainly by the degree of dissatisfaction felt by an individual professional when the current clinical situation is compared with the desired future.6*7 The presence or otherwise of dissatisfaction as a motivating factor is influenced to a significant extent by the values of the individuals concerned. Given the ethos of medicine, these values are likely to be derived from the interaction of doctors with patients and relatives. Indeed, successful commercial companies learn from the people they serve and pride themselves in being ‘close to the customer’.* The most valuable task of medical audit, therefore, is to capture the motivating influence of the personal feelings in health professionals of dissatisfaction with the clinical service arising from doctor-patient interactions, yet translate this into the objective recordings of an evaluation. This is potentially the most productive route for assembling information which can produce agreed policies and services for the improvement of care. The difficulties of implementing medical audit at this sensitive level should not be underestimated. For instance, in medical education teaching on topics such as the care of the dying, bereavement and communication were introduced into the curriculum at a relatively late stage.g Moreover, in current practice, for example in general practice and community child health, a plea is still being made for doctors to listen carefully to the parents when there is childhood illness.” A view from the natural sciences
In quantum physics, the simultaneous measurement of the position and momentum of atomic matter has an inherent indeterminacy (Heisenberg’s uncertainty principle).” The uncertainty is present regardless of the accuracy of the measuring instruments, technique or the observer. The
Uncertaintiesin medical audit DP Forster
68
uncertainties in measurement are only important when the ~~entals of matter are being investigated, for example atoms and subatomic particles. The act of trying to measure the position and moment of particles such as electrons disturbs the system in a manner which is unpredictable. That is to say, measurement itself creates change at this level. The similarity here with human behaviour is that, when the actions of humans at work are closely observed in experimental (or audit) situations, there is a change in work behaviour.i2 The important principle common to physical and human systems is that these discoveries concerning change refer to the most detailed levels of measurement. In human systems of course, as a prelude to decisions about change in behaviour, there is the added complexity of varying perceptions about the observers, particularly about the values they hold and about the merits or otherwise of the observations and measurements being made. The success or otherwise of medical audit in stimulating appropriate change depends, therefore, on the perceptions of those who are being audited. It is a small step, therefore, to see the advantages from professionals intemalising the concept of audit with an acceptable peer group and of self audit. The role of management
If the objective of medical audit is to improve the care of patients, then the context in which this is carried out should be one which encourages all aspects of quality. The concept of audit for quality sits uneasily in a National Health Service in which there are restrictive clauses in employment contracts concerning the right to speak freely on matters of public importance.‘3 The views expressed by individual caring professionals may not in themselves amount to audit since they do not futfil the criterion of being systematic, but they usually arise from genuine concerns. Since the essence of medical audit is that it is systematic, there has been a readiness by management to provide information system s~pport.‘~ If audit is successful, good practice with a favourable outcome for patients or ways of reducing risks in medical care are identified. Some risks will be remediable at little or no cost by appropriate changes in medical practice; some will require expenditure to reduce or eliminate them whilst others may prove to be unavoidable. The managerial role in audit is to specify the cost of reducing a particular risk to patients, identified from the medical audit. A doctor’s commitment to investigate systematically should be complemented by a manager’s concomitant and systematic response in indicating the threshold level of identified risk at which the provider would intervene with further expenditure. When funds for medical audit used to be hypothecated from the centre, there was, crucially, no comparable pledge about money for risk reduction measures successfully revealed through the medial audit process.”
professionals. Models from the seemingly disparate field of organisational theory and the natural sciences suggest that, in order to have the best chance of influencing clinical behaviour, medical audit should be focussed on the actual interactions between doctor and patient; the ways in which clinical teams work together; and how the lessons of audit are intemalised. The corollary is that effective medical audit may not be necessarily helped by the proliferation of distant audit committees, long lines of coruscation and remote information systems. The dangers lie in a displacement of the goals of the NBS away from curing and caring to an undue emphasis on management control systems and activities becoming focused on meeting the procedures of the organisation rather than on its true goals.‘4*‘6*‘7 It has been argued that, if medical audit is to be effective, it should be concentrated on those areas of practice in which clinical values are principally located. In addition, the objective data that are really required are often not obtainable or cannot always be captured in a sufficiently sensitive form by the use of questionnaires or surveys. Indeed, Powell has argued that in the changed NHS, attempts are being made to measure the unmeasurable.ls More likely, it is a matter of achieving the right balance between the objective and the subjective. In a different but relevant context, Charles Booth in the late nineteenth century was the pioneer of scientific surveys, the origins of which arose from his personal preception of and concern for the poverty and living conditions of the working class. The balance in his approach would not be amiss in considering medical audit today: ‘In intensity of feeling, and not in statistics lies the power to move the world. But by statistics must this power be guided if it would move the world aright’.” Decisions in medicine must remain an art as well as a science. Acknowledgement I am very grateful to Dr Christine Forster for discussions about Heisenberg’s uncertainty principle. References Department of Health. The Quality of Medical Care. Report of the Standing Medical Advisory Committee. HMSO: London, 1990. Committee for North of England Study of Standards and Performance in General P&tice. Medical audit in general practice. I: effects of doctors’ clinical behaviour for common childhood conditions. 3hQ 1992; 384: 1480-1484. Maynard A. Casefor auditing audit. Health ServJ 1991; 101: 26, Rutstein DD, Berenberg W, ChalmersTC, Child CG, Fishman AP, Perrin EB. Measuring the quality of medical care. N Engl JMed 1976; 11: 582-588. Mackenbach JP, Bouvier-Colle MI-I, Jougla E. ‘Avoidable’ mortality and health services: a review of aggregate data studies.J Epidemiol Comm Health 1990; 44: 106-l Il. 6 Kolb DA, Rubin IM, MacIntyre JM. Organizational Psychologv: an Experienrial Approach to Organizational Behaviour.
A balanced approach The increasing importance of the scientific approach to medicine is inevitable and necessary. An extension of this is the measurement approach of medical audit. In the fhtal analysis, however, medicine is about the care and treatment of an individual either by an individual or a small group of
Prentice Hall: Englewood Cliffs, New Jersey, 1984. 7 Forster DP, Hadley R. The NHS Reforms: conditions for successfulchange?Health Serv h@mt 1989; 85: 215-218. 8 PetersTJ, Waterman RH. In Search of Excellence.Harper and Row: London, 1982. 9 General Medical Council: Education Committee. Recommendations on basic medical education. GeneralMedical Council: London, 1980.
Uncertainties DP Forster
10 Roberts H. Listen to the parents. BMJ 1996; 313: 954-955.
11 Heisenberg W. ijber den anschaulichen Inhalt der QuantentheoretischenKinematik und Mechanik. ZPhysik 1927; 43: 172-l 98. 12 Mayo E. The Human Problems of an Industrial Civilisation. Macmillan: New York, 1933. 13 Greene D, Cooper J. Whistle blowers. BMJ 1992; 305: 13431344. 14 Bowden D, Walshe K. When medical audit starts to count. BMJ 1991; 303: 101-103.
in medical audit
15 NHS Management Executive. Medical audit--allocation of finds 1992-93 Executive Letter EL(92)21, Department of Health, 1992. 16 Hadley R, Forster DP (eds). Docfors as Managers: Experiences in the Front-Line of the NHS. Longmans: Harlow, 1993. 17 Etzioni A. Modem Organizations. Prentice Hall: Englewood Cliffs, New Jersey, 1964. 18 Powell E. Lost in a maze of hopelesschange. BMA News Rev 1992; 18: 11. 19 Booth C (ed). Lubour and Life of the People of London. Macmillan: London, 1889-1902.
69