- Email: [email protected]
Quality assurance issues and PACS
Int J Biomed Comput, 30 (1992) 249-253
249
Elsevier Scientific Publishers Ireland Ltd.
QUALITY ASSURANCE
ISSUES AND PACS
H.D. BANTA TNO/WHO Programme in Health Care Technology Assessment, Leiden (The Netherlands)
Quality assessment and assurance is a growing concern in all areas of health care. The concern is fueled by a body of evidence that indicates that quality of care is not optimal, and in many instances, is unacceptably low. Although different standards for quality have been proposed, health outcome is the ultimate standard, since improving health is the goal of the health care system. The effects of PACS on health outcome are not known. A PAC system has the potential to improve quality, especially if it makes the diagnostic process more efficient, but evidence that such improvement actually occurs is so far lacking. Keywords: PACS; Quality of care; Diagnostic process
Introduction
The quality of medical and health care is becoming an increasingly visible public issue in a number of countries. While the overall results of advances in health care seem relatively clear, there is considerable evidence that the care being given is not optimal..Many technologies have not been shown to be of benefit. Many others are obviously overused and inappropriately used. Variations in use of technologies from hospital to hospital, region to region, and country to country show that practice patterns are not based on best scientific evidence. The objective of quality assurance is to ‘improve the outcome of all health care in terms of health, functional ability, patient wellbeing and consumer satisfaction’ [ 11.The term ‘quality’ often is used in different ways and may denote characteristics such as effectiveness, efficacy, efficiency, equity, acceptability, accessibility, adequacy, and scientific/ technical quality. Standards for Quality Assurance
The Offtce of Technology Assessment [2] has defined quality of medical care as ‘the degree to which the process of care increases the probability of outcomes desired by patients and reduces the probability of undesired outcomes, given the state of medical knowledge.’ Quality assessment measures and perhaps monitors the quality of health care. Quality assurance seeks to safeguard and improve quality. Donabedian [3,4] provided the generally accepted classification of techniques when he used the terms structure, process, and outcome. As Donabedian points out, Correspondence to: H.D. Banta, TNO/WHO Programme in Health Care Technology Assessment, CMTRNO P.O. Box 430, 2300 AK Leiden, The Netherlands. 0020-7101/92/$05.00
0 1992 Elsevier Scientific Publishers lreland Ltd. Printed and Published in Ireland
250
H. D. Banta
evaluating structure requires the acceptance of two assumptions: that better care is more likely to be provided when better qualified staff, better physical facilities, and so forth are employed; and that available knowledge allows one to identify what is good in terms of staff, physical structure, and organization. Process is the evaluation of the actual activities of providers, but requires specifying which activities are appropriate, which means having specific knowledge about the links between certain health care processes and health outcomes. Assessment of outcomes is the evaluation of end results of health care in terms of health and satisfaction. It should be emphasized that health outcome involves more than survival [5]. Donabedian includes recovery and restoration of function as outcomes, for example. Most quality assurance activities have dealt with the structure or process of care, but there is a growing trend for quality assessment to use measures of health outcome. Still, the experience with quality assurance using outcome is limited. Evaluating process of care is easier, but it produces less definitive information. The problem is that the relationship between standard health care procedures and health outcomes is often not known. There are two problems [6]. One is knowing whether physicians follow currently accepted medical procedures in their practices. In other words, are they using technology effectively? The second is establishing the effect on outcomes under ideal circumstances, that is, efficacy. Another important issue is that of patient satisfaction. Donabedian includes satisfaction as a separate category to be assessed. However, satisfaction is clearly related to health outcome. If the patient does not feel that the outcome is satisfactory, physician evaluation of ‘objective’ factors such as physiologic state has limited meaning. Increasingly, patient satisfaction is thought of as a dimension of health outcome. The term quality implies a degree of excellence. To measure such excellence, a standard of comparison must exist and a method of measurement must be available. The development of broad goals for quality is not a problem. Making specific standards is. Using standards to evaluate quality of care requires criteria by which to judge how a health condition or disease has been diagnosed or treated [l]. The development of such criteria, if they are to be valid, must be based on knowledge about (at least) the efficacy and safety of the health care technology concerned. Despite the relative lack of such information, a start must be made. Limited knowledge is no excuse for lack of action. The Relation between PACS and Quality of Care
It is difficult to relate PACS to quality of care, as defined by health outcome. The effect of diagnosis on health is often difficult to demonstrate. Diagnosis must lead to effective therapy, and such therapy is often not available. In general, therefore, diagnostic technologies have been evaluated technically; occasionally, their precision has been evaluated; effects on therapy have sometimes been evaluated; they have seldom been evaluated for their effects on health. Since the main hoped-for effect from PACS is more efficient diagnosis, PACS will suffer from the same problem. Nonetheless, I think that standards of quality should be chosen because they are likely to be associated with improved outcomes. Crowe [7] has presented a rather comprehensive list of benefits to the patient. Several areas seem most interesting.
Quality assurance
251
X-ray Dose A major orientation in technological developments in the field of diagnostic imaging has been to reduce the dose of ionizing radiation delivered to people, since this radiation is associated with risks [8]. These efforts have been historically successful, through such innovations as control of the spread of the X-ray beam, development of image intensifiers, and ability to reconstruct and manipulate an image through computers. Also, the rapid growth of ultrasound imaging and magnetic resonance imaging is in part because they are probably safer than imaging based on X-rays or other ionizing radiation. One reason for excessive radiation dose to patients is repeat films because of poor quality X-ray images. Image quality can also be enhanced by image processing, which is not possible with film. Some evidence is available that computer assisted medical imaging can reduce radiation dose to the patient [9-l 11. Improved and More Effkient Diagnosis
Diagnostic imaging leads to better diagnosis. For the physician, the first concern with PACS will be that the diagnostic accuracy is at least as high as with films. The technical accuracy of digital images has been extensively studied [ 121. Another issue is that the diagnostic process should be just as efficient (if not more so) as it was under conventional methods. Since conventional radiology is now highly automated in many ways, this may be difficult to demonstrate. At present, images are available to the physician in three ways: (1) in emergencies, the images are sent immediately, and a report follows; (2) in ordinary situations, images and report are sent later, perhaps the next day; (3) in certain situations, and at the request of the clinical physician, consultations are held between referring physicians and radiologists to discuss diagnosis and sometimes therapy [8]. One possibly important effect of PACS is that clinical physicians must now often visit the radiology department to examine the images and discuss them with the radiologist. With PACS, it will be possible to transmit the image to any site with the appropriate equipment and discuss it by telephone, with significant time saving for the clinical physician. However, the possibly negative implications of less face-to-face consultation between physicians need to be considered as well. In addition, the ready access of other patient data through the Hospital Information System (HIS) may more than counter-balance such implications. Improved Access to Images
Sometimes it is critically important to have rapid access to images, as in emergency situations. A patient might have an abnormality on an image, for example, and only comparison with an older image could indicate whether it was new or not. It can sometimes be difficult to gain access to X-ray films from the archives. Sometimes it takes hours or longer to find or access an image in the present system, because of the limited number of copies, multiple archives, overloaded archive personnel, lost or wrongly placed images, or loaned images. Computer assisted imaging might improve access [13]. In particular, images can no longer be lost with such systems
252
H.D. Banta
(although discs can possibly be lost). Furthermore, multiple copies can be easily made, increasing availability. Some medical imaging services are only available in central locations, while others are quite diffused. CT scanners and MRI devices are generally quite centralized, in academic hospitals. Nuclear medicine is more widely diffused, but is only available through departments of radiology, nuclear medicine, or medical imaging. Conventional X-ray tends to be centralized into such departments, but can also be available at the primary care level and in remote sites. Ultrasound is widely distributed throughout the system. PACS could be particularly important when the patient is in a remote site. It could even be the case that no one with expertise in interpreting images was available. The image could be transmitted to a central site for interpretation. There is some data on the success of such a development for the physician [14]. Although such interpretation might be life-saving for the patient in some instances, no data are available to indicate how often such a development would be of benefit. Improving access to images might lead to reduced length of stay in the hospital. De Simone et al. [15] found that films were viewed earlier when PACS was in use; the authors assumed a shortened length of stay but did not document it. In the evaluation of the Utrecht prototype system, it was concluded that a reduction ‘might perhaps be achieved’ [16]. In addition to economic benefits, this has quality implications. For example, hospital-acquired infections might be reduced. Patient Satisfaction The present diagnostic process is not very efficient with the time of the patient. Patients must often return for additional examinations on a different day. Extra examinations are also taken when films are of poor quality or if they are lost. In addition, diagnostic services tend to be centralized, so the patient is sometimes required to travel relatively long distances. Presumably, a more efficient diagnostic process would be associated with a higher degree of patient satisfaction. If services were less centralized, patients would probably find this desirable. Shorter length of stays would probably be desirable to most patients. Inevitably, concerns about secrecy will appear [17]. Patient satisfaction with a PACS system would surely be less if secrecy were not fully protected. Conclusions The effects of PACS on quality are unknown, and have hardly been evaluated [18]. In this paper, I have speculated concerning some of the most important characteristics of PACS. These speculations need to be confirmed by evaluation as PACS becomes diffused. References I
World Health Organization. Regional Office for Europe: Quality assurance of health services. In: Thirty-eighth Session, Technical Discussions, Copenhagen, 12-11 September, 1988. (Document no. EUR/RC38TTech.Disc.A)
Quality assurance
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17
253
Offtce of Technology Assessment: The quality of medical care, information for consumers, US Government Printing O&X, Washington, DC, 1988. Donabedian A: Evaluating the quality in medical care, Milbanh Mem Fund Q, 44 (2) (1966) 166-206. Donabedian A: Explorations in Quality Assessment andMonitoring, Vol. 1, The Definition of Quality and Approaches to its Assessment, Health Administration Press, Ann Arbor, Michigan, 1980. Offtce of Technology Assessment: Assessing the Eeffkacy and Safety of Medical Technologies, US Government Printing Office, Washington, DC, 1978. Brook RH: Quality: can we measure it? New Engl J Med 296 (1977) 170-172. Crowe B: Overview of some Methodological Problems in Assessment of PACS, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. Economic Commission for Europe, United Nations. In: Digital imaging in Health Care, Working Party on Engineering Industries and Automation: New York, 1987 (ISBN 92-l-l 16380-3). Bailey R: Patient exposure requirements for high resolution in digital radiographic systems, Am J Radio1 142 (1984) 608-611. Fraser R, Sanders C and Barnes G: Digital imaging of the chest, Radiology 171 (1989) 769-773. McGrohan L, Patterson JF, Gagne RM and Goldstein HA: Average radiation doses in a standard head examination for 250 CT systems, Radiology 163 (1987) 263-268. Bameveld Binkhuysen F: Requiredfunctionality of PACS from a clinicalpoint of view, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. Banta HD and van Beekum WT (Eds): Computer Assisted Medical Imaging, The Case of Picture Archiving and Communications Systems (PACS), Kluwer Academic Publishers, Dordrecht, 1988. Horii S, Mun S, Elliott L et al: PACS clinical experience at Georgetown University, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. De Simone D, Kundel H, Arenson R, Seshadn S et al: Effect of a digital-imaging network on physician behavior in an intensive care unit, Rudiofogy 169 (1988) 41-44. Andriessen J, ter Haar Romeny B, Bamevald Binkhuysen F and van der Horst-Bruinsma I: Savings and costs of a picture archiving and communication system in the University Hospital Utrecht, Proc Med Im III Conf SPZE, 1093 (1989) 578-584. Irie G and Miyasaka K: Feasibility of PACS based on 2 years experiences of HV-PACS,
Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. 18 Flagle C: The inflwnce of PACS on quality of care at the clinical level, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991.
249
Elsevier Scientific Publishers Ireland Ltd.
QUALITY ASSURANCE
ISSUES AND PACS
H.D. BANTA TNO/WHO Programme in Health Care Technology Assessment, Leiden (The Netherlands)
Quality assessment and assurance is a growing concern in all areas of health care. The concern is fueled by a body of evidence that indicates that quality of care is not optimal, and in many instances, is unacceptably low. Although different standards for quality have been proposed, health outcome is the ultimate standard, since improving health is the goal of the health care system. The effects of PACS on health outcome are not known. A PAC system has the potential to improve quality, especially if it makes the diagnostic process more efficient, but evidence that such improvement actually occurs is so far lacking. Keywords: PACS; Quality of care; Diagnostic process
Introduction
The quality of medical and health care is becoming an increasingly visible public issue in a number of countries. While the overall results of advances in health care seem relatively clear, there is considerable evidence that the care being given is not optimal..Many technologies have not been shown to be of benefit. Many others are obviously overused and inappropriately used. Variations in use of technologies from hospital to hospital, region to region, and country to country show that practice patterns are not based on best scientific evidence. The objective of quality assurance is to ‘improve the outcome of all health care in terms of health, functional ability, patient wellbeing and consumer satisfaction’ [ 11.The term ‘quality’ often is used in different ways and may denote characteristics such as effectiveness, efficacy, efficiency, equity, acceptability, accessibility, adequacy, and scientific/ technical quality. Standards for Quality Assurance
The Offtce of Technology Assessment [2] has defined quality of medical care as ‘the degree to which the process of care increases the probability of outcomes desired by patients and reduces the probability of undesired outcomes, given the state of medical knowledge.’ Quality assessment measures and perhaps monitors the quality of health care. Quality assurance seeks to safeguard and improve quality. Donabedian [3,4] provided the generally accepted classification of techniques when he used the terms structure, process, and outcome. As Donabedian points out, Correspondence to: H.D. Banta, TNO/WHO Programme in Health Care Technology Assessment, CMTRNO P.O. Box 430, 2300 AK Leiden, The Netherlands. 0020-7101/92/$05.00
0 1992 Elsevier Scientific Publishers lreland Ltd. Printed and Published in Ireland
250
H. D. Banta
evaluating structure requires the acceptance of two assumptions: that better care is more likely to be provided when better qualified staff, better physical facilities, and so forth are employed; and that available knowledge allows one to identify what is good in terms of staff, physical structure, and organization. Process is the evaluation of the actual activities of providers, but requires specifying which activities are appropriate, which means having specific knowledge about the links between certain health care processes and health outcomes. Assessment of outcomes is the evaluation of end results of health care in terms of health and satisfaction. It should be emphasized that health outcome involves more than survival [5]. Donabedian includes recovery and restoration of function as outcomes, for example. Most quality assurance activities have dealt with the structure or process of care, but there is a growing trend for quality assessment to use measures of health outcome. Still, the experience with quality assurance using outcome is limited. Evaluating process of care is easier, but it produces less definitive information. The problem is that the relationship between standard health care procedures and health outcomes is often not known. There are two problems [6]. One is knowing whether physicians follow currently accepted medical procedures in their practices. In other words, are they using technology effectively? The second is establishing the effect on outcomes under ideal circumstances, that is, efficacy. Another important issue is that of patient satisfaction. Donabedian includes satisfaction as a separate category to be assessed. However, satisfaction is clearly related to health outcome. If the patient does not feel that the outcome is satisfactory, physician evaluation of ‘objective’ factors such as physiologic state has limited meaning. Increasingly, patient satisfaction is thought of as a dimension of health outcome. The term quality implies a degree of excellence. To measure such excellence, a standard of comparison must exist and a method of measurement must be available. The development of broad goals for quality is not a problem. Making specific standards is. Using standards to evaluate quality of care requires criteria by which to judge how a health condition or disease has been diagnosed or treated [l]. The development of such criteria, if they are to be valid, must be based on knowledge about (at least) the efficacy and safety of the health care technology concerned. Despite the relative lack of such information, a start must be made. Limited knowledge is no excuse for lack of action. The Relation between PACS and Quality of Care
It is difficult to relate PACS to quality of care, as defined by health outcome. The effect of diagnosis on health is often difficult to demonstrate. Diagnosis must lead to effective therapy, and such therapy is often not available. In general, therefore, diagnostic technologies have been evaluated technically; occasionally, their precision has been evaluated; effects on therapy have sometimes been evaluated; they have seldom been evaluated for their effects on health. Since the main hoped-for effect from PACS is more efficient diagnosis, PACS will suffer from the same problem. Nonetheless, I think that standards of quality should be chosen because they are likely to be associated with improved outcomes. Crowe [7] has presented a rather comprehensive list of benefits to the patient. Several areas seem most interesting.
Quality assurance
251
X-ray Dose A major orientation in technological developments in the field of diagnostic imaging has been to reduce the dose of ionizing radiation delivered to people, since this radiation is associated with risks [8]. These efforts have been historically successful, through such innovations as control of the spread of the X-ray beam, development of image intensifiers, and ability to reconstruct and manipulate an image through computers. Also, the rapid growth of ultrasound imaging and magnetic resonance imaging is in part because they are probably safer than imaging based on X-rays or other ionizing radiation. One reason for excessive radiation dose to patients is repeat films because of poor quality X-ray images. Image quality can also be enhanced by image processing, which is not possible with film. Some evidence is available that computer assisted medical imaging can reduce radiation dose to the patient [9-l 11. Improved and More Effkient Diagnosis
Diagnostic imaging leads to better diagnosis. For the physician, the first concern with PACS will be that the diagnostic accuracy is at least as high as with films. The technical accuracy of digital images has been extensively studied [ 121. Another issue is that the diagnostic process should be just as efficient (if not more so) as it was under conventional methods. Since conventional radiology is now highly automated in many ways, this may be difficult to demonstrate. At present, images are available to the physician in three ways: (1) in emergencies, the images are sent immediately, and a report follows; (2) in ordinary situations, images and report are sent later, perhaps the next day; (3) in certain situations, and at the request of the clinical physician, consultations are held between referring physicians and radiologists to discuss diagnosis and sometimes therapy [8]. One possibly important effect of PACS is that clinical physicians must now often visit the radiology department to examine the images and discuss them with the radiologist. With PACS, it will be possible to transmit the image to any site with the appropriate equipment and discuss it by telephone, with significant time saving for the clinical physician. However, the possibly negative implications of less face-to-face consultation between physicians need to be considered as well. In addition, the ready access of other patient data through the Hospital Information System (HIS) may more than counter-balance such implications. Improved Access to Images
Sometimes it is critically important to have rapid access to images, as in emergency situations. A patient might have an abnormality on an image, for example, and only comparison with an older image could indicate whether it was new or not. It can sometimes be difficult to gain access to X-ray films from the archives. Sometimes it takes hours or longer to find or access an image in the present system, because of the limited number of copies, multiple archives, overloaded archive personnel, lost or wrongly placed images, or loaned images. Computer assisted imaging might improve access [13]. In particular, images can no longer be lost with such systems
252
H.D. Banta
(although discs can possibly be lost). Furthermore, multiple copies can be easily made, increasing availability. Some medical imaging services are only available in central locations, while others are quite diffused. CT scanners and MRI devices are generally quite centralized, in academic hospitals. Nuclear medicine is more widely diffused, but is only available through departments of radiology, nuclear medicine, or medical imaging. Conventional X-ray tends to be centralized into such departments, but can also be available at the primary care level and in remote sites. Ultrasound is widely distributed throughout the system. PACS could be particularly important when the patient is in a remote site. It could even be the case that no one with expertise in interpreting images was available. The image could be transmitted to a central site for interpretation. There is some data on the success of such a development for the physician [14]. Although such interpretation might be life-saving for the patient in some instances, no data are available to indicate how often such a development would be of benefit. Improving access to images might lead to reduced length of stay in the hospital. De Simone et al. [15] found that films were viewed earlier when PACS was in use; the authors assumed a shortened length of stay but did not document it. In the evaluation of the Utrecht prototype system, it was concluded that a reduction ‘might perhaps be achieved’ [16]. In addition to economic benefits, this has quality implications. For example, hospital-acquired infections might be reduced. Patient Satisfaction The present diagnostic process is not very efficient with the time of the patient. Patients must often return for additional examinations on a different day. Extra examinations are also taken when films are of poor quality or if they are lost. In addition, diagnostic services tend to be centralized, so the patient is sometimes required to travel relatively long distances. Presumably, a more efficient diagnostic process would be associated with a higher degree of patient satisfaction. If services were less centralized, patients would probably find this desirable. Shorter length of stays would probably be desirable to most patients. Inevitably, concerns about secrecy will appear [17]. Patient satisfaction with a PACS system would surely be less if secrecy were not fully protected. Conclusions The effects of PACS on quality are unknown, and have hardly been evaluated [18]. In this paper, I have speculated concerning some of the most important characteristics of PACS. These speculations need to be confirmed by evaluation as PACS becomes diffused. References I
World Health Organization. Regional Office for Europe: Quality assurance of health services. In: Thirty-eighth Session, Technical Discussions, Copenhagen, 12-11 September, 1988. (Document no. EUR/RC38TTech.Disc.A)
Quality assurance
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17
253
Offtce of Technology Assessment: The quality of medical care, information for consumers, US Government Printing O&X, Washington, DC, 1988. Donabedian A: Evaluating the quality in medical care, Milbanh Mem Fund Q, 44 (2) (1966) 166-206. Donabedian A: Explorations in Quality Assessment andMonitoring, Vol. 1, The Definition of Quality and Approaches to its Assessment, Health Administration Press, Ann Arbor, Michigan, 1980. Offtce of Technology Assessment: Assessing the Eeffkacy and Safety of Medical Technologies, US Government Printing Office, Washington, DC, 1978. Brook RH: Quality: can we measure it? New Engl J Med 296 (1977) 170-172. Crowe B: Overview of some Methodological Problems in Assessment of PACS, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. Economic Commission for Europe, United Nations. In: Digital imaging in Health Care, Working Party on Engineering Industries and Automation: New York, 1987 (ISBN 92-l-l 16380-3). Bailey R: Patient exposure requirements for high resolution in digital radiographic systems, Am J Radio1 142 (1984) 608-611. Fraser R, Sanders C and Barnes G: Digital imaging of the chest, Radiology 171 (1989) 769-773. McGrohan L, Patterson JF, Gagne RM and Goldstein HA: Average radiation doses in a standard head examination for 250 CT systems, Radiology 163 (1987) 263-268. Bameveld Binkhuysen F: Requiredfunctionality of PACS from a clinicalpoint of view, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. Banta HD and van Beekum WT (Eds): Computer Assisted Medical Imaging, The Case of Picture Archiving and Communications Systems (PACS), Kluwer Academic Publishers, Dordrecht, 1988. Horii S, Mun S, Elliott L et al: PACS clinical experience at Georgetown University, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. De Simone D, Kundel H, Arenson R, Seshadn S et al: Effect of a digital-imaging network on physician behavior in an intensive care unit, Rudiofogy 169 (1988) 41-44. Andriessen J, ter Haar Romeny B, Bamevald Binkhuysen F and van der Horst-Bruinsma I: Savings and costs of a picture archiving and communication system in the University Hospital Utrecht, Proc Med Im III Conf SPZE, 1093 (1989) 578-584. Irie G and Miyasaka K: Feasibility of PACS based on 2 years experiences of HV-PACS,
Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991. 18 Flagle C: The inflwnce of PACS on quality of care at the clinical level, Presented at the International Workshop on Technology Assessment of PACS, Enkhuizen, May 26-27, 1991.