- Email: [email protected]
The assessment of diagnostic imaging technologies: a policy perspective
Health Policy 36 (1996) 185-197
The assessment of diagnostic imaging technologies: a policy perspective David Centre for
the Study
of Clinical
Hailey*,
Practice, Fitzroy,
Ian McDonald
St. Vincent5 Hospital, Victoria 3065, Australia
Melbourne,
41 Victoria
Parade,
Received 5 December 1995;accepted 19 January 1996
Abstract
Diagnosticimagingtechnologiesare essentialin health care but have high costsand poorly definedbenefits.Formulation and implementationof policy on their procurementand useis made difficult by the complexity of the diagnosticprocess,and the limitations of available data and assessment methodology. Informed policy decisionswill need to be basedon a synthesisof imperfect data from a variety of perspectives,and supplementedby effective disseminationand feedback of information. A list of attributes for consideration in the policy formulation processis presented. Keywords: Diagnostic imaging; Policy; Patient management;Economics;Assessment
1. Introduction Diagnostic imaging includes a range of technologies which have brought about profound change in the practice of medicine and are indispensable to modern health care. However, such technologies continue to give concerns to policy makers and program managers because of their high costs to health care budgets and a degree of uncertainty as to their effects on health status in relation to levels of utilization.
Policy areas, administrators and funders of health services have for
many years been faced with the reality of rising costs of services and increasing numbers of examinations. There is relatively little objective information on what some of these technologies are achieving in terms of their effects on patient management. * Corresponding author.Tel.: + 61 6 2316539; fax: + 61 6 2311090. 0168~8510/96/$15.00 0 1996ElsevierScienceIrelandLtd. All rightsreserved PZZ SO168-8510(95)00811-1
186
D. Hailey,
I. McDonald/Health
Policy
36 (1996)
185-197
In this paper we consider a number of aspects of diagnostic imaging methods that are relevant to their use and place in health care, and outline various considerations which might usefully be taken into account by those who have responsibility for helping shape decisions on the procurement and use of these technologies.
2. The nature and role of diagnostic imaging
In diagnostic imaging, the health care professional, viewing the displayed image, makes a decision based on knowledge of anatomy or properties of the tissue under examination and the characteristics of the particular imaging technology. Complex data are obtained and analysed in an interaction of the machine with the specialist’s interpretive skills. There is then a further interaction with the physician linking the information in the diagnostic imaging report with other clinical evidence, and previous training and experience, in coming to a decision on management of the patient. The process is inherently complex. The degree of complexity will vary (Table 1). In a number of situations, the imaging method has a dedicated, quite closely defined role, as an integral part of a procedure. However, more often diagnostic imaging technologies have multiple uses, covering many conditions and clinical situations. Each technology will have various roles, and may be used to confirm or exclude disease, to monitor a condition or to plan management. Imaging technologies may compete with each other, be complementary examinations, or perhaps be additive to previous tests without providing significant new data. It is this range and versatility of diagnostic imaging technologies which leads to some of the difficulties in their appraisal, management and use.
3. The diffusion of imaging methods and assessment
The evolution of some diagnostic imaging methods has in part been responsible for the development of health technology assessment initiatives. Banta and Lute [l] note that after its introduction in 1972, CT scanning rapidly became the most Table Roles
1 of diagnostic
imaging
technologies Examples
Role Specifically
procedure-related
Disease-specific, including screening General diagnostic method in a specific area General method with multiple applications
Imaging during coronary angioplasty, laparoscopic surgery Mammography Ultrasound in obstetrics CT, MRI in hospitals and radiology
clinics
D. Hailey.
I. McDonald
1 Health
Policy
36 (1996)
185-197
187
visible technological advance in health care, and led to significant assessment efforts in the USA and in Sweden. In Australia, the experience with the essentially uncontrolled introduction of CT was a factor leading to recommendations for the establishment of national health technology assessment initiatives [2]. While imaging technologies were a spur to the introduction of assessment programs, their appraisal remains comparatively limited. The literature on diagnostic imaging is vast, but few studies have considered the use, effects and costs of these modalities in routine use in any detail. The Australian experience with the level of use of imaging services resembles that of other countries in that there have been high rates of growth, of around 15%, for CT and ultrasound examinations over the last decade, with conventional radiology also continuing to expand, but to a lesser extent. Diffusion of the technology that triggered earlier concerns, CT scanning, continues with there now being over 370 scanners in Australia (20.5 per million population). Much more limited use has been made of MRI and PET, technologies with high capital costs when introduced, with support dependent very much on decisions by government agencies. Overall, about 25% of expenditure on technological services funded through the Medicare insurance scheme is related to imaging examinations [3]. The growth in use of imaging technologies in part reflects a long-term trend in medicine to wider use of more informative diagnostic methods, and to their integral involvement in new methods of treatment. However, as with other medical technologies, many factors affect their diffusion. Greer [4] has drawn attention to the importance of opinion leaders, such as radiologists, in influencing procurement decisions, and to the distrust of the scientific literature by many physicians. The dramatic images that can be produced provide powerful support for arguments in favour of increasing capacity, but too often are not matched by evidence of benefit in terms of effects on patient management. Other considerations will include profit, a wish to offer optimum care and fear of litigation. Diffusion of these technologies has been driven by a variety of factors. 4. Assessment
of diagnostic imaging methods
Health technology assessment has been able to develop successful approaches to the provision of objective advice on therapeutic technologies. It has proved more difficult to describe the efficacy, effectiveness and efficiency of diagnostic imaging methods. Assessment has contributed in a more limited way to the debate on their procurement and appropriate use. Fineberg et al. [5] formulated a well known conceptual framework for assessment of diagnostic imaging. Evaluation of efficacy was considered at five levels covering technical evaluation, diagnostic accuracy, diagnostic impact, therapeutic impact and health impact. Relatively few studies have progressed beyond the second of these. For example, a review of studies on the use of MRI in neuroradiology showed that there was abundant evidence of its technical capacity, a little evidence of reliable diagnostic accuracy and impact, and virtually no evidence of therapeutic or patient outcome impacts [6].
188
D. Hailey,
I. McDonald
1 Health
Policy
36 (1996)
185-197
In most situations, a diagnostic imaging test will not be used in isolation, but will be part of a range of evidence considered by the doctor in reaching a decision on management of the patient. This decision making process is complex, and the way that an imaging test might fit within it may often be difficult to describe with any precision. Assessors of diagnostic imaging methods are faced with the problem of defining a suitable outcome measure. Only quite rarely will the imaging test contribute in a measurable way to change in health status - its role is subject to uncertainty and there will be many other factors contributing to the patient’s condition. The final stage in the hierarchy described by Fineberg et al. may in practice be difficult to attain. For the most part, assessment of imaging tests will need to focus more on the question of the extent to which use of a particular technology affects the management decision. Investigation at this level is also complex and there is a danger that the approaches taken in a particular evaluation may not closely correspond to the way in which a test is used in routine practice. There is often a lack of good quality, verifiable data to substantiate the place of an imaging technology. There seems a tendency to often regard the imaging test as a ‘free good’ and a reluctance to consider the issue of whether use of this resource is really good value for money, given its likely influence on the eventual management of the patient. Further, there is a need to appreciate that use of an imaging test may inappropriately influence the management decision. Much of the existing literature refers to studies of limited power, typically involving small series and uncontrolled, unblinded designs. A challenge for the assessment process is to define practical methods of evaluation which are capable of reasonable description of the technology under consideration while not placing undue constraints and demands on those who are using the method. There is a considerable backlog of unmet evaluation need. Various diagnostic imaging methods have been introduced over many years and are widely established. In many cases their use has become integrated into routine health care but there is not necessarily good knowledge of the performance, strengths and weaknesses of each method or much formal appraisal. A recent overview of imaging services has pointed out that while common sense and expert opinion suggest that some X-ray examinations improve patient outcome, the benefit of much current practice is unproven. Further, in most cases evidence that X-ray examinations improve management of patients has been obtained only from non-experimental studies [7]. Appropriate patterns of practice are not necessarily assured. There are clearly problems for policy areas in obtaining clear advice on the performance of diagnostic imaging tests. However, it is still possible to describe steps in the appraisal process and to consider strategies for obtaining information on the performance of diagnostic imaging technologies. There will be a need to make an early choice of promising potential applications. With the introduction of a diagnostic imaging method there will commonly be a ‘first impressions’ phase. From a clinician’s perspective there will be indications that the method may be of value and have some comparative advantage relative to other technologies. Even in this early phase there will be a need for data on basic
D. Hailey,
I. McDonald
/ Health
Policy
36 (1996)
189
185-197 -4
Clinical
\ Clinical requirement
-
l l l
Technical performance
I operator training upgrades maintenance
-
Policy and program decisions
Economic factors
/ Patient, societal pYSpdiVl?S
Monitor, follow up I Fig. 1. Dimensions of a diagnostic imaging examination.
attributes related to cost of operation, time of examination and types of examination undertaken. Different types of assessment and reporting approaches may need to be formulated at this stage. A review of the assessment of magnetic resonance imaging noted the need to tailor assessments of a new diagnostic technology to the technical, logistical and clinical circumstances that prevail. There may be a need to begin with a careful observational study, moving later to more controlled approaches, and perhaps alternating between these levels of analysis [8]. It is very unlikely that a single assessment method will be suEicient to appropriately define the place of an imaging method. Randomised controlled trials have not often been used in the assessment of diagnostic imaging technologies. In addition to difficulties in defining an outcome which will realistically compare the method with other approaches, there may be problems in definition of selection criteria and in the generalisability of results to slightly different applications of the imaging test. At another level, there will often be pressure on both clinical staff and administrators to grant access to all patients, rather than persisting with a randomised approach, and there may be major difficulties with selection bias. These problems do not mean that the randomised trial should be ruled out in evaluation of imaging methods. However, the reality is that opportunities to
190
D. Hailey,
I. McDonald
1 Health
Policy
36 (1996)
185-197
undertake useful randomised trials may be limited and that such studies will need to be considered as complementary to other approaches. Consideration will need to be given to other methods to evaluate whether the diagnostic imaging test has a significant influence on the patient management decision. These may, for example, include ‘before/after’ studies comparing action taken by clinician with the management plan which was described prior to the diagnostic imaging test being performed. Such studies are useful in giving indications of when use of the diagnostic imaging method is appropriate and efficient, or where changes in practice may be needed. Broader modeling approaches, such as use of decision-analytic methodology, are perhaps a more realistic option after an imaging technology has matured and begun to diffuse into the health care system. They may also be helpful at a later stage, when the mature technology has to be compared with the use of newer approaches. Incremental changes to an imaging method ideally should be systematically identified by each group using the technology and appropriately reported. In practice, assessors and policy makers will be faced with forming a synthesis of imperfect information. 5. Interaction
of assessment
and policy
Assessment of costs and effectiveness of diagnostic imaging technologies to define their use and comparative benefits will inevitably be linked in policy areas to considerations of rationing, implicit or explicit, and of reimbursement or other approaches to support technology. At the level of the hospital and clinic, assessment will help to define appropriate standards of practice, in addition to informing decisions on allocation of resources. Assessment and policy on health technologies have been considered in terms of four relationships - between the health care system and assessment; between assessment and change in policy; between policy and changes in practice; and between changes in practice, monitoring and the need for reassessment [93. The first of these is concerned with the complex process of selecting topics for assessment. It is noteworthy that, while diagnostic imaging technologies meet suggested criteria for deciding whether an assessment should be undertaken, relatively few resources have yet been committed by funding authorities to their evaluation. Reasons may include the perceived complexity of the technologies, pressure from interest groups and persistence of older administrative practices in budget formulation. Similarly, with the second relationship, the influence of available assessments on policy has not always been very significant - other inputs may be more dominant. When assessment has successfully influenced policy, the ability of policy areas to then shape the way diagnostic imaging is used in practice has often been constrained. As with other types of technologies, the instruments available to policy makers in determining the place of diagnostic imaging methods and the degree to which they should be supported within a health care system tend to be limited. In
D. Hailey,
I. McDonald
/ Health
Policy
36 (1996)
185-197
191
the Australian context, available policy instruments are for the most part linked to financial incentives. These include reimbursement through the Medicare health insurance program and grants provided by governments. In the longer term there will be scope for sponsoring educational initiatives and best practice guidelines, though such measures are demanding and their effectiveness in influencing practice is not well established. The monitoring and review process implied in the final relationship has also been limited by available policy instruments and data, an absence of longer term assessment efforts and uncertainty by policy areas on what they are prepared to do with these. Links between the assessment and policy formulation processes remain limited. To an extent, governments have not followed up the creation of health technology assessment agencies with appropriate levels of resourcing to adequately investigate diagnostic imaging technologies. Diagnostic imaging techniques will inevitably be refined, sometimes very rapidly, with consequential change to clinical practice. There is a need to consider the economic implications of such technical progress. Decisions will need to be made at many levels on procurement and use of new technologies or replacement of older ones. However, in the area of diagnostic imaging, policy formulation and review has often not been a continuous, dynamic process. These processes are too often intermittent and unresponsive to change in the technologies and their applications. The initial policy decisions on an imaging technology may have to be taken on the basis of limited data. Even with new technologies, where there is an opportunity to influence introduction, for example through use of conditional funding linked to a trial, rather limited information may emerge. Data that do become available in such circumstances will tend to relate to the efficacy of the diagnostic method rather than its performance under average conditions of use. There is need to update and re-appraise the state of knowledge of the various techniques and applications, firstly as the new technology becomes more widely diffused so that its effectiveness rather than efficacy requires definition. At a later stage, further assessment is needed when new generations of the equipment have emerged (possibly including new competing technologies), bringing the possibility of different levels of comparative advantage, scope of application and cost consequences. Further data should feed into the policy process, not only in terms of funding decisions but also in relation to optimum patterns of practice. In general, rather little evaluation information tends to emerge at a later stage in the life of the diagnostic imaging technology. It often proves difficult to obtain funding for follow up trials and collection of good quality routine data is demanding and rarely resourced adequately. Eventually there will be a need to make informed judgments on the phasing out of obsolete diagnostic imaging methods and practices which are no longer considered optimal. This is an excellent principle, but can be difficult to achieve. There is strong practice conservatism and often older methods and practices exist in parallel with newer techniques for many years. In the case of diagnostic imaging technologies this undesirable situation may be accentuated by limited involvement of diagnosticians in the clinical process.
Estimate based on costs and performance (above), applications and patient numbers (below)
Cost per exam
Numbers of machines required
Increment to other tests
Measurement against a ‘gold standard’ with appropriate study design
Accuracy of study
Preferably, specific studies to indicate place and contribution of each test in management of patient
Monitor overall use of different diagnostic tests
‘Accuracy’ in routine use
Confirm/Exclude/Monitor/ Prevalence, severity
Nature of exam Disease/ condition
Plan/management
Derived from previous items
Number and time of exams
Items related to specik diseases/conditions
Staff-related; power, other machine consumables; film, other reporting/storage Measured patient throughput
Recurrent cost
to the imaging
Purchase/lease expenditure
generally
Measure(s)
Capita1 cost
Items related method
Attribute
Table 2 Attributes of diagnostic imaging technologies
Potential effect of test on patient management (see below) Availability of adequate ‘gold standard’; power of study, whether full description of procedure and analysis Measures used to obtain such indications (as above); relevance of such measures to actual diagnostic process Complex; algorithms useful, effect on costs, effects on certainty of result
Length of operating day; effect of updates on speed; overhead for clerical, etc.; complexity of exam Consider in context of benefits achieved, also cost consequences of incorrect result or exam not undertaken Difficult to estimate need for follow up exams, expanding indications, place of method in accepted approach for each disease
Negotiation with supplier; replacement timing; method of depreciation Level of staffing for effective operation
Question(s)
Number of examinations per case, having regard to time scale Change to diagnostic decision - determine e.g. from before/after study Effect on length of hospital stay, other health service indicators Effect on use of other tests, services Consequences of missed/incorrect result: l unnecessary procedures, unnecessary additional tests 0 additional morbidity, anxiety Assess impact on survival, morbidity data Consider effect on patient quality of life, e.g. reassurance, less invasive procedures Synthesis of points of comparison - performance, cost, effectiveness
Repeat tests
Relationship to other tests
Effect on outcomes
Adverse consequences
Effect on management
Measure(s)
Attribute
Table 2 Attributes of diagnostic imaging technologies
Explicit consideration of chance
Effect on clinical outcomes may be difficult to measure - often multifactorial, imaging method may have minor (though worthwhile) role
Important considerations when appraising the effectiveness of a diagnostic method
Complex, multifactorial; of influencing decision
Training required; reliability of equipment
Question(s)
194
6. Attributes
D. Hailey,
I. McDonald/Health
for consideration
Policy
36 (1996)
185-197
by policy makers
There is very often a need to base decisions on a synthesis of information concerning the technology in question, weighting the different attributes with regard to their relevance to administrative, equity and budgetary considerations. Fig. 1 illustrates some domains associated with diagnostic imaging that should be taken into account when considering policy and program decisions. The place of the technology will be informed and determined by clinical need, its technical performance, its performance in the clinical setting, economic considerations and the perspectives of the patient and of society generally. A synthesis is needed, as consideration of all such aspects is almost never undertaken by those who order or perform diagnostic imaging tests. Desirably, as with other types of technology, use and outcomes of diagnostic imaging methods will be monitored and followed up with feedback to an ongoing process of appraisal and policy formulation. Some factors relating to the different domains include: 6.1. Clinical need
The specification for the diagnostic test required here is the significance and prevalence of the disease gated. It will be hard to argue for increased support imaging method if the prevalence of the condition is to health status are limited. Other points are the limitations of other diagnostic approaches.
by the clinician. An issue or condition to be investifor a high cost diagnostic low, and the consequences availability, strengths and
6.2. Technical performance
A basic description of the technology will be important at the operator/institutional level and interactive with input to quality assurance programs. Aspects may include specification of image characteristics; ‘production’ costs and output, such as patient throughput and machine operating costs; possibly basic levels of ‘accuracy’ in comparison with other approaches; and consideration of safety issues such as radiation dose. Training considerations may be significant. Such factors will be necessary but limited components in the overall appraisal of a method. 6.3. Clinical performance
An important consideration is the place of the test in the algorithm of patient management and its relationship to other examinations. With some imaging methods there may be a large gap between efficacy and effectiveness. Outcome measures will relate to ‘diagnostic yield’ and ‘accuracy’, in the context of the prevalence of disease/condition and the effect on patient management. There will
D. Hailey.
I. McDonald
/ Health
Policy
36 (1996)
185-197
195
be further training considerations and a range of levels of difficulty in interpretation of findings, including possible problems with irrelevant pathology detected during the examination. Clinical (and technical) performance will change over time with upgrades to the technology, and indications for use may also change. 6.4. Economic factors
Economic measurement of the effect of diagnostic imaging technologies is difficult and controversial. Requirements here are measures of incremental gain over information from other tests, linked to costs and benefits. Benefits may include gains in quality of life for the patient, and consideration will be needed as to the validity of measurements of utility that are undertaken. Economic appraisal should include consideration of the consequences of not undertaking the imaging test, and those that may arise from an incorrect result. In considering costs, policy areas will often have a first concern of the implications for program budgets. There is a question of how the cost burden should be distributed, what are appropriate measures and to what extent consequential costs, including expenditure avoided through use of imaging, should be built into specific policies. Available policy approaches tend to be limited to reflections of macro measures of expenditure, such as the cost of production and interpretation of images, rather than specific consideration of potential effects (both economic and clinical) on patient management decisions and eventual outcomes. It is difficult to capture such subtleties with broad policy instruments. However, in seeking to evaluate the impact of a diagnostic imaging test it is desirable to make the separation between costing for the purpose of setting policy on reimbursement and wider economic appraisal to inform a more general position on the place and value of a particular technology in health care. Considerations of cost - including capital and site costs, recurrent items and consequential expenditure as a result of use of the diagnostic imaging technology - are complex issues and tend to be moving targets in their own right. 6.5. Patient and societal perspectives
The opinions of the patient usually receive too little attention. They form an important outcome measure and inform the economic analysis of diagnostic imaging methods. Important factors for the patient may include less painful procedures, less inconvenience and perhaps benefits through provision of additional information - though the value of further information will be highly dependent on how this is made available to the individual. The results from a study of echocardiography showed that patients were not reassured by demonstration of the technology of a normal heart [lo]. Table 2 summarises some of the important attributes of diagnostic imaging methods, approaches to measure these and questions to bear in mind throughout
196
D. Hailey,
I. McDonald/Health
Policy
36 (1996)
185-191
this process. The list is formidable and many of the issues are complex, but such information may be helpful as a guide when considering the place of these technologies. If the full range of attributes is not considered, the nature of diagnostic imaging technologies and the issues associated with their appropriate deployment and use will not be adequately appreciated. While definitive studies will not exist for most aspects of a technology, policy makers and administrators have a need to take decisions. They must therefore make judgments on the basis of available data, recognising that the various inputs need to be looked at critically and kept under review. Discussion and consideration of the attributes listed in Table 2, applied to various applications of a diagnostic imaging method, will help in obtaining descriptions of how and why they are used. The synthesis process will help in giving an overall picture for the policy area. There is then a question of how to take things further and influence use of the technology. The various elements listed in Table 2 will be useful components of assessment reports, but there is a need to go beyond that to a more systematic process of policy formulation, implementation and follow up. As discussed previously, available policy instruments may be limited in scope, and policy involvement in a specific technology can be intermittent. Such constraints point to the need for appropriate dissemination of assessment data and related recommendations and decisions, and for a process of dialogue with users and suppliers of diagnostic imaging technologies. Appraisal, procurement and use of these technologies will need to be part of an iterative process, with interaction between their funders and users, if the complex issues discussed here are to be adequately dealt with by health care systems.
References [I] Banta, D., and Lute, B., Health Care Technology and its Assessment, Oxford University Press, London, 1993, p. 159. [2] Sax, S. (Chairman), Report of the Committee on Applications and Costs of Modern Technology in Medical Practice, Australian Government Publishing Service, Canberra, 1978. [3] Australian Institute of Health and Welfare, HealthTechStats No. 4, Australian Institute of Health and Welfare, Canberra, 1994. [4] Greer, A.L., The state of the art versus the state of the science: the diffusion of new medical technologies into practice, International Journal of Technology Assessment in Health Care, 4 (1988) 5-26. [5] Drummond, M., Hailey, D. and Selby Smith, C., Maximising the impact of health technology assessment. In C. Selby Smith (Ed.), Economics and Health 1991, Monash University, Melbourne, 1992, pp. 234-271. [6] Fineberg, H.V., Baumann, R. and Sosman, M., Computerized cranial tomography: effect on diagnostic and therapeutic plans. Journal of the American Medical Association, 238 (1977) 2244221. [7] Kent, D.L. and Larson, E.B., Three dimensions of clinical efficacy assessment applied to magnetic resonance imaging, Investigative Radiology, 27 (1992) 245-254.
D. Hailey,
I. McDonald
/ Health
Policy
36
(1996) 185-197
197
[g] Hailey, D.M., Crowe, B.L. and McDonald, LG., Assessing the Place of MRI in Australia, Australian Institute of Health, Canberra, 1991. [9] Welsh Health Planning Forum, Getting the Best For Patients. Effectiveness in Medical Imaging, Cardiff, 1994. lo] Daly, J. and McDonald, I., The social impact of echocardiography. Australian Institute of Health and Welfare, Canberra, May 1993.
The assessment of diagnostic imaging technologies: a policy perspective David Centre for
the Study
of Clinical
Hailey*,
Practice, Fitzroy,
Ian McDonald
St. Vincent5 Hospital, Victoria 3065, Australia
Melbourne,
41 Victoria
Parade,
Received 5 December 1995;accepted 19 January 1996
Abstract
Diagnosticimagingtechnologiesare essentialin health care but have high costsand poorly definedbenefits.Formulation and implementationof policy on their procurementand useis made difficult by the complexity of the diagnosticprocess,and the limitations of available data and assessment methodology. Informed policy decisionswill need to be basedon a synthesisof imperfect data from a variety of perspectives,and supplementedby effective disseminationand feedback of information. A list of attributes for consideration in the policy formulation processis presented. Keywords: Diagnostic imaging; Policy; Patient management;Economics;Assessment
1. Introduction Diagnostic imaging includes a range of technologies which have brought about profound change in the practice of medicine and are indispensable to modern health care. However, such technologies continue to give concerns to policy makers and program managers because of their high costs to health care budgets and a degree of uncertainty as to their effects on health status in relation to levels of utilization.
Policy areas, administrators and funders of health services have for
many years been faced with the reality of rising costs of services and increasing numbers of examinations. There is relatively little objective information on what some of these technologies are achieving in terms of their effects on patient management. * Corresponding author.Tel.: + 61 6 2316539; fax: + 61 6 2311090. 0168~8510/96/$15.00 0 1996ElsevierScienceIrelandLtd. All rightsreserved PZZ SO168-8510(95)00811-1
186
D. Hailey,
I. McDonald/Health
Policy
36 (1996)
185-197
In this paper we consider a number of aspects of diagnostic imaging methods that are relevant to their use and place in health care, and outline various considerations which might usefully be taken into account by those who have responsibility for helping shape decisions on the procurement and use of these technologies.
2. The nature and role of diagnostic imaging
In diagnostic imaging, the health care professional, viewing the displayed image, makes a decision based on knowledge of anatomy or properties of the tissue under examination and the characteristics of the particular imaging technology. Complex data are obtained and analysed in an interaction of the machine with the specialist’s interpretive skills. There is then a further interaction with the physician linking the information in the diagnostic imaging report with other clinical evidence, and previous training and experience, in coming to a decision on management of the patient. The process is inherently complex. The degree of complexity will vary (Table 1). In a number of situations, the imaging method has a dedicated, quite closely defined role, as an integral part of a procedure. However, more often diagnostic imaging technologies have multiple uses, covering many conditions and clinical situations. Each technology will have various roles, and may be used to confirm or exclude disease, to monitor a condition or to plan management. Imaging technologies may compete with each other, be complementary examinations, or perhaps be additive to previous tests without providing significant new data. It is this range and versatility of diagnostic imaging technologies which leads to some of the difficulties in their appraisal, management and use.
3. The diffusion of imaging methods and assessment
The evolution of some diagnostic imaging methods has in part been responsible for the development of health technology assessment initiatives. Banta and Lute [l] note that after its introduction in 1972, CT scanning rapidly became the most Table Roles
1 of diagnostic
imaging
technologies Examples
Role Specifically
procedure-related
Disease-specific, including screening General diagnostic method in a specific area General method with multiple applications
Imaging during coronary angioplasty, laparoscopic surgery Mammography Ultrasound in obstetrics CT, MRI in hospitals and radiology
clinics
D. Hailey.
I. McDonald
1 Health
Policy
36 (1996)
185-197
187
visible technological advance in health care, and led to significant assessment efforts in the USA and in Sweden. In Australia, the experience with the essentially uncontrolled introduction of CT was a factor leading to recommendations for the establishment of national health technology assessment initiatives [2]. While imaging technologies were a spur to the introduction of assessment programs, their appraisal remains comparatively limited. The literature on diagnostic imaging is vast, but few studies have considered the use, effects and costs of these modalities in routine use in any detail. The Australian experience with the level of use of imaging services resembles that of other countries in that there have been high rates of growth, of around 15%, for CT and ultrasound examinations over the last decade, with conventional radiology also continuing to expand, but to a lesser extent. Diffusion of the technology that triggered earlier concerns, CT scanning, continues with there now being over 370 scanners in Australia (20.5 per million population). Much more limited use has been made of MRI and PET, technologies with high capital costs when introduced, with support dependent very much on decisions by government agencies. Overall, about 25% of expenditure on technological services funded through the Medicare insurance scheme is related to imaging examinations [3]. The growth in use of imaging technologies in part reflects a long-term trend in medicine to wider use of more informative diagnostic methods, and to their integral involvement in new methods of treatment. However, as with other medical technologies, many factors affect their diffusion. Greer [4] has drawn attention to the importance of opinion leaders, such as radiologists, in influencing procurement decisions, and to the distrust of the scientific literature by many physicians. The dramatic images that can be produced provide powerful support for arguments in favour of increasing capacity, but too often are not matched by evidence of benefit in terms of effects on patient management. Other considerations will include profit, a wish to offer optimum care and fear of litigation. Diffusion of these technologies has been driven by a variety of factors. 4. Assessment
of diagnostic imaging methods
Health technology assessment has been able to develop successful approaches to the provision of objective advice on therapeutic technologies. It has proved more difficult to describe the efficacy, effectiveness and efficiency of diagnostic imaging methods. Assessment has contributed in a more limited way to the debate on their procurement and appropriate use. Fineberg et al. [5] formulated a well known conceptual framework for assessment of diagnostic imaging. Evaluation of efficacy was considered at five levels covering technical evaluation, diagnostic accuracy, diagnostic impact, therapeutic impact and health impact. Relatively few studies have progressed beyond the second of these. For example, a review of studies on the use of MRI in neuroradiology showed that there was abundant evidence of its technical capacity, a little evidence of reliable diagnostic accuracy and impact, and virtually no evidence of therapeutic or patient outcome impacts [6].
188
D. Hailey,
I. McDonald
1 Health
Policy
36 (1996)
185-197
In most situations, a diagnostic imaging test will not be used in isolation, but will be part of a range of evidence considered by the doctor in reaching a decision on management of the patient. This decision making process is complex, and the way that an imaging test might fit within it may often be difficult to describe with any precision. Assessors of diagnostic imaging methods are faced with the problem of defining a suitable outcome measure. Only quite rarely will the imaging test contribute in a measurable way to change in health status - its role is subject to uncertainty and there will be many other factors contributing to the patient’s condition. The final stage in the hierarchy described by Fineberg et al. may in practice be difficult to attain. For the most part, assessment of imaging tests will need to focus more on the question of the extent to which use of a particular technology affects the management decision. Investigation at this level is also complex and there is a danger that the approaches taken in a particular evaluation may not closely correspond to the way in which a test is used in routine practice. There is often a lack of good quality, verifiable data to substantiate the place of an imaging technology. There seems a tendency to often regard the imaging test as a ‘free good’ and a reluctance to consider the issue of whether use of this resource is really good value for money, given its likely influence on the eventual management of the patient. Further, there is a need to appreciate that use of an imaging test may inappropriately influence the management decision. Much of the existing literature refers to studies of limited power, typically involving small series and uncontrolled, unblinded designs. A challenge for the assessment process is to define practical methods of evaluation which are capable of reasonable description of the technology under consideration while not placing undue constraints and demands on those who are using the method. There is a considerable backlog of unmet evaluation need. Various diagnostic imaging methods have been introduced over many years and are widely established. In many cases their use has become integrated into routine health care but there is not necessarily good knowledge of the performance, strengths and weaknesses of each method or much formal appraisal. A recent overview of imaging services has pointed out that while common sense and expert opinion suggest that some X-ray examinations improve patient outcome, the benefit of much current practice is unproven. Further, in most cases evidence that X-ray examinations improve management of patients has been obtained only from non-experimental studies [7]. Appropriate patterns of practice are not necessarily assured. There are clearly problems for policy areas in obtaining clear advice on the performance of diagnostic imaging tests. However, it is still possible to describe steps in the appraisal process and to consider strategies for obtaining information on the performance of diagnostic imaging technologies. There will be a need to make an early choice of promising potential applications. With the introduction of a diagnostic imaging method there will commonly be a ‘first impressions’ phase. From a clinician’s perspective there will be indications that the method may be of value and have some comparative advantage relative to other technologies. Even in this early phase there will be a need for data on basic
D. Hailey,
I. McDonald
/ Health
Policy
36 (1996)
189
185-197 -4
Clinical
\ Clinical requirement
-
l l l
Technical performance
I operator training upgrades maintenance
-
Policy and program decisions
Economic factors
/ Patient, societal pYSpdiVl?S
Monitor, follow up I Fig. 1. Dimensions of a diagnostic imaging examination.
attributes related to cost of operation, time of examination and types of examination undertaken. Different types of assessment and reporting approaches may need to be formulated at this stage. A review of the assessment of magnetic resonance imaging noted the need to tailor assessments of a new diagnostic technology to the technical, logistical and clinical circumstances that prevail. There may be a need to begin with a careful observational study, moving later to more controlled approaches, and perhaps alternating between these levels of analysis [8]. It is very unlikely that a single assessment method will be suEicient to appropriately define the place of an imaging method. Randomised controlled trials have not often been used in the assessment of diagnostic imaging technologies. In addition to difficulties in defining an outcome which will realistically compare the method with other approaches, there may be problems in definition of selection criteria and in the generalisability of results to slightly different applications of the imaging test. At another level, there will often be pressure on both clinical staff and administrators to grant access to all patients, rather than persisting with a randomised approach, and there may be major difficulties with selection bias. These problems do not mean that the randomised trial should be ruled out in evaluation of imaging methods. However, the reality is that opportunities to
190
D. Hailey,
I. McDonald
1 Health
Policy
36 (1996)
185-197
undertake useful randomised trials may be limited and that such studies will need to be considered as complementary to other approaches. Consideration will need to be given to other methods to evaluate whether the diagnostic imaging test has a significant influence on the patient management decision. These may, for example, include ‘before/after’ studies comparing action taken by clinician with the management plan which was described prior to the diagnostic imaging test being performed. Such studies are useful in giving indications of when use of the diagnostic imaging method is appropriate and efficient, or where changes in practice may be needed. Broader modeling approaches, such as use of decision-analytic methodology, are perhaps a more realistic option after an imaging technology has matured and begun to diffuse into the health care system. They may also be helpful at a later stage, when the mature technology has to be compared with the use of newer approaches. Incremental changes to an imaging method ideally should be systematically identified by each group using the technology and appropriately reported. In practice, assessors and policy makers will be faced with forming a synthesis of imperfect information. 5. Interaction
of assessment
and policy
Assessment of costs and effectiveness of diagnostic imaging technologies to define their use and comparative benefits will inevitably be linked in policy areas to considerations of rationing, implicit or explicit, and of reimbursement or other approaches to support technology. At the level of the hospital and clinic, assessment will help to define appropriate standards of practice, in addition to informing decisions on allocation of resources. Assessment and policy on health technologies have been considered in terms of four relationships - between the health care system and assessment; between assessment and change in policy; between policy and changes in practice; and between changes in practice, monitoring and the need for reassessment [93. The first of these is concerned with the complex process of selecting topics for assessment. It is noteworthy that, while diagnostic imaging technologies meet suggested criteria for deciding whether an assessment should be undertaken, relatively few resources have yet been committed by funding authorities to their evaluation. Reasons may include the perceived complexity of the technologies, pressure from interest groups and persistence of older administrative practices in budget formulation. Similarly, with the second relationship, the influence of available assessments on policy has not always been very significant - other inputs may be more dominant. When assessment has successfully influenced policy, the ability of policy areas to then shape the way diagnostic imaging is used in practice has often been constrained. As with other types of technologies, the instruments available to policy makers in determining the place of diagnostic imaging methods and the degree to which they should be supported within a health care system tend to be limited. In
D. Hailey,
I. McDonald
/ Health
Policy
36 (1996)
185-197
191
the Australian context, available policy instruments are for the most part linked to financial incentives. These include reimbursement through the Medicare health insurance program and grants provided by governments. In the longer term there will be scope for sponsoring educational initiatives and best practice guidelines, though such measures are demanding and their effectiveness in influencing practice is not well established. The monitoring and review process implied in the final relationship has also been limited by available policy instruments and data, an absence of longer term assessment efforts and uncertainty by policy areas on what they are prepared to do with these. Links between the assessment and policy formulation processes remain limited. To an extent, governments have not followed up the creation of health technology assessment agencies with appropriate levels of resourcing to adequately investigate diagnostic imaging technologies. Diagnostic imaging techniques will inevitably be refined, sometimes very rapidly, with consequential change to clinical practice. There is a need to consider the economic implications of such technical progress. Decisions will need to be made at many levels on procurement and use of new technologies or replacement of older ones. However, in the area of diagnostic imaging, policy formulation and review has often not been a continuous, dynamic process. These processes are too often intermittent and unresponsive to change in the technologies and their applications. The initial policy decisions on an imaging technology may have to be taken on the basis of limited data. Even with new technologies, where there is an opportunity to influence introduction, for example through use of conditional funding linked to a trial, rather limited information may emerge. Data that do become available in such circumstances will tend to relate to the efficacy of the diagnostic method rather than its performance under average conditions of use. There is need to update and re-appraise the state of knowledge of the various techniques and applications, firstly as the new technology becomes more widely diffused so that its effectiveness rather than efficacy requires definition. At a later stage, further assessment is needed when new generations of the equipment have emerged (possibly including new competing technologies), bringing the possibility of different levels of comparative advantage, scope of application and cost consequences. Further data should feed into the policy process, not only in terms of funding decisions but also in relation to optimum patterns of practice. In general, rather little evaluation information tends to emerge at a later stage in the life of the diagnostic imaging technology. It often proves difficult to obtain funding for follow up trials and collection of good quality routine data is demanding and rarely resourced adequately. Eventually there will be a need to make informed judgments on the phasing out of obsolete diagnostic imaging methods and practices which are no longer considered optimal. This is an excellent principle, but can be difficult to achieve. There is strong practice conservatism and often older methods and practices exist in parallel with newer techniques for many years. In the case of diagnostic imaging technologies this undesirable situation may be accentuated by limited involvement of diagnosticians in the clinical process.
Estimate based on costs and performance (above), applications and patient numbers (below)
Cost per exam
Numbers of machines required
Increment to other tests
Measurement against a ‘gold standard’ with appropriate study design
Accuracy of study
Preferably, specific studies to indicate place and contribution of each test in management of patient
Monitor overall use of different diagnostic tests
‘Accuracy’ in routine use
Confirm/Exclude/Monitor/ Prevalence, severity
Nature of exam Disease/ condition
Plan/management
Derived from previous items
Number and time of exams
Items related to specik diseases/conditions
Staff-related; power, other machine consumables; film, other reporting/storage Measured patient throughput
Recurrent cost
to the imaging
Purchase/lease expenditure
generally
Measure(s)
Capita1 cost
Items related method
Attribute
Table 2 Attributes of diagnostic imaging technologies
Potential effect of test on patient management (see below) Availability of adequate ‘gold standard’; power of study, whether full description of procedure and analysis Measures used to obtain such indications (as above); relevance of such measures to actual diagnostic process Complex; algorithms useful, effect on costs, effects on certainty of result
Length of operating day; effect of updates on speed; overhead for clerical, etc.; complexity of exam Consider in context of benefits achieved, also cost consequences of incorrect result or exam not undertaken Difficult to estimate need for follow up exams, expanding indications, place of method in accepted approach for each disease
Negotiation with supplier; replacement timing; method of depreciation Level of staffing for effective operation
Question(s)
Number of examinations per case, having regard to time scale Change to diagnostic decision - determine e.g. from before/after study Effect on length of hospital stay, other health service indicators Effect on use of other tests, services Consequences of missed/incorrect result: l unnecessary procedures, unnecessary additional tests 0 additional morbidity, anxiety Assess impact on survival, morbidity data Consider effect on patient quality of life, e.g. reassurance, less invasive procedures Synthesis of points of comparison - performance, cost, effectiveness
Repeat tests
Relationship to other tests
Effect on outcomes
Adverse consequences
Effect on management
Measure(s)
Attribute
Table 2 Attributes of diagnostic imaging technologies
Explicit consideration of chance
Effect on clinical outcomes may be difficult to measure - often multifactorial, imaging method may have minor (though worthwhile) role
Important considerations when appraising the effectiveness of a diagnostic method
Complex, multifactorial; of influencing decision
Training required; reliability of equipment
Question(s)
194
6. Attributes
D. Hailey,
I. McDonald/Health
for consideration
Policy
36 (1996)
185-197
by policy makers
There is very often a need to base decisions on a synthesis of information concerning the technology in question, weighting the different attributes with regard to their relevance to administrative, equity and budgetary considerations. Fig. 1 illustrates some domains associated with diagnostic imaging that should be taken into account when considering policy and program decisions. The place of the technology will be informed and determined by clinical need, its technical performance, its performance in the clinical setting, economic considerations and the perspectives of the patient and of society generally. A synthesis is needed, as consideration of all such aspects is almost never undertaken by those who order or perform diagnostic imaging tests. Desirably, as with other types of technology, use and outcomes of diagnostic imaging methods will be monitored and followed up with feedback to an ongoing process of appraisal and policy formulation. Some factors relating to the different domains include: 6.1. Clinical need
The specification for the diagnostic test required here is the significance and prevalence of the disease gated. It will be hard to argue for increased support imaging method if the prevalence of the condition is to health status are limited. Other points are the limitations of other diagnostic approaches.
by the clinician. An issue or condition to be investifor a high cost diagnostic low, and the consequences availability, strengths and
6.2. Technical performance
A basic description of the technology will be important at the operator/institutional level and interactive with input to quality assurance programs. Aspects may include specification of image characteristics; ‘production’ costs and output, such as patient throughput and machine operating costs; possibly basic levels of ‘accuracy’ in comparison with other approaches; and consideration of safety issues such as radiation dose. Training considerations may be significant. Such factors will be necessary but limited components in the overall appraisal of a method. 6.3. Clinical performance
An important consideration is the place of the test in the algorithm of patient management and its relationship to other examinations. With some imaging methods there may be a large gap between efficacy and effectiveness. Outcome measures will relate to ‘diagnostic yield’ and ‘accuracy’, in the context of the prevalence of disease/condition and the effect on patient management. There will
D. Hailey.
I. McDonald
/ Health
Policy
36 (1996)
185-197
195
be further training considerations and a range of levels of difficulty in interpretation of findings, including possible problems with irrelevant pathology detected during the examination. Clinical (and technical) performance will change over time with upgrades to the technology, and indications for use may also change. 6.4. Economic factors
Economic measurement of the effect of diagnostic imaging technologies is difficult and controversial. Requirements here are measures of incremental gain over information from other tests, linked to costs and benefits. Benefits may include gains in quality of life for the patient, and consideration will be needed as to the validity of measurements of utility that are undertaken. Economic appraisal should include consideration of the consequences of not undertaking the imaging test, and those that may arise from an incorrect result. In considering costs, policy areas will often have a first concern of the implications for program budgets. There is a question of how the cost burden should be distributed, what are appropriate measures and to what extent consequential costs, including expenditure avoided through use of imaging, should be built into specific policies. Available policy approaches tend to be limited to reflections of macro measures of expenditure, such as the cost of production and interpretation of images, rather than specific consideration of potential effects (both economic and clinical) on patient management decisions and eventual outcomes. It is difficult to capture such subtleties with broad policy instruments. However, in seeking to evaluate the impact of a diagnostic imaging test it is desirable to make the separation between costing for the purpose of setting policy on reimbursement and wider economic appraisal to inform a more general position on the place and value of a particular technology in health care. Considerations of cost - including capital and site costs, recurrent items and consequential expenditure as a result of use of the diagnostic imaging technology - are complex issues and tend to be moving targets in their own right. 6.5. Patient and societal perspectives
The opinions of the patient usually receive too little attention. They form an important outcome measure and inform the economic analysis of diagnostic imaging methods. Important factors for the patient may include less painful procedures, less inconvenience and perhaps benefits through provision of additional information - though the value of further information will be highly dependent on how this is made available to the individual. The results from a study of echocardiography showed that patients were not reassured by demonstration of the technology of a normal heart [lo]. Table 2 summarises some of the important attributes of diagnostic imaging methods, approaches to measure these and questions to bear in mind throughout
196
D. Hailey,
I. McDonald/Health
Policy
36 (1996)
185-191
this process. The list is formidable and many of the issues are complex, but such information may be helpful as a guide when considering the place of these technologies. If the full range of attributes is not considered, the nature of diagnostic imaging technologies and the issues associated with their appropriate deployment and use will not be adequately appreciated. While definitive studies will not exist for most aspects of a technology, policy makers and administrators have a need to take decisions. They must therefore make judgments on the basis of available data, recognising that the various inputs need to be looked at critically and kept under review. Discussion and consideration of the attributes listed in Table 2, applied to various applications of a diagnostic imaging method, will help in obtaining descriptions of how and why they are used. The synthesis process will help in giving an overall picture for the policy area. There is then a question of how to take things further and influence use of the technology. The various elements listed in Table 2 will be useful components of assessment reports, but there is a need to go beyond that to a more systematic process of policy formulation, implementation and follow up. As discussed previously, available policy instruments may be limited in scope, and policy involvement in a specific technology can be intermittent. Such constraints point to the need for appropriate dissemination of assessment data and related recommendations and decisions, and for a process of dialogue with users and suppliers of diagnostic imaging technologies. Appraisal, procurement and use of these technologies will need to be part of an iterative process, with interaction between their funders and users, if the complex issues discussed here are to be adequately dealt with by health care systems.
References [I] Banta, D., and Lute, B., Health Care Technology and its Assessment, Oxford University Press, London, 1993, p. 159. [2] Sax, S. (Chairman), Report of the Committee on Applications and Costs of Modern Technology in Medical Practice, Australian Government Publishing Service, Canberra, 1978. [3] Australian Institute of Health and Welfare, HealthTechStats No. 4, Australian Institute of Health and Welfare, Canberra, 1994. [4] Greer, A.L., The state of the art versus the state of the science: the diffusion of new medical technologies into practice, International Journal of Technology Assessment in Health Care, 4 (1988) 5-26. [5] Drummond, M., Hailey, D. and Selby Smith, C., Maximising the impact of health technology assessment. In C. Selby Smith (Ed.), Economics and Health 1991, Monash University, Melbourne, 1992, pp. 234-271. [6] Fineberg, H.V., Baumann, R. and Sosman, M., Computerized cranial tomography: effect on diagnostic and therapeutic plans. Journal of the American Medical Association, 238 (1977) 2244221. [7] Kent, D.L. and Larson, E.B., Three dimensions of clinical efficacy assessment applied to magnetic resonance imaging, Investigative Radiology, 27 (1992) 245-254.
D. Hailey,
I. McDonald
/ Health
Policy
36
(1996) 185-197
197
[g] Hailey, D.M., Crowe, B.L. and McDonald, LG., Assessing the Place of MRI in Australia, Australian Institute of Health, Canberra, 1991. [9] Welsh Health Planning Forum, Getting the Best For Patients. Effectiveness in Medical Imaging, Cardiff, 1994. lo] Daly, J. and McDonald, I., The social impact of echocardiography. Australian Institute of Health and Welfare, Canberra, May 1993.