- Email: [email protected]
From health technology assessment to research on the organisation and delivery of health services: addressing the balance
Health Policy 63 (2003) 155 /165 www.elsevier.com/locate/healthpol
From health technology assessment to research on the organisation and delivery of health services: addressing the balance Naomi Fulop *, Pauline Allen, Aileen Clarke, Nick Black Health Services Research Unit, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK Received 28 October 2001; accepted 26 March 2002
Abstract This paper argues that the focus of research to improve health services has, until recently, been on health technology assessment. The authors make the case for a greater emphasis on research on how health services are managed, organised and delivered, and refer to initiatives in a number of countries which are seeking to address this balance. The way two such initiatives in England and Canada have set priorities for this type of research, involving a wide range of stakeholders is described. The authors argue that a wide range of disciplines needs to be applied to research on the organisation and delivery of health services. Important theoretical differences between and within disciplines, and their implications for research methods, are discussed. An example of an issue in the delivery of organisation of health services (how best to deliver orthopaedic care) is used to illustrate how a number of different disciplines can be applied. The challenge for researchers from these disciplines is to see how far they can work together to carry out research in this important field. The challenge for this research is that the findings are valued and used by health service professionals, managers and users. # 2002 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Health services; Organisation; Delivery; Management; Research methods; Multi-disciplinary research
1. Introduction Throughout the world, societies are striving to determine how best to organise their health systems and deliver services. There is increasing recognition that advances in medicine are only part of the answer to better health care. These medical advances, alongside rising expectations of * Corresponding author E-mail address: [email protected] (N. Fulop).
health care users, mean that demands on the way we organise services will become even greater. It is therefore essential that, in parallel to biomedical research and health technology assessment (HTA), we advance our understanding and knowledge of how best to organise and deliver health services. Without this, we run the risk of failing to realise the potential benefits that health systems can provide. In addition, the need to be more responsive to the needs and wishes of those who use health care
0168-8510/02/$ - see front matter # 2002 Elsevier Science Ireland Ltd. All rights reserved. PII: S 0 1 6 8 - 8 5 1 0 ( 0 2 ) 0 0 0 6 2 - 3
156
N. Fulop et al. / Health Policy 63 (2003) 155 /165
is increasingly being accepted. The traditional tendency for health service organisations to prioritise staff needs over users’ needs is no longer acceptable, regardless of whether the system is publicly or privately financed. All of these pressures have led governments to recognise the need to increase their support for research that can help to improve the way we organise and deliver health services. While this is not a new area of study, the previously disjointed, ad hoc activities that characterised such research are giving way to a more strategic approach, with national programmes of research in some countries. This, in turn, will require a growth in both the volume and quality of research. To achieve these, the tendency for fragmentation between the very disciplines that can make key contributions must be overcome. In this paper, first we outline the development of research on the organisation and delivery of health services. Then we discuss ways of setting priorities for this area of research, drawing on the experience of the new national programme in England. The range of disciplines required is discussed, drawing on the different epistemological approaches underlying them. Finally, the contribution of a number of disciplines to this area is illustrated, using an example of a change in the delivery of health services (telemedicine), to show how these disciplines can address different aspects of health service delivery and organisation.
2. Development of research on the organisation and delivery of health services Research on the way health services are delivered and organised is part of the broader field of health services (or systems) research which has become well-established in the UK, North America and parts of Europe in the last 20 years. Health services research aims to ‘produce reliable and valid research data on which to base appropriate, effective, cost-effective, efficient and acceptable health services’ [1]. It is not a scientific discipline in itself but an area of applied research, which draws on a range of disciplines and perspectives including anthropology, economics, epidemiology,
history, medicine, nursing, political science, sociology, and statistics. It concentrates on the study of health services or systems rather than on the state of health and disease of individuals and of populations. Unlike clinical research, which has traditionally focused on individual patients in relation to their treatment and care, health services research adopts a population perspective. Questions considered by health services research include: How much health care should we have? How should services be funded? Who should receive health services? and, How well are services being delivered? [2]. The interests of different stakeholders in health services have led to the growth, development and increased expectations of health services research. Funders of health care have sought help to address the problem of cost containment; health care professionals, particularly doctors who have come under pressure to defend themselves against those who argued that many clinical interventions were unproven, ineffective, or even harmful [3 /5], have sought scientific evidence from health services research to defend their practices; and users of health services have looked to this research to support their growing demands for more information about the care they receive. These pressures have led to the tendency for the focus of health services research to be on HTA. This has supported, and in return been supported by, the development of the movement for evidence-based medicine [6] and an increased focus on HTA in the U.S. [7], the UK, and many other countries [8]. This focus has, until recently, underplayed the importance of research on how health services are managed, organised and delivered, and on how methods of finance, delivery and organisation may have an impact on the outcome of health technologies [9]. This is not to say that research on these issues has not been carried out in the last 20 years. On the contrary, many good examples of such research exist. Research has been conducted on different funding systems, such as different methods of paying for health care [10] and of purchasing health care [11]; the basis for resource allocation within a health care system [12]; the nature of organisational structures within health
N. Fulop et al. / Health Policy 63 (2003) 155 /165
systems, such as internal markets [13]; the impact of organisational change on health care providers [14]; evaluations of different models of care such as community mental health teams [15]; and hospitalat-home services [16]; the appropriateness of the use of acute hospital beds [17]; analyses of interactions between health care professionals and patients [18]; and the impact of different approaches to management, such as the introduction of general management [19]. There is not, of course, a clearly defined boundary between HTA and questions relating to service delivery and organisation (SDO). Indeed, some would argue that it is a false distinction to make, as technical and organisational or social considerations are so interwoven that they should not be studied in isolation. SDO research and HTA research complement one another. For example, HTA can provide evidence on the cost effectiveness of one particular intervention (such as a drug or surgical technique) compared with another. SDO research would then be required to determine, where the most cost-effective setting (e.g. primary or secondary care, for example), would be for the delivery of that intervention. In England, while research on the delivery and organisation of services has been broadly supported in the past, it received a major boost with the establishment of a national governmentfunded programme in 1999 (http://www.sdo.lshtm.ac.uk). Initiatives in other countries, such as the Canadian Health Services Research Foundation [20], the Agency for Healthcare Research and Quality in the U.S. (http://www.ahrq.gov) and the Australian Centre for Effective Healthcare (http://www.aceh.usyd.edu.au), have also increased the focus in this area internationally.
3. Setting priorities for research on the organisation and delivery of health services If SDO research is going to have an impact on policy and practice, then those concerned with the day to day organisation and delivery of health services need to be involved in the commissioning and dissemination of research. These stakeholders include policy-makers, managers, health care pro-
157
fessionals, and users. This approach to the identification and prioritisation of research topics has been developed by the Canadian Health Services Research Foundation [20] and the Service Delivery and Organisation Research and Development Programme funded by the NHS in England [21]. The latter conducted a national ‘listening exercise’ which took the form of focus groups across the range of stakeholders within and allied to the NHS in England (the other countries in the UK have their own R&D programmes), including users and carers (over 350 people in total). From this exercise, 10 major themes emerged (see Table 1). A commissioning board has been established for the English SDO programme to take forward these themes with membership from the wide range of stakeholders */the majority of members are managers, professionals and users, rather than researchers. In this way, the SDO programme aims to commission research which is relevant to these stakeholders, the results of which are, therefore, more likely to be implemented.
4. Disciplines required for SDO research The list of themes in Table 1 indicates the need for a range of methods to address them. As much of the focus of health services research has been on
Table 1 Emerging themes from national listening exercise to inform the English National Health Service SDO Research and Development Programme Organising health services around the needs of the patient User involvement Continuity of care Co-ordination/integration across organisations Inter-professional working Workforce issues Relationship between organisational form, function and outcomes Implications of the communication revolution The use of resources, such as ways of disinvesting in services and managing demand The implementation of major national policy initiatives Source: Fulop and Allen [21].
158
N. Fulop et al. / Health Policy 63 (2003) 155 /165
HTA, there has been a methodological emphasis on the randomised trial as indicated by the influential hierarchy of methods which places the randomised trial at the top [22]. It has been argued that this has ‘retarded the development of a broader view of evaluation’ [23] and ignored the potential contributors of other, non-experimental methods, which are often more appropriate for evaluating health services [24]. Key questions for the organisation and delivery of health care include: Which models of service delivery are the most appropriate? How should changes to services be implemented? What impact do new models of care have on relationships between users and professionals? How can different health care professionals work in teams to provide more integrated care to users? To address these questions, a wide range of disciplines and methods needs to be considered [25]. These disciplines include sociology, organisational psychology, organisational studies, epidemiology, policy analysis, economics and history.
5. Paradigms: positivism, interactionism and postmodernism It is important to recognise that different paradigms operate both within and between disciplines. These paradigms have contrasting theoretical approaches, based on different theories of knowledge. It is important to acknowledge these differences in order to appreciate the extent to which researchers from these different disciplines can work together. One of the most important differences is between objectivity or positivism, on the one hand, and subjectivity or interactionism, on the other. Positivism can be characterised by the assertion that the facts are definitely out there, they just require gathering by the researcher. That is, there is one version of reality. In contrast, an interactionist or an interpretivist works within a paradigm where an interpretation is sought for what is observed and ‘truths’ or ‘facts’ are not ‘out there’ but dependent on different actors’ viewpoints.
It is important to note that both ‘sides’ contain a wide range of views. There can be as many debates between those on the ‘subjective side’ as between, say, the ‘opposing sides’ of interactionists and positivists. Another set of ideas */postmodernism */illustrates this. These ideas have been very influential in a number of disciplines in relation to questions of objectivity. The huge variety of ideas labelled postmodern, and their core value of diversity, makes summarisation difficult. However, it is important to note that postmodernists believe that knowledge is fundamentally fragmented, that is ‘knowledge is produced in so many bits and pieces there can be no reasonable expectation that it will add up to an integrated and singular view’ [26]. This leads to criticisms of efforts at universal understanding as ‘grand narrative’ [27]. Most importantly, postmodernists challenge modern notions of truth and the search for one best way to understand phenomena. Postmodernism denies the priority of observation, which underpins most of modern science and thereby opens the modernist scientific view of knowledge to debate. Modernism is interpreted by postmodernists as a series of truth ‘claims’ [26]. Each paradigm, as characterised here, presents a potential problem for SDO research. In the case of interactionism or inter-pretivism, for example, are the results so subjective that generalisable truths cannot be drawn about the way a service is organised independently of the particular organisation researched or the particular researcher undertaking that research? And the converse problem pertains with an entirely positivist view in which the researcher is unaware of the inextricable effect the research may be having on the very ‘truths’ he or she is hoping to uncover and unaware that another researcher might derive a very different account from the same research. It is easy to characterise these paradigms as diametrically opposed and constituting unbridegable barriers to those who would adopt one or another */what has been termed ‘paradigm wars’ [28]. To some extent, ‘positivism’ has been caricatured: researchers based in a broadly positivist tradition often also recognise that observation is theory-impregnated [29] and that knowledge is
N. Fulop et al. / Health Policy 63 (2003) 155 /165
socially organised [30]. Similarly, researchers from other traditions have come to acknowledge the importance of being explicit about the methods used. Others argue that these ‘paradigm wars’ have been unproductive in addressing important social questions [31]. Hammersley [32] proposes an alternative*/ subtle realism */to transcend the dichotomy between positivism and subjectivity/relativism. The subtle realist believes that the world exists independently of the researcher and that it is possible to undertake research on organisations. However, he or she acknowledges that both the researcher and the organisation will inevitably be affected by the process. The research is then designed not so much to uncover ‘facts’ or ‘truths’, as to represent them accurately. There is recognition that other representations from a different perspective might also be possible. One would expect these representations to be complementary because all would relate to the independent, underlying reality of the situation.
6. Deductive and inductive approaches Another important difference within and between disciplines is that between deductive and inductive approaches. The deductive approach is theory-driven and research is undertaken with an a priori theoretical view. A hypothesis is generated and classically, the aim of the research is to falsify the hypothesis [29]. The inductive approach is data-driven-facts derived from the research are used to generate theory. These approaches to the acquisition of knowledge lead different disciplines to hold different views about how knowledge is built up. For some disciplines there is a belief in the progressive acquisition of ‘facts’ or ‘truths’, while for others there is a more inductive approach which is not linear and where knowledge is built up through observation [33,34]. The stances taken on these two key differences (positivism: interactionism and deductive or inductive approach) inform the way the world is viewed by each discipline. They probably account for the deepest divisions both between and within
159
disciplines, and between researchers. It is important to understand these differences as they have fundamental implications for how researchers from different perspectives view important aspects of research methods such as the types of research undertaken; the role of the researcher; the focus on process or outcomes; the role of context and the extent of generalisability; approaches to the issues of reliability and validity; and the types of methods used [25]. Whilst these are important issues for most research, they are particularly pertinent to SDO research questions where organisational context is so central [9].
7. Contribution of different disciplines Different disciplines make complementary and contrasting contributions to research on the organisation and delivery of health services. The following disciplines have been selected to provide an illustration of the different approaches taken to the theoretical issues discussed earlier, and their tendency to focus on different elements of health service organisation and delivery, such as process or outcomes. These are organisational studies, qualitative sociology, epidemiology, organisational psychology, policy analysis, economic evaluation, organisational economics, and history. This list is certainly not exhaustive. Other disciplines, which can contribute to research on the organisation and delivery of health services, include anthropology and geography, and approaches such as operational research [35] and action research [36]. Imagine a group of health care practitioners and managers who are concerned to improve services for orthopaedic patients. Practitioners and managers require research evidence on a number of different aspects of the way such services are delivered and organised to ensure the best possible service is provided within the resources available. At the interface between primary and secondary care, managers first want to improve communication between practitioners so as to enhance patient care. In particular, they might want to know whether they should introduce a system of teleorthopaedics within primary care. This system
160
N. Fulop et al. / Health Policy 63 (2003) 155 /165
would provide a consultation and diagnostic service for GP’s and their patients in the GP’s surgery from hospital-based orthopaedic specialists. The research would need to compare the teleorthopaedics system with the traditional model of service organisation i.e. the patient attending a hospital outpatient clinic. Assuming that the research shows that teleorthopaedics as a technology is potentially a superior model of delivery to the standard one, the orthopaedic managers would then want to know how best to introduce this new system i.e. how to implement change in health service delivery and organisation. Finally, the managers want to take a ‘whole system’ approach to improving care for orthopaedic patients. They want to see how they can improve the level of integration and commitment that patients receive from the different providers in primary, secondary (out-patients and inpatients) and social care.
8. Stage one: How can the delivery of orthopaedic care be improved? Introducing a teleorthopaedics service would allow general practitioners to obtain an orthopaedic opinion in their own surgery rather than refer patients to the hospital outpatient department. While the new service would mean that patients, many of whom have impaired mobility and are elderly, would not have to travel to obtain an orthopaedic opinion, there are several uncertainties regarding the potential impact of this new way of organising and delivering a service: will it be as effective in terms of accuracy of diagnosis? Will it be as efficient for the health service given the initial capital investment and the GP’s time involved in teleconsultations? What impact might telemedicine have on the relationship between the GP and the specialist? What impact might it have on the patient’s relationship with both the GP and the specialist? At least four disciplines could help answer these questions: epidemiology, economics, history and sociology.
8.1. Will telemedicine be as clinically effective in terms of accuracy of diagnosis?*/epidemiological research Epidemiological techniques can be applied to organisational problems in health services, particularly the evaluation of interventions. These methods concentrate on the comparison of the outcomes of different modes of service delivery or organisational structures, rather than studying the processes themselves [37]. It may be possible to carry out a randomised trial in which patients needing an orthopaedic opinion are randomly allocated either to the hospital outpatient department or to be seen in the GP’s surgery using the telemedicine link to the hospital. It would be necessary to recruit a sufficient number of GPs prepared to randomise their patients. Given that telemedicine is a new service, it could be decided that it was only available within the trial. Thus GPs or patients not wishing to be randomised would be referred to outpatients in the traditional way. Effectiveness would be assessed as the proportion of patients who were correctly diagnosed, as judged independently by another orthopaedic surgeon who was unaware which form of consultation had occurred. This is the kind of trial suited to cluster randomisation because if a GP has access to the telemedicine service, this might inadvertently affect the way that he or she manages the ‘control’ patients. This is known as ‘contamination’. Cluster randomisation where, for example, GP practices rather than individual patients are randomised reduces this risk of contamination though it increases the size of the sample needed for the study. 8.2. Will telemedicine be as efficient for the health service? */economic evaluation As well as assessing the relative effectiveness of telemedicine, an economic evaluation could also be conducted. This formal quantitative economic technique is concerned with questions about the efficient allocation of resources. It can be used to compare alternative courses of action in terms of their costs and consequences [38]. The costs of both types of consultation will need to be deter-
N. Fulop et al. / Health Policy 63 (2003) 155 /165
mined. The principal additional costs for telemedicine will be a contribution to the capital cost of the equipment, the cost of transmission of images and the GP’s time. The orthopaedic surgeon’s time and the cost of investigations might be the same as outpatients though this cannot be assumed. Both would need to be measured in both arms of the trial. The difference in costs borne by the patients and their families when using telemedicine or conventional care will need to be measured. All these costs could be collected during the randomised trial. It is possible that telemedicine might lead to a radically different organisation of working practices */if this were the case then the differences in costs would need to be measured.
8.3. What impact might telemedicine have on the relationship between GP and the specialist? */ historical research Some people have voiced concern that telemedicine might adversely affect the longstanding relationship between general practitioners and specialists. Others have suggested the converse that both parties would benefit from the joint consultations that are a feature of telemedicine. The nature of the existing relationship is the product of many years going back to the apothecaries and barber-surgeons of medieval times. Historical research could throw light on the nature of the relationship and how it has developed thus increasing our understanding of the impact of bringing the specialist into the GP’s consulting room by means of a television link, in the presence of the patient. Historical research on the introduction of new technologies in general might help us think about the impact of telemedicine in this instance. The value of historical research to the provision of contemporary health care is not usually recognised. It is particularly suited to analysing change (e.g. how current organisational structures and modes of delivery of care resemble or differ from previous ones), although, it cannot be used to answer evaluative questions about ‘what works?’ [39].
161
8.4. What impact might telemedicine have on the patient’s relationship with the GP and the specialist? */micro-sociology Qualitative sociology at the micro level can be used to explore interactions between individuals in health care settings [40]. The effectiveness (and as a result, cost-effectiveness) of consultations depends partly on the nature of the relationship between the doctor and the patient. The relationship can facilitate or impede communication and understanding, and can determine the extent to which the patient participates in decisions concerning treatment and management of their condition. It seems likely that teleconsultations will affect the relationship a patient has with his or her GP. This might be beneficial or adverse. Qualitative research using observational methods (audio or video recording of consultations) together with in-depth interviews of the participants could help our understanding of the impact of teleconsultations on the humanity of care from the patient’s perspective and on the doctor’s perception of his or her relationship with the patient.
9. Stage two: How can change to existing services be implemented? If research showed that teleorthopaedics is a better method than the traditional model, the next step would be to use research to understand how the delivery of services can be changed to include telemedicine. The research question therefore concerns the investigation and evaluation of methods to implement changes in how services are delivered. 9.1. What effect would the method of funding orthopaedic services have on implementing telemedicine? */organisational economics This discipline focuses on the effects on costs of different ways of organising the production of services (or products) and of organising their exchange between organisations. This type of economic analysis is relevant to questions such as
162
N. Fulop et al. / Health Policy 63 (2003) 155 /165
the appropriateness of boundaries between organisations, and the incentives, which influence the behaviour of individuals when responding to scarcity [41]. In this case, an analysis of the financial incentives relevant to the delivery of orthopaedic services within a hospital could be carried out. This could identify, for example, if the hospital would lose income if, instead of patients attending outpatients, they consulted with the orthopaedic surgeon by means of telemedicine. If that were the case, there would be an incentive for hospital managers to block the introduction of telemedicine, unless the hospital was able concomitantly to reduce its costs to take account of any reduction in income. 9.2. What impact does telemedicine have on the behaviour, attitudes and emotions of clinical staff? */organisational psychology Organisational psychology is concerned with the behaviour, thoughts and emotions of individuals and groups within organisations [42]. Theoretical models from this discipline concerning how people learn and how they can be induced to change their behaviour could be used to understand the relevant aspects of the orthopaedic surgeons’ personalities, motivation and cognitive processes, in order to facilitate change. These models could be used to design a survey of the surgeons to find out what motivational factors are most relevant in changing behaviour in these circumstances. Organisational psychology could also be used to study the effect of introducing telemedicine on clinical and managerial staff, for example, in terms of job satisfaction and levels of team working.
10. Stage three: How can care be improved further by changes at the organisational level? The hospital managers and practitioners now want to improve the overall quality of care patients receive from the different component organisations: primary, secondary (out-patients and inpatients) and social care, and how a more integrated service can be provided.
Whichever discipline, or combination of disciplines is used, one key conceptual task would be to unpack the term ‘integration’ and arrive at definitions which could be used in the study. This might be a focus of the early part of any study. 10.1. What factors facilitate or hinder integration of services? */organisational studies The focus of organisational studies is on the importance of organisations as a site for social action [43]. In organisational studies, behaviour of individuals and organisations are viewed as ‘socially embedded’ through such factors as norms, culture, and power relations. An important aspect of organisational studies is that it emphasises the importance of context in understanding organisational behaviour. Although this can make generalisation difficult, organisational studies can help us understand what it is about a particular context that, for example, facilitates the integration of services. Looking at a range of hospitals, case studies could identify organisational factors, which facilitate or hinder integration of services. These factors might include organisational form and the organisational division of labour, professional boundaries and organisational cultures. For example, does the integration of organisations providing orthopaedic services result in improved service integration? Have some organisations been more successful at breaking down professional barriers and creating multidisciplinary teams to improve service integration? 10.2. Which policies might facilitate greater integration? */policy analysis Policy analysis focuses on the processes by which policies are made and the context in which this occurs [44]. This approach could be used to provide an analysis of key actors’ values and perceptions of service integration and how better integration might be achieved. Key factors would include practitioners, managers, policy makers, planners, and service users. The analysis could be used to create an ‘ideal type’ model of how service integration might be achieved in an ideal world, or used to compare different actors’ ‘framing’ of the
N. Fulop et al. / Health Policy 63 (2003) 155 /165
issue and compare their perceptions with any available evidence. This would aid understanding of the issues in integrating services and what issues might arise in any proposed policy intervention. Policy analysis could also be used to evaluate particular policy interventions designed to improve integration of services. The study design might consist of comparative case studies or ‘natural experiments’. Where a policy was introduced universally, the comparative case study approach could be used to compare the different contexts within which the intervention had been introduced (e.g. doctor- or nurse-led, urban or rural setting). In the situation where a policy was not introduced universally, the comparative case study approach could compare areas where the policy had been introduced with those where it had not. In either case, such a study would seek to generalise the research findings not through aggregation, but rather through the development of theory.
11. Issues involved in undertaking multi-disciplinary research on SDO From this example it can be seen that many different disciplines have invaluable contributions to make to SDO research. However, there are important differences both between and within disciplines, which need to be understood if multidisciplinary research is possible. Multi-disciplinary work is easiest where researchers from the different disciplines share paradigms. Thus it is possible to imagine epidemiologists and economists sharing a broadly ‘positivist’ perspective working together to evaluate a change in the way a health service is delivered. Similarly, it is possible to imagine researchers from organisational studies and policy analysis with an interpretivist perspective working together to study the same change in delivery of a health service. The key question is whether it is possible, or indeed desirable, for researchers from different paradigms to work together to address the same research problem. As we have tried to emphasise, these perspectives are not necessarily at opposite poles */they form a range of views, along a
163
continuum. So, for example, it is possible to imagine a ‘subtle realist’ using qualitative methods to study a change in service delivery which has uniquely occurred in one organisation, while an epidemiologist uses this ‘natural experiment’ to compare this organisation with another one. It is also important to recognise that it will be not always be possible to reconcile positions within each perspective. It is more difficult to imagine an epidemiologist working alongside a researcher from a post-modernist perspective within organisational studies. Given both the desirability of researchers from different disciplines and perspectives to work together and that in many instances it should be feasible, how can multi-disciplinary research be encouraged? There is clearly a role for research funding bodies to provide encouragement and even incentives for such multi-disciplinary research by, at the very least, requiring evidence of it in applications for research grants. Members of the research community themselves also need to take responsibility for thinking outside their own paradigms or worldviews so that they can work with others. As we have argued, it is vital that research, such as that described above on telemedicine, takes account of the role of the context within which the organisation and delivery of health services takes place, as well as their underlying processes . This is probably even more important for SDO research than for HTA. Therefore, researchers who have traditionally taken a ‘black box’ approach focussing on inputs and outputs need to work with researchers who study context and processes, and vice-versa [25].
12. Conclusions HTA research is clearly vital but it is important to complement it by a focus of research efforts on the organisation and delivery of health services. The latter has recently started to receive more funding and attention. This is important if the balance between HTA and SDO research is to be corrected. The benefits of SDO research are potentially just as great as those of HTA research. Research on the organisation and delivery of
164
N. Fulop et al. / Health Policy 63 (2003) 155 /165
health services requires a wide array of disciplines and methods if these complex questions are going to be appropriately addressed. It is a challenge, therefore, for researchers from these disciplines to see how far they can work together to carry out research in this important field. There is a role for research funding bodies to encourage researchers from different disciplines and perspectives to work together. A further challenge for research on the organisation and delivery of health services is one shared with other applied fields of policy research*/how far can this research influence policy and practice? The real test for SDO research is that health care professionals, managers and users value its outputs. This complex question of the implementation of research findings is one which also concerns those involved in HTA research. Implementation issues for SDO research are particularly challenging because of the complexity of the interventions and topics researched, the methodological and paradigmatic differences between researchers (which mean that issues of generalisability are highly contested), and the wide range of stakeholders involved. These issues alongside the rich mix of disciplines and methods which can be brought to bear on SDO research questions mean that the expanding world of SDO research promises to be an exciting one.
[4] [5] [6]
[7]
[8]
[9]
[10]
[11]
[12]
[13] [14]
Acknowledgements We are grateful to two anonymous referees for their comments on an earlier version of this paper. NJF, PA and AC are funded by the NHS Service Delivery and Organisation R&D programme.
[15]
[16]
References [1] Bowling A. Research methods in health: investigating health and health services. Buckingham: Open University Press, 1997. [2] Black N. Health services research: saviour or chimera? Lancet 1997;349:1834 /6. [3] Cochrane, A. Effectiveness and efficiency: random reflections on health services. London: Nuffield Provincial
[17]
[18]
[19]
Hospitals Trust Illich I, 1972. Medical nemesis: limits to medicine. Harmondsworth: Penguin, 1976. McKeown T. The role of medicine, 2nd ed.. Oxford: Oxford University Press, 1979. Illich I. Medical nemesis: limits to medicine. Harmondsworth: Penguin, 1976. Sackett D, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. [editorial]. British Medical Journal 1996;312:71 /2. US Congress, Office of Technology Assessment. Identifying health technologies that work: searching for evidence */OTA-H-608. Washington DC, U.S.: Government Printing Office, 1994 Perry S, Gardner E, Thamer M. The status of health technology assessment worldwide. Results of an international survey. International Journal of Technology Assessment in Health Care 1997;j13:81 /98. Sheldon TA. It ain’t what you do but the way that you do it. Journal of Health Services Research and Policy 2001;6(1):3 /5. The Insurance Experiment Group, Newhouse JP. Free for all? Lessons from the RAND health insurance experiment. Cambridge, MA: Harvard University Press, 1993. (on behalf of the Total Purchasing National Evaluation Team), Mays N, Goodwin N, Bevan G, Wykes S. Total purchasing: a profile of national pilot projects. London: Kings Fund, 1997. Carr-Hill RA, Sheldon TA, Smith P, Martin S, Peacock S, Hardman G. Allocating resources to health authorities: development of method for small area analysis of use of inpatient services. British Medical Journal 1994;309:1046 / 9. Allen P. Contracts in the NHS internal market. Modern Law Review 1995;58:321 /42. Shortell SM, Jones RH, Rademaker AW, Gillies RR, Dranove DS, Hughes EFX, Budetti PP, Reynolds KSE, Huang C. Assessing the impact of total quality management and organisational culture on multiple outcomes of care for coronary artery bypass graft surgery patients. Medical Care 2000;38(2):207 /17. Tyrer P, Evans K, Gandhi N, Lamont A, Harrison-Read P, Johnson T. Randomised controlled trial of two models of care for discharged psychiatric patients. British Medical Journal 1998;316:106 /9. Fulop N, Hood S, Parsons S. Does the National Health Service want hospital at home? Journal of the Royal Society of Medicine 1997;90:212 /5. Victor CR, Nazareth B, Hudson M, Fulop N. Inappropriate use of acute hospital beds in an inner London health authority. Health Trends 1993;25:94 /7. Stimson G, Webb B. Going to see the Doctor. London: Routledge, 1975. Strong P. The ceremonial order of the clinic: parents, doctors and medical bureaucracies. London: Routledge, Kegan and Paul, 1979. Strong P, Robinson J. The NHS: under new management. Milton Keynes: Open University Press, 1990.
N. Fulop et al. / Health Policy 63 (2003) 155 /165 [20] Lomas J. Using ‘linkage and exchange’ to move research into policy at a Canadian foundation. Health Affairs 2000;19(3):236 /40. [21] Fulop N, Allen P. National listening exercise: report of the findings. London: NCCSDO, 2000. url: http:// www.sdo.lshtm.ac.uk. [22] Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guides to the medical literature. IX. A method for grading health care recommendations. Journal of American Medical Association 1995;274(22):1800 /4. [23] St Leger AS, Walsworth Bell JP. Change-promoting research for health services. Buckingham: Open University Press, 1999. [24] Black N. Why we need observational studies to evaluate the effectiveness of health care. British Medical Journal 1996;312:1215 /8. [25] Fulop N, Allen P, Clarke A, Black N. Issues in studying the organisation and delivery of health services. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [26] Hatch M. Organisation theory: modern symbolic and postmodern perspectives. Oxford: Oxford University Press, 1997. [27] Lyotard JF. The postmodern condition: a report on knowledge. Manchester: Manchester University Press, 1979. [28] Pawson RD, Tilley N. Realistic evaluation. London: Sage, 1997. [29] Popper KR. The logic of scientific discovery. London: Hutchinson, 1972. [30] Kuhn T. The structure of scientific revolutions. Chicago: University of Chicago Press, 1970. [31] Oakley A. Experiments in knowing. Cambridge: Polity Press, 2000. [32] Hammersley M. What’s wrong with ethnography?. London: Routledge, 1992. [33] Ovretveit J. Evaluating health interventions. Buckingham: Open University Press, 1998. [34] Guba EG, Lincoln YS. Fourth generation evaluation. Newbury Park, CA: Sage, 1989.
165
[35] Rosenhead J. Operational research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [36] Meyer J. Action research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [37] Grimshaw J. Epidemiological methods. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [38] Sculpher M. Economic evaluation. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [39] Berridge V. Historical research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [40] Murphy E. Micro-level qualitative research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [41] Dawson D. Organisational economics. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [42] Arnold J. Organisational psychology. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [43] Ferlie E. Organisational studies. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [44] Harrison S. Policy analysis. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001.
From health technology assessment to research on the organisation and delivery of health services: addressing the balance Naomi Fulop *, Pauline Allen, Aileen Clarke, Nick Black Health Services Research Unit, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK Received 28 October 2001; accepted 26 March 2002
Abstract This paper argues that the focus of research to improve health services has, until recently, been on health technology assessment. The authors make the case for a greater emphasis on research on how health services are managed, organised and delivered, and refer to initiatives in a number of countries which are seeking to address this balance. The way two such initiatives in England and Canada have set priorities for this type of research, involving a wide range of stakeholders is described. The authors argue that a wide range of disciplines needs to be applied to research on the organisation and delivery of health services. Important theoretical differences between and within disciplines, and their implications for research methods, are discussed. An example of an issue in the delivery of organisation of health services (how best to deliver orthopaedic care) is used to illustrate how a number of different disciplines can be applied. The challenge for researchers from these disciplines is to see how far they can work together to carry out research in this important field. The challenge for this research is that the findings are valued and used by health service professionals, managers and users. # 2002 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Health services; Organisation; Delivery; Management; Research methods; Multi-disciplinary research
1. Introduction Throughout the world, societies are striving to determine how best to organise their health systems and deliver services. There is increasing recognition that advances in medicine are only part of the answer to better health care. These medical advances, alongside rising expectations of * Corresponding author E-mail address: [email protected] (N. Fulop).
health care users, mean that demands on the way we organise services will become even greater. It is therefore essential that, in parallel to biomedical research and health technology assessment (HTA), we advance our understanding and knowledge of how best to organise and deliver health services. Without this, we run the risk of failing to realise the potential benefits that health systems can provide. In addition, the need to be more responsive to the needs and wishes of those who use health care
0168-8510/02/$ - see front matter # 2002 Elsevier Science Ireland Ltd. All rights reserved. PII: S 0 1 6 8 - 8 5 1 0 ( 0 2 ) 0 0 0 6 2 - 3
156
N. Fulop et al. / Health Policy 63 (2003) 155 /165
is increasingly being accepted. The traditional tendency for health service organisations to prioritise staff needs over users’ needs is no longer acceptable, regardless of whether the system is publicly or privately financed. All of these pressures have led governments to recognise the need to increase their support for research that can help to improve the way we organise and deliver health services. While this is not a new area of study, the previously disjointed, ad hoc activities that characterised such research are giving way to a more strategic approach, with national programmes of research in some countries. This, in turn, will require a growth in both the volume and quality of research. To achieve these, the tendency for fragmentation between the very disciplines that can make key contributions must be overcome. In this paper, first we outline the development of research on the organisation and delivery of health services. Then we discuss ways of setting priorities for this area of research, drawing on the experience of the new national programme in England. The range of disciplines required is discussed, drawing on the different epistemological approaches underlying them. Finally, the contribution of a number of disciplines to this area is illustrated, using an example of a change in the delivery of health services (telemedicine), to show how these disciplines can address different aspects of health service delivery and organisation.
2. Development of research on the organisation and delivery of health services Research on the way health services are delivered and organised is part of the broader field of health services (or systems) research which has become well-established in the UK, North America and parts of Europe in the last 20 years. Health services research aims to ‘produce reliable and valid research data on which to base appropriate, effective, cost-effective, efficient and acceptable health services’ [1]. It is not a scientific discipline in itself but an area of applied research, which draws on a range of disciplines and perspectives including anthropology, economics, epidemiology,
history, medicine, nursing, political science, sociology, and statistics. It concentrates on the study of health services or systems rather than on the state of health and disease of individuals and of populations. Unlike clinical research, which has traditionally focused on individual patients in relation to their treatment and care, health services research adopts a population perspective. Questions considered by health services research include: How much health care should we have? How should services be funded? Who should receive health services? and, How well are services being delivered? [2]. The interests of different stakeholders in health services have led to the growth, development and increased expectations of health services research. Funders of health care have sought help to address the problem of cost containment; health care professionals, particularly doctors who have come under pressure to defend themselves against those who argued that many clinical interventions were unproven, ineffective, or even harmful [3 /5], have sought scientific evidence from health services research to defend their practices; and users of health services have looked to this research to support their growing demands for more information about the care they receive. These pressures have led to the tendency for the focus of health services research to be on HTA. This has supported, and in return been supported by, the development of the movement for evidence-based medicine [6] and an increased focus on HTA in the U.S. [7], the UK, and many other countries [8]. This focus has, until recently, underplayed the importance of research on how health services are managed, organised and delivered, and on how methods of finance, delivery and organisation may have an impact on the outcome of health technologies [9]. This is not to say that research on these issues has not been carried out in the last 20 years. On the contrary, many good examples of such research exist. Research has been conducted on different funding systems, such as different methods of paying for health care [10] and of purchasing health care [11]; the basis for resource allocation within a health care system [12]; the nature of organisational structures within health
N. Fulop et al. / Health Policy 63 (2003) 155 /165
systems, such as internal markets [13]; the impact of organisational change on health care providers [14]; evaluations of different models of care such as community mental health teams [15]; and hospitalat-home services [16]; the appropriateness of the use of acute hospital beds [17]; analyses of interactions between health care professionals and patients [18]; and the impact of different approaches to management, such as the introduction of general management [19]. There is not, of course, a clearly defined boundary between HTA and questions relating to service delivery and organisation (SDO). Indeed, some would argue that it is a false distinction to make, as technical and organisational or social considerations are so interwoven that they should not be studied in isolation. SDO research and HTA research complement one another. For example, HTA can provide evidence on the cost effectiveness of one particular intervention (such as a drug or surgical technique) compared with another. SDO research would then be required to determine, where the most cost-effective setting (e.g. primary or secondary care, for example), would be for the delivery of that intervention. In England, while research on the delivery and organisation of services has been broadly supported in the past, it received a major boost with the establishment of a national governmentfunded programme in 1999 (http://www.sdo.lshtm.ac.uk). Initiatives in other countries, such as the Canadian Health Services Research Foundation [20], the Agency for Healthcare Research and Quality in the U.S. (http://www.ahrq.gov) and the Australian Centre for Effective Healthcare (http://www.aceh.usyd.edu.au), have also increased the focus in this area internationally.
3. Setting priorities for research on the organisation and delivery of health services If SDO research is going to have an impact on policy and practice, then those concerned with the day to day organisation and delivery of health services need to be involved in the commissioning and dissemination of research. These stakeholders include policy-makers, managers, health care pro-
157
fessionals, and users. This approach to the identification and prioritisation of research topics has been developed by the Canadian Health Services Research Foundation [20] and the Service Delivery and Organisation Research and Development Programme funded by the NHS in England [21]. The latter conducted a national ‘listening exercise’ which took the form of focus groups across the range of stakeholders within and allied to the NHS in England (the other countries in the UK have their own R&D programmes), including users and carers (over 350 people in total). From this exercise, 10 major themes emerged (see Table 1). A commissioning board has been established for the English SDO programme to take forward these themes with membership from the wide range of stakeholders */the majority of members are managers, professionals and users, rather than researchers. In this way, the SDO programme aims to commission research which is relevant to these stakeholders, the results of which are, therefore, more likely to be implemented.
4. Disciplines required for SDO research The list of themes in Table 1 indicates the need for a range of methods to address them. As much of the focus of health services research has been on
Table 1 Emerging themes from national listening exercise to inform the English National Health Service SDO Research and Development Programme Organising health services around the needs of the patient User involvement Continuity of care Co-ordination/integration across organisations Inter-professional working Workforce issues Relationship between organisational form, function and outcomes Implications of the communication revolution The use of resources, such as ways of disinvesting in services and managing demand The implementation of major national policy initiatives Source: Fulop and Allen [21].
158
N. Fulop et al. / Health Policy 63 (2003) 155 /165
HTA, there has been a methodological emphasis on the randomised trial as indicated by the influential hierarchy of methods which places the randomised trial at the top [22]. It has been argued that this has ‘retarded the development of a broader view of evaluation’ [23] and ignored the potential contributors of other, non-experimental methods, which are often more appropriate for evaluating health services [24]. Key questions for the organisation and delivery of health care include: Which models of service delivery are the most appropriate? How should changes to services be implemented? What impact do new models of care have on relationships between users and professionals? How can different health care professionals work in teams to provide more integrated care to users? To address these questions, a wide range of disciplines and methods needs to be considered [25]. These disciplines include sociology, organisational psychology, organisational studies, epidemiology, policy analysis, economics and history.
5. Paradigms: positivism, interactionism and postmodernism It is important to recognise that different paradigms operate both within and between disciplines. These paradigms have contrasting theoretical approaches, based on different theories of knowledge. It is important to acknowledge these differences in order to appreciate the extent to which researchers from these different disciplines can work together. One of the most important differences is between objectivity or positivism, on the one hand, and subjectivity or interactionism, on the other. Positivism can be characterised by the assertion that the facts are definitely out there, they just require gathering by the researcher. That is, there is one version of reality. In contrast, an interactionist or an interpretivist works within a paradigm where an interpretation is sought for what is observed and ‘truths’ or ‘facts’ are not ‘out there’ but dependent on different actors’ viewpoints.
It is important to note that both ‘sides’ contain a wide range of views. There can be as many debates between those on the ‘subjective side’ as between, say, the ‘opposing sides’ of interactionists and positivists. Another set of ideas */postmodernism */illustrates this. These ideas have been very influential in a number of disciplines in relation to questions of objectivity. The huge variety of ideas labelled postmodern, and their core value of diversity, makes summarisation difficult. However, it is important to note that postmodernists believe that knowledge is fundamentally fragmented, that is ‘knowledge is produced in so many bits and pieces there can be no reasonable expectation that it will add up to an integrated and singular view’ [26]. This leads to criticisms of efforts at universal understanding as ‘grand narrative’ [27]. Most importantly, postmodernists challenge modern notions of truth and the search for one best way to understand phenomena. Postmodernism denies the priority of observation, which underpins most of modern science and thereby opens the modernist scientific view of knowledge to debate. Modernism is interpreted by postmodernists as a series of truth ‘claims’ [26]. Each paradigm, as characterised here, presents a potential problem for SDO research. In the case of interactionism or inter-pretivism, for example, are the results so subjective that generalisable truths cannot be drawn about the way a service is organised independently of the particular organisation researched or the particular researcher undertaking that research? And the converse problem pertains with an entirely positivist view in which the researcher is unaware of the inextricable effect the research may be having on the very ‘truths’ he or she is hoping to uncover and unaware that another researcher might derive a very different account from the same research. It is easy to characterise these paradigms as diametrically opposed and constituting unbridegable barriers to those who would adopt one or another */what has been termed ‘paradigm wars’ [28]. To some extent, ‘positivism’ has been caricatured: researchers based in a broadly positivist tradition often also recognise that observation is theory-impregnated [29] and that knowledge is
N. Fulop et al. / Health Policy 63 (2003) 155 /165
socially organised [30]. Similarly, researchers from other traditions have come to acknowledge the importance of being explicit about the methods used. Others argue that these ‘paradigm wars’ have been unproductive in addressing important social questions [31]. Hammersley [32] proposes an alternative*/ subtle realism */to transcend the dichotomy between positivism and subjectivity/relativism. The subtle realist believes that the world exists independently of the researcher and that it is possible to undertake research on organisations. However, he or she acknowledges that both the researcher and the organisation will inevitably be affected by the process. The research is then designed not so much to uncover ‘facts’ or ‘truths’, as to represent them accurately. There is recognition that other representations from a different perspective might also be possible. One would expect these representations to be complementary because all would relate to the independent, underlying reality of the situation.
6. Deductive and inductive approaches Another important difference within and between disciplines is that between deductive and inductive approaches. The deductive approach is theory-driven and research is undertaken with an a priori theoretical view. A hypothesis is generated and classically, the aim of the research is to falsify the hypothesis [29]. The inductive approach is data-driven-facts derived from the research are used to generate theory. These approaches to the acquisition of knowledge lead different disciplines to hold different views about how knowledge is built up. For some disciplines there is a belief in the progressive acquisition of ‘facts’ or ‘truths’, while for others there is a more inductive approach which is not linear and where knowledge is built up through observation [33,34]. The stances taken on these two key differences (positivism: interactionism and deductive or inductive approach) inform the way the world is viewed by each discipline. They probably account for the deepest divisions both between and within
159
disciplines, and between researchers. It is important to understand these differences as they have fundamental implications for how researchers from different perspectives view important aspects of research methods such as the types of research undertaken; the role of the researcher; the focus on process or outcomes; the role of context and the extent of generalisability; approaches to the issues of reliability and validity; and the types of methods used [25]. Whilst these are important issues for most research, they are particularly pertinent to SDO research questions where organisational context is so central [9].
7. Contribution of different disciplines Different disciplines make complementary and contrasting contributions to research on the organisation and delivery of health services. The following disciplines have been selected to provide an illustration of the different approaches taken to the theoretical issues discussed earlier, and their tendency to focus on different elements of health service organisation and delivery, such as process or outcomes. These are organisational studies, qualitative sociology, epidemiology, organisational psychology, policy analysis, economic evaluation, organisational economics, and history. This list is certainly not exhaustive. Other disciplines, which can contribute to research on the organisation and delivery of health services, include anthropology and geography, and approaches such as operational research [35] and action research [36]. Imagine a group of health care practitioners and managers who are concerned to improve services for orthopaedic patients. Practitioners and managers require research evidence on a number of different aspects of the way such services are delivered and organised to ensure the best possible service is provided within the resources available. At the interface between primary and secondary care, managers first want to improve communication between practitioners so as to enhance patient care. In particular, they might want to know whether they should introduce a system of teleorthopaedics within primary care. This system
160
N. Fulop et al. / Health Policy 63 (2003) 155 /165
would provide a consultation and diagnostic service for GP’s and their patients in the GP’s surgery from hospital-based orthopaedic specialists. The research would need to compare the teleorthopaedics system with the traditional model of service organisation i.e. the patient attending a hospital outpatient clinic. Assuming that the research shows that teleorthopaedics as a technology is potentially a superior model of delivery to the standard one, the orthopaedic managers would then want to know how best to introduce this new system i.e. how to implement change in health service delivery and organisation. Finally, the managers want to take a ‘whole system’ approach to improving care for orthopaedic patients. They want to see how they can improve the level of integration and commitment that patients receive from the different providers in primary, secondary (out-patients and inpatients) and social care.
8. Stage one: How can the delivery of orthopaedic care be improved? Introducing a teleorthopaedics service would allow general practitioners to obtain an orthopaedic opinion in their own surgery rather than refer patients to the hospital outpatient department. While the new service would mean that patients, many of whom have impaired mobility and are elderly, would not have to travel to obtain an orthopaedic opinion, there are several uncertainties regarding the potential impact of this new way of organising and delivering a service: will it be as effective in terms of accuracy of diagnosis? Will it be as efficient for the health service given the initial capital investment and the GP’s time involved in teleconsultations? What impact might telemedicine have on the relationship between the GP and the specialist? What impact might it have on the patient’s relationship with both the GP and the specialist? At least four disciplines could help answer these questions: epidemiology, economics, history and sociology.
8.1. Will telemedicine be as clinically effective in terms of accuracy of diagnosis?*/epidemiological research Epidemiological techniques can be applied to organisational problems in health services, particularly the evaluation of interventions. These methods concentrate on the comparison of the outcomes of different modes of service delivery or organisational structures, rather than studying the processes themselves [37]. It may be possible to carry out a randomised trial in which patients needing an orthopaedic opinion are randomly allocated either to the hospital outpatient department or to be seen in the GP’s surgery using the telemedicine link to the hospital. It would be necessary to recruit a sufficient number of GPs prepared to randomise their patients. Given that telemedicine is a new service, it could be decided that it was only available within the trial. Thus GPs or patients not wishing to be randomised would be referred to outpatients in the traditional way. Effectiveness would be assessed as the proportion of patients who were correctly diagnosed, as judged independently by another orthopaedic surgeon who was unaware which form of consultation had occurred. This is the kind of trial suited to cluster randomisation because if a GP has access to the telemedicine service, this might inadvertently affect the way that he or she manages the ‘control’ patients. This is known as ‘contamination’. Cluster randomisation where, for example, GP practices rather than individual patients are randomised reduces this risk of contamination though it increases the size of the sample needed for the study. 8.2. Will telemedicine be as efficient for the health service? */economic evaluation As well as assessing the relative effectiveness of telemedicine, an economic evaluation could also be conducted. This formal quantitative economic technique is concerned with questions about the efficient allocation of resources. It can be used to compare alternative courses of action in terms of their costs and consequences [38]. The costs of both types of consultation will need to be deter-
N. Fulop et al. / Health Policy 63 (2003) 155 /165
mined. The principal additional costs for telemedicine will be a contribution to the capital cost of the equipment, the cost of transmission of images and the GP’s time. The orthopaedic surgeon’s time and the cost of investigations might be the same as outpatients though this cannot be assumed. Both would need to be measured in both arms of the trial. The difference in costs borne by the patients and their families when using telemedicine or conventional care will need to be measured. All these costs could be collected during the randomised trial. It is possible that telemedicine might lead to a radically different organisation of working practices */if this were the case then the differences in costs would need to be measured.
8.3. What impact might telemedicine have on the relationship between GP and the specialist? */ historical research Some people have voiced concern that telemedicine might adversely affect the longstanding relationship between general practitioners and specialists. Others have suggested the converse that both parties would benefit from the joint consultations that are a feature of telemedicine. The nature of the existing relationship is the product of many years going back to the apothecaries and barber-surgeons of medieval times. Historical research could throw light on the nature of the relationship and how it has developed thus increasing our understanding of the impact of bringing the specialist into the GP’s consulting room by means of a television link, in the presence of the patient. Historical research on the introduction of new technologies in general might help us think about the impact of telemedicine in this instance. The value of historical research to the provision of contemporary health care is not usually recognised. It is particularly suited to analysing change (e.g. how current organisational structures and modes of delivery of care resemble or differ from previous ones), although, it cannot be used to answer evaluative questions about ‘what works?’ [39].
161
8.4. What impact might telemedicine have on the patient’s relationship with the GP and the specialist? */micro-sociology Qualitative sociology at the micro level can be used to explore interactions between individuals in health care settings [40]. The effectiveness (and as a result, cost-effectiveness) of consultations depends partly on the nature of the relationship between the doctor and the patient. The relationship can facilitate or impede communication and understanding, and can determine the extent to which the patient participates in decisions concerning treatment and management of their condition. It seems likely that teleconsultations will affect the relationship a patient has with his or her GP. This might be beneficial or adverse. Qualitative research using observational methods (audio or video recording of consultations) together with in-depth interviews of the participants could help our understanding of the impact of teleconsultations on the humanity of care from the patient’s perspective and on the doctor’s perception of his or her relationship with the patient.
9. Stage two: How can change to existing services be implemented? If research showed that teleorthopaedics is a better method than the traditional model, the next step would be to use research to understand how the delivery of services can be changed to include telemedicine. The research question therefore concerns the investigation and evaluation of methods to implement changes in how services are delivered. 9.1. What effect would the method of funding orthopaedic services have on implementing telemedicine? */organisational economics This discipline focuses on the effects on costs of different ways of organising the production of services (or products) and of organising their exchange between organisations. This type of economic analysis is relevant to questions such as
162
N. Fulop et al. / Health Policy 63 (2003) 155 /165
the appropriateness of boundaries between organisations, and the incentives, which influence the behaviour of individuals when responding to scarcity [41]. In this case, an analysis of the financial incentives relevant to the delivery of orthopaedic services within a hospital could be carried out. This could identify, for example, if the hospital would lose income if, instead of patients attending outpatients, they consulted with the orthopaedic surgeon by means of telemedicine. If that were the case, there would be an incentive for hospital managers to block the introduction of telemedicine, unless the hospital was able concomitantly to reduce its costs to take account of any reduction in income. 9.2. What impact does telemedicine have on the behaviour, attitudes and emotions of clinical staff? */organisational psychology Organisational psychology is concerned with the behaviour, thoughts and emotions of individuals and groups within organisations [42]. Theoretical models from this discipline concerning how people learn and how they can be induced to change their behaviour could be used to understand the relevant aspects of the orthopaedic surgeons’ personalities, motivation and cognitive processes, in order to facilitate change. These models could be used to design a survey of the surgeons to find out what motivational factors are most relevant in changing behaviour in these circumstances. Organisational psychology could also be used to study the effect of introducing telemedicine on clinical and managerial staff, for example, in terms of job satisfaction and levels of team working.
10. Stage three: How can care be improved further by changes at the organisational level? The hospital managers and practitioners now want to improve the overall quality of care patients receive from the different component organisations: primary, secondary (out-patients and inpatients) and social care, and how a more integrated service can be provided.
Whichever discipline, or combination of disciplines is used, one key conceptual task would be to unpack the term ‘integration’ and arrive at definitions which could be used in the study. This might be a focus of the early part of any study. 10.1. What factors facilitate or hinder integration of services? */organisational studies The focus of organisational studies is on the importance of organisations as a site for social action [43]. In organisational studies, behaviour of individuals and organisations are viewed as ‘socially embedded’ through such factors as norms, culture, and power relations. An important aspect of organisational studies is that it emphasises the importance of context in understanding organisational behaviour. Although this can make generalisation difficult, organisational studies can help us understand what it is about a particular context that, for example, facilitates the integration of services. Looking at a range of hospitals, case studies could identify organisational factors, which facilitate or hinder integration of services. These factors might include organisational form and the organisational division of labour, professional boundaries and organisational cultures. For example, does the integration of organisations providing orthopaedic services result in improved service integration? Have some organisations been more successful at breaking down professional barriers and creating multidisciplinary teams to improve service integration? 10.2. Which policies might facilitate greater integration? */policy analysis Policy analysis focuses on the processes by which policies are made and the context in which this occurs [44]. This approach could be used to provide an analysis of key actors’ values and perceptions of service integration and how better integration might be achieved. Key factors would include practitioners, managers, policy makers, planners, and service users. The analysis could be used to create an ‘ideal type’ model of how service integration might be achieved in an ideal world, or used to compare different actors’ ‘framing’ of the
N. Fulop et al. / Health Policy 63 (2003) 155 /165
issue and compare their perceptions with any available evidence. This would aid understanding of the issues in integrating services and what issues might arise in any proposed policy intervention. Policy analysis could also be used to evaluate particular policy interventions designed to improve integration of services. The study design might consist of comparative case studies or ‘natural experiments’. Where a policy was introduced universally, the comparative case study approach could be used to compare the different contexts within which the intervention had been introduced (e.g. doctor- or nurse-led, urban or rural setting). In the situation where a policy was not introduced universally, the comparative case study approach could compare areas where the policy had been introduced with those where it had not. In either case, such a study would seek to generalise the research findings not through aggregation, but rather through the development of theory.
11. Issues involved in undertaking multi-disciplinary research on SDO From this example it can be seen that many different disciplines have invaluable contributions to make to SDO research. However, there are important differences both between and within disciplines, which need to be understood if multidisciplinary research is possible. Multi-disciplinary work is easiest where researchers from the different disciplines share paradigms. Thus it is possible to imagine epidemiologists and economists sharing a broadly ‘positivist’ perspective working together to evaluate a change in the way a health service is delivered. Similarly, it is possible to imagine researchers from organisational studies and policy analysis with an interpretivist perspective working together to study the same change in delivery of a health service. The key question is whether it is possible, or indeed desirable, for researchers from different paradigms to work together to address the same research problem. As we have tried to emphasise, these perspectives are not necessarily at opposite poles */they form a range of views, along a
163
continuum. So, for example, it is possible to imagine a ‘subtle realist’ using qualitative methods to study a change in service delivery which has uniquely occurred in one organisation, while an epidemiologist uses this ‘natural experiment’ to compare this organisation with another one. It is also important to recognise that it will be not always be possible to reconcile positions within each perspective. It is more difficult to imagine an epidemiologist working alongside a researcher from a post-modernist perspective within organisational studies. Given both the desirability of researchers from different disciplines and perspectives to work together and that in many instances it should be feasible, how can multi-disciplinary research be encouraged? There is clearly a role for research funding bodies to provide encouragement and even incentives for such multi-disciplinary research by, at the very least, requiring evidence of it in applications for research grants. Members of the research community themselves also need to take responsibility for thinking outside their own paradigms or worldviews so that they can work with others. As we have argued, it is vital that research, such as that described above on telemedicine, takes account of the role of the context within which the organisation and delivery of health services takes place, as well as their underlying processes . This is probably even more important for SDO research than for HTA. Therefore, researchers who have traditionally taken a ‘black box’ approach focussing on inputs and outputs need to work with researchers who study context and processes, and vice-versa [25].
12. Conclusions HTA research is clearly vital but it is important to complement it by a focus of research efforts on the organisation and delivery of health services. The latter has recently started to receive more funding and attention. This is important if the balance between HTA and SDO research is to be corrected. The benefits of SDO research are potentially just as great as those of HTA research. Research on the organisation and delivery of
164
N. Fulop et al. / Health Policy 63 (2003) 155 /165
health services requires a wide array of disciplines and methods if these complex questions are going to be appropriately addressed. It is a challenge, therefore, for researchers from these disciplines to see how far they can work together to carry out research in this important field. There is a role for research funding bodies to encourage researchers from different disciplines and perspectives to work together. A further challenge for research on the organisation and delivery of health services is one shared with other applied fields of policy research*/how far can this research influence policy and practice? The real test for SDO research is that health care professionals, managers and users value its outputs. This complex question of the implementation of research findings is one which also concerns those involved in HTA research. Implementation issues for SDO research are particularly challenging because of the complexity of the interventions and topics researched, the methodological and paradigmatic differences between researchers (which mean that issues of generalisability are highly contested), and the wide range of stakeholders involved. These issues alongside the rich mix of disciplines and methods which can be brought to bear on SDO research questions mean that the expanding world of SDO research promises to be an exciting one.
[4] [5] [6]
[7]
[8]
[9]
[10]
[11]
[12]
[13] [14]
Acknowledgements We are grateful to two anonymous referees for their comments on an earlier version of this paper. NJF, PA and AC are funded by the NHS Service Delivery and Organisation R&D programme.
[15]
[16]
References [1] Bowling A. Research methods in health: investigating health and health services. Buckingham: Open University Press, 1997. [2] Black N. Health services research: saviour or chimera? Lancet 1997;349:1834 /6. [3] Cochrane, A. Effectiveness and efficiency: random reflections on health services. London: Nuffield Provincial
[17]
[18]
[19]
Hospitals Trust Illich I, 1972. Medical nemesis: limits to medicine. Harmondsworth: Penguin, 1976. McKeown T. The role of medicine, 2nd ed.. Oxford: Oxford University Press, 1979. Illich I. Medical nemesis: limits to medicine. Harmondsworth: Penguin, 1976. Sackett D, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. [editorial]. British Medical Journal 1996;312:71 /2. US Congress, Office of Technology Assessment. Identifying health technologies that work: searching for evidence */OTA-H-608. Washington DC, U.S.: Government Printing Office, 1994 Perry S, Gardner E, Thamer M. The status of health technology assessment worldwide. Results of an international survey. International Journal of Technology Assessment in Health Care 1997;j13:81 /98. Sheldon TA. It ain’t what you do but the way that you do it. Journal of Health Services Research and Policy 2001;6(1):3 /5. The Insurance Experiment Group, Newhouse JP. Free for all? Lessons from the RAND health insurance experiment. Cambridge, MA: Harvard University Press, 1993. (on behalf of the Total Purchasing National Evaluation Team), Mays N, Goodwin N, Bevan G, Wykes S. Total purchasing: a profile of national pilot projects. London: Kings Fund, 1997. Carr-Hill RA, Sheldon TA, Smith P, Martin S, Peacock S, Hardman G. Allocating resources to health authorities: development of method for small area analysis of use of inpatient services. British Medical Journal 1994;309:1046 / 9. Allen P. Contracts in the NHS internal market. Modern Law Review 1995;58:321 /42. Shortell SM, Jones RH, Rademaker AW, Gillies RR, Dranove DS, Hughes EFX, Budetti PP, Reynolds KSE, Huang C. Assessing the impact of total quality management and organisational culture on multiple outcomes of care for coronary artery bypass graft surgery patients. Medical Care 2000;38(2):207 /17. Tyrer P, Evans K, Gandhi N, Lamont A, Harrison-Read P, Johnson T. Randomised controlled trial of two models of care for discharged psychiatric patients. British Medical Journal 1998;316:106 /9. Fulop N, Hood S, Parsons S. Does the National Health Service want hospital at home? Journal of the Royal Society of Medicine 1997;90:212 /5. Victor CR, Nazareth B, Hudson M, Fulop N. Inappropriate use of acute hospital beds in an inner London health authority. Health Trends 1993;25:94 /7. Stimson G, Webb B. Going to see the Doctor. London: Routledge, 1975. Strong P. The ceremonial order of the clinic: parents, doctors and medical bureaucracies. London: Routledge, Kegan and Paul, 1979. Strong P, Robinson J. The NHS: under new management. Milton Keynes: Open University Press, 1990.
N. Fulop et al. / Health Policy 63 (2003) 155 /165 [20] Lomas J. Using ‘linkage and exchange’ to move research into policy at a Canadian foundation. Health Affairs 2000;19(3):236 /40. [21] Fulop N, Allen P. National listening exercise: report of the findings. London: NCCSDO, 2000. url: http:// www.sdo.lshtm.ac.uk. [22] Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guides to the medical literature. IX. A method for grading health care recommendations. Journal of American Medical Association 1995;274(22):1800 /4. [23] St Leger AS, Walsworth Bell JP. Change-promoting research for health services. Buckingham: Open University Press, 1999. [24] Black N. Why we need observational studies to evaluate the effectiveness of health care. British Medical Journal 1996;312:1215 /8. [25] Fulop N, Allen P, Clarke A, Black N. Issues in studying the organisation and delivery of health services. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [26] Hatch M. Organisation theory: modern symbolic and postmodern perspectives. Oxford: Oxford University Press, 1997. [27] Lyotard JF. The postmodern condition: a report on knowledge. Manchester: Manchester University Press, 1979. [28] Pawson RD, Tilley N. Realistic evaluation. London: Sage, 1997. [29] Popper KR. The logic of scientific discovery. London: Hutchinson, 1972. [30] Kuhn T. The structure of scientific revolutions. Chicago: University of Chicago Press, 1970. [31] Oakley A. Experiments in knowing. Cambridge: Polity Press, 2000. [32] Hammersley M. What’s wrong with ethnography?. London: Routledge, 1992. [33] Ovretveit J. Evaluating health interventions. Buckingham: Open University Press, 1998. [34] Guba EG, Lincoln YS. Fourth generation evaluation. Newbury Park, CA: Sage, 1989.
165
[35] Rosenhead J. Operational research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [36] Meyer J. Action research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [37] Grimshaw J. Epidemiological methods. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [38] Sculpher M. Economic evaluation. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [39] Berridge V. Historical research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [40] Murphy E. Micro-level qualitative research. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [41] Dawson D. Organisational economics. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [42] Arnold J. Organisational psychology. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [43] Ferlie E. Organisational studies. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001. [44] Harrison S. Policy analysis. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: research methods. London: Routledge, 2001.