- Email: [email protected]
A New Approach to Eliciting Patients' Preferences for Palliative Day Care: The Choice Experiment Method
Vol. 29 No. 5 May 2005
Journal of Pain and Symptom Management
435
Original Article
A New Approach to Eliciting Patients’ Preferences for Palliative Day Care: The Choice Experiment Method Hannah-Rose Douglas, PhD, Charles E. Normand, DPhil, Irene J. Higginson, PhD, and Danielle M. Goodwin, PhD London School of Hygiene and Tropical Medicine (H.-R.D.), London, United Kingdom; Trinity College (C.E.N.), Dublin, Ireland; and Department of Palliative Care and Policy (I.J.H., D.M.G.), Guy’s, King’s and St. Thomas’ Medical School, London, United Kingdom
Abstract Palliative day care (PDC) provides individualized care to meet patients’ needs and preferences and has posed problems for economic evaluation. Current methods are limited in their ability to capture relevant outcomes. The choice experiment elicits preferences for multiple aspects of care rather than a single outcome. A choice experiment was undertaken at four centers in England. A random effects probit model was used. Interaction terms relating to patient and service characteristics were explored. Seventy-nine patients participated. All characteristics of PDC except bathing and hairdressing were significant (P ⬍ 0.001). Access to specialist therapies was three times as important as medical support and twice as important as staying all day. Interaction terms were not significant, except for age and preference for specialist therapies, although the sample may not have been adequate to detect differences. Choice experiments provided useful insights by quantifying preferences for services, providing an alternative to cost-effectiveness analysis. J Pain Symptom Manage 2005;29:435–445. 쑖 2005 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved. Key Words Choice experiment, palliative care, day care, evaluation, economic
Introduction All health services present problems for evaluation. In palliative care, there are specific difficulties in applying conventional research
Address reprint requests to: Hannah-Rose Douglas, PhD, Health Services Research Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT, United Kingdom. Accepted for publication: August 25, 2004.
쑖 2005 U.S. Cancer Pain Relief Committee Published by Elsevier Inc. All rights reserved.
techniques. First, the interventions can be complex and not easy to define. Second, testing effectiveness in controlled clinical trials to provide robust evidence is made difficult by problems of randomization when the essence of the services is that they are responsive to the preferences and needs of individuals. Third, this reduces the usefulness of observational studies because different individuals (at least to some extent) receive different packages of care as a result of decisions about their health state and life circumstances. Fourth, the outcomes of palliative care interventions are 0885-3924/05/$–see front matter doi:10.1016/j.jpainsymman.2004.08.017
436
Douglas et al.
multidimensional and tend not to fit well into conventional measurement. Economic evaluation commonly presents its findings in terms of cost per life year saved or episode of morbidity averted. These are crude measures of outcome. The quality-adjustedlife-year, which has become the gold standard measure in economic evaluation, is an improvement but is still a limited measure of outcome because it is not designed to capture the subtle differences in health and well-being reflected in the goals of palliative care. Palliative care outcome measures may better capture the specific benefits of palliative care, but are not very satisfactory for economic evaluation. Measures such as the McGill Quality of Life Questionnaire1 and the Palliative Care Outcome Scale2 were not designed for economic evaluation and it is not clear what the unit of benefit would be in a cost-effectiveness analysis incorporating these measures. In this context, it may be necessary to consider other ways of assessing the value of palliative care beyond cost-effectiveness analysis. Although it is important to subject palliative care to the same evaluation rigor and skepticism as other services, the question of whether or not it works is not the only one of interest, and is very hard to answer. It is possible to provide guidance to palliative health care providers by exploring what services are or are not valued by patients, rather than focusing on outcomes only. Given the many possible service elements in palliative care, analyzing which components are valued highly and which are of less interest to users is an appropriate aim of research. However, answering this type of question requires different techniques and approaches. Recent advances in health economics have developed in response to the need for research tools that can capture the multidimensionality of health care in evaluation. These tools focus on the value of specific aspects of care to the users of those services. A choice experiment is a technique for establishing the importance of individual attributes of a good or service.3,4 Choice experiments (also referred to in the literature as conjoint analysis, stated preference, and discrete choice modeling) were developed from the environmental literature5–7 and transport literature,8–10 where economists have been concerned with the valuation of specific characteristics or attributes of
Vol. 29 No. 5 May 2005
public policies and especially in forecasting demand for new services. The approach has been used by health economists working in evaluation for more than ten years11,12 and has been applied to a wide range of health care contexts and questions, including the evaluation of the outcomes of specific medical interventions such as magnetic resonance imaging13 and liver transplants;14 the organization of services such as blood clinics,15 cardiac services,16 and gastroenterology clinics;17 off-hours primary care provision;18 location of care;19,20 and patient held records.21 It has been used to explore general practitioner’s decisions about the way they practice and are remunerated,22,23 and to consider the value of the process of obstetric care beyond the final outcome.24,25 A choice experiment elicits respondents’ preferences for different dimensions of a service. The conceptual framework appeals to health economists working on public policy issues because it is grounded in Lancaster’s microeconomic theory of demand.26 The fundamental proposition of this theory is that all goods have characteristics that are objective and finite, and satisfy human wants. Lancaster proposed that goods or services are desired because they contain certain characteristics that contribute to welfare or utility. The choice experiment constructs a series of hypothetical but realistic descriptions of a good or service described in terms of characteristics or attributes. For example, a car might be described by its make, color, and price. Respondents are shown descriptions (or ‘scenarios’) of a good or service and asked which they would choose. In the car example, one choice might be a blue hatchback at UK£15,000 and another might be an orange sports car at £17,000. The respondent weighs the two alternatives and decides which would bring the most happiness or welfare, all other things being equal. This approach has had wide applications in the world of marketing, as well as in economic evaluation.27 Scenarios combine positive attributes with negative attributes so that individuals have to make trade-offs. The choice-set (usually of two scenarios in each, but sometimes three or more) is repeated with different combinations, and the respondents’ choices are recorded. Regression techniques are used to determine which attributes contribute most/least to respondents’ choices and, consequently, to their
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
welfare, assuming that individuals are the best judge of their welfare. The value of the choice experiment approach is that, unlike more straightforward ranking or rating exercises, it requires respondents to make judgments under conditions of constrained choice—that is, it prevents respondents seeing every attribute as equally important. Respondents are required to make decisions about what they would sacrifice to obtain more of an attribute they prefer. The technique has been shown to provide quantitative, comparative measures of the contribution of specific attributes to welfare in contexts where this cannot be measured directly.3 Choice experiment techniques are attractive to those evaluating multiple attribute health services because of their ability to incorporate a range of stimuli in an individual’s decision to access a particular service. They also can provide a large amount of information from a small amount of data collection.28 In this sense they are efficient research tools. Also, they do not rely on comparative data either over time or between patient groups. They are useful where patients may be too frail to participate in longitudinal research or die before the end of the study, which is a problem in palliative care research.29–31 In many contexts, economists prefer to observe individuals’ revealed preferences, that is, the choices people make when faced with real life choices, rather than asking about their preferences in a hypothetical way. For many contexts (shopping for cars, for example), this is adequate. However, in the context of health care, these observed preferences become very difficult to interpret because other factors may have contributed to an individual’s choice. If patients participate in a specific activity, they may not prefer that activity over one that is not offered to them. They may not have access to or knowledge of the full range of options available or feel able to make a free choice between them. They may feel pressured to choose an option they would not have chosen with a completely free choice. Also, in a palliative day care (PDC) setting, some aspects of the service (such as opening times or routine access to a doctor) are not individualized to each patient. Choice experiments can provide information on what an optimal service would be for an individual, or group of patients, where
437
such a service does not yet exist in the real world.
Methods There are five main stages to a choice experiment: identifying attributes, assigning levels to each attribute, designing the choice-sets, presenting them to respondents in the form of questionnaires, and analyzing their responses using regression techniques. The statistical method used is probit analysis. This estimates the impact of each attribute on the individual’s choice. Random-effects probit models are usually recommended for statistical reasons because the data include a number of observations from the same individual.32 The results of the probit analysis show the coefficients for each attribute and their P-value. The relative size of the coefficients for each attribute is interpreted as the propensity to choose a scenario with a specific attribute in it (e.g., the color blue in the car example). The higher the positive coefficient, the more likely an individual will chose the scenario with that attribute in it. The signs on the coefficients indicate the direction of association between the presence of an attribute and the propensity to choose that scenario. If the coefficient for that attribute is not significant (for example, a P-value below the 5% or 10% significance level), the attribute level is not important to the decision. The probit analysis is presented with a “goodness of fit” statistic (in this case, the McFadden R2 statistic). This statistic is shown as a proportion (range 0 to 1) and indicates how well the model fits the data or predicts a patient’s choice of scenario. This approach is consistent with guidance published in the health economic literature on choice experiments.11,33 The first stage in a choice experiment in PDC was to define the important attributes of a center. These attributes are the ones thought to have an influence on a patient’s decision to attend. These could be access to counseling, specialist medical services, or different kinds of therapies. Attributes also include opening times, location, provision of meals, or excursions. The attributes chosen for the PDC study were selected on the basis of descriptive evidence gathered from a previous study of PDC undertaken by the authors. This was a multicenter evaluation of PDC undertaken between 1998
438
Douglas et al.
and 2000 that used qualitative and quantitative analysis.34,35 This method of defining the attributes is common in designing choice experiments.12 The attributes chosen for this study represented characteristics of PDC that differed between centers. Some of these attributes were specific activities, others related to the organization of care. Six attributes were chosen. These were: provision of hairdressing and bathing, routine access to a doctor, type of access (appointment only or drop-in), opening hours, and specialist therapy (such as massage). Each attribute represented a dimension of PDC that was ‘discrete’ (not related to the presence of
Vol. 29 No. 5 May 2005
any other attribute), was conceptually straightforward, and varied across centers. For the activities (bathing, hairdressing, specialist therapies, and routine access to a doctor), the attribute levels were either ‘present’ or ‘absent’. For the attributes that defined the way that the center worked, in-depth discussions with providers of PDC led to the attribute for opening hours of 10 AM to 3 PM, and 1PM to 6 PM. The ‘access’ attribute levels were either ‘open access all day’ or ‘by appointment only’. Figure 1 shows how the respondents were presented with the alternative scenarios. No attribute pairs were the same in any choice set (say, specialist therapies available in
“Imagine the two palliative day care centers you could attend once a week. One is center A and one is center B. Both offer arts and crafts and a friendly, social environment. But other than that they are not the same and offer different kinds of services and activities. Some of the choices may appear easy; others will be more difficult. Just tell us which, if you had to choose, you would prefer to visit?” Card 1 Center A
Center B Stay for the full session
Access
You can only come for an hour or so but not stay all day
Opening time
10 am to 3 pm
1 pm to 6 pm
You can have a bath every visit if you want to
Not available
Bathing
Massage, aromatherapy and reflexology are available
Not available
Not available
Hairdressing is available
In emergencies only
You can see a doctor every visit if you want to
Specialized therapies and activities
Hairdressing
Doctor
Which center do you prefer?
A
B
Fig. 1. An example of a choice set presented to respondents (1 of 10 cards).
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
both choices) so that respondents were forced to consider each attribute in their decision. A fractional factorial design, which is a subset of all possible combinations, was used in the experiment. Fractional factorial designs are adopted when presenting all alternatives would be too time-consuming, cost too much, or might fatigue the respondents, thereby possibly invalidating responses.36 No interactions between attributes were assumed in the first instance but were explored in sub-group analysis. A computer program (SPSS software version 10, Orthoplan) was used to generate a balanced, orthogonal study design, where the combination of attributes was mutually independent of all others, and all combinations were equally likely to occur. This approach is widely used in the choice experiment literature and has been recommended in a review of the quality of choice experiment studies in health economics.12 The minimum number of choice sets for the design was determined by Orthoplan. Eight paired choice sets were specified. This lies at the low end of the range of 8–18 choice sets that has been adopted by other choice experiments.33 The experiment was conducted in four PDC centers in southern England. All patients who had experienced PDC for at least one month were eligible if they attended the center on the day that the interviewer attended. This inclusion criterion was agreed upon in order for respondents to have some knowledge of the value of the attributes of day care they were being asked to trade off in the experiment. The impact of this inclusion criteria to the generalizability of the study is discussed below. The only exclusion criterion was if the patient was too unwell to participate in an interview, as decided by the day care leader. The specific reasons for exclusion on these grounds were not recorded for ethical reasons. The data were collected using face-to-face interviews. Individuals were approached first by the day care leader for initial verbal consent, and written consent was obtained by the interviewer once the respondent had agreed to take part. Local ethics committee approval was obtained for all four participating PDC centers.
Sample Size The stated preference literature has tended to deal with the issues of sample size rather
439
informally, with early empirical work suggesting samples of around 30 per subgroup were adequate.3 The argument was that because these methods collect multiple observations per respondent, only small sample sizes were required.37 However, because the number of multiple observations per individual provides information about that individual and not the larger population of interest in economic research, larger sample sizes are now more common. The figure of 75–100 respondents per market segment has been quoted in later unpublished studies.38,39 The latter study tested how many respondents were needed to uncover known preferences. It suggested that values could be predicted with the greatest accuracy as the sample approached 100, beyond which the marginal benefit of each additional interview falls. Given that this was a pilot study, it was not known prior to the experiment the likely distribution of responses, that is, what proportion of respondents might be expected to choose scenario A or B for any given combination of attributes. Sample size for this study was driven in the first instance by practical considerations of how many patients could be recruited with the resources available and level of attendance at each center. A target of 20 patients per center was thought to be achievable in the timescale. One of the choice experiment software companies published guidance on sampling for small populations.40 The formula for a 95% confidence interval for a finite sample is given as:
⫾1.96
√
(1⫺f )
P·Q 1⫺n
(1)
where P is the expected proportion choosing scenario A, Q is (1⫺P), n is the sample size, and f is a finite population correction expressed as the ratio of sample size to population. With a population of approximately 400 palliative care patients eligible for PDC across the four sites, and assuming a worst case scenario (largest standard error), that is, for 50% to choose scenario A and 50% to choose B, the equation is solved for a sample size of 80 patients to achieve a margin of error of ⫾5%. This indicates that the sample recruited to the study was correct for a
440
Douglas et al.
sample population of four palliative care centers. However, using this approach, and not sampling from all patients eligible for PDC in the UK, the results cannot be extrapolated beyond this small population, which is one of the limitations of the study described in the Discussion.
Analysis Plan Using a probit model, the following regression equation was estimated: Y⫽constant⫹α1access⫹α2time⫹α3bath ⫹α4specialist⫹α5hair⫹α6doctor with Y representing the chance of choosing scenario A and α1–6 representing the importance of each attribute to the decision to choose scenario A. A significant constant term indicates that respondents had a bias towards choosing scenario A or B, whatever the attributes (for example, always choosing the left-hand choice). The interactions between attributes in the choice experiment and specific covariates were examined. Age and sex were considered to be important and other covariates relating to personal circumstances, such as whether respondents lived alone were included. The association between preferences and the characteristics of the respondent were examined. Dummy variables were created in the regression model. They were: OLD (if the respondent was 75 years or older at the time of interview); YOUNG (under 65 years); MALE (if the patient was male); LIVALONE (if the patient lived alone); MEDCENTRE (if the center the respondent attended had routine medical appointments); ALLDAY (if the center encouraged patients to stay from opening until closing time). Interaction terms were created using these dummy variables. They were: OLDACCESS and YOUNGACCESS to assess whether the age of respondent had an impact on their preferences for staying for the full session; ALONEACCESS to assess whether patients who lived alone had stronger preferences for staying for the full day; ALLDAYACCESS whether respondents who attended centers that encourage people to stay all day had a stronger preference for that attribute than those attending appointment-based centers; MEDCENTRE-DR to assess whether patients who attended centers with routine medical appointments preferred this more than those who did
Vol. 29 No. 5 May 2005
not; and YOUNGTHERAPY to estimate whether younger patients had a greater preference for more active, therapeutic interventions. A regression model was estimated for each of these interaction terms consecutively. MALE was combined with all attributes to assess whether there was a systematic difference between the preferences of men and women for specific attributes of palliative day care. In addition to the eight pairs of scenarios in the experiment, the orthogonal design program also produced two ‘hold-out’ choice sets. These are identical in structure to all the other choice sets but the responses to these choicesets are not used to estimate the regression model. The hold-out approach is a way of assessing the internal validity of the model. Holdout analysis tests whether a model correctly predicts the actual choice an individual makes in a specific choice set. If the hold-out analysis correctly predicts a high proportion of the individuals’ responses to the hold-out choice sets, then the model is seen to be a good fit (given that 50% of correct predictions would be expected to occur by chance). The χ2 test analyzes the observed over expected results and the Pvalue reports the likelihood that this result would have been obtained by chance.
Results In total, 81 of approximately 123 patients attending PDC on the day the interviewer was present (66%) agreed to take part across four centers. Reasons for not obtaining an interview were either because the day care leader thought they were too unwell (5 patients), refusal to participate (2 patients), or they were not asked due to lack of opportunity between activities or because they stayed for only a short time (35 patients). The potential bias introduced by excluding this final group is explored in the Discussion. Two patients were recruited to the study but stopped the interview before making any choices. These patients were included in the demographic analysis of the sample. The interviews took between 10 and 20 minutes, including explanation and additional discussion between the interviewer and patient. The actual choice experiment instrument took 10 minutes or less, although this was not timed exactly. There was some interview fatigue reported by a small number of the frailer patients
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
who did not complete the interview once started (3 respondents). One patient chose not to make a decision for half of the pairwise choices, saying that he would not choose either option. These “choose neither” responses were not included in the analysis. All other responses were included, whether patients completed the interview or not. Of all patients who consented and were recruited to the study, 46% were male and 54% female. Mean age was 61 years (median 63 years, range 20–89 years). Thirty-eight percent (38%) of patients said they lived alone, 45% with a partner, and 14% with other family members. The mean number of months that patients had attended a PDC center was 8.13 (median 6 months, range 1 month to 9 years). Table 1 shows the results of a random effects probit model. High levels of significance (Pvalue less than 5%) indicate that the presence of an attribute was correlated with the respondents’ decisions to choose a particular scenario. The results of the probit model show that all the signs on the coefficients are as expected, with the presence of a positive attribute increasing the likelihood that a respondent would prefer the scenario with that attribute in it. The model shows that all the attributes except bathing and hairdressing had a significant impact on respondents’ choices. The most important attribute was access to specialist therapies (coefficient for specialist therapies: 0.6118), which was more than twice as important as staying all day at the center (access: 0.2832), which was
441
more important than routine access to a doctor (doctor any time: 0.1857). In this sample of patients, hairdressing and bathing did not contribute to the decision to attend PDC. This is plausible because Table 2 confirms that these two services are the least used for all patients in the study, with only 15% and 9% of patients accessing these services in a PDC center. An analysis of dominant preferences was also undertaken. Dominant preferences are those where a respondent always chooses the scenario with a specific attribute, regardless of how good the other attributes in the alternative scenario are. Case by case analysis examining each choice made by every respondent revealed a high proportion of patients who expressed a preference for one dominant attribute. Nearly half of all patients expressed dominant preferences with almost a quarter indicating a dominant preference for specialist therapies (23%). This is a higher proportion of dominant preferences than has been reported in other choice experiments in the health economics literature. This is a result in itself (that PDC patients appear to have strong preferences for one attribute, although the dominant attribute differs between patients). However, it may also be a result of the design of the experiment because respondents were only offered two options for every attribute and this may have indicated that there were more dominant preferences that actually existed due to the lack of a wider choice of attribute levels. The design may, therefore,
Table 1 Results of the Random Effects Probit Model: All Patients Including Those Manifesting Dominant Preferences (n ⴝ 79) Attributes
Coefficient
Access 0.2832 Time 0.2987 Bath 0.0811 Specialist therapies 0.6118 Hairdressing 0.0446 Doctor any time 0.1856 Constant 0.0220 No. of observations: 624 (some patients did not answer all questions) No. of respondents 79 Mean number responses 7.9 (range 4 to 8) Goodness of fit (McFadden R2) 0.206 Proportion of correct choice predictions from the hold-out analysis: Y ⫽ 0 61% Y ⫽ 1 85% χ2 142.03 (P ⬍ 0.0001)
P-value ⬍0.0001 ⬍0.0001 0.154 ⬍0.0001 0.430 0.001 ⬍0.694
95% Confidence Interval 0.1720, ⫺0.4102, ⫺0.0303, 0.5003, ⫺0.0330, 0.0755, ⫺0.0880,
0.3944 ⫺0.1872 0.1926 0.7232 0.1551 0.2957 ⫺0.5972
442
Douglas et al.
Table 2 Activities Accessed in PDC by Study Respondents (n ⴝ 79) Activity Social interaction Doctor Physiotherapy Massage Reflexology Aromatherapy Counseling Arts/crafts Hairdressing Bath
No of respondents
%
60 41 36 32 24 28 16 33 12 7
76 52 46 41 30 35 20 42 15 9
have had an impact on the results of the regression model. Therefore, a second model was estimated, excluding patients with dominant preferences, to assess whether the model produced different attribute coefficients. The magnitude of the coefficients was very similar to those reported in the first model (Table 1). The coefficients for hairdressing and bathing were not significant in either model, with the only change being the level of significance in the attribute ‘doctor any time’, which improved from 0.001 to 0.0001. The P-values of all other attributes remained below 0.0001 for both models. The confidence intervals were wider because there were fewer observations, but the overall findings did not change. The results suggest that staying at a PDC center for the full session was preferable to attending for an appointment only; 10 AM to 3 PM was preferred to a later opening time of 1–6 PM. Access to specialist therapies influenced respondents’ decision to attend a center, as did access to the doctor any time. The prediction of respondents’ choices in the hold-out scenarios (that were not used to estimate the probit model) was compared with actual choices made by respondents. Table 1 shows the percentage of correctly predicted values for the hold-out scenarios. Chi-squared tests show the likelihood that this result could have been arrived at by chance. The P-value indicates that this is highly unlikely. Interaction terms were entered into the model to explore whether specified relationships between variables were significant. Significance was explored at the 10% level because the sample size was small and interactions were not expected to show a strong relationship. Table 3 shows the results of adding interaction terms to
Vol. 29 No. 5 May 2005
the model for all respondents (including those with dominant preferences). The coefficients indicate the positive relationship between patients/service characteristic and the importance of an attribute. It indicates that only the interaction term that combined the younger age group and preference for specialist therapy (YOUNGTHERAPY) was significant at the 10% level.
Discussion The purpose of the choice experiment was to assess the value of specific components of PDC by considering patients’ preferences for these attributes. The results indicate that, in this population of PDC patients, specialist therapies were the most highly valued of the six attributes of PDC presented in the experiment. A number of the PDC centers relied on volunteers to provide these services, so this raises the question of whether resources spent elsewhere in these centers should be re-directed towards specialist therapies. There was also a strong preference for the centers to stay open all day rather than for specific appointments only. One of the centers that took part in the study operated an appointments system and discouraged patients from staying all day. In the sample of patients that were interviewed, there was no association found between sub-groups of patients with particular characteristics except that patients under 65 may have had a stronger preference for specialist therapies than older patients. This concurs with reports from staff that younger patients often take a more active approach to their illness than older patients and may reflect the expectations of younger people and how they manage their illness. A survey of 40 PDC centers showed that almost half of them offered a separate day for younger patients in recognition Table 3 Probit Model Interaction Terms Explored in the Data (All Patients, n ⴝ 79) Interaction Term Coefficient P-value Confidence Interval OLDACCESS YOUNGTHERAPY YOUNGACCESS ALONEACCESS MECENTREDR ALLDAYACCESS
0.0012 0.2970 0.1986 0.0777 0.0936 0.1143
0.993 0.008 0.074 0.449 0.407 0.448
⫺0.2674, 0.0788, ⫺0.0192, ⫺0.1477, ⫺0.1277, ⫺0.2088,
0.2650 0.5152 0.4164 0.3031 0.3149 0.4373
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
that this group may have different needs from older patients.41 It is not possible to make generalizations about patients’ preferences for PDC from this study. Only the preferences of patients who were well enough to be interviewed as judged by the PDC leader were included in the analysis. They had to have attended at least four weeks and stayed in the center long enough to be approached by an interviewer to participate on the day of interview. Patients who are very unwell or only stay a short while in the day may have different preferences from the majority of patients interviewed. The views of all users of PDC should ideally be captured in research. This study, therefore, should be seen as only a partial picture of the preferences of users and potential users of PDC services. The study was also limited by the set of attributes used in the experiment. The attributes chosen here may not necessarily be most important to patients. But they are the attributes which those who are charged with running the service have some control over. Previous research suggests that other attributes (characteristics), such as positive social atmosphere, personal safety, hope for the future, the PDC ‘community’, are highly valued parts of the PDC experience.34,35,42,43 This study does not refute this claim. However, these aspects of the PDC experience are difficult to define in ways that mean the same things to all people, and difficult for day care leaders to put into action without more clear understanding of what contributes to these attributes and how they can influence them. The structure of day care and the provision of key services influence whether patients attend, whether they contribute to a positive social atmosphere, provide individual care or contribute to welfare in other undefined ways. Why a service or characteristic is valued by patients may not be expressed in words, but patients may still have strong preferences for it. In some areas of palliative care, the improvement in objectively measurable outcomes is very difficult and an alternative could be to explore patients’ preferences for specific aspects of care. The choice experiment approach forces respondents to trade off between different preferences and puts these preferences at the center of decision making in determining which services should be offered. These decisions are currently being made by managers
443
without robust evidence about which attributes contribute to patients’ overall welfare. The choice experiment approach provides guidance to decision makers about where they might concentrate their efforts. There is debate in the PDC field about the relative importance of the various activities it offers. Choice experiments can bypass the objective measurement of health outcome and consider individuals’ preferences for particular characteristics of a service directly. For PDC research, this reasoning addresses a fundamental problem: that the measurement of outcome may not be the appropriate way to judge the value of the service to society. If, using choice experiment methods, the value (or lack of value) of specific attributes of PDC services can be established, this could be a useful contribution to knowledge, in ways that mirror realworld decision making. As the evaluation culture spreads to more services that cross the divide between health and social care, the results of the clinical and cost-effectiveness studies are less and less insightful to decision makers working in care settings where these paradigms cannot answer some of the most important questions. Other methods such as choice experiments may need to be incorporated into the group of acceptable methods of economic evaluation by institutions such as the National Institute for Clinical Excellence (NICE) in the UK that currently favors cost-effectiveness and cost-utility methods.44,45 The debate continues about when and how values other than final outcome should be incorporated into economic analyses. A method of evaluation that adopts a preference-based approach may be a more useful way of obtaining evidence of the value of specific aspects of a complex service.
Acknowledgments The authors would like to acknowledge the support of managers and clinical staff of the four PDC centers that took part in the study and the enthusiastic participation of the patients who were interviewed, and their carers. The study was funded by South East Regional NHS Executive R&D. We would also like to thank the anonymous reviewer of this article for constructive comments on the first draft.
444
Douglas et al.
References 1. Cohen SR, Mount BM, Bruera E, et al. Validity of the McGill Quality of Life Questionnaire in the palliative care setting: a multi-centre Canadian study demonstrating the importance of the existential domain. Palliat Med 1997;11:3–20. 2. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care 1999;8:219–227. 3. Cave M, Cunningham D, Buxton M, et al. The valuation of changes in quality in the public services. London: Her Majesty’s Treasury London, HMSO, 1993. 4. Ryan M. Using consumer preferences in health care decision making: the application of conjoint analysis. London: Office of Health Economics, 1996. 5. Cummings JE, Hughes SL, Weaver FM, et al. Cost-effectiveness of Veterans Administration hospital-based home care. A randomized clinical trial. Arch Intern Med 1990;150:1274–1280. 6. Hanneman WM. Valuing the environment through contingent valuation. J Econ Perspectives 1994;8:19–43. 7. Louviere JJ, Hensher DA, Swait JD. Stated Choice Methods: Analysis and Application. Cambridge: Cambridge University Press, 2000. 8. Bates J. Sensitivity to level of service: evidence from stated preference work. Behav Res Transport Policy 1986;13:289–305. 9. Wardman M. A comparison of revealed preference and stated preference models. J Transport Econ Policy 1988;22:71–91. 10. Kroes EP, Sheldon RJ. Stated preference methods: an introduction. J Transport Econ 1988;22:11–25. 11. Ryan M, Bate A, Eastmond CJ, Ludbrook A. Use of discrete choice experiments to elicit preferences. Qual Health Care 2001;10(Suppl 1):i55–i60. 12. Ryan M, Scott DA, Reeves C, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess 2001;5:1–186. 13. Bryan S, Buxton M, Sheldon R, Grant A. Magnetic resonance imaging for the investigation of knee injuries: an investigation of preferences. Health Econ 1998;7:595–603. 14. Ratcliffe J, Buxton M. Patients’ preferences regarding the process and outcomes of life-saving technology. An application of conjoint analysis to liver transplantation. Int J Technol Assess Health Care 1999;15:340–351. 15. van der Pol M, Cairns J. Establishing patient preferences for blood transfusion support: an application of conjoint analysis. J Health Serv Res Policy 1998;3:70–76.
Vol. 29 No. 5 May 2005
16. Jan S, Mooney G, Ryan M, et al. The use of conjoint analysis to elicit community preferences in public health research: a case study of hospital services in South Australia. Aust NZ J Public Health 2000;24:64–70. 17. Moayyedi P, Wardman M, Toner J, et al. Establishing patient preferences for gastroenterology clinic reorganization using conjoint analysis. Eur J Gastroenterol Hepatol 2002;14:429–433. 18. Morgan A, Shackley P, Pickin M, Brazier J. Quantifying patient preferences for out-of-hours primary care. J Health Serv Res Policy 2000;5:214–218. 19. Shackley P, Slack R, Michaels J. Vascular patients’ preferences for local treatment: an application of conjoint analysis. J Health Serv Res Policy 2001;6: 151–157. 20. Ryan M, McIntosh E, Dean T, Old P. Trade-offs between location and waiting times in the provision of health care: the case of elective surgery on the Isle of Wight. J Public Health Med 2000;22:202–210. 21. Ryan M, McIntosh E, Shackley P. Using conjoint analysis to elicit the views of health service users: an application to the patient health card. Health Expect 1998;1:117–129. 22. Gosden T, Bowler I, Sutton M. How do general practitioners choose their practice? Preferences for practice and job characteristics. J Health Serv Res Policy 2000;5:208–213. 23. Scott A. Eliciting GPs’ preferences for pecuniary and non-pecuniary job characteristics. J Health Econ 2001;20:329–347. 24. Ryan M, Hughes J. Using conjoint analysis to assess women’s preferences for miscarriage management. Health Econ 1997;6:261–273. 25. Ryan M. Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilization. Soc Sci Med 1999;48:535–546. 26. Lancaster K. Consumer demand: a new approach. London: Columbia University Press, 1971. 27. Green PE, Srinivasan V. Conjoint analysis in marketing: new developments with implications for research and practice. J Marketing 1990;14:3–19. 28. Cattin P, Wittink DR. Commercial use of conjoint analysis: a survey. J Market Res 1982;46:44–53. 29. McWhinney IR, Bass MJ, Donner A. Evaluation of a palliative care service: problems and pitfalls. BMJ 1994;309:1340–1342. 30. Jarvis H, Burge FI, Scott CA. Evaluating a palliative care program: methodology and limitations. J Palliat Care 1996;12:23–33. 31. Grande GE, Todd CJ, Barclay SI, Farquhar MC. A randomized controlled trial of a hospital at home service for the terminally ill. Palliat Med 2000;14:375–385. 32. Jones A. Applied econometrics for health economists—a practical guide. London: Office of Health Economics, 2001.
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
33. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy 2003;2:55–64. 34. Douglas HR, Higginson IJ, Myers K, Normand C. Assessing structure, process and outcome in palliative day care: a pilot study for a multicentre trial. Health Soc Care Community 2000;8:336–344. 35. Goodwin DM, Higginson IJ, Myers K, et al. What is palliative day care? A patient perspective of five UK services. Support Care Cancer 2002;10:556–562. 36. Kanninen B. Optimal design of choice experiments for non-market valuation. In: Carlin A, ed. Stated preference: what do we know? Where do we go? Washington, DC: Environmental Policy and Economics Workshop Series, 2002. 37. Green PE, Wind Y. New ways of measuring consumer judgement. Harvard Business Rev 1975; 53:107–117. 38. Bradley MA, Kroes EP. Forecasting issues in stated preference survey research, 1990. Unpublished paper quoted in Cave M, Cunningham D, Buxton M, et al. The valuation of changes in quality in the public services. London: Her Majesty’s Treasury London, HMSO, 1993.
445
39. Swanson J, Pearmain D, Loughead K. Stated preference sample sizes, 1993. Unpublished paper quoted in Cave M, Cunningham D, Buxton M, et al., The valuation of changes in quality in the public services. London: Her Majesty’s Treasury London, HMSO, 1993. 40. Orme B. Sample size issues for conjoint analysis studies. Sawtooth Software research paper series. Washington, DC: Sawtooth Software Incorporated, 1998. http://www.sawtoothsoftware.com/download/ techpap/samplesz.pdf 41. Higginson IJ, Hearn J, Myers K, Naysmith A. Palliative day care: what do services do? Palliat Med 2000;14:277–286. 42. Goodwin DM, Higginson IJ, Myers K, et al. Effectiveness of palliative day care in improving pain, symptom control, and quality of life. J Pain Symptom Manage 2003;25:202–212. 43. Fisher RA, McDaid P. Palliative day care. London: Arnold Publishing, 1996. 44. Birch S, Gafni A. On being NICE in the UK: guidelines for technology appraisal for the NHS in England and Wales. Health Econ 2002;11:185–191. 45. Ryan M. Discrete choice experiments in health care. BMJ 2004;328:360–361.
Journal of Pain and Symptom Management
435
Original Article
A New Approach to Eliciting Patients’ Preferences for Palliative Day Care: The Choice Experiment Method Hannah-Rose Douglas, PhD, Charles E. Normand, DPhil, Irene J. Higginson, PhD, and Danielle M. Goodwin, PhD London School of Hygiene and Tropical Medicine (H.-R.D.), London, United Kingdom; Trinity College (C.E.N.), Dublin, Ireland; and Department of Palliative Care and Policy (I.J.H., D.M.G.), Guy’s, King’s and St. Thomas’ Medical School, London, United Kingdom
Abstract Palliative day care (PDC) provides individualized care to meet patients’ needs and preferences and has posed problems for economic evaluation. Current methods are limited in their ability to capture relevant outcomes. The choice experiment elicits preferences for multiple aspects of care rather than a single outcome. A choice experiment was undertaken at four centers in England. A random effects probit model was used. Interaction terms relating to patient and service characteristics were explored. Seventy-nine patients participated. All characteristics of PDC except bathing and hairdressing were significant (P ⬍ 0.001). Access to specialist therapies was three times as important as medical support and twice as important as staying all day. Interaction terms were not significant, except for age and preference for specialist therapies, although the sample may not have been adequate to detect differences. Choice experiments provided useful insights by quantifying preferences for services, providing an alternative to cost-effectiveness analysis. J Pain Symptom Manage 2005;29:435–445. 쑖 2005 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved. Key Words Choice experiment, palliative care, day care, evaluation, economic
Introduction All health services present problems for evaluation. In palliative care, there are specific difficulties in applying conventional research
Address reprint requests to: Hannah-Rose Douglas, PhD, Health Services Research Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT, United Kingdom. Accepted for publication: August 25, 2004.
쑖 2005 U.S. Cancer Pain Relief Committee Published by Elsevier Inc. All rights reserved.
techniques. First, the interventions can be complex and not easy to define. Second, testing effectiveness in controlled clinical trials to provide robust evidence is made difficult by problems of randomization when the essence of the services is that they are responsive to the preferences and needs of individuals. Third, this reduces the usefulness of observational studies because different individuals (at least to some extent) receive different packages of care as a result of decisions about their health state and life circumstances. Fourth, the outcomes of palliative care interventions are 0885-3924/05/$–see front matter doi:10.1016/j.jpainsymman.2004.08.017
436
Douglas et al.
multidimensional and tend not to fit well into conventional measurement. Economic evaluation commonly presents its findings in terms of cost per life year saved or episode of morbidity averted. These are crude measures of outcome. The quality-adjustedlife-year, which has become the gold standard measure in economic evaluation, is an improvement but is still a limited measure of outcome because it is not designed to capture the subtle differences in health and well-being reflected in the goals of palliative care. Palliative care outcome measures may better capture the specific benefits of palliative care, but are not very satisfactory for economic evaluation. Measures such as the McGill Quality of Life Questionnaire1 and the Palliative Care Outcome Scale2 were not designed for economic evaluation and it is not clear what the unit of benefit would be in a cost-effectiveness analysis incorporating these measures. In this context, it may be necessary to consider other ways of assessing the value of palliative care beyond cost-effectiveness analysis. Although it is important to subject palliative care to the same evaluation rigor and skepticism as other services, the question of whether or not it works is not the only one of interest, and is very hard to answer. It is possible to provide guidance to palliative health care providers by exploring what services are or are not valued by patients, rather than focusing on outcomes only. Given the many possible service elements in palliative care, analyzing which components are valued highly and which are of less interest to users is an appropriate aim of research. However, answering this type of question requires different techniques and approaches. Recent advances in health economics have developed in response to the need for research tools that can capture the multidimensionality of health care in evaluation. These tools focus on the value of specific aspects of care to the users of those services. A choice experiment is a technique for establishing the importance of individual attributes of a good or service.3,4 Choice experiments (also referred to in the literature as conjoint analysis, stated preference, and discrete choice modeling) were developed from the environmental literature5–7 and transport literature,8–10 where economists have been concerned with the valuation of specific characteristics or attributes of
Vol. 29 No. 5 May 2005
public policies and especially in forecasting demand for new services. The approach has been used by health economists working in evaluation for more than ten years11,12 and has been applied to a wide range of health care contexts and questions, including the evaluation of the outcomes of specific medical interventions such as magnetic resonance imaging13 and liver transplants;14 the organization of services such as blood clinics,15 cardiac services,16 and gastroenterology clinics;17 off-hours primary care provision;18 location of care;19,20 and patient held records.21 It has been used to explore general practitioner’s decisions about the way they practice and are remunerated,22,23 and to consider the value of the process of obstetric care beyond the final outcome.24,25 A choice experiment elicits respondents’ preferences for different dimensions of a service. The conceptual framework appeals to health economists working on public policy issues because it is grounded in Lancaster’s microeconomic theory of demand.26 The fundamental proposition of this theory is that all goods have characteristics that are objective and finite, and satisfy human wants. Lancaster proposed that goods or services are desired because they contain certain characteristics that contribute to welfare or utility. The choice experiment constructs a series of hypothetical but realistic descriptions of a good or service described in terms of characteristics or attributes. For example, a car might be described by its make, color, and price. Respondents are shown descriptions (or ‘scenarios’) of a good or service and asked which they would choose. In the car example, one choice might be a blue hatchback at UK£15,000 and another might be an orange sports car at £17,000. The respondent weighs the two alternatives and decides which would bring the most happiness or welfare, all other things being equal. This approach has had wide applications in the world of marketing, as well as in economic evaluation.27 Scenarios combine positive attributes with negative attributes so that individuals have to make trade-offs. The choice-set (usually of two scenarios in each, but sometimes three or more) is repeated with different combinations, and the respondents’ choices are recorded. Regression techniques are used to determine which attributes contribute most/least to respondents’ choices and, consequently, to their
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
welfare, assuming that individuals are the best judge of their welfare. The value of the choice experiment approach is that, unlike more straightforward ranking or rating exercises, it requires respondents to make judgments under conditions of constrained choice—that is, it prevents respondents seeing every attribute as equally important. Respondents are required to make decisions about what they would sacrifice to obtain more of an attribute they prefer. The technique has been shown to provide quantitative, comparative measures of the contribution of specific attributes to welfare in contexts where this cannot be measured directly.3 Choice experiment techniques are attractive to those evaluating multiple attribute health services because of their ability to incorporate a range of stimuli in an individual’s decision to access a particular service. They also can provide a large amount of information from a small amount of data collection.28 In this sense they are efficient research tools. Also, they do not rely on comparative data either over time or between patient groups. They are useful where patients may be too frail to participate in longitudinal research or die before the end of the study, which is a problem in palliative care research.29–31 In many contexts, economists prefer to observe individuals’ revealed preferences, that is, the choices people make when faced with real life choices, rather than asking about their preferences in a hypothetical way. For many contexts (shopping for cars, for example), this is adequate. However, in the context of health care, these observed preferences become very difficult to interpret because other factors may have contributed to an individual’s choice. If patients participate in a specific activity, they may not prefer that activity over one that is not offered to them. They may not have access to or knowledge of the full range of options available or feel able to make a free choice between them. They may feel pressured to choose an option they would not have chosen with a completely free choice. Also, in a palliative day care (PDC) setting, some aspects of the service (such as opening times or routine access to a doctor) are not individualized to each patient. Choice experiments can provide information on what an optimal service would be for an individual, or group of patients, where
437
such a service does not yet exist in the real world.
Methods There are five main stages to a choice experiment: identifying attributes, assigning levels to each attribute, designing the choice-sets, presenting them to respondents in the form of questionnaires, and analyzing their responses using regression techniques. The statistical method used is probit analysis. This estimates the impact of each attribute on the individual’s choice. Random-effects probit models are usually recommended for statistical reasons because the data include a number of observations from the same individual.32 The results of the probit analysis show the coefficients for each attribute and their P-value. The relative size of the coefficients for each attribute is interpreted as the propensity to choose a scenario with a specific attribute in it (e.g., the color blue in the car example). The higher the positive coefficient, the more likely an individual will chose the scenario with that attribute in it. The signs on the coefficients indicate the direction of association between the presence of an attribute and the propensity to choose that scenario. If the coefficient for that attribute is not significant (for example, a P-value below the 5% or 10% significance level), the attribute level is not important to the decision. The probit analysis is presented with a “goodness of fit” statistic (in this case, the McFadden R2 statistic). This statistic is shown as a proportion (range 0 to 1) and indicates how well the model fits the data or predicts a patient’s choice of scenario. This approach is consistent with guidance published in the health economic literature on choice experiments.11,33 The first stage in a choice experiment in PDC was to define the important attributes of a center. These attributes are the ones thought to have an influence on a patient’s decision to attend. These could be access to counseling, specialist medical services, or different kinds of therapies. Attributes also include opening times, location, provision of meals, or excursions. The attributes chosen for the PDC study were selected on the basis of descriptive evidence gathered from a previous study of PDC undertaken by the authors. This was a multicenter evaluation of PDC undertaken between 1998
438
Douglas et al.
and 2000 that used qualitative and quantitative analysis.34,35 This method of defining the attributes is common in designing choice experiments.12 The attributes chosen for this study represented characteristics of PDC that differed between centers. Some of these attributes were specific activities, others related to the organization of care. Six attributes were chosen. These were: provision of hairdressing and bathing, routine access to a doctor, type of access (appointment only or drop-in), opening hours, and specialist therapy (such as massage). Each attribute represented a dimension of PDC that was ‘discrete’ (not related to the presence of
Vol. 29 No. 5 May 2005
any other attribute), was conceptually straightforward, and varied across centers. For the activities (bathing, hairdressing, specialist therapies, and routine access to a doctor), the attribute levels were either ‘present’ or ‘absent’. For the attributes that defined the way that the center worked, in-depth discussions with providers of PDC led to the attribute for opening hours of 10 AM to 3 PM, and 1PM to 6 PM. The ‘access’ attribute levels were either ‘open access all day’ or ‘by appointment only’. Figure 1 shows how the respondents were presented with the alternative scenarios. No attribute pairs were the same in any choice set (say, specialist therapies available in
“Imagine the two palliative day care centers you could attend once a week. One is center A and one is center B. Both offer arts and crafts and a friendly, social environment. But other than that they are not the same and offer different kinds of services and activities. Some of the choices may appear easy; others will be more difficult. Just tell us which, if you had to choose, you would prefer to visit?” Card 1 Center A
Center B Stay for the full session
Access
You can only come for an hour or so but not stay all day
Opening time
10 am to 3 pm
1 pm to 6 pm
You can have a bath every visit if you want to
Not available
Bathing
Massage, aromatherapy and reflexology are available
Not available
Not available
Hairdressing is available
In emergencies only
You can see a doctor every visit if you want to
Specialized therapies and activities
Hairdressing
Doctor
Which center do you prefer?
A
B
Fig. 1. An example of a choice set presented to respondents (1 of 10 cards).
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
both choices) so that respondents were forced to consider each attribute in their decision. A fractional factorial design, which is a subset of all possible combinations, was used in the experiment. Fractional factorial designs are adopted when presenting all alternatives would be too time-consuming, cost too much, or might fatigue the respondents, thereby possibly invalidating responses.36 No interactions between attributes were assumed in the first instance but were explored in sub-group analysis. A computer program (SPSS software version 10, Orthoplan) was used to generate a balanced, orthogonal study design, where the combination of attributes was mutually independent of all others, and all combinations were equally likely to occur. This approach is widely used in the choice experiment literature and has been recommended in a review of the quality of choice experiment studies in health economics.12 The minimum number of choice sets for the design was determined by Orthoplan. Eight paired choice sets were specified. This lies at the low end of the range of 8–18 choice sets that has been adopted by other choice experiments.33 The experiment was conducted in four PDC centers in southern England. All patients who had experienced PDC for at least one month were eligible if they attended the center on the day that the interviewer attended. This inclusion criterion was agreed upon in order for respondents to have some knowledge of the value of the attributes of day care they were being asked to trade off in the experiment. The impact of this inclusion criteria to the generalizability of the study is discussed below. The only exclusion criterion was if the patient was too unwell to participate in an interview, as decided by the day care leader. The specific reasons for exclusion on these grounds were not recorded for ethical reasons. The data were collected using face-to-face interviews. Individuals were approached first by the day care leader for initial verbal consent, and written consent was obtained by the interviewer once the respondent had agreed to take part. Local ethics committee approval was obtained for all four participating PDC centers.
Sample Size The stated preference literature has tended to deal with the issues of sample size rather
439
informally, with early empirical work suggesting samples of around 30 per subgroup were adequate.3 The argument was that because these methods collect multiple observations per respondent, only small sample sizes were required.37 However, because the number of multiple observations per individual provides information about that individual and not the larger population of interest in economic research, larger sample sizes are now more common. The figure of 75–100 respondents per market segment has been quoted in later unpublished studies.38,39 The latter study tested how many respondents were needed to uncover known preferences. It suggested that values could be predicted with the greatest accuracy as the sample approached 100, beyond which the marginal benefit of each additional interview falls. Given that this was a pilot study, it was not known prior to the experiment the likely distribution of responses, that is, what proportion of respondents might be expected to choose scenario A or B for any given combination of attributes. Sample size for this study was driven in the first instance by practical considerations of how many patients could be recruited with the resources available and level of attendance at each center. A target of 20 patients per center was thought to be achievable in the timescale. One of the choice experiment software companies published guidance on sampling for small populations.40 The formula for a 95% confidence interval for a finite sample is given as:
⫾1.96
√
(1⫺f )
P·Q 1⫺n
(1)
where P is the expected proportion choosing scenario A, Q is (1⫺P), n is the sample size, and f is a finite population correction expressed as the ratio of sample size to population. With a population of approximately 400 palliative care patients eligible for PDC across the four sites, and assuming a worst case scenario (largest standard error), that is, for 50% to choose scenario A and 50% to choose B, the equation is solved for a sample size of 80 patients to achieve a margin of error of ⫾5%. This indicates that the sample recruited to the study was correct for a
440
Douglas et al.
sample population of four palliative care centers. However, using this approach, and not sampling from all patients eligible for PDC in the UK, the results cannot be extrapolated beyond this small population, which is one of the limitations of the study described in the Discussion.
Analysis Plan Using a probit model, the following regression equation was estimated: Y⫽constant⫹α1access⫹α2time⫹α3bath ⫹α4specialist⫹α5hair⫹α6doctor with Y representing the chance of choosing scenario A and α1–6 representing the importance of each attribute to the decision to choose scenario A. A significant constant term indicates that respondents had a bias towards choosing scenario A or B, whatever the attributes (for example, always choosing the left-hand choice). The interactions between attributes in the choice experiment and specific covariates were examined. Age and sex were considered to be important and other covariates relating to personal circumstances, such as whether respondents lived alone were included. The association between preferences and the characteristics of the respondent were examined. Dummy variables were created in the regression model. They were: OLD (if the respondent was 75 years or older at the time of interview); YOUNG (under 65 years); MALE (if the patient was male); LIVALONE (if the patient lived alone); MEDCENTRE (if the center the respondent attended had routine medical appointments); ALLDAY (if the center encouraged patients to stay from opening until closing time). Interaction terms were created using these dummy variables. They were: OLDACCESS and YOUNGACCESS to assess whether the age of respondent had an impact on their preferences for staying for the full session; ALONEACCESS to assess whether patients who lived alone had stronger preferences for staying for the full day; ALLDAYACCESS whether respondents who attended centers that encourage people to stay all day had a stronger preference for that attribute than those attending appointment-based centers; MEDCENTRE-DR to assess whether patients who attended centers with routine medical appointments preferred this more than those who did
Vol. 29 No. 5 May 2005
not; and YOUNGTHERAPY to estimate whether younger patients had a greater preference for more active, therapeutic interventions. A regression model was estimated for each of these interaction terms consecutively. MALE was combined with all attributes to assess whether there was a systematic difference between the preferences of men and women for specific attributes of palliative day care. In addition to the eight pairs of scenarios in the experiment, the orthogonal design program also produced two ‘hold-out’ choice sets. These are identical in structure to all the other choice sets but the responses to these choicesets are not used to estimate the regression model. The hold-out approach is a way of assessing the internal validity of the model. Holdout analysis tests whether a model correctly predicts the actual choice an individual makes in a specific choice set. If the hold-out analysis correctly predicts a high proportion of the individuals’ responses to the hold-out choice sets, then the model is seen to be a good fit (given that 50% of correct predictions would be expected to occur by chance). The χ2 test analyzes the observed over expected results and the Pvalue reports the likelihood that this result would have been obtained by chance.
Results In total, 81 of approximately 123 patients attending PDC on the day the interviewer was present (66%) agreed to take part across four centers. Reasons for not obtaining an interview were either because the day care leader thought they were too unwell (5 patients), refusal to participate (2 patients), or they were not asked due to lack of opportunity between activities or because they stayed for only a short time (35 patients). The potential bias introduced by excluding this final group is explored in the Discussion. Two patients were recruited to the study but stopped the interview before making any choices. These patients were included in the demographic analysis of the sample. The interviews took between 10 and 20 minutes, including explanation and additional discussion between the interviewer and patient. The actual choice experiment instrument took 10 minutes or less, although this was not timed exactly. There was some interview fatigue reported by a small number of the frailer patients
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
who did not complete the interview once started (3 respondents). One patient chose not to make a decision for half of the pairwise choices, saying that he would not choose either option. These “choose neither” responses were not included in the analysis. All other responses were included, whether patients completed the interview or not. Of all patients who consented and were recruited to the study, 46% were male and 54% female. Mean age was 61 years (median 63 years, range 20–89 years). Thirty-eight percent (38%) of patients said they lived alone, 45% with a partner, and 14% with other family members. The mean number of months that patients had attended a PDC center was 8.13 (median 6 months, range 1 month to 9 years). Table 1 shows the results of a random effects probit model. High levels of significance (Pvalue less than 5%) indicate that the presence of an attribute was correlated with the respondents’ decisions to choose a particular scenario. The results of the probit model show that all the signs on the coefficients are as expected, with the presence of a positive attribute increasing the likelihood that a respondent would prefer the scenario with that attribute in it. The model shows that all the attributes except bathing and hairdressing had a significant impact on respondents’ choices. The most important attribute was access to specialist therapies (coefficient for specialist therapies: 0.6118), which was more than twice as important as staying all day at the center (access: 0.2832), which was
441
more important than routine access to a doctor (doctor any time: 0.1857). In this sample of patients, hairdressing and bathing did not contribute to the decision to attend PDC. This is plausible because Table 2 confirms that these two services are the least used for all patients in the study, with only 15% and 9% of patients accessing these services in a PDC center. An analysis of dominant preferences was also undertaken. Dominant preferences are those where a respondent always chooses the scenario with a specific attribute, regardless of how good the other attributes in the alternative scenario are. Case by case analysis examining each choice made by every respondent revealed a high proportion of patients who expressed a preference for one dominant attribute. Nearly half of all patients expressed dominant preferences with almost a quarter indicating a dominant preference for specialist therapies (23%). This is a higher proportion of dominant preferences than has been reported in other choice experiments in the health economics literature. This is a result in itself (that PDC patients appear to have strong preferences for one attribute, although the dominant attribute differs between patients). However, it may also be a result of the design of the experiment because respondents were only offered two options for every attribute and this may have indicated that there were more dominant preferences that actually existed due to the lack of a wider choice of attribute levels. The design may, therefore,
Table 1 Results of the Random Effects Probit Model: All Patients Including Those Manifesting Dominant Preferences (n ⴝ 79) Attributes
Coefficient
Access 0.2832 Time 0.2987 Bath 0.0811 Specialist therapies 0.6118 Hairdressing 0.0446 Doctor any time 0.1856 Constant 0.0220 No. of observations: 624 (some patients did not answer all questions) No. of respondents 79 Mean number responses 7.9 (range 4 to 8) Goodness of fit (McFadden R2) 0.206 Proportion of correct choice predictions from the hold-out analysis: Y ⫽ 0 61% Y ⫽ 1 85% χ2 142.03 (P ⬍ 0.0001)
P-value ⬍0.0001 ⬍0.0001 0.154 ⬍0.0001 0.430 0.001 ⬍0.694
95% Confidence Interval 0.1720, ⫺0.4102, ⫺0.0303, 0.5003, ⫺0.0330, 0.0755, ⫺0.0880,
0.3944 ⫺0.1872 0.1926 0.7232 0.1551 0.2957 ⫺0.5972
442
Douglas et al.
Table 2 Activities Accessed in PDC by Study Respondents (n ⴝ 79) Activity Social interaction Doctor Physiotherapy Massage Reflexology Aromatherapy Counseling Arts/crafts Hairdressing Bath
No of respondents
%
60 41 36 32 24 28 16 33 12 7
76 52 46 41 30 35 20 42 15 9
have had an impact on the results of the regression model. Therefore, a second model was estimated, excluding patients with dominant preferences, to assess whether the model produced different attribute coefficients. The magnitude of the coefficients was very similar to those reported in the first model (Table 1). The coefficients for hairdressing and bathing were not significant in either model, with the only change being the level of significance in the attribute ‘doctor any time’, which improved from 0.001 to 0.0001. The P-values of all other attributes remained below 0.0001 for both models. The confidence intervals were wider because there were fewer observations, but the overall findings did not change. The results suggest that staying at a PDC center for the full session was preferable to attending for an appointment only; 10 AM to 3 PM was preferred to a later opening time of 1–6 PM. Access to specialist therapies influenced respondents’ decision to attend a center, as did access to the doctor any time. The prediction of respondents’ choices in the hold-out scenarios (that were not used to estimate the probit model) was compared with actual choices made by respondents. Table 1 shows the percentage of correctly predicted values for the hold-out scenarios. Chi-squared tests show the likelihood that this result could have been arrived at by chance. The P-value indicates that this is highly unlikely. Interaction terms were entered into the model to explore whether specified relationships between variables were significant. Significance was explored at the 10% level because the sample size was small and interactions were not expected to show a strong relationship. Table 3 shows the results of adding interaction terms to
Vol. 29 No. 5 May 2005
the model for all respondents (including those with dominant preferences). The coefficients indicate the positive relationship between patients/service characteristic and the importance of an attribute. It indicates that only the interaction term that combined the younger age group and preference for specialist therapy (YOUNGTHERAPY) was significant at the 10% level.
Discussion The purpose of the choice experiment was to assess the value of specific components of PDC by considering patients’ preferences for these attributes. The results indicate that, in this population of PDC patients, specialist therapies were the most highly valued of the six attributes of PDC presented in the experiment. A number of the PDC centers relied on volunteers to provide these services, so this raises the question of whether resources spent elsewhere in these centers should be re-directed towards specialist therapies. There was also a strong preference for the centers to stay open all day rather than for specific appointments only. One of the centers that took part in the study operated an appointments system and discouraged patients from staying all day. In the sample of patients that were interviewed, there was no association found between sub-groups of patients with particular characteristics except that patients under 65 may have had a stronger preference for specialist therapies than older patients. This concurs with reports from staff that younger patients often take a more active approach to their illness than older patients and may reflect the expectations of younger people and how they manage their illness. A survey of 40 PDC centers showed that almost half of them offered a separate day for younger patients in recognition Table 3 Probit Model Interaction Terms Explored in the Data (All Patients, n ⴝ 79) Interaction Term Coefficient P-value Confidence Interval OLDACCESS YOUNGTHERAPY YOUNGACCESS ALONEACCESS MECENTREDR ALLDAYACCESS
0.0012 0.2970 0.1986 0.0777 0.0936 0.1143
0.993 0.008 0.074 0.449 0.407 0.448
⫺0.2674, 0.0788, ⫺0.0192, ⫺0.1477, ⫺0.1277, ⫺0.2088,
0.2650 0.5152 0.4164 0.3031 0.3149 0.4373
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
that this group may have different needs from older patients.41 It is not possible to make generalizations about patients’ preferences for PDC from this study. Only the preferences of patients who were well enough to be interviewed as judged by the PDC leader were included in the analysis. They had to have attended at least four weeks and stayed in the center long enough to be approached by an interviewer to participate on the day of interview. Patients who are very unwell or only stay a short while in the day may have different preferences from the majority of patients interviewed. The views of all users of PDC should ideally be captured in research. This study, therefore, should be seen as only a partial picture of the preferences of users and potential users of PDC services. The study was also limited by the set of attributes used in the experiment. The attributes chosen here may not necessarily be most important to patients. But they are the attributes which those who are charged with running the service have some control over. Previous research suggests that other attributes (characteristics), such as positive social atmosphere, personal safety, hope for the future, the PDC ‘community’, are highly valued parts of the PDC experience.34,35,42,43 This study does not refute this claim. However, these aspects of the PDC experience are difficult to define in ways that mean the same things to all people, and difficult for day care leaders to put into action without more clear understanding of what contributes to these attributes and how they can influence them. The structure of day care and the provision of key services influence whether patients attend, whether they contribute to a positive social atmosphere, provide individual care or contribute to welfare in other undefined ways. Why a service or characteristic is valued by patients may not be expressed in words, but patients may still have strong preferences for it. In some areas of palliative care, the improvement in objectively measurable outcomes is very difficult and an alternative could be to explore patients’ preferences for specific aspects of care. The choice experiment approach forces respondents to trade off between different preferences and puts these preferences at the center of decision making in determining which services should be offered. These decisions are currently being made by managers
443
without robust evidence about which attributes contribute to patients’ overall welfare. The choice experiment approach provides guidance to decision makers about where they might concentrate their efforts. There is debate in the PDC field about the relative importance of the various activities it offers. Choice experiments can bypass the objective measurement of health outcome and consider individuals’ preferences for particular characteristics of a service directly. For PDC research, this reasoning addresses a fundamental problem: that the measurement of outcome may not be the appropriate way to judge the value of the service to society. If, using choice experiment methods, the value (or lack of value) of specific attributes of PDC services can be established, this could be a useful contribution to knowledge, in ways that mirror realworld decision making. As the evaluation culture spreads to more services that cross the divide between health and social care, the results of the clinical and cost-effectiveness studies are less and less insightful to decision makers working in care settings where these paradigms cannot answer some of the most important questions. Other methods such as choice experiments may need to be incorporated into the group of acceptable methods of economic evaluation by institutions such as the National Institute for Clinical Excellence (NICE) in the UK that currently favors cost-effectiveness and cost-utility methods.44,45 The debate continues about when and how values other than final outcome should be incorporated into economic analyses. A method of evaluation that adopts a preference-based approach may be a more useful way of obtaining evidence of the value of specific aspects of a complex service.
Acknowledgments The authors would like to acknowledge the support of managers and clinical staff of the four PDC centers that took part in the study and the enthusiastic participation of the patients who were interviewed, and their carers. The study was funded by South East Regional NHS Executive R&D. We would also like to thank the anonymous reviewer of this article for constructive comments on the first draft.
444
Douglas et al.
References 1. Cohen SR, Mount BM, Bruera E, et al. Validity of the McGill Quality of Life Questionnaire in the palliative care setting: a multi-centre Canadian study demonstrating the importance of the existential domain. Palliat Med 1997;11:3–20. 2. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care 1999;8:219–227. 3. Cave M, Cunningham D, Buxton M, et al. The valuation of changes in quality in the public services. London: Her Majesty’s Treasury London, HMSO, 1993. 4. Ryan M. Using consumer preferences in health care decision making: the application of conjoint analysis. London: Office of Health Economics, 1996. 5. Cummings JE, Hughes SL, Weaver FM, et al. Cost-effectiveness of Veterans Administration hospital-based home care. A randomized clinical trial. Arch Intern Med 1990;150:1274–1280. 6. Hanneman WM. Valuing the environment through contingent valuation. J Econ Perspectives 1994;8:19–43. 7. Louviere JJ, Hensher DA, Swait JD. Stated Choice Methods: Analysis and Application. Cambridge: Cambridge University Press, 2000. 8. Bates J. Sensitivity to level of service: evidence from stated preference work. Behav Res Transport Policy 1986;13:289–305. 9. Wardman M. A comparison of revealed preference and stated preference models. J Transport Econ Policy 1988;22:71–91. 10. Kroes EP, Sheldon RJ. Stated preference methods: an introduction. J Transport Econ 1988;22:11–25. 11. Ryan M, Bate A, Eastmond CJ, Ludbrook A. Use of discrete choice experiments to elicit preferences. Qual Health Care 2001;10(Suppl 1):i55–i60. 12. Ryan M, Scott DA, Reeves C, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess 2001;5:1–186. 13. Bryan S, Buxton M, Sheldon R, Grant A. Magnetic resonance imaging for the investigation of knee injuries: an investigation of preferences. Health Econ 1998;7:595–603. 14. Ratcliffe J, Buxton M. Patients’ preferences regarding the process and outcomes of life-saving technology. An application of conjoint analysis to liver transplantation. Int J Technol Assess Health Care 1999;15:340–351. 15. van der Pol M, Cairns J. Establishing patient preferences for blood transfusion support: an application of conjoint analysis. J Health Serv Res Policy 1998;3:70–76.
Vol. 29 No. 5 May 2005
16. Jan S, Mooney G, Ryan M, et al. The use of conjoint analysis to elicit community preferences in public health research: a case study of hospital services in South Australia. Aust NZ J Public Health 2000;24:64–70. 17. Moayyedi P, Wardman M, Toner J, et al. Establishing patient preferences for gastroenterology clinic reorganization using conjoint analysis. Eur J Gastroenterol Hepatol 2002;14:429–433. 18. Morgan A, Shackley P, Pickin M, Brazier J. Quantifying patient preferences for out-of-hours primary care. J Health Serv Res Policy 2000;5:214–218. 19. Shackley P, Slack R, Michaels J. Vascular patients’ preferences for local treatment: an application of conjoint analysis. J Health Serv Res Policy 2001;6: 151–157. 20. Ryan M, McIntosh E, Dean T, Old P. Trade-offs between location and waiting times in the provision of health care: the case of elective surgery on the Isle of Wight. J Public Health Med 2000;22:202–210. 21. Ryan M, McIntosh E, Shackley P. Using conjoint analysis to elicit the views of health service users: an application to the patient health card. Health Expect 1998;1:117–129. 22. Gosden T, Bowler I, Sutton M. How do general practitioners choose their practice? Preferences for practice and job characteristics. J Health Serv Res Policy 2000;5:208–213. 23. Scott A. Eliciting GPs’ preferences for pecuniary and non-pecuniary job characteristics. J Health Econ 2001;20:329–347. 24. Ryan M, Hughes J. Using conjoint analysis to assess women’s preferences for miscarriage management. Health Econ 1997;6:261–273. 25. Ryan M. Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilization. Soc Sci Med 1999;48:535–546. 26. Lancaster K. Consumer demand: a new approach. London: Columbia University Press, 1971. 27. Green PE, Srinivasan V. Conjoint analysis in marketing: new developments with implications for research and practice. J Marketing 1990;14:3–19. 28. Cattin P, Wittink DR. Commercial use of conjoint analysis: a survey. J Market Res 1982;46:44–53. 29. McWhinney IR, Bass MJ, Donner A. Evaluation of a palliative care service: problems and pitfalls. BMJ 1994;309:1340–1342. 30. Jarvis H, Burge FI, Scott CA. Evaluating a palliative care program: methodology and limitations. J Palliat Care 1996;12:23–33. 31. Grande GE, Todd CJ, Barclay SI, Farquhar MC. A randomized controlled trial of a hospital at home service for the terminally ill. Palliat Med 2000;14:375–385. 32. Jones A. Applied econometrics for health economists—a practical guide. London: Office of Health Economics, 2001.
Vol. 29 No. 5 May 2005
Choice Experiments in Palliative Day Care
33. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy 2003;2:55–64. 34. Douglas HR, Higginson IJ, Myers K, Normand C. Assessing structure, process and outcome in palliative day care: a pilot study for a multicentre trial. Health Soc Care Community 2000;8:336–344. 35. Goodwin DM, Higginson IJ, Myers K, et al. What is palliative day care? A patient perspective of five UK services. Support Care Cancer 2002;10:556–562. 36. Kanninen B. Optimal design of choice experiments for non-market valuation. In: Carlin A, ed. Stated preference: what do we know? Where do we go? Washington, DC: Environmental Policy and Economics Workshop Series, 2002. 37. Green PE, Wind Y. New ways of measuring consumer judgement. Harvard Business Rev 1975; 53:107–117. 38. Bradley MA, Kroes EP. Forecasting issues in stated preference survey research, 1990. Unpublished paper quoted in Cave M, Cunningham D, Buxton M, et al. The valuation of changes in quality in the public services. London: Her Majesty’s Treasury London, HMSO, 1993.
445
39. Swanson J, Pearmain D, Loughead K. Stated preference sample sizes, 1993. Unpublished paper quoted in Cave M, Cunningham D, Buxton M, et al., The valuation of changes in quality in the public services. London: Her Majesty’s Treasury London, HMSO, 1993. 40. Orme B. Sample size issues for conjoint analysis studies. Sawtooth Software research paper series. Washington, DC: Sawtooth Software Incorporated, 1998. http://www.sawtoothsoftware.com/download/ techpap/samplesz.pdf 41. Higginson IJ, Hearn J, Myers K, Naysmith A. Palliative day care: what do services do? Palliat Med 2000;14:277–286. 42. Goodwin DM, Higginson IJ, Myers K, et al. Effectiveness of palliative day care in improving pain, symptom control, and quality of life. J Pain Symptom Manage 2003;25:202–212. 43. Fisher RA, McDaid P. Palliative day care. London: Arnold Publishing, 1996. 44. Birch S, Gafni A. On being NICE in the UK: guidelines for technology appraisal for the NHS in England and Wales. Health Econ 2002;11:185–191. 45. Ryan M. Discrete choice experiments in health care. BMJ 2004;328:360–361.