- Email: [email protected]
Understanding Bereaved Caregiver Evaluations of the Quality of Dying and Death: An Application of Cognitive Interviewing Methodology to the Quality of Dying and Death Questionnaire
Vol. 43 No. 2 February 2012
Journal of Pain and Symptom Management
195
Original Article
Understanding Bereaved Caregiver Evaluations of the Quality of Dying and Death: An Application of Cognitive Interviewing Methodology to the Quality of Dying and Death Questionnaire Sarah Hales, MD, Lucia Gagliese, PhD, Rinat Nissim, PhD, Camilla Zimmermann, MD, PhD, and Gary Rodin, MD Princess Margaret Hospital, University Health Network, Toronto, Ontario, Canada
Abstract Context. To increase the interpretability of quality of dying and death measures, research is needed to understand potential sources of response variation. Objectives. The aim of this study was to understand how bereaved caregivers assess the quality of dying and death experience with the Quality of Dying and Death questionnaire (QODD) by exploring the cognitive processes that underlie their evaluations. Methods. Bereaved caregivers of former metastatic cancer patients were asked to take part in a cognitive interview protocol after formulating the 31 quality ratings that contribute to the total QODD score. Qualitative content analysis was applied to transcribed interviews, with a specific focus on the information retrieved, the judgment strategies used, and any difficulties participants reported. Results. Twenty-two bereaved caregivers were interviewed with the protocol. Information that formed the basis of quality ratings referred to the perspective of the patient, the caregiver, other family/friends, or a combination of perspectives. Quality rating judgment strategies were generally comparative, and the most common standards of comparison were to ‘‘a hoped for or ideal dying experience,’’ ‘‘a state before the dying phase,’’ ‘‘a state of distress/no distress,’’ or ‘‘normalcy/humanness.’’ All respondents relied on multiple perspectives and standards of comparison when answering the QODD. Conclusion. These results suggest that the quality of dying and death is a complex construct based on multiple perspectives and standards of comparison. These findings have implications for clinical care, which, if it aspires to improve how dying and death are evaluated, must ensure that the family is the unit of care and aid in preparation for the dying and death experience. J Pain Symptom
Address correspondence to: Sarah Hales, MD, Princess Margaret Hospital, University Health Network, 610 University Avenue, Room 16-747, Toronto, Ontario, Canada M5G 2M9. E-mail: [email protected] Ó 2012 U.S. Cancer Pain Relief Committee Published by Elsevier Inc. All rights reserved.
Accepted for publication: March 29, 2011.
0885-3924/$ - see front matter doi:10.1016/j.jpainsymman.2011.03.018
196
Hales et al.
Vol. 43 No. 2 February 2012
Manage 2012;43:195e204. Ó 2012 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved. Key Words Quality of death, good death, bad death, cognitive interviewing
Introduction If the ‘‘good death’’ is a goal of modern health care, research methods that go beyond enumeration of patient symptoms in the last week of life, and meaningfully capture subjective evaluation of the multiple dimensions of the quality of the dying and death experience, are required. Measures that evaluate the dying and death experience across large samples could explore determinants of better deaths and provide meaningful outcomes for interventions that aim to improve end-of-life care. However, the development, testing, and validation of measures to assess the quality of dying and death have proved challenging.1 Even after careful design and application in numerous studies, the interpretability of such measures has been questioned.2e4 Most instruments attempting to quantify the subjective evaluation of the multidimensional experience of the final days of life, such as the Quality of Dying and Death questionnaire (QODD)5dthe most reliable and valid measure of the quality of dying and death currently available1dare administered to bereaved caregivers or health care providers after the patient’s death. Although after-death assessments of this kind are the most feasible means to study the dying and death phase, as proxies for patient experience, concerns have been raised regarding the potential influence of the emotional state and frame of reference of the caregiver, motivation to recall, and the impact of the time elapsed since the death of the patient on recall of events.6 It also is recognized that numerous differences (e.g., diagnosis of the deceased, individual psychology, cultural group, and social roles of the patient and the caregiver) may influence how the quality of dying and death is constructed and, therefore, contribute to variability in dying and death evaluations.7 With respect to the QODD in particular, previous research has raised questions about the reliability and validity of respondent evaluations.
Mularski et al.8 found only moderate concordance among family member QODD scores, and Levy et al.9 found that family members and physicians tended to give more favorable ratings of intensive care unit (ICU) deaths than nurses and residents. These study authors suggested that sources of variation in the ratings of the same death could include both ‘‘noise’’ or measurement error and ‘‘real differences’’ as a result of different experiences or interpretations of these events. To enhance the interpretability of the QODD and similar quality of dying and death evaluations, research is needed to better identify potential sources of variation. More specifically, greater understanding of the information and strategies that respondents use to judge the dying and death experience would shed greater light on the meaning and significance of test scores. Cognitive interviewing is a qualitative methodology that closely examines the question and answer process. It is increasingly used to improve survey and questionnaire construction, explore potential threats to validity and reliability, and inform how measures are implemented and the way in which resulting quantitative data are interpreted.10,11 Cognitive interviewing is based on theories of survey response, such as the four-stage model by Tourangeau et al., which includes comprehension, retrieval, judgment, and response.12 There are a number of variations on the fourstage model, such as the model of qualityof-life appraisal by Rapkin and Schwartz.13 Because of the similarity in required cognitive processes, such models of quality-of-life appraisal could be adapted to quality of dying and death appraisal. The four meta-categories of their model are (1) frame of reference (i.e., experiences individuals deem relevant to their response), (2) sampling strategy (i.e., the specific experiences that are sampled), (3) standards of comparison (i.e., subjective standards by which the experience is judged), and (4) the combinatory algorithm (i.e., the
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
summative evaluation of relevant experiences and formulation of a response). Cognitive interviewing has been used previously to examine the limitations of after-death methodology and to better understand what factors bereaved caregivers take into account when they respond to a structured questionnaire regarding the symptoms of dying patients. McPherson and Addington-Hall14 used a cognitive interview protocol to study the underlying mental process of family members who provided after-death proxy ratings of patient pain, anxiety, and depression as part of the Views of Informal Carers-Evaluation of Services (VOICES) questionnaire.15 To date, no study has used cognitive interviewing with a quality of dying and death measure, such as the QODD. The aim of the present study was to understand how bereaved caregivers assess the quality of dying and death experience with the QODD by exploring the cognitive processes that underlie their evaluations. Particular attention was focused on (1) the information that provided the basis of quality-rating judgments, (2) quality-rating judgment strategies, and (3) any response difficulties encountered.
Methods Participants and Recruitment The present study was conducted as part of an ongoing longitudinal study of the quality of dying and death in patients with metastatic gastrointestinal cancer or lung cancer and the subsequent bereavement morbidity in their caregivers. Consecutive patients attending outpatient oncology clinics at Princess Margaret Hospital, a comprehensive cancer center in Toronto, Ontario, Canada, were recruited to participate. Patient deaths were subsequently identified through regular review of hospital records, obituary records, and inquiries made to patients’ family physicians. Six months after patient death, a letter was sent to the patient-identified primary caregiver informing them that they would be contacted by telephone regarding this study. The letter included a voicemail telephone number that they could call if they did not want further contact. One week after the letter was sent, the bereaved caregiver was contacted by telephone and invited to take part. If they were interested,
197
an in-person or telephone interview was scheduled for eight to 10 months after the patient’s death, at which time an interview (the QODD) and self-report questionnaire package were completed. Because deaths might have occurred in locations outside of the hospital, investigators could not rely on hospital records alone to identify patient deaths. The six-month time frame was, therefore, required to search obituary records and contact patients’ family physicians to ensure identification of the maximum number of patient deaths in the sample. A convenience sample of bereaved caregivers was asked to take part in a cognitive interview protocol while answering the QODD. This protocol made the interview more complex and was introduced only when the interview was conducted in person. These interviews were audiotaped and transcribed.
Measures Patient and caregiver demographics (age, gender, ethnicity, country of birth, level of education, and household income), specific caregiver variables (primary language of the caregiver and the nature and length of the patient-primary caregiver relationship), and patient illness variables (cancer type and length of illness) were obtained through patient chart review and caregiver self-report questionnaires. The quality of dying and death was measured with the QODD.5 This 31-item measure is administered to family members or health care providers after a patient’s death and asks for evaluation regarding events in the last week, or, if the patient was unconscious, the last month of life. It inquires about aspects of symptoms and personal care, treatment preferences, time with family and friends, whole person concerns, preparation for death, and the moment of death. For each item, respondents are asked first whether or how frequently an event occurred and then to provide a quality rating for that item with the question ‘‘How would you rate this aspect of the patient’s dying experience?’’ on a scale from 0 to 10 (with 0 representing a terrible experience and 10 representing an almost perfect experience). These 31 quality scores are averaged and multiplied by 10 to generate a QODD total score on a scale from 0 to 100. Since initial publication, the measure’s developers have reported on several shortened versions of the QODD intended
198
Hales et al.
to reduce respondent burden16 or use in the ICU setting,17 but the full 31-item QODD was used in the present study.
Cognitive Interview Protocol During the administration of the QODD interview to bereaved caregivers, participants were asked to verbalize their thought processes after answering each of the 31 quality-rating questions. This retrospective probing cognitive technique is distinct from a purely concurrent ‘‘think-aloud’’ technique, in which respondents are asked to describe their answering process as they complete the questionnaire, and has been deemed easiest for respondents in face-to-face interviews.18 Two probes were used to access the information retrieved and the judgment strategies that respondents relied on in providing quality ratings. These were (1) ‘‘What were you thinking about when you answered that question?,’’ and (2) ‘‘Why did you give it that rating?’’ If subjects expressed any difficulty providing a quality rating, they were asked, ‘‘Why is this question difficult for you to answer?’’
Analysis Qualitative content analysis was applied to transcribed interviews, with a specific focus on the information retrieved, the judgment strategies used, and any difficulties participants reported while formulating the 31 QODD quality ratings. Analysis was both inductive and deductive, beginning with fixed categories and subsequently allowing categories and codes to emerge as the analysis progressed until saturation. The initial coding scheme was informed by that used by McPherson and AddingtonHall14 in their evaluation of the VOICES questionnaire, based on the four-stage model of survey response by Tourangeau et al.12 For the purposes of this study, the foci were primarily the information respondents used as the basis of their judgments, which related most to their processes of comprehension and retrieval, and the judgment strategies that they employed, which related most to the processes of judgment and response. Response difficulties, it was understood, might be found at any stage of the answering process. The first author conducted initial analysis of the interview transcripts. To ensure valid interpretation of subject responses, comprehensiveness, and appropriateness of analysis, meetings were held with
Vol. 43 No. 2 February 2012
coinvestigators L.G, R.N, and G.R throughout the analysis process. Coded transcripts were reviewed and compared, and category descriptions were reviewed, amended, or further developed until consensus among the group was reached. The iterative process of data collection and analysis continued until saturation. Statistical analysis was used to describe sample characteristics, quality of dying and death scores, and the frequency of coded categories.
Results Sample Characteristics and QODD Scores Twenty-two bereaved caregivers evaluated 22 different patient deaths with the QODD and cognitive interview protocol. No caregivers refused participation in this protocol. Qualitative analysis was concurrent, and recruitment for this protocol was closed when saturation was reached. Characteristics of the 22 patients and bereaved caregivers and QODD scores are presented in Table 1. The mean age of the patients was 66.2 years and the majority were male, Caucasian, and Canadian born, with education beyond high school, and household income greater than C$60,000 per year. The mean age of the caregivers was 59.0 years and the majority were female, Caucasian, and Canadian born, with education beyond high school. Half of the bereaved caregiver sample had an income greater than C$60,000 per year. Most caregivers were patient spouses, and the average length of the relationship with the patient was 37.5 years. The mean QODD total score was 66.4 (on a 0e100 scale, with higher scores indicating better quality of dying and death).
Content Analysis Coding Analysis of the cognitive processes that provided the basis for evaluations when responding to the QODD focused on (1) the information retrieved, (2) the judgment strategy, and (3) any response difficulties reported. A summary of the codes and categories generated by the content analysis, with illustrative quotes, is presented in Table 2. The QODD allows for respondents to choose ‘‘I do not know’’ in response to quality-rating questions. If a quality rating was not provided by the respondent (i.e., there was no cognitive
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
199
Table 1 Sample Characteristics and QODD Scores (n ¼ 22) n (%) Patient variables Age, mean (SD), range Gender Female Male Ethnicity Caucasian Other Country of birth Canada Other Education #High school >High school Household income #$60,000/year >$60,000/year Primary cancer Gastrointestinal Lung Time from diagnosis to death, mean (SD), range
66.2 years (13.0), 34.3e88.8 8 (36.4) 14 (63.6) 21 (95.5) 1 (4.5) 14 (63.6) 8 (36.4) 6 (27.3) 16 (72.7) 6 (27.3) 16 (72.7) 14 (63.6) 8 (36.4) 39.3 months (33.3), 4.5e135.6
Caregiver variables Age, mean (SD), range Gender Female Male Ethnicity Caucasian Other Country of birth Canada Other Primary language English Other Education #High school >High school Household income #$60,000/year >$60,000/year Relationship to patient Spouse Parent Son/daughter Friend Length of relationship with patient, mean (SD), range
59.0 years (13.8), 28e80 17 (77.3) 5 (22.7) 21 (95.5) 1 (4.5) 18 (81.8) 4 (18.2) 22 (100) 0 (0) 6 (27.3) 16 (72.7) 11 (50) 11 (50) 15 1 4 2 37.5 years
Quality of dying and death QODD total score, mean (SD), range
process to analyze), this was coded as ‘‘not applicable.’’ If the respondent provided a quality rating but did not communicate what information or judgment strategy provided the basis for their quality rating, this was coded as ‘‘no information provided’’ or ‘‘no judgment strategy provided.’’ Of note, several respondents had difficulty with the cognitive interview protocol and could not verbalize their thought processes.
(68.2) (4.5) (18.2) (9.1) (15.6), 3e58
66.4 (10.3), 48.7e88.9
At least once during their interviews, 16 of 22 caregivers could not explain on what information they were basing their answers. Similarly, at least once during their interviews, 17 of 22 caregivers could not provide details of their judgment strategy.
Information Retrieved The types of information that provided the basis for the 31 QODD quality ratings were
200
Hales et al.
Vol. 43 No. 2 February 2012
Table 2 Content Analysis Coding Scheme Code Description Information retrieved Perspective of patient
Perspective of caregiver
Perspective of other family/friend
Judgment strategy Comparison to hoped for/ideal dying experience Comparison to prior to dying phase
Comparison to state of distress/no distress
Comparison to normalcy/humanness
Item reported to be not relevant Other strategy (e.g., comparison to a known past dying experience)
Response difficulty No communication with patient regarding that aspect of experience Question deemed irrelevant/not applicable
Unclear question meaning/wording
Frame of reference uncertain Difficulty using rating scale No knowledge of event
Example Quote Respondent 4, in response to probe regarding 7/10 quality rating for item that asks whether deceased appeared to have her/his pain under control: ‘‘Well, even though he was in pain a fair amount, it was important to him to remain lucid and in control.’’ Respondent 7, in response to probe regarding 9/10 quality rating for item that asks about deceased’s time with other family and friends: ‘‘Well, that was good for me. Because for me. for them coming around. it was mostly for me. And that’s what [he] didn’t understand. He’d be like, ‘What are they doing over here again?’’’ Respondent 7, in response to probe regarding 0/10 quality rating for item that asks whether deceased said goodbye to loved ones: ‘‘It would have been nice if he said to the kids. I would have liked him to say. for their self-esteem. ‘You are.you’ve been a good son.’’’ Respondent 20, in response to probe regarding 9/10 quality rating for item which asks whether anyone was present at the moment of death: ‘‘Well, she said she didn’t want to die alone. She got what she asked for. We were there all night long.’’ Respondent 2, in response to probe regarding 8/10 quality rating for item that asks whether deceased appeared to keep her/his dignity and selfrespect: ‘‘That was in his nature. This was one of the things I found, looking back. I did not find Bill to change. The real Bill came out throughout his dying period.’’ Respondent 13, in response to probe regarding 7/10 quality rating for item that asks whether deceased appeared to be worried about strain on her/ his loved ones: ‘‘I would say. I mean, obviously the less worrying he did the better, right?’’ Respondent 20, in response to probe regarding 3/10 quality rating for item which asks whether the deceased appeared to have the energy to do most of the things she/he wanted to do: ‘‘Well if she had no energy, she couldn’t do anything. So she wasn’t behaving like a human being any more.’’ Respondent 8, in response to probe regarding 5/10 quality rating for item that asks whether the deceased said goodbye to loved ones: ‘‘Well, she had done it earlier, so I think it was irrelevant at that stage, so I’d say five.’’ Respondent 9, in response to probe regarding 9/10 quality rating for item that asks whether the deceased appeared to have control over what was going on around her/him: ‘‘I would say almost perfect experience. I would have to say of all the people I’ve seen die she certainly was the most courageous person I’ve encountered. Even though she was very, very weak, she was still able to express that and be in control of it.’’ Respondent 11, following item that asks whether deceased appeared to find meaning and purpose in life: ‘‘I don’t know. I guess he’s the only one who can really tell us that. He never communicated that.’’ Respondent 15, following item that asks whether the deceased appeared to feel at peace with dying: ‘‘I couldn’t answer that question. I think he didn’t think he was dying, pretty much. Not then. I don’t think he thought that going to the hospital for a blood transfusion was going to be his demise, when in fact it was.’’ Respondent 1, following item that asks whether the deceased appeared to feel at peace with dying: ‘‘I’m not sure whether he was at peace with it, or whether his body was struggling. It’s hard for me to distinguish between the psychological aspect of it and the physiological. I know it’s all part of the one, the same big melting pot, but they are two different parts.’’ Respondent 17, following item that asks whether anyone was present at the moment of death: ‘‘It’s very hard. In my opinion poor, but maybe in his opinion not.You know what I’m saying?’’ Respondent 3, following item that asks where the deceased died: ‘‘It’s just the wording of this. You see, the thing is that you can’t say dying at all is ‘almost perfect experience.’ It’s so not. It’s the worst thing ever.’’ Respondent 10, following item that asks whether anyone was present at the moment of death: ‘‘I really don’t know, because I was there until midnight. She died about. I think 4:00 in the morning.’’
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
behaviors, communications, thoughts, feelings, or values. This information was categorized according to whether it referred to the perspective of the patient (i.e., the patient’s behavior, communication, and supposed thoughts/feelings/values), the caregiver (i.e., the caregiver’s thoughts, feelings, or values), another family member or friend (i.e., the family member or friend’s behavior, communication, and supposed thoughts/feelings/values), or a combination of perspectives. Over the QODD as a whole, all respondents relied on more than one perspective to answer the questionnaire. The most common perspective mentioned was that of the patient (the mean number of times the patients’ perspective was used over the 31-item QODD interview was 20.0, standard deviation [SD] 5.3), followed by that of the caregiver (mean 6.5, SD 4.2), and then by that of another family member or friend (mean 2.2, SD 2.3). The codes and related example quotes are provided in Table 2. All respondents used multiple perspectives (i.e., patient, caregiver, and/or other family member/friend) when providing a single rating at least once during the interview, with a range across respondents of one to nine uses of multiple perspectives. For example, in response to a probe regarding a five of 10 quality rating for the item that asks whether the deceased said goodbye to loved ones, Respondent 3 said, ‘‘I think the fact that he didn’t say goodbye to anybody wouldn’t have really bothered him, and it didn’t really bother us.’’ This response was coded as referencing all three perspectives: that of the patient, the caregiver, and other family/friends.
Judgment Strategies Quality-rating judgment strategies were generally based on comparisons. Standards of comparison were categorized by type for each of the 31 QODD quality-rating questions. The most common standards of comparison were to ‘‘a hoped for or ideal dying experience’’ (the mean number of times this strategy was used over the 31-item QODD interview was 8.8, SD 3.8), ‘‘a state before the dying phase’’ (mean 8.0, SD 4.2), ‘‘a state of distress/no distress’’ (mean 4.1, SD 5.2), and ‘‘normalcy/humanness’’ (mean 1.0, SD 1.3). Some respondents provided an item quality rating that was negative, neutral, or positive and explained their
201
rating by saying that the item was ‘‘not relevant to the dying and death experience’’ of the patient (mean 3.5, SD 3.1). Other less frequently used judgment strategies included comparison to ‘‘a known past dying experience’’ or ‘‘a modified ideal given known limitations.’’ The codes and related example quotes are provided in Table 2. All respondents relied on more than one strategy to answer the questionnaire, with a mean of 4.5 different strategies used per respondent (SD 1.0). Some respondents used multiple standards of comparison to generate a single rating, with a range of zero to eight uses of multiple standards. For example, in response to a probe regarding a nine of 10 rating for an item that asks whether the deceased appeared to be unafraid of dying, Respondent 9 said, ‘‘I think because she herself had experienced many people dying, and she had seen people who were afraid.she very much asked for a peaceful death, and I believe she had a peaceful death.’’ This judgment strategy was coded as both ‘‘comparison to a known past dying experience’’ and ‘‘comparison to a hoped for or ideal dying experience.’’
Response Difficulties When respondents had difficulty providing a quality rating for any of the 31 QODD items, they were asked why answering the question was difficult, and these responses were coded. Response difficulties were identified by 21 of 22 caregivers at least once during the interview. For 16 of 22 caregivers, this difficulty was great enough to prevent them from providing a quality rating at least once in the interview, although in most instances, caregivers still provided a rating despite indicating a response difficulty. Of those response difficulties reported, the most common reasons coded were ‘‘no communication with the patient regarding that aspect of experience,’’ followed by ‘‘question deemed irrelevant/not applicable,’’ followed by ‘‘unclear question meaning/wording.’’ Of the 31 quality-rating items in the QODD, 25 were associated by the pool of respondents with a response difficulty at least once. The item most commonly associated with difficulty was Item 25, ‘‘Did (patient) have the means to end her/his life if she/he needed to?,’’ and ‘‘How would you rate this aspect of (patient)’s dying experience?,’’ with seven of 22
202
Hales et al.
respondents having difficulty answering these questions. The most common reason given for this difficulty was ‘‘unclear question meaning/wording,’’ reported by four of seven respondents because they were unsure if they were meant to report on patient suicidality. Both Item 6, ‘‘How often did (patient) appear to feel at peace with dying?’’ and ‘‘How would you rate this aspect of (patient)’s dying experience?,’’ and Item 17, ‘‘Did (patient) appear to find meaning and purpose in her/his life?’’ and ‘‘How would you rate this aspect of (patient)’s experience?,’’ were associated with difficulty for six of 22 respondents. For both these items, the most common reason coded for the difficulty was ‘‘no communication with the patient regarding that aspect of experience’’ (four of six respondents and three of six respondents, respectively).
Discussion This study is the first to use cognitive interviewing methodology to evaluate a measure of the quality of dying and death. In answering the QODD, bereaved caregivers most often relied on the patient perspective but all also incorporated their own perspective and/or that of family and friends. This may occur because the bereaved caregiver and other family members are participant-observers in the death experience of the patient and also may have been the recipients of end-of-life care. As the disease progresses and patients become more debilitated and perhaps spend less time in a conscious state, caregivers often become more involved in the dying experience. This may include tending to the needs of the patient, communicating and liaising with health care providers, and receiving practical and emotional support to alleviate their own distress. The use of multiple perspectives by caregivers to evaluate the end-of-life experience, therefore, may reflect a summated rather than individualized proxy-for-patient approach, which matches the nature of end-oflife care delivery. In judging the quality of the dying and death with the QODD, bereaved caregivers took into account multiple standards against which the end-of-life experience could be compared. The most common standards applied
Vol. 43 No. 2 February 2012
were those of an ideal or hoped-for dying experience and the state before the dying phase, although there was variability among bereaved caregivers in their reliance on different comparative strategies. These results suggest that individual values, past experiences, or expectancies shape whether a particular death experience is judged positively or negatively. In other words, when asked to evaluate whether a death experience was ‘‘good’’ or ‘‘bad,’’ respondents must consider ‘‘for whom?’’ and ‘‘compared to what?’’ Reliance on different perspectives and standards of comparison may partly explain the reported variability in QODD scores obtained from different raters, with respect to a single patient death.8,9 Such variability in the processes that shape responses to the QODD may be regarded as threats to its reliability and validity. Indeed, individual variation in survey response is often perceived as a phenomenon that confounds interpretation of pooled quantitative results.19 It may be argued that such findings justify amendment of questionnaires to allow for more uniform responses. Alternatively, the demonstration of such variability might serve to adjust our understanding of the underlying constructs that are being measured. In this case, the reliance of bereaved caregivers on multiple perspectives and multiple comparisons in rating the quality of dying and death may reflect the multidetermined nature of this construct, which cannot be simplified by rewording or redesign of the questionnaire. This use of multiple perspectives and standards of comparison parallels findings in the field of quality-of-life evaluation.13 Quality-oflife measures have not been found to distinguish consistently among groups of raters, are often only weakly related to objective criteria of functioning, and show little convergence across measurement perspectives. It is acknowledged that quality-of-life evaluations may be informed by such personal reference points as prior functioning, lost capacities, extreme experiences, observations of others, past encounters with illness, and communication from providers. Rather than be considered to reflect measurement bias, denial, or willful distortion, however, these differences in quality-of-life appraisal are presumed to reflect individual differences and intraindividual changes in internal standards, values, and meaning of quality of
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
life. These differences, it has been argued, if well understood, could serve as potential targets for clinical intervention (i.e., helping patients to reprioritize values and priorities for more adaptive coping and adjustment).13 The individual differences in values, experiences, and expectations that influence evaluation of the quality of dying and death may similarly be potential targets for clinical intervention. Discussions with patients and family members throughout the illness course about possible future events, what to expect, and how best to prepare may help to shape how the dying and death experience is ultimately evaluated. Examination of response difficulties in providing quality of dying and death ratings was another objective of this study. We found that most caregivers reported difficulty in providing a quality rating at least once in their QODD interview and that most QODD items were associated with difficulty in at least one of the interviews. In some cases, caregivers identified a response difficulty and declined to provide a quality rating, whereas, in other cases, caregivers identified a response difficulty but, nevertheless, provided a rating. Differences in response difficulty may contribute to response variability. Although all quality ratings required respondents to assess the subjective experience of the patient, the questions associated with the most difficulty were those that asked about more internal experiences (e.g., meaning in life or peace with dying) or about aspects of the endof-life experience that might have been difficult for patients and caregivers to discuss openly (e.g., means to end life if needed or peace with dying). These findings are supported by previous research, indicating that proxies are most accurate with respect to concrete observable aspects of the patient experience but less accurate with respect to patients’ subjective states.6 Although eliminating items referencing more internal experience or sensitive topics would increase the weighting of patient experience, this would omit items considered highly relevant to the quality of dying and death construct. An alternative approach that may improve the validity of the after-death evaluation of the patient’s experience would be predeath identification by the patient of the caregiver they regard as the best and most reliable source of information.
203
Examination of other response difficulties suggests strategies that might improve the quality of some QODD item responses. For example, given that ‘‘unclear question meaning/ wording’’ was a common reason for response difficulty, providing additional explanation for some items (i.e., a definition for meaning and purpose in life) may assist some respondents in providing responses. Similarly, because ‘‘question deemed irrelevant/not applicable’’ was another common reason for response difficulty, providing respondents with the option of not answering because an item was irrelevant and/or allowing respondents to rate the relative importance of items might improve the validity of total scores. A 17-item QODD, based on research in which terminally ill patients and their associates prioritized rankings of QODD items,16 partly addresses this issue but ultimately sacrifices some comprehensiveness and cannot account completely for the individual and shifting nature of end-of-life priorities as patients and families move through the course of disease. In summary, as with other questionnaires attempting to measure constructs that are highly individual and subject to change over time, comprehensive global measures yielding summative scores are challenging to interpret when applied to large populations. There are limitations to this study. The use of cognitive interview methodology assumes that questionnaire respondents follow a logical or at least understandable process of response. It also is based on the assumption that respondents have conscious access to their cognitive processes as they answer questions and that they will report this information honestly and accurately, which may not be the case. Further, the majority of this sample had difficulty at least once in the cognitive interviewing process, which suggests that this is a challenging task for most individuals. An additional limitation is the small sample size and its homogeneity with respect to ethnicity and language, which limits generalization of the findings. Some potential sources of variability in response could not be explored with the methodology of this study and should be the focus of future research. These include error in memory or recall and the impact of time elapsed since death. Interestingly, no caregiver suggested that the inability to remember details of the
204
Hales et al.
time period in question caused response difficulty, although all interviews were conducted eight to 10 months after the patient’s death. This may suggest that events near the end of life are highly salient and, therefore, vividly recalled, although other memories may fade over time. However, this does not rule out possible retrospective bias and revision. This study focused on bereaved caregivers and how nonfamily respondents (such as nurses or physicians) make quality of dying and death judgments, and the potential influence of respondent psychological characteristics, such as distress and empathic ability, remain areas for future research using cognitive interview methodology. In conclusion, the results of the present study suggest that the quality of dying and death is a complex construct based on multiple perspectives and standards of comparison. Rather than being seen simply as errors or distortion in the measurement process, these multiple perspectives and standards may be viewed as valid targets for end-of-life care. Clinical attention to the family as the unit of care and to values, past experiences, and expectations in advance care planning may help ensure that the end of life is ultimately remembered and judged as positively as possible.
Disclosures and Acknowledgments Funding sources included Canadian Institutes of Health Research operating grant 74684 and a National Cancer Institute of Canada Post-MD Fellowship (Dr. Hales). The authors declare no conflicts of interest. The authors acknowledge the contributions of the research staff responsible for subject recruitment, data collection, and management, including Anne Rydall, MSc, Leanne Ferreira, BSc, and Andrew Walsh, MA.
References 1. Hales S, Zimmermann C, Rodin G. The quality of dying and death: a systematic review of measures. Palliat Med 2010;24:127e144. 2. Kaasa S, Loge JH. Quality of life in palliative care: principles and practice. Palliat Med 2003;17:11e20. 3. Steinhauser KE. Measuring end-of-life care outcomes prospectively. J Palliat Med 2005;8:S30eS41.
Vol. 43 No. 2 February 2012
4. Teno JM. Measuring end-of-life care outcomes retrospectively. J Palliat Med 2005;8:S42eS49. 5. Curtis JR, Patrick DL, Engelberg RA, et al. A measure of the quality of dying and death: initial validation using after-death interviews with family members. J Pain Symptom Manage 2002;24:17e30. 6. Addington-Hall J, McPherson C. After-death interviews with surrogates/bereaved family members: some issues of validity. J Pain Symptom Manage 2001;22:784e790. 7. Hales S, Zimmermann C, Rodin G. The quality of dying and death. Arch Intern Med 2008;168: 912e918. 8. Mularski R, Curtis JR, Osborne M, Engelberg RA, Ganzini L. Agreement among family members in their assessment of the quality of dying and death. J Pain Symptom Manage 2004;28:306e315. 9. Levy CR, Ely W, Payne K, et al. Quality of dying and death in two medical ICUs. Chest 2005;127: 1775e1783. 10. McColl E, Meadows K, Barofsky I. Cognitive aspects of survey methodology and quality of life assessment. Qual Life Res 2003;12:217e218. 11. Murtagh FEM, Addington-Hall JM, Higginson IJ. The value of cognitive interviewing techniques in palliative care research. Palliat Med 2007;21:87e89. 12. Tourangeau R, Rips LJ, Rasinski K. The psychology of survey response. Cambridge, UK: Cambridge University Press, 2000. 13. Rapkin BD, Schwartz CE. Toward a theoretical model of quality-of-life appraisal: implications of findings from studies of response shift. Health Qual Life Outcomes 2004;2:14e26. 14. McPherson CJ, Addington-Hall JM. Evaluating palliative care: bereaved family members’ evaluations of patients’ pain, anxiety and depression. J Pain Symptom Manage 2004;28:104e114. 15. Addington-Hall J, Walker L, Jones C, et al. A randomized controlled trial of postal versus interviewer administration of a questionnaire measuring satisfaction with, and use of, services received in the year before death. J Epidemiol Commun Health 1998;52:802e807. 16. Downey L, Engelberg RA, Curtis JR, Lafferty WE, Patrick DL. Shared priorities for the end-of-life period. J Pain Symptom Manage 2009; 37:175e188. 17. Curtis JR, Treece PD, Nielsen EL, et al. Integrating palliative and critical care: evaluation of a quality-improvement intervention. Am J Respir Crit Care Med 2008;178:269e275. 18. Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003;12: 229e238. 19. Norman G. Hi! How are you? Response shift, implicit theories and differing epistemologies. Qual Life Res 2003;12:239e249.
Journal of Pain and Symptom Management
195
Original Article
Understanding Bereaved Caregiver Evaluations of the Quality of Dying and Death: An Application of Cognitive Interviewing Methodology to the Quality of Dying and Death Questionnaire Sarah Hales, MD, Lucia Gagliese, PhD, Rinat Nissim, PhD, Camilla Zimmermann, MD, PhD, and Gary Rodin, MD Princess Margaret Hospital, University Health Network, Toronto, Ontario, Canada
Abstract Context. To increase the interpretability of quality of dying and death measures, research is needed to understand potential sources of response variation. Objectives. The aim of this study was to understand how bereaved caregivers assess the quality of dying and death experience with the Quality of Dying and Death questionnaire (QODD) by exploring the cognitive processes that underlie their evaluations. Methods. Bereaved caregivers of former metastatic cancer patients were asked to take part in a cognitive interview protocol after formulating the 31 quality ratings that contribute to the total QODD score. Qualitative content analysis was applied to transcribed interviews, with a specific focus on the information retrieved, the judgment strategies used, and any difficulties participants reported. Results. Twenty-two bereaved caregivers were interviewed with the protocol. Information that formed the basis of quality ratings referred to the perspective of the patient, the caregiver, other family/friends, or a combination of perspectives. Quality rating judgment strategies were generally comparative, and the most common standards of comparison were to ‘‘a hoped for or ideal dying experience,’’ ‘‘a state before the dying phase,’’ ‘‘a state of distress/no distress,’’ or ‘‘normalcy/humanness.’’ All respondents relied on multiple perspectives and standards of comparison when answering the QODD. Conclusion. These results suggest that the quality of dying and death is a complex construct based on multiple perspectives and standards of comparison. These findings have implications for clinical care, which, if it aspires to improve how dying and death are evaluated, must ensure that the family is the unit of care and aid in preparation for the dying and death experience. J Pain Symptom
Address correspondence to: Sarah Hales, MD, Princess Margaret Hospital, University Health Network, 610 University Avenue, Room 16-747, Toronto, Ontario, Canada M5G 2M9. E-mail: [email protected] Ó 2012 U.S. Cancer Pain Relief Committee Published by Elsevier Inc. All rights reserved.
Accepted for publication: March 29, 2011.
0885-3924/$ - see front matter doi:10.1016/j.jpainsymman.2011.03.018
196
Hales et al.
Vol. 43 No. 2 February 2012
Manage 2012;43:195e204. Ó 2012 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved. Key Words Quality of death, good death, bad death, cognitive interviewing
Introduction If the ‘‘good death’’ is a goal of modern health care, research methods that go beyond enumeration of patient symptoms in the last week of life, and meaningfully capture subjective evaluation of the multiple dimensions of the quality of the dying and death experience, are required. Measures that evaluate the dying and death experience across large samples could explore determinants of better deaths and provide meaningful outcomes for interventions that aim to improve end-of-life care. However, the development, testing, and validation of measures to assess the quality of dying and death have proved challenging.1 Even after careful design and application in numerous studies, the interpretability of such measures has been questioned.2e4 Most instruments attempting to quantify the subjective evaluation of the multidimensional experience of the final days of life, such as the Quality of Dying and Death questionnaire (QODD)5dthe most reliable and valid measure of the quality of dying and death currently available1dare administered to bereaved caregivers or health care providers after the patient’s death. Although after-death assessments of this kind are the most feasible means to study the dying and death phase, as proxies for patient experience, concerns have been raised regarding the potential influence of the emotional state and frame of reference of the caregiver, motivation to recall, and the impact of the time elapsed since the death of the patient on recall of events.6 It also is recognized that numerous differences (e.g., diagnosis of the deceased, individual psychology, cultural group, and social roles of the patient and the caregiver) may influence how the quality of dying and death is constructed and, therefore, contribute to variability in dying and death evaluations.7 With respect to the QODD in particular, previous research has raised questions about the reliability and validity of respondent evaluations.
Mularski et al.8 found only moderate concordance among family member QODD scores, and Levy et al.9 found that family members and physicians tended to give more favorable ratings of intensive care unit (ICU) deaths than nurses and residents. These study authors suggested that sources of variation in the ratings of the same death could include both ‘‘noise’’ or measurement error and ‘‘real differences’’ as a result of different experiences or interpretations of these events. To enhance the interpretability of the QODD and similar quality of dying and death evaluations, research is needed to better identify potential sources of variation. More specifically, greater understanding of the information and strategies that respondents use to judge the dying and death experience would shed greater light on the meaning and significance of test scores. Cognitive interviewing is a qualitative methodology that closely examines the question and answer process. It is increasingly used to improve survey and questionnaire construction, explore potential threats to validity and reliability, and inform how measures are implemented and the way in which resulting quantitative data are interpreted.10,11 Cognitive interviewing is based on theories of survey response, such as the four-stage model by Tourangeau et al., which includes comprehension, retrieval, judgment, and response.12 There are a number of variations on the fourstage model, such as the model of qualityof-life appraisal by Rapkin and Schwartz.13 Because of the similarity in required cognitive processes, such models of quality-of-life appraisal could be adapted to quality of dying and death appraisal. The four meta-categories of their model are (1) frame of reference (i.e., experiences individuals deem relevant to their response), (2) sampling strategy (i.e., the specific experiences that are sampled), (3) standards of comparison (i.e., subjective standards by which the experience is judged), and (4) the combinatory algorithm (i.e., the
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
summative evaluation of relevant experiences and formulation of a response). Cognitive interviewing has been used previously to examine the limitations of after-death methodology and to better understand what factors bereaved caregivers take into account when they respond to a structured questionnaire regarding the symptoms of dying patients. McPherson and Addington-Hall14 used a cognitive interview protocol to study the underlying mental process of family members who provided after-death proxy ratings of patient pain, anxiety, and depression as part of the Views of Informal Carers-Evaluation of Services (VOICES) questionnaire.15 To date, no study has used cognitive interviewing with a quality of dying and death measure, such as the QODD. The aim of the present study was to understand how bereaved caregivers assess the quality of dying and death experience with the QODD by exploring the cognitive processes that underlie their evaluations. Particular attention was focused on (1) the information that provided the basis of quality-rating judgments, (2) quality-rating judgment strategies, and (3) any response difficulties encountered.
Methods Participants and Recruitment The present study was conducted as part of an ongoing longitudinal study of the quality of dying and death in patients with metastatic gastrointestinal cancer or lung cancer and the subsequent bereavement morbidity in their caregivers. Consecutive patients attending outpatient oncology clinics at Princess Margaret Hospital, a comprehensive cancer center in Toronto, Ontario, Canada, were recruited to participate. Patient deaths were subsequently identified through regular review of hospital records, obituary records, and inquiries made to patients’ family physicians. Six months after patient death, a letter was sent to the patient-identified primary caregiver informing them that they would be contacted by telephone regarding this study. The letter included a voicemail telephone number that they could call if they did not want further contact. One week after the letter was sent, the bereaved caregiver was contacted by telephone and invited to take part. If they were interested,
197
an in-person or telephone interview was scheduled for eight to 10 months after the patient’s death, at which time an interview (the QODD) and self-report questionnaire package were completed. Because deaths might have occurred in locations outside of the hospital, investigators could not rely on hospital records alone to identify patient deaths. The six-month time frame was, therefore, required to search obituary records and contact patients’ family physicians to ensure identification of the maximum number of patient deaths in the sample. A convenience sample of bereaved caregivers was asked to take part in a cognitive interview protocol while answering the QODD. This protocol made the interview more complex and was introduced only when the interview was conducted in person. These interviews were audiotaped and transcribed.
Measures Patient and caregiver demographics (age, gender, ethnicity, country of birth, level of education, and household income), specific caregiver variables (primary language of the caregiver and the nature and length of the patient-primary caregiver relationship), and patient illness variables (cancer type and length of illness) were obtained through patient chart review and caregiver self-report questionnaires. The quality of dying and death was measured with the QODD.5 This 31-item measure is administered to family members or health care providers after a patient’s death and asks for evaluation regarding events in the last week, or, if the patient was unconscious, the last month of life. It inquires about aspects of symptoms and personal care, treatment preferences, time with family and friends, whole person concerns, preparation for death, and the moment of death. For each item, respondents are asked first whether or how frequently an event occurred and then to provide a quality rating for that item with the question ‘‘How would you rate this aspect of the patient’s dying experience?’’ on a scale from 0 to 10 (with 0 representing a terrible experience and 10 representing an almost perfect experience). These 31 quality scores are averaged and multiplied by 10 to generate a QODD total score on a scale from 0 to 100. Since initial publication, the measure’s developers have reported on several shortened versions of the QODD intended
198
Hales et al.
to reduce respondent burden16 or use in the ICU setting,17 but the full 31-item QODD was used in the present study.
Cognitive Interview Protocol During the administration of the QODD interview to bereaved caregivers, participants were asked to verbalize their thought processes after answering each of the 31 quality-rating questions. This retrospective probing cognitive technique is distinct from a purely concurrent ‘‘think-aloud’’ technique, in which respondents are asked to describe their answering process as they complete the questionnaire, and has been deemed easiest for respondents in face-to-face interviews.18 Two probes were used to access the information retrieved and the judgment strategies that respondents relied on in providing quality ratings. These were (1) ‘‘What were you thinking about when you answered that question?,’’ and (2) ‘‘Why did you give it that rating?’’ If subjects expressed any difficulty providing a quality rating, they were asked, ‘‘Why is this question difficult for you to answer?’’
Analysis Qualitative content analysis was applied to transcribed interviews, with a specific focus on the information retrieved, the judgment strategies used, and any difficulties participants reported while formulating the 31 QODD quality ratings. Analysis was both inductive and deductive, beginning with fixed categories and subsequently allowing categories and codes to emerge as the analysis progressed until saturation. The initial coding scheme was informed by that used by McPherson and AddingtonHall14 in their evaluation of the VOICES questionnaire, based on the four-stage model of survey response by Tourangeau et al.12 For the purposes of this study, the foci were primarily the information respondents used as the basis of their judgments, which related most to their processes of comprehension and retrieval, and the judgment strategies that they employed, which related most to the processes of judgment and response. Response difficulties, it was understood, might be found at any stage of the answering process. The first author conducted initial analysis of the interview transcripts. To ensure valid interpretation of subject responses, comprehensiveness, and appropriateness of analysis, meetings were held with
Vol. 43 No. 2 February 2012
coinvestigators L.G, R.N, and G.R throughout the analysis process. Coded transcripts were reviewed and compared, and category descriptions were reviewed, amended, or further developed until consensus among the group was reached. The iterative process of data collection and analysis continued until saturation. Statistical analysis was used to describe sample characteristics, quality of dying and death scores, and the frequency of coded categories.
Results Sample Characteristics and QODD Scores Twenty-two bereaved caregivers evaluated 22 different patient deaths with the QODD and cognitive interview protocol. No caregivers refused participation in this protocol. Qualitative analysis was concurrent, and recruitment for this protocol was closed when saturation was reached. Characteristics of the 22 patients and bereaved caregivers and QODD scores are presented in Table 1. The mean age of the patients was 66.2 years and the majority were male, Caucasian, and Canadian born, with education beyond high school, and household income greater than C$60,000 per year. The mean age of the caregivers was 59.0 years and the majority were female, Caucasian, and Canadian born, with education beyond high school. Half of the bereaved caregiver sample had an income greater than C$60,000 per year. Most caregivers were patient spouses, and the average length of the relationship with the patient was 37.5 years. The mean QODD total score was 66.4 (on a 0e100 scale, with higher scores indicating better quality of dying and death).
Content Analysis Coding Analysis of the cognitive processes that provided the basis for evaluations when responding to the QODD focused on (1) the information retrieved, (2) the judgment strategy, and (3) any response difficulties reported. A summary of the codes and categories generated by the content analysis, with illustrative quotes, is presented in Table 2. The QODD allows for respondents to choose ‘‘I do not know’’ in response to quality-rating questions. If a quality rating was not provided by the respondent (i.e., there was no cognitive
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
199
Table 1 Sample Characteristics and QODD Scores (n ¼ 22) n (%) Patient variables Age, mean (SD), range Gender Female Male Ethnicity Caucasian Other Country of birth Canada Other Education #High school >High school Household income #$60,000/year >$60,000/year Primary cancer Gastrointestinal Lung Time from diagnosis to death, mean (SD), range
66.2 years (13.0), 34.3e88.8 8 (36.4) 14 (63.6) 21 (95.5) 1 (4.5) 14 (63.6) 8 (36.4) 6 (27.3) 16 (72.7) 6 (27.3) 16 (72.7) 14 (63.6) 8 (36.4) 39.3 months (33.3), 4.5e135.6
Caregiver variables Age, mean (SD), range Gender Female Male Ethnicity Caucasian Other Country of birth Canada Other Primary language English Other Education #High school >High school Household income #$60,000/year >$60,000/year Relationship to patient Spouse Parent Son/daughter Friend Length of relationship with patient, mean (SD), range
59.0 years (13.8), 28e80 17 (77.3) 5 (22.7) 21 (95.5) 1 (4.5) 18 (81.8) 4 (18.2) 22 (100) 0 (0) 6 (27.3) 16 (72.7) 11 (50) 11 (50) 15 1 4 2 37.5 years
Quality of dying and death QODD total score, mean (SD), range
process to analyze), this was coded as ‘‘not applicable.’’ If the respondent provided a quality rating but did not communicate what information or judgment strategy provided the basis for their quality rating, this was coded as ‘‘no information provided’’ or ‘‘no judgment strategy provided.’’ Of note, several respondents had difficulty with the cognitive interview protocol and could not verbalize their thought processes.
(68.2) (4.5) (18.2) (9.1) (15.6), 3e58
66.4 (10.3), 48.7e88.9
At least once during their interviews, 16 of 22 caregivers could not explain on what information they were basing their answers. Similarly, at least once during their interviews, 17 of 22 caregivers could not provide details of their judgment strategy.
Information Retrieved The types of information that provided the basis for the 31 QODD quality ratings were
200
Hales et al.
Vol. 43 No. 2 February 2012
Table 2 Content Analysis Coding Scheme Code Description Information retrieved Perspective of patient
Perspective of caregiver
Perspective of other family/friend
Judgment strategy Comparison to hoped for/ideal dying experience Comparison to prior to dying phase
Comparison to state of distress/no distress
Comparison to normalcy/humanness
Item reported to be not relevant Other strategy (e.g., comparison to a known past dying experience)
Response difficulty No communication with patient regarding that aspect of experience Question deemed irrelevant/not applicable
Unclear question meaning/wording
Frame of reference uncertain Difficulty using rating scale No knowledge of event
Example Quote Respondent 4, in response to probe regarding 7/10 quality rating for item that asks whether deceased appeared to have her/his pain under control: ‘‘Well, even though he was in pain a fair amount, it was important to him to remain lucid and in control.’’ Respondent 7, in response to probe regarding 9/10 quality rating for item that asks about deceased’s time with other family and friends: ‘‘Well, that was good for me. Because for me. for them coming around. it was mostly for me. And that’s what [he] didn’t understand. He’d be like, ‘What are they doing over here again?’’’ Respondent 7, in response to probe regarding 0/10 quality rating for item that asks whether deceased said goodbye to loved ones: ‘‘It would have been nice if he said to the kids. I would have liked him to say. for their self-esteem. ‘You are.you’ve been a good son.’’’ Respondent 20, in response to probe regarding 9/10 quality rating for item which asks whether anyone was present at the moment of death: ‘‘Well, she said she didn’t want to die alone. She got what she asked for. We were there all night long.’’ Respondent 2, in response to probe regarding 8/10 quality rating for item that asks whether deceased appeared to keep her/his dignity and selfrespect: ‘‘That was in his nature. This was one of the things I found, looking back. I did not find Bill to change. The real Bill came out throughout his dying period.’’ Respondent 13, in response to probe regarding 7/10 quality rating for item that asks whether deceased appeared to be worried about strain on her/ his loved ones: ‘‘I would say. I mean, obviously the less worrying he did the better, right?’’ Respondent 20, in response to probe regarding 3/10 quality rating for item which asks whether the deceased appeared to have the energy to do most of the things she/he wanted to do: ‘‘Well if she had no energy, she couldn’t do anything. So she wasn’t behaving like a human being any more.’’ Respondent 8, in response to probe regarding 5/10 quality rating for item that asks whether the deceased said goodbye to loved ones: ‘‘Well, she had done it earlier, so I think it was irrelevant at that stage, so I’d say five.’’ Respondent 9, in response to probe regarding 9/10 quality rating for item that asks whether the deceased appeared to have control over what was going on around her/him: ‘‘I would say almost perfect experience. I would have to say of all the people I’ve seen die she certainly was the most courageous person I’ve encountered. Even though she was very, very weak, she was still able to express that and be in control of it.’’ Respondent 11, following item that asks whether deceased appeared to find meaning and purpose in life: ‘‘I don’t know. I guess he’s the only one who can really tell us that. He never communicated that.’’ Respondent 15, following item that asks whether the deceased appeared to feel at peace with dying: ‘‘I couldn’t answer that question. I think he didn’t think he was dying, pretty much. Not then. I don’t think he thought that going to the hospital for a blood transfusion was going to be his demise, when in fact it was.’’ Respondent 1, following item that asks whether the deceased appeared to feel at peace with dying: ‘‘I’m not sure whether he was at peace with it, or whether his body was struggling. It’s hard for me to distinguish between the psychological aspect of it and the physiological. I know it’s all part of the one, the same big melting pot, but they are two different parts.’’ Respondent 17, following item that asks whether anyone was present at the moment of death: ‘‘It’s very hard. In my opinion poor, but maybe in his opinion not.You know what I’m saying?’’ Respondent 3, following item that asks where the deceased died: ‘‘It’s just the wording of this. You see, the thing is that you can’t say dying at all is ‘almost perfect experience.’ It’s so not. It’s the worst thing ever.’’ Respondent 10, following item that asks whether anyone was present at the moment of death: ‘‘I really don’t know, because I was there until midnight. She died about. I think 4:00 in the morning.’’
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
behaviors, communications, thoughts, feelings, or values. This information was categorized according to whether it referred to the perspective of the patient (i.e., the patient’s behavior, communication, and supposed thoughts/feelings/values), the caregiver (i.e., the caregiver’s thoughts, feelings, or values), another family member or friend (i.e., the family member or friend’s behavior, communication, and supposed thoughts/feelings/values), or a combination of perspectives. Over the QODD as a whole, all respondents relied on more than one perspective to answer the questionnaire. The most common perspective mentioned was that of the patient (the mean number of times the patients’ perspective was used over the 31-item QODD interview was 20.0, standard deviation [SD] 5.3), followed by that of the caregiver (mean 6.5, SD 4.2), and then by that of another family member or friend (mean 2.2, SD 2.3). The codes and related example quotes are provided in Table 2. All respondents used multiple perspectives (i.e., patient, caregiver, and/or other family member/friend) when providing a single rating at least once during the interview, with a range across respondents of one to nine uses of multiple perspectives. For example, in response to a probe regarding a five of 10 quality rating for the item that asks whether the deceased said goodbye to loved ones, Respondent 3 said, ‘‘I think the fact that he didn’t say goodbye to anybody wouldn’t have really bothered him, and it didn’t really bother us.’’ This response was coded as referencing all three perspectives: that of the patient, the caregiver, and other family/friends.
Judgment Strategies Quality-rating judgment strategies were generally based on comparisons. Standards of comparison were categorized by type for each of the 31 QODD quality-rating questions. The most common standards of comparison were to ‘‘a hoped for or ideal dying experience’’ (the mean number of times this strategy was used over the 31-item QODD interview was 8.8, SD 3.8), ‘‘a state before the dying phase’’ (mean 8.0, SD 4.2), ‘‘a state of distress/no distress’’ (mean 4.1, SD 5.2), and ‘‘normalcy/humanness’’ (mean 1.0, SD 1.3). Some respondents provided an item quality rating that was negative, neutral, or positive and explained their
201
rating by saying that the item was ‘‘not relevant to the dying and death experience’’ of the patient (mean 3.5, SD 3.1). Other less frequently used judgment strategies included comparison to ‘‘a known past dying experience’’ or ‘‘a modified ideal given known limitations.’’ The codes and related example quotes are provided in Table 2. All respondents relied on more than one strategy to answer the questionnaire, with a mean of 4.5 different strategies used per respondent (SD 1.0). Some respondents used multiple standards of comparison to generate a single rating, with a range of zero to eight uses of multiple standards. For example, in response to a probe regarding a nine of 10 rating for an item that asks whether the deceased appeared to be unafraid of dying, Respondent 9 said, ‘‘I think because she herself had experienced many people dying, and she had seen people who were afraid.she very much asked for a peaceful death, and I believe she had a peaceful death.’’ This judgment strategy was coded as both ‘‘comparison to a known past dying experience’’ and ‘‘comparison to a hoped for or ideal dying experience.’’
Response Difficulties When respondents had difficulty providing a quality rating for any of the 31 QODD items, they were asked why answering the question was difficult, and these responses were coded. Response difficulties were identified by 21 of 22 caregivers at least once during the interview. For 16 of 22 caregivers, this difficulty was great enough to prevent them from providing a quality rating at least once in the interview, although in most instances, caregivers still provided a rating despite indicating a response difficulty. Of those response difficulties reported, the most common reasons coded were ‘‘no communication with the patient regarding that aspect of experience,’’ followed by ‘‘question deemed irrelevant/not applicable,’’ followed by ‘‘unclear question meaning/wording.’’ Of the 31 quality-rating items in the QODD, 25 were associated by the pool of respondents with a response difficulty at least once. The item most commonly associated with difficulty was Item 25, ‘‘Did (patient) have the means to end her/his life if she/he needed to?,’’ and ‘‘How would you rate this aspect of (patient)’s dying experience?,’’ with seven of 22
202
Hales et al.
respondents having difficulty answering these questions. The most common reason given for this difficulty was ‘‘unclear question meaning/wording,’’ reported by four of seven respondents because they were unsure if they were meant to report on patient suicidality. Both Item 6, ‘‘How often did (patient) appear to feel at peace with dying?’’ and ‘‘How would you rate this aspect of (patient)’s dying experience?,’’ and Item 17, ‘‘Did (patient) appear to find meaning and purpose in her/his life?’’ and ‘‘How would you rate this aspect of (patient)’s experience?,’’ were associated with difficulty for six of 22 respondents. For both these items, the most common reason coded for the difficulty was ‘‘no communication with the patient regarding that aspect of experience’’ (four of six respondents and three of six respondents, respectively).
Discussion This study is the first to use cognitive interviewing methodology to evaluate a measure of the quality of dying and death. In answering the QODD, bereaved caregivers most often relied on the patient perspective but all also incorporated their own perspective and/or that of family and friends. This may occur because the bereaved caregiver and other family members are participant-observers in the death experience of the patient and also may have been the recipients of end-of-life care. As the disease progresses and patients become more debilitated and perhaps spend less time in a conscious state, caregivers often become more involved in the dying experience. This may include tending to the needs of the patient, communicating and liaising with health care providers, and receiving practical and emotional support to alleviate their own distress. The use of multiple perspectives by caregivers to evaluate the end-of-life experience, therefore, may reflect a summated rather than individualized proxy-for-patient approach, which matches the nature of end-oflife care delivery. In judging the quality of the dying and death with the QODD, bereaved caregivers took into account multiple standards against which the end-of-life experience could be compared. The most common standards applied
Vol. 43 No. 2 February 2012
were those of an ideal or hoped-for dying experience and the state before the dying phase, although there was variability among bereaved caregivers in their reliance on different comparative strategies. These results suggest that individual values, past experiences, or expectancies shape whether a particular death experience is judged positively or negatively. In other words, when asked to evaluate whether a death experience was ‘‘good’’ or ‘‘bad,’’ respondents must consider ‘‘for whom?’’ and ‘‘compared to what?’’ Reliance on different perspectives and standards of comparison may partly explain the reported variability in QODD scores obtained from different raters, with respect to a single patient death.8,9 Such variability in the processes that shape responses to the QODD may be regarded as threats to its reliability and validity. Indeed, individual variation in survey response is often perceived as a phenomenon that confounds interpretation of pooled quantitative results.19 It may be argued that such findings justify amendment of questionnaires to allow for more uniform responses. Alternatively, the demonstration of such variability might serve to adjust our understanding of the underlying constructs that are being measured. In this case, the reliance of bereaved caregivers on multiple perspectives and multiple comparisons in rating the quality of dying and death may reflect the multidetermined nature of this construct, which cannot be simplified by rewording or redesign of the questionnaire. This use of multiple perspectives and standards of comparison parallels findings in the field of quality-of-life evaluation.13 Quality-oflife measures have not been found to distinguish consistently among groups of raters, are often only weakly related to objective criteria of functioning, and show little convergence across measurement perspectives. It is acknowledged that quality-of-life evaluations may be informed by such personal reference points as prior functioning, lost capacities, extreme experiences, observations of others, past encounters with illness, and communication from providers. Rather than be considered to reflect measurement bias, denial, or willful distortion, however, these differences in quality-of-life appraisal are presumed to reflect individual differences and intraindividual changes in internal standards, values, and meaning of quality of
Vol. 43 No. 2 February 2012
Application of Cognitive Interviewing to QODD
life. These differences, it has been argued, if well understood, could serve as potential targets for clinical intervention (i.e., helping patients to reprioritize values and priorities for more adaptive coping and adjustment).13 The individual differences in values, experiences, and expectations that influence evaluation of the quality of dying and death may similarly be potential targets for clinical intervention. Discussions with patients and family members throughout the illness course about possible future events, what to expect, and how best to prepare may help to shape how the dying and death experience is ultimately evaluated. Examination of response difficulties in providing quality of dying and death ratings was another objective of this study. We found that most caregivers reported difficulty in providing a quality rating at least once in their QODD interview and that most QODD items were associated with difficulty in at least one of the interviews. In some cases, caregivers identified a response difficulty and declined to provide a quality rating, whereas, in other cases, caregivers identified a response difficulty but, nevertheless, provided a rating. Differences in response difficulty may contribute to response variability. Although all quality ratings required respondents to assess the subjective experience of the patient, the questions associated with the most difficulty were those that asked about more internal experiences (e.g., meaning in life or peace with dying) or about aspects of the endof-life experience that might have been difficult for patients and caregivers to discuss openly (e.g., means to end life if needed or peace with dying). These findings are supported by previous research, indicating that proxies are most accurate with respect to concrete observable aspects of the patient experience but less accurate with respect to patients’ subjective states.6 Although eliminating items referencing more internal experience or sensitive topics would increase the weighting of patient experience, this would omit items considered highly relevant to the quality of dying and death construct. An alternative approach that may improve the validity of the after-death evaluation of the patient’s experience would be predeath identification by the patient of the caregiver they regard as the best and most reliable source of information.
203
Examination of other response difficulties suggests strategies that might improve the quality of some QODD item responses. For example, given that ‘‘unclear question meaning/ wording’’ was a common reason for response difficulty, providing additional explanation for some items (i.e., a definition for meaning and purpose in life) may assist some respondents in providing responses. Similarly, because ‘‘question deemed irrelevant/not applicable’’ was another common reason for response difficulty, providing respondents with the option of not answering because an item was irrelevant and/or allowing respondents to rate the relative importance of items might improve the validity of total scores. A 17-item QODD, based on research in which terminally ill patients and their associates prioritized rankings of QODD items,16 partly addresses this issue but ultimately sacrifices some comprehensiveness and cannot account completely for the individual and shifting nature of end-of-life priorities as patients and families move through the course of disease. In summary, as with other questionnaires attempting to measure constructs that are highly individual and subject to change over time, comprehensive global measures yielding summative scores are challenging to interpret when applied to large populations. There are limitations to this study. The use of cognitive interview methodology assumes that questionnaire respondents follow a logical or at least understandable process of response. It also is based on the assumption that respondents have conscious access to their cognitive processes as they answer questions and that they will report this information honestly and accurately, which may not be the case. Further, the majority of this sample had difficulty at least once in the cognitive interviewing process, which suggests that this is a challenging task for most individuals. An additional limitation is the small sample size and its homogeneity with respect to ethnicity and language, which limits generalization of the findings. Some potential sources of variability in response could not be explored with the methodology of this study and should be the focus of future research. These include error in memory or recall and the impact of time elapsed since death. Interestingly, no caregiver suggested that the inability to remember details of the
204
Hales et al.
time period in question caused response difficulty, although all interviews were conducted eight to 10 months after the patient’s death. This may suggest that events near the end of life are highly salient and, therefore, vividly recalled, although other memories may fade over time. However, this does not rule out possible retrospective bias and revision. This study focused on bereaved caregivers and how nonfamily respondents (such as nurses or physicians) make quality of dying and death judgments, and the potential influence of respondent psychological characteristics, such as distress and empathic ability, remain areas for future research using cognitive interview methodology. In conclusion, the results of the present study suggest that the quality of dying and death is a complex construct based on multiple perspectives and standards of comparison. Rather than being seen simply as errors or distortion in the measurement process, these multiple perspectives and standards may be viewed as valid targets for end-of-life care. Clinical attention to the family as the unit of care and to values, past experiences, and expectations in advance care planning may help ensure that the end of life is ultimately remembered and judged as positively as possible.
Disclosures and Acknowledgments Funding sources included Canadian Institutes of Health Research operating grant 74684 and a National Cancer Institute of Canada Post-MD Fellowship (Dr. Hales). The authors declare no conflicts of interest. The authors acknowledge the contributions of the research staff responsible for subject recruitment, data collection, and management, including Anne Rydall, MSc, Leanne Ferreira, BSc, and Andrew Walsh, MA.
References 1. Hales S, Zimmermann C, Rodin G. The quality of dying and death: a systematic review of measures. Palliat Med 2010;24:127e144. 2. Kaasa S, Loge JH. Quality of life in palliative care: principles and practice. Palliat Med 2003;17:11e20. 3. Steinhauser KE. Measuring end-of-life care outcomes prospectively. J Palliat Med 2005;8:S30eS41.
Vol. 43 No. 2 February 2012
4. Teno JM. Measuring end-of-life care outcomes retrospectively. J Palliat Med 2005;8:S42eS49. 5. Curtis JR, Patrick DL, Engelberg RA, et al. A measure of the quality of dying and death: initial validation using after-death interviews with family members. J Pain Symptom Manage 2002;24:17e30. 6. Addington-Hall J, McPherson C. After-death interviews with surrogates/bereaved family members: some issues of validity. J Pain Symptom Manage 2001;22:784e790. 7. Hales S, Zimmermann C, Rodin G. The quality of dying and death. Arch Intern Med 2008;168: 912e918. 8. Mularski R, Curtis JR, Osborne M, Engelberg RA, Ganzini L. Agreement among family members in their assessment of the quality of dying and death. J Pain Symptom Manage 2004;28:306e315. 9. Levy CR, Ely W, Payne K, et al. Quality of dying and death in two medical ICUs. Chest 2005;127: 1775e1783. 10. McColl E, Meadows K, Barofsky I. Cognitive aspects of survey methodology and quality of life assessment. Qual Life Res 2003;12:217e218. 11. Murtagh FEM, Addington-Hall JM, Higginson IJ. The value of cognitive interviewing techniques in palliative care research. Palliat Med 2007;21:87e89. 12. Tourangeau R, Rips LJ, Rasinski K. The psychology of survey response. Cambridge, UK: Cambridge University Press, 2000. 13. Rapkin BD, Schwartz CE. Toward a theoretical model of quality-of-life appraisal: implications of findings from studies of response shift. Health Qual Life Outcomes 2004;2:14e26. 14. McPherson CJ, Addington-Hall JM. Evaluating palliative care: bereaved family members’ evaluations of patients’ pain, anxiety and depression. J Pain Symptom Manage 2004;28:104e114. 15. Addington-Hall J, Walker L, Jones C, et al. A randomized controlled trial of postal versus interviewer administration of a questionnaire measuring satisfaction with, and use of, services received in the year before death. J Epidemiol Commun Health 1998;52:802e807. 16. Downey L, Engelberg RA, Curtis JR, Lafferty WE, Patrick DL. Shared priorities for the end-of-life period. J Pain Symptom Manage 2009; 37:175e188. 17. Curtis JR, Treece PD, Nielsen EL, et al. Integrating palliative and critical care: evaluation of a quality-improvement intervention. Am J Respir Crit Care Med 2008;178:269e275. 18. Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003;12: 229e238. 19. Norman G. Hi! How are you? Response shift, implicit theories and differing epistemologies. Qual Life Res 2003;12:239e249.