Design Issues for Clinical Research in Health Psychology

Copyright © 1998 Elsevier Science Ltd. All rights reserved.

8.05 Design Issues for Clinical Research in Health Psychology CAROLYN E. SCHWARTZ Frontier Science & Technology Research Foundation Inc., Chestnut Hill and Harvard Medical School, Boston, MA, USA 8.05.1 INTRODUCTION

137

8.05.2 IMPLEMENTING RESEARCH IN A CLINICAL SETTING

138

8.05.2.1 Key Considerations for Meaningful Research 138 8.05.2.1.1 Ask a meaningful research question 138 8.05.2.1.2 Have a strong theoretical foundation 138 8.05.2.1.3 Measure appropriate outcomes 138 8.05.2.1.4 Use an appropriate study design 139 8.05.2.1.5 Maintain a high response rate 139 8.05.2.1.6 Maintain complete data 139 8.05.2.1.7 Plan and implement appropriate statistical analysis 140 8.05.2.1.8 Plan study with adequate statistical power 140 8.05.2.1.9 Disseminate findings to appropriate audiences 140 8.05.2.2 Study Designs to Evaluate Clinical Interventions 140 8.05.2.2.1 Needs assessment followed by randomized consent 141 8.05.2.2.2 Three-arm crossover design 145 8.05.2.2.3 Case±control 146 8.05.2.2.4 Solomon four-group pre±post design 146 8.05.2.2.5 Pre±post one-group design 146 8.05.2.3 Selecting the Appropriate Study Design for Evaluating Treatment Intervention Effects: Balancing Research Questions and Assumptions 146 8.05.2.4 Practical Challenges to Implementing Research in a Clinical Setting 147 8.05.2.4.1 Who pays for treatments being evaluated? 147 8.05.2.4.2 Incentives of capitated care 148 8.05.2.4.3 Legal considerations 148 8.05.2.4.4 Unavoidable costs of doing research 149 8.05.2.4.5 Ethical considerations 149 8.05.2.5 An Emerging Theoretical Challenge for Outcomes Research 149 8.05.3 SUMMARY

150

8.05.4 REFERENCES

150

8.05.1 INTRODUCTION

This documentation requires relying not only on empirical data to demonstrate the impact of treatment, but also on research designs which are rigorous yet feasible in active clinical settings where funding may not be available to support resource-intensive efforts. Given the

With the increasing focus on cost-containment and cost-effectiveness, clinical health psychologists are finding it increasingly necessary to document the value of their services. 137

138

Design Issues for Clinical Research in Health Psychology

emphasis of health psychology on the individual's well-being, it would seem incumbent on such research to link the patient's perspective with objective clinical outcomes as well as cost. Such linkage would also be consistent with the biopsychosocial model which drives much of health psychology theory and practice. The purpose of this chapter will be to discuss key designs, considerations, and challenges of doing outcomes studies of interventions in a clinical health psychology and/or medical setting. Although similar issues will be relevant for health psychology research which is correlative/ predictive in nature, it is beyond the scope of the present chapter. Further information on some of the various concepts and designs discussed can be found in Volume 3. 8.05.2 IMPLEMENTING RESEARCH IN A CLINICAL SETTING 8.05.2.1 Key Considerations for Meaningful Research Active clinical settings present an inviting paradox to the health psychologist. On the one hand, they have the ultimate resource for relevant outcomes research: an ample source of patients. Further, there is an obvious potential to link patient outcome data to clinical and cost data. The challenge of such settings is implementing data collection efforts from patients which are both externally and internally valid, often without financial and personnel resources dedicated to research endeavors. Thus, key considerations in study implementation need to be planned and integrated into the daily clinical routine so that the resulting data yield meaningful and useful glimpses into the patient's experience of illness and its management. The considerations which are crucial to successful study implementation (Table 1) will be discussed briefly. 8.05.2.1.1 Ask a meaningful research question Since any well-done research endeavor is likely to require substantial effort and organization, it would be important that the research be motivated by research questions which are meaningful to patients, providers, payers, and other parties. A clinical setting provides the health psychology researcher with ample opportunities to hear about potentially interesting research issues and to develop hypotheses about what social and behavioral factors would be worthy of exploration in a formal study. By listening to patients, their caregivers, healthcare providers, payers, and others, researchers can develop questions which are responsive to

current forces in healthcare delivery and health enhancement in general. Meaningful research questions can be developed in response to maintaining an open ear when interacting with patients and others in the clinical setting. Other more formal approaches might include focus groups (Morgan, 1988) and semistructured individual interviews with people with various relevant perspectives. 8.05.2.1.2 Have a strong theoretical foundation A reasonable background of both knowledge and theory would be necessary conditions for a research study to be worth the effort and resources it will cost. Such a background can be gleaned by consulting with health psychologists who have a current and broad foundational understanding of models of change or growth from fields such as health education, public health, clinical and health psychology, and psychiatry. These models would provide a necessary foundation for hypothesizing what psychosocial and medical factors would be relevant and thus should be measured (Slife & Williams, 1997). This understanding might be supplemented with recent literature reviews as well as searches of computer databases of medical and psychological publications. 8.05.2.1.3 Measure appropriate outcomes In beginning to plan a study with a strong theoretical foundation, one must select relevant outcomes. This selection process should also be informed by clinical acumen and patient experience. Focus groups (Morgan, 1988) with providers and patients can be helpful in identifying germane outcomes. This method usually identifies relevant groups whose perspective should be considered and begins with a series of open-ended questions. The underlying theme of focus groups is to expand the realm of possibilities rather than to contract them. One continues focus groups and various provider and patient subgroups until the information gleaned becomes redundant (Morgan, 1988). Focus groups can also be useful for pretesting questionnaire packets and for selecting one questionnaire among many which ostensibly measure the same thing. For example, there are numerous measures of functional status which might be used in an outcomes study. In the pilottesting phase of a study evaluating the impact of two psychosocial interventions (Schwartz, 1994; Schwartz & Rogers, 1994; Schwartz & Sprangers, under review), Schwartz pretested a set of functional status measures, and asked chronically ill patients to track their mood and energy level before and after completing each measure.

Implementing Research in a Clinical Setting

139

Table 1 Key considerations for successful study implementation. Consideration

Method for achieving

a. Meaningful research question

Listening to patients and clinicians, literature review, attenting to regulatory forces in health care delivery Literature review; expert consultation Focus groups Match with research question and assumptions (see Table 2) Total Design Method Telephone follow-up Involvement of statistician when planning study Involvement of statistician when planning study Peer-review journals; patient education; managed care journals

b. Theory-based c. Appropriate outcomes d. Appropriate study design e. Response rate f. Complete data g. Appropriate analysis h. Statistical power i. Dissemination to appropriate audiences

The measure which was selected finally was the one which was the least depressing and was rated by patients as asking questions they deemed pertinent to their physical well-being. 8.05.2.1.4 Use an appropriate study design Although collecting data from consecutive patients who come into a clinic might be considered research by some people, most professionals would agree that studies must be designed before they are implemented if they are to result in data worth analyzing. It is believed that study design is a crucial stumbling block for many aspiring researchers, and an entire section of this chapter has been devoted to describing five study design options which can be implemented feasibly in busy clinical settings (see below). 8.05.2.1.5 Maintain a high response rate Ascertaining the generalizability of the study results will be an important consideration in assessing the impact that it may have. Studies which lose a large proportion of eligible participants are necessarily suspect of suffering from selection bias which may hamper the investigators' ability to make any valid conclusions about the treatments being evaluated. Further, substantial drop-out rates can result in more time required for accrual and more costly resources required to implement studies. Thus, planning carefully to maximize response rate would be a necessary condition to successful research. In his Total Design Method, Dillman (1978) provides an accessible and feasible method which integrates the large literature on survey research to specific steps for assuring a high response rate. These steps can be easily implemented by the clinical researcher. For example, Dillman provides details regarding the letter which is sent to patients to inform them

and invite them to participate in a given research study. He outlines the information which should be included in each paragraph, the letterhead parameters which have been documented to enhance response rate, the color ink which should be used to sign the letter, whose names should grace the letter, and the way in which the letter should be folded. Dillman (1978) provides a similar level of detail regarding preparation of the questionnaire booklets. Although this level of detail might seem unnecessary, he documents the increased percentage of response rate which results from their consideration. If one follows Dillman's protocol, one can be assured of a 90% response rate. Indeed, following these steps very carefully has allowed our group to maintain a high response rate in a number of studies by mail. 8.05.2.1.6 Maintain complete data Missing data can pose significant problems to data analysis and interpretation. Most selfreport questionnaires require data on at least 75% of the items within a subscale to allow imputation of a subscale score. Similarly, most statistical methods require complete data to include a participant's data in the analysis. Although some statistical methods can accommodate missing data (e.g., growth curve; Francis, Fletcher, Stuebing, Davidson, & Thompson, 1991), the meaning of the missingness can be an important consideration. For example, if data are ªinformedº missing, then their absence is indicative of some notable cause, such as dying during the course of the study. If they drop out due to health problems or access to care, then their missing data would also be informative. Thus, it is incumbent on the health psychology investigator to maintain complete data as much as possible, and to document reasons for drop-outs and missing data. Developing a good rapport with study participants can be crucial for attaining complete

140

Design Issues for Clinical Research in Health Psychology

data. This rapport may be developed initially in an intake interview, and maintained via telephone follow-up. For example, if a participant has not returned a questionnaire packet within two weeks of mailing it out, the investigator would be well-advised to inquire via telephone and to negotiate a date by which the participant is able to return the completed packet. Similarly, if the packet is returned but there are items which have not been answered, a respectful phone call can be beneficial. In general, it has been found that such calls are perceived by patients to be indicative of the seriousness and meaningfulness of the research endeavor, and they are cooperative in answering the missing items. If the patient purposefully left the item blank, then the investigator should respect the confidentiality of the patient, code the item as ªrefused,º and not press the participant for an answer.

8.05.2.1.7 Plan and implement appropriate statistical analysis A well-planned investigation would include planning the statistical analysis before the study is implemented. Such intention would ensure that the appropriate data (e.g., covariates, predictors, and outcomes) are collected, and that the research questions will be addressed adequately by the statistical analysis. It is recommended to include on the research team a professional who has expertise in statistical analysis. This person's expertise should be included at the planning stage of any study.

8.05.2.1.8 Plan study with adequate statistical power By planning the statistical analysis when the study's research questions are being defined, the thoughtful investigator paves the way for adequate statistical power. Simply stated, an investigator's power to detect meaningful differences depends on three parameters: the sample size, the magnitude of the effect which is deemed clinically significant, and the inherent variability of the outcome measure. The power curve is exponential, requiring few subjects per group for large effects, and large numbers of subjects per group for moderate and small effects (Figure 1) (Cohen, 1988; Pocock, 1983). It is recommended that power calculation be used during the planning phases of a study by a professional who has expertise in statistical analysis, or with the aid of software designed for behavioral science studies (e.g., Bornstein & Cohen, 1988).

8.05.2.1.9 Disseminate findings to appropriate audiences One of the most important products of a research study is developing vehicles for disseminating study results. Such dissemination should take place through peer-reviewed professional journals, a mechanism which can enhance the state of knowledge in the broader arena. Other mechanisms exist which can influence other highly relevant audiences. For example, there are private foundations which focus on a specific disease and which would provide conduits by which investigators might implement patient educational programs based on the results of their research. There might also be access to other healthcare providers and payers by publishing jargon-free articles in managed care journals and health management organizational newsletters. The more mechanisms by which access to study results can be provided, the more influential such results can be. Even negative (i.e., nonsignificant) findings can be informative, so investigators are to be encouraged to disseminate study results in all cases. 8.05.2.2 Study Designs to Evaluate Clinical Interventions Thus far, important considerations in successful study implementation to enhance the external and internal validity of the research endeavor have been reviewed. Although the clinical setting does not necessarily pose challenges to assessing needs or concerns of patient populations, many important research questions may be more focused on how efficacious and cost-effective an intervention may be. Despite the above suggestions, the clinical setting may pose challenges to implementing the standard randomized controlled trial. In this design, eligible patients are approached and asked to participate in a trial in which they will be randomized to one of two treatment arms. Study designs which maximize statistical power are placebo-controlled randomized trials with longitudinal follow-up (Kraemer & Thiemann, 1989). Given the ethical, theoretical, and practical problems with placebo-controlled studies (Schwartz, Chesney, Irvine, & Keefe, 1997), the modern investigator often may be forced to consider statistical power issues in the context of intervention studies of two active treatments, such as standard care as compared to an experimental treatment or psychological adjunct (Figure 2(a)). The more one controls for nonspecific effects, the smaller the anticipated effect size and hence the larger the sample size requirements (Kazdin & Bass,

Number of Patients per Treatment Arm

Implementing Research in a Clinical Setting

141

400

300 Parallel design 200 Cross over design 100

0.8

0.7

0.6

0.5

0.4

0.3 Moderate

Large

0.2

0.1 Small

Effect Size Figure 1 Statistical power curves for parallel design and crossover trial. This plot shows how the sample size requirements for a study increase with the decreasing magnitude of the anticipated effect size. Using a crossover design can reduce the required sample to approximately one-quarter the number of subjects as a parallel-arm study for the same power (Fleiss, 1986). This estimation assumes that the within-subject variability is equal to the between-subject variability.

1989). Similarly, the more the intervention is tailored to the individual patient, the more heterogeneous the resulting comparison groups. This heterogeneity also reduces the statistical power due to the increased variability in the outcomes. The increased cost and challenge of implementing such a large study might lead to considering the several design alternatives outlined below. 8.05.2.2.1 Needs assessment followed by randomized consent An alternative to the standard randomized controlled trial is the randomized consent design (Zelen, 1979, 1990). In this design, eligible patients are randomized prior to obtaining consent, and consent is obtained for participation in that arm to which the patient has been randomized (Figure 2(b)). This design might be beneficial in a clinical setting where the efficacy or cost-effectiveness of an adjunctive treatment which is a reimbursable clinical service is evaluated. In this case, ªreimbursableº refers to the idea that a third party (i.e., insurance company) would deem the service worthy of coverage by the individual's healthcare insurance. Such a concern is particularly relevant in countries which do not have

universal healthcare coverage (e.g., the USA). By using a randomized design, the statistical comparability of the two treatment arms is ensured, thereby reducing the systematic error encountered in more quasi-experimental designs. The primary caveat of the randomized consent design is that it is feasible only if participants will be highly likely to accept the treatment to which they have been randomized. This caveat is due to the intention-to-treat principle of randomized trials: that is, that patient data are analyzed according to the intervention to which they were assigned, regardless of how much of the treatment they actually received. This principle ensures that the more consistent the efficacy (i.e., effectiveness and feasibility) of an intervention, the more ecologically valid the representation of the realworld impact it is likely to have. To ensure that the participants will be highly likely to accept the treatment to which they have been randomized, implementing a needs assessment survey of clinic patients is suggested. This survey can be used to help determine the psychosocial needs and meaningful interventions for a given target population (Edwards & White, 1987). Data from such surveys can also be used to evaluate the sociodemographic and medical correlates of expressed needs (Kraft,

142

Design Issues for Clinical Research in Health Psychology

a. Standard Randomized Controlled Trial Data Collection (post)

Data Collection (pre) R A N D O M I Z E

Yes

PATIENT ELIGIBLE

Informed Consent

b. Randomized Consent Design

PATIENT ELIGIBLE

Treatment B

Dropped

No

NEEDS ASSESSMENT SURVEY

Treatment A

Data Collection (pre)

R A N D O M I Z E

Seek consent for observational study

Seek consent: Will you accept treatment A?

Data Collection (post)

Observation (control)

No

Dropped

Yes

Treatment A

Implementing Research in a Clinical Setting

c. Three-arm Crossover Design

Informed Consent

Data Data Data Data Collection Collection Collection Collection (pre) (post) (post) (post)

R A N D O M I Z E

}

PATIENT ELIGIBLE

143

a arm

arm b arm

Optional Washout Period

}

c

Time

Wait-list or standard therapy control Active intervention tailored to individual patient’s need

d. Case-control Design

Data Collection (post)

Yes PATIENT ELIGIBLE

Informed Consent

Patients matched by relevant sociodemographic and medical characteristics

Exposure to Intervention? No

144

Design Issues for Clinical Research in Health Psychology

e. Four-arm Pre-Post Design

PATIENT ELIGIBLE

Informed Consent

Data Data Collection Collection (post) (post)

R A N D O M I Z E

f. Pre-post One Group Design

PATIENT ELIGIBLE

Intervention

X

X

No Intervention

X

X

Intervention

X

No Intervention

X

Data Collection (post)

Data Collection (post)

Yes

Treatment A

No

Dropped

Informed Consent

Figure 2 Research designs which would be feasible to implement in an active clinical setting. (a) Schema for a standard randomized controlled trial for the sake of comparison with the subsequent designs. (b) Schema for a needs assessment survey followed by a randomized consent design. (c) Schema for the three-arm crossover design, which allows one to evaluate the impact of duration of treatment on outcomes. (d) Case±control design which would require matching patients on relevant sociodemographic and medical characteristics. (e) The Solomon four-arm pre±post design controls for maturation effects and investigates the impact of salience cuing on treatment outcomes. (f) Schema for a pre±post one-group design which is the simplest and least expensive study to implement. This study allows one to evaluate patient factors associated with better treatment outcomes.

Implementing Research in a Clinical Setting Freal, & Coryell, 1986). By beginning with a needs assessment survey, a needy target population can be identified for a specific trial who would be likely to want the intervention(s) being evaluated (Figure 2(b)). 8.05.2.2.2 Three-arm crossover design Given the difficulty in randomizing patients to a placebo±control condition, Schwartz et al. (1996) have proposed a randomized three-arm variation of a standard crossover trial. This design allows evaluation of treatment, order, and dose±response effects. In the first two arms, patients would receive active treatment followed by a control condition (arm ªaº in Figure 2c), or vice versa (arm ªbº in Figure 2(c)). The patients randomized to the third arm (arm ªcº in Figure 2(c)) would receive the active treatment for the entire duration of the other treatment arms (i.e., treatment and control). This arm would allow the investigator to address the longer-term effect of the intervention, without dramatically increasing the sample size. The control condition in the first two arms (i.e., arms ªaº and ªbº in Figure 2(c)) would be a wait-list which could involve standard care or no treatment. The decision about whether to use a standard care or no-treatment wait-list control would depend on the usual treatment approach for the study population and symptom complex. If there were a standard treatment for the symptom complex (e.g., pharmaceuticals, support groups, antidepressants, etc.), then the wait-list patients would have normal access to them. If the syndrome is one that does not interfere significantly with daily life and is not threatening to health then a no-treatment control would be appropriate. If the trial population is in crisis, then the control group would get standard care. Since crossover trials compare the impact of an active treatment to standard care and/or wait list control, the designs avoid the need to create a placebo±control. Thus, the three-arm crossover similarly minimizes the ethical dilemma by providing treatment to all patients. The enhanced statistical efficiency of the crossover design would allow the use of individuallytailored interventions, and thus increase the clinical validity of the trial results. Such treatment protocol heterogeneity would normally reduce the statistical power of the trial and lead to an expensive increase in the number of patients required to detect a difference between treatments. However, crossover designs use patients as their own controls and are thereby statistically efficient: they require approximately one quarter the number of

145

patients per study arm as those required for a standard parallel design (Figure 1, Fleiss, 1986). This estimation assumes that the within-subject is equal to the between-subject variability. This increased power would offset the reduced statistical power of the protocol heterogeneity. The proposed three-arm crossover design allows for an examination of dose±response relationships, whereas the standard crossover trial limits the investigator's ability to learn about the effects of longer-term intervention. The three arms provide information about the effects of shorter-term interventions (i.e., arm ªaº or ªbº) and about longer-term interventions (i.e., arm ªcº). The investigator might seek to evaluate the maintenance of effects over time (i.e., carryover effects) by shorter-term interventions using data from arm ªaº or design the study to include a washout period between treatment and follow-up so that the follow-up could be a clear no-treatment comparison condition. Consequently, treatment outcomes should be measured pre- and postintervention as well as before and after the washout and notreatment periods. Thus, the proposed design responds to the standard problem of crossover designs, that of not allowing the investigator to examine whether continued adherence to an intervention produces benefits over a long period of time or follow-up. This use of the three-arm crossover trial data is similar to an approach used by Thoresen (1991) to evaluate the long-term impact of a behavioral intervention for postinfarction patients who had participated in a crossover trial. They were able to show that the larger the ªdoseº received of the behavioral intervention, the greater behavior change and health benefit. Finally, the three-arm crossover design can allow the investigator to evaluate order effects, which may prove illuminating for hypothesis generation. For example, assume that condition ªaº is nonspecific support (e.g., social support), condition ªbº is coping effectiveness training (i.e., the experimental intervention), and condition ªcº is two cycles of the coping effectiveness training. The design is: AB, BA, and BB. Suppose that analysis of the results revealed that AB was better than BA and BB. The interpretation, framed as hypothesis generation, might be that social support prepared patients for the coping skills training, whereas coping skills training given first inadvertently increased resistance in some patients. Therefore, this three-arm crossover design could identify possible timing or staging effects of the different therapies. The appropriateness of the three-arm crossover trial will depend on the objectives of the research. If the intervention is not expected to

146

Design Issues for Clinical Research in Health Psychology

have extended carryover effects (e.g., acupuncture, massage), then no washout period will be required. If the intervention is expected to have brief carryover effects (e.g., pharmacotherapy), then a brief washout period may be necessary. If the intervention is a cognitive-behavioral treatment for stress management, then a washout period may not be feasible because it may not be possible to remove the effects of the exposure. If the objective is to study carryover effects, then a standard parallel-arm design would be appropriate (i.e., treatment vs. control with no crossover). For example, the experimental treatment, best standard care, and a wait-list control or usual care (if the latter is not the same as the comparison condition) could be compared. The latter design will be expensive to implement due to the sample size requirements in parallel-arm trials comparing active treatments (i.e., small effects sizes, see Figure 1). Further, it may not be feasible among proactive patients who refuse to participate if randomized to a wait-list control. The three-arm crossover design is a feasible and appropriate design if carryover effects are not the primary endpoints. It represents one of several possible approaches to design. 8.05.2.2.3 Case±control Borrowed from the field of epidemiology, the case±control design can be useful for health psychology research on interventions where some naturalistic assignment has occurred. In this design, patients are naturalistically ªexposedº to one of the interventions under study and followed over time to determine the effect of the intervention (Figure 2(d)). This design might be used in cases where similar patient groups at difference sites receive distinct interventions. For example, if two sites of a managed care organization had similar patient populations in a diabetes clinic, and only one of the clinics implemented a patient education self-management program. Patients from both sites could be followed over time to evaluate the patient education program. 8.05.2.2.4 Solomon four-group pre±post design This design allows the explicit consideration of external validity factors (Campbell, 1963). By paralleling a standard pretest±post-test control group design with experimental and control groups lacking the pretest (Figure 2(e)), the main effects of both the intervention and testing can be evaluated, as well as the interaction of the intervention and testing. This design consequently allows for an investigation of the impact of salience cuing on treatment outcomes. For

example, measuring an outcome requires asking patients to self-report about that outcome. Thus the outcome's salience may be increased as a function of study participation, thereby leading to a shift in internal standards (i.e., response shift as described subsequently) on behalf of study participants. The Solomon four-group design would provide a vehicle for examining this aspect of response shift in clinical health psychology research. 8.05.2.2.5 Pre±post one-group design Many investigators may not feel that they have the expertise to implement studies with more complex designs. Consequently, they rely on recruiting consecutive patients for treatment evaluation studies and asking patients to selfreport on various quality-of-life outcomes before and after the intervention (Figure 2(f)). Because there is no comparison group, such data are not appropriate for testing the effectiveness of an intervention. However, this design can facilitate correlative or predictive studies in health psychology which seek to understand which factors are associated with better outcomes. One example of this use of the pre±post one-group design was a study done by Mohr et al. (1996) in which they examined how patient expectations were related to treatment compliance among a cohort of multiple sclerosis patients. They found that adverse events and unrealistic expectations were important predictors of treatment compliance in 86% of patients initiating an immunomodulating therapy. This use of the pre±post one-group design exemplifies how clinically relevant information can be gleaned with this simple and feasible research design. 8.05.2.3 Selecting the Appropriate Study Design for Evaluating Treatment Intervention Effects: Balancing Research Questions and Assumptions The above five study designs are reasonable alternatives to randomized controlled trials for health psychology research. However, selecting the appropriate design will require balancing the research question being asked as well as the assumptions on which each design relies (Table 2). Although the first five study designs address the relative effectiveness of treatment A vs. treatment B, they differ in their assumptions about patient characteristics. For example, the standard randomized controlled trial assumes that patients will accept randomization to the control group, and will not seek alternative and similar treatments to the experimental group

Implementing Research in a Clinical Setting due to resentful demoralization (Cook & Campbell, 1978). In contrast, the randomized consent design only asks that patients accept the singular treatment option to which they have already been randomized. Thus, the former may take longer to accrue participants if the control group is obvious. On the other hand, the randomized consent design is only feasible if the researcher is fairly certain that patients will be likely to accept the treatment option to which they have been assigned. A common assumption of randomized designs is that the random allocation method results in groups which are similar on sociodemographic and medical characteristics. As this is not always the case, stratification may be necessary for the crucial characteristics. Covariate adjustment can be done in the analysis phase for other pretreatment differences which occurred despite randomization. Although covariate adjustment can be done in both case±control and pre±post one-group designs, the former would benefit from matching study participants on relevant sociodemographic and medical factors to enhance the comparability of the two groups. This matching will prevent confounding due to differential representation of various prognoses by study arms, a design problem which cannot be resolved with covariate adjustment. One example of such a problem would be comparing the impact of a behavioral intervention on people with epilepsy, and not matching for location and focality of brain lesions. It is well-known that patients with focal temporal lesions are more likely to be considered as candidates for neurosurgery (Lechtenberg, 1990), and hence have a better psychosocial prognosis than those with diffuse lesions in other parts of the brain. Not matching on such important prognostic factors can lead to unnecessary confounding and perhaps even selection bias. Given the similarities in research questions and assumptions across the several designs suggested, clinical investigators may need to base their research design selection on characteristics of the context in which they would like to initiate the research study. For example, randomized trials may not be feasible in settings in which there is no support staff to oversee the blind allocation of treatment or vigilant followup. In these contexts, the case±control or pre±post one-group designs may be optimal as they would allow one to dovetail data collection efforts to standard clinic visits and can address relevant, albeit limited, questions about treatment outcomes. The clinical investigator therefore has several design options which can facilitate outcomes research despite limited resources. Selecting an appropriate

147

design will require the careful balance of the research question one seeks to ask, the assumptions one is comfortable making, and the constraints imposed by the clinical context. 8.05.2.4 Practical Challenges to Implementing Research in a Clinical Setting Once an appropriate design has been selected, clinical investigators may wish to begin collecting data immediately. However, there are other practical challenges which they may confront. These challenges reflect the emerging financial and legal considerations which are playing an increasing role in healthcare. Highlighting a few of these considerations should make it easier for potential investigators to prepare for them. 8.05.2.4.1 Who pays for treatments being evaluated? The appeal of implementing research in a clinical setting is partially founded on the assumption that the cost of such research can be reduced by having it dovetail with standard clinical work. Indeed the goal of outcomes research is to integrate an evaluation process into standard clinical practice and to provide relevant and timely feedback to clinicians so that they might improve the quality of care provided. However, many healthcare payers (e.g., insurance companies) may consider that such attempts to evaluate clinical practice indicate that the practice is ªexperimentalº and therefore not reimbursable. Consequently, the investigator is faced with the following dilemma: if the payers think it is research, they may not cover it. Indeed studies of treatments which are already standard clinical practice might be more analogous to Phase IV (i.e., postmarketing) clinical trials than to Phase III (i.e., effectiveness) clinical trials. This tautological problem within outcomes research may be solved in various ways, depending on the resources available to the institution. For example, some institutions may opt for internal funding of clinical work being investigated, thereby not seeking reimbursement for clinical services. Others may opt for the pre±post onegroup design which may only require collecting data from patients during standard clinic visits. Others may hire independent private organizations to manage the data collection effort so that the apparent investigators are not obviously linked to the clinical facility. Whatever the solution, it will be advantageous for the clinical investigator to consider this tautological problem before initiating any research study.

148

Design Issues for Clinical Research in Health Psychology

Table 2 Feasible clinical health psychology research study designs: research questions and assumptions. Study design

Research question

Assumption

a. Standard randomized controlled trial

Is treatment A better than treatment B on x outcome(s)?

b. Randomized consent

Is treatment A better than treatment B on x outcome(s)?

c. Three-arm crossover

i. Is treatment A better than treatment B on x outcome(s)? ii. Does duration of treatment influence outcome(s)? Is treatment A better than treatment B on x outcome(s)?

Patients will accept randomization to a control group; randomization takes care of pretreatment group differences Patients will be likely to accept the treatment to which they have been randomized Washout period duration is known

d. Case±control e. Solomon four-group pre±post

f. Pre±post one-group

Is treatment A better than treatment B on x outcome(s)? Does measurement cue patients about salience and thereby influence treatment outcomes? What factors are associated with better outcome(s) of treatment A?

Groups are matched on relevant sociodemographic and medical variables All groups are similar at baseline on outcome variables

Relevant factors are measured

8.05.2.4.2 Incentives of capitated care

8.05.2.4.3 Legal considerations

Since health care becomes increasingly controlled by managed care, the incentives of this system must be considered by clinical investigators. The idea behind capitated care is that the provider is paid in advance for all medical care on a per patient (i.e., capitated) basis. Unlike the fee-for-service arrangement where the incentive is to provide more medical services, capitated care presents an incentive to minimize the medical services provided so that the cost per patient is reduced. Given this incentive structure, research which seeks to evaluate adjunctive treatments such as behavioral medicine interventions may face additional barriers. Such programs may reduce unnecessary medical care use in the future by attending to current psychological morbidities. These morbidities can lead to increased healthcare utilization (Browne, Aprin, Corey, Fitch, & Gafini, 1990), but treating them with adjunctive behavioral interventions might lead to a cost offset in the longer term (Friedman, Sobel, Myers, Candill, & Benson, 1995). However, managed care providers may be focused on short-term savings, while psychological interventions may appear to increase rather than reduce healthcare costs. Investigators may need to consider that short-term increases in utilization may appear to work against the incentives of the system. Consequently short- and longterm costs and outcomes should be addressed in clinical research studies.

Although informed consent has become an assumed step in any research process, there are still legal issues which may arise due to an increasingly fearful and litigious culture. Legal issues may play a larger role as clinical investigators are required to seek consultations from medical attorneys rather than focusing on the best research design to ask the research question. A real-life example of such a situation was one where researchers sought to evaluate the predictive value of inability to answer a standard quality-of-life questionnaire, the SF36 (Ware, Snow, Kosinski, & Gandek, 1993). Their original design was a standard case± control design which followed elderly patients in a primary care setting. Eligible participants were to be those patients who were unable to complete the questionnaire without missing data. Half of the patients were to be observed without additional intervention, and half were to be telephoned monthly by a clinical nurse to ensure that they adhered to treatment regimens and/or saw a physician if needed. After consulting with lawyers, however, the investigators were informed that such a design put the clinic at risk of malpractice: if one of the control patients died and a relative learned that the patient had participated in this study, the relative might sue the clinic for withholding treatment even though the efficacy of the treatment was exactly the focus of the investigation. The researchers were advised to change

Implementing Research in a Clinical Setting

149

their design so that half the patients got a preand post-test along with the telephone intervention. The other half provided data only at the post-test. Clearly such a design limited the investigatorsº ability to adjust statistically for baseline health status. However, the fear of possible litigation over an unproven treatment dominated the design selection. Future clinical investigators might be wise to consider the legal perspective, and to ensure informed consent so that scientific rigor does not lose priority to litigious concerns.

study ends, it is incumbent upon the investigator to make the implications of treatment termination explicit. Further, the investigator may need to provide access to social service providers who can direct outreach after study termination and identify resources which would be otherwise unavailable to patients. Such considerations may not be apparent in the initial review by the committee for the protection of human subjects and may arise in the course of co-investigator meetings to clarify study procedures.

8.05.2.4.4 Unavoidable costs of doing research

8.05.2.5 An Emerging Theoretical Challenge for Outcomes Research

No matter how much a clinical research study attempts to dovetail with standard clinical care, there are some costs which cannot be ignored. For a study to be rigorous and to maintain a reasonable response rate, some staff time will need to be devoted to the initial training as well as for implementing the Total Design Method (Dillman, 1978). Further, it may be necessary to pay for some of the clinicians' time for aspects of the research which do not strictly involve delivering clinical care. For example, some time will be needed for occasional meetings to confirm that the study procedures are clear and implemented correctly, for interpreting results of the data analysis, and for completing manuscript preparation for peer review. Finally, for studies implemented by mail or by telephone there will be costs associated with pretesting instruments, printing, postage, and telephone follow-up. Although these costs are minimal, there are necessary components of the research endeavor and should be considered before beginning the study. 8.05.2.4.5 Ethical considerations The integration into any research endeavor of review by the appropriate committee for the protection of human subjects and informed consent have become crucial steps for maintaining ethically sound research. Nonetheless, some ethical concerns may be raised which are beyond the scope of such processes, especially as research efforts seek to extend clinical service evaluations to lower income patient populations. One example would be research which seeks to evaluate differences in clinical effectiveness of a behavioral medicine intervention on higher vs. lower income patients. Although standard informed consent procedures might list the risks and benefits of such benign interventions, they would be unlikely to highlight the impact of treatment termination. When the referent patient population is unlikely to be able to afford continuing the treatment after the

An emerging area of interest and work by researchers from a broad range of disciplines is focused on understanding ªresponse shift phenomenon.º This term refers to the idea that individuals facing a significant health challenge may experience a change in internal standards, values and the meaning of quality of life (Breetvelt and VanDam, 1991; Schwartz & Sprangers, under review; Sprangers 1988, 1996; Sprangers et al., 1995; Sprangers, Rogemuller, VanDen Berk, Bowen, & VanDam, 1994). As both Heraclitus and Alphonse Karr noted long ago, change is a constant in life; yet there is an underlying and undeniable structure to personality (Funder & Colvin, 1991). Quality-of-life investigators have documented a type of inconsistency in self-reported health outcomes which is likely to have important implications for evaluating the impact of cognitively-based interventions. Acute health-state changes may have an impact on psychological morbidity, followed by accommodation and adaptation to the functional limitations imposed by the illness. This shift explains how an individual's life satisfaction may not be directly related to their functional status. Bach and Tilton (1994) found, for example, that individuals with tetraplegia who were dependent on a ventilator reported higher life satisfaction than tetraplegics who were able to breathe independently. Individuals facing a significant health challenge may scale down their expectations of health, may be more appreciative of the social resources which support their daily living activities, and may be making significant adjustments in the importance of life domains (Bach & Tilton, 1994). These individual values may also play an important role in determining the complex inter-relationships underlying quality of life. For example, satisfaction with one's functional status has been found to be related to psychological well-being only among those

150

Design Issues for Clinical Research in Health Psychology

individuals who viewed the abilities being evaluated as very important (Blalock, B. M. DeVellis, R. F. DeVellis, & Santer 1993). Interventions which improve social support may affect patient values, priorities, and appreciation of the resources they have (Norman & Parker, 1996). These social support interventions may allow them to maximize their quality of life despite important physical setbacks. Thus, it may be hard to differentiate change due to active interventions and truly improved functional status from change due to patient accommodation to level of function. Response shift represents a challenge to health researchers. It lies under the surface of measurement, camouflaged by an apparent lack of change in treatment outcomes. Intra-individual shifts in referents and priorities may mediate both well-being and functional status (Sprangers & Schwartz, under review). Understanding the predictive significance of this intra-individual variability is at the heart of meaningful outcomes measurement, and has the potential to lead to a paradigm shift in the many fields of investigation that rely on patient self-report. 8.05.3 SUMMARY This chapter reviewed various design issues which might facilitate implementing health psychology intervention research in an active clinical setting. Key considerations were described to improve the ability to detect treatment effects, and several research design options were presented to highlight how one's assumptions and research questions determine the optimal research design. Finally, response shift phenomenon was discussed to emphasize how internal standards, values, and an individual's concept of quality of life can be a dynamic process which is under the surface of current measurement techniques. This emerging construct should be considered in future clinical health psychology research, so that the full impact of interventions is more likely to be revealed in clinical research. ACKNOWLEDGMENTS The author would like to thank Elissa Laitin for her assistance in manuscript preparation. The project was supported by grant number R01 HS08582-01A1 from the agency for Health Care Policy and Research to Dr. Schwartz. 8.05.4 REFERENCES Bach, J. R., & Tilton, M. C. (1994). Life satisfaction and well-being measures in ventilator assisted individuals with traumatic tetraplegia. Archives of Physical Medicine

& Rehabilitation, 75, 626±632. Blalock, S. J., DeVellis, B. M., DeVellis, R. F., & Santer, S. C. (1997). Self-evaluation processes and adjustment to rheumatoid arthritis. Arthritis & Rheumatism, 31, 1245±1251. Bornstein, M., & Cohen, J. (1988). Statistical Power Analysis: A Computer Program. Hillsdale, NJ: LEA Software and Alternative Media. Breetvelt, I. S., & VanDam, F. S. (1991) Underreporting by cancer patients: The case of response-shift. Social Science and Medicine, 32, 981±987. Browne, G. B., Arpin, K., Corey. P., Fitch, M., & Gafni, A. (1990) Individual correlates of health service utilization and the cost of poor adjustment to chronic illness. Medical Care, 28, 43±58. Campbell, D. T. (1963) Experimental and quasi-experimental designs for research. Chicago: Rand McNally. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin. Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: Wiley. Edwards, M., & White, N. (1987). A cancer patient survey to help determine psychosocial needs, design, and implement meaningful interventions. Advances in Cancer ControlÐ15 Years of Progress, 248, 153±180. Francis, D. J., Fletcher, J. M., Stuebing, K. K., Davidson, K. C., & Thompson, N. M. (1991). Analysis of change: Modeling individual growth. Journal of Consulting and Clinical Psychology, 59, 27±37. Fleiss, J. L. (1986). The design and analysis of clinical experiments. New York: Wiley. Friedman, R., Sobel, D., Myers, P., Caudill, M., & Benson, H. (1995). Behavioral medicine, clinical health psychology, and cost offset. Health Psychology, 14, 509±518. Funder, D. C., & Colvin, C. R. (1991). Explorations in behavioral consistency: Properties of persons, situations, and behaviors. Journal of Personality and Social Psychology, 60(5), 773±794. Kazdin, A. E., & Bass, D. (1989). Power to detect differences between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 57, 138±147. Kraemer, H. C., & Thiemann, S. (1989) A strategy to use soft data effectively in randomized controlled clinical trials. Journal of Consulting and Clinical Psychology, 57, 148±154. Kraft, G. H., Freal, J. E., & Coryell, J. K. (1986). Disability, disease duration, and rehabilitation service needs in multiple sclerosis: Patient perspectives. Archives of Physical Medicine & Rehabilitation, 67, 164±168. Lechtenberg, R. (1990). Seizure recognition and treatment. New York: Churchill Livingstone. Mohr, D. C., Goodkin, D. E., Gatto, N., Neilley, L. K., Griffen, C., Likosky, W., & Stiebling, B. (1996). Therapeutic expectations of patients with multiple sclerosis upon initiating interferon beta 1-b: Relationship to adherence to treatment. Multiple Sclerosis, 2, 222±226. Morgan, D. L. (1988). Focus groups as qualitative research. Newbury Park, CA: Sage. Norman, P., & Parker, S. (1996). The interpretation of change in verbal reports: Implications for health psychology. Psychology and Health, 11, 301±314. Pocock, S. J. (1983). Clinical trials: A practical approach (p. 182). New York: Wiley. Schwartz, C. E. (1994, July). How do psychosocial interventions influence functional status in multiple sclerosis? Results of a randomized trial [Abstract]. Proceedings of the Third International Congress of Behavioral Medicine, Amsterdam.

References Schwartz, C. E. (under review). The psychosocial impact of two social support interventions: Results of a randomized trial. Schwartz, C. E., Chesney, M. A., Irvine, M. J., & Keefe, F. J. (1997). The control group dilemma in clinical research: Applications for psychosocial and behavioral medicine trials. Psychosomatic Medicine, 59, 362±371. Schwartz, C. E., & Rogers, M. (1994). Designing a psychosocial intervention to teach coping flexibility. Rehabilitation Psychology, 39(1), 57±72. Schwartz, C. E., & Sprangers, M. (under review). Methodological approaches for assessing response shift in longitudinal quality of life research. Slife, B. D., & Williams, R. N. (1997). Toward a theoretical psychology: Should a subdiscipline be formally recognized? American Psychologist, 52, 117±129. Sprangers M. (1988). Response shift and the retrospective pretest: On the usefulness of retrospective pretest±posttest designs in detecting training related response shifts. Amsterdam: SVO. Sprangers, M. (1996). Response-shift bias: A challenge to the assessment of patientsº quality of life in cancer clinical trials. Cancer Treatment Reviews, 22, 55±62.

151

Sprangers, M., Broersen, J., Lodder, L., Wever, L., Smets, E., & VanDam, F. S. (1995). The need to control for response shift bias in longitudinal quality of life research [Abstract]. Quality of Life Research, 4, 488. Sprangers, M., Rozemuller, N., Vanden Berk, M. B. P., Boven, S. V., & VanDam, F. S. (1994). Response shift bias in longitudinal quality of life research [Abstract]. Quality of Life Research, 3, 49. Sprangers, M., & Schwartz, C. E., (under review). Integrating response shift into health-related quality-oflife research: A theoretical model. Thoresen, C. E. (1991). Long-term results of recurrent coronary prevention project at eight years. Invited paper presentation at the First International Congress of Behavioral Medicine, Uppsala, Sweden. Ware, J. E., Snow, K. K., Kosinski, M., & Gandek, B. (1993). SF-36 Health survey: Manual and interpretation guide. Boston: The Health Institute. Zelen, M. (1979). A new design for randomized clinical trials. New England Journal of Medicine, 300, 1242±1245. Zelen, M. (1990). Randomized consent designs for clinical trials: An update. Statistics in Medicine, 9, 645±656