- Email: [email protected]
Using ethnographic methodology in substance abuse treatment outcome research
Journal of Substance Abuse Treatment 18 (2000) 1–8
Article
Using ethnographic methodology in substance abuse treatment outcome research Gerald J. Stahler, Ph.D.a,*, Eric Cohen, Ph.D.b a
Department of Geography and Urban Studies, 309 Gladfelter Hall, Temple University (025-26), Philadelphia, PA 19122, USA b Department of Sociology, Pennsylvania State University—Fayette Campus, P.O. Box 519, Uniontown, PA 15437, USA Received 28 August 1998; accepted 22 January 1999
Abstract The purpose of this article is to argue for a greater inclusion of ethnography as an adjunctive methodology within the context of traditional substance abuse treatment outcome studies. First we describe what is meant by “ethnographic methodology,” then discuss the various methodological elements of ethnographic research that are relevant to substance abuse research. A number of suggested applications of ethnographic research pertaining to a substance abuse treatment outcome context are then presented. The article concludes with a discussion about how ethnographic methods can enhance and broaden our understanding of important questions relating to substance abuse treatment processes and outcomes. © 1999 Elsevier Science Inc. All rights reserved. Keywords: Ethnography; Treatment outcome research; Qualitative
1. Introduction During the past 30 years, applied ethnographic methodologies have made significant contributions to the study of substance abuse and specific substance-abusing subpopulations (Agar, 1985; Akins & Beschner, 1980; Feldman & Aldrich, 1990). Although ethnographic research has contributed to public policy by increasing the field’s understanding of the nature of substance abuse, its epidemiology, and the experience and lifestyle of substance users, relatively little work has been done on utilizing ethnographic research methods to assist in evaluating treatment effectiveness. There does exist a limited number of ethnographic studies of substance abuse treatment, mainly descriptive accounts of the organization and processes of therapeutic communities (Anderson, 1992; Skoll, 1992; Sugarman, 1974; Weppner, 1983) and how various treatment ideologies influence practice in methadone maintenance clinics (Rosenbaum, 1985). The present article argues for the use of ethnographic methods as not only an important and useful methodological tool to study drug abuse and drug abusers, but also as a means for enhancing traditional quantitative outcome studies of treatment effectiveness. We will first describe what we mean by “ethnographic” methodology and then present
* Corresponding author. Tel.: 215-204-6939. E-mail address: [email protected] (G.J. Stahler)
an overview of the basic approaches of ethnographic field methods that can be utilized within the context of treatment outcome studies. We will then describe a number of specific applications of ethnography that can be incorporated within the context of traditional treatment outcome studies. Some examples will be drawn from the published literature and others from our own work. The latter will be drawn primarily from several ethnographic studies that were conducted as part of a randomized field experiment funded by the National Institute of Alcohol Abuse and Alcoholism (NIAAA) and the National Institute on Drug Abuse (NIDA) that assessed the effectiveness of alternative treatment modalities for homeless, male, crack-cocaine users (Stahler, Shipley, Bartelt, & DuCette, 1995). 2. Ethnography as methodology 2.1. Defining ethnography Although some ethnographers will consider our definition as too broad, we use the term ethnography to refer to different qualitative methodologies that all relate to the process of doing fieldwork. Ethnography is essentially the methodological framework utilized to characterize all qualitative research, from unstructured and semi-structured interviewing, to extensive observation to obtain an insider’s understanding of the social context. Ethnographic methodology in its broadest sense can refer to any qualitative method that involves participant observation, nonparticipant observa-
0740-5472/00/$ – see front matter © 1999 Elsevier Science Inc. All rights reserved. PII: S0740-5472(99)00 0 2 9 - X
2
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
tion, or qualitative interviewing. The common thread that underlies these methodologies is the attempt to see the world from the point of view of the research participants, to attempt to understand their reality from their perspective within their natural environment or social context (Brooks, 1994; Glaser & Strauss, 1967). As Jennifer James stated, ethnography is “the study of a culture from within” (cited in Walters, 1980, p. 17). In traditional ethnography, the ethnographer typically spends time with members of the group he or she is attempting to study, observing what goes on, taking notes, and asking questions based on these observations and analysis. Within this context, research subjects are generally referred to as “key informants.” They are “insiders” from the point of view of either staff or user populations. Hypotheses are generated through inductive analysis; underlying theories are grounded in the observations, interpretations, and analysis extracted from the field or people one is studying (Glaser & Strauss, 1967). The major advantage of ethnographic methods is the ability to obtain in-depth and detailed information from the point of view of the research participant that is not constrained by predetermined categories (Patton, 1990). If trust and rapport with research respondents are established, ethnographers are often able to elicit rich information that is usually out of the reach of other methods (Walters, 1980). In general, ethnographic research methods usually include some type of observational method, some form of interviewing, a sampling strategy for selecting informants, and a thematic analysis and integration of this information. These areas will be briefly described below. 2.2. Observational methods Observational methods may involve either participation or nonparticipation by the researchers within the context that is being studied. 2.2.1. Participant-observational research This is the most widely used general ethnographic approach employed in the study of substance abuse with both treatment and street-based populations. Within a treatment setting context, this method involves observing and interacting with participants, and represents a combination of observing and informal interviewing. This may occur during treatment activities (e.g., during group counseling sessions) or during periods of unstructured time between activities. Systematic, participant-observationally based research often generates questions that may be posed in more formal, qualitative interviews. Some participant-observational ethnographic studies have been conducted by ethnographers who have a legitimate role in the program. For example, Skoll (1992) conducted his extensive ethnography of a midwestern therapeutic community from the vantage point of a program counselor. Both clients and staff were aware of his dual role of counselor and researcher.
2.2.2. Nonparticipant observation In terms of conducting ethnographic field studies of treatment programs, the goal is often to understand the program or intervention under study as thoroughly and comprehensively as possible. The nonparticipant-observer role usually involves intensive observation of a particular field setting without providing a tangible service to the treatment program. The goal of nonparticipant observation in drug treatment settings is often to develop an understanding of how a program functions, as well as the impact of the program on the participants. Predefined hypotheses are usually outside the range of concern of the observing ethnographer. The ethnographer in this capacity observes phenomena in order to elicit what is important in a general sense, and how specific details fit into the overall understanding of the program. The avoidance of a priori hypotheses and predefined conceptual schemes, and remaining open to many possible interpretations of the particular milieu one is studying are important contributions of ethnographic observational field studies (Brooks, 1994). Nonparticipant-observer approaches to gaining an understanding of a treatment setting are viable when the presence of researchers do not substantially alter the culture of the program. Sometimes the mere presence of observers can lead both staff and clients to change their behavior to meet the expectations of the observer (e.g., the Hawthorne Effect). Ethnographic observers can make staff and clients nervous, thus altering the natural flow of interaction in the particular setting. This type of disruptive effect can be minimized through careful planning and discussion with program staff and clients. In some of our work that has involved the use of trained graduate student ethnographers (i.e., Stahler, Cohen, Greene, Shipley, & Bartelt, 1995), the observers were viewed by program staff as rather benign and unobtrusive in each milieu they observed. This was probably because staff and clients were informed of their presence prior to their observation, they were relatively young and not viewed as authority figures, and they had already established rapport with both the treatment staff and the respective client populations in their role as interviewers for the quantitative portion of the main study. 2.3. Qualitative interviewing Qualitative interviews are typically open-ended, unstructured or loosely structured, survey formats designed to elicit detailed material about the treatment setting, or the impact of specific elements of the program on the client population. The interviewer attempts to assist informants to use their own words, values, and thoughts when answering questions (Patton, 1990; Posavac & Carey, 1997). As opposed to highly structured quantitative interviews, qualitative interviewers typically use questions that encourage the study informant to talk and elaborate about a particular concern. For example, ethnographers often conduct in-depth semi-structured interviews to obtain rich detail about the
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
backgrounds and biographies of addict populations. This type of research is often called life-history interviews. The advantage of a more ethnographically oriented interview schedule is that its primary purpose, independent of gaining a detailed understanding of the addicted client’s way of life, is as an exploratory heuristic device. The life-history interview, for example, is often governed by broad life-cycle conceptual categories (e.g., school experiences, family background, drug “careers,” and survival strategies on the street), or a timeline that the interviewee can use to anchor meaningful life events. The advantage here is the overall flexibility of the interview design and the ability of the interviewer to develop new questions, or change the course of the interview if the information warrants such a change. In addition to the importance of rich, descriptive case-oriented data, ideas gleaned from life-history interviews can be utilized to better articulate (a) cause-and-effect relationships related to substance abuse onset (e.g., traumatic events that trigger substance use); (b) problematic patterns of use and mitigating factors that may have influenced an abusive pattern (e.g., the influence of a significant relationship on the participant’s substance use); (c) factors that may have influence on the motivation to seek treatment (e.g., being fired from one’s job, eviction from an apartment); (d) or an assessment of the impact in qualitative terms of a particular drug treatment, or particular elements of the program on one’s life. 2.4. Sampling methods Because ethnographic research methodologies are relatively labor-intensive and usually involve studies that permit the researcher to examine social phenomena in-depth, samples tend to be small compared to more quantitative designs that produce information with greater breadth. As a result, deciding on whom to interview or observe is of critical importance in ethnographic research. Random sampling to achieve representativeness of the population is rarely done because of the large numbers required. Consequently, ethnography generally uses a variety of purposive sampling strategies, including extreme case sampling, typical case sampling, chain referral or “snowball” sampling, and convenience sampling (Patton, 1990). 2.4.1. “Extreme” case sampling This involves sampling cases that are unusual in some way. For example, case studies of clients who achieved extreme success in treatment may provide insights into how addicts can overcome their addiction (Stahler, Cohen, et al., 1995). A second example would be to identify clients who might be considered treatment “failures,” to better understand why a program had such little positive impact on them (Stahler, Cohen, Shipley, & Bartelt, 1993). Selecting clients to be interviewed can be done either through the use of quantitative data using empirically derived criteria (e.g., low scores on a treatment outcome measure), qualitatively
3
through information from such key informants as treatment staff, or a combination of the two. 2.4.2. “Typical” case sampling This type of strategy is generally used to illustrate rather than make definitive conclusions about a treatment program or its participants. For example, a case study of what a typical client experiences as he or she progresses through treatment may be useful to describe the treatment program. Cases are selected either from information provided by key informants concerning what they feel constitutes “typical” cases, or through quantitative data on program participants. Optimally, both methods should be utilized. Unfortunately, it is often difficult to define what constitutes a “typical” case. The notion behind selecting “typical” cases is to provide information about clients who are “average” or roughly representative of the general population. However, as described above, random sampling is usually not appropriate, given the small numbers to be selected. Therefore, typical cases might be selected on the basis of certain predefined categories or subpopulations that may be meaningful within the study context, such as older clients, women with children, the dually diagnosed, or the homeless. The ethnographer can select categories of clients whose experience and background may be different, and then select within those strata. It must be remembered, however, that qualitative information gleaned from this interview strategy is not meant to be conclusive as in a quantitative paradigm, but rather the object is to deepen one’s understanding, to develop new insights, and to point to further questions that may be worthy of study about these client populations. 2.4.3. Chain referral or “snowball” sampling Chain referral sampling techniques are typically utilized to gain access to relatively hidden populations of drug users. This sampling approach allows researchers to gain entry to a particular subculture of street users or other difficult to access substance-abusing populations through the use of key informants. By utilizing the social networks of drug users in the community or those already in treatment, the ethnographer can acquire an adequate sample of a population that may not otherwise be accessible to quantitatively oriented researchers. While this technique is not directly related to treatment outcome assessment, it is commonly used in developing samples of “hidden” or street-based substanceabusing populations and was necessary in our own investigation of homeless substance abusers. In our study of treatment dropouts (Stahler et al., 1993), we utilized a chain referral technique to locate clients who dropped out of treatment. They were found in various domains related to the homeless lifestyle, such as sections of the city where homeless people frequented, homeless settlements on the street, drop-in centers, soup kitchens, and the like. Through the use of informants who were still in treatment, we were able to find, interview, and help men reenter treatment, thus improving the efficacy of the overall project, and helping to
4
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
maintain an effective sample size for the quantitative outcome assessment. 2.4.4. Convenience sample This is the least desirable, but perhaps the most common type of sampling strategy, where informants are selected by convenience. An example would be to interview clients in a treatment program according to whomever is most easy to approach. In this strategy, there is no strategic basis used to select informants (Patton, 1990). 2.5. Data analyses Observations, impressions, and informant interviews are usually either in the form of field notes or tape-recorded and then transcribed. The main task of analyzing qualitative information is to identify the primary patterns in the data (Patton, 1990). Generally, some form of content analysis is used where the field notes or interviews are coded and categorized according to certain themes that are inductively derived by the analyst. Usually, direct quotations are used to illustrate the themes. These quotations not only help substantiate the themes, but also serve to bring the reader closer to the direct experience of the participant’s perspective. Reading direct quotations in the participant’s vernacular also has the benefit of reminding the research audience of the “subject’s” humanity, a perspective that is too easily forgotten in the course of conducting research. In our ethnographic study of treatment success (Stahler, Cohen, et al., 1995), we asked clients who were identified by counselors as having made considerable progress in treatment to what they attributed their success. After a thorough review of the interview transcriptions, field notes, and other supportive documents, we then set up indices of key conceptual areas; integrated our conceptual impressions; shared our interpretations with staff, clients and research staff in the relevant settings; and then cross-checked their interpretations with our findings. Their responses were categorized inductively to yield a variety of themes, which will be described later. There does exist relatively sophisticated qualitative data analysis software that performs this task of analyzing qualitative information interactively on a personal computer. Some of the more popular packages are ETHNOGRAPH, HyperRESEARCH, and NUD*IST 4 (Non-numerical Unstructured Data-Indexing, Searching and Theorizing). 3. The application of ethnographic methods in substance abuse treatment outcome research In general, the ethnographic methods described above can be applied to several areas of substance abuse research that relate to better understanding treatment outcomes. Examples of how ethnographic methodology can complement quantitative evaluation methods and provide important insights into evaluating substance abuse treatment programs focus on the following four areas: epidemiology, client and
staff satisfaction and perceptions of treatment, treatment dropout, and the social environment of the treatment program. In addition, ethnographic methods can assist in broadening our ability to interpret the results of treatment outcome research (see, e.g., Stahler & Cohen, 1995). 3.1. Epidemiology Ethnographic research has been particularly helpful in defining, augmenting, and clarifying an epidemiological understanding of substance use and abuse. For example, ethnography has helped identify the epidemiological connection between crack-cocaine addiction and HIV infection (Inciardi, Lockwood, & Pottieger, 1993). Over the last decade or so, ethnography has also made major contributions in gathering information concerning ways to reach and provide acceptable outreach to hidden and high risk populations such as intravenous drug users and crack-using women who engage in sex-for-crack exchanges (Ratner, 1993). Other ethnographic studies have been useful in identifying how some heroin addicts recover without treatment and why many addicts do not seek treatment (Biernacki, 1986). Ethnographic research on the epidemiology of substance abuse, as shown by these examples, can help define hypotheses, which can then be tested using experimental designs. They also help inform program planners and treatment staff. However, there are other applications of ethnography that can be utilized more directly within the context of treatment outcome research. 3.2. Client and staff satisfaction and perceptions of treatment Of particular importance in evaluating the success of the program is the level of acceptability and satisfaction with the program on the part of both clients and staff. If service providers resist a particular treatment approach, then it is unlikely that it will be implemented effectively. Quantitative scales that assess satisfaction and perceptions of treatment may be one of the most common evaluation approaches undertaken by treatment programs. However, qualitative approaches have the advantage of providing a more open-ended and thorough means of understanding from the staff’s and client’s perspective—what they like about the treatment program, what they do not like, and suggestions for improvement. Quantitative scales may produce a summary picture of the level of satisfaction with treatment and the treatment environment, but it will generally not provide specific information on why clients may be dissatisfied with their treatment experience, or why they may feel particularly positive about it. Open-ended, qualitative questions administered during treatment may elicit important information on client concerns, which, if addressed, may reduce treatment attrition. It may also be important to understand how staff perceive the treatment program. Staff morale may impact upon the quality of services provided and may affect the imple-
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
mentation of treatment under study. The process of asking clients and staff about their reactions to the treatment program also has the advantage of letting them feel that their point of view has been heard, which in itself may be of value. Assessing client and staff perceptions offers an especially important source of feedback for treatment interventions that are newly implemented or are in their exploratory phase of development. This type of qualitative assessment may be particularly useful when treatment strategies that are found to be effective in research settings are tested in community-based treatment settings. 3.3. Assessing treatment attrition In studying the effectiveness of substance abuse treatment interventions, ethnography can be employed to document how programs are actually implemented. Observation and interviews with staff and clients can provide information concerning barriers to optimal implementation, as well as qualitative information concerning the fidelity by which the interventions are implemented as intended. For example, in a case management intervention, quantitative indicators can be developed to assess the number and patterns of referral, and whether the clients actually receive the services to which they are referred. However, this method provides little information pertaining to why clients decide whether or not to go to referred services, or why case managers refer or do not refer for certain services. Another example concerns treatment attrition. Studies that have assessed client dropout from substance abuse treatment have generally focused upon quantitative measures that attempt to determine what types of clients drop out or stay, or what types of characteristics best predict client dropout (Stahler et al., 1993). However, understanding why they drop out may provide valuable information that can be fed back into improving the program. For example, in our ethnographic study of treatment attrition mentioned earlier (Stahler et al., 1993), we were able to find and interview a sample of 34 homeless clients who willingly left treatment prior to graduation. As mentioned earlier, we did this through checking lists at shelters and visiting locations frequented by the homeless. Clients were asked why they had dropped out of treatment, at what point in the program did they drop out, and to describe the events that led up to when they left the program. We found that there were often multiple reasons and most frequently a combination of “pushes” and “pulls” from external forces, factors relating to the treatment and shelter environment, and individual propensities. The most common reason concerned a basic lack of motivation and interest in obtaining treatment for their addiction. Others were dissatisfied with the intervention group to which they were randomly assigned. Some clients who wanted less structure ended up in the highly structured residential treatment group, whereas others who wanted more intensive services were assigned to the shelter-based case-management group. This mismatch
5
between client preferences and assigned treatment intervention resulted in many occurrences of attrition. The restrictiveness of the program structure, the culture and climate of the shelter, altercations with staff and clients, and exogenous factors, such as being drawn away by a job, girlfriend, or family members were also commonly mentioned reasons for leaving the program. As a result of this ethnographically derived information, a number of programmatic changes consistent with the original research design were made. The primary benefit of our ethnographic interviews was that it informed us about why clients were dropping out from their perspective in a more in-depth way than a more closedended, quantitative assessment would have captured. We learned to see from the clients’ reality and not from our preconceived notions about why we thought they were leaving the program. 3.4. Descriptions of program context A major question in disseminating treatment approaches to other programs is whether the treatment approach can be successfully replicated in different contexts with different program managers, staff, local conditions, and populations (e.g., external validity). Qualitative information can be useful in describing the environmental context and how this relates to the program implementation. Ethnographic descriptions of the community context can be obtained from key informants in the community or treatment agency, as well as from observation in the community. In addition, information from key informants and from extant documents can describe the current existence of other human service programs and resources in the community, and external events, such as changes in drug use patterns (e.g., introduction of new drugs), that may affect the research. 3.5. Understanding the social environment of the program While it is possible to specify a treatment intervention in some detail and to utilize manuals to assist in the standardizing of a substance abuse intervention, the actual “culture” or social environment of a treatment setting may lead to important differences in actual service delivery and outcomes, even within the same treatment modalities. For example, methadone clinics are often viewed as a single category of treatment intervention for heroin addiction. However, Rosenbaum’s (1985) ethnographic study of methadone clinics showed that there is considerable variability in the treatment “cultures” of clinics, particularly in regard to level of client control and treatment philosophy. The client’s relationship to the more humanistic, nonpunitive, “consumer”oriented models that allows clients to participate in their own treatment had more satisfied clients than the more punitive, authoritarian treatment models where patients were often viewed as hopelessly addicted and psychiatrically impaired. Ethnography can be utilized to assess how a treatment program’s culture develops from its origin, and to describe
6
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
and analyze the organizational structure of the treatment setting (Johnston, Rowe, & Swift, 1995). When the quantitative data are analyzed longitudinally, changes in client behavior and treatment model effectiveness can be joined with process ethnographic data regarding the dynamics of program structure and operation. As a result, a more complete and integrated understanding emerges. Not only does this result in a more systematic and thorough outcome assessment, but it also introduces new ways to conceptualize the treatment intervention from its original design (Brooks, 1994). The involvement of ethnographers in the treatment setting also presents the possibility of fostering closer relationships with its staff and client populations, thus allowing for smoother implementation of the research protocol 3.6. Helping to interpret treatment outcomes Ethnography can assist in the interpretation of quantitative data on treatment outcomes because of this methodology’s relatively open-ended nature and its focus on client experience. This may be particularly valuable in situations where no differences among groups are found. In well-conducted, rigorous treatment outcome studies using experimental designs, finding significant differences between control and experimental groups as hypothesized leads to relatively few difficulties in interpreting findings. However, treatment outcome studies conducted in the field frequently do not produce such straightforward results. Often some hypotheses are supported, and others are not. Some outcome variables may be significantly improved, and others not. Ethnography can often shed light on interpreting mixed findings or unexpected results. For example, in our outcome study on homeless, male, crack users (Stahler, Shipley, et al., 1995), the quantitative outcome data did not show statistically significant differences among the three different interventions at follow-up, even though all three groups did improve from baseline to follow-up. Exploratory field immersion by our ethnographers, as well as interviews with “successful” clients, provided some insights into why client improvement rates were not different among the three groups. While all three programs represented different therapeutic levels to address addiction, clients in all the groups had the opportunity to utilize 12-step meetings on a daily basis. Based on both quantitative services data as well as ethnographic interviews with staff and client informants from all three programs regarding treatment success, it was found that almost all successful clients reported attending Narcotics Anonymous (NA) and Alcoholics Anonymous (AA) meetings on a regular basis, and attributed aspects of 12-step programming (i.e., the elements they also offer, such as community, support, identity, etc.) as key ingredients to why they were progressing in their movement away from drugs and alcohol (Stahler, Cohen, et al., 1995). In addition, we also discovered that the services provided by city employees in the usual care condition (control group), involving standard shelter-based case-management services, had unexpectedly improved during the course of the study.
As a result of knowing that they were involved in the study, case managers appeared to become more responsive to clients than had been the case prior to the study. A second contribution of this ethnographic research to the same study concerned interpreting treatment outcomes. Our ethnographic research helped to identify both individual and programmatic factors that related to positive outcomes (e.g., within group differences). As mentioned earlier, clients were interviewed concerning their own attributions for why they were “successful.” This ethnographic effort was parallel to one of the project’s goal of identifying through multivariate analyses individual and program factors that were predictive of positive outcomes. The most salient theme identified was the role of self-motivation. This is certainly consistent with the burgeoning literature on the critical importance of client motivation to treatment outcomes (e.g., DeLeon & Jainchill, 1986; Miller & Rollnick, 1991; Prochaska, DiClemente, & Norcross, 1992). However, treatment intervention factors, such as the program structure, 12-step meetings, and counseling and support from staff, were also described by clients as essential to their progress, as was social support from other clients and family. According to most of the clients, it was not any one single factor alone that made a difference in their lives, but rather a combination of several of these factors that were essential for their progress. The results underscored that selfmotivation is often not sufficient to enhance outcomes, but needs to be nurtured with responsive treatment programming and social support from other clients, staff, and family and friends. The importance of social support was further supported by the quantitative portion of the study that examined predictors of success using multivariate analyses. 4. Discussion As we have attempted to show, the true virtues of ethnographic applications to in and out-of-treatment based substance abuse populations are the rich descriptions based on the user’s perspective and their overall interpretive power. Ethnography provides a useful adjunctive and complementary methodology to traditional quantitative substance abuse treatment research approaches. As a heuristic device, ethnography can aid in the research discovery process in helping to define and demarcate the phenomenology of the setting one is studying. The development of an in-depth understanding of the treatment process, the “real world” application of a particular treatment intervention, or attempting to isolate the factors in treatment that may positively or negatively affect treatment outcome, are all areas that benefit significantly from ethnographic methodologies. In-depth key informant interviews with clinical staff, clients, or administrative personnel can often shed light on problems or intricacies of the research project, which are typically hidden from the standard quantitative research strategies. Observational studies of treatment cultures can often lead to a clearer, more focused understanding of why certain clients
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
change and others do not, the contextual contingencies that may affect program compliance, or reasons why particular clientele may leave programs prior to completing treatment. The perspective of the substance abuser, particularly feelings toward treatment, staff, and the outcome assessment process itself, are useful in evaluating a particular treatment’s effectiveness. Studying the treatment and research process can often help disentangle cause and effect issues related to defining progress in treatment, or the isolation of barriers that may inhibit client progression in the program. However, ethnographic methodology does have its limitations. Field studies in natural settings by their very nature are not designed to control variables and, therefore, are not amenable to testing research hypotheses. Experimental methodologies using quantitative measures are, without question, the most effective means of demonstrating causal relationships and are best able to formally test a priori hypotheses. Hypothesis testing warrants a rigor and a control of conditions that ethnography cannot satisfy. However, ethnographic research can contribute more directly to quantitative research as questions that emerge in the discovery process become more focused and clear. While inductive approaches like ethnography are weak at demonstrating causal relationships, they excel at detecting general and specific patterns among social phenomena in field settings. For example, once ethnographic impressions of a treatment setting become more developed, consistent, and tangible patterns emerge, it may be then possible to form testable hypotheses about the expected pattern of certain quantitative measures relating to treatment outcome. A second frequently mentioned criticism is that ethnographic research is case-oriented and, therefore, uses small samples. As a result, the data may be idiosyncratic, not likely to be representative, have questionable validity, and will probably not be generalizable to other study settings or populations (Walters, 1980). This question of validity of ethnographic research was addressed by Donald Campbell. He argued that ethnographic research methods are particularly useful in obtaining valid and reliable data for three reasons: (a) the ethnographer’s familiarity with the respondent’s vernacular, (b) the ethnographer’s length of interaction in the setting, and (c) the ethnographer’s degree of participation in the respondent’s world (Walters, 1980). There is a long tradition of debate in the social sciences concerning the use of quantitative versus qualitative research approaches, and the relative validity of these approaches (e.g., Reichardt & Rallis, 1994), which will not be resolved here. Our position is that each looks at different aspects of social phenomena and that they are complementary. We believe that ethnography has a useful place, and should be routinely incorporated, within the context of traditional quantitative substance abuse treatment outcome research. Ethnographic methodology can be utilized within a traditional quantitative research paradigm to enlarge and broaden our understanding of treatment processes and outcomes. By attempting to understand how clients view and experience addiction
7
and addiction treatment programs in a more in-depth way and from the participants’ point of view, we can develop and improve existing interventions to be more responsive to client needs. We can also generate more meaningful hypotheses that are “grounded” in the subjective realities of clients and yet are still testable within an experimental paradigm. By understanding what “really went on” in treatment from an “insider’s” point of view (a perspective usually lacking in most outcome studies), interpretation of either significant or nonsignificant findings can be greatly facilitated. In the main, ethnography’s primary substantive contribution to the treatment outcome assessment field is to help provide a fuller, more accurate, and humanistic view of substance abuse, substance abusers, and the process and impact of substance abuse treatment. The question is not whether to use quantitative or qualitative methods; but rather how to use different methods so that they complement each other in the most effective ways possible (Posavac & Carey, 1997). As Light and Pillemer (1984) conclude: “The pursuit of good science should transcend personal preferences for numbers or narrative” (p. 143). References Agar, M. (1985). Folks and professionals: different models for the interpretation of drug use. The International Journal of Addictions 20, 173–182. Akins, C., & Beschner, G. (Eds.). (1980). Ethnography: A Research Tool for Policymakers in the Drug and Alcohol Fields (DHHS Pub. No. [ADM] 80-946). Rockville, MD: National Institute on Drug Abuse. Anderson, E. N. (1992). A healing place: ethnographic notes on a treatment center. Alcoholism Treatment Quarterly 9, 1–21. Biernacki, P. (1986). Pathways from Heroin Addiction Without Treatment. Philadelphia, PA: Temple University Press. Brooks, C. R. (1994). Using ethnography in the evaluation of drug prevention and intervention programs. The International Journal of the Addictions 29, 791–801. DeLeon, G., & Jainchill, N. (1986). Circumstances, motivation, readiness, and suitability as correlates of tenure in treatment. Journal of Psychoactive Drugs 18, 203–208. Feldman, H. W., & Aldrich, M. R. (1990). The role of ethnography in substance abuse research and public policy: historical precedent and future prospects. In E. Lambert (Ed.), The Collection and Interpretation of Data from Hidden Populations (NIDA Research Monograph No. 98). Rockville, MD: National Institute on Drug Abuse. Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine. Inciardi, J. A., Lockwood, D., & Pottieger, A. E. (1993). Women and Crack-Cocaine. New York: Macmillan. Johnston, P., Rowe, M., & Swift, P. (1995). Dilemmas of human service reform in New Haven: integrating three levels of organizational analysis. Contemporary Drug Problems 22, 363–391. Light, R. J., & Pillemer, D. B. (1984). Summing Up: The Science of Reviewing Research. Cambridge, MA: Harvard University Press. Miller, W. R., & Rollnick, S. (1991). Motivational Interviewing: Preparing People to Change Addictive Behavior. New York: Guilford Press. Patton, M. Q. (1990). Qualitative Evaluation and Research Methods (2nd ed.). Newbury Park, CA: Sage. Posavac, E. J., & Carey, R. G. (1997). Program Evaluation: Methods and Case Studies. Englewood Cliffs, NJ: Prentice-Hall Books. Prochaska, J. O., DiClemente, C. C., & Norcross, J. C. (1992). In search of how people change: applications to addictive behaviors. American Psychologist 47, 1102–1114.
8
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
Ratner, M. S. (Ed.). (1993). Crack Pipe as Pimp: An Ethnographic Investigation of Sex-for-Crack Exchanges. New York: Lexington Books. Reichardt, C. S., & Rallis, S. F. (Eds.). (1994). The Qualitative-Quantitative Debate: New Perspectives. New Directions in Program Evaluation, No. 61. San Francisco: Jossey-Bass. Rosenbaum, M. (1985). A matter of style: Variation among methadone clinics in the control of clients. Contemporary Drug Problems 13, 375–397. Skoll, G. (1992). Walk the Walk and Talk the Talk: An Ethnography of a Drug Abuse Treatment Facility. Philadelphia, PA: Temple University Press. Stahler, G. J., & Cohen, E. (Eds.). (1995). Homelessness and substance abuse in the 1990s: qualitative studies from service demonstration projects. Contemporary Drug Problems 22, 169–391. Stahler, G. J., Cohen, E., Greene, M., Shipley, T., & Bartelt, D. (1995). A qualitative study of treatment success among homeless crack-addicted men: definitions and attributions. Contemporary Drug Problems 22, 237–264.
Stahler, G. J., Cohen, E., Shipley, T., & Bartelt, D. (1993). Why clients drop out of treatment: ethnographic perspectives on client attrition among homeless male “crack” cocaine users. Contemporary Drug Problems 20, 651–680. Stahler, G., Shipley, T., Bartelt, D., & DuCette, J. (1995). Evaluating alternative treatments for homeless substance-abusing men: outcomes and predictors of success. Journal of Addictive Diseases 14, 151–167. Sugarman, B. (1974). Daytop Village: A Therapeutic Community. New York: Holt, Rinehart, and Winston. Walters, J. M. (1980). What is ethnography? In C. Akins & G. Beschner (Eds.), Ethnography: A Research Tool for Policymakers in the Drug and Alcohol Fields (pp. 15–20) (DHHS Pub. No. [ADM] 80-946). Rockville, MD: National Institute on Drug Abuse. Weppner, R. (1983). The Untherapeutic Community. Lincoln, NE: University of Nebraska Press.
Article
Using ethnographic methodology in substance abuse treatment outcome research Gerald J. Stahler, Ph.D.a,*, Eric Cohen, Ph.D.b a
Department of Geography and Urban Studies, 309 Gladfelter Hall, Temple University (025-26), Philadelphia, PA 19122, USA b Department of Sociology, Pennsylvania State University—Fayette Campus, P.O. Box 519, Uniontown, PA 15437, USA Received 28 August 1998; accepted 22 January 1999
Abstract The purpose of this article is to argue for a greater inclusion of ethnography as an adjunctive methodology within the context of traditional substance abuse treatment outcome studies. First we describe what is meant by “ethnographic methodology,” then discuss the various methodological elements of ethnographic research that are relevant to substance abuse research. A number of suggested applications of ethnographic research pertaining to a substance abuse treatment outcome context are then presented. The article concludes with a discussion about how ethnographic methods can enhance and broaden our understanding of important questions relating to substance abuse treatment processes and outcomes. © 1999 Elsevier Science Inc. All rights reserved. Keywords: Ethnography; Treatment outcome research; Qualitative
1. Introduction During the past 30 years, applied ethnographic methodologies have made significant contributions to the study of substance abuse and specific substance-abusing subpopulations (Agar, 1985; Akins & Beschner, 1980; Feldman & Aldrich, 1990). Although ethnographic research has contributed to public policy by increasing the field’s understanding of the nature of substance abuse, its epidemiology, and the experience and lifestyle of substance users, relatively little work has been done on utilizing ethnographic research methods to assist in evaluating treatment effectiveness. There does exist a limited number of ethnographic studies of substance abuse treatment, mainly descriptive accounts of the organization and processes of therapeutic communities (Anderson, 1992; Skoll, 1992; Sugarman, 1974; Weppner, 1983) and how various treatment ideologies influence practice in methadone maintenance clinics (Rosenbaum, 1985). The present article argues for the use of ethnographic methods as not only an important and useful methodological tool to study drug abuse and drug abusers, but also as a means for enhancing traditional quantitative outcome studies of treatment effectiveness. We will first describe what we mean by “ethnographic” methodology and then present
* Corresponding author. Tel.: 215-204-6939. E-mail address: [email protected] (G.J. Stahler)
an overview of the basic approaches of ethnographic field methods that can be utilized within the context of treatment outcome studies. We will then describe a number of specific applications of ethnography that can be incorporated within the context of traditional treatment outcome studies. Some examples will be drawn from the published literature and others from our own work. The latter will be drawn primarily from several ethnographic studies that were conducted as part of a randomized field experiment funded by the National Institute of Alcohol Abuse and Alcoholism (NIAAA) and the National Institute on Drug Abuse (NIDA) that assessed the effectiveness of alternative treatment modalities for homeless, male, crack-cocaine users (Stahler, Shipley, Bartelt, & DuCette, 1995). 2. Ethnography as methodology 2.1. Defining ethnography Although some ethnographers will consider our definition as too broad, we use the term ethnography to refer to different qualitative methodologies that all relate to the process of doing fieldwork. Ethnography is essentially the methodological framework utilized to characterize all qualitative research, from unstructured and semi-structured interviewing, to extensive observation to obtain an insider’s understanding of the social context. Ethnographic methodology in its broadest sense can refer to any qualitative method that involves participant observation, nonparticipant observa-
0740-5472/00/$ – see front matter © 1999 Elsevier Science Inc. All rights reserved. PII: S0740-5472(99)00 0 2 9 - X
2
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
tion, or qualitative interviewing. The common thread that underlies these methodologies is the attempt to see the world from the point of view of the research participants, to attempt to understand their reality from their perspective within their natural environment or social context (Brooks, 1994; Glaser & Strauss, 1967). As Jennifer James stated, ethnography is “the study of a culture from within” (cited in Walters, 1980, p. 17). In traditional ethnography, the ethnographer typically spends time with members of the group he or she is attempting to study, observing what goes on, taking notes, and asking questions based on these observations and analysis. Within this context, research subjects are generally referred to as “key informants.” They are “insiders” from the point of view of either staff or user populations. Hypotheses are generated through inductive analysis; underlying theories are grounded in the observations, interpretations, and analysis extracted from the field or people one is studying (Glaser & Strauss, 1967). The major advantage of ethnographic methods is the ability to obtain in-depth and detailed information from the point of view of the research participant that is not constrained by predetermined categories (Patton, 1990). If trust and rapport with research respondents are established, ethnographers are often able to elicit rich information that is usually out of the reach of other methods (Walters, 1980). In general, ethnographic research methods usually include some type of observational method, some form of interviewing, a sampling strategy for selecting informants, and a thematic analysis and integration of this information. These areas will be briefly described below. 2.2. Observational methods Observational methods may involve either participation or nonparticipation by the researchers within the context that is being studied. 2.2.1. Participant-observational research This is the most widely used general ethnographic approach employed in the study of substance abuse with both treatment and street-based populations. Within a treatment setting context, this method involves observing and interacting with participants, and represents a combination of observing and informal interviewing. This may occur during treatment activities (e.g., during group counseling sessions) or during periods of unstructured time between activities. Systematic, participant-observationally based research often generates questions that may be posed in more formal, qualitative interviews. Some participant-observational ethnographic studies have been conducted by ethnographers who have a legitimate role in the program. For example, Skoll (1992) conducted his extensive ethnography of a midwestern therapeutic community from the vantage point of a program counselor. Both clients and staff were aware of his dual role of counselor and researcher.
2.2.2. Nonparticipant observation In terms of conducting ethnographic field studies of treatment programs, the goal is often to understand the program or intervention under study as thoroughly and comprehensively as possible. The nonparticipant-observer role usually involves intensive observation of a particular field setting without providing a tangible service to the treatment program. The goal of nonparticipant observation in drug treatment settings is often to develop an understanding of how a program functions, as well as the impact of the program on the participants. Predefined hypotheses are usually outside the range of concern of the observing ethnographer. The ethnographer in this capacity observes phenomena in order to elicit what is important in a general sense, and how specific details fit into the overall understanding of the program. The avoidance of a priori hypotheses and predefined conceptual schemes, and remaining open to many possible interpretations of the particular milieu one is studying are important contributions of ethnographic observational field studies (Brooks, 1994). Nonparticipant-observer approaches to gaining an understanding of a treatment setting are viable when the presence of researchers do not substantially alter the culture of the program. Sometimes the mere presence of observers can lead both staff and clients to change their behavior to meet the expectations of the observer (e.g., the Hawthorne Effect). Ethnographic observers can make staff and clients nervous, thus altering the natural flow of interaction in the particular setting. This type of disruptive effect can be minimized through careful planning and discussion with program staff and clients. In some of our work that has involved the use of trained graduate student ethnographers (i.e., Stahler, Cohen, Greene, Shipley, & Bartelt, 1995), the observers were viewed by program staff as rather benign and unobtrusive in each milieu they observed. This was probably because staff and clients were informed of their presence prior to their observation, they were relatively young and not viewed as authority figures, and they had already established rapport with both the treatment staff and the respective client populations in their role as interviewers for the quantitative portion of the main study. 2.3. Qualitative interviewing Qualitative interviews are typically open-ended, unstructured or loosely structured, survey formats designed to elicit detailed material about the treatment setting, or the impact of specific elements of the program on the client population. The interviewer attempts to assist informants to use their own words, values, and thoughts when answering questions (Patton, 1990; Posavac & Carey, 1997). As opposed to highly structured quantitative interviews, qualitative interviewers typically use questions that encourage the study informant to talk and elaborate about a particular concern. For example, ethnographers often conduct in-depth semi-structured interviews to obtain rich detail about the
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
backgrounds and biographies of addict populations. This type of research is often called life-history interviews. The advantage of a more ethnographically oriented interview schedule is that its primary purpose, independent of gaining a detailed understanding of the addicted client’s way of life, is as an exploratory heuristic device. The life-history interview, for example, is often governed by broad life-cycle conceptual categories (e.g., school experiences, family background, drug “careers,” and survival strategies on the street), or a timeline that the interviewee can use to anchor meaningful life events. The advantage here is the overall flexibility of the interview design and the ability of the interviewer to develop new questions, or change the course of the interview if the information warrants such a change. In addition to the importance of rich, descriptive case-oriented data, ideas gleaned from life-history interviews can be utilized to better articulate (a) cause-and-effect relationships related to substance abuse onset (e.g., traumatic events that trigger substance use); (b) problematic patterns of use and mitigating factors that may have influenced an abusive pattern (e.g., the influence of a significant relationship on the participant’s substance use); (c) factors that may have influence on the motivation to seek treatment (e.g., being fired from one’s job, eviction from an apartment); (d) or an assessment of the impact in qualitative terms of a particular drug treatment, or particular elements of the program on one’s life. 2.4. Sampling methods Because ethnographic research methodologies are relatively labor-intensive and usually involve studies that permit the researcher to examine social phenomena in-depth, samples tend to be small compared to more quantitative designs that produce information with greater breadth. As a result, deciding on whom to interview or observe is of critical importance in ethnographic research. Random sampling to achieve representativeness of the population is rarely done because of the large numbers required. Consequently, ethnography generally uses a variety of purposive sampling strategies, including extreme case sampling, typical case sampling, chain referral or “snowball” sampling, and convenience sampling (Patton, 1990). 2.4.1. “Extreme” case sampling This involves sampling cases that are unusual in some way. For example, case studies of clients who achieved extreme success in treatment may provide insights into how addicts can overcome their addiction (Stahler, Cohen, et al., 1995). A second example would be to identify clients who might be considered treatment “failures,” to better understand why a program had such little positive impact on them (Stahler, Cohen, Shipley, & Bartelt, 1993). Selecting clients to be interviewed can be done either through the use of quantitative data using empirically derived criteria (e.g., low scores on a treatment outcome measure), qualitatively
3
through information from such key informants as treatment staff, or a combination of the two. 2.4.2. “Typical” case sampling This type of strategy is generally used to illustrate rather than make definitive conclusions about a treatment program or its participants. For example, a case study of what a typical client experiences as he or she progresses through treatment may be useful to describe the treatment program. Cases are selected either from information provided by key informants concerning what they feel constitutes “typical” cases, or through quantitative data on program participants. Optimally, both methods should be utilized. Unfortunately, it is often difficult to define what constitutes a “typical” case. The notion behind selecting “typical” cases is to provide information about clients who are “average” or roughly representative of the general population. However, as described above, random sampling is usually not appropriate, given the small numbers to be selected. Therefore, typical cases might be selected on the basis of certain predefined categories or subpopulations that may be meaningful within the study context, such as older clients, women with children, the dually diagnosed, or the homeless. The ethnographer can select categories of clients whose experience and background may be different, and then select within those strata. It must be remembered, however, that qualitative information gleaned from this interview strategy is not meant to be conclusive as in a quantitative paradigm, but rather the object is to deepen one’s understanding, to develop new insights, and to point to further questions that may be worthy of study about these client populations. 2.4.3. Chain referral or “snowball” sampling Chain referral sampling techniques are typically utilized to gain access to relatively hidden populations of drug users. This sampling approach allows researchers to gain entry to a particular subculture of street users or other difficult to access substance-abusing populations through the use of key informants. By utilizing the social networks of drug users in the community or those already in treatment, the ethnographer can acquire an adequate sample of a population that may not otherwise be accessible to quantitatively oriented researchers. While this technique is not directly related to treatment outcome assessment, it is commonly used in developing samples of “hidden” or street-based substanceabusing populations and was necessary in our own investigation of homeless substance abusers. In our study of treatment dropouts (Stahler et al., 1993), we utilized a chain referral technique to locate clients who dropped out of treatment. They were found in various domains related to the homeless lifestyle, such as sections of the city where homeless people frequented, homeless settlements on the street, drop-in centers, soup kitchens, and the like. Through the use of informants who were still in treatment, we were able to find, interview, and help men reenter treatment, thus improving the efficacy of the overall project, and helping to
4
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
maintain an effective sample size for the quantitative outcome assessment. 2.4.4. Convenience sample This is the least desirable, but perhaps the most common type of sampling strategy, where informants are selected by convenience. An example would be to interview clients in a treatment program according to whomever is most easy to approach. In this strategy, there is no strategic basis used to select informants (Patton, 1990). 2.5. Data analyses Observations, impressions, and informant interviews are usually either in the form of field notes or tape-recorded and then transcribed. The main task of analyzing qualitative information is to identify the primary patterns in the data (Patton, 1990). Generally, some form of content analysis is used where the field notes or interviews are coded and categorized according to certain themes that are inductively derived by the analyst. Usually, direct quotations are used to illustrate the themes. These quotations not only help substantiate the themes, but also serve to bring the reader closer to the direct experience of the participant’s perspective. Reading direct quotations in the participant’s vernacular also has the benefit of reminding the research audience of the “subject’s” humanity, a perspective that is too easily forgotten in the course of conducting research. In our ethnographic study of treatment success (Stahler, Cohen, et al., 1995), we asked clients who were identified by counselors as having made considerable progress in treatment to what they attributed their success. After a thorough review of the interview transcriptions, field notes, and other supportive documents, we then set up indices of key conceptual areas; integrated our conceptual impressions; shared our interpretations with staff, clients and research staff in the relevant settings; and then cross-checked their interpretations with our findings. Their responses were categorized inductively to yield a variety of themes, which will be described later. There does exist relatively sophisticated qualitative data analysis software that performs this task of analyzing qualitative information interactively on a personal computer. Some of the more popular packages are ETHNOGRAPH, HyperRESEARCH, and NUD*IST 4 (Non-numerical Unstructured Data-Indexing, Searching and Theorizing). 3. The application of ethnographic methods in substance abuse treatment outcome research In general, the ethnographic methods described above can be applied to several areas of substance abuse research that relate to better understanding treatment outcomes. Examples of how ethnographic methodology can complement quantitative evaluation methods and provide important insights into evaluating substance abuse treatment programs focus on the following four areas: epidemiology, client and
staff satisfaction and perceptions of treatment, treatment dropout, and the social environment of the treatment program. In addition, ethnographic methods can assist in broadening our ability to interpret the results of treatment outcome research (see, e.g., Stahler & Cohen, 1995). 3.1. Epidemiology Ethnographic research has been particularly helpful in defining, augmenting, and clarifying an epidemiological understanding of substance use and abuse. For example, ethnography has helped identify the epidemiological connection between crack-cocaine addiction and HIV infection (Inciardi, Lockwood, & Pottieger, 1993). Over the last decade or so, ethnography has also made major contributions in gathering information concerning ways to reach and provide acceptable outreach to hidden and high risk populations such as intravenous drug users and crack-using women who engage in sex-for-crack exchanges (Ratner, 1993). Other ethnographic studies have been useful in identifying how some heroin addicts recover without treatment and why many addicts do not seek treatment (Biernacki, 1986). Ethnographic research on the epidemiology of substance abuse, as shown by these examples, can help define hypotheses, which can then be tested using experimental designs. They also help inform program planners and treatment staff. However, there are other applications of ethnography that can be utilized more directly within the context of treatment outcome research. 3.2. Client and staff satisfaction and perceptions of treatment Of particular importance in evaluating the success of the program is the level of acceptability and satisfaction with the program on the part of both clients and staff. If service providers resist a particular treatment approach, then it is unlikely that it will be implemented effectively. Quantitative scales that assess satisfaction and perceptions of treatment may be one of the most common evaluation approaches undertaken by treatment programs. However, qualitative approaches have the advantage of providing a more open-ended and thorough means of understanding from the staff’s and client’s perspective—what they like about the treatment program, what they do not like, and suggestions for improvement. Quantitative scales may produce a summary picture of the level of satisfaction with treatment and the treatment environment, but it will generally not provide specific information on why clients may be dissatisfied with their treatment experience, or why they may feel particularly positive about it. Open-ended, qualitative questions administered during treatment may elicit important information on client concerns, which, if addressed, may reduce treatment attrition. It may also be important to understand how staff perceive the treatment program. Staff morale may impact upon the quality of services provided and may affect the imple-
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
mentation of treatment under study. The process of asking clients and staff about their reactions to the treatment program also has the advantage of letting them feel that their point of view has been heard, which in itself may be of value. Assessing client and staff perceptions offers an especially important source of feedback for treatment interventions that are newly implemented or are in their exploratory phase of development. This type of qualitative assessment may be particularly useful when treatment strategies that are found to be effective in research settings are tested in community-based treatment settings. 3.3. Assessing treatment attrition In studying the effectiveness of substance abuse treatment interventions, ethnography can be employed to document how programs are actually implemented. Observation and interviews with staff and clients can provide information concerning barriers to optimal implementation, as well as qualitative information concerning the fidelity by which the interventions are implemented as intended. For example, in a case management intervention, quantitative indicators can be developed to assess the number and patterns of referral, and whether the clients actually receive the services to which they are referred. However, this method provides little information pertaining to why clients decide whether or not to go to referred services, or why case managers refer or do not refer for certain services. Another example concerns treatment attrition. Studies that have assessed client dropout from substance abuse treatment have generally focused upon quantitative measures that attempt to determine what types of clients drop out or stay, or what types of characteristics best predict client dropout (Stahler et al., 1993). However, understanding why they drop out may provide valuable information that can be fed back into improving the program. For example, in our ethnographic study of treatment attrition mentioned earlier (Stahler et al., 1993), we were able to find and interview a sample of 34 homeless clients who willingly left treatment prior to graduation. As mentioned earlier, we did this through checking lists at shelters and visiting locations frequented by the homeless. Clients were asked why they had dropped out of treatment, at what point in the program did they drop out, and to describe the events that led up to when they left the program. We found that there were often multiple reasons and most frequently a combination of “pushes” and “pulls” from external forces, factors relating to the treatment and shelter environment, and individual propensities. The most common reason concerned a basic lack of motivation and interest in obtaining treatment for their addiction. Others were dissatisfied with the intervention group to which they were randomly assigned. Some clients who wanted less structure ended up in the highly structured residential treatment group, whereas others who wanted more intensive services were assigned to the shelter-based case-management group. This mismatch
5
between client preferences and assigned treatment intervention resulted in many occurrences of attrition. The restrictiveness of the program structure, the culture and climate of the shelter, altercations with staff and clients, and exogenous factors, such as being drawn away by a job, girlfriend, or family members were also commonly mentioned reasons for leaving the program. As a result of this ethnographically derived information, a number of programmatic changes consistent with the original research design were made. The primary benefit of our ethnographic interviews was that it informed us about why clients were dropping out from their perspective in a more in-depth way than a more closedended, quantitative assessment would have captured. We learned to see from the clients’ reality and not from our preconceived notions about why we thought they were leaving the program. 3.4. Descriptions of program context A major question in disseminating treatment approaches to other programs is whether the treatment approach can be successfully replicated in different contexts with different program managers, staff, local conditions, and populations (e.g., external validity). Qualitative information can be useful in describing the environmental context and how this relates to the program implementation. Ethnographic descriptions of the community context can be obtained from key informants in the community or treatment agency, as well as from observation in the community. In addition, information from key informants and from extant documents can describe the current existence of other human service programs and resources in the community, and external events, such as changes in drug use patterns (e.g., introduction of new drugs), that may affect the research. 3.5. Understanding the social environment of the program While it is possible to specify a treatment intervention in some detail and to utilize manuals to assist in the standardizing of a substance abuse intervention, the actual “culture” or social environment of a treatment setting may lead to important differences in actual service delivery and outcomes, even within the same treatment modalities. For example, methadone clinics are often viewed as a single category of treatment intervention for heroin addiction. However, Rosenbaum’s (1985) ethnographic study of methadone clinics showed that there is considerable variability in the treatment “cultures” of clinics, particularly in regard to level of client control and treatment philosophy. The client’s relationship to the more humanistic, nonpunitive, “consumer”oriented models that allows clients to participate in their own treatment had more satisfied clients than the more punitive, authoritarian treatment models where patients were often viewed as hopelessly addicted and psychiatrically impaired. Ethnography can be utilized to assess how a treatment program’s culture develops from its origin, and to describe
6
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
and analyze the organizational structure of the treatment setting (Johnston, Rowe, & Swift, 1995). When the quantitative data are analyzed longitudinally, changes in client behavior and treatment model effectiveness can be joined with process ethnographic data regarding the dynamics of program structure and operation. As a result, a more complete and integrated understanding emerges. Not only does this result in a more systematic and thorough outcome assessment, but it also introduces new ways to conceptualize the treatment intervention from its original design (Brooks, 1994). The involvement of ethnographers in the treatment setting also presents the possibility of fostering closer relationships with its staff and client populations, thus allowing for smoother implementation of the research protocol 3.6. Helping to interpret treatment outcomes Ethnography can assist in the interpretation of quantitative data on treatment outcomes because of this methodology’s relatively open-ended nature and its focus on client experience. This may be particularly valuable in situations where no differences among groups are found. In well-conducted, rigorous treatment outcome studies using experimental designs, finding significant differences between control and experimental groups as hypothesized leads to relatively few difficulties in interpreting findings. However, treatment outcome studies conducted in the field frequently do not produce such straightforward results. Often some hypotheses are supported, and others are not. Some outcome variables may be significantly improved, and others not. Ethnography can often shed light on interpreting mixed findings or unexpected results. For example, in our outcome study on homeless, male, crack users (Stahler, Shipley, et al., 1995), the quantitative outcome data did not show statistically significant differences among the three different interventions at follow-up, even though all three groups did improve from baseline to follow-up. Exploratory field immersion by our ethnographers, as well as interviews with “successful” clients, provided some insights into why client improvement rates were not different among the three groups. While all three programs represented different therapeutic levels to address addiction, clients in all the groups had the opportunity to utilize 12-step meetings on a daily basis. Based on both quantitative services data as well as ethnographic interviews with staff and client informants from all three programs regarding treatment success, it was found that almost all successful clients reported attending Narcotics Anonymous (NA) and Alcoholics Anonymous (AA) meetings on a regular basis, and attributed aspects of 12-step programming (i.e., the elements they also offer, such as community, support, identity, etc.) as key ingredients to why they were progressing in their movement away from drugs and alcohol (Stahler, Cohen, et al., 1995). In addition, we also discovered that the services provided by city employees in the usual care condition (control group), involving standard shelter-based case-management services, had unexpectedly improved during the course of the study.
As a result of knowing that they were involved in the study, case managers appeared to become more responsive to clients than had been the case prior to the study. A second contribution of this ethnographic research to the same study concerned interpreting treatment outcomes. Our ethnographic research helped to identify both individual and programmatic factors that related to positive outcomes (e.g., within group differences). As mentioned earlier, clients were interviewed concerning their own attributions for why they were “successful.” This ethnographic effort was parallel to one of the project’s goal of identifying through multivariate analyses individual and program factors that were predictive of positive outcomes. The most salient theme identified was the role of self-motivation. This is certainly consistent with the burgeoning literature on the critical importance of client motivation to treatment outcomes (e.g., DeLeon & Jainchill, 1986; Miller & Rollnick, 1991; Prochaska, DiClemente, & Norcross, 1992). However, treatment intervention factors, such as the program structure, 12-step meetings, and counseling and support from staff, were also described by clients as essential to their progress, as was social support from other clients and family. According to most of the clients, it was not any one single factor alone that made a difference in their lives, but rather a combination of several of these factors that were essential for their progress. The results underscored that selfmotivation is often not sufficient to enhance outcomes, but needs to be nurtured with responsive treatment programming and social support from other clients, staff, and family and friends. The importance of social support was further supported by the quantitative portion of the study that examined predictors of success using multivariate analyses. 4. Discussion As we have attempted to show, the true virtues of ethnographic applications to in and out-of-treatment based substance abuse populations are the rich descriptions based on the user’s perspective and their overall interpretive power. Ethnography provides a useful adjunctive and complementary methodology to traditional quantitative substance abuse treatment research approaches. As a heuristic device, ethnography can aid in the research discovery process in helping to define and demarcate the phenomenology of the setting one is studying. The development of an in-depth understanding of the treatment process, the “real world” application of a particular treatment intervention, or attempting to isolate the factors in treatment that may positively or negatively affect treatment outcome, are all areas that benefit significantly from ethnographic methodologies. In-depth key informant interviews with clinical staff, clients, or administrative personnel can often shed light on problems or intricacies of the research project, which are typically hidden from the standard quantitative research strategies. Observational studies of treatment cultures can often lead to a clearer, more focused understanding of why certain clients
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
change and others do not, the contextual contingencies that may affect program compliance, or reasons why particular clientele may leave programs prior to completing treatment. The perspective of the substance abuser, particularly feelings toward treatment, staff, and the outcome assessment process itself, are useful in evaluating a particular treatment’s effectiveness. Studying the treatment and research process can often help disentangle cause and effect issues related to defining progress in treatment, or the isolation of barriers that may inhibit client progression in the program. However, ethnographic methodology does have its limitations. Field studies in natural settings by their very nature are not designed to control variables and, therefore, are not amenable to testing research hypotheses. Experimental methodologies using quantitative measures are, without question, the most effective means of demonstrating causal relationships and are best able to formally test a priori hypotheses. Hypothesis testing warrants a rigor and a control of conditions that ethnography cannot satisfy. However, ethnographic research can contribute more directly to quantitative research as questions that emerge in the discovery process become more focused and clear. While inductive approaches like ethnography are weak at demonstrating causal relationships, they excel at detecting general and specific patterns among social phenomena in field settings. For example, once ethnographic impressions of a treatment setting become more developed, consistent, and tangible patterns emerge, it may be then possible to form testable hypotheses about the expected pattern of certain quantitative measures relating to treatment outcome. A second frequently mentioned criticism is that ethnographic research is case-oriented and, therefore, uses small samples. As a result, the data may be idiosyncratic, not likely to be representative, have questionable validity, and will probably not be generalizable to other study settings or populations (Walters, 1980). This question of validity of ethnographic research was addressed by Donald Campbell. He argued that ethnographic research methods are particularly useful in obtaining valid and reliable data for three reasons: (a) the ethnographer’s familiarity with the respondent’s vernacular, (b) the ethnographer’s length of interaction in the setting, and (c) the ethnographer’s degree of participation in the respondent’s world (Walters, 1980). There is a long tradition of debate in the social sciences concerning the use of quantitative versus qualitative research approaches, and the relative validity of these approaches (e.g., Reichardt & Rallis, 1994), which will not be resolved here. Our position is that each looks at different aspects of social phenomena and that they are complementary. We believe that ethnography has a useful place, and should be routinely incorporated, within the context of traditional quantitative substance abuse treatment outcome research. Ethnographic methodology can be utilized within a traditional quantitative research paradigm to enlarge and broaden our understanding of treatment processes and outcomes. By attempting to understand how clients view and experience addiction
7
and addiction treatment programs in a more in-depth way and from the participants’ point of view, we can develop and improve existing interventions to be more responsive to client needs. We can also generate more meaningful hypotheses that are “grounded” in the subjective realities of clients and yet are still testable within an experimental paradigm. By understanding what “really went on” in treatment from an “insider’s” point of view (a perspective usually lacking in most outcome studies), interpretation of either significant or nonsignificant findings can be greatly facilitated. In the main, ethnography’s primary substantive contribution to the treatment outcome assessment field is to help provide a fuller, more accurate, and humanistic view of substance abuse, substance abusers, and the process and impact of substance abuse treatment. The question is not whether to use quantitative or qualitative methods; but rather how to use different methods so that they complement each other in the most effective ways possible (Posavac & Carey, 1997). As Light and Pillemer (1984) conclude: “The pursuit of good science should transcend personal preferences for numbers or narrative” (p. 143). References Agar, M. (1985). Folks and professionals: different models for the interpretation of drug use. The International Journal of Addictions 20, 173–182. Akins, C., & Beschner, G. (Eds.). (1980). Ethnography: A Research Tool for Policymakers in the Drug and Alcohol Fields (DHHS Pub. No. [ADM] 80-946). Rockville, MD: National Institute on Drug Abuse. Anderson, E. N. (1992). A healing place: ethnographic notes on a treatment center. Alcoholism Treatment Quarterly 9, 1–21. Biernacki, P. (1986). Pathways from Heroin Addiction Without Treatment. Philadelphia, PA: Temple University Press. Brooks, C. R. (1994). Using ethnography in the evaluation of drug prevention and intervention programs. The International Journal of the Addictions 29, 791–801. DeLeon, G., & Jainchill, N. (1986). Circumstances, motivation, readiness, and suitability as correlates of tenure in treatment. Journal of Psychoactive Drugs 18, 203–208. Feldman, H. W., & Aldrich, M. R. (1990). The role of ethnography in substance abuse research and public policy: historical precedent and future prospects. In E. Lambert (Ed.), The Collection and Interpretation of Data from Hidden Populations (NIDA Research Monograph No. 98). Rockville, MD: National Institute on Drug Abuse. Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine. Inciardi, J. A., Lockwood, D., & Pottieger, A. E. (1993). Women and Crack-Cocaine. New York: Macmillan. Johnston, P., Rowe, M., & Swift, P. (1995). Dilemmas of human service reform in New Haven: integrating three levels of organizational analysis. Contemporary Drug Problems 22, 363–391. Light, R. J., & Pillemer, D. B. (1984). Summing Up: The Science of Reviewing Research. Cambridge, MA: Harvard University Press. Miller, W. R., & Rollnick, S. (1991). Motivational Interviewing: Preparing People to Change Addictive Behavior. New York: Guilford Press. Patton, M. Q. (1990). Qualitative Evaluation and Research Methods (2nd ed.). Newbury Park, CA: Sage. Posavac, E. J., & Carey, R. G. (1997). Program Evaluation: Methods and Case Studies. Englewood Cliffs, NJ: Prentice-Hall Books. Prochaska, J. O., DiClemente, C. C., & Norcross, J. C. (1992). In search of how people change: applications to addictive behaviors. American Psychologist 47, 1102–1114.
8
G.J. Stahler, E. Cohen / Journal of Substance Abuse Treatment 18 (2000) 1–8
Ratner, M. S. (Ed.). (1993). Crack Pipe as Pimp: An Ethnographic Investigation of Sex-for-Crack Exchanges. New York: Lexington Books. Reichardt, C. S., & Rallis, S. F. (Eds.). (1994). The Qualitative-Quantitative Debate: New Perspectives. New Directions in Program Evaluation, No. 61. San Francisco: Jossey-Bass. Rosenbaum, M. (1985). A matter of style: Variation among methadone clinics in the control of clients. Contemporary Drug Problems 13, 375–397. Skoll, G. (1992). Walk the Walk and Talk the Talk: An Ethnography of a Drug Abuse Treatment Facility. Philadelphia, PA: Temple University Press. Stahler, G. J., & Cohen, E. (Eds.). (1995). Homelessness and substance abuse in the 1990s: qualitative studies from service demonstration projects. Contemporary Drug Problems 22, 169–391. Stahler, G. J., Cohen, E., Greene, M., Shipley, T., & Bartelt, D. (1995). A qualitative study of treatment success among homeless crack-addicted men: definitions and attributions. Contemporary Drug Problems 22, 237–264.
Stahler, G. J., Cohen, E., Shipley, T., & Bartelt, D. (1993). Why clients drop out of treatment: ethnographic perspectives on client attrition among homeless male “crack” cocaine users. Contemporary Drug Problems 20, 651–680. Stahler, G., Shipley, T., Bartelt, D., & DuCette, J. (1995). Evaluating alternative treatments for homeless substance-abusing men: outcomes and predictors of success. Journal of Addictive Diseases 14, 151–167. Sugarman, B. (1974). Daytop Village: A Therapeutic Community. New York: Holt, Rinehart, and Winston. Walters, J. M. (1980). What is ethnography? In C. Akins & G. Beschner (Eds.), Ethnography: A Research Tool for Policymakers in the Drug and Alcohol Fields (pp. 15–20) (DHHS Pub. No. [ADM] 80-946). Rockville, MD: National Institute on Drug Abuse. Weppner, R. (1983). The Untherapeutic Community. Lincoln, NE: University of Nebraska Press.