- Email: [email protected]
Using focus groups to evaluate client satisfaction in an alcohol and drug treatment program
Journal of Substance Abuse Treatment 18 (2000) 313–320
Article
Using focus groups to evaluate client satisfaction in an alcohol and drug treatment program Nicola A. Conners, M.Ed.a,*, Kathy K. Franklin, Ed.D.b a
University of Arkansas for Medical Sciences, 4301 W. Marham, Slot 711-1, Little Rock, AR 72205, USA Department of Educational Leadership, University of Arkansas at Little Rock, 2801 S. University Avenue, Little Rock, AR 72204-1099, USA Received 25 January 1999; accepted 9 August 1999
b
Abstract Often in studies of the program effectiveness of alcohol and drug treatment programs, behavioral outcomes are the only focus. While important, behavioral outcomes do not explore the viewpoint of the client about the success of the program, and evaluators are now recognizing the importance of measuring the performance of addictions treatment programs in terms of client satisfaction. The primary purpose of this article is to describe the experience of evaluators in their attempts to assess client satisfaction through the use of focus groups at a comprehensive drug treatment program for women and their children. The results of these focus groups are shared, and the appropriateness of the focus group methodology in assessing client satisfaction with comprehensive drug treatment services for women is discussed. © 2000 Elsevier Science Inc. All rights reserved. Keywords: Client satisfaction; Focus groups; Women’s AOD treatment program evaluation; Substance abuse
1. Introduction Drug abuse has often been viewed as a disorder primarily affecting men, therefore, treatment programs have historically been designed for males. However, surveys show that drug abuse is a problem that also impacts many women. Results from the 1997 National Household Survey on Drug Abuse indicate that 4.5% of women used an illicit drug during the month before the survey, 4.1 million women of childbearing age were current illicit drug users, and about 1.4 million of these women had children living with them (Substance Abuse and Mental Health Services Administration, 1998). Because of the obvious need, today more substance abuse treatment providers are recognizing the treatment needs of women, including pregnant and parenting women, and implementing programs to address those needs. 2. Arkansas Center for Addictions Research, Education, and Services The Arkansas Center for Addictions Research, Education, and Services (AR-CARES) is one program that seeks to address all of the complex treatment needs of addicted women. Located in Little Rock, AR-CARES is a licensed * Corresponding author. Tel.: 501-296-1760. E-mail address: [email protected] (N.A. Conners).
facility that provides residential and outpatient comprehensive substance abuse prevention and treatment services to low-income pregnant and parenting women, and their children. The component of AR-CARES that is the focus of this article is a residential program for parenting women and their children. A list of the services provided by this program can be found in Table 1. This part of AR-CARES is funded as a demonstration project by the Centers for Substance Abuse Treatment, and a study evaluating the effectiveness of this project is ongoing. 2.1. Program evaluation and drug treatment programs Often in studies of program effectiveness such as these, behavioral outcomes are the only focus. While important, behavioral outcomes do not explore the viewpoint of the client about the success of the program. This individual perspective is needed to judge the acceptability of the program in the eyes of potential clients, as well as the perceived effectiveness of the program in giving clients the tools they need to be successful (Chan et al., 1997). Recognizing this, the examination of client satisfaction with treatment services has been an important component of the evaluation of the AR-CARES program. As part of the evaluation of women-specific comprehensive drug treatment programs, it seems particularly appropriate to address the issue of client satisfaction because
0740-5472/00/$ – see front matter © 2000 Elsevier Science Inc. All rights reserved. PII: S 0740-5472(99)000 8 3 - 5
314
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
these types of programs are relatively new. Program managers are still learning which kinds of services are working for their clients, and why those services are working. Recognizing the importance of this, the primary purpose of this article is to describe the experience of the AR-CARES evaluation team in their attempts to assess client satisfaction through the use of focus groups. In addition, the results of these focus groups will be shared, and the appropriateness of the focus group methodology in assessing client satisfaction with comprehensive drug treatment services for women will be discussed. 3. Client satisfaction with alcohol and other drug treatment programs Client satisfaction with services is recognized as an important outcome measure for health care services, and efforts towards understanding client’s perceptions of the care they receive have increased dramatically over the past 10 years (Williams, 1994). This is true not only for general health care services, but also for mental health services. In fact, in the mental health field, consumer satisfaction surveys have been common for many years (Lebow, 1982). The area of addictions treatment has not been exempt from this increasing emphasis on the role of the consumer in Table 1 Services provided by the AR-CARES program 7–8-hour treatment day Alcohol and drug use assessment, education, and treatment Licensed mental health services for client, child, and family On-site residence Licensed early intervention services On-site licensed child care Infant and toddler Preschool and school-aged School-aged summer program Parenting education and support Health services and education Community and on-site 12-step meetings Life-skills training through the Cooperative Extension Service Case management Group and individual counseling, covering the following areas: Denial The disease process of addiction Physiology and pharmacology of tabacco, alcohol, and other drugs Effect of tobacco, alcohol, and other drugs on the mother and fetus Parenting the drug-exposed infant Women in relationships Women’s issues in recovery Loss and grief Anger and blame Family dynamics in recovery 12-step recovery Self-esteem building Relapse prevention AIDS and other sexually transmitted diseases Family planning Child development and care AR-CARES ⫽ Arkansas Center for Addictions, Research, Education, and Services.
quality improvement efforts. This has led to a shift in evaluation efforts toward measuring the performance of addictions treatment programs in terms of client satisfaction, in addition to the more traditional indicators of programs success, such as abstinence and employment (McLellan & Hunkeler, 1998). In spite of this increased emphasis, the literature regarding client satisfaction with substance abuse treatment remains limited. One study that examined client satisfaction with drug abuse day treatment versus residential care found that clients in both day and residential treatment were highly satisfied with overall services, and satisfaction scores did not differ between treatment modalities (Chan et al., 1997). In another study of 36 public alcohol treatment programs, the relationships between client satisfaction and program size, staffing patterns, Alcoholics Anonymous (AA) influence, and staff recovery status was examined. For residential clients, AA influence on treatment and AA beliefs held by staff were found to be related to client satisfaction, while program size and staffing patterns were not (Mavis & Stoffelmayr, 1994). The literature regarding client satisfaction with treatment is not only sparse, but contradictory. For example, in a study of patients in a psychiatric unit and patients in a substance abuse treatment unit, patient satisfaction was found to be positively related to treatment gains in quality of life, symptoms, and level of functioning (Holcomb et al., 1997). The previously described study of satisfaction with day versus residential care also found a correlation between client satisfaction and treatment retention as well as some treatment outcomes (Chan et al., 1997). However, in another study of 435 patients in four alcohol and drug abuse treatment programs, levels of client satisfaction were not found to be related to performance indicators (McLellan & Hunkeler, 1998). While there are some contradictions in the literature, one area of consistency in the findings about client satisfaction with substance abuse treatment is that reported levels of satisfaction are almost always high. This appears to hold true regardless of treatment modality (Etheridge et al., 1995; Holcomb et al., 1997; McLellan & Hunkeler, 1998). High reported levels of satisfaction have also been found consistently in studies examining satisfaction with mental health treatment (Lebow, 1982; Perreault et al., 1993). This has led to some concern that many client-satisfaction surveys may tend to inflate reports of satisfaction (Lebow, 1982; Perreault et al., 1993; Williams, 1994). Thus, evaluators seeking a true reflection of client satisfaction with services are challenged to think carefully about the tools they choose to assess client satisfaction. While the literature about client satisfaction with substance abuse treatment in general is sparse and contradictory in some areas, there is even less information regarding women’s satisfaction with treatment. What is available is largely anecdotal, and it is clear that more information on this topic could be very helpful for treatment program managers and evaluators. One study that did address this issue
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
involved individual interviews with 24 women who had participated in traditional substance abuse treatment programs as well as a treatment program that offered specialized services for women. In addition to the interview, each participant was also asked to indicate from a list of 24 possible services, which were available in each treatment program they had attended, and how helpful each service was to their recovery. The study findings indicated that substance dependent women have unique needs that go unmet in most treatment settings. In addition, the women in the study expressed a preference for women-specific treatment over more traditional programs (Nelson-Zlupko et al., 1996). Efforts to understand client satisfaction in a comprehensive substance abuse treatment program for women and children are complicated by the fact that these programs incorporate many different fields of service into one setting: health care, vocational education, child care and early intervention, drug and alcohol education, parenting education, etc. In order to fully understand client satisfaction with treatment, all these service areas must be considered. In addition, clients must be questioned regarding not only their satisfaction with the services they receive, but also the services received by their children. All of these problems pose challenges to evaluators interested in the issue of client satisfaction in these programs. Additionally, there do not appear to be any published surveys designed to assess client satisfaction in comprehensive drug treatment programs for women. With no readily available survey tool designed for use with this client group, evaluators in women’s treatment programs must look for other methods of assessing client satisfaction. 4. Using qualitative measures to evaluate client satisfaction A strong argument can be made for the use of a qualitative methodology, particularly in the early attempts at assessing client satisfaction in these relatively new treatment programs. First, there are no published surveys specifically designed for this population, and efforts to create a survey without first documenting clients’ concerns could result in a tool that is not valid. In fact, it has been suggested that one way to improve the content validity of satisfaction scales is to first use a qualitative approach to better understand the aspects of service delivery judged important by clients (Perreault et al., 1993). More information is needed concerning the factors that contribute most to clients’ satisfaction or dissatisfaction with the treatment programs. Second, it has been suggested that qualitative methodologies may address some concerns about surveys that result in inflated satisfaction scores. Clients often are more critical when qualitative methodologies are utilized, and are free to express their concerns about all aspects of care in a way that is not possible with many surveys (Perreault et al., 1993; Williams, 1994). For new and innovative programs seeking to mold their programming around the needs of their clients, it is especially important that their client satisfaction reports
315
be as accurate as possible, not inflated through the use of a poorly crafted survey. Finally, qualitative methods are invaluable in providing depth to the exploration of client satisfaction that is not possible with quantitative surveys. According to Merriam (1998), “the product of a qualitative study is richly descriptive” (p. 8). If it is true that women face multiple barriers to successful drug treatment outcomes, then it is reasonable to suggest that the exploration depth intrinsic to the qualitative inquiry is crucial to developing strategies and programs to eliminate those barriers. 5. Using a focus group method of inquiry for client satisfaction One qualitative method that has been used successfully to assess client satisfaction in a variety of fields is the focus group method. In the health care industry, focus groups have been used to gain information needed to design surveys on client satisfaction (Budreau & Chase, 1994; Glass, 1995; Omar & Schiffman, 1995). One strength of the focus group method is that it brings clients together to discuss their perceptions about the services that they have received. This allows for interaction between group members, which stimulates thoughts and recall of experiences. Focus groups can be particularly helpful for the discovery of service problems, and suggestions for fixing those problems (Ford et al., 1997). The drawbacks to this method include the expense and time involved, as well as concerns about whether the views expressed in the focus group are representative of all clients (Ford et al., 1997). When deciding the best way to begin assessing client satisfaction with services at AR-CARES, the focus group method appealed to the evaluation team primarily for two reasons. First, the program management and evaluators believed that the brief program-created survey form currently being used was inadequate, and was not giving the management the breadth nor depth of information they needed about client satisfaction. Also, it was unclear which aspects of the AR-CARES program are important to clients’ satisfaction with the program. Second, while many AR-CARES clients are quiet and at times somewhat withdrawn when speaking with staff on an individual basis, they seem to feel more comfortable sharing their feelings and opinions in a group setting. Many therapy groups and educational activities take place in small group settings at AR-CARES, and thus clients are familiar with the small group discussion format. Thus, it was believed more information could be obtained from focus group sessions than from individual interviews. 6. Evaluation of client satisfaction at AR-CARES 6.1. Participants Having decided to conduct a series of focus group sessions with AR-CARES clients, in late summer of 1997, pro-
316
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
gram evaluators began to contact current and former clients of the program to recruit focus group participants. They attempted to contact by phone or mail, or both, all 40 clients that had entered the program since new management took over in August, 1995. Of the 40 possible clients, 16 volunteered to participate in the focus group sessions. Each volunteer had been admitted to the residential treatment program at AR CARES, but their lengths of stay varied considerably. All participants were mothers whose children had entered the treatment program with them. The majority of the volunteers were African American. An equal effort was made to include clients currently enrolled in the AR-CARES program, as well as successful “program completers,” program dropouts, and clients who were administratively discharged. Of the 16 participants, 7 were successful program completers, 2 were program dropouts, 2 were administratively discharged, and 5 were current clients. 6.2. Focus group sessions The program evaluators planned three focus group sessions to accommodate the number of volunteers. The evaluators made a special effort to keep each group small, as it had been the experience of program staff that large group sessions were often chaotic or unproductive with these clients. Each focus group session was held on the AR-CARES campus in a large office area with transportation and childcare offered to all clients. The staff of AR-CARES served an evening meal to all focus group participants, and allowed time for them to eat and visit with one another. Clients signed an informed consent form outlining the protection of their anonymity in the transcription of the audiorecording from the session and the future reporting of evaluation results. The focus group moderator encouraged the volunteers to speak freely and share all of their opinions and ideas. The author, who is a research assistant at AR-CARES, served as the moderator for each focus group session. The moderator of the focus group sessions designed a focus group guide to elicit information about client satisfaction in several different ways. First, with the knowledge that many participants may have been involved in a variety of treatment programs in the past, including treatment programs traditionally designed for male clients, information was needed about client expectations toward treatment. Because of these past experiences, it was hypothesized that some clients entering AR-CARES might have divergent ideas about the nature of substance abuse treatment. There was a concern that these divergent perspectives of treatment might influence client expectations about, and satisfaction with, treatment. Second, the moderator hoped to encourage participants to think beyond their past or even current ideas about substance abuse treatment to identify a variety of ways in which they could better be served. Thus, the second question on the focus group guide encouraged participants to brainstorm about the types of services they need, without
limiting their discussion to the particular services they had received in the AR-CARES program or in other treatment settings. Finally, more specific information was needed about the services participants had received at AR-CARES, and how well those services had met their needs. Therefore, the moderator designed two questions to elicit more specific information about client satisfaction with AR-CARES programming and services. In these two questions clients responded to a query about what they found to be most, and the least, helpful at AR-CARES. 6.3. Analysis of focus group data After the focus groups were completed, an administrative assistant with AR-CARES transcribed all audiotapes. The moderator reviewed and critiqued the final transcripts for accuracy as compared with written notes taken during each session. If the moderator discovered a discrepancy between the written notes and the transcript, the audiotape was consulted. Next, the moderator coded the transcripts into categories based on attitudes related to client satisfaction with substance abuse treatment. The transcripts were coded with the aid of Ethnograph v. 4.0 (Seidel et al., 1995). In the coding process, the unit of analysis included all word groups or sentences that related to client satisfaction. Each unit of analysis that was identified as part of the same concept was given a common label (Miles & Huberman, 1994). Next, a code book was developed in which each code word was given a definition. Working from these definitions, the codes were then grouped into common attitude themes corresponding to client satisfaction. Finally, the moderator merged the attitude themes into six overarching attitude patterns to develop a theoretical framework explaining client satisfaction.
7. Results from the focus group conversations These six attitude patterns related to client (a) expectations prior to entering treatment; (b) perception of treatment benefits; (c) attitudes about specific programming/service issues; (d) comments on the structure and policies of the drug treatment center; (e) preferences related to program staff; and (f) feelings about the inclusion of children in the residential treatment program. 7.1. Client expectations prior to entering treatment Participants’ ideas about treatment prior to entry varied from not even knowing that drug treatment programs existed, to having participated in many different programs. Clients reported that prior to entry into a treatment program they never knew what to expect, or how one program might be different from another. Participants also described feeling absolutely desperate, having tried and failed at every attempt to abstain from drugs on their own. They also reported that they found this level of desperation to be
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
necessary for successful completion of the treatment program. One client described that feeling of desperation in the following way: I remember how I felt. I didn’t want to live. It was like I wanted to live and I didn’t want to live. And that was because I thought that was the way I was supposed to go. I though I was supposed to smoke crack until I die. I said, I guess that’s what God had me here for. And that’s the feeling I had. 7.2. Client perceptions of treatment benefits Participants’ described the many ways in which treatment programs helped them by teaching them “to live life on life’s terms.” Participants described learning about themselves in treatment, including learning that they are not “bad people,” but that they have a disease for which they need treatment, and learning to love themselves regardless of what others think of them. They also reported that in treatment they learn to “live again” in all areas of life. As a result of treatment, they are better prepared not only for a life of sobriety, but also a life in which they can better handle relationships, finances, children, jobs, etc. They also described learning that they cannot succeed in treatment on their own. They need the support of others to help them in their recovery. It is not a “me” or an “I” program. It’s a “we” and an “us” program. Out there it was all about me, what I wanted, how I was gonna get it. It’s still all about me, but in a positive way. I know I can’t get it without the help of others. I know that I can’t do it alone. Participants also described how treatment helped them learn how to recognize their emotions and find positive ways to deal with them. In treatment they learned how to think about their feelings before acting impulsively, and they also learned the importance of a positive mental attitude. Also, they reported gaining a new “God consciousness,” and an understanding of the role of spirituality in their recovery. However, some participants expressed feeling that religion is often forced upon them in treatment programs. 7.3. Client attitudes about specific programming/service issues Many participants agreed that the most helpful groups and classes were the ones most closely related to their addiction issues. Examples of such groups include: Alcohol and Drug Education, Relapse Prevention, and Recovery Dynamics. Participants believe that the main focus of treatment should be to help clients learn about themselves and their disease. Many participants said that, particularly during clients’ early days in treatment, classes focusing on addiction issues should take precedence over classes that do not directly relate to addiction: nutrition, parenting, budgeting, etc. Some participants suggested that “life skills” classes should be optional, or perhaps delayed until clients have
317
been in the program for some reasonable length of time. They also expressed a desire to learn about life skills in a hands-on, practical manner. For example, they suggested that clients be allowed to take charge of the residence in terms of shopping, meal planning, etc. Participants also discussed programming issues around job preparation, saying that their lack of job preparation is a serious problem for them. Many stated that attending GED classes was helpful to them, and that work preparation classes and more job referrals are necessary to get them back into the work world. Participants also expressed that one of the benefits of long-term treatment is that there is time to help clients develop these work skills as well as related skills like coping and relationship building. On the issue of job training, one client stated: A lot of addicts and alcoholics are addicts and alcoholics just because they can’t support or take care of their family. It’s just like being on welfare. They are on welfare because they don’t have the education or the know how to go out there and get a job to provide for their family. And so the addicts and alcoholics are in jail, or in prison, because they are not able to go out here and get job, because they don’t have the education, they don’t know. Give me something I can use, some type of computer training, something I can use when I leave out of here, that I can provide for my family. Some type of job training, or be able to at least point me in the direction that I need to go in, you know. Another programming issue addressed by participants was group and individual therapy. They described how both group and individual sessions with their therapist were helpful in their recovery. In their view, the strength of the group therapy session is that it allows clients to see that they are not alone, and that there are others who have had similar experiences, feelings, and struggles. It is comforting for clients to discover that “it’s not just me.” They need to know that there are other people like them and they are not the only ones. Because you feel like you are the only one. You have one foot in the grave and the other foot on a banana peel. I thought I was crazy and had lost my mind, and you know, I should have known better. Individual therapy sessions were described as important for clients because in that setting they can share issues that may be too painful to discuss in a group setting. Participants also stated that it is especially helpful for them to be actively involved in the local recovery community while they are still in treatment. They shared a desire to attend as many AA/CA meetings as possible as part of their treatment. They also described the importance of sponsorship in their recovery process. On another issue, participants stated that they would like to see more emphasis placed upon giving assistance to cli-
318
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
ents who are transitioning out of treatment. Participants discussed the difficulty in finding housing, jobs, and transportation on their own. One suggestion which emerged from the focus groups was to add a transitional facility to the treatment program where clients who are working could live. In this way they could be spared from having to face all of the pressures of the outside world at once. Others emphasized needing referrals to help them find jobs, and transportation to help them get to work. 7.4. Clients comments on the structure and policies of the drug treatment center Participants expressed a very strong desire for a program where the rules are clear, are consistently enforced by all staff, and are consistently applied to all clients. They explained that the rules at AR-CARES were at times unclear and frequently changing, or inconsistently enforced. Participants reported that clients’ behaviors will not change until staff enforce rules consistently. They shared that when clients enter treatment, they lack discipline in their life, and they expect a treatment program to assist them in learning to live by the rules of society. Participants stated that a “phase” or “level” system is helpful in treatment, and they shared a variety of ideas about how that might work. One suggestion that emerged was to implement a 3- to 4-day detoxification period where clients are not responsible for normal caregiving and program activities; a 30-day restriction period for new clients in which they have no phone or pass privileges; an intensive phase in which clients deal exclusively with recovery issues; and a less intensive phase where clients might take a variety of classes or work on developing other skills. Participants reported that an important part of the phase system is that it should be used to hold clients accountable for their behavior and level of effort in treatment. Another policy issue raised by the participants was the issue of drug screens. They emphasized that drug screening should be frequent and consistently administered. They even commented that staff should watch clients while they urinate, and should use urine specimen cups with temperature gauges to prevent tampering. Participants also described the problems and pressures related to the facilities of the treatment program. They found it difficult to share a room not only with their children, but also with another family, as was the case at ARCARES. Participants described a need for some level of privacy, as well as a need to have a place where they can spend time as a family. Whether or not clients were disturbed by the living situation in part depends upon their expectation. For clients who expected a one-room dormitory based on their previous experience in treatment, sharing a room with another family did not seem like such a hardship. Another policy issue that participants’ expressed strong opinions on was the issue of smoking. Participants expressed that they should not be required to quit smoking while in drug treatment, as was emphasized at AR-CARES.
They feel that it is hard enough for them to give up drugs and alcohol in their lives without giving up smoking as well. They also commented that they receive mixed messages about smoking because some of the 12-step programs do not place the same emphasis on quitting smoking as some treatment programs do. 7.5. Client preferences related to program staff Participants agreed that the most important quality in a staff person in a treatment center is that they be able to “relate” to clients. Participants described the importance of a staff person understanding their experiences and feelings. Some participants felt that in order for a staff person to have that understanding, they needed to be a recovering person themselves. However, other participants described having had good relationships with staff members who never had a drug problem, but who nonetheless have a good understanding of clients. Participants also shared a concern regarding the qualifications of staff members. They stated that they do not want any “uncertified” person teaching a class. Participants also emphasized their need to feel supported by staff, not to be “beaten down” or reminded of what they have done wrong in their lives. Participants reported being acutely aware of all of their mistakes, and reminders and criticism are not viewed as helpful. If staff approach them too harshly they will be turned off to treatment. One participant described her experience, saying: When I came here, I was beat down enough. I needed support and care, and what I got was, “you was this and you was that.” Well, I know that, you know? Tell me what I can do to not be like that. Don’t tell me what I did and did wrong, and how I messed up. They told me “well your child was in pitiful shape cause her mother and daddy messed up.” Don’t tell me stuff like that. I can’t take it. Participants also reported feeling that it is important that staff members respect their feelings and concerns. When their complaints are dismissed by staff, it leaves the impression that client’s feeling are not taken seriously. Participants also shared concerns about staff becoming too personally involved with clients, saying that if staff members do not maintain a professional relationship with clients, then the client’s recovery process is jeopardized. Participants described good staff members as ones who do their job in a structured manner, and do not take their job home with them. Instead they are able to separate their personal and professional lives. 7.6. Client feelings about the inclusion of children in the residential treatment program Participants reported that they appreciate the ability to bring their children into treatment with them for a variety of reasons. Participants feel that having their children in treatment with them allows them to focus on their recovery rather than worrying about where their children are, or
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
whether they might lose custody of them. Participants also stated that they need to take on the responsibility of being mothers and acting as head of the household. Having their children in treatment with them allows them to do so. In addition, participants recognized that their children need treatment too. They have suffered because of their mother’s addiction and need to get help for their problems. One mother described how she felt about bringing her son into treatment with her. I’m grateful there was a place for me and my son. Because the first time, I couldn’t take him with me. The second time I knew I had to find some place for the both of us to go, because they told me that when I was out getting sick, he was getting sick too. Participants also shared that having their children in treatment with them allowed them to rebuild their relationships. They described learning about their children, and learning to identify their problems, understand their behaviors, and rebuild a trust relationship with their children. It’s teaching me like a closeness, like a bonding. I’m seeing things in him that I didn’t choose to ignore, but I did ignore in my addiction. And I know him a little bit better than I did when I walked in here. I’m learning him as I’m learning me. I’m learning to accept my son as he is. Participants also stated that more family therapy was needed to help them repair their relationships with their children. They feel that having separate therapists for mothers and children, as was the policy at AR-CARES, makes if difficult for the family to communicate. While they want their children to have an outlet for expressing their feelings, they want the child to learn to communicate with family members as well as therapists. They feel that more family therapy would help facilitate communication and strengthen their bond. Participants also described the difficulty of being in a living situation with children who have serious behavior problems. It is stressful for clients to be around not only their own children, but also other clients’ children who may also have behavior problems. Some parents view their children or other’s children as generally “bad.” Some participants feel that children who have serious behavior problems should be sent to another type of treatment facility, perhaps a psychiatric treatment center for children. Others believe that all of their families have problems to some degree, and families should be allowed to remain intact as they seek help. However, in spite of the difficulties, participants described seeing positive changes in their children as a result of their time in treatment. They believe that mothers and children alike are benefiting from treatment. 8. Discussion of focus group results In a way that quantitative surveys never did, the results from these focus groups offered clear evidence to ARCARES program evaluators and managers that clients do
319
have strong feelings about their experiences in treatment, as well as many suggestions about how treatment programs could be improved. Drawing on their experiences in a variety of treatment programs, both traditional programs and the more comprehensive women-specific AR-CARES program, the focus group participants were able to identify which services were most important to their recovery, what needs remain unmet, and what steps AR-CARES can take to further improve their treatment program. One interesting theme that emerged from the focus group sessions, was the preference the participants expressed for classes focusing primarily on addiction issues. Given the emphasis in the literature on the need for very comprehensive treatment for women, this finding is somewhat surprising. It has been repeatedly suggested that women want and need classes that build their “life skills” in areas such as budgeting, parenting, nutrition, and so forth. (Finkelstein, 1994; Nunes-Dinis & Barth, 1993). However, many of these participants expressed some frustration with these classes, and voiced a preference for strictly addiction related classes, such as relapse prevention. Longterm treatment programs for women may want to take seriously the participants’ suggestion that the earliest portion of treatment be devoted primarily to the issues most closely related to addiction, and that an emphasis on building “life skills” be postponed until later in the treatment episode. A related theme that emerged from the focus group sessions, was the frequently mentioned need for skill-building and support around job preparation. While GED classes were viewed as helpful for many participants, they need more support to move successfully into the work world. Participants suggested ways in which treatment programs could help them make this transition more effectively, including by offering more job training, job referrals, and transportation support. Additionally, participants affirmed the importance of group and individual therapy in the treatment program, and indicated that they were satisfied with the therapy they received. However, they also stressed a need for more family therapy, a recommendation that treatment programs may wish to heed. Participants also emphasized the impact of program staff on their satisfaction with treatment. Many described the good relationships they had developed with staff, and the importance of those relationships to their recovery. However, others expressed dissatisfaction with staff who were unprofessional, lacked qualifications, or were not supportive. Certainly these focus groups did not resolve the controversial question of whether or not clients feel it is necessary for program staff to be recovering from addictions themselves, however, participants did express strong opinions on the topic. Clearly, treatment program management should consider these diverse and strong opinions as they hire program staff. These are just a few of the important findings that emerged from the focus group sessions with AR-CARES clients. While these findings are important to evaluators and program managers, an examination of the AR-CARES experience using the focus group method to evaluate client satisfaction may be just as valuable.
320
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
9. Validity of the focus group method to program evaluation For AR-CARES, using focus group methodology to explore client satisfaction proved to be a useful endeavor for several reasons. First, most participants in each session seemed comfortable in the small group setting, and they freely shared details of their experiences both at ARCARES and in other treatment programs. As they shared their experiences with one another, the words of one participant often brought to mind something another participant had experienced, and thus they were able to share more completely than might have been possible otherwise. Second, the focus groups were judged successful because they resulted in participants’ sharing the type of information that the focus groups were designed to elicit: information about their expectations about treatment, their ideas about the services they need, as well as feedback about the specific services provided by AR-CARES. This information proved very useful to management at AR-CARES in their continual efforts to respond to the needs of their clients. Third, the focus groups provided the information necessary for the development of a client satisfaction survey form for use in this program. By allowing clients to identify what services and program issues are important to them, the program gained the information they needed to develop a useful and relevant survey tool. Fourth, and possibly most important, this process allowed for the recognition of clients as experts on their own treatment experience. An opportunity was provided for clients to voice their opinion and feel that they were making an important contribution to the treatment program. This is an important outcome of the focus group process, as it has been suggested that when clients are allowed input into the program design, the treatment program is enhanced and client retention improves (Laken & Hutchins, 1996).
10. Conclusion Overall, the focus group method of inquiry served as a valuable tool in eliciting rich and detailed data concerning the satisfaction of female clients participating in a residential substance abuse treatment program. While the focus group method was a time consuming, and labor intensive process, the data collected from these sessions has been useful to decision-makers at AR-CARES as they continue to develop programs and services to meet the needs of parenting women. Based on the experiences of the program evaluators, it is recommended that agencies consider the use of the focus group as an appropriate method for evaluating client satisfaction with drug treatment services. However, whether it is through surveys, individual interviews, focus groups, or other methods, what is important is that treatment programs make an effort to include clients’ perceptions of services as a part of the evaluation of their program. While it is not necessary that
evaluators or management always agree with clients’ assessment of their treatment needs or the helpfulness of services, it is important that their opinions be included in the debate.
Acknowledgment Support was provided by the Centers for Substance Abuse Treatment (6HD8 T100970-05-01)
References Budreau, G., & Chase, L. (1994). A family-centered approach to the development of a pediatric family satisfaction questionnaire. Pediatric Nursing 20, 604–608. Chan, M., Sorensen, J. L., Guydish, J., Tajima, B., & Acampora, A. (1997). Client satisfaction with drug abuse day treatment versus residential care. Journal of Drug Issues 27, 367–377. Etheridge, R. M., Craddock, S. G., Dunteman, G. H., & Hubbard, R. L. (1995). Treatment services in two national studies of community-based drug abuse treatment programs. Journal of Substance Abuse 7, 9–26. Finkelstein, N. (1994). Treatment issues for alcohol-and-drug-dependent pregnant and parenting women. Health and Social Work 19, 7–15. Ford, R. C., Bach, S. A., & Fottler, M. D. (1997). Methods of measuring patient satisfaction in health care organizations. Health Care Management Review 22, 74–89. Glass, A. P. (1995). Identifying issues important to patients on a hospital satisfaction questionnaire. Psychiatric Services 46, 83–85. Holcomb, W. R., Parker, J. C., & Leong, G. B. (1997). Outcomes of inpatients treated on a VA psychiatric unit and a substance abuse treatment unit. Psychiatric Services 48, 699–704. Laken, M. P., & Hutchins, E. (1996). Recruitment and Retention of Substance Using Pregnant and Parenting Women: Lessons Learned. Arlington, VA: National Center for Education in Maternal and Child Health. Lebow, J. (1982). Consumer satisfaction with mental health treatment. Psychological Bulletin 91, 244–259. Mavis, B. E., & Stoffelmayr, B. E. (1994). Program factors influencing client satisfaction in alcohol treatment. Journal of Substance Abuse 6, 345–354. McLellan, A. T., & Hunkeler, E. (1998). Patient satisfaction and outcomes in alcohol and drug abuse treatment. Psychiatric Services 49, 573–575. Merriam, S. B. (1998). Qualitative Research and Case Study Applications in Education. San Francisco: Jossey-Bass. Miles, M. B., & Huberman, A. M. (1994). An Expanded Sourcebook: Qualitative Data Analysis (2nd ed.). Thousand Oaks, CA: Sage. Nelson-Zlupko, L., Morrison Dore, M., Kauffman, E., & Kaltenbach, K. (1996). Women in recovery: their perceptions of treatment effectiveness. Journal of Substance Abuse Treatment 13, 51–59. Nunes-Dinis, M., & Barth, R. P. (1993). Cocaine treatment and outcome. Social Work 38, 611–617. Omar, M. A., & Schiffman, R. F. (1995). Pregnant women’s perceptions of prenatal care. Maternal-Child Nursing Journal 23, 132–144. Perreault, M., Leichner, P., Sabourin, S., & Gendreau, P. (1993). Patient satisfaction with outpatient psychiatric services: qualitative and quantitative assessments. Evaluation and Program Planning 16, 109–118. Seidel, J., Friese, S., & Leonard, D. C. (1995). The Ethnograph v4.0: A Users Guide. Amherst, MA: Qualis Research Associates. Substance Abuse and Mental Health Services Administration. (1998). Preliminary Results from the 1997 National Household Survey on Drug Abuse [On-line]. Available: http://www.samhsa.gov/oas/nhsda/nhsdafls.htm Williams, B. (1994). Patient satisfaction: a valid concept? Social Science and Medicine 38, 509–516.
Article
Using focus groups to evaluate client satisfaction in an alcohol and drug treatment program Nicola A. Conners, M.Ed.a,*, Kathy K. Franklin, Ed.D.b a
University of Arkansas for Medical Sciences, 4301 W. Marham, Slot 711-1, Little Rock, AR 72205, USA Department of Educational Leadership, University of Arkansas at Little Rock, 2801 S. University Avenue, Little Rock, AR 72204-1099, USA Received 25 January 1999; accepted 9 August 1999
b
Abstract Often in studies of the program effectiveness of alcohol and drug treatment programs, behavioral outcomes are the only focus. While important, behavioral outcomes do not explore the viewpoint of the client about the success of the program, and evaluators are now recognizing the importance of measuring the performance of addictions treatment programs in terms of client satisfaction. The primary purpose of this article is to describe the experience of evaluators in their attempts to assess client satisfaction through the use of focus groups at a comprehensive drug treatment program for women and their children. The results of these focus groups are shared, and the appropriateness of the focus group methodology in assessing client satisfaction with comprehensive drug treatment services for women is discussed. © 2000 Elsevier Science Inc. All rights reserved. Keywords: Client satisfaction; Focus groups; Women’s AOD treatment program evaluation; Substance abuse
1. Introduction Drug abuse has often been viewed as a disorder primarily affecting men, therefore, treatment programs have historically been designed for males. However, surveys show that drug abuse is a problem that also impacts many women. Results from the 1997 National Household Survey on Drug Abuse indicate that 4.5% of women used an illicit drug during the month before the survey, 4.1 million women of childbearing age were current illicit drug users, and about 1.4 million of these women had children living with them (Substance Abuse and Mental Health Services Administration, 1998). Because of the obvious need, today more substance abuse treatment providers are recognizing the treatment needs of women, including pregnant and parenting women, and implementing programs to address those needs. 2. Arkansas Center for Addictions Research, Education, and Services The Arkansas Center for Addictions Research, Education, and Services (AR-CARES) is one program that seeks to address all of the complex treatment needs of addicted women. Located in Little Rock, AR-CARES is a licensed * Corresponding author. Tel.: 501-296-1760. E-mail address: [email protected] (N.A. Conners).
facility that provides residential and outpatient comprehensive substance abuse prevention and treatment services to low-income pregnant and parenting women, and their children. The component of AR-CARES that is the focus of this article is a residential program for parenting women and their children. A list of the services provided by this program can be found in Table 1. This part of AR-CARES is funded as a demonstration project by the Centers for Substance Abuse Treatment, and a study evaluating the effectiveness of this project is ongoing. 2.1. Program evaluation and drug treatment programs Often in studies of program effectiveness such as these, behavioral outcomes are the only focus. While important, behavioral outcomes do not explore the viewpoint of the client about the success of the program. This individual perspective is needed to judge the acceptability of the program in the eyes of potential clients, as well as the perceived effectiveness of the program in giving clients the tools they need to be successful (Chan et al., 1997). Recognizing this, the examination of client satisfaction with treatment services has been an important component of the evaluation of the AR-CARES program. As part of the evaluation of women-specific comprehensive drug treatment programs, it seems particularly appropriate to address the issue of client satisfaction because
0740-5472/00/$ – see front matter © 2000 Elsevier Science Inc. All rights reserved. PII: S 0740-5472(99)000 8 3 - 5
314
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
these types of programs are relatively new. Program managers are still learning which kinds of services are working for their clients, and why those services are working. Recognizing the importance of this, the primary purpose of this article is to describe the experience of the AR-CARES evaluation team in their attempts to assess client satisfaction through the use of focus groups. In addition, the results of these focus groups will be shared, and the appropriateness of the focus group methodology in assessing client satisfaction with comprehensive drug treatment services for women will be discussed. 3. Client satisfaction with alcohol and other drug treatment programs Client satisfaction with services is recognized as an important outcome measure for health care services, and efforts towards understanding client’s perceptions of the care they receive have increased dramatically over the past 10 years (Williams, 1994). This is true not only for general health care services, but also for mental health services. In fact, in the mental health field, consumer satisfaction surveys have been common for many years (Lebow, 1982). The area of addictions treatment has not been exempt from this increasing emphasis on the role of the consumer in Table 1 Services provided by the AR-CARES program 7–8-hour treatment day Alcohol and drug use assessment, education, and treatment Licensed mental health services for client, child, and family On-site residence Licensed early intervention services On-site licensed child care Infant and toddler Preschool and school-aged School-aged summer program Parenting education and support Health services and education Community and on-site 12-step meetings Life-skills training through the Cooperative Extension Service Case management Group and individual counseling, covering the following areas: Denial The disease process of addiction Physiology and pharmacology of tabacco, alcohol, and other drugs Effect of tobacco, alcohol, and other drugs on the mother and fetus Parenting the drug-exposed infant Women in relationships Women’s issues in recovery Loss and grief Anger and blame Family dynamics in recovery 12-step recovery Self-esteem building Relapse prevention AIDS and other sexually transmitted diseases Family planning Child development and care AR-CARES ⫽ Arkansas Center for Addictions, Research, Education, and Services.
quality improvement efforts. This has led to a shift in evaluation efforts toward measuring the performance of addictions treatment programs in terms of client satisfaction, in addition to the more traditional indicators of programs success, such as abstinence and employment (McLellan & Hunkeler, 1998). In spite of this increased emphasis, the literature regarding client satisfaction with substance abuse treatment remains limited. One study that examined client satisfaction with drug abuse day treatment versus residential care found that clients in both day and residential treatment were highly satisfied with overall services, and satisfaction scores did not differ between treatment modalities (Chan et al., 1997). In another study of 36 public alcohol treatment programs, the relationships between client satisfaction and program size, staffing patterns, Alcoholics Anonymous (AA) influence, and staff recovery status was examined. For residential clients, AA influence on treatment and AA beliefs held by staff were found to be related to client satisfaction, while program size and staffing patterns were not (Mavis & Stoffelmayr, 1994). The literature regarding client satisfaction with treatment is not only sparse, but contradictory. For example, in a study of patients in a psychiatric unit and patients in a substance abuse treatment unit, patient satisfaction was found to be positively related to treatment gains in quality of life, symptoms, and level of functioning (Holcomb et al., 1997). The previously described study of satisfaction with day versus residential care also found a correlation between client satisfaction and treatment retention as well as some treatment outcomes (Chan et al., 1997). However, in another study of 435 patients in four alcohol and drug abuse treatment programs, levels of client satisfaction were not found to be related to performance indicators (McLellan & Hunkeler, 1998). While there are some contradictions in the literature, one area of consistency in the findings about client satisfaction with substance abuse treatment is that reported levels of satisfaction are almost always high. This appears to hold true regardless of treatment modality (Etheridge et al., 1995; Holcomb et al., 1997; McLellan & Hunkeler, 1998). High reported levels of satisfaction have also been found consistently in studies examining satisfaction with mental health treatment (Lebow, 1982; Perreault et al., 1993). This has led to some concern that many client-satisfaction surveys may tend to inflate reports of satisfaction (Lebow, 1982; Perreault et al., 1993; Williams, 1994). Thus, evaluators seeking a true reflection of client satisfaction with services are challenged to think carefully about the tools they choose to assess client satisfaction. While the literature about client satisfaction with substance abuse treatment in general is sparse and contradictory in some areas, there is even less information regarding women’s satisfaction with treatment. What is available is largely anecdotal, and it is clear that more information on this topic could be very helpful for treatment program managers and evaluators. One study that did address this issue
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
involved individual interviews with 24 women who had participated in traditional substance abuse treatment programs as well as a treatment program that offered specialized services for women. In addition to the interview, each participant was also asked to indicate from a list of 24 possible services, which were available in each treatment program they had attended, and how helpful each service was to their recovery. The study findings indicated that substance dependent women have unique needs that go unmet in most treatment settings. In addition, the women in the study expressed a preference for women-specific treatment over more traditional programs (Nelson-Zlupko et al., 1996). Efforts to understand client satisfaction in a comprehensive substance abuse treatment program for women and children are complicated by the fact that these programs incorporate many different fields of service into one setting: health care, vocational education, child care and early intervention, drug and alcohol education, parenting education, etc. In order to fully understand client satisfaction with treatment, all these service areas must be considered. In addition, clients must be questioned regarding not only their satisfaction with the services they receive, but also the services received by their children. All of these problems pose challenges to evaluators interested in the issue of client satisfaction in these programs. Additionally, there do not appear to be any published surveys designed to assess client satisfaction in comprehensive drug treatment programs for women. With no readily available survey tool designed for use with this client group, evaluators in women’s treatment programs must look for other methods of assessing client satisfaction. 4. Using qualitative measures to evaluate client satisfaction A strong argument can be made for the use of a qualitative methodology, particularly in the early attempts at assessing client satisfaction in these relatively new treatment programs. First, there are no published surveys specifically designed for this population, and efforts to create a survey without first documenting clients’ concerns could result in a tool that is not valid. In fact, it has been suggested that one way to improve the content validity of satisfaction scales is to first use a qualitative approach to better understand the aspects of service delivery judged important by clients (Perreault et al., 1993). More information is needed concerning the factors that contribute most to clients’ satisfaction or dissatisfaction with the treatment programs. Second, it has been suggested that qualitative methodologies may address some concerns about surveys that result in inflated satisfaction scores. Clients often are more critical when qualitative methodologies are utilized, and are free to express their concerns about all aspects of care in a way that is not possible with many surveys (Perreault et al., 1993; Williams, 1994). For new and innovative programs seeking to mold their programming around the needs of their clients, it is especially important that their client satisfaction reports
315
be as accurate as possible, not inflated through the use of a poorly crafted survey. Finally, qualitative methods are invaluable in providing depth to the exploration of client satisfaction that is not possible with quantitative surveys. According to Merriam (1998), “the product of a qualitative study is richly descriptive” (p. 8). If it is true that women face multiple barriers to successful drug treatment outcomes, then it is reasonable to suggest that the exploration depth intrinsic to the qualitative inquiry is crucial to developing strategies and programs to eliminate those barriers. 5. Using a focus group method of inquiry for client satisfaction One qualitative method that has been used successfully to assess client satisfaction in a variety of fields is the focus group method. In the health care industry, focus groups have been used to gain information needed to design surveys on client satisfaction (Budreau & Chase, 1994; Glass, 1995; Omar & Schiffman, 1995). One strength of the focus group method is that it brings clients together to discuss their perceptions about the services that they have received. This allows for interaction between group members, which stimulates thoughts and recall of experiences. Focus groups can be particularly helpful for the discovery of service problems, and suggestions for fixing those problems (Ford et al., 1997). The drawbacks to this method include the expense and time involved, as well as concerns about whether the views expressed in the focus group are representative of all clients (Ford et al., 1997). When deciding the best way to begin assessing client satisfaction with services at AR-CARES, the focus group method appealed to the evaluation team primarily for two reasons. First, the program management and evaluators believed that the brief program-created survey form currently being used was inadequate, and was not giving the management the breadth nor depth of information they needed about client satisfaction. Also, it was unclear which aspects of the AR-CARES program are important to clients’ satisfaction with the program. Second, while many AR-CARES clients are quiet and at times somewhat withdrawn when speaking with staff on an individual basis, they seem to feel more comfortable sharing their feelings and opinions in a group setting. Many therapy groups and educational activities take place in small group settings at AR-CARES, and thus clients are familiar with the small group discussion format. Thus, it was believed more information could be obtained from focus group sessions than from individual interviews. 6. Evaluation of client satisfaction at AR-CARES 6.1. Participants Having decided to conduct a series of focus group sessions with AR-CARES clients, in late summer of 1997, pro-
316
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
gram evaluators began to contact current and former clients of the program to recruit focus group participants. They attempted to contact by phone or mail, or both, all 40 clients that had entered the program since new management took over in August, 1995. Of the 40 possible clients, 16 volunteered to participate in the focus group sessions. Each volunteer had been admitted to the residential treatment program at AR CARES, but their lengths of stay varied considerably. All participants were mothers whose children had entered the treatment program with them. The majority of the volunteers were African American. An equal effort was made to include clients currently enrolled in the AR-CARES program, as well as successful “program completers,” program dropouts, and clients who were administratively discharged. Of the 16 participants, 7 were successful program completers, 2 were program dropouts, 2 were administratively discharged, and 5 were current clients. 6.2. Focus group sessions The program evaluators planned three focus group sessions to accommodate the number of volunteers. The evaluators made a special effort to keep each group small, as it had been the experience of program staff that large group sessions were often chaotic or unproductive with these clients. Each focus group session was held on the AR-CARES campus in a large office area with transportation and childcare offered to all clients. The staff of AR-CARES served an evening meal to all focus group participants, and allowed time for them to eat and visit with one another. Clients signed an informed consent form outlining the protection of their anonymity in the transcription of the audiorecording from the session and the future reporting of evaluation results. The focus group moderator encouraged the volunteers to speak freely and share all of their opinions and ideas. The author, who is a research assistant at AR-CARES, served as the moderator for each focus group session. The moderator of the focus group sessions designed a focus group guide to elicit information about client satisfaction in several different ways. First, with the knowledge that many participants may have been involved in a variety of treatment programs in the past, including treatment programs traditionally designed for male clients, information was needed about client expectations toward treatment. Because of these past experiences, it was hypothesized that some clients entering AR-CARES might have divergent ideas about the nature of substance abuse treatment. There was a concern that these divergent perspectives of treatment might influence client expectations about, and satisfaction with, treatment. Second, the moderator hoped to encourage participants to think beyond their past or even current ideas about substance abuse treatment to identify a variety of ways in which they could better be served. Thus, the second question on the focus group guide encouraged participants to brainstorm about the types of services they need, without
limiting their discussion to the particular services they had received in the AR-CARES program or in other treatment settings. Finally, more specific information was needed about the services participants had received at AR-CARES, and how well those services had met their needs. Therefore, the moderator designed two questions to elicit more specific information about client satisfaction with AR-CARES programming and services. In these two questions clients responded to a query about what they found to be most, and the least, helpful at AR-CARES. 6.3. Analysis of focus group data After the focus groups were completed, an administrative assistant with AR-CARES transcribed all audiotapes. The moderator reviewed and critiqued the final transcripts for accuracy as compared with written notes taken during each session. If the moderator discovered a discrepancy between the written notes and the transcript, the audiotape was consulted. Next, the moderator coded the transcripts into categories based on attitudes related to client satisfaction with substance abuse treatment. The transcripts were coded with the aid of Ethnograph v. 4.0 (Seidel et al., 1995). In the coding process, the unit of analysis included all word groups or sentences that related to client satisfaction. Each unit of analysis that was identified as part of the same concept was given a common label (Miles & Huberman, 1994). Next, a code book was developed in which each code word was given a definition. Working from these definitions, the codes were then grouped into common attitude themes corresponding to client satisfaction. Finally, the moderator merged the attitude themes into six overarching attitude patterns to develop a theoretical framework explaining client satisfaction.
7. Results from the focus group conversations These six attitude patterns related to client (a) expectations prior to entering treatment; (b) perception of treatment benefits; (c) attitudes about specific programming/service issues; (d) comments on the structure and policies of the drug treatment center; (e) preferences related to program staff; and (f) feelings about the inclusion of children in the residential treatment program. 7.1. Client expectations prior to entering treatment Participants’ ideas about treatment prior to entry varied from not even knowing that drug treatment programs existed, to having participated in many different programs. Clients reported that prior to entry into a treatment program they never knew what to expect, or how one program might be different from another. Participants also described feeling absolutely desperate, having tried and failed at every attempt to abstain from drugs on their own. They also reported that they found this level of desperation to be
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
necessary for successful completion of the treatment program. One client described that feeling of desperation in the following way: I remember how I felt. I didn’t want to live. It was like I wanted to live and I didn’t want to live. And that was because I thought that was the way I was supposed to go. I though I was supposed to smoke crack until I die. I said, I guess that’s what God had me here for. And that’s the feeling I had. 7.2. Client perceptions of treatment benefits Participants’ described the many ways in which treatment programs helped them by teaching them “to live life on life’s terms.” Participants described learning about themselves in treatment, including learning that they are not “bad people,” but that they have a disease for which they need treatment, and learning to love themselves regardless of what others think of them. They also reported that in treatment they learn to “live again” in all areas of life. As a result of treatment, they are better prepared not only for a life of sobriety, but also a life in which they can better handle relationships, finances, children, jobs, etc. They also described learning that they cannot succeed in treatment on their own. They need the support of others to help them in their recovery. It is not a “me” or an “I” program. It’s a “we” and an “us” program. Out there it was all about me, what I wanted, how I was gonna get it. It’s still all about me, but in a positive way. I know I can’t get it without the help of others. I know that I can’t do it alone. Participants also described how treatment helped them learn how to recognize their emotions and find positive ways to deal with them. In treatment they learned how to think about their feelings before acting impulsively, and they also learned the importance of a positive mental attitude. Also, they reported gaining a new “God consciousness,” and an understanding of the role of spirituality in their recovery. However, some participants expressed feeling that religion is often forced upon them in treatment programs. 7.3. Client attitudes about specific programming/service issues Many participants agreed that the most helpful groups and classes were the ones most closely related to their addiction issues. Examples of such groups include: Alcohol and Drug Education, Relapse Prevention, and Recovery Dynamics. Participants believe that the main focus of treatment should be to help clients learn about themselves and their disease. Many participants said that, particularly during clients’ early days in treatment, classes focusing on addiction issues should take precedence over classes that do not directly relate to addiction: nutrition, parenting, budgeting, etc. Some participants suggested that “life skills” classes should be optional, or perhaps delayed until clients have
317
been in the program for some reasonable length of time. They also expressed a desire to learn about life skills in a hands-on, practical manner. For example, they suggested that clients be allowed to take charge of the residence in terms of shopping, meal planning, etc. Participants also discussed programming issues around job preparation, saying that their lack of job preparation is a serious problem for them. Many stated that attending GED classes was helpful to them, and that work preparation classes and more job referrals are necessary to get them back into the work world. Participants also expressed that one of the benefits of long-term treatment is that there is time to help clients develop these work skills as well as related skills like coping and relationship building. On the issue of job training, one client stated: A lot of addicts and alcoholics are addicts and alcoholics just because they can’t support or take care of their family. It’s just like being on welfare. They are on welfare because they don’t have the education or the know how to go out there and get a job to provide for their family. And so the addicts and alcoholics are in jail, or in prison, because they are not able to go out here and get job, because they don’t have the education, they don’t know. Give me something I can use, some type of computer training, something I can use when I leave out of here, that I can provide for my family. Some type of job training, or be able to at least point me in the direction that I need to go in, you know. Another programming issue addressed by participants was group and individual therapy. They described how both group and individual sessions with their therapist were helpful in their recovery. In their view, the strength of the group therapy session is that it allows clients to see that they are not alone, and that there are others who have had similar experiences, feelings, and struggles. It is comforting for clients to discover that “it’s not just me.” They need to know that there are other people like them and they are not the only ones. Because you feel like you are the only one. You have one foot in the grave and the other foot on a banana peel. I thought I was crazy and had lost my mind, and you know, I should have known better. Individual therapy sessions were described as important for clients because in that setting they can share issues that may be too painful to discuss in a group setting. Participants also stated that it is especially helpful for them to be actively involved in the local recovery community while they are still in treatment. They shared a desire to attend as many AA/CA meetings as possible as part of their treatment. They also described the importance of sponsorship in their recovery process. On another issue, participants stated that they would like to see more emphasis placed upon giving assistance to cli-
318
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
ents who are transitioning out of treatment. Participants discussed the difficulty in finding housing, jobs, and transportation on their own. One suggestion which emerged from the focus groups was to add a transitional facility to the treatment program where clients who are working could live. In this way they could be spared from having to face all of the pressures of the outside world at once. Others emphasized needing referrals to help them find jobs, and transportation to help them get to work. 7.4. Clients comments on the structure and policies of the drug treatment center Participants expressed a very strong desire for a program where the rules are clear, are consistently enforced by all staff, and are consistently applied to all clients. They explained that the rules at AR-CARES were at times unclear and frequently changing, or inconsistently enforced. Participants reported that clients’ behaviors will not change until staff enforce rules consistently. They shared that when clients enter treatment, they lack discipline in their life, and they expect a treatment program to assist them in learning to live by the rules of society. Participants stated that a “phase” or “level” system is helpful in treatment, and they shared a variety of ideas about how that might work. One suggestion that emerged was to implement a 3- to 4-day detoxification period where clients are not responsible for normal caregiving and program activities; a 30-day restriction period for new clients in which they have no phone or pass privileges; an intensive phase in which clients deal exclusively with recovery issues; and a less intensive phase where clients might take a variety of classes or work on developing other skills. Participants reported that an important part of the phase system is that it should be used to hold clients accountable for their behavior and level of effort in treatment. Another policy issue raised by the participants was the issue of drug screens. They emphasized that drug screening should be frequent and consistently administered. They even commented that staff should watch clients while they urinate, and should use urine specimen cups with temperature gauges to prevent tampering. Participants also described the problems and pressures related to the facilities of the treatment program. They found it difficult to share a room not only with their children, but also with another family, as was the case at ARCARES. Participants described a need for some level of privacy, as well as a need to have a place where they can spend time as a family. Whether or not clients were disturbed by the living situation in part depends upon their expectation. For clients who expected a one-room dormitory based on their previous experience in treatment, sharing a room with another family did not seem like such a hardship. Another policy issue that participants’ expressed strong opinions on was the issue of smoking. Participants expressed that they should not be required to quit smoking while in drug treatment, as was emphasized at AR-CARES.
They feel that it is hard enough for them to give up drugs and alcohol in their lives without giving up smoking as well. They also commented that they receive mixed messages about smoking because some of the 12-step programs do not place the same emphasis on quitting smoking as some treatment programs do. 7.5. Client preferences related to program staff Participants agreed that the most important quality in a staff person in a treatment center is that they be able to “relate” to clients. Participants described the importance of a staff person understanding their experiences and feelings. Some participants felt that in order for a staff person to have that understanding, they needed to be a recovering person themselves. However, other participants described having had good relationships with staff members who never had a drug problem, but who nonetheless have a good understanding of clients. Participants also shared a concern regarding the qualifications of staff members. They stated that they do not want any “uncertified” person teaching a class. Participants also emphasized their need to feel supported by staff, not to be “beaten down” or reminded of what they have done wrong in their lives. Participants reported being acutely aware of all of their mistakes, and reminders and criticism are not viewed as helpful. If staff approach them too harshly they will be turned off to treatment. One participant described her experience, saying: When I came here, I was beat down enough. I needed support and care, and what I got was, “you was this and you was that.” Well, I know that, you know? Tell me what I can do to not be like that. Don’t tell me what I did and did wrong, and how I messed up. They told me “well your child was in pitiful shape cause her mother and daddy messed up.” Don’t tell me stuff like that. I can’t take it. Participants also reported feeling that it is important that staff members respect their feelings and concerns. When their complaints are dismissed by staff, it leaves the impression that client’s feeling are not taken seriously. Participants also shared concerns about staff becoming too personally involved with clients, saying that if staff members do not maintain a professional relationship with clients, then the client’s recovery process is jeopardized. Participants described good staff members as ones who do their job in a structured manner, and do not take their job home with them. Instead they are able to separate their personal and professional lives. 7.6. Client feelings about the inclusion of children in the residential treatment program Participants reported that they appreciate the ability to bring their children into treatment with them for a variety of reasons. Participants feel that having their children in treatment with them allows them to focus on their recovery rather than worrying about where their children are, or
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
whether they might lose custody of them. Participants also stated that they need to take on the responsibility of being mothers and acting as head of the household. Having their children in treatment with them allows them to do so. In addition, participants recognized that their children need treatment too. They have suffered because of their mother’s addiction and need to get help for their problems. One mother described how she felt about bringing her son into treatment with her. I’m grateful there was a place for me and my son. Because the first time, I couldn’t take him with me. The second time I knew I had to find some place for the both of us to go, because they told me that when I was out getting sick, he was getting sick too. Participants also shared that having their children in treatment with them allowed them to rebuild their relationships. They described learning about their children, and learning to identify their problems, understand their behaviors, and rebuild a trust relationship with their children. It’s teaching me like a closeness, like a bonding. I’m seeing things in him that I didn’t choose to ignore, but I did ignore in my addiction. And I know him a little bit better than I did when I walked in here. I’m learning him as I’m learning me. I’m learning to accept my son as he is. Participants also stated that more family therapy was needed to help them repair their relationships with their children. They feel that having separate therapists for mothers and children, as was the policy at AR-CARES, makes if difficult for the family to communicate. While they want their children to have an outlet for expressing their feelings, they want the child to learn to communicate with family members as well as therapists. They feel that more family therapy would help facilitate communication and strengthen their bond. Participants also described the difficulty of being in a living situation with children who have serious behavior problems. It is stressful for clients to be around not only their own children, but also other clients’ children who may also have behavior problems. Some parents view their children or other’s children as generally “bad.” Some participants feel that children who have serious behavior problems should be sent to another type of treatment facility, perhaps a psychiatric treatment center for children. Others believe that all of their families have problems to some degree, and families should be allowed to remain intact as they seek help. However, in spite of the difficulties, participants described seeing positive changes in their children as a result of their time in treatment. They believe that mothers and children alike are benefiting from treatment. 8. Discussion of focus group results In a way that quantitative surveys never did, the results from these focus groups offered clear evidence to ARCARES program evaluators and managers that clients do
319
have strong feelings about their experiences in treatment, as well as many suggestions about how treatment programs could be improved. Drawing on their experiences in a variety of treatment programs, both traditional programs and the more comprehensive women-specific AR-CARES program, the focus group participants were able to identify which services were most important to their recovery, what needs remain unmet, and what steps AR-CARES can take to further improve their treatment program. One interesting theme that emerged from the focus group sessions, was the preference the participants expressed for classes focusing primarily on addiction issues. Given the emphasis in the literature on the need for very comprehensive treatment for women, this finding is somewhat surprising. It has been repeatedly suggested that women want and need classes that build their “life skills” in areas such as budgeting, parenting, nutrition, and so forth. (Finkelstein, 1994; Nunes-Dinis & Barth, 1993). However, many of these participants expressed some frustration with these classes, and voiced a preference for strictly addiction related classes, such as relapse prevention. Longterm treatment programs for women may want to take seriously the participants’ suggestion that the earliest portion of treatment be devoted primarily to the issues most closely related to addiction, and that an emphasis on building “life skills” be postponed until later in the treatment episode. A related theme that emerged from the focus group sessions, was the frequently mentioned need for skill-building and support around job preparation. While GED classes were viewed as helpful for many participants, they need more support to move successfully into the work world. Participants suggested ways in which treatment programs could help them make this transition more effectively, including by offering more job training, job referrals, and transportation support. Additionally, participants affirmed the importance of group and individual therapy in the treatment program, and indicated that they were satisfied with the therapy they received. However, they also stressed a need for more family therapy, a recommendation that treatment programs may wish to heed. Participants also emphasized the impact of program staff on their satisfaction with treatment. Many described the good relationships they had developed with staff, and the importance of those relationships to their recovery. However, others expressed dissatisfaction with staff who were unprofessional, lacked qualifications, or were not supportive. Certainly these focus groups did not resolve the controversial question of whether or not clients feel it is necessary for program staff to be recovering from addictions themselves, however, participants did express strong opinions on the topic. Clearly, treatment program management should consider these diverse and strong opinions as they hire program staff. These are just a few of the important findings that emerged from the focus group sessions with AR-CARES clients. While these findings are important to evaluators and program managers, an examination of the AR-CARES experience using the focus group method to evaluate client satisfaction may be just as valuable.
320
N.A. Conners, K.K. Franklin / Journal of Substance Abuse Treatment 18 (2000) 313–320
9. Validity of the focus group method to program evaluation For AR-CARES, using focus group methodology to explore client satisfaction proved to be a useful endeavor for several reasons. First, most participants in each session seemed comfortable in the small group setting, and they freely shared details of their experiences both at ARCARES and in other treatment programs. As they shared their experiences with one another, the words of one participant often brought to mind something another participant had experienced, and thus they were able to share more completely than might have been possible otherwise. Second, the focus groups were judged successful because they resulted in participants’ sharing the type of information that the focus groups were designed to elicit: information about their expectations about treatment, their ideas about the services they need, as well as feedback about the specific services provided by AR-CARES. This information proved very useful to management at AR-CARES in their continual efforts to respond to the needs of their clients. Third, the focus groups provided the information necessary for the development of a client satisfaction survey form for use in this program. By allowing clients to identify what services and program issues are important to them, the program gained the information they needed to develop a useful and relevant survey tool. Fourth, and possibly most important, this process allowed for the recognition of clients as experts on their own treatment experience. An opportunity was provided for clients to voice their opinion and feel that they were making an important contribution to the treatment program. This is an important outcome of the focus group process, as it has been suggested that when clients are allowed input into the program design, the treatment program is enhanced and client retention improves (Laken & Hutchins, 1996).
10. Conclusion Overall, the focus group method of inquiry served as a valuable tool in eliciting rich and detailed data concerning the satisfaction of female clients participating in a residential substance abuse treatment program. While the focus group method was a time consuming, and labor intensive process, the data collected from these sessions has been useful to decision-makers at AR-CARES as they continue to develop programs and services to meet the needs of parenting women. Based on the experiences of the program evaluators, it is recommended that agencies consider the use of the focus group as an appropriate method for evaluating client satisfaction with drug treatment services. However, whether it is through surveys, individual interviews, focus groups, or other methods, what is important is that treatment programs make an effort to include clients’ perceptions of services as a part of the evaluation of their program. While it is not necessary that
evaluators or management always agree with clients’ assessment of their treatment needs or the helpfulness of services, it is important that their opinions be included in the debate.
Acknowledgment Support was provided by the Centers for Substance Abuse Treatment (6HD8 T100970-05-01)
References Budreau, G., & Chase, L. (1994). A family-centered approach to the development of a pediatric family satisfaction questionnaire. Pediatric Nursing 20, 604–608. Chan, M., Sorensen, J. L., Guydish, J., Tajima, B., & Acampora, A. (1997). Client satisfaction with drug abuse day treatment versus residential care. Journal of Drug Issues 27, 367–377. Etheridge, R. M., Craddock, S. G., Dunteman, G. H., & Hubbard, R. L. (1995). Treatment services in two national studies of community-based drug abuse treatment programs. Journal of Substance Abuse 7, 9–26. Finkelstein, N. (1994). Treatment issues for alcohol-and-drug-dependent pregnant and parenting women. Health and Social Work 19, 7–15. Ford, R. C., Bach, S. A., & Fottler, M. D. (1997). Methods of measuring patient satisfaction in health care organizations. Health Care Management Review 22, 74–89. Glass, A. P. (1995). Identifying issues important to patients on a hospital satisfaction questionnaire. Psychiatric Services 46, 83–85. Holcomb, W. R., Parker, J. C., & Leong, G. B. (1997). Outcomes of inpatients treated on a VA psychiatric unit and a substance abuse treatment unit. Psychiatric Services 48, 699–704. Laken, M. P., & Hutchins, E. (1996). Recruitment and Retention of Substance Using Pregnant and Parenting Women: Lessons Learned. Arlington, VA: National Center for Education in Maternal and Child Health. Lebow, J. (1982). Consumer satisfaction with mental health treatment. Psychological Bulletin 91, 244–259. Mavis, B. E., & Stoffelmayr, B. E. (1994). Program factors influencing client satisfaction in alcohol treatment. Journal of Substance Abuse 6, 345–354. McLellan, A. T., & Hunkeler, E. (1998). Patient satisfaction and outcomes in alcohol and drug abuse treatment. Psychiatric Services 49, 573–575. Merriam, S. B. (1998). Qualitative Research and Case Study Applications in Education. San Francisco: Jossey-Bass. Miles, M. B., & Huberman, A. M. (1994). An Expanded Sourcebook: Qualitative Data Analysis (2nd ed.). Thousand Oaks, CA: Sage. Nelson-Zlupko, L., Morrison Dore, M., Kauffman, E., & Kaltenbach, K. (1996). Women in recovery: their perceptions of treatment effectiveness. Journal of Substance Abuse Treatment 13, 51–59. Nunes-Dinis, M., & Barth, R. P. (1993). Cocaine treatment and outcome. Social Work 38, 611–617. Omar, M. A., & Schiffman, R. F. (1995). Pregnant women’s perceptions of prenatal care. Maternal-Child Nursing Journal 23, 132–144. Perreault, M., Leichner, P., Sabourin, S., & Gendreau, P. (1993). Patient satisfaction with outpatient psychiatric services: qualitative and quantitative assessments. Evaluation and Program Planning 16, 109–118. Seidel, J., Friese, S., & Leonard, D. C. (1995). The Ethnograph v4.0: A Users Guide. Amherst, MA: Qualis Research Associates. Substance Abuse and Mental Health Services Administration. (1998). Preliminary Results from the 1997 National Household Survey on Drug Abuse [On-line]. Available: http://www.samhsa.gov/oas/nhsda/nhsdafls.htm Williams, B. (1994). Patient satisfaction: a valid concept? Social Science and Medicine 38, 509–516.