- Email: [email protected]
Methodological Issues Associated With Group Intervention Research
Methodological Issues Associated With Group Intervention Research Shirley A. Murphy and L. Clark Johnson Interventions using a group format can be powerful treatment modalities. However, a review of nursing journals most likely to report the conduct of group research by nurses showed that less than 1% of nursing research reports used this approach and that none accounted for group-level effects in the analysis. This article discusses methodological issues inherent in group intervention research. We begin by offering examples of variables that can be incorporated into group research. We then present some challenges that researchers must address in collecting data when group formats are used. We end with recommendations for conducting group intervention research and by addressing issues associated with interpreting and reporting results. D 2006 Elsevier Inc. All rights reserved.
W
HEN INDIVIDUALS ARE brought together to achieve common goals, group members affect and influence one another (Bonito, 2002; Corey & Corey, 2002; Kenny, Mannetti, Pierro, Livi, & Kashy, 2002; Yalom, 1995). The posited benefits of group involvement for individuals include identification (i.e., the opportunity to confer with others who have had the same or similar experience; Baumeister & Leary, 1995; Corey & Corey, 2002); efficiency and cost-effectiveness (i.e., reaching several persons with the same message simultaneously; Yalom, 1995); and efficacy (i.e., feedback based on group consensus is said to make a more powerful impression on group members as compared with suggestions or advice given by an individual therapist; Gottlieb, 1988). In addition to the advantages of identification, efficiency, and efficacy, group environments can facilitate trust and safety for learning and practicing new behaviors. From the Department of Psychosocial and Community Health, School of Nursing, University of Washington, Seattle, WA. Address reprint requests to Shirley A. Murphy, PhD, Professor Emeritus, Department of Psychosocial and Community Health, School of Nursing, University of Washington, Box 357263, Seattle, WA 98195-7263. E-mail address: [email protected] (S.A Murphy). B 2006 Elsevier Inc. All rights reserved. 0883-9417/1801-0005$30.00/0 doi:10.1016/j.apnu.2006.05.003
276
However, the data obtained from groups are frequently analyzed using models intended for individuals rather than groups. The benefits of group involvement are recognized by psychiatric/mental health nurses. The group intervention research reports cited in this paragraph are examples of studies conducted by nurses in the specialty and/or that involve issues relevant to the specialty. Gross and Grady (2002) and WebsterStratton and Hammond (1998) used group formats to test the efficacy of various approaches to increase effective parenting behaviors. Hatton and Kaiser (2004) used a group format to test the impact of specified regimens on health deficits among homeless women. Kreidler (2005) conducted a 50-session group intervention study on sexually abused women with and those without a chronic mental illness. According to the published reports, the authors did not consider the influence of one group member on another, referred to in the literature as independence of observations (Bonito, 2002; Kenny et al., 2002). Perraud, Farran, Loukissa, and Paun (2004) described group characteristics (GCs) observed during the course of a clinical trial involving caregivers of persons with Alzheimer’s disease, but the identification of GCs apparently was not incorporated into the initial design of the study. Despite the foregoing examples, group intervention studies are surprisingly rare in the published nursing literature. Perhaps nurse investigators
Archives of Psychiatric Nursing, Vol. 20, No. 6 (December), 2006: pp 276–281
GROUP RESEARCH
publish group intervention results in non-nursing journals more frequently than they do in nursing journals. This seems unlikely because nursing research journals are used widely to report the results of non-group intervention studies. We conducted a review of the literature to determine the extent to which nurse investigators published group studies in the nursing literature and to note the type of data collected and analyzed. We first selected five nursing journals likely to publish group intervention data, Nursing Research, Western Journal of Nursing Research, Research in Nursing & Health, Archives in Psychiatric Nursing, and Issues in Mental Health Nursing. We hand-searched all issues for the past 6 years (1999– 2005) of these selected journals and selected review articles that indicated an intervention was conducted. We then read the abstracts of these research reports. If a group intervention was conducted, we read the methods, results, and discussion sections to determine if and how group variables were handled by the investigators. A 6-year period was selected to provide a sufficient amount of time to identify a trend revealing the number of recent articles published on the topic of interest. Approximately 900 articles were published in the five aforementioned journals between 1999 and 2005. Only 19 publications that included group research reports based on our search methods were identified. The number of group intervention articles that we discovered led us to ask ourselves another question: are nurses interested in conducting group research exposed to group methods in nursing research journals and textbooks? To answer this question, we reviewed the same five journals listed for articles on group methods and nine basic and advanced nursing research textbooks published over the past 20 years (Brink & Wood, 1998; Burns & Grove, 2001; Munro, 2001; Polit & Beck, 2004; Sidani & Braden, 1998; Treece & Treece, 1986; Wilson, 1985; Wood & Haber, 2002; Woods & Catanzaro, 1988). The review showed that the collection and analysis of data from dyads, groups, or families were not included in any of these texts. Our review suggested that group intervention research is an underutilized method in nursing science whereas issues concerning group research are currently addressed by other disciplines, specifically that data collected from groups for research purposes require careful planning and management to avoid potential bias in reporting
277
study results (Bonito, 2002; Kenny et al., 2002; Moritz & Watson, 1998). It is the paucity of methodological concerns involving group research in the nursing literature that prompted us to write this article. The goal of our review was to initiate a discussion of methodological issues inherent in group intervention research. We address three major concerns—nonindependent observations, selection of the unit of analysis, and sampling— and have written this article for researchers rather than for those who provide statistical and other expert consultations. A HEURISTIC MODEL ILLUSTRATING GROUP DATA
In a typical nursing intervention study, the researcher will likely administer questionnaires and/or obtain biobehavioral data for pretest and posttest observations to determine the impact of an intervening treatment. This study design assumes that the effect (c) can be directly estimated as the difference between pretest and posttest intervention observations as shown in the following equation: c ¼ ðOBSPost OBSPre Þ This approach fails to account for the systematic and potentially crucial impact of person characteristics (PCs) and GCs, which, as pointed out by several authors (Hoyle, Georgesen, & Webster, 2001, Kenny et al., 2002, Moritz & Watson, 1998), can result in a biased estimate of an intervention’s impact. Therefore, group intervention studies need to be designed such that they can be analyzed with a model that incorporates variations in PCs and GCs and their interactions, as shown in the following equation: c ¼ ðOBSPost OBSPre Þ ðPC þ GC þ PCTGCÞ The second term added to the model (i.e., the second set of parentheses) implies that the apparent treatment effect has been adjusted to remove the bias associated with PCs and GCs. We assert that group processes are important and that interventions should be conducted so that these additional components of the intervention can be estimated. To accomplish this objective, investigators must collect and appropriately incorporate into the analysis data from two distinct albeit intertwined units of analysis: participants and groups. Table 1 lists some of these PCs and GCs. The column on PCs lists factors that individuals
278
MURPHY AND JOHNSON
Table 1. Examples of PCs and GCs Inherent in Group Research PCs
GCs
Sex Symptom level Readiness to change Frequency of attendance Extent of participation Member perceptions of other members Perceptions of group leader(s) Perceptions of potential group success Sense of belonging Identification with other members
Location of group meetings Group composition Group size Frequency and duration of sessions Stage of group life Group norms Level of group cohesion Leader characteristics Leader skills/behavior Collective efficacy
contribute to a group environment. For example, member perceptions of other group members are PCs. The second column in Table 1 on GCs lists some properties of groups that govern group behavior, such as confidentiality, stages of group life, group size, and group norms that determine how trust is developed, how conflict is resolved, and what beliefs about the group develop over time. These lists are not intended to be exhaustive but are used here to illustrate the types of data that can be collected when using a group format. The examples in Table 1 come from the numerous articles and texts cited in this article. The following vignette illustrates the types of data that can be included in an intervention study. Suppose researchers are conducting a study to prevent osteoporosis. A community-based sample with a global recruitment goal of 400 premenopausal women is initiated. As study participants become available, they are recruited and randomized into group cohorts (e.g., n = 10/group) that represent the experimental and control conditions of the study. The treatment protocols include two components, diet and exercise, with each component having two parts. The diet component includes calcium-rich food and calcium supplements, whereas the exercise component includes weight resistance and an additional choice of activity (e.g., swimming, walking, or bicycling). The strength of the intervention (Yeaton & Sechrest, 1981) is addressed by the inclusion of in-group and at-home components. During group sessions, members would be instructed about diet and exercise. The at-home component consists of
self-recording and monitoring the targeted behaviors between group sessions. Each cohort is to be observed for 6 months. The research team therefore expects to conduct 20 cohorts and wants to capture data that describe individual differences and the constituent differences that characterize the variation across cohorts. What might some of these variables be? How does one measure them? What impact do they have on the design and analysis of the study? These are methodological issues that require careful consideration as the study is being planned. To provide an indication of how this might begin, we now provide examples of both PCs and GCs in the context of our hypothetical research study. In as much as osteoporosis is asymptomatic in its early stages (National Osteoporosis Foundation, 1998), it may be difficult to recruit premenopausal women to participate in a study with the highly demanding protocol identified above. Therefore, readiness to change is a PC (Table 1) that is likely play an important role in the treatment’s efficacy (e.g., high levels of readiness to change will tend to potentiate the effect). In fact, this person-specific factor is of such central importance to our hypothetical study that it could be an important predictor of the success of the intervention. Group cohesion, a potent group process variable (Table 1), has both instrumental (purpose) and affective (social bonding) dimensions (Carron, Brawley, & Widmeyer, 1998). Cohesion affects members’ decisions about remaining in a group. It can be influenced by group leaders but is highly dependent on the chemistry that develops in the group. Because motivation to continue in the study is an essential part of success, the level of group cohesion may be an important concomitant variable. Person characteristics and GCs also interact and influence the results of an intervention in this way. This variable interaction is shown by the formula presented at the beginning of this section. For example, the PC sense of belonging influences and is influenced by the GC level of group cohesion. CHALLENGES ASSOCIATED WITH INTERVENTIONS USING GROUP FORMATS
Nonindependent Observations Interactions among group members are the central tenet that explains group behavior and in most cases are not independent observations. Because each
GROUP RESEARCH
member’s behavior influences at least one other member’s behavior, the data obtained from each group member are not considered independent. Three factors are likely to lead to nonindependence of group data: compositional effects, common fate, and mutual influence (Kenny & Judd, 1986). Compositional effects occur when study participants are nonrandomly assigned to groups (e.g., married couples being assigned to the same group). However, nonrandom assignment is a common approach because it fulfills the goals of the study (e.g., increasing parenting skills and awareness among the marital dyads recruited for study). Common fate occurs when members of a group are linked to another unit in their environments (e.g., each has the same primary care physician). Mutual influence occurs when one group member adopts a behavior of another group member (e.g., making increasingly positive remarks about being a member of the group after hearing another group member do so). The independence of observations is a fundamental assumption of linear models such as analysis of variance and multiple regression (Hays, 1988; Tabachnick & Fidell, 2001). If data are not independent, then the use of linear models is inappropriate. According to Kenny et al. (2002), violating the independence assumption is more serious than violating other assumptions of linear models (i.e., normality or homogeneity of variance). The consequences of nonindependence of data are serious and can result in the distortion of the estimate of the error variance such that standard errors, P values, confidence intervals, and most effect-size measures are invalid (Hoyle et al., 2001; Kenny et al., 2002). Although the assumption of normality can easily be determined by plotting the data, this is not the case for the independence assumption. Because there is no simple way to discover this, the problem may go undetected and bias the study results. Therefore, it is important to determine the independence or nonindependence of data before the analyses planned to determine the effects of a group intervention. Investigators have two choices: they can assume that the group data are nonindependent and account for this factor in the analyses of choice or they can compute the intraclass correlation for the dependent variable(s). Kenny et al. provided two methods to compute intraclass correlation. Both methods yield very similar estimates as long as there are five or more
279
groups included in the computation; however, both measures have a slight negative bias: they tend to underestimate the level of nonindependence. If the correlation is statistically significant with an a level of .25, then there is evidence of nonindependence. Statistical consultation is recommended to select an appropriate test to compute the intraclass correlation and to interpret the results for subsequent analysis. Selection of the Unit of Analysis The data collected from a group intervention study inherently contain two components that affect the intervention: individual and group data. The study design must include data collected from both of these components because group members reflect their own thoughts and feelings as well as those shared by other members of the group. In other words, the data obtained have both unique and shared variabilities. Separating the interactions between the two components and separating their effects from the intervention are critical issues. Moritz and Watson (1998) provided an excellent discussion of the shortcomings associated with the omission of one level of data over another. According to Moritz and Watson, a so-called single-level analysis of outcomes is inappropriate because this analysis strategy suffers from three fundamental biases: overgeneralization, underestimation of cross-level effects, and underestimation of the effects that individuals have on their environments. Overgeneralization may occur in a single-level analysis when the researcher assumes that a concept at one level will have the same relationships as a similar concept at another level (e.g., if self-efficacy is assumed to have the same effects as collective efficacy). Single-level research also underestimates cross-level effects; that is, the effects of individuals on the group and the effects of group membership on individuals. For example, the single-level analysis of group as the unit of analysis underestimates the effects that individual members have on the group environment, such as some members being more bon taskQ than others. Hoyle et al. (2001) and Pollack (1998) suggested the use of hierarchical linear modeling as a way to analyze data simultaneously from cross-level (individual and group) studies. A discussion of data analysis is needed but requires extensive explanation, and hence beyond the scope of the current article.
280
MURPHY AND JOHNSON
Sample Considerations One implication of the foregoing discussion is the need to implement a complex sampling plan. Essential to this discussion is the recognition that estimates of both personal and group effects will require a sampling plan that incorporates both units of observation. Just as many individuals are needed to estimate the population effects for persons, so too are many groups needed to estimate effects for the group level of analysis. In the hypothetical study presented in the previous section, this is accomplished by the enrollment of many cohorts. Subsequent analysis of the data views group cohorts as a unit of analysis (n = 20) and estimates the impact of GCs from the variability of GC indices selected for inclusion in the study. A study with single intervention and control group is, in this context, not an adequate basis from which to infer an intervention’s effectiveness (Kenny et al., 2002). This fact presents a dilemma for the researcher conducting a pilot study or a single-group cohort study. A single cohort with intervention and control groups cannot be generalized to the larger population of interest because the group variables (e.g., those that change as the intervention is repeated) are fixed. In a single group (n = 1), the investigator cannot observe variance and hence cannot estimate the extent to which the group-level variables affect efficacy. Investigators reporting the results of a study in which a single intervention group was observed can only avoid the nonindependence problem if they explicitly limit the interpretation of results to the group that they observed. In other words, a pilot study can only be used to demonstrate the feasibility of the intervention. To move past the hypothesis generation phase, investigators must enroll multiple cohorts simultaneously and/or enroll cohorts over time. We did not find any specific recommendation in the literature as to the number of group cohorts. Needless to say, investigators will want sufficient data to show effect sizes if they exist. CONCLUSIONS
In this article, methodological issues associated with group intervention research have been identified. Our intent was to target researchers rather than experts on methods. Although further discussion among researchers is recommended, the
implications of this article suggest that intervention studies must be designed to accommodate individual and group effects. Three recommendations to do this are offered to investigators when conducting group studies: (1) recognize that interdependence, not independence, is the hallmark of group dynamics; (2) when conducting intervention studies, measure GCs (these data assist researchers in separating their effects from treatment effects); and (3) understand that inferences concerning interventions in which group dynamics play a significant role can only be made if the analysis incorporates an impact of these dynamics (this requires that an adequate sample of groups from which these effects can be estimated be available). REFERENCES Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497 – 529. Bonito, J. A. (2002). The analysis of participation in small groups. Methodological and conceptual issues related to interdependence. Small Group Research, 31, 412 – 438. Brink, P. J., & Wood, M. J. (1998). Advanced design in nursing research (2nd ed.). Thousand Oaks, CA7 Sage. Burns, N., & Grove, S. K. (2001). The practice of nursing research (4th ed.). Philadelphia7 W.B. Saunders Co. Carron, A. V., Brawley, L. R., & Widmeyer, W. N. (1998). The measurement of cohesiveness in sport groups. In J. L. Duda (Ed.), Advances in sport and exercise psychology measurement (pp. 213 – 226). Morgantown, WV7 Fitness Information Technology. Corey, M. S., & Corey, G. (2002). Groups: Process and practice (6th ed., pp. 224–244). Pacific Grove, CA7 Brooks/ Cole. Gottlieb, B. H. (1988). Marshalling social support: The state of the art in research and practice. In B. H. Gottlieb (Ed.), Marshalling social support: Formats, processes, and effects (pp. 11–51). Newbury Park, CA7 Sage. Gross, D., & Grady, J. (2002). Group-based parent training for preventing mental health disorders in children. Issues in Mental Health Nursing, 23, 367 – 383. Hatton, D. C., & Kaiser, L. (2004). Methodological and ethical issues emerging from pilot testing an intervention with women in a transitional shelter. Western Journal of Nursing Research, 26, 129 – 136. Hays, W. L. (1988). Statistics (4th ed., pp. 325–375). New York7 Holt, Rinehart, & Winston. Hoyle, R. H., Georgesen, J. C., & Webster, J. M. (2001). Analyzing data from individuals in groups: The past, the present, and the future. Group Dynamics: Theory, Research, and Practice, 5, 41 – 47. Kenny, D. A., & Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422 – 431.
GROUP RESEARCH
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83, 126 – 137. Kreidler, M. (2005). Group therapy for survivors of childhood sexual abuse who have chronic mental illness. Archives of Psychiatric Nursing, 19, 176 – 183. Moritz, S. E., & Watson, C. B. (1998). Levels of analysis issues in group psychology: Using efficacy as an example of a multilevel model. Group Dynamics: Theory, Research, and Practice, 6(4), 267 – 276. Munro, B. H. (2001). Statistical methods for health care research (4th ed.). Philadelphia7 Lippincott Williams & Wilkins. National Osteoporosis Foundation (1998). Physician’s guide to prevention and treatment of osteoporosis. Washington, DC7 Author. Perraud, S., Farran, C. J., Loukissa, D., & Paun, O. (2004). Alzheimer’s disease caregiving information and skills: Part III. Group process issues and concerns. Research in Nursing & Health, 27, 110 – 120. Polit, D. F., & Beck, C. T. (2004). Nursing research: Principles and methods (7th ed.). Philadelphia7 Lippincott Williams & Wilkins. Pollack, B. N. (1998). Hierarchical linear modeling and the bunit of analysisQ problem: A solution for analyzing responses of intact group members. Group Dynamics: Theory, Research, and Practice, 2, 299 – 312.
281
Sidani, S., & Braden, C. J. (1998). Evaluating nursing interventions: A theory-driven approach. Thousand Oaks, CA7 Sage. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed., pp. 111–170). Boston7 Allyn & Bacon. Treece, E., & Treece, J. (1986). Elements of nursing research. St. Louis, MO7 Mosby. Webster-Stratton, C., & Hammond, M. (1998). Conduct problems and level of social competence in Head Start children: Prevalence, pervasiveness, and associated risk factors. Clinical Child and Family Psychology Review, 1, 101 – 124. Wilson, H. S. (1985). Research in nursing. Menlo Park, CA7 Addison-Wesley Publishing Co. Wood, G., & Haber, J. (2002). Nursing research, methods, critical appraisal & utilization (5th ed.). St. Louis, MO7 Mosby. Woods, N. F., & Catanzaro, M. (1988). Nursing research: Theory and practice. St. Louis, MO7 Mosby. Yalom, I. (1995). The theory and practice of group psychotherapy (4th ed., pp. 1– 99). New York7 Basic Books. Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156 – 167.
W
HEN INDIVIDUALS ARE brought together to achieve common goals, group members affect and influence one another (Bonito, 2002; Corey & Corey, 2002; Kenny, Mannetti, Pierro, Livi, & Kashy, 2002; Yalom, 1995). The posited benefits of group involvement for individuals include identification (i.e., the opportunity to confer with others who have had the same or similar experience; Baumeister & Leary, 1995; Corey & Corey, 2002); efficiency and cost-effectiveness (i.e., reaching several persons with the same message simultaneously; Yalom, 1995); and efficacy (i.e., feedback based on group consensus is said to make a more powerful impression on group members as compared with suggestions or advice given by an individual therapist; Gottlieb, 1988). In addition to the advantages of identification, efficiency, and efficacy, group environments can facilitate trust and safety for learning and practicing new behaviors. From the Department of Psychosocial and Community Health, School of Nursing, University of Washington, Seattle, WA. Address reprint requests to Shirley A. Murphy, PhD, Professor Emeritus, Department of Psychosocial and Community Health, School of Nursing, University of Washington, Box 357263, Seattle, WA 98195-7263. E-mail address: [email protected] (S.A Murphy). B 2006 Elsevier Inc. All rights reserved. 0883-9417/1801-0005$30.00/0 doi:10.1016/j.apnu.2006.05.003
276
However, the data obtained from groups are frequently analyzed using models intended for individuals rather than groups. The benefits of group involvement are recognized by psychiatric/mental health nurses. The group intervention research reports cited in this paragraph are examples of studies conducted by nurses in the specialty and/or that involve issues relevant to the specialty. Gross and Grady (2002) and WebsterStratton and Hammond (1998) used group formats to test the efficacy of various approaches to increase effective parenting behaviors. Hatton and Kaiser (2004) used a group format to test the impact of specified regimens on health deficits among homeless women. Kreidler (2005) conducted a 50-session group intervention study on sexually abused women with and those without a chronic mental illness. According to the published reports, the authors did not consider the influence of one group member on another, referred to in the literature as independence of observations (Bonito, 2002; Kenny et al., 2002). Perraud, Farran, Loukissa, and Paun (2004) described group characteristics (GCs) observed during the course of a clinical trial involving caregivers of persons with Alzheimer’s disease, but the identification of GCs apparently was not incorporated into the initial design of the study. Despite the foregoing examples, group intervention studies are surprisingly rare in the published nursing literature. Perhaps nurse investigators
Archives of Psychiatric Nursing, Vol. 20, No. 6 (December), 2006: pp 276–281
GROUP RESEARCH
publish group intervention results in non-nursing journals more frequently than they do in nursing journals. This seems unlikely because nursing research journals are used widely to report the results of non-group intervention studies. We conducted a review of the literature to determine the extent to which nurse investigators published group studies in the nursing literature and to note the type of data collected and analyzed. We first selected five nursing journals likely to publish group intervention data, Nursing Research, Western Journal of Nursing Research, Research in Nursing & Health, Archives in Psychiatric Nursing, and Issues in Mental Health Nursing. We hand-searched all issues for the past 6 years (1999– 2005) of these selected journals and selected review articles that indicated an intervention was conducted. We then read the abstracts of these research reports. If a group intervention was conducted, we read the methods, results, and discussion sections to determine if and how group variables were handled by the investigators. A 6-year period was selected to provide a sufficient amount of time to identify a trend revealing the number of recent articles published on the topic of interest. Approximately 900 articles were published in the five aforementioned journals between 1999 and 2005. Only 19 publications that included group research reports based on our search methods were identified. The number of group intervention articles that we discovered led us to ask ourselves another question: are nurses interested in conducting group research exposed to group methods in nursing research journals and textbooks? To answer this question, we reviewed the same five journals listed for articles on group methods and nine basic and advanced nursing research textbooks published over the past 20 years (Brink & Wood, 1998; Burns & Grove, 2001; Munro, 2001; Polit & Beck, 2004; Sidani & Braden, 1998; Treece & Treece, 1986; Wilson, 1985; Wood & Haber, 2002; Woods & Catanzaro, 1988). The review showed that the collection and analysis of data from dyads, groups, or families were not included in any of these texts. Our review suggested that group intervention research is an underutilized method in nursing science whereas issues concerning group research are currently addressed by other disciplines, specifically that data collected from groups for research purposes require careful planning and management to avoid potential bias in reporting
277
study results (Bonito, 2002; Kenny et al., 2002; Moritz & Watson, 1998). It is the paucity of methodological concerns involving group research in the nursing literature that prompted us to write this article. The goal of our review was to initiate a discussion of methodological issues inherent in group intervention research. We address three major concerns—nonindependent observations, selection of the unit of analysis, and sampling— and have written this article for researchers rather than for those who provide statistical and other expert consultations. A HEURISTIC MODEL ILLUSTRATING GROUP DATA
In a typical nursing intervention study, the researcher will likely administer questionnaires and/or obtain biobehavioral data for pretest and posttest observations to determine the impact of an intervening treatment. This study design assumes that the effect (c) can be directly estimated as the difference between pretest and posttest intervention observations as shown in the following equation: c ¼ ðOBSPost OBSPre Þ This approach fails to account for the systematic and potentially crucial impact of person characteristics (PCs) and GCs, which, as pointed out by several authors (Hoyle, Georgesen, & Webster, 2001, Kenny et al., 2002, Moritz & Watson, 1998), can result in a biased estimate of an intervention’s impact. Therefore, group intervention studies need to be designed such that they can be analyzed with a model that incorporates variations in PCs and GCs and their interactions, as shown in the following equation: c ¼ ðOBSPost OBSPre Þ ðPC þ GC þ PCTGCÞ The second term added to the model (i.e., the second set of parentheses) implies that the apparent treatment effect has been adjusted to remove the bias associated with PCs and GCs. We assert that group processes are important and that interventions should be conducted so that these additional components of the intervention can be estimated. To accomplish this objective, investigators must collect and appropriately incorporate into the analysis data from two distinct albeit intertwined units of analysis: participants and groups. Table 1 lists some of these PCs and GCs. The column on PCs lists factors that individuals
278
MURPHY AND JOHNSON
Table 1. Examples of PCs and GCs Inherent in Group Research PCs
GCs
Sex Symptom level Readiness to change Frequency of attendance Extent of participation Member perceptions of other members Perceptions of group leader(s) Perceptions of potential group success Sense of belonging Identification with other members
Location of group meetings Group composition Group size Frequency and duration of sessions Stage of group life Group norms Level of group cohesion Leader characteristics Leader skills/behavior Collective efficacy
contribute to a group environment. For example, member perceptions of other group members are PCs. The second column in Table 1 on GCs lists some properties of groups that govern group behavior, such as confidentiality, stages of group life, group size, and group norms that determine how trust is developed, how conflict is resolved, and what beliefs about the group develop over time. These lists are not intended to be exhaustive but are used here to illustrate the types of data that can be collected when using a group format. The examples in Table 1 come from the numerous articles and texts cited in this article. The following vignette illustrates the types of data that can be included in an intervention study. Suppose researchers are conducting a study to prevent osteoporosis. A community-based sample with a global recruitment goal of 400 premenopausal women is initiated. As study participants become available, they are recruited and randomized into group cohorts (e.g., n = 10/group) that represent the experimental and control conditions of the study. The treatment protocols include two components, diet and exercise, with each component having two parts. The diet component includes calcium-rich food and calcium supplements, whereas the exercise component includes weight resistance and an additional choice of activity (e.g., swimming, walking, or bicycling). The strength of the intervention (Yeaton & Sechrest, 1981) is addressed by the inclusion of in-group and at-home components. During group sessions, members would be instructed about diet and exercise. The at-home component consists of
self-recording and monitoring the targeted behaviors between group sessions. Each cohort is to be observed for 6 months. The research team therefore expects to conduct 20 cohorts and wants to capture data that describe individual differences and the constituent differences that characterize the variation across cohorts. What might some of these variables be? How does one measure them? What impact do they have on the design and analysis of the study? These are methodological issues that require careful consideration as the study is being planned. To provide an indication of how this might begin, we now provide examples of both PCs and GCs in the context of our hypothetical research study. In as much as osteoporosis is asymptomatic in its early stages (National Osteoporosis Foundation, 1998), it may be difficult to recruit premenopausal women to participate in a study with the highly demanding protocol identified above. Therefore, readiness to change is a PC (Table 1) that is likely play an important role in the treatment’s efficacy (e.g., high levels of readiness to change will tend to potentiate the effect). In fact, this person-specific factor is of such central importance to our hypothetical study that it could be an important predictor of the success of the intervention. Group cohesion, a potent group process variable (Table 1), has both instrumental (purpose) and affective (social bonding) dimensions (Carron, Brawley, & Widmeyer, 1998). Cohesion affects members’ decisions about remaining in a group. It can be influenced by group leaders but is highly dependent on the chemistry that develops in the group. Because motivation to continue in the study is an essential part of success, the level of group cohesion may be an important concomitant variable. Person characteristics and GCs also interact and influence the results of an intervention in this way. This variable interaction is shown by the formula presented at the beginning of this section. For example, the PC sense of belonging influences and is influenced by the GC level of group cohesion. CHALLENGES ASSOCIATED WITH INTERVENTIONS USING GROUP FORMATS
Nonindependent Observations Interactions among group members are the central tenet that explains group behavior and in most cases are not independent observations. Because each
GROUP RESEARCH
member’s behavior influences at least one other member’s behavior, the data obtained from each group member are not considered independent. Three factors are likely to lead to nonindependence of group data: compositional effects, common fate, and mutual influence (Kenny & Judd, 1986). Compositional effects occur when study participants are nonrandomly assigned to groups (e.g., married couples being assigned to the same group). However, nonrandom assignment is a common approach because it fulfills the goals of the study (e.g., increasing parenting skills and awareness among the marital dyads recruited for study). Common fate occurs when members of a group are linked to another unit in their environments (e.g., each has the same primary care physician). Mutual influence occurs when one group member adopts a behavior of another group member (e.g., making increasingly positive remarks about being a member of the group after hearing another group member do so). The independence of observations is a fundamental assumption of linear models such as analysis of variance and multiple regression (Hays, 1988; Tabachnick & Fidell, 2001). If data are not independent, then the use of linear models is inappropriate. According to Kenny et al. (2002), violating the independence assumption is more serious than violating other assumptions of linear models (i.e., normality or homogeneity of variance). The consequences of nonindependence of data are serious and can result in the distortion of the estimate of the error variance such that standard errors, P values, confidence intervals, and most effect-size measures are invalid (Hoyle et al., 2001; Kenny et al., 2002). Although the assumption of normality can easily be determined by plotting the data, this is not the case for the independence assumption. Because there is no simple way to discover this, the problem may go undetected and bias the study results. Therefore, it is important to determine the independence or nonindependence of data before the analyses planned to determine the effects of a group intervention. Investigators have two choices: they can assume that the group data are nonindependent and account for this factor in the analyses of choice or they can compute the intraclass correlation for the dependent variable(s). Kenny et al. provided two methods to compute intraclass correlation. Both methods yield very similar estimates as long as there are five or more
279
groups included in the computation; however, both measures have a slight negative bias: they tend to underestimate the level of nonindependence. If the correlation is statistically significant with an a level of .25, then there is evidence of nonindependence. Statistical consultation is recommended to select an appropriate test to compute the intraclass correlation and to interpret the results for subsequent analysis. Selection of the Unit of Analysis The data collected from a group intervention study inherently contain two components that affect the intervention: individual and group data. The study design must include data collected from both of these components because group members reflect their own thoughts and feelings as well as those shared by other members of the group. In other words, the data obtained have both unique and shared variabilities. Separating the interactions between the two components and separating their effects from the intervention are critical issues. Moritz and Watson (1998) provided an excellent discussion of the shortcomings associated with the omission of one level of data over another. According to Moritz and Watson, a so-called single-level analysis of outcomes is inappropriate because this analysis strategy suffers from three fundamental biases: overgeneralization, underestimation of cross-level effects, and underestimation of the effects that individuals have on their environments. Overgeneralization may occur in a single-level analysis when the researcher assumes that a concept at one level will have the same relationships as a similar concept at another level (e.g., if self-efficacy is assumed to have the same effects as collective efficacy). Single-level research also underestimates cross-level effects; that is, the effects of individuals on the group and the effects of group membership on individuals. For example, the single-level analysis of group as the unit of analysis underestimates the effects that individual members have on the group environment, such as some members being more bon taskQ than others. Hoyle et al. (2001) and Pollack (1998) suggested the use of hierarchical linear modeling as a way to analyze data simultaneously from cross-level (individual and group) studies. A discussion of data analysis is needed but requires extensive explanation, and hence beyond the scope of the current article.
280
MURPHY AND JOHNSON
Sample Considerations One implication of the foregoing discussion is the need to implement a complex sampling plan. Essential to this discussion is the recognition that estimates of both personal and group effects will require a sampling plan that incorporates both units of observation. Just as many individuals are needed to estimate the population effects for persons, so too are many groups needed to estimate effects for the group level of analysis. In the hypothetical study presented in the previous section, this is accomplished by the enrollment of many cohorts. Subsequent analysis of the data views group cohorts as a unit of analysis (n = 20) and estimates the impact of GCs from the variability of GC indices selected for inclusion in the study. A study with single intervention and control group is, in this context, not an adequate basis from which to infer an intervention’s effectiveness (Kenny et al., 2002). This fact presents a dilemma for the researcher conducting a pilot study or a single-group cohort study. A single cohort with intervention and control groups cannot be generalized to the larger population of interest because the group variables (e.g., those that change as the intervention is repeated) are fixed. In a single group (n = 1), the investigator cannot observe variance and hence cannot estimate the extent to which the group-level variables affect efficacy. Investigators reporting the results of a study in which a single intervention group was observed can only avoid the nonindependence problem if they explicitly limit the interpretation of results to the group that they observed. In other words, a pilot study can only be used to demonstrate the feasibility of the intervention. To move past the hypothesis generation phase, investigators must enroll multiple cohorts simultaneously and/or enroll cohorts over time. We did not find any specific recommendation in the literature as to the number of group cohorts. Needless to say, investigators will want sufficient data to show effect sizes if they exist. CONCLUSIONS
In this article, methodological issues associated with group intervention research have been identified. Our intent was to target researchers rather than experts on methods. Although further discussion among researchers is recommended, the
implications of this article suggest that intervention studies must be designed to accommodate individual and group effects. Three recommendations to do this are offered to investigators when conducting group studies: (1) recognize that interdependence, not independence, is the hallmark of group dynamics; (2) when conducting intervention studies, measure GCs (these data assist researchers in separating their effects from treatment effects); and (3) understand that inferences concerning interventions in which group dynamics play a significant role can only be made if the analysis incorporates an impact of these dynamics (this requires that an adequate sample of groups from which these effects can be estimated be available). REFERENCES Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497 – 529. Bonito, J. A. (2002). The analysis of participation in small groups. Methodological and conceptual issues related to interdependence. Small Group Research, 31, 412 – 438. Brink, P. J., & Wood, M. J. (1998). Advanced design in nursing research (2nd ed.). Thousand Oaks, CA7 Sage. Burns, N., & Grove, S. K. (2001). The practice of nursing research (4th ed.). Philadelphia7 W.B. Saunders Co. Carron, A. V., Brawley, L. R., & Widmeyer, W. N. (1998). The measurement of cohesiveness in sport groups. In J. L. Duda (Ed.), Advances in sport and exercise psychology measurement (pp. 213 – 226). Morgantown, WV7 Fitness Information Technology. Corey, M. S., & Corey, G. (2002). Groups: Process and practice (6th ed., pp. 224–244). Pacific Grove, CA7 Brooks/ Cole. Gottlieb, B. H. (1988). Marshalling social support: The state of the art in research and practice. In B. H. Gottlieb (Ed.), Marshalling social support: Formats, processes, and effects (pp. 11–51). Newbury Park, CA7 Sage. Gross, D., & Grady, J. (2002). Group-based parent training for preventing mental health disorders in children. Issues in Mental Health Nursing, 23, 367 – 383. Hatton, D. C., & Kaiser, L. (2004). Methodological and ethical issues emerging from pilot testing an intervention with women in a transitional shelter. Western Journal of Nursing Research, 26, 129 – 136. Hays, W. L. (1988). Statistics (4th ed., pp. 325–375). New York7 Holt, Rinehart, & Winston. Hoyle, R. H., Georgesen, J. C., & Webster, J. M. (2001). Analyzing data from individuals in groups: The past, the present, and the future. Group Dynamics: Theory, Research, and Practice, 5, 41 – 47. Kenny, D. A., & Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422 – 431.
GROUP RESEARCH
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83, 126 – 137. Kreidler, M. (2005). Group therapy for survivors of childhood sexual abuse who have chronic mental illness. Archives of Psychiatric Nursing, 19, 176 – 183. Moritz, S. E., & Watson, C. B. (1998). Levels of analysis issues in group psychology: Using efficacy as an example of a multilevel model. Group Dynamics: Theory, Research, and Practice, 6(4), 267 – 276. Munro, B. H. (2001). Statistical methods for health care research (4th ed.). Philadelphia7 Lippincott Williams & Wilkins. National Osteoporosis Foundation (1998). Physician’s guide to prevention and treatment of osteoporosis. Washington, DC7 Author. Perraud, S., Farran, C. J., Loukissa, D., & Paun, O. (2004). Alzheimer’s disease caregiving information and skills: Part III. Group process issues and concerns. Research in Nursing & Health, 27, 110 – 120. Polit, D. F., & Beck, C. T. (2004). Nursing research: Principles and methods (7th ed.). Philadelphia7 Lippincott Williams & Wilkins. Pollack, B. N. (1998). Hierarchical linear modeling and the bunit of analysisQ problem: A solution for analyzing responses of intact group members. Group Dynamics: Theory, Research, and Practice, 2, 299 – 312.
281
Sidani, S., & Braden, C. J. (1998). Evaluating nursing interventions: A theory-driven approach. Thousand Oaks, CA7 Sage. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed., pp. 111–170). Boston7 Allyn & Bacon. Treece, E., & Treece, J. (1986). Elements of nursing research. St. Louis, MO7 Mosby. Webster-Stratton, C., & Hammond, M. (1998). Conduct problems and level of social competence in Head Start children: Prevalence, pervasiveness, and associated risk factors. Clinical Child and Family Psychology Review, 1, 101 – 124. Wilson, H. S. (1985). Research in nursing. Menlo Park, CA7 Addison-Wesley Publishing Co. Wood, G., & Haber, J. (2002). Nursing research, methods, critical appraisal & utilization (5th ed.). St. Louis, MO7 Mosby. Woods, N. F., & Catanzaro, M. (1988). Nursing research: Theory and practice. St. Louis, MO7 Mosby. Yalom, I. (1995). The theory and practice of group psychotherapy (4th ed., pp. 1– 99). New York7 Basic Books. Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156 – 167.