- Email: [email protected]
The Effectiveness of the Olweus Bullying Prevention Program in Public Middle Schools: A Controlled Trial
Journal of Adolescent Health 40 (2007) 266 –274
Original article
The Effectiveness of the Olweus Bullying Prevention Program in Public Middle Schools: A Controlled Trial Nerissa S. Bauer, M.D., M.P.H.a,*, Paula Lozano, M.D., M.P.H.a, and Frederick P. Rivara, M.D., M.P.H.a,b b
a Department of Pediatrics, Child Health Institute, University of Washington, Seattle, Washington Department of Epidemiology, Child Health Institute, University of Washington, Seattle, Washington Manuscript received June 22, 2006; manuscript accepted October 2, 2006
Abstract
Purpose: To examine the effectiveness of a widely disseminated bullying prevention program. Methods: A nonrandomized controlled trial with 10 public middle schools (7 intervention and 3 control) was conducted. Student-reported relational (e.g., spreading rumors, social exclusion) and physical victimization, and whether the program improved student attitudes and perceptions toward bullying were assessed pre- and post-implementation using available school survey data. Results: Regression analyses controlling for baseline prevalence and school characteristics showed no overall effect on student victimization. However, when stratified by ethnicity/race, reports of relational and physical victimization decreased by 28% (RR ⫽ .72, 95% CI: .53–.98) and 37% (RR ⫽ .63, 95% CI: .42–.97), respectively, among white students relative to those in comparison schools. No similar effect was found for students of other races/ethnicities; there were no differences by gender or by grade. Students in intervention schools were more likely to perceive other students as actively intervening in bullying incidents, and 6th graders were more likely to feel sorry and want to help victims. Conclusions: The program had some mixed positive effects varying by gender, ethnicity/race, and grade but no overall effect. Schools implementing the program, especially with a heterogeneous student body, should monitor outcomes and pay particular attention to the impact of culture, race and family influences on student behavior. Future studies of large-scale bullying prevention programs in the community must be rigorously evaluated to ensure they are effective. © 2007 Society for Adolescent Medicine. All rights reserved.
Keywords:
Bullying; School-based intervention; Prevention; Victimization; Middle schools
Bullying is aggressive behavior marked by an imbalance of power occurring repetitively with intent to harm [1,2] and can be either physical (e.g., fighting, pushing) or relational (e.g., social exclusion, spreading rumors). Bullying is a social phenomenon, with each child’s role— bully, victim, bully-victim, or bystander [1,3]— dependent on the situation. Children who either join in bullying or observe without trying to stop it reinforce the bully’s behavior [4]. Children who refrain from intervening are termed bystanders. Bul*Address correspondence to: Nerissa S. Bauer, M.D., M.P.H., Indiana University, Department of General & Community Pediatrics, 1001 West 10th Street, Wishard-Bryce Building, B2007, Indianapolis, IN 46202. E-mail address: [email protected]
lies, victims, and bully-victims are at risk for negative mental health and social outcomes that may persist into adulthood [5–7]. Without appropriate intervention, bullying behaviors tend to increase and contribute to a negative school environment [8 –10]. The Olweus Bullying Prevention Program (OBPP) was designed to improve peer relations and promote a safe and positive school environment by fostering school-wide awareness of bullying [8]. Program core components target school-, classroom-, individual-, and community-level interventions and include regular discussions about antibullying rules and other activities designed to engage students, with the long-term goal of changing student attitudes and perceptions surrounding bullying. The original study of
1054-139X/07/$ – see front matter © 2007 Society for Adolescent Medicine. All rights reserved. doi:10.1016/j.jadohealth.2006.10.005
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
267
Figure 1. Study design and implementation timeline of the Olweus Bullying Prevention Program. *One of seven intervention schools implemented program in 2004 –2005 academic year.
OBPP involving 2500 students in 42 schools in Norway reported a 50% reduction in student bullying behavior 2 years after implementation [8]. Since then, other studies, including one in U.S. rural schools [11], using interventions modeled after or replicating the OBPP, have found mixed results [12–14]. Nonetheless, the OBPP has been disseminated widely in the United States and abroad. In 2003–2004, as part of a system-wide focus on highrisk youth [15], leaders from 7 Seattle middle schools (grades 6 – 8) implemented the OBPP to address bullying and victimization. Three remaining middle schools chose less formal activities for bullying prevention in the same district. Concurrently, in the fall of 2002, a state-level mandate required all schools to implement anti-bullying schoollevel policies by August 2003. In 2004, the school district requested a program evaluation to assess its effectiveness and identify areas for improvement for future adaptation of the program in other schools. This afforded us the unique opportunity to evaluate the OBPP in a nonrandomized trial 1–2 years after implementation. The objective of our study was two-fold: (1) to characterize the implementation of the OBPP in the seven schools, and (2) to compare schools with (N ⫽ 7) and without (N ⫽ 3) the OBPP to determine if the program was effective with regard to: (a) reducing studentreported victimization (primary outcome), (b) improving student attitudes toward bullying and perceptions of others’ readiness to intervene (key program targets), and (c) improving the general school experience beyond bullying. Methods Sample We conducted a controlled trial with a cohort of 10 middle schools (grades 6 – 8) following a statewide mandate
requiring all middle schools to implement anti-bullying policies and measures. We chose to limit our sample to middle schools because the Olweus Bullying Prevention Program was originally targeted for this age range. School leaders decided how they would satisfy the mandate, thus randomization was not possible. Seven schools elected to implement the Olweus Bullying Prevention Program and the remainder pursued less formal activities (Figure 1). Procedures We assessed the type and extent of bullying prevention efforts in OBPP intervention and comparison schools. With the help of district trainers overseeing the OBPP, we invited key informants from each middle school to participate in in-depth interviews. In 2002, the district proactively integrated specific questions from the Olweus survey verbatim into existing school climate surveys to learn more about the prevalence of bullying. As this survey was administered to all schools, regardless of participation in the OBPP, it provided us the ability to determine the effectiveness of the OBPP. We used all available annual school climate survey data from the Olweus indicators and select others felt to measure key program target outcomes at two different time points, pre- and post-implementation. Study procedures were approved by the University of Washington Human Subjects Protection Committee. Measures Primary outcome (victimization experiences) Students were asked to respond to four questions regarding relational and physical bullying victimization. Questions were from the Revised Olweus Bully/Victim Questionnaire [8,16] as shown in Figure 2. The section was prefaced by the following general question, “Have you been
268
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
Figure 2. Survey questions.
bullied at school in the past couple of months in one or more of the following ways?” Students responded on the frequency of each of these events using a five-point scale. Responses were dichotomized using the cut-off of “2–3 times a month” or greater to capture the repetitive nature specific to the definition of bullying, consistent with prior literature [8,16 –18]. The two relational and two physical indicators were collapsed into separate composite measures. If a student responded positively to at least one of the pair of indicators, the student was coded as experiencing relational or physical victimization, respectively. Secondary outcomes Selected questions were chosen from the student climate survey to measure student attitudes, perceptions of others’ readiness to intervene, and school experience (Figure 2). Program targets As the program objective was to reduce student bullying by increasing awareness, we wanted to measure students’ pro-victim attitudes. To examine the desired program effect of shifting more students’ attitudes towards wanting to help victims, we dichotomized this indicator at “feel sorry and want to help.” Perceptions of others’ readiness to intervene were measured by two separate questions, one for the actions of other students and the other for actions of teachers or other adults. Responses were dichotomized so that “sometimes” or greater was considered to be a positive response.
General school experiences Additional questions related to the perception of safety, support, and school engagement were selected to measure the general school experience beyond bullying. In the present study, one indicator (“I feel safe in my classroom”) was chosen to measure the perception of safety. There were seven variables that related to the students’ perception of school support. A factor analysis was conducted and revealed all seven variables loaded on one factor, “being supported.” These 7 variables explained 42% of the variance. A new composite variable was created so that a positive response to any of the seven indicators resulted in the student being coded as feeling supported. School engagement was measured by one indicator. Intervention The 10 schools were assigned to 1 of 2 groups based on whether they elected to implement the OBPP (intervention group) or pursued less formal prevention efforts (comparison group). Each intervention school underwent consultation by district trainers prior to implementation. Covariates Variables correlated within a particular school were used in the statistical analyses to account for interschool variations. In this study, school size, percent of students eligible for free/reduced lunch, and percent of students meeting state standards for a reading achievement test were obtained from
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
269
Table 1 Definitions of “Olweus Standards” for core component implementation* Component School-wide 1. OBPP survey 2. School assembly 3. Student supervision 4. Staff discussions 5. Coordinating committee 6. School rules 7. Engaging parents 8. Engaging students 9. Tracking & identification of “hot spots” 10. Staff training Classroom 1. Class discussions 2. Reinforcement of school rules Community 1. Raise community awareness
Olweus Standard definition 1. Administer survey & results shared with school 2. Official implementation start date. Show commitment to bullying prevention, introduce concepts & school rules, raise enthusiasm 3. Schools should revise problem spots as identified, intermittently revisit protocol to ensure adequacy 4. Hold regular meetings to discuss problematic issues, with goal of fostering collaboration in implementation efforts 5. Identify core group of people responsible for initial planning & oversight of implementation 6. Set common language & expectation for student behavior 7. If engage parents, expectations & follow-through of consequences of student behavior at school can occur 8. In effort to change student attitudes and perceptions, involve students in activities to raise awareness. Target students who are otherwise known as bystanders 9. Not by one-on-one experience with individual students per se, but by examining patterns of problem areas with intent to shift supervision as needed 10. After coordinating committee trained, committee members train remaining adult staff 1. Regular & consistent discussions of schoolwide rules; teach skills to deal with bullying, foster empathy for others 2. Teachers should feel comfortable in intervening in bullying incidents, either by actively stopping it themselves or at least identifying and reporting to administration/counselors 1. Public relations and foster OBPP-inspired program development based in the community
* Schools rated on 4-point scale: 0-no implementation, 1-attempted but not to Olweus standard, 2-meets Olweus standard, 3-exceeds Olweus standard.
each school’s annual reports and entered into the database (Table 1). These covariates were predictive of victimization in a previous study [19]. Clustering by schools was not possible because the results became unstable due to the small number of clusters. Data analysis Key informant interviews and analysis Key informant interviews were conducted by the primary investigator (NB) at each of the schools. Informed consent was obtained prior to the interview. Information concerning the timeline of implementation of any prevention efforts (OBPP for intervention or other methods for comparison schools) was ascertained. As many of the OBPP core components are not unique to the program, both intervention and comparison schools were asked further questions about each core component. We were not able to collect information regarding individual-level components, as records of student incidents and meetings with parents were not available at the time of the interview and not consistently documented. Each interview lasted approximately 60 minutes. School efforts were evaluated with respect to an “Olweus Standard,” to adequately capture teachers’ use of core components with the program’s intended regularity and consistency to remain involved with students (Table 1). The Olweus standard was developed from a review of the literature [8,11]. A blinded coder (an outside expert in the field of developmental psychology and particular expertise in bul-
lying prevention) and a study investigator (NB) independently coded the 10 schools’ activities across the 3 levels, as well as, implementation fidelity of individual core components based on a 4-point scale (0 ⫽ no effort/activity, 1 ⫽ attempted but not to Olweus standards, 2 ⫽ meets Olweus standard, or 3 ⫽ exceeds Olweus standard). Inter-rater reliability was 87% (113 of the 130 coded observations were in 100% agreement). Sixteen of the 17 discordant observations were coded within a 1-point difference. Conflicts in coding were resolved by consensus. Mean school scores were aggregated by group (intervention or comparison) and calculated. To test for statistically significant differences between implementation fidelity, t-tests with equal variances were performed to examine differences between implementation efforts by school-, classroom-, and communitylevel scores. Quantitative analysis Because surveys lacked identifying information, individual students could not be linked to survey responses. Student data were aggregated by school, and school-level data were used in the analyses. Poisson regression was performed, controlling for the school-level covariates. Baseline frequencies of victimization were included in the models to adjust for differences at baseline. Separate models were examined for each of the secondary outcomes using the same procedures. As bullying trends for both gender and age have been documented
270
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
previously [8,12,17,20], we stratified each outcome by gender and grade level. Given the ethnicity/race diversity in these schools, we elected to examine the effect of ethnicity/ race on our outcomes of interest. Results Qualitative data On average, intervention schools implemented more core components at the school-wide and classroom levels to the Olweus Standard than comparison schools (school-wide score: mean, 9.1 intervention vs. 2.7 comparison, p ⬍ .05 [10 points max]; classroom score: 1.3 intervention vs. 0 comparison, p ⬍ .05 [2 points max]). Only one of the intervention schools implemented a community-level component by the Olweus Standard by offering bullying prevention training to businesses surrounding the school to raise awareness of bullying that may occur to and from the school. Comparison schools tried implementing some activities, but without regularity or within the context of a whole school approach. The greatest sources of variation within the intervention schools were activities to engage parents, methods to keep abreast of bullying incidents, holding regular staff discussions, and classroom meetings. Most of the OBPP schools, at a minimum, notified parents about the start of the OBPP, but only a few held additional events to actively educate parents. All intervention schools used the OBPP survey data to identify “hot spots” (problematic areas prone to bullying), but some actively monitored these hot spots by having students complete bullying reporting
forms and marking where the incident took place on a school map. Reporting forms were dropped in designated locked boxes or specific lockers and reviewed by counselors and/or administration. Schools differed in their tracking of incidents, with the majority of schools having an informal tracking system in place. One school started an electronic system by year two. Holding regular staff discussions and class meetings was difficult for most intervention schools. Some schools designated specific class periods to discuss school rules against bullying, for role playing, and other curricular activities. Among intervention schools, two consistently maintained high cooperation and enthusiasm for the program, whereas the others faced obstacles related to funding, effort, and administrative support. One of the comparison schools elected to start the OBPP in fall of 2004. This school underwent consultation with district trainers and formed a school coordinating committee but did not get started during the time of the present study and therefore remained classified as a comparison school. The other two comparison schools were involved with less formal measures such as peer mediation and curriculum dealing with racist attitudes. Quantitative analysis–student characteristics at baseline Comparison schools had a higher proportion of African American students (28% vs. 12%), while the intervention schools had more white students (40% vs. 23%). Other demographic characteristics were similar between the two groups (Table 2).
Table 2 Baseline student characteristics and school-level covariates Characteristic
Intervention students N ⫽ 4959 N (%)
Comparison students N ⫽ 1559 N (%)
Female gender Ethnicity/Race* Black/African American Hispanic/Latino Asian White Native American Other Grade level 6th 7th 8th Achievement Mostly As Mostly Bs/Cs Mostly D/F School-level covariates School size % Free lunch % Meeting state standards reading WASL (55.0)
2522 (51)
782 (50)
610 (12) 362 (7) 1148 (24) 1941 (40) 95 (2) 434 (9)
422 (28)* 102 (7) 384 (25) 348 (23)* 22 (1) 159 (10)
1672 (34) 1629 (33) 1588 (32)
570 (37) 515 (33) 449 (29)
2241 (47) 2262 (47) 274 (6) Intervention group N ⫽ 7 877 (617–1247) 45 (16–69) 53.8 (38.8–81)
661 (44) 724 (48) 116 (8) Comparison group N ⫽ 3 714 (467–994) 56 (36–71) 44 (31.9–66.7)
* Test of proportions, p-value ⬍ .001.
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
271
Table 3 Intervention and comparison schools’ student-reported prevalence of student victimization, attitudes and perceptions, baseline and follow-up Student-reported
2003: Baseline
Relational victimization* Physical victimization* Student attitude Probably what student deserved Don’t feel much Feel sorry for student Feel sorry & want to help Students help other students# Teachers help other students#
2005: Follow-up
Comparison N/total N (%)
Intervention N/total N (%)
Comparison N/total N (%)
Intervention N/total N (%)
428/1408 (30.4) 224/1373 (16.3) N ⫽ 1559 131 (8.4) 214 (13.7) 511 (32.8) 544 (34.9) 44/731 (6) 670/1100 (60.9)
1144/4607 (24.8) 627/4531 (13.8) N ⫽ 4959 513 (10.3) 705 (14.2) 1838 (37.1) 1594 (32.1) 140/2518 (5.6) 2324/3664 (63.4)
439/1456 (30.2) 254/1448 (17.5) N ⫽ 1580 162 (10.3) 217 (13.7) 513 (32.5) 575 (36.4) 219/1075 (20.4) 764/1167 (65.5)
1105/4480 (24.7) 643/4419 (14.6) N ⫽ 4843 516 (10.7) 655 (13.5) 1556 (32.1) 1773 (36.6) 769/3193 (24.1) 2545/3730 (68.2)
* At least 2–3 times per month. # Sometimes or more often.
Victimization Middle school students of both groups at baseline reported being victims of relational bullying more frequently than physical bullying (Table 3). For both types of victimization, baseline frequencies in the intervention group were lower than in the comparison group. Males consistently reported more relational victimization at baseline than females (29.5% vs. 20.4% intervention, 34.6% vs. 26.2% comparison, p ⱕ .001). This was also true for physical victimization at baseline (18.4% vs. 9.5% intervention, 19.8% and 12.9% comparison, p ⱕ .001). Attitudes and perceptions Fewer students in intervention schools held pro-victim attitudes and wanted to help others at baseline when compared to students in comparison schools (34% vs. 39%, p ⱕ .001). With regard to perceiving others’ readiness to inter-
vene, almost two thirds of the students or greater felt teachers were already actively intervening at baseline among all schools. Only 6% of students in both intervention and comparison groups felt other students were actively intervening on behalf of student victims at baseline. Multivariable analyses Primary outcome: victimization Overall, there was no difference in relational (RR ⫽ .96, 95% CI: .86 –1.08) or physical (RR ⫽ 1.00, 95% CI: .87– 1.17) victimization reports for the intervention schools versus comparison schools over the two-year period (Table 4). When stratified by ethnicity/race, white students in intervention schools were 27.5% less likely to report relational (RR ⫽ .72, 95% CI: .53–.98) and 36.6% less likely to report physical victimization (RR ⫽ .63, 95% CI: .42–.97) compared to white students in comparison schools. There were
Table 4 Multivariable analyses of primary outcomes and program targets Student-reported
Relational victimization RR (95% CI)
Physical victimization RR (95% CI)
Attitude RR (95% CI)
Students help other students RR (95% CI)
Teachers help other students RR (95% CI)
Overall Gender Female Male Ethnicity/Race Black White Asian Other Grade 6th 7th 8th
0.96 (0.86–1.08)
1.01 (0.87–1.17)
1.04 (0.94–1.14)
1.21 (1.05–1.40)
0.98 (0.93–1.03)
1.01 (0.84–1.21) 0.94 (0.80–1.09)
0.91 (0.71–1.16) 1.05 (0.87–1.26)
1.07 (0.95–1.20) 0.97 (0.82–1.15)
1.18 (0.97–1.43) 1.30 (1.04–1.62)
1.02 (0.95–1.09) 0.94 (0.87–1.02)
0.89 (0.66–1.19) 0.72 (0.53–0.98) 1.00 (0.77–1.31) 0.90 (0.70–1.16)
1.24 (0.88–1.75) 0.63 (0.42–0.97) 1.07 (0.81–1.43) 0.84 (0.61–1.16)
1.13 (0.89–1.44) 1.20 (0.98–1.48) 0.91 (0.75–1.10) 1.00 (0.80–1.24)
1.01 (0.75–1.36) 1.33 (0.83–2.13) 1.50 (1.13–1.99) 1.04 (0.76–1.41)
0.96 (0.85–1.08) 1.14 (0.93–1.40) 0.95 (0.87–1.03) 0.97 (0.86–1.10)
1.10 (0.90–1.34) 0.87 (0.71–1.07) 0.89 (0.71–1.11)
0.88 (0.68–1.14) 1.11 (0.85–1.44) 0.98 (0.76–1.28)
1.21 (1.05–1.40) 0.82 (0.68–0.99) 0.95 (0.79–1.15)
1.30 (1.05–1.60) 1.06 (0.81–1.40) 1.16 (0.86–1.57)
1.02 (0.94–1.10) 0.95 (0.87–1.04) 0.95 (0.86–1.06)
ⴱAdjusted for school size; % students eligible free lunch, % students passing achievement test and baseline frequencies for each outcome. Bold: p ⱕ .05.
272
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
no effects for students of other ethnicities/races. No effects were seen when examined by gender or grade. Secondary outcomes (program targets): attitudes and perceptions There was no difference between the intervention and comparison schools in student attitudes to intervene overall (RR ⫽ 1.04, 95% CI: .94 –1.14). When stratified by grade, 6th grade students in the intervention schools were more likely to feel sorry for students being bullied and want to help. No effect was found based on gender, ethnicity/race, or in the other grades. Students in intervention schools were more likely to perceive other students as actively intervening in bullying incidents (RR ⫽ 1.21, 95% CI: 1.05–1.40) compared to those in comparison schools. Similar analyses for student perception of teachers’ or other adults’ readiness to intervene did not show intervention effects. Secondary outcomes: general school experience Perception of support, perception of safety, and school engagement were not affected by the intervention (data not shown). Discussion Bullying is a common experience as reported by middle school students in our sample. Almost one third of our sample reported being a victim of frequent relational bullying in the past couple of months, and a smaller proportion reported being physically bullied. In this controlled trial, there was no overall effect of the Olweus Bullying Prevention Program on student-reported victimization. However, when stratified by ethnicity/race, white students were less likely to report relational and physical victimization over the two-year period. Among intervention schools, we found students were 21% more likely to perceive other students actively intervening on behalf of student victims. With regard to student attitude, 6th graders were 21% more likely to feel sorry for victims and want to help. The prevalence of bullying victimization in our sample, and the higher prevalence in boys, fall within the range reported in the literature [3,14,17,21,22], although there are differences among these studies in the time period and whether overall victimization was reported or separated by type experienced. The lack of overall effect of OBPP in these Seattle middle schools suggests that the OBPP may not be as effective as hoped. Evaluations of OBPP (or OBPP-inspired programs) in Europe, Australia, and Canada have found mixed results [11–13,23,24] but are difficult to compare due to differences in sampling, design, and analyses. The Norwegian evaluation found a 50% reduction in both types of bullying over 2.5 years [5], but had no comparison schools and used pre-post comparisons between age-equivalent groups, a design that is subject to historical threats to internal validity. Our findings of the apparent
effect of OBPP on student attitudes add to scant literature on these intermediate outcomes, and contrast with a report finding no effect of a bullying prevention program on 10- to 12-year-olds’ attitudes [24]. Because questions were generally phrased and not linked to individual students, we were not able to assess contextual variables associated with student attitudes toward bullying. In general, students possess pro-victim attitudes. Yet, we cannot conclude the OBPP was responsible for the difference in perception because these feelings may have been influenced by gender [25], ethnicity/race [23,24,26,27], and age of students. Several possibilities exist for our findings. The OBPP was developed in Norway for a relatively homogenous population; the program may not translate easily into a multi-ethnic society. Secondly, the major developmental task of adolescence is the formation of one’s identity. In schools with a diverse student body, ethnic identity and attitudes toward others could be influenced by experiences with peer groups. One study examining middle school students’ perception of discrimination and attitudes toward others found European-Americans held more positive attitudes toward other ethnic groups and reported less discrimination when compared to the attitudes of African Americans, Vietnamese Americans, and Mexican Americans [28]. Therefore, it is quite possible that white students in our sample held similar beliefs and thus, schools with a higher proportion of white students did not require as intensive or tailored approaches to bullying prevention. It is also possible that direct consultation to the schools by Olweus was responsible for the treatment effect seen in the original evaluation, although another study using dedicated consultants did not show an intervention effect [24]. In addition, a variety of factors affect students’ perception and reporting of victimization, including cultural biases, making the transfer of a program developed for one culture to another difficult. Our study found post-implementation studentreported victimization rates remained stable or increased. A recent study found student perception of experienced victimization influenced responses to the Olweus Bully/Victim Questionnaire items [29]. Therefore, post-implementation prevalence could be made up of previously unlabeled victims and may be a direct consequence of the district’s effort to raise awareness through bullying prevention policies as increases occurred for both intervention and comparison schools. Lastly, we were not able to control for additional home and family factors previously linked to student aggression, such as exposure to intimate partner violence in the home [30], history of sexual or physical abuse [31], or harsh parenting [32]. As we found a substantial decrease in victimization only among white students in our sample, it raises the question of the overlap between racism and bullying. There are few studies that examine the issue of racism and children specifically [28,33]. Racist remarks do not need to be experienced repeatedly to result in harm [33]; whereas, the current
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
accepted definition of bullying emphasizes the repetitive nature of an aggressive act to be considered bullying. Moreover, the Olweus Bully/Victim Questionnaire has only one indicator to capture racism (“I was called nasty names about my color or race”). Eslea et al found Asian students experience a high prevalence of racism regarding student skin color, language and dress, and religious practice [33]. Because our sample had a higher proportion of heterogeneity than previous studies, racism could explain some of the observed variability of effect. A cross-sectional survey conducted by Romero and Roberts is the only study we are aware of that examined the association between ethnic identity, perceived discrimination, and attitudes toward other racial groups among an ethnically diverse sample consisting of 3071 middle school students (grades 6 – 8) [28]. In this study, the authors explored two distinct but related factors within adolescent ethnic identity— ethnic exploration and ethnic affiliation. Students who engaged in more ethnic exploration or possessed more negative attitudes toward others were more likely to perceive more discrimination [28]. Conversely, students who reported a higher sense of belonging to one’s ethnic group (ethnic affirmation—the social identity aspect of ethnic identity) were more likely to hold more positive attitudes toward other groups [28]. More research is needed to further the work done by Romero and Roberts by examining the interplay between ethnic identity, bullying, and racism among this segment of children. In light of the various roles of the family and ethnicity/ race on student behavior and attitudes, we encourage schools not to stop implementing the Olweus Bullying Prevention Program. One reason is that this program is the only available bullying prevention program that is comprehensive and encompasses a whole school approach. Additionally, it is a vehicle for schools to bring about change because it establishes a common language and provides the necessary framework for schools to tackle bullying. The fact that we found positive effects among white students is encouraging but, given the likelihood that increased heterogeneity exists in schools across North America as compared to Norway, schools should be aware that the role of ethnicity/ race and culture must be considered for reasons already stated. Moreover, students come from varied family/home environments and schools should work to engage parents to effectively bring about change in student behavior. Implementation efforts of intervention schools were broad and encompassed a significant regularity and consistency when compared to efforts of comparison schools. This “whole school” approach is the distinguishing feature of the OBPP from less formal bullying prevention activities. Teacher/staff communication and school attention to bullying problems have been predictive of implementation on the school level [34]. These elements are essential to continued education and commitment of the staff, parents, and administration and should remain a priority for schools implementing or sustaining the program despite our
273
mixed findings on the program’s effectiveness. Given the link between students’ personal experiences with victimization and school avoidance [35], schools should continue planning specific, structured curriculum and interventions in the classroom. Peer groups are usually aware of the bullying that occurs [36] and are desirable targets for intervention. Teaching victim assertiveness and bolstering friendship quality among both bullies and victims [37] can serve as two important strategies. We acknowledge certain limitations of our study. This study was a natural experiment and there were measurement issues that we had to account for (establishing the “Olweus standard” and qualitative interviews). Key informant interviews are subject to recall bias, as well as sampling error. One investigator (NB) conducted all the interviews and confirmed obtained data through follow-up contact with key informants to ensure data accuracy and help filling in missing timeline information. Reliance on student reports may have led to under-reporting of victimization, although our rates are similar or higher than previously published studies [17,22,38]. In addition, from year to year, we had varying rates of missing data, due to students not responding to all questions. The school climate survey used in the present study did not have an operational definition and may have caused students to report on incidents involving another student of equal strength, which does not meet the criteria for bullying behavior. Self-reported bullying behavior and student knowledge about what constitutes bullying was not measured, nor did we have observational or cross-informant reports of bullying victimization. Lastly, our results may not be generalizable to other schools wishing to implement the program, given the inherent complexity required for program implementation and historical factors in our study setting. In summary, the Olweus Bullying Prevention Program had some positive effects varying by gender, ethnicity/race, and grade, but no overall effect in these Seattle middle schools. Implementation of the OBPP program may lead to variable differences in effectiveness based on factors related to culture, race, and the influence of the family/home environment. Therefore, schools should be aware of the influence that home, culture, and society have on student behavior, and tailor preventive measures accordingly. A major implication of our study is for schools with an ethnically-diverse student body not to stop implementing the program, but rather encourage the process of ethnic exploration to nurture adolescents’ emerging ethnic identity, while advocating the development of tolerance and sensitivity to other ethnic groups. Identification of resources, administrative support, and commitment to implementation fidelity is an ongoing process. Future studies of large-scale violence prevention programs in the community need to be rigorously evaluated to ensure they are working as intended.
274
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
Acknowledgments This study was supported by a grant from Health Resources and Service Administration (HRSA) and a National Research Service Award (NRSA) Primary Care Research Fellowship. The authors thank the following people for their contributions to this project: James F. Lymp, Ph.D., with the Child Health Institute for consultation on statistical analyses; Susan Haws, M.P.H., and Michael Arthur, Ph.D., from the Social Development Research Group (SDRG), for their input in the development phase of the project; the district trainers from the Seattle school district and key informants from the participating middle schools for their time, effort, and their continued support; and Susan P. Limber, Ph.D., from the Institute on Family & Neighborhood Life at Clemson University, for her consultation and for coding the qualitative data. References [1] Olweus D. Bullying at School: Blackwell Publishing, 1993. [2] Smith PK, Morita, Y, Junger-Tas J, et al (eds). The Nature of School Bullying: A Cross-National Perspective. New York: Routledge, 1999. [3] Haynie DL, Nansel T, Eitel P, et al. Bullies, victims, and bully/victims: Distinct groups of at-risk youth. J Early Adolesc 2001;21:29 – 49. [4] Schwartz D, Dodge KA, Coie JD. The emergence of chronic peer victimization in boys’ play groups. Child Dev 1993;64:1755–72. [5] Olweus D. Bully/victim problems in school: Facts and intervention. Eur J Psychol Educ 1997;12:495–510. [6] Juvonen J, Graham S, Schuster MA. Bullying among young adolescents: The strong, the weak, and the troubled. Pediatrics 2003;112(6 Pt 1):1231–7. [7] Kim YS, Koh YJ, Leventhal B. School bullying and suicidal risk in Korean middle school students. Pediatrics 2005;115:357– 63. [8] Olweus D. Bullying at school: Basic facts and effects of a school based intervention program. J Child Psychol Psychiatry 1994;35: 1171–90. [9] Patterson GR. Families: Applications of Social Learning to Family Life. Revised. Champaign, IL: Research Press, 1975. [10] Webster-Stratton C, Mihalic S, Fagan A, et al. Blueprints for Violence Prevention, Book Eleven: The Incredible Years—Parent, Teacher, and Child Training Series. Boulder, CO: Center for the Study and Prevention of Violence, 2001. [11] Limber SP, Nation M, Tracy AJ, et al. Implementation of the Olweus Bullying Prevention Programme in the Southeastern United States. In: Smith PK, Pepler D, Rigby K (eds). Bullying in Schools: How Successful can Interventions Be? Cambridge, UK: Cambridge University Press, 2004:334. [12] Eslea M, Smith, PK. The long-term effectiveness of anti-bullying work in primary schools. Educ Res 1998;40:203–18. [13] Salmivalli C, Kaukiainen A, Voeten M. Anti-bullying intervention: Implementation and outcome. Br J Educ Psychol 2005;75(Pt 3):465– 87. [14] Limber SP, Nation M, Tracy AJ, et al. Implementation of the Olweus Bullying Prevention programme in the Southeastern United States. In: Smith PK, Pepler D, Rigby K (Eds.) Bullying in Schools: How Successful Can Interventions Be? Cambridge: Cambridge University Press, 2004, 55–79. [15] Hawkins JD, Catalano, RF, Arthur MW. Promoting science-based prevention in communities. Addict Behav 2002;27:951–76. [16] Solberg M, Olweus D. Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire. Aggressive Behav 2003;29: 239 – 68.
[17] Nansel TR, Overpeck M, Pilla RS, et al. Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. JAMA 2001;285:2094 –100. [18] Glew GM, Fan MY, Katon W, et al. Bullying, psychosocial adjustment, and academic performance in elementary school. Arch Pediatr Adolesc Med 2005;159:1026 –31. [19] Wilson D. The interface of school climate and school connectedness and relationships with aggression and victimization. J School Health 2004;74:293–9. [20] Eslea M, Rees J. At what age are children most likely to be bullied at school? Aggressive Behav 2001;27:419 – 429. [21] Nansel TR, Craig W, Overpeck MD, et al. Cross-national consistency in the relationship between bullying behaviors and psychosocial adjustment. Arch Pediatr Adolesc Med 2004;158:730 – 6. [22] DeVoe JF. Student Reports of Bullying: Results from the 2001 School Crime Supplement to the National Crime Victimization Survey, Statistical Analysis Report (NCES 2005-310). In: National Department for Education Statistics (ed). U.S. Department of Education IoES.Washington, DC: U.S. Government Printing Office, 2005. [23] Rigby K, Slee PT. Bullying among Australian school children: Reported behavior and attitudes toward victims. J Soc Psychol 1991; 131:615–27. [24] Stevens V, De Bourdeaudhuij I, Van Oost P. Bullying in Flemish schools: An evaluation of anti-bullying intervention in primary and secondary schools. Br J Educ Psychol 2000;70(Pt 2):195–210. [25] Baldry AC. “What about bullying?” An experimental field study to understand students attitudes towards bullying and victimization in Italian middle schools. Br J Educ Psychol 2004;74(Pt 4):583–98. [26] Smith PK, Ananiadou K, Cowie H. Interventions to reduce school bullying. Can J Psychiatry 2003;48:591–9. [27] Stevens V, Van Oost P, De Bourdeaudhuij I. The effects of an anti-bullying intervention programme on peers’ attitudes and behaviour. J Adolesc 2000;23:21–34. [28] Romero AJ, Roberts RE. Perception of discrimination and ethnocultural variables in a diverse group of adolescents. J Adolesc 1998;21: 641–56. [29] Theriot MT, Dulmus CN, Sowers KM, et al. Factors relating to self-identification among bullying victims. Child Youth Serv Rev 2005;27:979 –94. [30] Bauer NS, Herrenkohl TI, Lozano P, et al. Childhood bullying involvement and exposure to intimate partner violence. Pediatrics 2006; 118:e235– 42. [31] Shields A, Cicchetti D. Parental maltreatment and emotion dysregulation as risk factors for bullying and victimization in middle childhood. J Clin Child Psychol 2001;30:349 – 63. [32] Carney AG, Merrell KW. Bullying in schools: Perspectives on understanding and preventing an international problem. School Psychol Int 2001;22:364 – 82. [33] Eslea M, Mukhtar K. Bullying and racism among Asian schoolchildren in Britain. Educ Res 2000;42:207–17. [34] Kallestad JH, Olweus D. Predicting teachers’ and schools’ implementation of the Olweus Bullying Prevention Program: A multilevel study. Prev Treat 2003;6:23. [35] Astor RA, Benbenishty R, Zeira A, et al. School climate, observed risky behaviors, and victimization as predictors of high school students’ fear and judgments of school violence as a problem. Health Educ Behav 2002;29:716 –36. [36] Craig WM, Pepler DJ. Peer processes in bullying and victimization: An observational study. Except Educ Canada 1995;5:81–95. [37] Bollmer JM, Milich R, Harris MJ, et al. A friend in need: The role of friendship quality as a protective factor in peer victimization and bullying. J Interpers Violence 2005;20:701–12. [38] Genta ML, Menesini E, Fonzi A, et al. Bullies and victims in schools in central and southern Italy. Eur J Psychol Educ 1996;11:97–110.
Original article
The Effectiveness of the Olweus Bullying Prevention Program in Public Middle Schools: A Controlled Trial Nerissa S. Bauer, M.D., M.P.H.a,*, Paula Lozano, M.D., M.P.H.a, and Frederick P. Rivara, M.D., M.P.H.a,b b
a Department of Pediatrics, Child Health Institute, University of Washington, Seattle, Washington Department of Epidemiology, Child Health Institute, University of Washington, Seattle, Washington Manuscript received June 22, 2006; manuscript accepted October 2, 2006
Abstract
Purpose: To examine the effectiveness of a widely disseminated bullying prevention program. Methods: A nonrandomized controlled trial with 10 public middle schools (7 intervention and 3 control) was conducted. Student-reported relational (e.g., spreading rumors, social exclusion) and physical victimization, and whether the program improved student attitudes and perceptions toward bullying were assessed pre- and post-implementation using available school survey data. Results: Regression analyses controlling for baseline prevalence and school characteristics showed no overall effect on student victimization. However, when stratified by ethnicity/race, reports of relational and physical victimization decreased by 28% (RR ⫽ .72, 95% CI: .53–.98) and 37% (RR ⫽ .63, 95% CI: .42–.97), respectively, among white students relative to those in comparison schools. No similar effect was found for students of other races/ethnicities; there were no differences by gender or by grade. Students in intervention schools were more likely to perceive other students as actively intervening in bullying incidents, and 6th graders were more likely to feel sorry and want to help victims. Conclusions: The program had some mixed positive effects varying by gender, ethnicity/race, and grade but no overall effect. Schools implementing the program, especially with a heterogeneous student body, should monitor outcomes and pay particular attention to the impact of culture, race and family influences on student behavior. Future studies of large-scale bullying prevention programs in the community must be rigorously evaluated to ensure they are effective. © 2007 Society for Adolescent Medicine. All rights reserved.
Keywords:
Bullying; School-based intervention; Prevention; Victimization; Middle schools
Bullying is aggressive behavior marked by an imbalance of power occurring repetitively with intent to harm [1,2] and can be either physical (e.g., fighting, pushing) or relational (e.g., social exclusion, spreading rumors). Bullying is a social phenomenon, with each child’s role— bully, victim, bully-victim, or bystander [1,3]— dependent on the situation. Children who either join in bullying or observe without trying to stop it reinforce the bully’s behavior [4]. Children who refrain from intervening are termed bystanders. Bul*Address correspondence to: Nerissa S. Bauer, M.D., M.P.H., Indiana University, Department of General & Community Pediatrics, 1001 West 10th Street, Wishard-Bryce Building, B2007, Indianapolis, IN 46202. E-mail address: [email protected]
lies, victims, and bully-victims are at risk for negative mental health and social outcomes that may persist into adulthood [5–7]. Without appropriate intervention, bullying behaviors tend to increase and contribute to a negative school environment [8 –10]. The Olweus Bullying Prevention Program (OBPP) was designed to improve peer relations and promote a safe and positive school environment by fostering school-wide awareness of bullying [8]. Program core components target school-, classroom-, individual-, and community-level interventions and include regular discussions about antibullying rules and other activities designed to engage students, with the long-term goal of changing student attitudes and perceptions surrounding bullying. The original study of
1054-139X/07/$ – see front matter © 2007 Society for Adolescent Medicine. All rights reserved. doi:10.1016/j.jadohealth.2006.10.005
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
267
Figure 1. Study design and implementation timeline of the Olweus Bullying Prevention Program. *One of seven intervention schools implemented program in 2004 –2005 academic year.
OBPP involving 2500 students in 42 schools in Norway reported a 50% reduction in student bullying behavior 2 years after implementation [8]. Since then, other studies, including one in U.S. rural schools [11], using interventions modeled after or replicating the OBPP, have found mixed results [12–14]. Nonetheless, the OBPP has been disseminated widely in the United States and abroad. In 2003–2004, as part of a system-wide focus on highrisk youth [15], leaders from 7 Seattle middle schools (grades 6 – 8) implemented the OBPP to address bullying and victimization. Three remaining middle schools chose less formal activities for bullying prevention in the same district. Concurrently, in the fall of 2002, a state-level mandate required all schools to implement anti-bullying schoollevel policies by August 2003. In 2004, the school district requested a program evaluation to assess its effectiveness and identify areas for improvement for future adaptation of the program in other schools. This afforded us the unique opportunity to evaluate the OBPP in a nonrandomized trial 1–2 years after implementation. The objective of our study was two-fold: (1) to characterize the implementation of the OBPP in the seven schools, and (2) to compare schools with (N ⫽ 7) and without (N ⫽ 3) the OBPP to determine if the program was effective with regard to: (a) reducing studentreported victimization (primary outcome), (b) improving student attitudes toward bullying and perceptions of others’ readiness to intervene (key program targets), and (c) improving the general school experience beyond bullying. Methods Sample We conducted a controlled trial with a cohort of 10 middle schools (grades 6 – 8) following a statewide mandate
requiring all middle schools to implement anti-bullying policies and measures. We chose to limit our sample to middle schools because the Olweus Bullying Prevention Program was originally targeted for this age range. School leaders decided how they would satisfy the mandate, thus randomization was not possible. Seven schools elected to implement the Olweus Bullying Prevention Program and the remainder pursued less formal activities (Figure 1). Procedures We assessed the type and extent of bullying prevention efforts in OBPP intervention and comparison schools. With the help of district trainers overseeing the OBPP, we invited key informants from each middle school to participate in in-depth interviews. In 2002, the district proactively integrated specific questions from the Olweus survey verbatim into existing school climate surveys to learn more about the prevalence of bullying. As this survey was administered to all schools, regardless of participation in the OBPP, it provided us the ability to determine the effectiveness of the OBPP. We used all available annual school climate survey data from the Olweus indicators and select others felt to measure key program target outcomes at two different time points, pre- and post-implementation. Study procedures were approved by the University of Washington Human Subjects Protection Committee. Measures Primary outcome (victimization experiences) Students were asked to respond to four questions regarding relational and physical bullying victimization. Questions were from the Revised Olweus Bully/Victim Questionnaire [8,16] as shown in Figure 2. The section was prefaced by the following general question, “Have you been
268
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
Figure 2. Survey questions.
bullied at school in the past couple of months in one or more of the following ways?” Students responded on the frequency of each of these events using a five-point scale. Responses were dichotomized using the cut-off of “2–3 times a month” or greater to capture the repetitive nature specific to the definition of bullying, consistent with prior literature [8,16 –18]. The two relational and two physical indicators were collapsed into separate composite measures. If a student responded positively to at least one of the pair of indicators, the student was coded as experiencing relational or physical victimization, respectively. Secondary outcomes Selected questions were chosen from the student climate survey to measure student attitudes, perceptions of others’ readiness to intervene, and school experience (Figure 2). Program targets As the program objective was to reduce student bullying by increasing awareness, we wanted to measure students’ pro-victim attitudes. To examine the desired program effect of shifting more students’ attitudes towards wanting to help victims, we dichotomized this indicator at “feel sorry and want to help.” Perceptions of others’ readiness to intervene were measured by two separate questions, one for the actions of other students and the other for actions of teachers or other adults. Responses were dichotomized so that “sometimes” or greater was considered to be a positive response.
General school experiences Additional questions related to the perception of safety, support, and school engagement were selected to measure the general school experience beyond bullying. In the present study, one indicator (“I feel safe in my classroom”) was chosen to measure the perception of safety. There were seven variables that related to the students’ perception of school support. A factor analysis was conducted and revealed all seven variables loaded on one factor, “being supported.” These 7 variables explained 42% of the variance. A new composite variable was created so that a positive response to any of the seven indicators resulted in the student being coded as feeling supported. School engagement was measured by one indicator. Intervention The 10 schools were assigned to 1 of 2 groups based on whether they elected to implement the OBPP (intervention group) or pursued less formal prevention efforts (comparison group). Each intervention school underwent consultation by district trainers prior to implementation. Covariates Variables correlated within a particular school were used in the statistical analyses to account for interschool variations. In this study, school size, percent of students eligible for free/reduced lunch, and percent of students meeting state standards for a reading achievement test were obtained from
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
269
Table 1 Definitions of “Olweus Standards” for core component implementation* Component School-wide 1. OBPP survey 2. School assembly 3. Student supervision 4. Staff discussions 5. Coordinating committee 6. School rules 7. Engaging parents 8. Engaging students 9. Tracking & identification of “hot spots” 10. Staff training Classroom 1. Class discussions 2. Reinforcement of school rules Community 1. Raise community awareness
Olweus Standard definition 1. Administer survey & results shared with school 2. Official implementation start date. Show commitment to bullying prevention, introduce concepts & school rules, raise enthusiasm 3. Schools should revise problem spots as identified, intermittently revisit protocol to ensure adequacy 4. Hold regular meetings to discuss problematic issues, with goal of fostering collaboration in implementation efforts 5. Identify core group of people responsible for initial planning & oversight of implementation 6. Set common language & expectation for student behavior 7. If engage parents, expectations & follow-through of consequences of student behavior at school can occur 8. In effort to change student attitudes and perceptions, involve students in activities to raise awareness. Target students who are otherwise known as bystanders 9. Not by one-on-one experience with individual students per se, but by examining patterns of problem areas with intent to shift supervision as needed 10. After coordinating committee trained, committee members train remaining adult staff 1. Regular & consistent discussions of schoolwide rules; teach skills to deal with bullying, foster empathy for others 2. Teachers should feel comfortable in intervening in bullying incidents, either by actively stopping it themselves or at least identifying and reporting to administration/counselors 1. Public relations and foster OBPP-inspired program development based in the community
* Schools rated on 4-point scale: 0-no implementation, 1-attempted but not to Olweus standard, 2-meets Olweus standard, 3-exceeds Olweus standard.
each school’s annual reports and entered into the database (Table 1). These covariates were predictive of victimization in a previous study [19]. Clustering by schools was not possible because the results became unstable due to the small number of clusters. Data analysis Key informant interviews and analysis Key informant interviews were conducted by the primary investigator (NB) at each of the schools. Informed consent was obtained prior to the interview. Information concerning the timeline of implementation of any prevention efforts (OBPP for intervention or other methods for comparison schools) was ascertained. As many of the OBPP core components are not unique to the program, both intervention and comparison schools were asked further questions about each core component. We were not able to collect information regarding individual-level components, as records of student incidents and meetings with parents were not available at the time of the interview and not consistently documented. Each interview lasted approximately 60 minutes. School efforts were evaluated with respect to an “Olweus Standard,” to adequately capture teachers’ use of core components with the program’s intended regularity and consistency to remain involved with students (Table 1). The Olweus standard was developed from a review of the literature [8,11]. A blinded coder (an outside expert in the field of developmental psychology and particular expertise in bul-
lying prevention) and a study investigator (NB) independently coded the 10 schools’ activities across the 3 levels, as well as, implementation fidelity of individual core components based on a 4-point scale (0 ⫽ no effort/activity, 1 ⫽ attempted but not to Olweus standards, 2 ⫽ meets Olweus standard, or 3 ⫽ exceeds Olweus standard). Inter-rater reliability was 87% (113 of the 130 coded observations were in 100% agreement). Sixteen of the 17 discordant observations were coded within a 1-point difference. Conflicts in coding were resolved by consensus. Mean school scores were aggregated by group (intervention or comparison) and calculated. To test for statistically significant differences between implementation fidelity, t-tests with equal variances were performed to examine differences between implementation efforts by school-, classroom-, and communitylevel scores. Quantitative analysis Because surveys lacked identifying information, individual students could not be linked to survey responses. Student data were aggregated by school, and school-level data were used in the analyses. Poisson regression was performed, controlling for the school-level covariates. Baseline frequencies of victimization were included in the models to adjust for differences at baseline. Separate models were examined for each of the secondary outcomes using the same procedures. As bullying trends for both gender and age have been documented
270
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
previously [8,12,17,20], we stratified each outcome by gender and grade level. Given the ethnicity/race diversity in these schools, we elected to examine the effect of ethnicity/ race on our outcomes of interest. Results Qualitative data On average, intervention schools implemented more core components at the school-wide and classroom levels to the Olweus Standard than comparison schools (school-wide score: mean, 9.1 intervention vs. 2.7 comparison, p ⬍ .05 [10 points max]; classroom score: 1.3 intervention vs. 0 comparison, p ⬍ .05 [2 points max]). Only one of the intervention schools implemented a community-level component by the Olweus Standard by offering bullying prevention training to businesses surrounding the school to raise awareness of bullying that may occur to and from the school. Comparison schools tried implementing some activities, but without regularity or within the context of a whole school approach. The greatest sources of variation within the intervention schools were activities to engage parents, methods to keep abreast of bullying incidents, holding regular staff discussions, and classroom meetings. Most of the OBPP schools, at a minimum, notified parents about the start of the OBPP, but only a few held additional events to actively educate parents. All intervention schools used the OBPP survey data to identify “hot spots” (problematic areas prone to bullying), but some actively monitored these hot spots by having students complete bullying reporting
forms and marking where the incident took place on a school map. Reporting forms were dropped in designated locked boxes or specific lockers and reviewed by counselors and/or administration. Schools differed in their tracking of incidents, with the majority of schools having an informal tracking system in place. One school started an electronic system by year two. Holding regular staff discussions and class meetings was difficult for most intervention schools. Some schools designated specific class periods to discuss school rules against bullying, for role playing, and other curricular activities. Among intervention schools, two consistently maintained high cooperation and enthusiasm for the program, whereas the others faced obstacles related to funding, effort, and administrative support. One of the comparison schools elected to start the OBPP in fall of 2004. This school underwent consultation with district trainers and formed a school coordinating committee but did not get started during the time of the present study and therefore remained classified as a comparison school. The other two comparison schools were involved with less formal measures such as peer mediation and curriculum dealing with racist attitudes. Quantitative analysis–student characteristics at baseline Comparison schools had a higher proportion of African American students (28% vs. 12%), while the intervention schools had more white students (40% vs. 23%). Other demographic characteristics were similar between the two groups (Table 2).
Table 2 Baseline student characteristics and school-level covariates Characteristic
Intervention students N ⫽ 4959 N (%)
Comparison students N ⫽ 1559 N (%)
Female gender Ethnicity/Race* Black/African American Hispanic/Latino Asian White Native American Other Grade level 6th 7th 8th Achievement Mostly As Mostly Bs/Cs Mostly D/F School-level covariates School size % Free lunch % Meeting state standards reading WASL (55.0)
2522 (51)
782 (50)
610 (12) 362 (7) 1148 (24) 1941 (40) 95 (2) 434 (9)
422 (28)* 102 (7) 384 (25) 348 (23)* 22 (1) 159 (10)
1672 (34) 1629 (33) 1588 (32)
570 (37) 515 (33) 449 (29)
2241 (47) 2262 (47) 274 (6) Intervention group N ⫽ 7 877 (617–1247) 45 (16–69) 53.8 (38.8–81)
661 (44) 724 (48) 116 (8) Comparison group N ⫽ 3 714 (467–994) 56 (36–71) 44 (31.9–66.7)
* Test of proportions, p-value ⬍ .001.
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
271
Table 3 Intervention and comparison schools’ student-reported prevalence of student victimization, attitudes and perceptions, baseline and follow-up Student-reported
2003: Baseline
Relational victimization* Physical victimization* Student attitude Probably what student deserved Don’t feel much Feel sorry for student Feel sorry & want to help Students help other students# Teachers help other students#
2005: Follow-up
Comparison N/total N (%)
Intervention N/total N (%)
Comparison N/total N (%)
Intervention N/total N (%)
428/1408 (30.4) 224/1373 (16.3) N ⫽ 1559 131 (8.4) 214 (13.7) 511 (32.8) 544 (34.9) 44/731 (6) 670/1100 (60.9)
1144/4607 (24.8) 627/4531 (13.8) N ⫽ 4959 513 (10.3) 705 (14.2) 1838 (37.1) 1594 (32.1) 140/2518 (5.6) 2324/3664 (63.4)
439/1456 (30.2) 254/1448 (17.5) N ⫽ 1580 162 (10.3) 217 (13.7) 513 (32.5) 575 (36.4) 219/1075 (20.4) 764/1167 (65.5)
1105/4480 (24.7) 643/4419 (14.6) N ⫽ 4843 516 (10.7) 655 (13.5) 1556 (32.1) 1773 (36.6) 769/3193 (24.1) 2545/3730 (68.2)
* At least 2–3 times per month. # Sometimes or more often.
Victimization Middle school students of both groups at baseline reported being victims of relational bullying more frequently than physical bullying (Table 3). For both types of victimization, baseline frequencies in the intervention group were lower than in the comparison group. Males consistently reported more relational victimization at baseline than females (29.5% vs. 20.4% intervention, 34.6% vs. 26.2% comparison, p ⱕ .001). This was also true for physical victimization at baseline (18.4% vs. 9.5% intervention, 19.8% and 12.9% comparison, p ⱕ .001). Attitudes and perceptions Fewer students in intervention schools held pro-victim attitudes and wanted to help others at baseline when compared to students in comparison schools (34% vs. 39%, p ⱕ .001). With regard to perceiving others’ readiness to inter-
vene, almost two thirds of the students or greater felt teachers were already actively intervening at baseline among all schools. Only 6% of students in both intervention and comparison groups felt other students were actively intervening on behalf of student victims at baseline. Multivariable analyses Primary outcome: victimization Overall, there was no difference in relational (RR ⫽ .96, 95% CI: .86 –1.08) or physical (RR ⫽ 1.00, 95% CI: .87– 1.17) victimization reports for the intervention schools versus comparison schools over the two-year period (Table 4). When stratified by ethnicity/race, white students in intervention schools were 27.5% less likely to report relational (RR ⫽ .72, 95% CI: .53–.98) and 36.6% less likely to report physical victimization (RR ⫽ .63, 95% CI: .42–.97) compared to white students in comparison schools. There were
Table 4 Multivariable analyses of primary outcomes and program targets Student-reported
Relational victimization RR (95% CI)
Physical victimization RR (95% CI)
Attitude RR (95% CI)
Students help other students RR (95% CI)
Teachers help other students RR (95% CI)
Overall Gender Female Male Ethnicity/Race Black White Asian Other Grade 6th 7th 8th
0.96 (0.86–1.08)
1.01 (0.87–1.17)
1.04 (0.94–1.14)
1.21 (1.05–1.40)
0.98 (0.93–1.03)
1.01 (0.84–1.21) 0.94 (0.80–1.09)
0.91 (0.71–1.16) 1.05 (0.87–1.26)
1.07 (0.95–1.20) 0.97 (0.82–1.15)
1.18 (0.97–1.43) 1.30 (1.04–1.62)
1.02 (0.95–1.09) 0.94 (0.87–1.02)
0.89 (0.66–1.19) 0.72 (0.53–0.98) 1.00 (0.77–1.31) 0.90 (0.70–1.16)
1.24 (0.88–1.75) 0.63 (0.42–0.97) 1.07 (0.81–1.43) 0.84 (0.61–1.16)
1.13 (0.89–1.44) 1.20 (0.98–1.48) 0.91 (0.75–1.10) 1.00 (0.80–1.24)
1.01 (0.75–1.36) 1.33 (0.83–2.13) 1.50 (1.13–1.99) 1.04 (0.76–1.41)
0.96 (0.85–1.08) 1.14 (0.93–1.40) 0.95 (0.87–1.03) 0.97 (0.86–1.10)
1.10 (0.90–1.34) 0.87 (0.71–1.07) 0.89 (0.71–1.11)
0.88 (0.68–1.14) 1.11 (0.85–1.44) 0.98 (0.76–1.28)
1.21 (1.05–1.40) 0.82 (0.68–0.99) 0.95 (0.79–1.15)
1.30 (1.05–1.60) 1.06 (0.81–1.40) 1.16 (0.86–1.57)
1.02 (0.94–1.10) 0.95 (0.87–1.04) 0.95 (0.86–1.06)
ⴱAdjusted for school size; % students eligible free lunch, % students passing achievement test and baseline frequencies for each outcome. Bold: p ⱕ .05.
272
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
no effects for students of other ethnicities/races. No effects were seen when examined by gender or grade. Secondary outcomes (program targets): attitudes and perceptions There was no difference between the intervention and comparison schools in student attitudes to intervene overall (RR ⫽ 1.04, 95% CI: .94 –1.14). When stratified by grade, 6th grade students in the intervention schools were more likely to feel sorry for students being bullied and want to help. No effect was found based on gender, ethnicity/race, or in the other grades. Students in intervention schools were more likely to perceive other students as actively intervening in bullying incidents (RR ⫽ 1.21, 95% CI: 1.05–1.40) compared to those in comparison schools. Similar analyses for student perception of teachers’ or other adults’ readiness to intervene did not show intervention effects. Secondary outcomes: general school experience Perception of support, perception of safety, and school engagement were not affected by the intervention (data not shown). Discussion Bullying is a common experience as reported by middle school students in our sample. Almost one third of our sample reported being a victim of frequent relational bullying in the past couple of months, and a smaller proportion reported being physically bullied. In this controlled trial, there was no overall effect of the Olweus Bullying Prevention Program on student-reported victimization. However, when stratified by ethnicity/race, white students were less likely to report relational and physical victimization over the two-year period. Among intervention schools, we found students were 21% more likely to perceive other students actively intervening on behalf of student victims. With regard to student attitude, 6th graders were 21% more likely to feel sorry for victims and want to help. The prevalence of bullying victimization in our sample, and the higher prevalence in boys, fall within the range reported in the literature [3,14,17,21,22], although there are differences among these studies in the time period and whether overall victimization was reported or separated by type experienced. The lack of overall effect of OBPP in these Seattle middle schools suggests that the OBPP may not be as effective as hoped. Evaluations of OBPP (or OBPP-inspired programs) in Europe, Australia, and Canada have found mixed results [11–13,23,24] but are difficult to compare due to differences in sampling, design, and analyses. The Norwegian evaluation found a 50% reduction in both types of bullying over 2.5 years [5], but had no comparison schools and used pre-post comparisons between age-equivalent groups, a design that is subject to historical threats to internal validity. Our findings of the apparent
effect of OBPP on student attitudes add to scant literature on these intermediate outcomes, and contrast with a report finding no effect of a bullying prevention program on 10- to 12-year-olds’ attitudes [24]. Because questions were generally phrased and not linked to individual students, we were not able to assess contextual variables associated with student attitudes toward bullying. In general, students possess pro-victim attitudes. Yet, we cannot conclude the OBPP was responsible for the difference in perception because these feelings may have been influenced by gender [25], ethnicity/race [23,24,26,27], and age of students. Several possibilities exist for our findings. The OBPP was developed in Norway for a relatively homogenous population; the program may not translate easily into a multi-ethnic society. Secondly, the major developmental task of adolescence is the formation of one’s identity. In schools with a diverse student body, ethnic identity and attitudes toward others could be influenced by experiences with peer groups. One study examining middle school students’ perception of discrimination and attitudes toward others found European-Americans held more positive attitudes toward other ethnic groups and reported less discrimination when compared to the attitudes of African Americans, Vietnamese Americans, and Mexican Americans [28]. Therefore, it is quite possible that white students in our sample held similar beliefs and thus, schools with a higher proportion of white students did not require as intensive or tailored approaches to bullying prevention. It is also possible that direct consultation to the schools by Olweus was responsible for the treatment effect seen in the original evaluation, although another study using dedicated consultants did not show an intervention effect [24]. In addition, a variety of factors affect students’ perception and reporting of victimization, including cultural biases, making the transfer of a program developed for one culture to another difficult. Our study found post-implementation studentreported victimization rates remained stable or increased. A recent study found student perception of experienced victimization influenced responses to the Olweus Bully/Victim Questionnaire items [29]. Therefore, post-implementation prevalence could be made up of previously unlabeled victims and may be a direct consequence of the district’s effort to raise awareness through bullying prevention policies as increases occurred for both intervention and comparison schools. Lastly, we were not able to control for additional home and family factors previously linked to student aggression, such as exposure to intimate partner violence in the home [30], history of sexual or physical abuse [31], or harsh parenting [32]. As we found a substantial decrease in victimization only among white students in our sample, it raises the question of the overlap between racism and bullying. There are few studies that examine the issue of racism and children specifically [28,33]. Racist remarks do not need to be experienced repeatedly to result in harm [33]; whereas, the current
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
accepted definition of bullying emphasizes the repetitive nature of an aggressive act to be considered bullying. Moreover, the Olweus Bully/Victim Questionnaire has only one indicator to capture racism (“I was called nasty names about my color or race”). Eslea et al found Asian students experience a high prevalence of racism regarding student skin color, language and dress, and religious practice [33]. Because our sample had a higher proportion of heterogeneity than previous studies, racism could explain some of the observed variability of effect. A cross-sectional survey conducted by Romero and Roberts is the only study we are aware of that examined the association between ethnic identity, perceived discrimination, and attitudes toward other racial groups among an ethnically diverse sample consisting of 3071 middle school students (grades 6 – 8) [28]. In this study, the authors explored two distinct but related factors within adolescent ethnic identity— ethnic exploration and ethnic affiliation. Students who engaged in more ethnic exploration or possessed more negative attitudes toward others were more likely to perceive more discrimination [28]. Conversely, students who reported a higher sense of belonging to one’s ethnic group (ethnic affirmation—the social identity aspect of ethnic identity) were more likely to hold more positive attitudes toward other groups [28]. More research is needed to further the work done by Romero and Roberts by examining the interplay between ethnic identity, bullying, and racism among this segment of children. In light of the various roles of the family and ethnicity/ race on student behavior and attitudes, we encourage schools not to stop implementing the Olweus Bullying Prevention Program. One reason is that this program is the only available bullying prevention program that is comprehensive and encompasses a whole school approach. Additionally, it is a vehicle for schools to bring about change because it establishes a common language and provides the necessary framework for schools to tackle bullying. The fact that we found positive effects among white students is encouraging but, given the likelihood that increased heterogeneity exists in schools across North America as compared to Norway, schools should be aware that the role of ethnicity/ race and culture must be considered for reasons already stated. Moreover, students come from varied family/home environments and schools should work to engage parents to effectively bring about change in student behavior. Implementation efforts of intervention schools were broad and encompassed a significant regularity and consistency when compared to efforts of comparison schools. This “whole school” approach is the distinguishing feature of the OBPP from less formal bullying prevention activities. Teacher/staff communication and school attention to bullying problems have been predictive of implementation on the school level [34]. These elements are essential to continued education and commitment of the staff, parents, and administration and should remain a priority for schools implementing or sustaining the program despite our
273
mixed findings on the program’s effectiveness. Given the link between students’ personal experiences with victimization and school avoidance [35], schools should continue planning specific, structured curriculum and interventions in the classroom. Peer groups are usually aware of the bullying that occurs [36] and are desirable targets for intervention. Teaching victim assertiveness and bolstering friendship quality among both bullies and victims [37] can serve as two important strategies. We acknowledge certain limitations of our study. This study was a natural experiment and there were measurement issues that we had to account for (establishing the “Olweus standard” and qualitative interviews). Key informant interviews are subject to recall bias, as well as sampling error. One investigator (NB) conducted all the interviews and confirmed obtained data through follow-up contact with key informants to ensure data accuracy and help filling in missing timeline information. Reliance on student reports may have led to under-reporting of victimization, although our rates are similar or higher than previously published studies [17,22,38]. In addition, from year to year, we had varying rates of missing data, due to students not responding to all questions. The school climate survey used in the present study did not have an operational definition and may have caused students to report on incidents involving another student of equal strength, which does not meet the criteria for bullying behavior. Self-reported bullying behavior and student knowledge about what constitutes bullying was not measured, nor did we have observational or cross-informant reports of bullying victimization. Lastly, our results may not be generalizable to other schools wishing to implement the program, given the inherent complexity required for program implementation and historical factors in our study setting. In summary, the Olweus Bullying Prevention Program had some positive effects varying by gender, ethnicity/race, and grade, but no overall effect in these Seattle middle schools. Implementation of the OBPP program may lead to variable differences in effectiveness based on factors related to culture, race, and the influence of the family/home environment. Therefore, schools should be aware of the influence that home, culture, and society have on student behavior, and tailor preventive measures accordingly. A major implication of our study is for schools with an ethnically-diverse student body not to stop implementing the program, but rather encourage the process of ethnic exploration to nurture adolescents’ emerging ethnic identity, while advocating the development of tolerance and sensitivity to other ethnic groups. Identification of resources, administrative support, and commitment to implementation fidelity is an ongoing process. Future studies of large-scale violence prevention programs in the community need to be rigorously evaluated to ensure they are working as intended.
274
N.S. Bauer et al. / Journal of Adolescent Health 40 (2007) 266 –274
Acknowledgments This study was supported by a grant from Health Resources and Service Administration (HRSA) and a National Research Service Award (NRSA) Primary Care Research Fellowship. The authors thank the following people for their contributions to this project: James F. Lymp, Ph.D., with the Child Health Institute for consultation on statistical analyses; Susan Haws, M.P.H., and Michael Arthur, Ph.D., from the Social Development Research Group (SDRG), for their input in the development phase of the project; the district trainers from the Seattle school district and key informants from the participating middle schools for their time, effort, and their continued support; and Susan P. Limber, Ph.D., from the Institute on Family & Neighborhood Life at Clemson University, for her consultation and for coding the qualitative data. References [1] Olweus D. Bullying at School: Blackwell Publishing, 1993. [2] Smith PK, Morita, Y, Junger-Tas J, et al (eds). The Nature of School Bullying: A Cross-National Perspective. New York: Routledge, 1999. [3] Haynie DL, Nansel T, Eitel P, et al. Bullies, victims, and bully/victims: Distinct groups of at-risk youth. J Early Adolesc 2001;21:29 – 49. [4] Schwartz D, Dodge KA, Coie JD. The emergence of chronic peer victimization in boys’ play groups. Child Dev 1993;64:1755–72. [5] Olweus D. Bully/victim problems in school: Facts and intervention. Eur J Psychol Educ 1997;12:495–510. [6] Juvonen J, Graham S, Schuster MA. Bullying among young adolescents: The strong, the weak, and the troubled. Pediatrics 2003;112(6 Pt 1):1231–7. [7] Kim YS, Koh YJ, Leventhal B. School bullying and suicidal risk in Korean middle school students. Pediatrics 2005;115:357– 63. [8] Olweus D. Bullying at school: Basic facts and effects of a school based intervention program. J Child Psychol Psychiatry 1994;35: 1171–90. [9] Patterson GR. Families: Applications of Social Learning to Family Life. Revised. Champaign, IL: Research Press, 1975. [10] Webster-Stratton C, Mihalic S, Fagan A, et al. Blueprints for Violence Prevention, Book Eleven: The Incredible Years—Parent, Teacher, and Child Training Series. Boulder, CO: Center for the Study and Prevention of Violence, 2001. [11] Limber SP, Nation M, Tracy AJ, et al. Implementation of the Olweus Bullying Prevention Programme in the Southeastern United States. In: Smith PK, Pepler D, Rigby K (eds). Bullying in Schools: How Successful can Interventions Be? Cambridge, UK: Cambridge University Press, 2004:334. [12] Eslea M, Smith, PK. The long-term effectiveness of anti-bullying work in primary schools. Educ Res 1998;40:203–18. [13] Salmivalli C, Kaukiainen A, Voeten M. Anti-bullying intervention: Implementation and outcome. Br J Educ Psychol 2005;75(Pt 3):465– 87. [14] Limber SP, Nation M, Tracy AJ, et al. Implementation of the Olweus Bullying Prevention programme in the Southeastern United States. In: Smith PK, Pepler D, Rigby K (Eds.) Bullying in Schools: How Successful Can Interventions Be? Cambridge: Cambridge University Press, 2004, 55–79. [15] Hawkins JD, Catalano, RF, Arthur MW. Promoting science-based prevention in communities. Addict Behav 2002;27:951–76. [16] Solberg M, Olweus D. Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire. Aggressive Behav 2003;29: 239 – 68.
[17] Nansel TR, Overpeck M, Pilla RS, et al. Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. JAMA 2001;285:2094 –100. [18] Glew GM, Fan MY, Katon W, et al. Bullying, psychosocial adjustment, and academic performance in elementary school. Arch Pediatr Adolesc Med 2005;159:1026 –31. [19] Wilson D. The interface of school climate and school connectedness and relationships with aggression and victimization. J School Health 2004;74:293–9. [20] Eslea M, Rees J. At what age are children most likely to be bullied at school? Aggressive Behav 2001;27:419 – 429. [21] Nansel TR, Craig W, Overpeck MD, et al. Cross-national consistency in the relationship between bullying behaviors and psychosocial adjustment. Arch Pediatr Adolesc Med 2004;158:730 – 6. [22] DeVoe JF. Student Reports of Bullying: Results from the 2001 School Crime Supplement to the National Crime Victimization Survey, Statistical Analysis Report (NCES 2005-310). In: National Department for Education Statistics (ed). U.S. Department of Education IoES.Washington, DC: U.S. Government Printing Office, 2005. [23] Rigby K, Slee PT. Bullying among Australian school children: Reported behavior and attitudes toward victims. J Soc Psychol 1991; 131:615–27. [24] Stevens V, De Bourdeaudhuij I, Van Oost P. Bullying in Flemish schools: An evaluation of anti-bullying intervention in primary and secondary schools. Br J Educ Psychol 2000;70(Pt 2):195–210. [25] Baldry AC. “What about bullying?” An experimental field study to understand students attitudes towards bullying and victimization in Italian middle schools. Br J Educ Psychol 2004;74(Pt 4):583–98. [26] Smith PK, Ananiadou K, Cowie H. Interventions to reduce school bullying. Can J Psychiatry 2003;48:591–9. [27] Stevens V, Van Oost P, De Bourdeaudhuij I. The effects of an anti-bullying intervention programme on peers’ attitudes and behaviour. J Adolesc 2000;23:21–34. [28] Romero AJ, Roberts RE. Perception of discrimination and ethnocultural variables in a diverse group of adolescents. J Adolesc 1998;21: 641–56. [29] Theriot MT, Dulmus CN, Sowers KM, et al. Factors relating to self-identification among bullying victims. Child Youth Serv Rev 2005;27:979 –94. [30] Bauer NS, Herrenkohl TI, Lozano P, et al. Childhood bullying involvement and exposure to intimate partner violence. Pediatrics 2006; 118:e235– 42. [31] Shields A, Cicchetti D. Parental maltreatment and emotion dysregulation as risk factors for bullying and victimization in middle childhood. J Clin Child Psychol 2001;30:349 – 63. [32] Carney AG, Merrell KW. Bullying in schools: Perspectives on understanding and preventing an international problem. School Psychol Int 2001;22:364 – 82. [33] Eslea M, Mukhtar K. Bullying and racism among Asian schoolchildren in Britain. Educ Res 2000;42:207–17. [34] Kallestad JH, Olweus D. Predicting teachers’ and schools’ implementation of the Olweus Bullying Prevention Program: A multilevel study. Prev Treat 2003;6:23. [35] Astor RA, Benbenishty R, Zeira A, et al. School climate, observed risky behaviors, and victimization as predictors of high school students’ fear and judgments of school violence as a problem. Health Educ Behav 2002;29:716 –36. [36] Craig WM, Pepler DJ. Peer processes in bullying and victimization: An observational study. Except Educ Canada 1995;5:81–95. [37] Bollmer JM, Milich R, Harris MJ, et al. A friend in need: The role of friendship quality as a protective factor in peer victimization and bullying. J Interpers Violence 2005;20:701–12. [38] Genta ML, Menesini E, Fonzi A, et al. Bullies and victims in schools in central and southern Italy. Eur J Psychol Educ 1996;11:97–110.