Quest for the Golden Rule: An effective social skills promotion and bullying prevention program

Computers & Education 56 (2011) 166–175

Contents lists available at ScienceDirect

Computers & Education journal homepage: www.elsevier.com/locate/compedu

Quest for the Golden Rule: An effective social skills promotion and bullying prevention program Alice Rubin-Vaughan a, *, Debra Pepler b, Steven Brown c, Wendy Craig d a

Graduate Psychology Department, York University, 4700 Keele Street, Toronto, Ontario, Canada M3J 1P3 York University and The Hospital for Sick Children, 555 University Avenue, Toronto, Ontario, Canada M5G 1X8 c Practi-Quest Corp, Canada d Queen’s University, 99 University Avenue, Kingston, Ontario, Canada K7L 3N6 b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 28 April 2010 Received in revised form 4 August 2010 Accepted 9 August 2010

Everyday many students face bullying situations that they are ill equipped to manage. E-learning has recently emerged as a potentially effective tool in teaching children social skills, in addition to academic subject matter. Quest for the Golden Rule is one of the first bullying prevention e-learning programs available, designed by the Practi-Quest Corporation, for children in grades 2 – 5. The purpose of the current study was to explore data collected as part of standard program quality assurance practices to evaluate the impact of the gaming modules on how much children learned through interacting with the modules. Sample sizes ranged from 226 to 438 depending on the module; with approximately equal numbers of boys and girls. Following their interactions with each module, children’s knowledge of bullying and their identification of strategies to prevent bullying improved significantly. The majority of children reported that they enjoyed the game and felt confident that they could help solve bullying problems. Quest for the Golden Rule is an engaging, effective, and efficient means of raising awareness, fostering positive attitudes, and promoting effective problem-solving for bullying prevention in schools. Ó 2010 Elsevier Ltd. All rights reserved.

Keywords: Bullying prevention Program evaluation E-learning Educational computer gaming Elementary school age

1. Introduction Everyday many students face bullying situations that they are often ill equipped to manage. Bullying has repercussions, not only for children’s social and emotional well-being (e.g., Arseneault et al., 2006), but also for academic achievement and school absenteeism (e.g., Kshirsagar, Agarwal, & Bavdekar, 2007; Nansel et al., 2004; Ybarra & Mitchell, 2004). Adults must support children in managing these situations, as they often require skills exceeding children’s developmental capacities. Teachers and caregivers alike are charged with the responsibility of educating children in both academic and social domains. Although Canadian children are performing well on the international stage in terms of academics, ranking 3rd, 4th, and 7th out of 57 countries on science, reading, and mathematics, respectively (Bussiere, Knighton, & Pennock, 2007), the same is not true in terms of children’s social experiences at school. Based on responses to the World Health Organization–Health Behaviors in School Aged Children survey, Canadian girls ranked 26th and boys ranked 21st out of 40 countries on measures of bullying and victimization, respectively (Craig et al., 2009). Given the discrepancy between students’ academic and social experiences it is clear that we must focus on further supporting children’s social development at school. There are several ways in which adults can support children individually and in the context of their peer groups. According to Pepler (2006), adults can provide support to individual children through scaffolding or coaching, which children require to achieve skills beyond their developmental level. Scaffolding is a critical element of bullying prevention initiatives for children who bully, children who are victimized, and children who witness bullying (Pepler, 2006). Scaffolding can be provided in innovative and engaging ways through educational gaming. The current study provides a preliminary evaluation of a pioneering suite of web-based bullying prevention games, called Quest for the Golden Rule: Bullying Prevention Software. These games are designed to engage children in bullying prevention exercises

* Corresponding author. Tel.: þ1 416 678 6661. E-mail addresses: [email protected] (A. Rubin-Vaughan), [email protected] (D. Pepler), [email protected] (S. Brown), [email protected] (W. Craig). 0360-1315/$ – see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2010.08.009

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

167

through computer-based gaming modules, and are based on the principle of scaffolding through the provision of tailored and progressive support for each student. Although many students experience bullying and can benefit from education and support in addressing these challenging social dynamics, there is a small group of children (10%) who reports consistently high levels of bullying over time and who require more intensive support including practice and coaching to learn essential social skills (Pepler, Jiang, Craig, & Connolly, 2008). Because public schools tend to be under-resourced, it is often beyond the capacity of classroom teachers to provide the intensive and unique learning opportunities that some students require. Quest for the Golden Rule may provide an alternative means to provide intensive support to students in school through individualized experiences, an opportunity not typically provided through traditional approaches to bullying prevention. Early prevention and intervention for bullying problems are crucial to support children in developing healthy academic, social, and emotional coping skills (e.g., Mitchell, Ybarra, & Finkelhor, 2007). In a comprehensive review of thirty stringent studies of bullying prevention programs, Ttofi and Farrington, (2009) demonstrated the effectiveness in reducing bullying and victimization by an average of 20–23%. There is modest evidence to support the effectiveness of bullying prevention programs for elementary school students (e.g., Craig, Pepler, & Shelley, 2004; Smokowski & Kopasz, 2005); however, in their meta-analysis, Ttofi and Farrington, (2009) found that programs for children older than eleven were more effective. Because of the importance of prevention and early intervention, it is essential to increase the effectiveness of bullying programs for young children to support them before early adolescence, which is a developmental period of increased risk for bullying (Pellegrini & Bartini, 2001; Pepler, Jiang, Craig, & Connolly, 2008). Quest for the Golden Rule was designed to incorporate many of the effective characteristics of established bullying prevention programs (as recommended by Ttofi & Farrington, 2009). In addition to the primary elements, Quest for the Golden Rule provides an enhanced learning experience through the use of interactive gaming technology, which is individualized, accessible, and attractive for younger children. Few computer-based bullying prevention programs have been developed for students in elementary school. An objective of the current study was to provide preliminary data regarding the effectiveness of interactive bullying prevention software designed to increase knowledge and change attitudes. 2. Why use educational gaming? The issue of “play” may be particularly salient in work with children and pre-adolescents. At this developmental stage, the importance of play in facilitating learning has long been recognized. Vygotsky (1978) posited that play creates the opportunity for children to experiment with acting more maturely than their developmental stages, thus creating a “zone for proximal development”. Children’s social development can be fostered through play and experimentation with new social skills. Despite recognition of the importance of play in teaching children, there is a dearth of software programs that are acceptable to both children (as being fun and engaging) and adults (as providing “serious” education) (De Castell & Jenson, 2003). 3. Learning and educational gaming Active learning, metacognition, and transfer of knowledge have been identified as crucial components of learning (Huffaker & Calvert, 2003; Shih, Feng, & Tsai, 2008). Huffaker and Calvert (2003) reviewed the literature and concluded that e-learning may support the critical components of learning through: 1. children’s motivation to engage in the learning process; 2. self-direction such that children learn to plan and monitor their learning; and 3. collaborative activities and entertainment features that encourage the application of learning to situations outside the classroom. Educational gaming programs capitalize on experiential learning by actively engaging students with the computer program. Recently, researchers have begun to address the gap in the literature regarding whether elementary and secondary school students fare better in educational gaming environments versus traditional or combined programs. Much of this research has focused on secondary school students with mixed results regarding whether the students learned more from a traditional versus combined curriculum (Annetta, Minogue, Holmes, & Cheng, 2009; Chandra & Lloyd, 2008). Only engagement and motivation were consistently found to be higher with the inclusion of educational gaming experiences. Research on elementary school students’ acquisition of sun safety knowledge revealed that learning was heightened, for younger students, by a combination of traditional teacher presentations and computer programs compared to traditional presentations. In comparison, there was no difference between combined and traditional programs for older students (Buller et al., 2008). Given that the development of language skills is an essential task for early elementary school students, the presentation of learning materials through both traditional and audiovisual and graphic means may augment the learning process (Buller et al., 2008). 4. Bullying prevention through educational gaming The detrimental consequences for those involved in bullying, as the child who bullies, the child who is victimized, or children who witness bullying incidents provide the impetus for the development of bullying prevention programming through educational gaming. Although overall, traditional bullying prevention programs have been found to be effective, there is considerable variability in the results, particularly for younger students (Baldry & Farrington, 2007; Ttofi & Farrington, 2009). Early prevention is critical because of the potential during early childhood to build skills at a developmental stage during which students tend to be more accepting of adult-directed curriculum, more willing to talk to adults about bullying, and have more trust that adults are able to help with bullying problems (Craig, Pepler, & Blais, 2007; Rigby, 2002). Educational gaming may be one new direction to pursue in bolstering the effectiveness of bullying prevention programs for young children. There are several areas of social skills promotion and bullying prevention that are ripe for education programs geared for younger children. Elementary school students benefit from learning a variety of skills that are readily reinforced by teachers at school, including: 1. empathy, 2. emotional and behavioral regulation (e.g., recognize emotions, stop to regulate emotions through strategies like counting or deep breathing), 3. coping with feelings of sadness or anxiety, 4. social skills (e.g., joining group of peers, turn taking, getting positive attention), 5. positive leadership skills – to engage power dynamics positively, 6. alternative problem-solving, and 7. withstanding peer

168

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

pressure (awareness, stop and think, explore responses to peers) (Vaughan & Pepler, 2007). Skill-building computer games have been developed to achieve both bullying and more general violence prevention goals for elementary and secondary school students. Specific educational lessons are required in considering bullying, which represent a unique social problem involving challenging dynamics and a pattern of behavior that signify a power imbalance in the relationship with multiple players in different roles (Olweus, 1993). The current review yielded only two published articles with data regarding bullying prevention programs including multimedia components. McLaughlin, Laux, and Pescara-Kovach (2006) designed a unique experimental study to explore the impact of adding multimedia interventions (CD-ROM with relaxation exercises and a quiz show) to the traditional counselor/teacher bullying prevention program for grade three children. The findings suggested that the interventions were effective in significantly reducing bullying and victimization; however, were inconclusive regarding whether the degree of change was related to the addition of the multimedia components (McLaughlin, Laux, & Pescara-Kovach, 2006). Although multimedia programming was used, the format failed to meet the recommendations of Huffaker and Calvert (2003) in terms of how e-learning may facilitate the educational process. The multimedia component was not animated nor was it interactive or self-directed, therefore, it may not have led to increased motivation for students to engage in the learning process. FearNot! (Fun with Empathetic Agents to Reach out Novel Outcomes in Teaching) is a virtual role playing bullying prevention program (Hall, Woods, Hall, & Wolke, 2007). The FearNot! Program, developed for students aged 8–12, focuses on helping children to develop a deeper understanding of bullying issues and coping strategies, through their ability to empathize with the virtual characters and act as an invisible friend to a child who was victimized (Paiva et al., 2005). Students were found to benefit from this program, such that FearNot! significantly increased the likelihood that students reporting victimization at baseline would escape victimization by the first follow-up assessment compared to the control group, particularly those who interacted more with the characters (Sapouna et al., in press). Furthermore, greater levels of empathy and story comprehension occurred when animated characters were of the same gender as the participant, particularly for boys (Paiva et al., 2005; Woods, Hall, Dautenhahn, & Wolke, 2007). Evaluation of the FearNot! program provides evidence for the utility of educational gaming in addressing bullying, though it does not extend our understanding of the contribution of educational gaming over the traditional bullying prevention approach. Based on the immense success of the gaming industry and the clear appeal that gaming holds for children, the use of gaming may facilitate engagement with educational content. Children are tech-savvy, with 94% having access to computers with the Internet at home. Furthermore, online games are a favorite pastime for younger children – particularly among grade four students, 89% of whom report playing games online (ERIN Research & Wings, 2005). Despite the prominent societal role of technology in education, recreation, and business, and the subsequent opportunity to explore the utility of e-learning for intervening in serious social problems such as bullying, innovation into bullying prevention gaming has been slow to emerge. Although there is a relative dearth of research, emerging evidence from social skills promotion and bullying prevention programs using interactive computer software provides support for the potential effectiveness of an educational gaming approach in teaching students social skills and coping strategies to manage bullying situations. In the present study, we expected that Quest for the Golden Rule would be effective in teaching children bullying prevention messages. 5. Program description: QUEST for the Golden Rule The current study provides a preliminary evaluation of a new set of innovative prevention and intervention tools designed by the PractiQuest Corporation (www.practiquest.com), in consultation with leading researchers in the area of bullying from PREVNet (Promoting Relationships and Eliminating Violence Network). Quest for the Golden Rule uses principles of effective bullying prevention, translated into a modality of particular interest for children and youth – interactive, animated web-based games. The games are designed in such a way as to encourage experiential learning within a virtual and interactive format, facilitated through a safe and private virtual environment free of peer pressure. Children interact individually with animated characters in virtual role-plays, which provide them with the opportunity to learn and practice social skills and try out different strategies to cope with bullying. Students are unable to move on from a social problemsolving situation until they are able to provide a prosocial solution, ensuring that children are supported in learning skills that they may be lacking. Quest for the Golden Rule provides a solution to the typical problems of providing bullying prevention education within traditional classroom settings in which teachers are often overwhelmed, resources are not available to identify students who may need additional support, or teachers lack the time to provide the intensive coaching and practice required by some students. Companion guides for teachers accompany the software and include targeted curriculum and follow-up activities. Students are encouraged to participate in additional activities at home. Three modules have been designed for students in grades two to five each addressing a different topic related to bullying prevention. 5.1. Bark Academy This module provides an introduction to social justice, safety and fairness in school, introduces bullying and depicts three forms of bullying behavior (social, verbal, and physical). Students are invited to attend Bark Academy, a dog school where bullying manifests itself in a variety of unfair behaviors, where they help solve cases of bullying by using the concept of The Golden Rule. Captain Fairness, a super-dog who helps address bullying and creates fair environments for kids, assists students as they work through the module. In the first case, Farid, a much bigger and stronger student, is bullying Kate and Kerri. Upon closer investigation, students learn that Farid is upset at being disrespected and excluded from play by Kerri and Kate. The students must help Kerri and Kate realize they were participating in social bullying and help Farid discover more appropriate ways to respond when he feels left out. In the second case, Jermaine is physically bullying Hershey while a number of bystanders watch and laugh. Jermaine is also verbally bullying Hershey by making comments about his weight. The students must help Jermaine and the bystanders understand how they contributed to the bullying situation and explain the importance of following The Golden Rule. The module ends as the bystanders are confronted with their roles in the bullying situation. Students are left to question what impact the bystanders have in Hershey’s case and are encouraged to return for part two of Bark Academy to find out what the bystanders should have done differently. In the final case, Bobby

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

169

participates in social and cyber-bullying by threatening to send an email to the entire school that teases Yong about his ugly coat. The students must help Bobby realize that he should not send the email by using the concept of The Golden Rule. Moreover, the students help Yong learn that reporting bullying is not tattling. 5.2. Mission to Mars This module teaches students about the social skills involved in making friends. The role-play begins as the students, as space cadets, travel to Mars to be interviewed by a group of astronauts called the “eXplorers”. If the students impress the eXplorer astronauts, Wynona, Adam and Jessie, with qualities such as kindness, and respect, they are invited to become the newest eXplorer and walk on Mars. Once the students arrive on Mars, they are introduced to Lionel, an assistant whom the students must mentor. Lionel and the students build a space helmet together. Before being interviewed by the eXplorers the students watch Reggie, another candidate, complete his interview. Reggie is unsuccessful in becoming an eXplorer because he tries to impress Wynona, Adam and Jessie with his talents rather than listening and asking questions about the eXplorers. The eXplorers decide his self-centeredness could be dangerous on Mars. The eXplorers then interview the students, and when the correct options are chosen, they are invited to become the newest eXplorer. During the interview Lionel interrupts to show how well he can blow bubbles. He accidentally drops a pile of bubble gum on one of the eXplorer’s desks. The students must treat Lionel with kindness during this incident in order to win the respect of the eXplorer team. Immediately after becoming an eXplorer, students are given the task of rescuing Reggie, who had decided to explore Mars on his own and fell into a giant hole. Once the students decide to use the help of the eXplorer team to save Reggie, the team then works together to explore a strange golden light seen on the planet. The students and the team discover this light to be The Golden Rule. 5.3. Ghoul School This module addresses the important issue of how to respond when faced with bullying situations and empowers students to safely refuse and report bullying. In this module there is ample opportunity for students to explore, discuss, and practice appropriate and inappropriate responses to bullying. Students are taught three important strategies for responding to bullying: “Lead with Respect”, “Let an Adult Help” and “Leave with Respect”. Students are also taught to monitor their decisions to determine whether their actions make bullying problems bigger or smaller. In the second part of Ghoul School, students are introduced to Mousey. Mousey is a two-headed monster that enjoys telling silly jokes. He is often bullied by Molnar. Mousey has trouble staying calm and the students must first help Mousey become calm before he can respond to the bullying. As Mousey calms down, the bullying nearly stops and Mousey is free to respond to his situation by either leading through assertiveness, letting an adult help, or leaving. The students help the monsters decide to incorporate The Golden Rule into their own lives, and in the end, they all seek out new professions where they are accepted for who they are. 6. Current study Three key research questions were explored in this study, with corresponding hypotheses. 1. Were there any significant gender or grade differences in the children’s initial levels of knowledge demonstrated in the Bark Academy, Mission to Mars, and Ghoul School modules?  We expected that boys and girls would have similar levels of knowledge but that average knowledge of bullying would increase with each grade. 2. Did children’s knowledge of how to cope with bullying improve significantly following interaction with the Bark Academy, Mission to Mars, and Ghoul School and was this related to gender or grade?  We expected that students would demonstrate significant knowledge gains following interaction with the games, and that the degree of learning would be related to gender and grade. 3. Did children enjoy the Bark Academy, Mission to Mars, and Ghoul School modules?  We expected that the majority of children would favorably rate their subjective enjoyment of the software.

7. Method Quest for the Golden Rule was investigated independently of Practi-Quest by PREVNet researchers. The current study represents a preliminary investigation with plans for subsequent formal evaluation in the upcoming years. 7.1. Participants The participant pool constituted children who received this interactive computer programming in fall 2008 and winter 2009. Although Quest for the Golden Rule was designed for children in grades two to five, grade six students were also included in this study. Several sixth grade teachers in schools that were adopting the modules were interested in having their classes participate, thus it was left to their discretion to determine if the modules were developmentally appropriate for their students. Schools at arms-length from the software developers were targeted for the sale of the software. Schools were recruited to participate in preliminary testing of these modules through the endorsement of school board representatives or willingness of the principal to adopt this new form of bullying prevention programming. The modules were offered freely in a small portion of schools, with the remaining schools having purchased, for a small fee, the rights to use the software in their classrooms as part of their educational curriculum. Access to the program was provided to each student individually – with teachers having online access, enabling them to log in to see how their students were doing.

170

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

Although data about ethnicity were not collected systematically, the schools from which data were collected represented a large Canadian urban centre, populated by cultural, religious and ethnically diverse groups. Because students often only completed one module, we had different participants and sample sizes for each game. The number of students who completed each module was 307 for Bark Academy, 226 for Mission to Mars, and 438 for Ghoul School. The breakdown of participants with data at pre- and post-module by gender and grade follows in Table 1. 7.2. Measures Practi-quest staff and PREVNet researchers worked collaboratively to develop questions to address the specific knowledge and attitudes related to the skills taught within each module. Children were asked to respond to the questions for each module prior to, and immediately following, their use of the module. For the majority of questions, the response options were based on a five-point Likert scale (ranging from “1 – strongly disagree” to “5 – strongly agree”). Because the Bark Academy and Ghoul School modules were found to be too long for children to complete in one class session, they were broken into two sections that could be completed at different times. For these two modules, the questions were similarly divided into two sections depending on the information contained in each section. For the purpose of the current study, only students who completed both sections of a module were included in the analyses to understand what students learned from the full module. To assess engagement with each of the modules, students were asked at the end of each module to rate their agreement with the statement “This computer game was fun” on a five-point Likert scale. 7.2.1. The Bark Academy Commensurate with topic areas covered by the modules, the Bark Academy–Fairness Scale (BA: Fairness Scale) questions addressed attitudes about fairness and social justice and about bullying and coping strategies. The fairness and social justice items include ratings on statements such as: I should always be fair even to people who I do not like; I am willing to work harder to make my school fair for everybody; and I believe there is never an excuse to bully another person. The attitudes about bullying and coping strategies items include ratings on statements such as: Watching other students bully someone is always taking part; If I see someone getting bullied I would keep asking more adults for help until the problem is solved; and If I am being bullied I will always ask an adult to keep me safe. Seven items were combined to make a scale, labeled the BA: Fairness Scale; however, the reliability was best when one statement was removed (It is always tattle-telling to tell an adult if someone is being bullied). Although this question was reverse coded so that higher scores reflected a more positive response, the question still may not have fit with the other items because students were more likely to respond negatively to this question than the others. With the remaining six items (maximum scale total of 30) the internal consistency was still low, but adequate, with the Cronbach’s alphas being .55 and .68 at pre- and post-module, respectively. 7.2.2. The Mission to Mars Friendliness Scale (MM: Friendliness Scale) questions addressed attitudes and knowledge about respectful treatment of friends and peers, and strategies to stay safe from bullying. Eleven items rated on a five-point Likert scale were combined to make a total score for the MM: Friendliness Scale and included the following items about friendship: If a student is mean to me I should be mean back to them; If a student won’t talk to me it means they don’t like me; I would like to learn how to be more respectful and friendly; I should learn to compliment others more often; I always deserve to be treated with respect and friendliness; I have thought about joining a new group of friends soon; I have thought about inviting a new person to join my group of friends; I will always treat someone with friendliness even if I don’t like him/her; The best way to stay safe from bullying is staying near good friends; and I feel comfortable making new friends. The reliability was best when one statement was removed (My friends could be more friendly and respectful to me). This item may not have fit with the other items because students were more likely to respond negatively. The ten remaining items from this module were combined into a scale (maximum scale total of 50), found to have good internal consistency, with Cronbach’s alphas of .70 and .72 for pre- and postmodule scales, respectively. 7.2.3. The Ghoul School Ghoul School was assessed using questions in several domains including children’s knowledge of how to identify bullying and strategies to respond when faced with bullying situations, including safely refusing and reporting bullying. Fifteen items, each rated on a five-point Likert scale were combined to make the Ghoul School: Attitudes Scale (GS: Attitudes Scale), with a maximum scale total of 75. Items on this scale capture the attitudes of students toward their right to be treated respectfully, and strategies they might use in the face of bullying situations and included statements such as: When being bullied, the most important thing for me to do is to make the problem smaller; My

Table 1 Number and Percentage of Participants for each Module, by Gender and Grade. Variable

Gender Girls Boys Grade 2 3 4 5 6

Number and Percentage of Participants Bark Academy

Mission to Mars

Ghoul School

142 (46.4%) 165 (53.6%)

113 (50.0%) 113 (50.0%)

220 (50.3%) 218 (49.7%)

30 57 108 108 4

(9.6%) (18.5%) (35.3%) (35.3%) (1.4%)

1 49 76 99 1

(0.5%) (21.5%) (33.6%) (43.9%) (0.5%)

25 96 144 148 25

(5.7%) (21.8%) (32.8%) (33.9%) (5.7%)

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

171

friends should always treat me with respect and fairness; My friends always treat me with respect and fairness; I am a unique and special person; I would let an adult help me if a bullying problem is too big for me to handle on my own or if I feel unsafe; I think that respectful, calm people are less likely to be bullied; Showing that I am upset will make the bullying worse; I can avoid being bullied if I stay in a safe place or stay near respectful friends; I can make bullying problems smaller; I would like to practice staying calm when being bullied; I would like to practice ways to respectfully say no to bullying; I have the right to always be safe; To leave with respect I should make eye contact then calmly and proudly say I don’t accept being bullied and leave; Acting weak will always make the bullying worse; and Being aggressive will always make the bullying worse. This scale was found to have good internal consistency, with Cronbach’s alphas of .88 and .91 for preand post-module scales, respectively. The Ghoul School: Strategies Scale (GS: Strategies Scale), which assessed strategies to prevent and cope with physical victimization, was based on the number of strategies endorsed through discrete responses (“yes” or “no”) of students to the item “If someone has unfairly shoved me many times this week, and just shoved me again, I will.”. Students were required to select all the correct answers given the ten possible options including: make myself calm before I do anything, push them back to show them that I am not afraid, ignore the pushing and quietly wait for the problem to go away, if I feel upset not show that I am upset, make sure I stay safe and even run away if that’s what’s needed, leave with respect, let and adult help if I can’t fix the problem alone, stand tall, shoulders back and use a calm clear voice while I make eye contact, calmly and proudly say I don’t accept being bullied, or leave before the problem becomes bigger. The Ghoul School Prevention scale had a maximum scale total of ten items endorsed, and was found to have good internal consistency with Cronbach’s alphas of .89 and .91 for pre- and post-module scales, respectively. Students’ recognition of four key bullying prevention strategies were assessed through their responses to Item 9, “To make a bullying problem smaller, I can.”. The potential responses students could select, given the instruction to select all that apply, included: leave with respect by maintaining my own respect and respect for others, be with others who treat me with respect, let a trusted adult help, lead with respect, and say that bullying is not cool. Finally, students’ identification of elements of the bullying definition was evaluated through their responses to Item 10, “Veronica and Ibolya always point and laugh when Ivan gets on the school bus. How do we know if Ivan is being bullied?” Students were instructed to select all of the correct answers out of three options: it’s not fair to Ivan, it keeps happening, and it hurts Ivan. Because these two items were comprised of a small number of discrete responses, with maximum scale totals of four and three, respectively, they were explored descriptively. 7.3. Procedure Practiquest software was adopted by whole schools, or classrooms within a school, to be used as part of the educational curriculum. Students within each classroom were assigned usernames and passwords to access the modules. Although the program was delivered at school, some students took home their passwords and later accessed the games and continued to play from their home computers. As part of participation in this program Practiquest collected data for program research and development as well as for quality control. Data from every student who used the program were accessible to the program developers and these data were used in the current study. 8. Results 8.1. Data Screening From the descriptive statistics, it was clear that before and after the games most children responded in a manner reflecting ample understanding of the definition of bullying, and with knowledge of some skills required to cope with bullying situations (see Table 2 for descriptive statistics). On average, before using the modules students “agreed” with prosocial attitudes and knowledge on items contained within the BA: Fairness Scale, MM: Friendliness Scale, and GS: Attitudes and Strategies Scales. Children had some understanding of the definition of bullying, correctly identifying 1.8 of 3 elements of the definition. They were also able to identify, on average, 5.5 out of 10 strategies to cope with physical bullying and 2.4 out of 4 strategies to address bullying problems. Diagnostics were completed to explore each variable prior to statistical analyses. On the BA: Fairness Scale and GS: Attitudes and Strategies Scales, square root and natural log transformations were used due to concerns regarding violations of normality and equal variances. Analyses were run with and without these transformations. Because no substantive differences emerged for these scales, and given that ANOVA is robust despite violations of the assumption of equal variances (particularly given roughly equal numbers in each group), we decided to use to the original variables. To control for type 1 error, given the number of analyses conducted, we used a Bonferroni correction with a criterion of p < .01. 8.2. Were there gender or grade differences in children’s initial levels of knowledge and attitudes? 8.2.1. Bark Academy No significant gender differences were found in knowledge on the BA: Fairness Scale prior to using the module, t(346) ¼ .61, p ¼ .545. There were significant grade differences on pre-module scores F(4) ¼ 4.39, p ¼ .002. Post hoc tests revealed that our hypothesis that Table 2 Descriptive Statistics: Means, Standard Deviations, and 95% Confidence Intervals. Variables

Pre Mean (SD) Confidence Interval

Post Mean (SD) Confidence Interval

BA: Fairness Scale (n ¼ 307) MM: Friendliness Scale (n ¼ 226) GS: Attitudes Scale (n ¼ 438) GS: Strategies Scale (n ¼ 438)

26.32 40.37 63.70 5.66

26.88 42.20 66.24 6.04

(3.04) (5.61) (8.82) (3.33)

(26.00, 26.64) (49.62, 41.13) (62.88, 64.53) (5.34, 5.97)

(3.30) (5.56) (9.03) (3.47)

(26.51, 27.25) (41.45, 42.95) (65.40, 67.09) (5.72, 6.34)

172

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

Table 3 Descriptive Statistics for Pre Data Across Grade Level: Mean (Standard Deviation).

Grade 2 3 4 5 6 a

BA: Fairness Scale

MM: Friendliness Scale

10.92 9.01 10.21 9.17 11.33

30.00 18.57 19.90 19.87 15.00

(4.25) (2.51) (3.24) (2.77) (3.55)

(n/a)a (7.58) (5.35) (4.52) (n/a) a

GS: Attitudes Scale

GS: Strategies Scale

27.80 26.38 27.52 24.38 28.00

4.05 5.03 5.65 6.55 4.60

(12.34) (9.99) (8.46) (6.78) (10.15)

(3.46) (3.63) (3.21) (2.97) (3.91)

Standard deviation was not applicable because there was only one participant in the group.

knowledge would be greater among student in higher grades was only partially supported. Grade six students scored significantly higher than students in grades three and five but there were no other significant differences among grades (see Table 3 for descriptive statistics by grade). 8.2.2. Mission to Mars No significant gender or grade differences were found in knowledge on the MM: Friendliness Scale prior to using the module, t (212) ¼ .93, p ¼ .356 and F(4) ¼ 1.54, p ¼ .192, respectively. 8.2.3. Ghoul School Across both GS: Attitudes and GS: Strategies Scales there were statistically significant gender differences such that the girls’ pre-module knowledge was greater than that of the boys, t (436) ¼ 2.23, p ¼ .026 and t (436) ¼ 3.17, p ¼ .002 respectively. Each scale showed significant grade differences, F (3) ¼ 3.76, p ¼ .011 and F (3) ¼ 7.92, p < .000, respectively. Post hoc tests revealed that although grade five students had the lowest average GS: Attitudes Scale score, they were only significantly lower than students in grade four. Conversely, grade five students scored highest on the GS: Strategies Scale and the mean score was significantly higher than for students in grades two and three. Grade four students, likewise, had the second highest mean score and were trending in the direction of knowing significantly more strategies to prevent bullying than students in grade two (see Table 3 for descriptive statistics by grade). Our hypothesis that knowledge increased corresponding to advancing grade was not supported for the GS: Attitudes Scale but was somewhat supported for the GS: Strategies Scale, suggesting that knowledge of bullying prevention strategies increased with development. 8.3. Were there improvements in children’s knowledge following interaction with each of the modules, and was this related to gender or grade? 8.3.1. Bark Academy On the BA: Fairness Scale, a significant time by gender interaction was found, F (1, 307) ¼ 4.29, p ¼ .039. Effect size was estimated by partial h2 ¼ .014 with observed power ¼ .542. Girls’ knowledge improved significantly more than boys’ knowledge (see Fig. 1). Grade was included as a covariate in this analysis but was not significant, F (1, 307) ¼ .20, p ¼ .657. 8.3.2. Mission to Mars The MM: Friendliness Scale revealed a significant main effect of time, F (1, 214) ¼ 32.37, p ¼ .000. Effect size was estimated by partial h2 ¼ .132 with observed power ¼ 1.000. Because there was no time by gender interaction, F (1, 214) ¼ .031, p ¼ .861, the bullying knowledge of boys and girls improved similarly through this module (see Fig. 2). Grade was not included as a covariate because no significant differences were found between grades in the pre-module scores. 8.3.3. Ghoul School Trends were identified for both the GS: Attitudes Scale and GS: Strategies Scales, in terms of a main effect of time, F (1, 430) ¼ 3.01, p ¼ .084 and F (1, 430) ¼ 2.76, p ¼ .097, respectively. Effect size was estimated by partial h2 ¼ .007 with observed power ¼ .409, and partial h2 ¼ .006 observed power ¼ .383, respectively. No time by gender interactions, F (1, 430) ¼ .09, p ¼ .763 and F (1, 430) ¼ 1.79, p ¼ .193,

Fig. 1. BA: Fairness Scale changes in knowledge from pre to post-module.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

173

Fig. 2. MM: Friendliness Scale changes in knowledge from pre to post-module.

respectively, nor covariate effects of grade emerged. These trends are suggestive that students’ attitudes towards empowerment and knowledge of bullying prevention strategies improved after interacting with the module. The additional items were explored using z-test of proportions. Knowledge of four strategies students could use to make bullying problems smaller was measured in the Ghoul School module. Children were able to identify significantly more bullying prevention strategies after the module, compared to before (z ¼ 6.45). After using the module, 315 (71.9%) of the children were able to identify at least 3 out of 4 strategies compared to 222 (50.7%) before. Children’s knowledge of the elements of the bullying definition was also measured in the Ghoul School module. Children were able to identify significantly more elements of the bullying definition after the module, compared to before (z ¼ 6.23). After using the module, 299 (68.3%) of the children were able to identify at least 2 out of 3 elements of the bullying definition compared to 208 (47.5%) before. 8.4. Did children enjoy using the interactive computer modules? The majority of children reported that they enjoyed the modules. When asked if the computer game was fun, 93.4% who played Bark Academy, 93.0% who played Mission to Mars, and 83.6% who played Ghoul School “strongly agreed” or “agreed”. Of those children who played Ghoul School, 84.5% “strongly agreed” or “agreed” that they could make bullying problems smaller, reflecting confidence in the skills that they learned or were reinforced through the computer game. In summary, there is evidence that children’s knowledge of bullying and their identification of strategies to prevent bullying improved following their interaction with each module. By the typical conventions of observed power analysis (greater than .8 representing adequate power), we had relatively low power for these statistical tests to detect significant differences. With larger samples or over a longer followup period, the differences detected may have been stronger, and we may have had more power to detect gender or grade differences. Furthermore, the estimates of effect size (partial h2) were small for all the scales examined, suggesting that although we found statistically significant improvements, they were relatively modest, thus we must be cautious in our interpretation. 9. Discussion Bullying is a relationship problem that is challenging for both children and adults to manage. Early education and bullying prevention programs are crucial to provide children with opportunities to learn and practice skills so they are better equipped to manage bullying problems. Educational gaming, such as Quest for the Golden Rule, provides a fun and interactive way for students to learn individually and for classes to have a collective experience that promotes a safe and positive climate. Because educational gaming – as a method of teaching social skills or bullying prevention – is fairly new, there is limited research available to explore whether these programs are effective and for whom. Evidence from the preliminary investigation of the suite of games in Quest for the Golden Rule shows significant improvements in knowledge and attitudes across each of the three modules. Children gained knowledge about issues of fairness and safety in their schools (Bark Academy), social skills (Mission to Mars), and strategies to refuse and cope with bullying (Ghoul School). The vast majority of students reported that they enjoyed using the games; further evidence of their enjoyment and engagement was provided through log-on records indicating that many students took their passwords home and played the games from their home computers. Based on research suggesting that boys and girls interact differently with online gaming environments (e.g., Paiva et al., 2005), we hypothesized that gender would be a factor in understanding students’ learning through the modules. As children completed the games the only evident gender difference in girls’ and boys’ responses was found on the Bark Academy, with girls’ knowledge of fairness and safety improving significantly more than that of boys. On the MM: Friendliness Scale, knowledge of social skills increased for both boys and girls and, despite initial gender differences on the GS: Attitudes and Strategies Scales a trend was found for both boys and girls toward an increase in knowledge of empowerment and physical victimization prevention strategies. Although gender differences were evident on the BA: Fairness Scale in gains of knowledge and changes in attitude related to respect and fairness, overall there were many commonalities indicating that Quest for the Golden Rule modules are effective in teaching both boys and girls about issues related to bullying and bullying prevention strategies. Before the children had experienced these educational games, there were significant grade differences in children’s scores on the BA: Fairness Scale, GS: Attitudes and Strategies Scales. It is notable, however, that grade was not statistically significant in understanding how

174

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

much girls and boys learn about bullying prevention through these two games. Because of the relatively low power in these analyses, it may also be that the power to detect grade-based variations was limited. Another interpretation of these findings is that because Quest for the Golden Rule is an interactive computer game that children complete in a self-directed manner, it is equally effective across elementary school grades in leveling out differences in understanding of bullying. These findings highlight the universal applicability of the program for students in grades two to five, such that children work at their own pace, attend to elements that are of greatest interest to them, experience success in resolving difficult social situations, and spend additional time acquiring skills and understanding in areas in which their skills are under-developed. As a result, children’s experiences with Quest for the Golden Rule modules are tailored in a manner that supports optimal learning. This type of individualized support provides the scaffolding recommended by Pepler (2006) that is an integral element of bullying prevention. Given that teachers are often overburdened and do not have time to provide the individualized coaching that some children require, the design of this computer program enables students in need of additional support to receive more opportunities to practice their skills and develop social competence. This fun learning experience is consistent with Vygotsky’s (1978) observation that through play children are able to develop skills and understanding beyond their existing levels of competence. 9.1. Limitations Although the current study was not conducted using a full experimental design, results are generally consistent with those of the FearNot! program (Sapouna et al., in press) in the sense that improvements in knowledge and attitudes were found for students who used the games. Caution must be taken in interpreting the results due to the low power to detect change and modest effect sizes. These difficulties may be attributed, in part, to the post-testing immediately after the students had completed the modules. With a subsequent follow-up assessment after a period of time in which the lessons from the games can be reinforced by teachers, peers, and parents, we might find an even stronger program effect. An important aspect of program evaluation is to determine for whom programs work and why. In the present study, we did not have information about students’ involvement in bullying, victimization, or witnessing bullying, which may have been a factor in successful learning through the program. Given the overall high level of bullying prevention knowledge demonstrated by these students, it would be helpful to understand the nature of students’ involvement in bullying and their associated levels of knowledge. Conducting research using an experimental design would strengthen the analyses through provision of a control group with which to make comparisons. An important future direction will be to explore, through an experimental design, how bullying prevention games alone, traditional bullying prevention, or a combination of gaming and traditional bullying programs compare to a control condition in terms of children’s acquisition of bullying prevention knowledge and skills, as well as the longer-term impact on the bullying experiences of students. 10. Conclusion Educational gaming appears to be a promising area through which to bolster the effectiveness of early bullying prevention programs for elementary school students. Bullying prevention computer games complement a whole-school approach by providing cumulative and individualized learning of prosocial knowledge, attitudes, and skills within the classroom context that may help shift the social norms within the class and decrease bullying behavior. This preliminary investigation of Quest for the Golden Rule provided a unique opportunity to explore the effectiveness of gaming in bullying prevention. One of the greatest strengths of this program is the individualized nature of the games, and subsequent support given to students in the form of scaffolding to help them develop skills above their existing levels of competence. This program shows great promise in teaching elementary school students about bullying and strategies to cope with these difficult social situations. Supported through whole-school policies and adult guidance, Quest for the Golden Rule contributes important bullying prevention education that will help set children up for healthy social development and healthy relationships throughout their lives. Acknowledgments First and foremost, we would like to thank the students and teachers who participated in this study. We are grateful to the staff members at Practi-Quest Corp who have engaged in this important and innovative effort to make appealing and educational bullying prevention gaming software. This project was also greatly supported by PREVNet (Promoting Relationships and Eliminating Violence Network), through the researchers who collaborated in developing Quest for the Golden Rule, and in providing graduate student funding for research on this suite of bullying prevention games. References Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education, 53, 74–85. Arseneault, L., Walsh, E., Trzesniewski, K., Newcombe, R., Caspi, A., & Moffitt, T. E. (2006). Bullying victimization uniquely contributes to adjustment problems in young children: a nationally representative cohort study. Pediatrics, 118(1), 130–138. Baldry, A. C., & Farrington, D. P. (2007). Effectiveness of programs to prevent school bullying. Victims & Offenders. Special Issue on Early Intervention, 2, 183–204. Buller, M. K., Kane, I. L., Martin, R. C., Giese, A. J., Cutter, G. R., Saba, L. M., et al. (2008). Randomized trial evaluating computer-based sun safety education for children in elementary school. Journal of Cancer Education, 23, 74–79. Bussiere, P., Knighton, T., & Pennock, D. (2007). Measuring up: Canadian results of the OECD PISA study, the Performance of Canada’s youth in Science, Reading and Mathematics. Ottawa: Human Resources and Social Development Canada & Statistics Canada. Chandra, V., & Lloyd, M. (2008). The methodological nettle: ICT and student achievement. British Journal of Educational Technology, 39(6), 1087–1098. Craig, W., Harel-Fisch, Y., Fogel-Grinvald, H., Dostaler, S., Hetland, J., Simons-Morton, B., et al. (2009). A cross-national profile of bullying and victimization among adolescents in 40 countries. International Journal of Public Health, 54, S1–S9. Craig, W. M., Pepler, D. J., & Blais, J. (2007). Responding to bullying: what works? International Journal of School Psychology, 28, 15–24. Craig, W., Pepler, D., & Shelley, D. (2004). Summary of interventions to address Bullying problems at school. Report prepared for the Ministry of Education. Government of Ontario.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175

175

De Castell, S., & Jenson, J. (2003). OP-ED: serious play. Journal of Curriculum Studies, 35(6), 649–665. ERIN Research Inc., & Wings, C. (2005). Young Canadians in a wired world – Phase II: Student survey. Canada: Media Awareness Network. Hall, L., Woods, S., Hall, M., & Wolke, D. (2007). Children’s emotional interpretation of synthetic character interactions. Affective Computing and Intelligent Interaction, 4738, 642–653. Huffaker, D. A., & Calvert, S. L. (2003). The new science of learning: active learning, metacognition, and transfer of knowledge in e-learning applications. Journal of Educational Computing Research, 29(3), 325–334. Kshirsagar, V. Y., Agarwal, R., & Bavdekar, S. B. (2007). Bullying in schools: Prevalence and short-term impact. Indian Pediatrics, 44(1), 25–28. McLaughlin, L., Laux, J. M., & Pescara-Kovach, L. (2006). Using multimedia to reduce bullying and victimization in third-grade urban schools. Professional School Counselling, 10 (2), 153–161. Mitchell, K. J., Ybarra, M., & Finkelhor, D. (2007). The relative importance of online victimization in understanding depression, delinquency, and substance use. Child Maltreatment, 12, 314–324. Nansel, T. R., Craig, W., Overpeck, M. D., Saluja, G., Ruan, W. J., & Health Behavior in School-aged Children Bullying Analyses Working Group. (2004). Cross-national consistency in the relationship between bullying behaviors and psychosocial adjustment. Archives of Pediatrics & Adolescent Medicine, 158(8), 730–736. Olweus, D. (1993). Bullying at school: What we know and what we can do. Oxford: Blackwell. Paiva, A., Dias, J., Sobral, D., Aylett, R., Woods, S., Hall, L., et al. (2005). Learning by feeling: evoking empathy with synthetic characters. Applied Artificial Intelligence, 19, 235–266. Pellegrini, A., & Bartini, M. (2001). Dominance in early adolescent boys: affiliative and aggressive dimensions and possible functions. Merrill-Palmer Quarterly, 47, 142–163. Pepler, D. (2006). Bullying Interventions: a binocular perspective. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 15, 16–20. Pepler, D., Jiang, D., Craig, W., & Connolly, J. (2008). Developmental trajectories of bullying and associated factors. Child Development, 79, 325–338. Rigby, K. (2002). A meta-evaluation of methods and approaches to reducing bullying in pre- schools and in early primary school in Australia. Canberra: Commonwealth AttorneyGeneral’s Department. Sapouna, M., Wolke, D., Vannini, N., Watson, S., Woods, S., Schneider, W., Enz, S., et al. Virtual learning intervention to reduce victimization to primary school: a controlled trial. Journal of Child Psychology and Psychiatry, in press. Shih, M., Feng, J., & Tsai, C. (2008). Research and trends in the field of e-learning from 2001 to 2005: a content analysis of cognitive studies in selected journals. Computers and Education, 51, 955–967. Smokowski, P. R., & Kopasz, K. H. (2005). Bullying in school: an overview of types, effects, family characteristics, and intervention strategies. Children & Schools, 27, 101–110. Ttofi, M. M., & Farrington, D. P. (2009). What works in preventing bullying: effective elements of anti-bullying programmes. Journal of Aggression, Conflict and Peace Research, 1 (1), 13–24. Vaughan, A., & Pepler, D. (2007). Bullying. In L. Beal (Ed.), The ABC’s of Mental Health Teacher Resource. The Hincks-Dellcrest Centre. http://www.brocku.ca/teacherresource/ ABC/. Vygotsky, L. (1978). Mind in Society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Woods, S., Hall, L., Dautenhahn, K., & Wolke, D. (2007). Implications of gender differences for the development of animated characters for the study of bullying behavior. Computers in Human Behavior, 23, 770–786. Ybarra, M. L., & Mitchell, K. J. (2004). Youth engaging in online harassment: associations with caregiver-child relationships, Internet use, and personal characteristics. Journal of Adolescence, 27(3), 319–336.