- Email: [email protected]
Are violent video game-aggression researchers biased?
Are Violent Video Game-Aggression Researchers Biased? David A. Lishner, Christopher L. Groves, Quin M. Chrobak PII: DOI: Reference:
S1359-1789(15)00095-6 doi: 10.1016/j.avb.2015.07.010 AVB 938
To appear in:
Aggression and Violent Behavior
Received date: Revised date: Accepted date:
17 September 2014 14 July 2015 15 July 2015
Please cite this article as: Lishner, D.A., Groves, C.L. & Chrobak, Q.M., Are Violent Video Game-Aggression Researchers Biased?, Aggression and Violent Behavior (2015), doi: 10.1016/j.avb.2015.07.010
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT 1
NU
SC
RI
PT
Running Head: RESEARCHER BIAS
Are Violent Video Game-Aggression Researchers Biased?
MA
David A. Lishner
University of Wisconsin Oshkosh
TE
D
Christopher L. Groves Iowa State University
AC CE P
Quin M. Chrobak
University of Wisconsin Oshkosh
Corresponding Author:
David A. Lishner, Ph.D. Department of Psychology University of Wisconsin Oshkosh 800 Algoma Blvd. Oshkosh, WI 54901 Phone: 920-424-2301 Email: [email protected]
ACCEPTED MANUSCRIPT RESEARCHER BIAS
2 Author Note
David A. Lishner, Department of Psychology, University of Wisconsin Oshkosh;
RI
Department of Psychology, University of Wisconsin Oshkosh
PT
Christopher L. Groves, Department of Psychology, Iowa State University; Quin M. Chrobak,
Correspondence concerning this manuscript should be addressed to David A. Lishner,
SC
Department of Psychology, University of Wisconsin Oshkosh, 800 Algoma Blvd., Oshkosh, WI
AC CE P
TE
D
MA
NU
54901. E-mail: [email protected]
ACCEPTED MANUSCRIPT RESEARCHER BIAS
3 Abstract
Several recent commentaries have suggested possible researcher bias on the part of scientists
PT
conducting studies that find evidence of a causal link between violent video game play and
RI
aggression. The present paper argues that patterns of authorship, publication, and research findings within the experimental violent video game-aggression literature are inconsistent with
SC
the researcher bias hypothesis. It is concluded that the claim of a causal link between violent
NU
video game play and aggression is a defensible interpretation of the current experimental and
MA
meta-analytic literatures.
AC CE P
TE
D
Keywords: Video Games; Violent; Aggression; Experiments; Meta-Analysis; Bias
ACCEPTED MANUSCRIPT RESEARCHER BIAS
4
Are violent video game-aggression researchers biased? To date, at least nine meta-analyses have been conducted to examine the magnitude and
PT
nature of the association between violent video game play and aggression (Anderson, 2004;
RI
Anderson & Bushman, 2001, Anderson, Carnagey, Flanagan, Benjamin, Eubanks, & Valentine, 2004; Anderson et al., 2010; Ferguson, 2007a; Ferguson 2007b; Ferguson & Kilburn, 2009;
SC
Greitemeyer & Mügge, 2014; Sherry, 2001). The most comprehensive of the meta-analyses,
NU
based on 140 published and unpublished studies consisting of 68,132 participants, estimated the association between violent video game play and behavioral aggression to be approximately r =
MA
.19 (Anderson et al., 2010). Anderson et al. (2010) also evaluated this association among the subset of studies in which violent video game play was experimentally manipulated and
TE
D
behavioral aggression was subsequently measured. This analysis, which included 45 published and unpublished studies and consisted of 3,464 participants, resulted in an effect of comparable
aggression.
AC CE P
magnitude (r = .18), suggesting that violent video game play has a causal influence on behavioral
A Critical View of Violent Video Game-Aggression Researchers Despite these empirical findings, the claim that violent video game play causes behavioral aggression has been contested and vigorously debated in several prominent journals in recent years (Bushman, Rothstein, & Anderson, 2010; Ferguson, 2007b, 2010; Ferguson & Kilburn, 2010; Huesmann, 2010). These debates have focused primarily on statistical and methodological issues regarding interpretation of the empirical literature and the various metaanalytic findings. However, some commentaries have implied a more insidious state of affairs with respect to the behavior of researchers who believe in the existence of a causal effect.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
5
Specifically, some have implied that these researchers may be biased (Ferguson, 2010; Ferguson & Kilburn, 2010).
PT
According to Ferguson (2010), this bias is evident in several ways. First, he has argued
RI
that video game researchers have been overly selective in their reporting of the literature. For example, in commenting on the 2005 American Psychological Association resolution on video
SC
game violence (APA, 2005), Ferguson (2010) claimed that the resolution was “written by a
NU
committee of causal hypothesis scholars commenting largely on their own work and ignoring that of skeptics or research with opposing findings . . .” (p. 72). Ferguson (2010) described the
MA
review of meta-analytic findings reported by Anderson and Bushman (2001) and Anderson (2004) during a court case (ESA, VSDA and IRMA v. Blagojevich, Madigan, & Devine, 2005).
D
He stated that the review “revealed that the authors may have simply ignored research that didn’t
TE
fit with their hypotheses” (Ferguson, 2010, p.73). In support of this claim, he then quoted a
AC CE P
portion of the case proceedings that summarizes the conclusions of several expert witnesses who argued that C. A. Anderson, a prominent violent video game effects researcher, “. . . failed to cite any peer-reviewed studies that had shown a definitive causal link between violent video game play and aggression, but also ignored research that reached conflicting results” (ESA, VSDA and IRMA v. Blagojevich, Madigan, & Devine, 2005, cited in Ferguson, 2010, p. 73). Ferguson & Kilburn (2010) further noted the apparent "neglect" by Anderson et al. (2010) to mention the inverse relation between video game sales and youth violence rates, which they believe offers evidence against the claim of a causal violent video game-aggression effect. A second form of bias implied by Ferguson and colleagues is the way violent video game researchers tend to measure aggression. Their concern is that researchers are overly flexible in how they compute dependent variables derived from laboratory measures of aggression (Elson &
ACCEPTED MANUSCRIPT RESEARCHER BIAS
6
Ferguson, 2014; Ferguson, 2010). According to them, this unstandardized use of aggression measures is favored because researchers are “free to choose outcomes that supported their
PT
hypotheses and ignore outcomes which did not” (Ferguson, 2010, p. 75). Ferguson’s (2010) criticisms of violent video game-aggression research has led him to suggest that the field is
RI
contributing to a “moral panic,” which he defines as “a quest by some members of society to
SC
impose their moral beliefs on the greater society through the tactic of fear” (p. 70).
NU
It is worth noting that critics of violent video game-aggression research also have been accused of engaging in practices one may interpret as biased in nature. For example, three meta-
MA
analyses conducted by Ferguson and colleagues (Ferguson 2007a, b; Ferguson & Killburn, 2009), which have been used to argue against the existence of a violent video-game aggression
TE
D
effect, have been criticized for excessive redundancy and lack of transparency in terms of the studies that were included in the meta-analyses. Bushman et al. (2010) pointed out that these
AC CE P
meta-analyses contain between 54% and 100% overlap in the sampled studies. Also, the number of sampled studies was quite limited compared to the number of studies in the literature available for analysis, yet little rationale was provided for why these studies were selected as opposed to others. Thus, Ferguson’s and colleagues’ meta-analyses may create a false sense of how much disconfirming evidence actually exits for the violent video game-aggression effect. Questions also have been raised regarding the soundness of Ferguson and colleagues criticisms of evidence supporting a causal video game-aggression effect. For example, Ferguson and colleagues have argued that including unpublished studies in meta-analyses tends to create publication bias. This argument has then been used as a basis for criticizing the Anderson et al. (2010) meta-analysis (Ferguson & Kilburn, 2010), which found support for a causal effect of violent game play on aggression. However, Bushman, Rothman, and colleagues countered that
ACCEPTED MANUSCRIPT RESEARCHER BIAS
7
the justification for this argument rests in part on citing the viewpoints of other meta-analytic scholars out of context (Bushman et al., 2010; see also Rothstein & Bushman, 2012).
PT
Bushman et al. (2010) also raised concerns about Ferguson and colleagues claim that
RI
overly flexible analyses of behavioral aggression measures is responsible for inflated estimates of the causal violent video game-aggression effect. Commentaries forwarding this claim (Elson
SC
& Ferguson, 2014a; Ferguson, 2010) have focused primarily on findings that rely on one
NU
laboratory measure of aggression in particular—the Taylor Competitive Reaction Time Task (TCRTT). Yet, Bushman et al. noted that those who espouse this claim appear to disregard
MA
Anderson et al. (2010)’s findings that use of the TCRTT (as opposed to other measures of aggression) failed to moderate the experimental effect of violent game play on aggression.
TE
D
These and other criticisms of Ferguson’s and colleagues’ work can be found in Bushman, Rothstein, and Anderson (2010) and Rothstein and Bushman (2012).
AC CE P
Taken as a whole, bias has been implied on the part of causal violent video game effects researchers as well as on the part of those critical of these researchers.. However, claims of bias are more serious with regard to the former group, because they raise questions about whether the empirical findings documenting the existence of a causal effect can even be trusted. Despite the importance of this issue, it appears to have received little attention in the literature. We sought to remedy this state of affairs by evaluating the tenability of the researcher bias claim as it pertains to findings supporting the causal effect of violent game play on behavioral aggression. This evaluation involved consideration of three forms of potential researcher bias and whether evidence of each type exists. Forms of Researcher Bias
ACCEPTED MANUSCRIPT RESEARCHER BIAS
8
There are at least three forms of researcher bias that would seriously undermine the integrity of the experimental violent video game-aggression literature. First, researcher bias
PT
might be limited to a single, but prominent scholar who is able to unduly influence the direction
RI
of findings in the field as a result of his or her higher empirical output. Second, researcher bias might be limited to a group of scholars, who, given their numbers or ability to organize, are able
SC
to unduly influence the direction of findings in the field. Third, researcher bias toward belief in
NU
the effect might be largely systemic, and is present among most psychologists, even those not contributing directly to the violent video game-aggression literature.
MA
Prominent Scholar Bias
Could the bias of a prominent scholar be unduly influencing the apparent magnitude of a
TE
D
causal association between violent video game play and behavioral aggression? It is important to note that for such a scholar to be a plausible influence on the average effect he or she would need
AC CE P
to have broad influence on the publication or generation of findings in the violent video gameaggression literature. Perhaps he or she has been an editor or associate editor of journals that tend to publish findings supporting the existence of the causal effect. Or perhaps he or she has published highly cited reviews or meta-analyses that concluded in favor of the effect's existence. Additionally, a scholar who conducts experimental work that finds evidence in favor of the causal effect could potentially "stack the deck" by engaging in fraudulent or ethically questionable research practices. Most relevant would be a scholar who meets all of these criteria because he or she has multiple points of influence. But does such a prominent scholar exist in the violent video game effects field? We suggest one does: Craig A. Anderson.1 Not only is his name periodically associated with many of the implied claims of bias mentioned above (Ferguson, 2010; Ferguson & Kilburn, 2010), but he
ACCEPTED MANUSCRIPT RESEARCHER BIAS
9
has been an associate editor at Aggressive Behavior and Personality and Social Psychology Bulletin, as well as an editorial board member for a host of other journals, many of which have
PT
published experimental work supporting the causal effect. He has been the leading author on
RI
multiple meta-analytic reviews that find evidence in favor of a violent video-game aggression effect more generally (Anderson, 2004; Anderson & Bushman, 2001; Anderson, Carnagey,
SC
Flanagan, Benjamin, Ebanks, & Valentine, 2004), and a causal violent video game-aggression
NU
effect more specifically (Anderson et al., 2010). Most significantly, of the 45 experimental studies of the violent video game play effect on behavioral aggression used in the Anderson et al.
MA
(2010) meta-analysis, 11 (24.4%) were conducted by Anderson. Moreover, all 11 of these experiments were included within a subset of 27 experiments that Anderson et al. (2010)
TE
D
identified as meeting “best practice” standards. Within the best practice set of experiments the percentage conducted by Anderson rises to 40.7.
AC CE P
Of course, even if an author is prominent and possesses multiple points of influence, it does not mean he or she is necessarily biased. Nevertheless, is there any evidence that Anderson’s influence is driven by bias? We suggest that examining Anderson’s meta-analytic contributions and experimental work on the causal violent video game-behavioral aggression effect provides a reasonably clear answer to this question. First, although he is an author on several earlier meta-analytic reviews that have been criticized for being limited in their selection (Ferguson 2010), he is first author on the most comprehensive meta-analysis to date (Anderson et al., 2010). This meta-analysis included a larger and broader range of studies than any other meta-analysis published so far, including meta-analyses conducted by those who have been highly critical of evidence suggesting a violent video game-aggression effect (Ferguson 2007a;
ACCEPTED MANUSCRIPT RESEARCHER BIAS
10
Ferguson, 2007b; Ferguson & Kilburn, 2009). The overall comprehensiveness of this most recent meta-analysis would presumably argue against a claim of bias on Anderson’s part.
PT
Second, close inspection of Anderson et al.’s (2010) meta-analysis of studies using
RI
experimental methods reveals that the average causal effect of playing violent video games on behavioral aggression is similar (r = .18) to that of the overall effect (r = .19), which was
SC
computed while including longitudinal and cross sectional studies along with the experimental
NU
studies (Table 4, p.162). Interestingly, the causal effect size becomes a bit bigger if it is computed using only experiments that Anderson et al. (2010) classified as best practice studies (r
MA
= .21; Table 4, p. 162). Recall, that 11 of the experiments contributing to both estimates of the causal effect were conducted by Anderson. Such a preponderance of studies among all the
TE
D
experiments, and most notably the best practice experiments, raises an important question: Is Anderson's experimental work unduly stacking the deck in favor of finding evidence of a causal
AC CE P
effect?
We evaluated this possibility by conducting a moderator analysis of the two sets of experiments identified in Anderson et al. (2010). This was accomplished by computing the difference in average effect size depending on whether an experiment was reported in an article authored by Anderson or not. Our own meta-analytic findings revealed that the average effect of playing violent video games on aggression in experiments conducted by Anderson was r = .19 (14, .24), p < .001 (K = 11; N = 1,488). The average effect was r = .18 (.13, .22), p < .001 (K = 34; N = 1,976) in experiments reported in articles not authored by Anderson. The difference in these average effects sizes was not significantly different, Q(1) = .07, p = .79.2 Among best practices studies, the average effect in experiments not conducted by Anderson was r = .25 (.19, .30), p < .001 (K = 16; N = 1,025). The difference in this average effect compared to the average
ACCEPTED MANUSCRIPT RESEARCHER BIAS
11
effect of experiments conducted by Anderson was not significantly different, Q(1) = 2.30, p = .13. In other words, compared to other violent video game-aggression researchers who conducted
PT
experiments, Anderson’s work produces roughly similar effect sizes.
RI
These findings are consistent with an independent meta-analysis recently reported by Greitemeyer and Mügge (2014) who found that experimental and non-experimental work
SC
published by either Craig Anderson or Brad Bushman (a frequent collaborator) produced an
NU
average effect size comparable to the average effect size found in work published by others (r = .19, K = 20, N = 8,595, p < .001, versus r = .20, K = 58, N = 23,415, p < .001, respectively).
MA
Interestingly, the average effect size found in work published by a prominent critic of violent video game effects, Christopher Ferguson, was substantially smaller than that of other
TE
D
researchers (r = .02, K = 7, N = 2,444, p = .540). Thus, in the absence of evidence of data fraudulence, there appears to be no compelling evidence that Anderson is introducing biased
AC CE P
findings into the causal violent video game-aggression effect literature. Overall, his experimental work appears consistent with the majority of other scholars in the field. Consequently, we found no evidence to support the notion that prominent scholar bias is contributing to evidence supporting a causal link between violent video game play and aggression. Researcher Cabal Bias
Although the above analysis suggests that Anderson himself is not unduly influencing the causal violent video game-aggression empirical literature, it is possible a small group of researchers exists who are doing so in unison. Perhaps Anderson’s empirical work seems unbiased because he is one of several experimental researchers operating in a biased manner, either independently or in collusion with each another. Such a cabal could generate enough data to produce an inflated causal violent video game-behavioral aggression average effect.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
12
To evaluate these possibilities, we again examined the 45 experiments and the subset of 27 best practice experiments identified by Anderson et al. (2010). We reasoned that potential
PT
members of a cabal would most likely be researchers who are repeat contributors to either set of
RI
experimental studies. Among the 27 best practice studies, 16 experiments were conducted by at least one individual who in turn conducted at least one of the other experiments (eleven of the 16
SC
are the Anderson experiments). The average effect among these experiments was r = .20 (.15,
NU
.24), p < .001 (K = 16; N = 1,720). Eleven experiments were conducted by individuals who were unaffiliated with any other experiment in the set. The average effect among these experiments
MA
was r = .24 (.17, .30), p < .001 (K = 11; N = 793), which was not significantly different from the average effect among repeat contributors, Q(1) = 1.00, p = .32.
TE
D
Among all 45 experiments, 24 experiments were conducted by at least one individual who in turn conducted at least one of the other experiments (eleven of the 24 are the Anderson
AC CE P
experiments). The average effect among these experiments was r = .20 (.16, .24), p < .001 (K = 24; N = 2,175). Twenty-one were conducted by individuals who were unaffiliated with any other experiment in the set. The average effect among these experiments was r = .14 (.09, .20), p < .001 (K = 21; N = 1,289), which was not significantly different from the average effect among repeat contributors, Q(1) = 2.87, p = .09. Overall, there is little evidence of potential researcher cabal bias. Among all experiments identified by Anderson et al. (2010) there was a tendency for experiments conducted by repeat contributors (who would most likely reflect a researcher cabal) to produce higher average violent game play-aggression effects than experiments conducted by one-time contributors, but this difference was not statistically significant. Moreover, the direction of difference in average effect
ACCEPTED MANUSCRIPT RESEARCHER BIAS
13
size reverses when one considers experiments that meet more rigorous best practice criteria. Thus, we find little compelling evidence indicating the presence of a researcher cabal bias.
PT
Systemic Bias
RI
Perhaps bias against research that refutes the notion of a causal link between violent video games and aggression exists among most psychologists and social scientists more
SC
generally. As a result, it could be that researchers reporting contradictory results or
NU
interpretations of the empirical literature are systematically excluded from publication. However, publication patterns seem at odds with this notion. One need only look at the published work of
MA
Ferguson, one of the most vocal critics of the violent video game-aggression effect. He and his colleagues have published in a number of peer-juried, health and behavioral journals, including
TE
D
American Psychologist, European Psychologist, Psychological Bulletin, Review of General Psychology, and Social and Personality Compass (Elson & Ferguson, 2014a; Ferguson, 2007a,
AC CE P
b, 2010, 2013; Ferguson & Kilburn, 2009, 2010). Ferguson (2010) was even published in a special section for which Ferguson was the section editor. If systemic bias were promoting a spurious causal effect of violent game play on behavioral aggression, then it seems unlikely that Ferguson and his colleagues would be granted so far ranging a voice in so many well regarded journals.
Summary We suggest that a close look at the patterns of authorship, publication, and research findings makes highly untenable a claim that researcher bias unduly influences experimental evidence of a causal violent video game-aggression effect. A prominent scholar whose influence is widespread enough to potentially alter the magnitude of the average causal effect has published findings with effect sizes that are consistent in magnitude compared to those of other
ACCEPTED MANUSCRIPT RESEARCHER BIAS
14
scholars. His review work also includes the most comprehensive meta-analysis to date. Among repeat contributors of findings to the experimental literature (individuals most likely to represent
PT
a potential biased cabal of researchers) no evidence is found of a meaningful difference in
RI
average effect size produced by their work compared to the work of single contributors of findings to the experimental literature. Finally, respected peer-juried publication outlets in the
SC
health and behavioral sciences, including multiple highly regarded psychology journals, have
NU
given voice to scholars who have argued against the veracity or importance of the violent video game-aggression effect.
MA
Conclusion
Given the outcome of this analysis, we argue that researcher bias should not be
D
considered a valid criticism of the literature regarding the causal effect of violent video game
TE
play on behavioral aggression. Moreover, the current findings seem inconsistent with Ferguson’s
AC CE P
(2010) moral panic claim, at least as it applies to the potential role of empirical contributions by violent video game-aggression researchers. Absent resolution to ongoing methodological and statistical debate (Bushman & Huesmann, 2013; Bushman et al., 2010; Elson & Ferguson, 2014b; Ferguson & Killburn, 2010; Huesmann, 2010; Krahé, 2014; Warburton, 2014), we suggest a conclusion that violent video game play causes more aggression than nonviolent video game play is a defensible interpretation of the current experimental and meta-analytic literatures.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
15 References
American Psychological Association. (2005). Resolution of violence in video games and
PT
interactive media. Retrieved from http://www.apa.org/about/policy/interactive-media.pdf.
RI
Anderson, C.A. (2004). An update on the effects of violent video games. Journal of Adolescence, 27, 113-122.
SC
Anderson, C.A. & Bushman, B. J. (2001). Effects of violent video games on aggressive behavior,
NU
aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12(5), 353-359.
MA
Anderson, C.A., Carnagey, N. L., Flanagan, M., Benjamin, Jr., A. J., Eubanks, J, & Valentine, J. C. (2004). Violent video games: Specific effects of violent content on aggressive
TE
D
thoughts and behavior. Advances in Experimental Social Psychology, 36, 199-249. Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein,
AC CE P
H. R., & Saleem, M. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: A meta-analytic review. Psychological Bulletin, 136, 151-173. Bushman, B. J., & Huesmann, L. R. (2014). Twenty-five years of research on violence in digital games and aggression revisited: A reply to Elson and Ferguson (2013). European Psychologist, 19, 47-55. Bushman, B. J., Rothstein, H. R., & Anderson, C. A. (2010). Much ado about something: Violent video game effects and a school of red herring: Reply to Ferguson and Kilburn (2010). Psychological Bulletin, 136, 182-187. Elson, M., & Ferguson, C. J. (2014a). Twenty-five years of research on violence in digital games and aggression. European Psychologist, 19, 33-46.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
16
Elson, M., & Ferguson, C. J. (2014b). Does doing media violence research make one aggressive?: The ideological rigidity of social-cognitive theories of media violence and a
PT
response to Bushman and Huesmann (2013), Krahé (2013), and Warburton (2013).
RI
European Psychologist, 19, 68-75.
Ferguson, C. J. (2007a). Evidence of publication bias in video game violence effects literature: A
SC
meta-analytic review. Aggression and Violent Behavior, 12, 470-483.
NU
Ferguson, C. J. (2007b). The good, the bad and the ugly: A meta-analytical review of positive and negative effects of violent video games. Psychiatric Quarterly, 78, 309-316.
MA
Ferguson, C. J. (2013). Violent Video Games and the Supreme Court. American Psychologist, 68, 57-74.
TE
D
Ferguson, C. J. (2010). Blazing angels or resident evil? Can violent video games be a force for good? Review of General Psychology,14, 68-81.
AC CE P
Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for indentifying and controlling, and implications for the use of metal-analyses. Psychological Methods, 17, 120-128. Ferguson, C. J., & Kilburn, J. (2009). The public health risks of media violence: A meta-analytic review. Journal of Pediatrics, 154, 759-763 Ferguson, C. J., & Kilburn, J. (2010). Much ado about nothing: the misestimation and overinterpretation of violent video game effects in eastern and western nations: comment on Anderson et al. (2010). Psychological Bulletin, 136(2), 174-178. Greitemeyer, T., & Mügge, D. O. (In press). Video games do affect social outcomes: A metaanalytic review of the effects of violent and prosocial video game play. Personality and Social Psychology Bulletin.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
17
Huesmann, L. R. (2010). Nailing the coffin shut on doubts that violent video games stimulate aggression: Comment on Anderson et al. (2010). Psychological Bulletin, 136, 179-181.
PT
Krahé, B. (2014). Restoring the spirit of fair play in the debate about violent video games: A
RI
comment on Elson and Ferguson (2013). European Psychologist, 19, 56-59. Rothstein, H. R., & Bushman, B. (2012). Publication bias in psychological science: Comment on
SC
Ferguson and Brannick (2012). Psychological Methods, 17, 129-136.
Communication Research, 27, 409-431.
NU
Sherry, J. L. (2001). The effects of violent video games on aggression: A meta-analysis. Human
MA
Warburton, W. (2014). Apples, oranges, and the burden of proof—putting media violence findings into context: A comment on Elson and Ferguson (2013). European Psychologist,
AC CE P
TE
D
19, 60-67.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
18 Notes
1
For the sake of disclosure we note that the second author is now a graduate student
PT
working with Craig A. Anderson at Iowa State University. Evaluation of the researcher bias claim was initiated by the first author and the conclusions forwarded were largely developed
RI
prior to the second author’s acceptance into Craig Anderson’s research lab. The first author has
2
NU
author has never had contact with Craig Anderson.
SC
corresponded with Craig Anderson several times by email in a professional capacity. The third
We thank Craig A. Anderson for providing these data for re-analysis. All meta-analyses
AC CE P
TE
D
MA
are reported as fixed effects given lack of a significant test of heterogeneity in effect sizes.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
19 Highlights
TE
D
MA
NU
SC
RI
PT
Researcher bias among violent video game-aggression researchers was examined. Studies that manipulated violent game play and measured aggression were examined. Meta-analysis was used to examine differences in effect size based on authorship. No evidence was found that researcher bias explained effect size differences. Researcher bias is unlikely to explain the causal violent game-aggression effect.
AC CE P
S1359-1789(15)00095-6 doi: 10.1016/j.avb.2015.07.010 AVB 938
To appear in:
Aggression and Violent Behavior
Received date: Revised date: Accepted date:
17 September 2014 14 July 2015 15 July 2015
Please cite this article as: Lishner, D.A., Groves, C.L. & Chrobak, Q.M., Are Violent Video Game-Aggression Researchers Biased?, Aggression and Violent Behavior (2015), doi: 10.1016/j.avb.2015.07.010
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT 1
NU
SC
RI
PT
Running Head: RESEARCHER BIAS
Are Violent Video Game-Aggression Researchers Biased?
MA
David A. Lishner
University of Wisconsin Oshkosh
TE
D
Christopher L. Groves Iowa State University
AC CE P
Quin M. Chrobak
University of Wisconsin Oshkosh
Corresponding Author:
David A. Lishner, Ph.D. Department of Psychology University of Wisconsin Oshkosh 800 Algoma Blvd. Oshkosh, WI 54901 Phone: 920-424-2301 Email: [email protected]
ACCEPTED MANUSCRIPT RESEARCHER BIAS
2 Author Note
David A. Lishner, Department of Psychology, University of Wisconsin Oshkosh;
RI
Department of Psychology, University of Wisconsin Oshkosh
PT
Christopher L. Groves, Department of Psychology, Iowa State University; Quin M. Chrobak,
Correspondence concerning this manuscript should be addressed to David A. Lishner,
SC
Department of Psychology, University of Wisconsin Oshkosh, 800 Algoma Blvd., Oshkosh, WI
AC CE P
TE
D
MA
NU
54901. E-mail: [email protected]
ACCEPTED MANUSCRIPT RESEARCHER BIAS
3 Abstract
Several recent commentaries have suggested possible researcher bias on the part of scientists
PT
conducting studies that find evidence of a causal link between violent video game play and
RI
aggression. The present paper argues that patterns of authorship, publication, and research findings within the experimental violent video game-aggression literature are inconsistent with
SC
the researcher bias hypothesis. It is concluded that the claim of a causal link between violent
NU
video game play and aggression is a defensible interpretation of the current experimental and
MA
meta-analytic literatures.
AC CE P
TE
D
Keywords: Video Games; Violent; Aggression; Experiments; Meta-Analysis; Bias
ACCEPTED MANUSCRIPT RESEARCHER BIAS
4
Are violent video game-aggression researchers biased? To date, at least nine meta-analyses have been conducted to examine the magnitude and
PT
nature of the association between violent video game play and aggression (Anderson, 2004;
RI
Anderson & Bushman, 2001, Anderson, Carnagey, Flanagan, Benjamin, Eubanks, & Valentine, 2004; Anderson et al., 2010; Ferguson, 2007a; Ferguson 2007b; Ferguson & Kilburn, 2009;
SC
Greitemeyer & Mügge, 2014; Sherry, 2001). The most comprehensive of the meta-analyses,
NU
based on 140 published and unpublished studies consisting of 68,132 participants, estimated the association between violent video game play and behavioral aggression to be approximately r =
MA
.19 (Anderson et al., 2010). Anderson et al. (2010) also evaluated this association among the subset of studies in which violent video game play was experimentally manipulated and
TE
D
behavioral aggression was subsequently measured. This analysis, which included 45 published and unpublished studies and consisted of 3,464 participants, resulted in an effect of comparable
aggression.
AC CE P
magnitude (r = .18), suggesting that violent video game play has a causal influence on behavioral
A Critical View of Violent Video Game-Aggression Researchers Despite these empirical findings, the claim that violent video game play causes behavioral aggression has been contested and vigorously debated in several prominent journals in recent years (Bushman, Rothstein, & Anderson, 2010; Ferguson, 2007b, 2010; Ferguson & Kilburn, 2010; Huesmann, 2010). These debates have focused primarily on statistical and methodological issues regarding interpretation of the empirical literature and the various metaanalytic findings. However, some commentaries have implied a more insidious state of affairs with respect to the behavior of researchers who believe in the existence of a causal effect.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
5
Specifically, some have implied that these researchers may be biased (Ferguson, 2010; Ferguson & Kilburn, 2010).
PT
According to Ferguson (2010), this bias is evident in several ways. First, he has argued
RI
that video game researchers have been overly selective in their reporting of the literature. For example, in commenting on the 2005 American Psychological Association resolution on video
SC
game violence (APA, 2005), Ferguson (2010) claimed that the resolution was “written by a
NU
committee of causal hypothesis scholars commenting largely on their own work and ignoring that of skeptics or research with opposing findings . . .” (p. 72). Ferguson (2010) described the
MA
review of meta-analytic findings reported by Anderson and Bushman (2001) and Anderson (2004) during a court case (ESA, VSDA and IRMA v. Blagojevich, Madigan, & Devine, 2005).
D
He stated that the review “revealed that the authors may have simply ignored research that didn’t
TE
fit with their hypotheses” (Ferguson, 2010, p.73). In support of this claim, he then quoted a
AC CE P
portion of the case proceedings that summarizes the conclusions of several expert witnesses who argued that C. A. Anderson, a prominent violent video game effects researcher, “. . . failed to cite any peer-reviewed studies that had shown a definitive causal link between violent video game play and aggression, but also ignored research that reached conflicting results” (ESA, VSDA and IRMA v. Blagojevich, Madigan, & Devine, 2005, cited in Ferguson, 2010, p. 73). Ferguson & Kilburn (2010) further noted the apparent "neglect" by Anderson et al. (2010) to mention the inverse relation between video game sales and youth violence rates, which they believe offers evidence against the claim of a causal violent video game-aggression effect. A second form of bias implied by Ferguson and colleagues is the way violent video game researchers tend to measure aggression. Their concern is that researchers are overly flexible in how they compute dependent variables derived from laboratory measures of aggression (Elson &
ACCEPTED MANUSCRIPT RESEARCHER BIAS
6
Ferguson, 2014; Ferguson, 2010). According to them, this unstandardized use of aggression measures is favored because researchers are “free to choose outcomes that supported their
PT
hypotheses and ignore outcomes which did not” (Ferguson, 2010, p. 75). Ferguson’s (2010) criticisms of violent video game-aggression research has led him to suggest that the field is
RI
contributing to a “moral panic,” which he defines as “a quest by some members of society to
SC
impose their moral beliefs on the greater society through the tactic of fear” (p. 70).
NU
It is worth noting that critics of violent video game-aggression research also have been accused of engaging in practices one may interpret as biased in nature. For example, three meta-
MA
analyses conducted by Ferguson and colleagues (Ferguson 2007a, b; Ferguson & Killburn, 2009), which have been used to argue against the existence of a violent video-game aggression
TE
D
effect, have been criticized for excessive redundancy and lack of transparency in terms of the studies that were included in the meta-analyses. Bushman et al. (2010) pointed out that these
AC CE P
meta-analyses contain between 54% and 100% overlap in the sampled studies. Also, the number of sampled studies was quite limited compared to the number of studies in the literature available for analysis, yet little rationale was provided for why these studies were selected as opposed to others. Thus, Ferguson’s and colleagues’ meta-analyses may create a false sense of how much disconfirming evidence actually exits for the violent video game-aggression effect. Questions also have been raised regarding the soundness of Ferguson and colleagues criticisms of evidence supporting a causal video game-aggression effect. For example, Ferguson and colleagues have argued that including unpublished studies in meta-analyses tends to create publication bias. This argument has then been used as a basis for criticizing the Anderson et al. (2010) meta-analysis (Ferguson & Kilburn, 2010), which found support for a causal effect of violent game play on aggression. However, Bushman, Rothman, and colleagues countered that
ACCEPTED MANUSCRIPT RESEARCHER BIAS
7
the justification for this argument rests in part on citing the viewpoints of other meta-analytic scholars out of context (Bushman et al., 2010; see also Rothstein & Bushman, 2012).
PT
Bushman et al. (2010) also raised concerns about Ferguson and colleagues claim that
RI
overly flexible analyses of behavioral aggression measures is responsible for inflated estimates of the causal violent video game-aggression effect. Commentaries forwarding this claim (Elson
SC
& Ferguson, 2014a; Ferguson, 2010) have focused primarily on findings that rely on one
NU
laboratory measure of aggression in particular—the Taylor Competitive Reaction Time Task (TCRTT). Yet, Bushman et al. noted that those who espouse this claim appear to disregard
MA
Anderson et al. (2010)’s findings that use of the TCRTT (as opposed to other measures of aggression) failed to moderate the experimental effect of violent game play on aggression.
TE
D
These and other criticisms of Ferguson’s and colleagues’ work can be found in Bushman, Rothstein, and Anderson (2010) and Rothstein and Bushman (2012).
AC CE P
Taken as a whole, bias has been implied on the part of causal violent video game effects researchers as well as on the part of those critical of these researchers.. However, claims of bias are more serious with regard to the former group, because they raise questions about whether the empirical findings documenting the existence of a causal effect can even be trusted. Despite the importance of this issue, it appears to have received little attention in the literature. We sought to remedy this state of affairs by evaluating the tenability of the researcher bias claim as it pertains to findings supporting the causal effect of violent game play on behavioral aggression. This evaluation involved consideration of three forms of potential researcher bias and whether evidence of each type exists. Forms of Researcher Bias
ACCEPTED MANUSCRIPT RESEARCHER BIAS
8
There are at least three forms of researcher bias that would seriously undermine the integrity of the experimental violent video game-aggression literature. First, researcher bias
PT
might be limited to a single, but prominent scholar who is able to unduly influence the direction
RI
of findings in the field as a result of his or her higher empirical output. Second, researcher bias might be limited to a group of scholars, who, given their numbers or ability to organize, are able
SC
to unduly influence the direction of findings in the field. Third, researcher bias toward belief in
NU
the effect might be largely systemic, and is present among most psychologists, even those not contributing directly to the violent video game-aggression literature.
MA
Prominent Scholar Bias
Could the bias of a prominent scholar be unduly influencing the apparent magnitude of a
TE
D
causal association between violent video game play and behavioral aggression? It is important to note that for such a scholar to be a plausible influence on the average effect he or she would need
AC CE P
to have broad influence on the publication or generation of findings in the violent video gameaggression literature. Perhaps he or she has been an editor or associate editor of journals that tend to publish findings supporting the existence of the causal effect. Or perhaps he or she has published highly cited reviews or meta-analyses that concluded in favor of the effect's existence. Additionally, a scholar who conducts experimental work that finds evidence in favor of the causal effect could potentially "stack the deck" by engaging in fraudulent or ethically questionable research practices. Most relevant would be a scholar who meets all of these criteria because he or she has multiple points of influence. But does such a prominent scholar exist in the violent video game effects field? We suggest one does: Craig A. Anderson.1 Not only is his name periodically associated with many of the implied claims of bias mentioned above (Ferguson, 2010; Ferguson & Kilburn, 2010), but he
ACCEPTED MANUSCRIPT RESEARCHER BIAS
9
has been an associate editor at Aggressive Behavior and Personality and Social Psychology Bulletin, as well as an editorial board member for a host of other journals, many of which have
PT
published experimental work supporting the causal effect. He has been the leading author on
RI
multiple meta-analytic reviews that find evidence in favor of a violent video-game aggression effect more generally (Anderson, 2004; Anderson & Bushman, 2001; Anderson, Carnagey,
SC
Flanagan, Benjamin, Ebanks, & Valentine, 2004), and a causal violent video game-aggression
NU
effect more specifically (Anderson et al., 2010). Most significantly, of the 45 experimental studies of the violent video game play effect on behavioral aggression used in the Anderson et al.
MA
(2010) meta-analysis, 11 (24.4%) were conducted by Anderson. Moreover, all 11 of these experiments were included within a subset of 27 experiments that Anderson et al. (2010)
TE
D
identified as meeting “best practice” standards. Within the best practice set of experiments the percentage conducted by Anderson rises to 40.7.
AC CE P
Of course, even if an author is prominent and possesses multiple points of influence, it does not mean he or she is necessarily biased. Nevertheless, is there any evidence that Anderson’s influence is driven by bias? We suggest that examining Anderson’s meta-analytic contributions and experimental work on the causal violent video game-behavioral aggression effect provides a reasonably clear answer to this question. First, although he is an author on several earlier meta-analytic reviews that have been criticized for being limited in their selection (Ferguson 2010), he is first author on the most comprehensive meta-analysis to date (Anderson et al., 2010). This meta-analysis included a larger and broader range of studies than any other meta-analysis published so far, including meta-analyses conducted by those who have been highly critical of evidence suggesting a violent video game-aggression effect (Ferguson 2007a;
ACCEPTED MANUSCRIPT RESEARCHER BIAS
10
Ferguson, 2007b; Ferguson & Kilburn, 2009). The overall comprehensiveness of this most recent meta-analysis would presumably argue against a claim of bias on Anderson’s part.
PT
Second, close inspection of Anderson et al.’s (2010) meta-analysis of studies using
RI
experimental methods reveals that the average causal effect of playing violent video games on behavioral aggression is similar (r = .18) to that of the overall effect (r = .19), which was
SC
computed while including longitudinal and cross sectional studies along with the experimental
NU
studies (Table 4, p.162). Interestingly, the causal effect size becomes a bit bigger if it is computed using only experiments that Anderson et al. (2010) classified as best practice studies (r
MA
= .21; Table 4, p. 162). Recall, that 11 of the experiments contributing to both estimates of the causal effect were conducted by Anderson. Such a preponderance of studies among all the
TE
D
experiments, and most notably the best practice experiments, raises an important question: Is Anderson's experimental work unduly stacking the deck in favor of finding evidence of a causal
AC CE P
effect?
We evaluated this possibility by conducting a moderator analysis of the two sets of experiments identified in Anderson et al. (2010). This was accomplished by computing the difference in average effect size depending on whether an experiment was reported in an article authored by Anderson or not. Our own meta-analytic findings revealed that the average effect of playing violent video games on aggression in experiments conducted by Anderson was r = .19 (14, .24), p < .001 (K = 11; N = 1,488). The average effect was r = .18 (.13, .22), p < .001 (K = 34; N = 1,976) in experiments reported in articles not authored by Anderson. The difference in these average effects sizes was not significantly different, Q(1) = .07, p = .79.2 Among best practices studies, the average effect in experiments not conducted by Anderson was r = .25 (.19, .30), p < .001 (K = 16; N = 1,025). The difference in this average effect compared to the average
ACCEPTED MANUSCRIPT RESEARCHER BIAS
11
effect of experiments conducted by Anderson was not significantly different, Q(1) = 2.30, p = .13. In other words, compared to other violent video game-aggression researchers who conducted
PT
experiments, Anderson’s work produces roughly similar effect sizes.
RI
These findings are consistent with an independent meta-analysis recently reported by Greitemeyer and Mügge (2014) who found that experimental and non-experimental work
SC
published by either Craig Anderson or Brad Bushman (a frequent collaborator) produced an
NU
average effect size comparable to the average effect size found in work published by others (r = .19, K = 20, N = 8,595, p < .001, versus r = .20, K = 58, N = 23,415, p < .001, respectively).
MA
Interestingly, the average effect size found in work published by a prominent critic of violent video game effects, Christopher Ferguson, was substantially smaller than that of other
TE
D
researchers (r = .02, K = 7, N = 2,444, p = .540). Thus, in the absence of evidence of data fraudulence, there appears to be no compelling evidence that Anderson is introducing biased
AC CE P
findings into the causal violent video game-aggression effect literature. Overall, his experimental work appears consistent with the majority of other scholars in the field. Consequently, we found no evidence to support the notion that prominent scholar bias is contributing to evidence supporting a causal link between violent video game play and aggression. Researcher Cabal Bias
Although the above analysis suggests that Anderson himself is not unduly influencing the causal violent video game-aggression empirical literature, it is possible a small group of researchers exists who are doing so in unison. Perhaps Anderson’s empirical work seems unbiased because he is one of several experimental researchers operating in a biased manner, either independently or in collusion with each another. Such a cabal could generate enough data to produce an inflated causal violent video game-behavioral aggression average effect.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
12
To evaluate these possibilities, we again examined the 45 experiments and the subset of 27 best practice experiments identified by Anderson et al. (2010). We reasoned that potential
PT
members of a cabal would most likely be researchers who are repeat contributors to either set of
RI
experimental studies. Among the 27 best practice studies, 16 experiments were conducted by at least one individual who in turn conducted at least one of the other experiments (eleven of the 16
SC
are the Anderson experiments). The average effect among these experiments was r = .20 (.15,
NU
.24), p < .001 (K = 16; N = 1,720). Eleven experiments were conducted by individuals who were unaffiliated with any other experiment in the set. The average effect among these experiments
MA
was r = .24 (.17, .30), p < .001 (K = 11; N = 793), which was not significantly different from the average effect among repeat contributors, Q(1) = 1.00, p = .32.
TE
D
Among all 45 experiments, 24 experiments were conducted by at least one individual who in turn conducted at least one of the other experiments (eleven of the 24 are the Anderson
AC CE P
experiments). The average effect among these experiments was r = .20 (.16, .24), p < .001 (K = 24; N = 2,175). Twenty-one were conducted by individuals who were unaffiliated with any other experiment in the set. The average effect among these experiments was r = .14 (.09, .20), p < .001 (K = 21; N = 1,289), which was not significantly different from the average effect among repeat contributors, Q(1) = 2.87, p = .09. Overall, there is little evidence of potential researcher cabal bias. Among all experiments identified by Anderson et al. (2010) there was a tendency for experiments conducted by repeat contributors (who would most likely reflect a researcher cabal) to produce higher average violent game play-aggression effects than experiments conducted by one-time contributors, but this difference was not statistically significant. Moreover, the direction of difference in average effect
ACCEPTED MANUSCRIPT RESEARCHER BIAS
13
size reverses when one considers experiments that meet more rigorous best practice criteria. Thus, we find little compelling evidence indicating the presence of a researcher cabal bias.
PT
Systemic Bias
RI
Perhaps bias against research that refutes the notion of a causal link between violent video games and aggression exists among most psychologists and social scientists more
SC
generally. As a result, it could be that researchers reporting contradictory results or
NU
interpretations of the empirical literature are systematically excluded from publication. However, publication patterns seem at odds with this notion. One need only look at the published work of
MA
Ferguson, one of the most vocal critics of the violent video game-aggression effect. He and his colleagues have published in a number of peer-juried, health and behavioral journals, including
TE
D
American Psychologist, European Psychologist, Psychological Bulletin, Review of General Psychology, and Social and Personality Compass (Elson & Ferguson, 2014a; Ferguson, 2007a,
AC CE P
b, 2010, 2013; Ferguson & Kilburn, 2009, 2010). Ferguson (2010) was even published in a special section for which Ferguson was the section editor. If systemic bias were promoting a spurious causal effect of violent game play on behavioral aggression, then it seems unlikely that Ferguson and his colleagues would be granted so far ranging a voice in so many well regarded journals.
Summary We suggest that a close look at the patterns of authorship, publication, and research findings makes highly untenable a claim that researcher bias unduly influences experimental evidence of a causal violent video game-aggression effect. A prominent scholar whose influence is widespread enough to potentially alter the magnitude of the average causal effect has published findings with effect sizes that are consistent in magnitude compared to those of other
ACCEPTED MANUSCRIPT RESEARCHER BIAS
14
scholars. His review work also includes the most comprehensive meta-analysis to date. Among repeat contributors of findings to the experimental literature (individuals most likely to represent
PT
a potential biased cabal of researchers) no evidence is found of a meaningful difference in
RI
average effect size produced by their work compared to the work of single contributors of findings to the experimental literature. Finally, respected peer-juried publication outlets in the
SC
health and behavioral sciences, including multiple highly regarded psychology journals, have
NU
given voice to scholars who have argued against the veracity or importance of the violent video game-aggression effect.
MA
Conclusion
Given the outcome of this analysis, we argue that researcher bias should not be
D
considered a valid criticism of the literature regarding the causal effect of violent video game
TE
play on behavioral aggression. Moreover, the current findings seem inconsistent with Ferguson’s
AC CE P
(2010) moral panic claim, at least as it applies to the potential role of empirical contributions by violent video game-aggression researchers. Absent resolution to ongoing methodological and statistical debate (Bushman & Huesmann, 2013; Bushman et al., 2010; Elson & Ferguson, 2014b; Ferguson & Killburn, 2010; Huesmann, 2010; Krahé, 2014; Warburton, 2014), we suggest a conclusion that violent video game play causes more aggression than nonviolent video game play is a defensible interpretation of the current experimental and meta-analytic literatures.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
15 References
American Psychological Association. (2005). Resolution of violence in video games and
PT
interactive media. Retrieved from http://www.apa.org/about/policy/interactive-media.pdf.
RI
Anderson, C.A. (2004). An update on the effects of violent video games. Journal of Adolescence, 27, 113-122.
SC
Anderson, C.A. & Bushman, B. J. (2001). Effects of violent video games on aggressive behavior,
NU
aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12(5), 353-359.
MA
Anderson, C.A., Carnagey, N. L., Flanagan, M., Benjamin, Jr., A. J., Eubanks, J, & Valentine, J. C. (2004). Violent video games: Specific effects of violent content on aggressive
TE
D
thoughts and behavior. Advances in Experimental Social Psychology, 36, 199-249. Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein,
AC CE P
H. R., & Saleem, M. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: A meta-analytic review. Psychological Bulletin, 136, 151-173. Bushman, B. J., & Huesmann, L. R. (2014). Twenty-five years of research on violence in digital games and aggression revisited: A reply to Elson and Ferguson (2013). European Psychologist, 19, 47-55. Bushman, B. J., Rothstein, H. R., & Anderson, C. A. (2010). Much ado about something: Violent video game effects and a school of red herring: Reply to Ferguson and Kilburn (2010). Psychological Bulletin, 136, 182-187. Elson, M., & Ferguson, C. J. (2014a). Twenty-five years of research on violence in digital games and aggression. European Psychologist, 19, 33-46.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
16
Elson, M., & Ferguson, C. J. (2014b). Does doing media violence research make one aggressive?: The ideological rigidity of social-cognitive theories of media violence and a
PT
response to Bushman and Huesmann (2013), Krahé (2013), and Warburton (2013).
RI
European Psychologist, 19, 68-75.
Ferguson, C. J. (2007a). Evidence of publication bias in video game violence effects literature: A
SC
meta-analytic review. Aggression and Violent Behavior, 12, 470-483.
NU
Ferguson, C. J. (2007b). The good, the bad and the ugly: A meta-analytical review of positive and negative effects of violent video games. Psychiatric Quarterly, 78, 309-316.
MA
Ferguson, C. J. (2013). Violent Video Games and the Supreme Court. American Psychologist, 68, 57-74.
TE
D
Ferguson, C. J. (2010). Blazing angels or resident evil? Can violent video games be a force for good? Review of General Psychology,14, 68-81.
AC CE P
Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for indentifying and controlling, and implications for the use of metal-analyses. Psychological Methods, 17, 120-128. Ferguson, C. J., & Kilburn, J. (2009). The public health risks of media violence: A meta-analytic review. Journal of Pediatrics, 154, 759-763 Ferguson, C. J., & Kilburn, J. (2010). Much ado about nothing: the misestimation and overinterpretation of violent video game effects in eastern and western nations: comment on Anderson et al. (2010). Psychological Bulletin, 136(2), 174-178. Greitemeyer, T., & Mügge, D. O. (In press). Video games do affect social outcomes: A metaanalytic review of the effects of violent and prosocial video game play. Personality and Social Psychology Bulletin.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
17
Huesmann, L. R. (2010). Nailing the coffin shut on doubts that violent video games stimulate aggression: Comment on Anderson et al. (2010). Psychological Bulletin, 136, 179-181.
PT
Krahé, B. (2014). Restoring the spirit of fair play in the debate about violent video games: A
RI
comment on Elson and Ferguson (2013). European Psychologist, 19, 56-59. Rothstein, H. R., & Bushman, B. (2012). Publication bias in psychological science: Comment on
SC
Ferguson and Brannick (2012). Psychological Methods, 17, 129-136.
Communication Research, 27, 409-431.
NU
Sherry, J. L. (2001). The effects of violent video games on aggression: A meta-analysis. Human
MA
Warburton, W. (2014). Apples, oranges, and the burden of proof—putting media violence findings into context: A comment on Elson and Ferguson (2013). European Psychologist,
AC CE P
TE
D
19, 60-67.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
18 Notes
1
For the sake of disclosure we note that the second author is now a graduate student
PT
working with Craig A. Anderson at Iowa State University. Evaluation of the researcher bias claim was initiated by the first author and the conclusions forwarded were largely developed
RI
prior to the second author’s acceptance into Craig Anderson’s research lab. The first author has
2
NU
author has never had contact with Craig Anderson.
SC
corresponded with Craig Anderson several times by email in a professional capacity. The third
We thank Craig A. Anderson for providing these data for re-analysis. All meta-analyses
AC CE P
TE
D
MA
are reported as fixed effects given lack of a significant test of heterogeneity in effect sizes.
ACCEPTED MANUSCRIPT RESEARCHER BIAS
19 Highlights
TE
D
MA
NU
SC
RI
PT
Researcher bias among violent video game-aggression researchers was examined. Studies that manipulated violent game play and measured aggression were examined. Meta-analysis was used to examine differences in effect size based on authorship. No evidence was found that researcher bias explained effect size differences. Researcher bias is unlikely to explain the causal violent game-aggression effect.
AC CE P