- Email: [email protected]
Managing Research Environments: Heterarchies in Academia. A Response to Cumming
impossible, we argue that video recording facilitates blinding and is possible in most well-devised experiments. In other studies, extensive photographic or audio evidence may be more appropriate. Representative videos, photos, and audio can be easily archived on journal websites or other free public repositories (e.g., figshare) and the remaining files made available by authors on request. Biomedicine is implementing significant training in scientific integrity–the same is needed in E&E. From the outset, E&E graduate students should be trained to document their data collection in a transparent way (e.g., using video/photo evidence). This will require creativity and customised solutions for each study, especially those conducted in the field, but the benefits to transparency and reproducibility will far outweigh the logistical costs.
Concluding Remarks Scientific discovery relies on high-quality research that is transparent, reproducible, and available for outside scrutiny. Unfortunately, the current reward system selects for poor quality research by incentivising practices that favour sensationalism over quality [11]. Parker et al. highlight key areas in the analysis and reporting of results that accentuate issues of bias in the scientific literature, and suggest that scientists may be unwittingly contributing to the lack of transparency in E&E. We agree, but stress that combatting this issue must also acknowledge the uncomfortable reality of scientific misconduct. Frank and public discussions of this issue are needed to encourage progress and reform. 1
University of Tasmania and CSIRO Agriculture and Food, Hobart, Tasmania, Australia
2
Institute of Biology, University of Neuchâtel, Neuchâtel, Switzerland 3 Great Lakes Institute for Environmental Research, University of Windsor, Canada 4 Department of Biology, University of New Brunswick, Saint John, New Brunswick, Canada 5 Department of Neuroscience, Uppsala University, Uppsala, Sweden Department of Biology, Norwegian University of Science and Technology, Trondheim, Norway
6
900
*Correspondence: [email protected] (T.D. Clark). http://dx.doi.org/10.1016/j.tree.2016.09.006 References 1. Parker, T.H. et al. (2016) Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 2. Fang, F.C. et al. (2012) Misconduct accounts for the majority of retracted scientific publications. Proc. Natl. Acad. Sci. U.S.A. 109, 17028–17033 3. Fanelli, D. (2009) How many scientists fabricate and falsify research?. A systematic review and meta-analysis of survey data. PLoS ONE 4, e5738 4. Begley, C.G. and Ellis, L.M. (2012) Drug development: raise standards for preclinical cancer research. Nature 483, 531– 533 5. Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349, aac4716 6. Ioannidis, J.P.A. (2005) Why most published research findings are false. PLoS Med. 2, e124 7. Freedman, L.P. et al. (2015) The economics of reproducibility in preclinical research. PLoS Biol. 13, e1002165 8. Enserink, M. (2016) Karolinska Institute fires fallen star surgeon Paolo Macchiarini. Science Published online March 23, 2016. http://dx.doi.org/10.1126/science.aaf9825 9. Callaway, E. (2011) Report finds massive fraud at Dutch universities. Nature 479, 15 10. De Los Angeles, A. et al. (2015) Failure to replicate the STAP cell phenomenon. Nature 525, E6–E9 11. Smaldino, P.E. and McElreath, R. (2016) The natural selection of bad science. https://arxiv.org/pdf/1605.09511v1.pdf 12. Roche, D.G. et al. (2015) Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295
exploring relatively well-developed empirical evidence of insufficient transparency in ecology and evolution. We did not review fraud because there is an absence of empirical evidence indicating that fraud is a major problem in ecology and evolution. However, as Clark et al. also note, despite possible growth in fraud in other disciplines over recent decades, it still accounts for only a very small proportion of irreproducible results in those fields [2]. This would be true even if only 1% of fraud is detected. Thus, the available evidence suggests that fraud is not a primary hindrance to scientific progress. We agree with Clark et al. that increased transparency might help limit fraud and we look forward to continued discussion of various ways to promote transparency. 1
Whitman College, Walla Walla, WA, USA Max Planck Institute for Ornithology, Seewiessen, Germany 3 Royal Holloway University of London, Egham, UK 4 University of Melbourne, Melbourne, Australia 5 University of Edinburgh, Edinburgh, UK 6 Université du Québec à Montréal, Montréal, Canada 2
7 8
Stony Brook University, Stony Brook, NY, USA University of New South Wales, Sydney, Australia
*Correspondence: [email protected] (T.H. Parker).
Letter
Fraud Not a Primary Cause of Irreproducible Results: A Reply to Clark et al. Timothy H. Parker,1,* Wolfgang Forstmeier,2 Julia Koricheva,3 Fiona Fidler,4 Jarrod D. Hadfield,5 Yung En Chee,4 Clint D. Kelly,6 Jessica Gurevitch,7 and Shinichi Nakagawa8
http://dx.doi.org/10.1016/j.tree.2016.09.004 References 1. Clark, T.D. et al. Scientific misconduct: the elephant in the lab. Trends Ecol. Evol. (in press). http://dx.doi.org/10.1016/j. tree.2016.09.006. 2. Fang, F.C. et al. (2012) Misconduct accounts for the majority of retracted scientific publications. Proc. Natl Acad. Sci. U.S. A. 109, 17028–17033
Letter
Managing Research Environments: Heterarchies in Academia. A Response to Cumming 1,
We welcome the comment from Clark Joern Fischer * et al. [1] and are gratified that they found value in our article. In response, we would Cumming [1] recently reviewed the conlike to state that our paper was devoted to cept of heterarchy, and discussed its
Trends in Ecology & Evolution, December 2016, Vol. 31, No. 12
potential utility for the analysis of ecosystems and social-ecological systems. The concept of heterarchy invites thinking about system architecture in terms of (i) how hierarchical it is, and (ii) how connected the elements within a given system are. As Cumming showed, these two axes of system architecture are relevant in a wide range of contexts. I argue here that considering heterarchies can generate practical insights for the management of academic (research) environments. Research environments are inherently characterized both by elements of hierarchy among actors, and by elements of connection among them. Nevertheless, the degrees of hierarchy and connectedness differ markedly between workplaces. Considering four stereotypical environments of high versus low connectedness, and strong versus weak hierarchy, enables the explicit consideration of the advantages and disadvantages of different academic environments–thus providing a simple analytical tool for research managers.
projects. Exchange among them is not strictly necessary and remains limited– that is, even within the lab, there is no strong culture of lateral collaboration (e. g., among PhD students or among postdocs). The overall environment is competitive and, arguably, some very successful professors have organized their labs in this way.
The ‘Visionary Facilitator Model’
The second stereotypical environment – the ‘visionary facilitator model’ – is also hierarchical but is highly networked. As in the previous environment, there is a clear leader, such as a head of department or professor running a lab group. The focus in this case, however, is not on ‘controlling’ all projects but rather on providing leadership that facilitates creative research in the direction of a general, agreed upon, vision. For this reason, a ‘visionary facilitator’ will not only provide leadership but will also encourage interactions among researchers, across levels of experience, and between subprojects. In this environment, senior researchers have open doors for more junior researchers, but still provide direction and take responsibility on key decisions.
In the following I stereotype four types of academic environment. Each of these stereotypes exists to some extent in the real world, and each has both strengths and weaknesses. The ‘Collegiate Model’ Third, some research environments have The ‘Guru Model’ a very flat hierarchy, but many interactions The first stereotypical academic environ- among people – the ‘collegiate model’. ment – the ‘guru model’ – is strongly This model is characterized by flat decihierarchical but not highly networked. sion-making structures and joint proThis environment is one of strong silos, cesses to define and work towards such as relatively isolated laboratory goals. It is a model that may be seen, groups. Each such lab group is headed for example, in large, collaborative by a professor (the ‘guru’) and responds research projects involving consortia of to a dean or head of department. In this many actors, but not being led very environment, hierarchy is also pro- strongly by anyone. Communication nounced within lab groups. Postdocs channels, in this model, are accessible occupy places between professors and to researchers at all levels of experience, PhD students, acting as intermediaries, but there is little or no guidance as to or perhaps de facto supervising PhD stu- which research ought to be pursued, or dents (very possibly without receiving how. Arguably, the ‘collegiate’ model official credit for it). In this environment exists in projects or departments where many individuals work on different colleagues enjoy working together, but
nobody wants to take the lead, or in cultures that pride themselves of flat hierarchies.
The ‘Individualistic Model’
Finally, an ‘individualistic model’ exists in research environments that lack strong hierarchy but also lack networking. Such an environment may exist where individual researchers work ‘behind closed doors’ for their own survival, engaging very little with one another, and acting individualistically without immediate guidance or control of a superior. Such research environments may exist, for example, where there are few junior or mid-career researchers, and in disciplines where senior researchers can work largely independently (e.g., in relatively theoretical fields that do not require large lab groups).
Concluding Remarks As emphasized by Cumming [1], there is nothing inherently good or bad about different types of system architecture. However, some architectures will be better adapted to particular challenges than others. Investigating our own workplace, we can ask which of the above stereotypes best describes its architecture. From a management perspective, we might ask: which of the above environments best fosters creativity? Which might generate the highest academic impact? Which is most pleasant to work in? Which is most resilient to major funding cuts, or to higher-level (exogenous) restructuring? Thinking through questions such as these shows the immediate heuristic value of the concept of heterarchies. Arguably, this is one of those relatively rare concepts that are simple but have immediate appeal and utility–hallmarks for their likely uptake in a wide range of contexts. 1
Faculty of Sustainability, Leuphana University Lueneburg, Scharnhorststrasse 1, 21335 Lueneburg, Germany
Trends in Ecology & Evolution, December 2016, Vol. 31, No. 12
901
Reference 1. Cumming, G.S. (2016) Heterarchies: reconciling networks and hierarchies. Trends Ecol. Evol. 31, 622–632
Forum
What Do You Mean, ‘Tipping Point’? Egbert H. van Nes,1,* Babak M.S. Arani,1 Arie Staal,1 Bregje van der Bolt,1 Bernardo M. Flores,1 Sebastian Bathiany,1 and Marten Scheffer1
In this book Malcolm Gladwell describes various social examples, such as fashion trends and changes in criminality rates, where small initial changes led to a runaway process, causing big transitions. A more recent example is the bankruptcy of Lehman Brothers investment bank on 15 September 2008, initiating a global financial crisis [3]. Such relatively small events can accelerate in surprising ways as the transition unfolds. For instance, after Lehman Brothers fell, confidence in the stability of the financial systems was rapidly declining. The rising panic on the markets led banks to increase their liquidity, which contributed further to transmitting the crisis to other economic sectors [3]. In ecology there are also well-known (A)
150
All Environmental
100
50
0 1960
1980
2000
2020
Year
Figure 1. The Recent Rise of the Term ‘Tipping Point’ in the Scientific Literature (Source: ISI Web of Science). The red line shows the number of articles that are labeled with the research field ‘environmental science and ecology’.
examples of such self-propelled accelerating change. For instance, when tree mortality opens up the canopy of tropical forests, grasses might invade, increasing the chances of wildfires that subsequently
Potenal
Original stability landscape
State of system
Potenal
Potenal
Over the past 10 years the use of the term ‘tipping point’ in the scientific literature has exploded. It was originally used loosely as a metaphor for the phenomenon that, beyond a certain threshold, runaway change propels a system to a new state. Although several specific mathematical definitions have since been proposed, we (B) Tipping due to change in condions argue that these are too narrow and that it is better to retain the original definition.
Key: Gladwell’s book “The pping point”
http://dx.doi.org/10.1016/j.tree.2016.09.010
200
book The Tipping Point [2], published in 2000. Number of arcles
*Correspondence: joern.fi[email protected] (J. Fischer).
(C)
Tipping due to change in state
The oldest reference to the metaphor ‘tipping point’ that we encountered was in studies about racial segregation, to denote the set of conditions that led to the rapid flight of the existing white majority class from neighborhoods in US State of system State of system cities in the 1950s [1]. For decades the term was used solely in this context. After 2000 the popularity of the term Figure 2. Two Types of Tipping Points (after [10]). (A) The original potential landscape. (B) Change in rose exponentially, especially in climate external conditions until the current state becomes unstable in a bifurcation, and (C) change in state until the current state becomes unstable in an unstable equilibrium (or saddle point). Red arrows show how the system science, environmental sciences, and changes; black arrows show the accelerated change of the system. We used the well-known model of x x ecology (Figure 1). This sudden increase overgrazing dx dt ¼ x 1 K c x þH ; where c = 2.1 or 1.7; H = 1; K = 10, and the potentials are calculated using the formula of Strogatz [12] (see also Videos V1 and V2). was most likely induced by the popular 2
2
902
Trends in Ecology & Evolution, December 2016, Vol. 31, No. 12
2
Concluding Remarks Scientific discovery relies on high-quality research that is transparent, reproducible, and available for outside scrutiny. Unfortunately, the current reward system selects for poor quality research by incentivising practices that favour sensationalism over quality [11]. Parker et al. highlight key areas in the analysis and reporting of results that accentuate issues of bias in the scientific literature, and suggest that scientists may be unwittingly contributing to the lack of transparency in E&E. We agree, but stress that combatting this issue must also acknowledge the uncomfortable reality of scientific misconduct. Frank and public discussions of this issue are needed to encourage progress and reform. 1
University of Tasmania and CSIRO Agriculture and Food, Hobart, Tasmania, Australia
2
Institute of Biology, University of Neuchâtel, Neuchâtel, Switzerland 3 Great Lakes Institute for Environmental Research, University of Windsor, Canada 4 Department of Biology, University of New Brunswick, Saint John, New Brunswick, Canada 5 Department of Neuroscience, Uppsala University, Uppsala, Sweden Department of Biology, Norwegian University of Science and Technology, Trondheim, Norway
6
900
*Correspondence: [email protected] (T.D. Clark). http://dx.doi.org/10.1016/j.tree.2016.09.006 References 1. Parker, T.H. et al. (2016) Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 2. Fang, F.C. et al. (2012) Misconduct accounts for the majority of retracted scientific publications. Proc. Natl. Acad. Sci. U.S.A. 109, 17028–17033 3. Fanelli, D. (2009) How many scientists fabricate and falsify research?. A systematic review and meta-analysis of survey data. PLoS ONE 4, e5738 4. Begley, C.G. and Ellis, L.M. (2012) Drug development: raise standards for preclinical cancer research. Nature 483, 531– 533 5. Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349, aac4716 6. Ioannidis, J.P.A. (2005) Why most published research findings are false. PLoS Med. 2, e124 7. Freedman, L.P. et al. (2015) The economics of reproducibility in preclinical research. PLoS Biol. 13, e1002165 8. Enserink, M. (2016) Karolinska Institute fires fallen star surgeon Paolo Macchiarini. Science Published online March 23, 2016. http://dx.doi.org/10.1126/science.aaf9825 9. Callaway, E. (2011) Report finds massive fraud at Dutch universities. Nature 479, 15 10. De Los Angeles, A. et al. (2015) Failure to replicate the STAP cell phenomenon. Nature 525, E6–E9 11. Smaldino, P.E. and McElreath, R. (2016) The natural selection of bad science. https://arxiv.org/pdf/1605.09511v1.pdf 12. Roche, D.G. et al. (2015) Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295
exploring relatively well-developed empirical evidence of insufficient transparency in ecology and evolution. We did not review fraud because there is an absence of empirical evidence indicating that fraud is a major problem in ecology and evolution. However, as Clark et al. also note, despite possible growth in fraud in other disciplines over recent decades, it still accounts for only a very small proportion of irreproducible results in those fields [2]. This would be true even if only 1% of fraud is detected. Thus, the available evidence suggests that fraud is not a primary hindrance to scientific progress. We agree with Clark et al. that increased transparency might help limit fraud and we look forward to continued discussion of various ways to promote transparency. 1
Whitman College, Walla Walla, WA, USA Max Planck Institute for Ornithology, Seewiessen, Germany 3 Royal Holloway University of London, Egham, UK 4 University of Melbourne, Melbourne, Australia 5 University of Edinburgh, Edinburgh, UK 6 Université du Québec à Montréal, Montréal, Canada 2
7 8
Stony Brook University, Stony Brook, NY, USA University of New South Wales, Sydney, Australia
*Correspondence: [email protected] (T.H. Parker).
Letter
Fraud Not a Primary Cause of Irreproducible Results: A Reply to Clark et al. Timothy H. Parker,1,* Wolfgang Forstmeier,2 Julia Koricheva,3 Fiona Fidler,4 Jarrod D. Hadfield,5 Yung En Chee,4 Clint D. Kelly,6 Jessica Gurevitch,7 and Shinichi Nakagawa8
http://dx.doi.org/10.1016/j.tree.2016.09.004 References 1. Clark, T.D. et al. Scientific misconduct: the elephant in the lab. Trends Ecol. Evol. (in press). http://dx.doi.org/10.1016/j. tree.2016.09.006. 2. Fang, F.C. et al. (2012) Misconduct accounts for the majority of retracted scientific publications. Proc. Natl Acad. Sci. U.S. A. 109, 17028–17033
Letter
Managing Research Environments: Heterarchies in Academia. A Response to Cumming 1,
We welcome the comment from Clark Joern Fischer * et al. [1] and are gratified that they found value in our article. In response, we would Cumming [1] recently reviewed the conlike to state that our paper was devoted to cept of heterarchy, and discussed its
Trends in Ecology & Evolution, December 2016, Vol. 31, No. 12
potential utility for the analysis of ecosystems and social-ecological systems. The concept of heterarchy invites thinking about system architecture in terms of (i) how hierarchical it is, and (ii) how connected the elements within a given system are. As Cumming showed, these two axes of system architecture are relevant in a wide range of contexts. I argue here that considering heterarchies can generate practical insights for the management of academic (research) environments. Research environments are inherently characterized both by elements of hierarchy among actors, and by elements of connection among them. Nevertheless, the degrees of hierarchy and connectedness differ markedly between workplaces. Considering four stereotypical environments of high versus low connectedness, and strong versus weak hierarchy, enables the explicit consideration of the advantages and disadvantages of different academic environments–thus providing a simple analytical tool for research managers.
projects. Exchange among them is not strictly necessary and remains limited– that is, even within the lab, there is no strong culture of lateral collaboration (e. g., among PhD students or among postdocs). The overall environment is competitive and, arguably, some very successful professors have organized their labs in this way.
The ‘Visionary Facilitator Model’
The second stereotypical environment – the ‘visionary facilitator model’ – is also hierarchical but is highly networked. As in the previous environment, there is a clear leader, such as a head of department or professor running a lab group. The focus in this case, however, is not on ‘controlling’ all projects but rather on providing leadership that facilitates creative research in the direction of a general, agreed upon, vision. For this reason, a ‘visionary facilitator’ will not only provide leadership but will also encourage interactions among researchers, across levels of experience, and between subprojects. In this environment, senior researchers have open doors for more junior researchers, but still provide direction and take responsibility on key decisions.
In the following I stereotype four types of academic environment. Each of these stereotypes exists to some extent in the real world, and each has both strengths and weaknesses. The ‘Collegiate Model’ Third, some research environments have The ‘Guru Model’ a very flat hierarchy, but many interactions The first stereotypical academic environ- among people – the ‘collegiate model’. ment – the ‘guru model’ – is strongly This model is characterized by flat decihierarchical but not highly networked. sion-making structures and joint proThis environment is one of strong silos, cesses to define and work towards such as relatively isolated laboratory goals. It is a model that may be seen, groups. Each such lab group is headed for example, in large, collaborative by a professor (the ‘guru’) and responds research projects involving consortia of to a dean or head of department. In this many actors, but not being led very environment, hierarchy is also pro- strongly by anyone. Communication nounced within lab groups. Postdocs channels, in this model, are accessible occupy places between professors and to researchers at all levels of experience, PhD students, acting as intermediaries, but there is little or no guidance as to or perhaps de facto supervising PhD stu- which research ought to be pursued, or dents (very possibly without receiving how. Arguably, the ‘collegiate’ model official credit for it). In this environment exists in projects or departments where many individuals work on different colleagues enjoy working together, but
nobody wants to take the lead, or in cultures that pride themselves of flat hierarchies.
The ‘Individualistic Model’
Finally, an ‘individualistic model’ exists in research environments that lack strong hierarchy but also lack networking. Such an environment may exist where individual researchers work ‘behind closed doors’ for their own survival, engaging very little with one another, and acting individualistically without immediate guidance or control of a superior. Such research environments may exist, for example, where there are few junior or mid-career researchers, and in disciplines where senior researchers can work largely independently (e.g., in relatively theoretical fields that do not require large lab groups).
Concluding Remarks As emphasized by Cumming [1], there is nothing inherently good or bad about different types of system architecture. However, some architectures will be better adapted to particular challenges than others. Investigating our own workplace, we can ask which of the above stereotypes best describes its architecture. From a management perspective, we might ask: which of the above environments best fosters creativity? Which might generate the highest academic impact? Which is most pleasant to work in? Which is most resilient to major funding cuts, or to higher-level (exogenous) restructuring? Thinking through questions such as these shows the immediate heuristic value of the concept of heterarchies. Arguably, this is one of those relatively rare concepts that are simple but have immediate appeal and utility–hallmarks for their likely uptake in a wide range of contexts. 1
Faculty of Sustainability, Leuphana University Lueneburg, Scharnhorststrasse 1, 21335 Lueneburg, Germany
Trends in Ecology & Evolution, December 2016, Vol. 31, No. 12
901
Reference 1. Cumming, G.S. (2016) Heterarchies: reconciling networks and hierarchies. Trends Ecol. Evol. 31, 622–632
Forum
What Do You Mean, ‘Tipping Point’? Egbert H. van Nes,1,* Babak M.S. Arani,1 Arie Staal,1 Bregje van der Bolt,1 Bernardo M. Flores,1 Sebastian Bathiany,1 and Marten Scheffer1
In this book Malcolm Gladwell describes various social examples, such as fashion trends and changes in criminality rates, where small initial changes led to a runaway process, causing big transitions. A more recent example is the bankruptcy of Lehman Brothers investment bank on 15 September 2008, initiating a global financial crisis [3]. Such relatively small events can accelerate in surprising ways as the transition unfolds. For instance, after Lehman Brothers fell, confidence in the stability of the financial systems was rapidly declining. The rising panic on the markets led banks to increase their liquidity, which contributed further to transmitting the crisis to other economic sectors [3]. In ecology there are also well-known (A)
150
All Environmental
100
50
0 1960
1980
2000
2020
Year
Figure 1. The Recent Rise of the Term ‘Tipping Point’ in the Scientific Literature (Source: ISI Web of Science). The red line shows the number of articles that are labeled with the research field ‘environmental science and ecology’.
examples of such self-propelled accelerating change. For instance, when tree mortality opens up the canopy of tropical forests, grasses might invade, increasing the chances of wildfires that subsequently
Potenal
Original stability landscape
State of system
Potenal
Potenal
Over the past 10 years the use of the term ‘tipping point’ in the scientific literature has exploded. It was originally used loosely as a metaphor for the phenomenon that, beyond a certain threshold, runaway change propels a system to a new state. Although several specific mathematical definitions have since been proposed, we (B) Tipping due to change in condions argue that these are too narrow and that it is better to retain the original definition.
Key: Gladwell’s book “The pping point”
http://dx.doi.org/10.1016/j.tree.2016.09.010
200
book The Tipping Point [2], published in 2000. Number of arcles
*Correspondence: joern.fi[email protected] (J. Fischer).
(C)
Tipping due to change in state
The oldest reference to the metaphor ‘tipping point’ that we encountered was in studies about racial segregation, to denote the set of conditions that led to the rapid flight of the existing white majority class from neighborhoods in US State of system State of system cities in the 1950s [1]. For decades the term was used solely in this context. After 2000 the popularity of the term Figure 2. Two Types of Tipping Points (after [10]). (A) The original potential landscape. (B) Change in rose exponentially, especially in climate external conditions until the current state becomes unstable in a bifurcation, and (C) change in state until the current state becomes unstable in an unstable equilibrium (or saddle point). Red arrows show how the system science, environmental sciences, and changes; black arrows show the accelerated change of the system. We used the well-known model of x x ecology (Figure 1). This sudden increase overgrazing dx dt ¼ x 1 K c x þH ; where c = 2.1 or 1.7; H = 1; K = 10, and the potentials are calculated using the formula of Strogatz [12] (see also Videos V1 and V2). was most likely induced by the popular 2
2
902
Trends in Ecology & Evolution, December 2016, Vol. 31, No. 12
2