“It is important to reinforce the importance of …”: ‘Hype’ in reports of randomized controlled trials

English for Specific Purposes 54 (2019) 139–151

Contents lists available at ScienceDirect

English for Specific Purposes journal homepage: http://ees.elsevier.com/esp/default.asp

“It is important to reinforce the importance of .”: ‘Hype’ in reports of randomized controlled trials Neil Millar a, *, Françoise Salager-Meyer b, Brian Budgell c a

University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8573, Japan Universidad de los Andes, Merida, Av Principal de Los Chorros de Milla, Mérida 5101, Mérida, Venezuela c Canadian Memorial Chiropractic College, 6100 Leslie St, North York, ON M2H 3J1, Canada b

a r t i c l e i n f o

a b s t r a c t

Article history:

Research articles serve not just to inform but also to convince. Consequently, authors may be inclined to employ language to ‘sell’ aspects of their study. Such language may undermine objective and disinterested interpretation and bias readers’ evaluation of new knowledge. Randomized Controlled Trials (RCTs) are a type of study that aims to minimise bias when testing treatments and, in medicine, RCTs are generally regarded as the ‘gold standard’. This study provides quantitative and qualitative descriptions of how authors of RCTs use hyperbolic and/or subjective language to glamorise, promote and/or exaggerate aspects of their research – a phenomenon we refer to as ‘hype’. From a corpus of twentyfour RCTs in orthopaedic medicine we identified 161 hypes which we categorised for functional target and linguistic realization. Hypes in RCTs are most prevalent in Discussion sections and most frequently serve to aggrandize the methodology and sell the paper. Findings are discussed in relation to competition, pressure to publish, and the influence of standardised guidelines. Implications for the producers and consumers of the medical literature are considered. Ó 2019 Elsevier Ltd. All rights reserved.

Keywords: Written discourse analysis Evaluative language Corpus linguistics Medical discourse English for medical purposes

1. Introduction Recent years have seen inexorable growth in scientific information – especially in science, technology and medicine. About 2.5 million peer reviewed scientific articles (89% in English) – the purpose of which is to report original research and assess its contribution in a given area – are published yearly in approximately 25,400 peer reviewed research journals across all disciplines in all languages the world over (Ulrich’s Periodicals Directory, 2017). This number is increasing by 3.5% a year. In the medical field alone, PubMed today cites over 20 million papers. The increasing number of researchers worldwide (cf. Lillis & Curry, 2013) is one factor that explains this “avalanche of information” (Fraser & Dunstan, 2010: 314). But the ethos of “Publish or Perish” is also responsible for this hyperinflation of information. Pressure is placed on scholars all over the world (cf. Habibzadeh, 2014; Salager-Meyer, 2014) to (rapidly) publish their research results in high-status, international (English-medium) journals for tenure, promotion and career advancement (Nature Editorial, 2015). Indeed, unread is unknown. However, the highly competitive selection process makes it increasingly difficult for scholars to get their studies published in highly-ranked journals.

* Corresponding author. E-mail address: [email protected] (N. Millar). https://doi.org/10.1016/j.esp.2019.02.004 0889-4906/Ó 2019 Elsevier Ltd. All rights reserved.

140

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

Indeed, in recent years the number of grant applications and the demands of promotion and tenure systems have steadily increased. Cronin (2013: 1) argues that junior scholars are now running “like rats in a treadmill”, and Brauser (2014: 2) reports that a substantial proportion of Dutch medical professors consider the process as a “horse race” that leads to emotional exhaustion and burnout. It is against this backdrop that a number of undesirable and ethically questionable behaviours have been increasing. These range from “mild” research misconduct (e.g. the publication of ‘least publishable units’) to serious cases of fraud, data fabrication and plagiarism (Fanelli, 2009; Raghav, et al., 2015). In this respect, approximately 2% of scientists across a range of disciplines admit violating the rules of ethics (Martinson, Anderson, & de Vries, 2005). Such behaviour can be linked to a recent increase in article retractions (Grieneisen & Zhang, 2012; Haug, 2015; Steen, Casadevall, & Fang, 2013). According to Bailey (2013), the number of paper retractions due to academic fraud has increased up to tenfold in the past three decades with plagiarism and duplicate publications at the root of about 25% of those retractions. But a more mundane and subtle ethical issue arises with what Ecklund, Johnson, and Matthews (2015) refer to as ‘overhyping’. This consists in making one’s research methodology or findings look ‘sexier’ and more ground breaking than these actually are and/or in exaggerating or overstating the potential implications of one’s research results. Lately, over-interpretation, overstatement and misreporting of scientific results have been frequently reported and criticized not only in research articles (Ioannidis, 2005; Macleod et al., 2014; Ochodo et al., 2013), but also in academic press releases (Dodds et al., 2015). The present study assesses how authors in the field of medicine use hyperbolic and/or subjective language to glamorize, promote and/or exaggerate aspects of their research – a phenomenon referred to as ‘hype’ in the literature – and suggests pedagogical applications of the findings. 2. Background The use of hyperbolic subjective language in academic discourse to glamorize results, secure grants and/or to get published has received several names, such as ‘marketization’ (Fairclough, 1993), ‘elements of selling’ (Bhatia, 1993: 98), ‘boosterism’ (Swales, 2004), ‘boosting’ (Hyland, 2012), and ‘quasi advertising discourse’ (Lindeberg, 2004). Hyland (2001) points out that academic research articles have typically been viewed as a self-effacing and humble genre – for example, conventions require that authors mark claims as provisional and pending acceptance in the literature and by the community. However, it seems that conventions are changing and that the use of promotional rhetoric has increased. This ‘promotional strategy’ phenomenon was noticed by Swales and Najjar (1987) who found a growing frequency of statements of main research findings in the introductions of articles published in Physical Review. This foregrounding of newsworthy information was later reported in research papers from the biological sciences (Berkenkotter & Huckin, 1995) and computer science (Shehzad, 2010). More specifically, the lexis per se of promotional rhetoric in academic publications has been studied from different perspectives and mostly in the field of medicine. Fraser and Martin (2009: 1), for example, looked at the use of adjectives imposing subjective value (e.g. important, critical, original) on “an otherwise neutral knowledge claim” in fundamental and clinical research journals of different impact factors over a 20-year period (1985–2005). Their research suggests an increasing loss of objectivity in the presentation of scientific data. Vinkers at al. (2015), for their part, found an increase from 2% to 14% in the frequency of 25 individual positive words (e.g. crucial, novel, innovative, unprecedented) in PubMed abstracts published between 1974 and 2014. They suggest that scientists have come to “assume that results and their implications have to be exaggerated and overstated in order to get published” (Vinkers, Tijdink, & Otte, 2015: 3). Their data also suggest that such lexis is more frequent in papers written by non-native English-speaking scientists. The growing use of ‘drama’ words (e.g., adverbs, such as importantly, surprisingly, dramatically, strongly, markedly) in scientific research articles in general has also been evinced by Wheatley (2014, 2015), former Editor in Chief of Cell Biology International, who argued that science has become a theatrical business. The former editor-physician illustrates his arguments with the following “frequent and annoying phrase” in primary research articles: “We have revealed for the first time that xxxxx.” While this may be true, he argues, the sentence conjures up a moment of real drama, the authors hailing themselves as true pioneers by making prior claim, when the whole purpose of a primary research article is precisely to communicate new findings. Promotional elements that underline the worthiness, validity and/or significance of a research study have also been analyzed in the introductions of applied linguistics research articles (Wang & Yang, 2015). Overall, their study demonstrates that, in this field, claiming centrality is one of the main means to promote research. More narrowly, Martin and León Pérez (2014) examined the occurrence of such elements in the CARS section (Creating a Research Space, Swales, 2004) of Englishand Spanish-medium research articles from the health sciences and the humanities/social sciences. The findings of this crossdisciplinary and cross-linguistic study indicate that across the disciplines the English texts present a higher degree of rhetorical promotion than the Spanish ones. However, when comparing the two broad fields, the Spanish texts in the health sciences present a higher degree of promotion than the English (and Spanish) texts in the humanities/social sciences. These studies suggest that scholars today increasingly tend to emphasize the importance, uniqueness and novelty of their research. The researchers who have studied this particular linguistic issue point out that the hyping phenomenon reveals authors’ attitudes rather than the real significance of the reported research. Vinkers et al. (2015: 3) argue that the phenomenon impairs “the ability of science to find true effects and leads to an unnecessary focus on research marketability”, while Fraser and Martin (2009) suggest that promotional language might “bias” readers’ interpretation.

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

141

The present study assesses this phenomenon in medical research writing. Research articles in this field are typically categorized according the design of the study.(1) This study focuses on a specific type of research design called the Randomized Controlled Trial (RCT), a genre that specifically aims to minimize bias and maximize objectivity. RCTs are a genre common in medicine when testing treatments. They involve the blind and random assignment of subjects to either a group receiving the treatment under investigation or a group administered an alternative treatment or placebo. RCTs are generally regarded as a ‘gold standard’ in assessing treatments in medicine and can have a substantial influence on health care practice and policy (Cook, Guyatt, Laupacis, & Sackett, 1992). Reporting of RCTs has been shown to bias interpretation (Hahn, Williamson, & Hutton, 2002; Dwan et al., 2008), which in turn can make evidence unreliable for health care decision making. The CONSORT Statement is an initiative that seeks to minimize bias and ensure quality in the reporting of RCTs. It provides a set of guidelines explicitly detailing the content requirements for reports of RCTs (see, e.g. Moher et al., 2001). The guidelines, which have been widely adopted by journals, place constraints on format (e.g. use of an IMRD format and a structured abstract) and include a checklist of the information to be reported in each section (see Appendix 1 in ‘supplementray data’). Failure to include content items may result in a manuscript being deemed of inferior quality (see, for example, To, Jones, Emara, & Jadad, 2013). The ‘Uniform Requirements’ of the International Committee of Medical Journal Editors provide similar guidelines for medical research articles in general (the updated recommendations may be found at www.icmje.org). In addition to referring authors to external guidelines, journals often place their own restrictions on what is reported and, sometimes, how it should be reported. For example, it is not uncommon for journals to require that manuscripts avoid using the passive voice. Analysing a corpus of RCTs published in the ‘big five’ medical journals, Millar, Budgell, and Fuller (2012) assessed the impact of such guidelines on authors’ linguistic choices and argued that they do not necessarily improve reporting of research. With this exception, we are not aware of any studies on the discourse of RCTs. In sum, it seems that scientific writing, in general, and medical research articles, in particular, increasingly tend to use more promotional language, and, as argued by others (Fraser & Martin, 2009; Wheatley, 2014; Vinkers, et al., 2015), this phenomenon risks imposing judgments on readers that might undermine objective and disinterested evaluation of new knowledge. By providing quantitative and qualitative descriptions of how authors of RCTs use ‘hype’, we hope to contribute to the understanding of this phenomenon and the debate surrounding it, and might be approached in the EMP/ESP classroom. 3. Corpus and methods We define ‘hype’ as hyperbolic and/or subjective language that authors use to glamorize, promote, embellish and/or exaggerate aspects of their research. According to this definition, the following phrases discussed in Wheatly’s (2014) editorial would be interpreted as hype – Our results display new and exciting evidence of...; It will be enormously important to examine...; Importantly, we noted...; Interestingly, the evidence was... . Similarly, search terms used by Fraser and Martin (2009) and Vinkers et al. (2015) would, when directed at aspects of the authors’ research, also likely represent instances of hype – e.g. pivotal, crucial, essential, innovative, major, promising, groundbreaking, spectacular. This study was guided by the following research questions: RQ1. To what extent do authors of RCTs hype their research? RQ2. How are hypes distributed across the sections of RCTs? RQ3. What aspects of the research do hypes target? RQ4. How are hypes linguistically realized?

3.1. Data We compiled a corpus of 24 RCTs (75,927 tokens) published between 2010 and 2015 in the four highest impact journals in the field of orthopedic medicine and spine care (listed below; see Appendix 2 in ‘supplementary data’). RCTs (six from each journal) were selected based on comparability of topic. As all journals adhere to the CONSORT guidelines (http://www. consort-statement.org/), the RCTs were also comparable in terms of the type of information reported and the use of the IMRAD macrostructure (Introduction, Materials and Methods, Results and Discussion).





European Spine Journal (Impact Factor 2.132) Journal of Bone and Joint Surgery (Impact Factor 2.9) Spine (Impact Factor 2.3) Journal of Spinal Disorders and Techniques (Impact Factor 2.2)

The sub-disciplinary choice, orthopedic medicine and spine care, was motivated by the fact that one of the co-authors (BB) is a researcher and writer in the field (moreover, interested in linguistic issues). This enabled him to act as both a specialist

142

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

informant and a data analyst. His familiarity with the wider socio-cultural framework in which the texts were created helped elucidate doubts and resolve discrepancies in classifying the hyping words and expressions. The relatively small size of our corpus of RCTs limits the generalizability of our findings. However, as noted by Flowerdew (2005), working with small corpora, “the compiler-cum-analyst can act as a kind of mediating ethnographic specialist informant to shed light on the corpus data.” (Flowerdew, 2005: 329). Small corpora can also enable investigation of textlinguistics variables, i.e. in situations, such as in the present study, where entire texts must be taken into account so as to capture writers’ rhetorical strategies (Mauranen, 1993; Banks, 2005). 3.2. Analysis We chose to annotate the corpus manually, a procedure that is common in corpus-based social-semiotic studies. Although this limits the overall size of the data set compared with automated analyses, it reduces the chance of missing hard-to-discern hype linguistic items as well as features of realization that could be specific to individual texts (Kaltenbacher, 2006). An automated procedure based on a pre-established list of words/expressions, such as that used by Fraser and Martin (2009) and Vinkers et al. (2015), would risk under-identification of the phenomenon (i.e. some realizations of hype are not mentioned in the list). While automated searches were performed to check that all instances a given word had been identified, reliance on this approach alone would risk over-identification (i.e. in certain contexts, some items in the list do not actually represent hype). Thus, as a pilot exercise, the three authors (two linguists and a medical professional) independently read the same four RCTs and manually identified the hyping items in each rhetorical section, including the abstract. Identification of hype was essentially subjective in nature and involved judging whether a particular linguistic item could be removed or replaced with an objective or neutral alternative. Hyping items were then classified into broad ad hoc functional categories based on the targets of the hyping words and expressions – i.e. the aspect of the study that the words or expression serve to embellish. The results from these four RCTs were cross-checked and discussed, and eight target categories were adopted for further analysis of the whole corpus. When applied to the whole corpus, two categories were later conflated, resulting in seven target categories. Table 1 shows these seven target categories and examples, with the feature that we considered hyperbolic highlighted in bold –we refer to this as the node of the hype. The code in parentheses refers to the source of the extract (references listed as Supplementary Data), the rhetorical section where the hype occurred, and the functional target. Broad Research Area (BRA) comprises hypes targeted at the general field of study, while Specific Research Topic (SRT) relates to the particular topic under investigation. These two targets, thus, differ in their level of specificity. Hypes in these categories are often closely related with the CARS model – e.g. claiming centrality in (1) and outlining the purpose in (2); however, they are not necessarily restricted to the Introduction section. The third category, Authors’ Prior Research (APR) comprises hypes where the authors explicitly identify themselves as the source of related research – usually by use of the first-person pronoun, as in (3). Here we concur with Hyland (2001: 214) who argues that selfcitation and personal reference are not just stylistic features of research writing but “an important means of demonstrating one’s disciplinary credentials and credibility.” The fourth category, Research Methods (RM), comprises hypes targeted at how the study was conducted – e.g. methods, design and individuals. In extract (4), the description of the practice as ‘high-volume’ depicts the surgeon as particularly experienced in the procedure. This seems, to the authors, gratuitous, as in an RCT one would not recruit a practitioner who had a ‘low volume’ practice or lack of experience in the procedure of interest. The final three categories all relate to the product of the research. Research Outcome (RO) comprises hypes targeted at the results – in (5) the absence of differences between groups. Research Primacy (RP) comprises hypes that describe the research as superior or assign it priority, often in terms of novelty (e.g. 6). Finally,

Table 1 Hype target (functional) categories and examples. 1. Broad Research Area (BRA) (1) Accurate identification of the pain generators is critical for . (spine_1_intro_BRA) 2. Specific Research Topic (SRT) (2) Most importantly, in this study we wanted to examine prospectively the therapeutic effects of physiotherapy (esj_3_disc_SRT) 3. Authors’ Prior Research (APR) (3) We previously showed a thirteenfold reduction in HO formation following ... (jbjs_3_disc_APR) 4. Research Method (RM) (4) All arthroscopic procedures were performed by a single surgeon with a high-volume hip arthroscopy practice. (jbjs_3_meth_RM) 5. Research Outcome (RO) (5) Importantly, there were no differences between the treatment groups ... (Y2015_jsdt_28_301. RES) 6. Research Primacy (RP) (6) The current study provides a notable contribution to the literature as being the first to demonstrate that the ill-effects of BMP .... (spine_6_disc_RP) 7. Research Conclusion (RC) (7) We believe that meticulous hemostasis before closure and the use of absorbable gelatin sponge are essential for the appropriate management of these . (jsdt_4_disc_RC)

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

143

Research Conclusion (RC) comprises hypes that embellish the interpretation of results – e.g. the clinical implications in (7). The three analysts identified a total of 201 instances of potential hypes. Agreement on items identified as hype was marginally higher between the two linguists (NM and FSM ¼ 67%) than between the subject specialist and the linguists (BB and NM ¼ 55%, BB and FSM ¼ 59%). Discrepancies in the identification were dealt with on a case-by-case basis and resolved through several rounds of discussion resulting in 161 instances of hype. These were then categorised independently by each analyst for their functional target, and inter-rater reliability was calculated (Fleiss’ kappa, K ¼ 0.87), indicating good agreement among the raters. In addition to the functional target, the sentence containing the hype, the node of the hype (i.e. the feature that we considered hyperbolic) and the grammatical category of the node (e.g. adjective, adverb, etc.) were recorded. The category of multiple hype was also created: it refers to the juxtaposition of several hyping words (e.g., most importantly – extract 2 in Table 1) or to a sentence with several hyping words in it (Table 1, extract 7). Statistical analyses were carried out in the R statistical programing environment (R Core Team, 2018). 4. Results and discussion 4.1. Frequency and distribution of hypes (RQ1 and 2) We identified a total of 161 hypes corresponding to 6.7 occurrences per-article, or 2.0 occurrences per 1000 words (henceforth ptw). Figure 1A shows the normalized frequency of hypes per article. The degree to which authors employ hypes varies considerably. Six articles contained just over 55% (89) of all hypes while two contained no instances. Table 2 shows the frequency of hypes cross-tabulated by target and section. Figure 1A shows the frequency of hypes per section as a bar plot. Hypes occur most frequently in Discussions and Introductions (3.9 ptw and 2.8 ptw, respectively). As frequency alone does not illustrate how hypes are distributed across the sample, we also calculated the ‘dispersion’ of hypes – this is the percentage of article sections that contained a hype. Over three quarters of the 24 Discussion sections contained one or more hypes (disp. ¼ 79%), compared to only half of the Introductions (disp. ¼ 50%). Compared to Introductions, more Methods sections contain one or more hypes (disp. ¼ 58%), but at a lower frequency (1.3 ptw). Hypes are least frequent in Abstract and Results sections (1 ptw/disp. ¼ 20%, and 0.3 ptw/disp. ¼ 13%, respectively). The box plots(1) in 1B show the distribution of hypes in individual articles. Statistical analyses confirm the association of hypes with the Discussion and Introduction sections. A non-parametric one-way analysis of variance (Kruskal Wallis test using the PMCMR package in R - Pohlert, 2014) shows a significant difference in the frequency of hypes in RCTs across sections – c2(4, 24) ¼ 27.6, p < 0.001. Applying the Nemenyi-tests for multiple comparisons of mean rank sums (Tukey method) indicates that Discussions contain significantly more hypes than Abstracts (p < 0.05) and Results (p < 0.001), while Introductions contain significantly more hypes than Results (p < 0.05). All other comparisons were not significant. In part, the distribution of hypes appears to be consistent with the amount of rhetorical work we might expect each section to perform. Introductions serve to motivate and justify the authors’ work and usually involve (re-)establishing the significance of the broader field and actual research, while Discussion sections serve to situate the findings within the field and

Figure 1. (A) Distribution of hypes in corpus by section (normalized per corpus section); (B) Distribution of hypes in articles by section (normalized per article section).

144

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

Table 2 Frequency of hypes cross-tabulated by section and functional target. Target Section

1. BRA Broad Research Area

2. SRT Specific Research Topic

3. APR Authors’ Prior Research

4. RM Research Method

5. RO Research Outcome

6. RP Research Priority

7. RC Research Conclusion

Total

Abstract Introduction Methods Results Discussion Total

0.29 (2) 0.70 (5) – – 0.16 (4) 11

0.29 (2) 1.25 (9) – – 0.32 (8) 19

– 0.42 (3) – – 0.49 (12) 15

– 0.14 (1) 1.28 (31) – 1.26 (31) 63

0.44 – 0.04 0.33 1.09 36

– 0.28 (2) – – 0.45 (11) 13

– – – – 0.16 (4) 4

1.02 2.79 1.32 0.33 3.39 161

(3) (1) (5) (27)

(7) (20) (32) (5) (97)

Note: Frequency standardized per 1000 words. Raw frequency given in parentheses. Column totals given as raw frequency.

Figure 2. (A) Distribution of hypes by target and section; (B) Distribution of hypes in articles by target.

make claims that show the authors’ contribution to knowledge (Horton, 1995). Accordingly, hypes are associated with these argumentative, speculative, and hypothetical sections. Although Methods are the most expository, factual and least persuasive sections of research articles in general (Martínez, 2003; Samraj, 2016), our results show that authors consistently use hypes in this section – over half (58%) of Methods sections contain at least one hype. In medicine, as we said before, RCTs are widely viewed as the most rigorous study design (Grossman & Mackenzie, 2005) for the investigation of clinical effectiveness. We thus speculate that the distinct nature of RCTs places authors under pressure to demonstrate the value of their study in terms of the rigor of their methodology. At the same time, Abstracts, which are typically characterized as performing a promotional and persuasive function akin to that of Introductions (Samraj, 2016), contain very few hypes. The CONSORT guidelines, referred to above, view Abstracts as being of special significance (Hopewell et al., 2008). The CONSORT Statement provides a set of additional guidelines that specify a minimum of fifteen items of essential information that should be reported in abstracts. We

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

145

speculate that pressure to ensure that all items are reported within the given word limit restricts authors’ use of hyperbolic language. 4.2. Targets of hypes (RQ2) Figure 2A shows the distribution of hypes across the functional targets (and sections) – see also Table 2. Hypes targeting Research Methodology (RM) predominate (39.1%). Indeed, these are almost twice as frequent as hypes that target the rationale for the research: Broad Research Area (BRA) and Specific Research Topic (SRT) (combined 18.6%). Hypes that target Research Outcomes (RO) account for 22.4%. It seems likely that the prevalence of hypes targeted at methodology reflects the prestige assigned to RCTs (a methodology in itself) (Cook et al., 1992). The box plots in Figure 2B show how the hypes in individual articles are distributed across targets. A Kruskal Wallis test shows a significant difference in the frequency of hypes in articles across targets – c2(6, 24) ¼ 44.1, p < 0.001. Pairwise comparisons show that hypes targeting RM are significantly more frequent than those targeting BRA (p < 0.05), Authors’ Prior Research (APR), Research Primacy (RP) and Research Conclusion (RC) (in all these cases, p < 0.001). Hypes targeting RO are significantly more frequent than APR and RC (in both cases, p < 0.05). Figure 3 provides a breakdown of section by target (counts normalized across sections). The distribution of targets shown in this figure corresponds to the typical IMRAD ‘hourglass’ structure (Hill, Soppelsa, & West, 1982). That is, as we said before, the Introduction serves to position the research, and, as such, requires a wider range of rhetorical structures. Accordingly, authors in our sample employ a wider range of hypes. As the Methods section then narrows to focus on the description of the study and the findings being reported, the targets of the hypes tapers to become more focused. Although in terms of overall frequency the Methods sections contain fewer hypes, more Methods sections contain one or more hypes (i.e. dispersion). In the Results section hypes are all but absent. Finally, in the Discussion, as authors’ focus broadens to situate the research in the wider context, make claims and highlight strengths of the study, the use of hyperbolic language is at its highest level in terms of both the quantity and range of hypes – a bottom-heavy hourglass. In what follows we provide a discussion of the functional targets and their distribution over the sections (Figure 2A). Hypes targeted at the research methodology (RM), the most frequent target, are, on the whole, distributed evenly between the Methods and Discussion sections (31 instances in each and one occurrence in an Introduction). There are, however, differences in the focus of the hypes in the two sections. In the Methods, just under half (45%) have a human target (e.g. researcher(s), technician(s)) and serve to emphasize the expertise, qualifications, and/or experience of individuals (7–9). Items in this sub-category are limited to the Methods section and might be considered particularly hyperbolic. One would certainly hope that surgeons are trained, certified and, therefore, experts! Other aspects of the methodology targeted include the suitability (10), care taken (11) and accuracy of the chosen procedure (8 and 12). (7) Patients eligible for inclusion were those undergoing primary total hip arthroplasty performed by the same trained surgeon (E.S.) . (jbjs_6_meth_RM) (8) Assisted by 4 experienced nurse investigators (TMJ, LLD, AGM, and SMZ), a board-certified orthotist (CO) (RWT) measured the subjects to determine the correct collar size and ... (spine_4_meth_RM) (9) A primary diagnosis of degenerative or isthmic spondylolisthesis and/or LSS determined by expert spine surgeons . (esj_6_meth_RM)

Figure 3. Distribution of hypes by section and target.

146

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

(10) .we collected data using both Bazaz scale and hoarseness VAS to provide the most useful information. (jstd_1_meth_RM) (11) Immediately after the patients had given their consent, the physiatrists mailed the principal investigator (PI), who randomised the subjects [.] using a list of blinded treatment [.] to assure (sic) the concealment of the allocation. (esj_6_meth_RM) (12) For measuring pressure over curved surfaces, the system is highly accurate (2% error) when compared with resistive ink technology . (spine_3_meth_RM) In the Discussion section, hypes tend to be more explicit, often stating a specific aspect of the study design as a ‘strength’. For example, in the below extract (13), the authors highlight as strengths aspects of their study (underlined) which are arguably standard in RCTs. (13) The strengths of this study included a randomized design, rigorous study entry criteria, and use of validated outcome measures. (spine_1_disc_RM) In medical writing, this strategy (i.e. highlighting strengths) is not uncommon. Indeed, the BMJ guidelines recommend that authors devote two paragraphs to the ‘strengths and weaknesses’ of their research (see http://www.bmj.com/aboutbmj/resources-authors/article-types/research). Of the journals in our sample, only Spine requests that authors highlight ‘strengths’ – albeit indirectly through reference to external guidelines (Standards for Quality Improvement Reporting Excellence - http://www.squire-statement.org). Nevertheless, we find that of the explicit references to ‘strengths’, half occur in papers in this journal. Thus, the use of this strategy could, to some degree, be forced on the authors by standardized guidelines and journal policy. That is, authors’ perception of a need to highlight strengths may lead to the inclusion of unnecessary promotion. Not surprisingly, hypes that promote the product of the research (RO and RC), the next most frequent target, are also most prevalent in the Discussion section (respectively, 75% and 100% of all occurrences). Examples are provided in extracts (14–15). (14) Finally, it is important to reinforce the importance of target delivery of the injectate to the optimum site of pathology. (jstd_2_disc_RO) (15) Thus, it is not hard to see the vital importance of using fluoroscopy to overcome this ... (jsdt_2_disc_RO) The above extracts are instances of what we coded as ‘multiple hypes’ - the juxtaposition of several hyping items or a sentence with several hyping words in it. This subcategory was most strongly associated with Discussions sections – in total 37 were identified of which 27 occurred in the Discussion sections. The above extracts are illustrative of how greater levels of promotional intensity are found in Discussion sections. As Horton (1995: 985), former editor of The LANCET, put it: “The discussion is the section of a paper that most obviously seeks to cajole and convince. Hypes targeting the rationale of the study (BRA and SRT) are more frequent in the Abstract, Introduction and Discussion. Both types of hypes most often seek to establish importance – the adjective important (and derived forms) occur in 42% of all instances. In this regard, journals may explicitly ask reviewers to judge the importance of a manuscript e.g. Spine Journal asks reviewers to judge “whether it is novel and interesting, whether it has a sufficient impact and adds to the knowledge base.” The European Spine Journal asks referees to rank the novelty of a study and whether the area of the study is worthy of investigation. The following are examples of hypes targeting the importance of BRA and SRT: (16) Another key issue to be recognized is the importance of patients maintaining improved health outcome over time, . (spine_2_disc_BRA) (17) Although the clinical effects of a new intervention may seem advantageous off hand, an economic evaluation of the intervention is essential in order for decision makers to .... (spine_1_intro_SRT) Hypes targeted at authors’ previous research (APR) are limited to the Introduction and Discussion. These are instances of self-citation where the authors explicitly identify the cited work as their own by use of a personal pronoun or similar phrase (e.g. the authors) - see extract (18) (and also 3 in Table 1). While some journals recommend using the first person, none of our target journals do so; reference to self is thus a choice the authors make. In (18), it would have been equally acceptable from the journal’s point of view to refer to ‘an original study’. (18) Since the completion of our original study in 2005, which compared 4 standard collars, and the subsequent implementation of our findings into practice, our institution’s incidence ... (jbjs_3_disc_APR) The use of the first-person pronoun as a rhetorical strategy has been widely discussed in the literature. Hyland (2001: 223), for instance, argues that self-reference of this sort can function as “an expression of the participants’ custody and personal ownership of what they report”. Although we cannot say whether it was the authors’ intention, we suggest that the use of

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

147

first-person reference, in addition to conveying ownership, portrays the authors as experts in this field, and is, therefore, promotional in nature. In our sample, self-reference of this sort is relatively infrequent – we recorded fifteen instances only, and these were limited to three articles (Figure 2B). Indeed, two thirds of these (ten instances) were found in a single article (spine_4), which, with 25 instances, is the most hyped RCT in our sample. Although we did not consider ‘anonymous’ self-citation as a form of hype, we did conduct post hoc analyses which showed no statistical correlation between the number of anonymous selfcitations and the use of hyperbolic language. In total 13 hypes target research primacy (RP). With the exception of one instance recorded in an Introduction, these all occur in Discussion sections and are mainly realized through the use of adjectives. About half of these focus on what is new, novel, unique and innovative about the technique, procedure or design of the study (19, also 18 above). (19) Our present study is unique in that we studied TIP in addition to CROM restrictiveness of cervical collars. (spine_4_disc_RP) (20) The current study provides a notable contribution to the literature as being the first to demonstrate that ... (spine_6_disc_RP) Hypes also focus on what has been learnt from the research being reported and on how it is the first to reveal this new contribution to knowledge (20). Guidelines on ‘Preparing for Submission’ published by the ICMJE on this topic are somewhat confusing. Indeed, in the Discussion sections, authors are encouraged to “emphasize the new and important aspects of your study”, while the same guidelines also urge authors to “avoid claiming priority”. 4.3. Linguistic realization of hypes (RQ4) We also assessed how hypes are linguistically realized. In many cases, the node of the hype was identifiable as a single word (e.g. critical, important), while others were realized by a longer phrase (e.g. It should be emphasized that .). Table 3 provides a breakdown of the grammatical forms by which hypes are realized, and the word clouds in Figure 4 summarize the lexis present in these forms. Adjectives, the word class prototypically associated with evaluation (Hunston, 2010), are the most frequent form by which hypes are realized (40%), followed by adverbs (23%). Although adjectives and adverbs are used to hype all targets (except for APR), they are most strongly associated with RM, predominantly focusing on technical expertise (e.g. experienced, senior, specialized), rigor and care taken (e.g. precise, detailed, exhaustive), and general positive evaluation often used to emphasize the suitability of a chosen approach (e.g. attractive, appropriate, excellent). Items in Figure 4 exhibit overlap with linguistic taxonomies of stance and engagement (Hyland, 2005; Martin & White, 2005; Martin & Rose, 2007, inter alia) (3). Attitude markers are prominent – e.g. adjectives conveying authors’ attitudes of importance, such as important, significant (in the non-statistical sense), essential, critical. Boosters are also prevalent – e.g. very, highly, greatly, entirely, obviously. Although most frequently realized as adverbs, boosters also take the form of longer phrases – e.g. it is not hard to see, and without question. Cognitive directives – statements instructing the reader to interpret the results or the argument in a certain way are realized by longer phrases; examples include A key issue to recognize ., It is noteworthy that ., and It should be emphasized that.... Hyland’s (2005) analysis of a corpus of multidisciplinary RAs reports these three features (attitude markers, boosters and directives) as occurring at a combined rate of 14.1 items ptw. Although hypes in our sample of RCTs are less frequent (6.7 ptw), our analyses are restricted to hyperbolic language that targets the authors’ research; as a concept, our definition of ‘hype’ is, therefore, far narrower than stance and engagement. The results also overlap with the two studies assessing promotional language in medical research. Vinkers et al.’s (2015), for instance, found that over the past 40 years there has been an increase in the use of overtly positive adjectives in scientific abstracts. The three adjectives that show the largest gains, and, as such, might be considered prototypical of increased promotion, feature in our data as well (robust, novel and innovative). Similarly, Fraser and Martin (2009) found an increase in the use of “value-laden words” to modify knowledge claims. Of their 21 search terms (adjectives and nouns), 11 were also found in our data - central, crucial, critical, vital, essential, first, important, innovative, key, major, unique. Both studies suggest that the reporting of results and their implications has become exaggerated and overstated. Through the manual analysis of

Table 3 Grammatical realisation of hypes. Hype node

Example

Frequency










critical; experienced; novel particularly; markedly; obviously importance; robustness; strength our; we (referring to prior research) it should be emphasised; it is not hard to see strengthen; maximize; highlight all; much; few

73 41 16 15 13 8 3

adjective adverb noun possessive adj./personal pron. phrase verb determiner

148

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

Figure 4. Lexical realization of hypes (size and shading proportional to frequency). Note that, as Figure 4 contains items occurring both as single word hypes (e.g. important) and as part of a phrase (e.g. It is important to emphasise that .), the totals differ from those displayed in Table 3.

complete texts, the present study indicates that promotional language, in the case of RCTs at least, extends beyond the reporting of results and their implications, and that it is actually more prevalent in the reporting of methodology. 4.3.1. Pedagogical implications Randomised controlled trials (RCTs) are generally regarded as the highest level of experimental evidence by which to judge the clinical value of a medical intervention (Wallace, 2010; “Definition of Levels of Evidence (LoE) & Overall Strength of Evidence (SoE)”, 2014). Consequently, they are likely to strongly influence clinical guidelines and the practices of individual clinicians (see, for example, Nuti et al., 2018). Given the tangible impact of RCTs on human health, research findings and their clinical applications should be conveyed with fidelity to stakeholders including policy makers, health administrators and practitioners. However, dishonesty (Noble, 2017) and exaggeration (Hesterman, Szperka, & Turner, 2018) in the reporting of biomedical research are not uncommon. For example, a recent analysis of RCTs in psychiatry found that one third of articles claimed levels of effectiveness which were not justified by their own statistical analyses (Sugunama et al., 2017). To the extent that hype can modulate fidelity and so, in the long run, modulate clinical practice, EMP students and ESP/ EMP teachers and researchers should be aware of hype in reports of RCTs. Models taught in ESP typically seek to reflect the linguistic and rhetorical features of the target domain texts, and, in doing so, reinforce conformity. However, in the case of hype, encouraging students to mimic what may, by some, be viewed as conventional practice may not be in the interests of stakeholders. At the same time, telling students that hype should be avoided fails to recognize that it is a grey area and may, at times, serve useful functions - for example, by enlivening a text or drawing attention to salient features. Teachers are, therefore, likely to struggle in deciding how best to deal with the feature. As Hyland (2013: 108) puts it: The question, essentially, is whether ESP is a pragmatic exercise, working to help students to fit unquestioningly into subordinate roles in their professions, disciplines and courses, or whether it has a responsibility to help students understand the power relations of those contexts. We would suggest that teachers might consider adopting a critical pedagogical approach in order to help students unpack possible ethical issues surrounding hype in research writing. For example, to contextualise the topic, students might first be introduced to ethical issues in the reporting of research, such as the increases in plagiarism, data fabrication and salami-slicing (see Introduction), leading to a discussion of possible contributing factors - e.g. pressure to publish, increasing academic competition. The frequent use of hypes in the introductions, methods and discussion sections of RCTs put forward in this research could then form the basis of awareness-raising exercises designed so as to draw students’ attention to their presence. Sentences that exemplify different types of hype could be presented, and students could be asked whether and/or how they

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

149

serve a promotional function. Students could be provided with hyped and unhyped versions of the same texts and asked which version they think is most convincing, and why. This could give rise to critical discussions concerning the use of hype. For example, speculation on author motivation for using a hype and discussion of where and how hype might encroach upon the fidelity of the reporting. In addition, students could be asked to produce their own academic prose with a conscious attempt to strengthen their views and claims. A ‘peer-review workshop’ could then be conducted to offer feedback on one another’s work, and this could be followed by teacher-student conferencing to tackle the issue on an individual basis. Such activities could help students develop a critical awareness of hype, both as producers and consumers of medical research texts. 4.3.2. Limitations and future research Although the present study documents the use of hype in reports of RCTs, explanations as to why authors use hype (e.g. the influence of content and style guidelines, editorial practices, professional competition, formal/informal writing instruction) remain speculative. Furthermore, although the use of hype has been interpreted as a feature of writing that might influence readers’ interpretations, there is no evidence that hyping achieves this objective. Future research will therefore draw on qualitative/ethnographic data from producers and consumers of medical research papers to explore why authors of medical research articles use hype and how it influences the reader. Answers to these questions can help inform editorial and review processes and have implications for how this phenomenon should be treated in formal reading and writing instruction. 5. Conclusion The present study has demonstrated how authors of RCTs use hyperbolic and/or subjective language that glamorizes, promotes and/or exaggerates different aspects of their research – a phenomenon we have referred to as ‘hype’. To the extent that they enliven text, engage readers, and draw attention to the more salient features of a study, hypes may perform a useful role. However, when they erode the fidelity of conveying information, they perform a disservice to science. Where any particular instance of hype falls on this scale is a matter of judgment. Furthermore, since most (peer-reviewed) papers in our study demonstrated the phenomenon, we might assume that to some degree it is tolerable to authors, editors and publishers. Indeed, a number of guidelines seem to encourage practices which we have identified as hyping. We referred above to both the CONSORT guidelines (Moher et al., 2001), which are specific to RCTs, and to the guidelines of the International Committee of Medical Journal Editors (www.icmje.org), which encourage authors to “emphasize the new and important aspects of their [sic] study” while concurrently avoiding claiming priority. Given the somewhat conflicted advice of guidelines, we suggest that hyping may be seen as something of a gray area in medical writing and publishing, where the right amount is right. We have suggested that some of the pressures on authors to hype derive from the competitive nature of research funding and publishing (Nature Editorial, 2015). A defining feature of credible journals is that they do not accept all submitted manuscripts, and so an author is competing against others in a less than transparent marketplace for exposure which then impacts academic rank, tenure and so forth (see, for example, Crawford & Roche-Nagle, 2017; Hing, Higgs, Hooper, Donell, & Song, 2011). In this sort of environment, what author would not strive to put their best foot forward? What remains unclear, however, is how much and what type of hyping achieve the authors’ aims. At what point does hyping cease to convince and begin to disaffect readers, for example, those who may make decisions about publication, funding and/or academic rank? In part, the answer to this question may be parsed into lexical, functional and rhetorical domains. In our study, authors seemed to gravitate towards certain word families (important, importance, importantly; strength, strengthen). In the instance of RCTs, language choice appears to often be directed towards convincing the reader of the strength of a study’s methodology and the implications of research findings. This is logical since the defining feature of the RCT lies in the methods used, and its position as a gold standard derives from them (Cook et al., 1992). Hypes serve to portray the study as rigorous, and, thus, persuade the reader that the foundations for interpreting their evidence are empirical. However, we have also argued that hypes targeted at aspects of the methodology are often gratuitous, – e.g. unnecessarily highlighting researchers’ skills, qualifications and experience, and statements that present standard features of RCTs as ‘strengths’. RCTs’ discussion sections were found to be the most heavily hyped sections, and we have argued that this derives from the conventional use of the discussion section to advance the external validity, generalizability or implications of a study. This is because, ultimately, the purpose of an RCT is to inform and optimize clinical behaviors, particularly to influence how medical treatments are applied. Other aspects authors choose to hype include the importance of the research topic, the novelty, and strength of the study and their own prior work. Hence, authors, editors, reviewers and readers in general need to judge both the intent of the language of a manuscript, and whether such language conveys with fidelity the clinical implications of the actual investigation. Further, we need to consider whether, in any particular manuscript, hyping strikes an appropriate balance between what may be the incongruent interests of authors and consumers of the medical literature. Notes 1. There are many specific designs each directed towards answering a different type of question, and each with generally accepted content requirements. As a measure of the diversity of research designs, the Equator Network (http://www.

150

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

equator-network.org/reporting-guidelines/accessed on 21 August 21 2018), which acts as a clearinghouse for content guidelines, makes available 404 different sets of standardized guidelines stipulating, among other things, the required sections and information to be included in each. Discourse conventions are, thus, to a large part, dictated by standardized guidelines and writing is characterized by a high degree of formulaicity (Millar et al., 2012). 2. Box plots provide a graphical summary of the distribution of data. The heavy solid line shows the median and the box drawn around that line shows the data that falls within the inter-quartile range (i.e. the middle 50% of the data). The skew of the data can be inferred from the orientation of the median relative to the quartiles. The lines extending from the box provide a cut off value for the identification of potential outliers, which are marked individually by the circles on the plot. The arbitrary cut-off value is 1.5 times the inter quartile spread in length and is measured from the median. 3. Stance is an attitudinal dimension through which the textual voice expresses its “judgments, opinions and commitments while engagement is an alignment dimension where writers acknowledge and connect to others.” (Hyland, 2005: 176). Hyland further points out that “even the most rhetorically innocent sections reveal writers’ efforts to persuade their audience of their claims, so that stance and engagement are likely to figure, in different ways, across the research paper.” (Hyland, 2005: 190). Supplementary data Supplementary data to this article can be found online at https://doi.org/10.1016/j.esp.2019.02.004.

References Bailey, J. (2013). Defending against plagiarism. The scientist magazine. http://www.the-scientist.com/?articles.view/articleNo/35677/title/Defending-AgainstPlagiarism/. (Accessed 7 March 2017). Banks, D. (2005). The case of Perrin and Thomson: An example of the use of a mini-corpus. English for Specific Purposes, 24(2), 201-211. Berkenkotter, C., & Huckin, T. N. (1995). Genre knowledge in disciplinary communication: Cognition/Culture/Power. Hillsdale. Bhatia, V. K. (1993). Analyzing genre: Language use in professional settings. Longman. Brauser, D. (2014). Pressure to publish leads to scientific misconduct. Medscape. http://www.medscape.com/viewarticle/834250. (Accessed 7 March 2017) Cook, D. J., Guyatt, G. H., Laupacis, A., & Sackett, D. L. (1992). Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest, 102(4), 305-311. Crawford, S. A., & Roche-Nagle, G. (2017). Publication outcomes for research presented at a Canadian surgical conference. Canadian Journal of Surgery, 60(2), 108. Cronin, B. (2013). Slow food for thought – Editorial. Journal of the American Society for Information Science and Technology, 64(1), 1-1. Definition of Levels of Evidence (LoE), & Overall Strength of Evidence (SoE). (2014). Evidence-Based Spine-Care Journal, 5(2), 166. https://doi.org/10.1055/s0034-1394106. Dodds, P. S., Clark, E. M., Desu, S., Frank, M. R., Reagan, A. J., Williams, J. R., & ...Megerdoomian, K. (2015). Human language reveals a universal positivity bias. Proceedings of the National Academy of Sciences, 112(8), 2389-2394. Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., & Ghersi, D. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One, 3(8), e3081. Ecklund, E. H., Johnson, D. R., & Matthews, K. R. W. (2015). Commentary: Study highlights ethical ambiguity in physics. Physics Today, 68(6), 8-10. Fairclough, N. (1993). Critical discourse analysis and the marketisation of public discourse: The universities. Discourse & Society, 4(2), 133-168. Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), e5738. Flowerdew, L. (2005). An integration of corpus-based and genre-based approaches to text analysis in EAP/ESP: Countering criticisms against corpus-based methodologies. English for Specific Purposes, 24(3), 321-332. Fraser, A. G., & Dunstan, F. D. (2010). On the impossibility of being an expert. British Medical Journal, 341, c6815. Fraser, V. J., & Martin, J. G. (2009). Marketing data: Has the rise of impact factor led to the fall of objective language in the scientific literature? Respiratory Research, 10(35). https://doi.org/10.1186/1465-9921-10-3. Grieneisen, M. L., & Zhang, M. (2012). A comprehensive survey of retracted articles from the scholarly literature. PLoS One, 7(19), e44118. Grossman, J., & Mackenzie, F. J. (2005). The randomized controlled trial: Gold standard, or merely standard? Perspectives in Biology and Medicine, 48(4), 516534. Habibzadeh, F. (2014). Plagiarism: What does the future hold for science writing? European Science Editing, 40(4), 91-93. Hahn, S., Williamson, P. R., & Hutton, J. L. (2002). Investigation of within-study selective reporting in clinical research: Follow-up of applications submitted to a local research ethics committee. Journal of Evaluation in Clinical Practice, 8(3), 353-359. Haug, C. (2015). Peer-review fraud - hacking the scientific publication process. New England Journal of Medicine, 373(25), 2393-2395. Hesterman, C. M., Szperka, C. L., & Turner, D. P. (2018). Reasons for manuscript rejection after peer review from the journal headache. Headache, 58(10), 1511-1518. Hill, S. S., Soppelsa, B. F., & West, G. K. (1982). Teaching ESL students to read and write experimental research papers. Tesol Quarterly, 16(3), 333-347. Hing, C. B., Higgs, D., Hooper, L., Donell, S. T., & Song, F. (2011). A survey of orthopaedic journal editors determining the criteria of manuscript selection for publication. Journal of Orthopaedic Surgery and Research, 6(1), 19. Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., Schulz, K. F., &, CONSORT Group. (2008). CONSORT for reporting randomized controlled trials in journal and conference abstracts: Explanation and elaboration. PLoS Medicine, 5(1), e20. Horton, R. (1995). The rhetoric of research. British Medical Journal, 310, 985-987. Hunston, S. (2010). Corpus approaches to evaluation: Phraseology and evaluative language. Routledge. Hyland, K. (2001). Humble servants of the discipline? Self-mention in research articles. English for Specific Purposes, 20(3), 207-226. Hyland, K. (2005). Stance and engagement: A model of interaction in academic discourse. Discourse Studies, 7(2), 173-192. Hyland, K. (2012). “The past is the future with the lights on”: Reflections on AELFE’s 20th birthday. Iberica, 24, 29-43. Hyland, K. (2013). ESP and writing. In B. Paltridgey, & S. Starfield (Eds.), The handbook of English for specific purposes (pp. 95-112). London: Wiley-Blackwell. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. Kaltenbacher, M. (2006). Culture related linguistic differences in tourist websites: the emotive and the factual. A corpus analysis within the framework of Appraisal (pp. 269-292). Equinox Publishing. Lillis, T., & Curry, M. J. (2013). English, scientific publishing and participation in the global knowledge economy. In E. J. Erling, & P. Seargeant (Eds.), English and development: Policy, pedagogy and globalization. Multilingual Matters. Lindeberg, A. C. (2004). Promotion and politeness: Conflicting scholarly rhetoric in three disciplines. Abo Akademi University Press.

N. Millar et al. / English for Specific Purposes 54 (2019) 139–151

151

Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P., & ...Glasziou, P. (2014). Biomedical research: Increasing value, reducing waste. Lancet, 383, 101-104. Martínez, I. (2003). Aspects of theme in the method and discussion sections of biology journal articles in English. Journal of English for Academic Purposes, 2(2), 303-323. Martin, P., & León Pérez, I. K. (2014). Convincing peers of the value of one’s research: A genre analysis of rhetorical promotion in academic texts. English for Specific Purposes, 34, 1-13. Martin, J. R., & Rose, D. (2007). Working with discourse: Meaning beyond the clause. Open linguistics series. Continuum. Martinson, B. C., Anderson, S., & de Vries, R. (2005). Scientists behave badly. Nature, 435(7043), 737-738. Martin, J. R., & White, P. R. R. (2005). The language of evaluation: Appraisal in English. Palgrave Macmillan. Mauranen, A. (1993). Contrastive ESP rhetoric: Metatext in Finnish-English economics texts. English for specific Purposes, 12(1), 3-22. Millar, N., Budgell, B., & Fuller, K. (2012). ‘Use the active voice whenever possible’: The impact of style guidelines in medical journals. Applied Linguistics, 34(4), 393-414. Moher, D., Schulz, K. F., & Altman, D. G. (2001). The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology, 1(1), 2. Nature Editorial. (2015). Publish or perish. Nature, 521, 259. Noble, J. H. (2017). Truth in research labelling. Indian Journal of Medical Ethics, 2(3), 194-199. Nuti, R., Brandi, M. L., Checchia, G., Di Munno, O., Dominguez, L., Falaschi, P., & ...Isaia, G. C. (2018). Guidelines for the management of osteoporosis and fragility fractures. Internal and Emergency Medicine. https://doi.org/10.1007/s11739-018-1874-2. Ochodo, E. A., de Haan, M. C., Reitsma, J. B., Hooft, L., Bossuyt, P. M., & Leeflang, M. M. (2013). Overinterpretation and misreporting of diagnostic accuracy studies: Evidence of “spin”. Radiology, 267(2), 581-588. Pohlert, T. (2014). The pairwise multiple comparison of mean ranks package (PMCMR). R package. http://CRAN.R-project.org/package¼PMCMR. R Core Team. (2018). R: A Language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. http://www.R-project. org/ Raghav, K. P., Mahajan, S., Yao, J. C., Hobbs, H. B., Berry, D. A., & Pentz, R. D. (2015). From protocols to publications: A study in selective reporting of outcomes in randomized trials in oncology. Journal of Clinical Oncology, 33(31), 3583-3590. Salager-Meyer, F. (2014). Writing and publishing in peripheral scholarly journals: How to enhance the global influence of multilingual scholars? Journal of English for Academic Purposes, 13, 78-82. Samraj, B. (2016). Research articles. In K. Hyland, & P. Shaw (Eds.), The routledge handbook of English for academic purposes (pp. 403-415). Routledge. Shehzad, W. (2010). Announcement of the principal findings and value addition in computer science research papers. Iberica, 19, 97-119. Steen, R. G., Casadevall, A., & Fang, F. C. (2013). Why Has the Number of Scientific Retractions Increased? PLoS One, 8(7), e68397. Suganuma, A. M., Imai, H., Takeshima, N., Hayasaka, Y., & Furukawa, T. A. (2017). Overstatements in abstract conclusions claiming effectiveness of interventions in psychiatry: A meta-epidemiological investigation’. PLoS One, 12(9), e0184786. Swales, J. (2004). Research genres: Explorations and applications. Cambridge University Press. Swales, J. M., & Najjar, H. (1987). The writing of research article introductions. Written Communication, 2(4), 175-192. To, M. J., Jones, J., Emara, M., & Jadad, A. R. (2013). Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals. PLoS One, 8(12), e84779. Ulrich’s Periodicals Directory. (2017). http://ulrichsweb.serialssolutions.com. Accessed 8 January 2017. Vinkers, C. H., Tijdink, J. K., & Otte, W. M. (2015). Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: Retrospective analysis. British Medical Journal, 351, h6467. Wallace, D. K. (2010). Evidence-based medicine and levels of evidence. American Orthoptic Journal, 60(1), 1-5. Wang, W., & Yang, C. (2015). Claiming centrality as promotion in applied linguistics research article introductions. Journal of English for Academic Purposes, 20, 162-175. Wheatley, D. (2014). Drama in research papers. European Science Editing, 40(1), 14-16. Wheatley, D. (2015). Out with the old, in with the new: Words and phrases in fashion. European Science Editing, 41(2), 43-44. Neil Millar’s research centres around the practical applications of corpus-based methodologies. In addition to biomedical discourse, he has published on topics including language change, psycholinguistics, language teaching and evaluative language. He is an associated professor at the University of Tsukuba, Japan and holds a PhD from Lancaster University. Françoise Salager-Meyer has published widely on written medical discourse. In 1994 and 2004, she was awarded the Horowitz Prize for her works on the pragmatics of written scholarly communication. She was the section editor of the “Language and Medicine section” of the Encyclopaedia of Language and Linguistics (Elsevier) and is currently coordinating the Research Group on Medical Discourse Analysis (University of The Andes, Venezuela). Brian Budgell is Director of the Laboratory of Neurophysiology at Canadian Memorial Chiropractic College. His research interests include regulation of spinal cord blood flow, and biomedical linguistics. He has published on a range of linguistic topics including grammatical voice, pedagogic word lists and assessment. He the author of the Springer guide to Writing a Biomedical Research Paper.