What is a research article?: Genre variability and data selection in genre research

Journal of English for Academic Purposes 29 (2017) 1e11

Contents lists available at ScienceDirect

Journal of English for Academic Purposes journal homepage: www.elsevier.com/locate/jeap

What is a research article?: Genre variability and data selection in genre research Anneke van Enk a, *, Kate Power b a Centre for Health Education Scholarship, University of British Columbia, 429 e 2194 Health Sciences Mall, Vancouver, BC V6T 1Z3, Canada b Arts Studies in Research and Writing, University of British Columbia, #412-1873 East Mall, Vancouver, BC B6T 1Z1, Canada

a r t i c l e i n f o

a b s t r a c t

Article history: Received 14 February 2017 Accepted 13 July 2017

The written forms of the academic community, particularly the research article, are a frequent focus of genre research. Yet the criteria used to select “research articles” from among the different text types published in scholarly journals are not always made apparent. We argue that an elastic yet operational set of criteria for identifying the “research article” is both necessary and possible, and we offer a summary of our own process for developing such criteria in a project focused on the theory-practice tension in academic research in education. While genre theory's interest in variability may make researchers wary of setting boundaries, defining a prestige knowledge-making genre like the “research article” is not just methodologically but also politically significant. In unpacking tacit assumptions about what we select as data we become more aware not just of sampling biases but of which forms of knowledge we legitimate and exclude. © 2017 Elsevier Ltd. All rights reserved.

Keywords: Knowledge-making genres Genre definition Research and practice Academic writing Genre-based research methodology

Genre theory explores the recurrent discursive forms through which communities constitute themselves. The written forms of the academic community, particularly the research article, are a frequent focus for genre theorists. Yet, for all that is known about the typical linguistic features of research articlesdand there is a wealth of knowledge, particularly in EAP/ESP, where Swales' (1990) work has been foundationaldthe criteria researchers have used to select “research articles” from among the different text types published in scholarly journals are not always made apparent. Of course, like all genres, the research article is only a “relatively stable” entity (Bakhtin, 1986, p. 60), not a fixed form. This contention is amply borne out in empirical investigations; the conventions that shape and constrain the research article vary across disciplines (notable here is extensive work by Hyland (2004)) as well as over time (book length studies include Bazerman (1988) and Gross, Harmon & Reidy (2002)). Precisely because of this variability, data sampling for genre research requires a degree of explicitness and at least temporary fixity. While in everyday usage most genres are readily recognizable despite variations in their form and content, categorizing scholarly genres for research purposes demands more than commonsense knowledge. In this paper, we propose that an elastic yet operational, “stable-enough-for-now” (Schryer, 1993, p. 229) set of criteria for identifying the “research article” is both necessary and possible in genre research. We offer a summary of our own process for developing such criteria in a project focused on the theory-practice tension in academic research in education. The criteria themselves are not intended as definitive, of course, but we believe they offer a sound and useful starting point for other inquiries. We also discuss methodological and political implications of our work for studies of knowledge-making genres. In particular, we stress the importance of thinking through and laying out explicitly the parameters of genres under study. This is

* Corresponding author. E-mail addresses: [email protected] (A. van Enk), [email protected] (K. Power). http://dx.doi.org/10.1016/j.jeap.2017.07.002 1475-1585/© 2017 Elsevier Ltd. All rights reserved.

2

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

true even of genres like the research article, with which we assume our readers will be very familiar. In fact, we argue it is particularly true of such “familiar” genres, because defining a knowledge-making genre is not just a methodological practice but also a political one. In unpacking tacit assumptions about what we select as data in our studies we become more aware not just of sampling biases but of which forms of knowledge we frame as legitimate and which forms we exclude. 1. Genre sampling in writing research Discussion of sampling is relatively limited in the genre and writing studies literature. Possibly, this is attributable to Bhatia's notion of “generic integrity” (2001, p. 88). If, as he claims, “the most important aspect of a genre is that it is recognizable, sufficiently standardized and is based on a set of mutually accessible conventions” (2001, p. 88), then it may be that the identification of particular genres is simply taken for granted. As we have already noted above, this is, of course, how genre operates in the everydaydmore or less by tacit recognition. Yet, Bazerman and Prior (2004) is one of few scholars to point out that, however tempting this commonsense foundation for identifying genres may be, it is methodologically problematic to work on this basis alone. He offers a set of useful approaches to “go beyond the cataloging of features of genres that we already know” (p. 324), but he does not offer advice on working with boundaries, that is, on how to determine whether or not a given text represents the particular genre being studied. In a section headed “What Is a Genre and How Do You Know One?” he acknowledges that “there is no magic equation to determine what gives you adequate evidence of a genre” (p. 327). Yet, his suggestion to collect samples to “the point of diminishing returns, plus a couple more,” (p. 327) while sound in principle, may not always be workable and still leaves the problem of genre definition unaddressed. Peck MacDonald (2002) suggests that researchers should “read or skim broadly in trying to develop a sense of what is representative” (p. 116). Here again, however, the advice to work inductively is sensible but ultimately not enough to be useful in setting parameters, particularly when accompanied by the broad admonition to avoid “atypical articles” (p. 117). (What constitutes typical and atypical articles? And how might one recognize exemplars of either category?) Bhatia (2004) advises ensuring “that one's criteria for deciding whether a text belongs to a specific genre are clearly stated” (p. 164). He adds, helpfully but without much elaboration, that a genre's definition may be based on its communicative purposes, its situational context, distinctive textual characteristics, or a combination of these. In the absence of ready direction from more theoretical discussions of genre sampling in writing studies, we looked at other sources that might provide guidance. We searched for existing corpora of research articles and found The Corpus of Research Articles (CRA), maintained by the Research Centre for Professional Communication at Hong Kong Polytechnic University. The CRA is sizabled5,609,407-words, with articles from high impact factor-journals in 39 disciplinesdbut we found no documentation indicating how actual research articles were selected. We also looked for selection criteria in empirical studies of the research article genre. Here, we found a predominance of studies about specific elements of the research article rather than holistic analyses of the research article per se. We also found that investigationseof both the research article and its elementseseem to operate with a common-sense notion of the genre, even as they look for contextual variation. The research article genre is rarely defined, even in fields where it is far less conventionalized than in the “hard” sciences. One notable exception is Gray's (2015) study of linguistic variation within the research article, in which the author draws on a taxonomy of over-arching article types (empirical, theoretical and evaluative) and sub-types to build a corpus of articles across six disciplines and three registers. While her study differs in aim from ours, it was useful to us in modeling an inductive and systematic approach to rendering selection criteria for articles explicit. In sum, just as we had initially expected to do in our own study, most authors working with a corpus of research articles tend to describe how they selected particular journals but not how they selected research articles. Most frequently, the selection of research articles is described as “random.” Occasionally, methods sections specify criteria for choosing from among research articles. For instance, selection is sometimes restricted to reports of certain types of studies (for example, Sheldon, 2011) or single-authored papers (for example, Molino, 2010). However, few studies set out criteria for distinguishing research articles from other types of content, and those that do commonly employ the IMRD framework as an identifier (for example, Valipouri & Nassaji, 2013; Martinez, Beck, & Panza, 2009). Yet, in our project, the IMRD structure presented challenges as a basis for data selection. Before we move on to discuss this challenge (as well as the complications involved in our attempt to draw on editorial headings and placement as a guide for selection) we describe the project itself and our process for selecting journals. 2. The larger project: “Scholarly and practical orientations in education research articles: a genre-based study” This paper reports on sampling issues that arose in a larger project examining how scholars' differing levels of commitment to theory and practice are realized in the genre of the research article.1 More specifically, the project looks at research articles from three contrasting subfields of education, namely health professions education (HPE), early childhood education (ECE), and philosophy of education (P of E). The relationship between research for understanding and research for application has been the subject of longstanding discussion, but its discursive aspects have received little attention. Given that the research article is both one of the primary outputs of research activity (Knorr-Cetina, 1999; Swales, 1990) and a major focus in writing studies, we sought to understand better the part this genre plays in constituting the theory-practice relationship. As an applied field with a large number of contrasting subfields, education makes a good site for this exploration of the research article. Indeed, the

1

This project was generously funded by a Research Initiative award from the Conference on College Composition and Communication.

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

3

relationship between theory and practice is a central tension in the field, with a number of commentators, including Labaree (1998, 2004) and Tierney and Holley (2008), attributing the relatively low status of education research to the field's divided loyalties between theory and practice. Our sampling strategy for this study involved the selection of six journals from each of our three education sub-fields on the grounds that the journals represented different prioritizations of theory and practice. Research articles would then be selected from each journal to form a corpus for comparative analysis, focusing on how differing commitments to scholarly and practical aims play out textually. Participating consultants from each sub-field assisted us both to select relevant journals and to categorize them, using a model mapped out by Donald Stokes (1997) in his book Pasteur's Quadrant. Stokes critiques onedimensional models that place theory and practice at opposite ends of a spectrum, forcing a choice between the two. He proposes instead a pair of intersecting axes, which represent relative degrees of commitment to the advancement of knowledge (the vertical axis) and to its application (the horizontal axis). Research can be plotted into one of four resulting quadrants, with work motivated by high interest in both knowledge advancement and practical application falling into the top right-hand quadrant, which Stokes dubs “Pasteur's quadrant.”

The corpus study would then be followed up by a close analysis of select articles identified by journal editors as especially representative of work in Pasteur's quadrant and by interviews with authors of those articles about the process of producing them. 3. Locating “research articles” With Stokes’ model, we felt confident that we had established a relevant, sound, and innovative approach to exploring discursive representations of diverse theory and practice priorities across our three education subfields. Our working assumption was that the research articles we collected from the selected journals would reflect these prioritiesdan assumption that would be tested in a subsequent analysis of article titles. But we ran into an unexpected complication: in beginning to select research articles for our corpus, we encountered such variation in the types of articles published across our three sub-fieldsdreflecting and constituting diverse positions on the relationship between theory and practicedthat we could not simply rely on our own experience of and familiarity with the genre as a grounds for identifying “research articles” in these sub-fields. Before we could proceed with our data collection, we needed either to locate an extant set of identifying criteria or to develop our own. As noted above, however, we had found no definitions of the research article. And the most commonly used sampling strategy in existing research studiesdthat is, identifying conventional IMRD macrostructuresdproved problematic for us, as we explain below. We also discuss below the challenges presented by another strategy we considered, namely, using journals' own categorizations of content. This strategy doesn't appear to be widespread in the literature on research articles. Nevertheless, it suggested itself as a reasonable possibility: Why not rely on headings supplied by the editors and common organizational principles of scholarly journals to identify research articles? 3.1. Using IMRD The standard IMRD macrostructure (Swales, 1990) is most often associated with research in the natural and social sciences. However, it is not constant (Swales (2002) himself has warned against too rigid an uptake), and drawing on it as a guide can be problematic, as several researchers have already noted. For one thing, “obligatory” IMRD moves are often accompanied by “optional” moves, and what counts as “optional” and “obligatory” also appears to be changing over time (Li & Ge, 2009). New technology, such as open-access publishing, contributes to a changing environment in which genres shift in response to “audience [expectations] as well as the ideology of the publisher” (Kittle Autry, 2013, p. 98). And the standard IMRD moves are not always transparently sign-posted (Ruiying & Allison, 2004), either by section headings or other linguistic features, making classification of an article based on those moves potentially both difficult and time-consuming.

4

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

Even more challenging for our study was the fact that the subfields of education on which we focused did not necessarily share the epistemological commitments that give rise to, and sustain, the IMRD macrostructure. Philosophy of education (P of E) is the obvious outlier, given the nature of knowledge-creation in its parent discipline, philosophy. But even in health professions education (HPE) and early childhood education (ECE), the fit is sometimes awkward. The HPE journals in our corpus make frequent use of the IMRD framework, which is possibly a function of the field's close relationship with the biomedical sciences. The practice of medicine is anchored in experimental research, and aspiring medical practitioners are instructed accordingly in knowledge derived from biomedical research. Medical (or, more broadly, health professions) education, however, is a different form of practice, and there are calls in the field for the adoption of a wider range of knowledge-making methods and epistemological models (Eva, 2009; Regehr, 2010; Yardley, 2014). But the pull exerted by the knowledge-making traditions of medicine appears to be strong, and we often see the discursive resources of these traditionsdin particular, the IMRD structuredused in scholarly publication, sometimes even with content that has no “methods” or “results” to report. Moreover, the research community in HPE is clearly oriented to practice, and there is extensive participation in research by clinician educators. Yet the latter are often not trained in research, and many of their articles seek primarily to share practical resources for teaching, evaluation, and program development rather than to pursue gaps in scholarly knowledge; again, however, the field's close connection with experimental science appears to push such articles into an IMRD structure. In contrast with HPE, there is a relatively strong divide between scholars and practitioners in ECE, and, while ECE practitioners do participate in research, they tend to create a very different type of knowledge than do scholars. Traditions of practitioner inquiry value personal and context-specific insights (e.g., Schon, 1983) and often adopt a critical/emancipatory agenda (e.g., Carr & Kemmis, 1986), none of which is particularly well served by the IMRD macrostructure. Yet we came across one journal of practitioner research in ECE in which the publication guidelines explicitly insisted on an IMRD format, with sections labeled accordingly. There are, of course, various possible explanations for such academicization; often, for example, journals publishing practitioner research are directed by university-based researchers, and it is possible that the influence of these boards may in some instances academicize the articles published. Whatever the reason, however, seeing the macrostructure applied in the aforementioned journal to a mode of inquiry so removed from traditional science contributed further to our hesitation in using it as a selection criterion. 3.2. Relying on journal categorizations As noted in section 3 above, another approach to locating research articles is to rely on journals’ own categorizations of the articles they publish. According to this sampling strategy, anything labeled a “research article” (or “original research” or something similar) would be included. Anything entitled “commentary,” “response” or “reflection,” etc., would automatically be excluded. Where headings were missing, we might reasonably consider typical organization of content as a guide; hence, we might accept as research articles anything between front contents like editorials and back contents like book reviews. On closer inspection, however, we discovered that journal categories (or their organizational formatting) are, in fact, not always reliable as a guide. Some journals offer descriptions in their submission guidelines of the types of texts various categories are intended to contain (in Medical Education we even found entire editorials dedicated to the introduction of new categories over the years). In other journals, however, no such explanation is offered, and the rationales for different categories are difficult to decipher on the basis of content. Although in principle allowing each of the scholarly communities in our study to define for themselves how they recognize the “research article” genre could be said to enhance interpretive validity, we decided that following only this approach creates potential problems in a cross-contextual study; given the significant epistemological differences between our education subfields, the “research article” genre may be defined so differently by each subfield that it is rendered too unstable and we risk comparing apples and oranges. The problem of journals' own categorizations was particularly evident in the field of HPE, where there appeared to be a larger variety of submission types than in the other two fields. In Academic Medicine, for instance, we came across two pieces in the same issue describing novel education programs, one classed as an “Innovation Report” and the other as a “Perspective.” Similarly, in an issue of Advances in Health Sciences Education, we found two articles that described and systematically evaluated an innovation, one a write-up of a pedagogic strategy classed as a “Reflection,” the other a report on an assessment tool, which the publishers deemed an “Original Paper.” In other journals, too, the publisher's categories were unclear to usdwhat is the difference between an “Article” and a “Research Article” in Medical Teacher? What is the difference between “Applied Research’ and “Research Basic to Medical Education” in Teaching and Learning in Medicine? Again, looking at the types of articles classed under these headings offered no real clues. Further complicating matters, journals' electronic tables of contents and the pdfs of articles do not always assign the same category heading for individual articles. Consequently, while the journals' own classifications did eventually play a role in our selection criteria (see below), we decided not to rely on them alone. 4. Research article selection criteria With these possible avenues for selection exhausted, we turned to formulating our own criteria. We began by returning to Miller's (1984) articulation of genre as social action and spelled out what the research article is typically understood to be doing, namely, presenting new knowledge that has been systematically produced. (The social action of the research article is, of course, richer and more complex than this, but we wanted to build out from a common and basic articulation.) We elaborated on the possible meanings of “new knowledge” and “systematically produced” based on our own background knowledge as researchers,

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

5

developing a set of working criteria that would be elastic enough to include scholarship operating in all three of Stokes' named quadrants mentioned above. From here we worked inductively and reflexively to refine our selection criteria. Our research assistants first applied them to the contents of each issue of every journal selected for our study, published in 2014 and 2015. Their resulting queries were discussed with us and our participating field consultants. We also applied the same criteria to select articles. Proposed revisions to our working criteria were circulated among all participants in the studydinvestigators, consultants and research assistantsdfor input before being adopted. The results of this process are summed up in our final criteria, as follows:

Criteria for selecting research articles for “Scholarly and Practical Orientations in Education Research Articles: A Genre-based Study” We use “research article” to refer to a genre that - has its origins in physical science but is now widely used across academic disciplines, including (even if often uncomfortably) by those who produce conceptual work. - is generally but by no means exclusively produced by and for those in the academy. - reports on new knowledge or new perspectives systematically produced. 1. The article presents new knowledge or new perspectives. 1.1 The article does not merely describe, summarize, or explicate extant knowledge or already documented perspectives (even if this extant knowledge or these documented perspectives may be new to the community being addressed). 1.2 The article may describe outcomes in terms of private insight. While practitioner inquiry values the advancement of practitioners’ personal knowledge, we will assume that personal revelations are, by dint of being published, also considered meaningful additions to collective knowledge. 1.3 “New” may be understood in relation to an existing body of scholarly work, either conceptually or empirically addressing a “gap” or “deficit” in the literature. 1.4 “New” may also be understood in relation to the lore of a professional community, either conceptually or empirically responding to (i) commonly held views or normative practices in need of revision, or (ii) widely recognized problems relating to practice. These views, practices, or problems, while not necessarily documented elsewhere, are described in the article. 1.5 New contributions may also be ambiguously addressed to both scholarly and professional communities, and, as such, they may benefit theory and/or practice. 1.6 Where an article chiefly concerns practical innovations and/or novel applications, whether concrete or conceptual, these have been carried out by the authors and are not merely speculated upon as potentially interesting. 1.7 Articles that merely describe the implementation of innovations, without indicating their systematic evaluation, are not “research articles.” 1.8 Research articles may include pilot studies and reports of preliminary research results. 2. The article shows that the new knowledge or new perspective is based on empirical evidence or argued warrants that are, in turn, the product of systematic processes. 2.1 It is clear in empirical studies that data was collected and analyzed in a considered way. Typically, this will involve methodologies recognized in qualitative or quantitative handbooks. 2.2 In conceptual work, too, processes are methodical. In the absence of methods sections, steps in reasoning and/ or interpretive lenses are still in evidence, although these may not necessarily be explicitly identified by the authors. Journal headings and formatting may serve as an additional guide here (see point 3 below), as may checks with our field consultants. 2.3 Articles reporting on the testing of innovations may not always include extensive discussion of methods but will include evidence of a degree of systematicity, via reference to evaluation tools, for instance. (For example, “we received lots of positive feedback about the course” does not satisfy our criteria for a “research article,” but “written exit surveys showed the overall course was positively regarded by all those participating” does.)

6

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

3. Journal category headings and formatting may serve as additional guides. Journal headings and formatting may serve as an additional guide in sorting research articles where the above criteria for systematicity are insufficient. However, such labels do not automatically take precedence over the above criteria. Texts labeled “research article” that do not conform to the criteria above (or, conversely, texts labeled something like “viewpoint,” which do meet the above criteria) will be individually considered. 4. Text types listed below are excluded from this study. The following text types are not included in this study: -

book reviews (including review essays dealing with multiple books) editorials “state of knowledge”/narrative reviews (systematic reviews will be individually considered) direct responses to an article in the same or immediately previous issue

5. Discussion Below we discuss several questions that needed answering in order to establish these sampling criteria for our research. 5.1. Is “research article” the best term? Our project focuses on the research article, but we considered whether we might need to use a different term. “Research” is often associated with empirical investigation, but our study is not restricted to that kind of scholarship. Our P of E consultant pointed out, for example, that most philosophers would use the term “scholarly article” instead of “research article” to describe what they produce. However, a decision to use the term “scholarly article” would seem to locate the object of our study more fully in the academy and might thereby exclude inquiry by those who are not (or only loosely) affiliated with universitiesdsomething we obviously wanted to avoid in a study focused on theory-practice relationships. Coining a new term for a genre that reports on new knowledge or perspectives systematically produced not only seems cumbersome but would also negate both the history of and recognition garnered by the term “research article.” Consequently, we retained the term “research article,” which we felt was warranted at least in part by the fact that the genre literature our project builds upon uses this term. In the introduction to our criteria, therefore, we note that the research article, as generally discussed in genre studies, has its origins in the physical sciences but that it is now widely used across disciplines. Most scholars in the social sciences are comfortable with the term, and, as our P of E consultant noted, even those in the humanities are under pressure to identify with it (even if less comfortably) in modern universities, many of which frame themselves as “research” institutions and reward faculty for “research, teaching and service.”2 5.2. What do we mean by “new”? Defining “newness” was perhaps the most complex part of determining our sampling strategy, because “new” is clearly a deictic term and, as such, its use generates several further questions. 5.2.1. New relative to a readership? to the writer? “New” might suggest that the knowledge presented is new relative to the community being addressed by the research article. Hence, for example, contents of a regular feature in the journal Medical Education entitled “When I Say …,” which briefly and accessibly explains concepts likely to be new to the broader HPE community (such as grounded theory, for instance) could be deemed new knowledge. But, of course, knowledge that is new to a given community may already exist elsewheredclearly grounded theory is not new to sociologistsdso we excluded from our study articles that chiefly described, summarized, or explained previously articulated knowledge or perspectives. “New” might also refer to knowledge that is new, not to a community, but rather to a particular individual. Practitioner inquiry argues that the advancement of practitioners' personal knowledge is a valuable and even sufficient goal for inquiry (see Cochrane-Smith & Lytle, 2015), and our criteria were developed to include articles from this tradition. We recognize the fact that research findings are usually meant to contribute to collective knowledge. And we are aware of the critiques of practitioner inquiry in the literature, in particular Huberman's (1996) concerns about “emic” perspectives. Nevertheless, in the ECE literature, in particular, we found numerous articles written by teachers that are intensely personal both in terms of their investigative processes and in terms of the insights they describe. To exclude such articles would have been to neglect an important segment of ECE research involving practitioners, and so, to do justice to the field as a whole, we had to expand our

2 She also pointed to an American Philosophical Association statement (1996) that asserts the field can be said to conduct “research” insofar as in common parlance the term is interchangeable with “inquiry”.

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

7

view of the objectives of research to include the development of personal knowledge among practitioners. Moreover, we reasoned that, if such knowledge is articulated in a publication addressed to a community of readers, it is also, by implication, considered by journal editors to be a meaningful addition to collective knowledge. 5.2.2. New in relation to academic theory or professional practice? Research articles, particularly in the (social) sciences, typically frame the originality of their contributions by first identifying a knowledge gap or deficit in the scholarly literature. Newness is established in relation to what has already been argued or established in the writings of a given academic field, and the aim is to add to existing theoretical knowledge. But what of (a) investigations that are practically oriented and (b) studies that contribute new knowledge that appears to be both theoretical and practical? Because we came across significant numbers of both types, we found we had to adjust our assumptions about what “new” meantdin other words, to consider that “new” might not be new always or exclusively in relation to documented theoretical knowledgedin order to include this work and thereby represent more adequately the range of research in our three subfields. The clearest examples of new practical knowledge in our study are articles from ECE produced in traditions of practitioner inquiry. Here practice is, of course, central, and while scholarly theory might be engaged, contributing to it is not the chief aim. “New” is new in relation to practitioners’ knowledge, which is, for the most part, not written down. While it may draw from textual sourcesdprofessional literature, policy documents, etc.dpractitioner knowledge is also shaped powerfully by individual experience in the classroom and through oral sharing with (mostly local) colleagues. The resulting knowledge is virtually impossible to pull together into anything as formalized as a scholarly literature review. Nevertheless, we posit a body of collective practitioner knowledge in terms of professional lore, tradition, and commonly held views. This knowledge involves amalgamations of texts, experience, and orally shared anecdotes and advice. While such knowledge is generally undocumented and thus not citable, our criteria suggest that it is still important that authors describe in some reasonable detail the aspect of collective practical knowledge they are engaging. While in many articles it is relatively easy to discern whether the aim is to contribute something new to a theoretical or to a practical body of knowledge, this is not always the case. In this study, we repeatedly found research articles that set themselves up to address scholarly needs but report outcomes that seem to speak to practical as much as, or more than, theoretical interests. This is the case with studies reporting on practical innovations and practical applications of theory. In HPE, we came across many articles that begin with references, often cursory and small in number, to academic work. These studies are usually structured according to academic IMRD formats, too. However, such references and formatting often seem to be journal-mandated gestures to scholarly interests, whereas the authors’ chief aim appears to be to relate experiences with new tools and strategies. Links between such practical tools or strategies and extant theory are usually thin and translations into theoretical significance, mostly absent. In ECE, we also found articles reporting on innovations. Here, the reviews of the literature are more substantial than in HPE, and the introductions tend to locate the impetus for the project very clearly in scholarly gaps. The innovations themselves often also draw much more explicitly than in HPE on existing theoretical work. In spite of this greater scholarly sophistication, though, the problems addressed by the innovations are in essence practical, and the results of the projects described appear to have as much, if not more, value for practice than for theory. Newness in both the HPE and ECE studies, then, presented itself to us somewhat ambiguously in relation to academic and practical interestsda possibility we felt was important to allow for in our criteria. 5.2.3. Does answering the question “does it work?” produce new knowledge? The HPE articles on practical innovations also raised another concern. As noted above, they do appear, on the surface at least, to meet our criteria for newness in that the intervention being described is usually framed as a novel response to existing scholarly findings and/or theory. Again, however, the articles often seem to be aimed not so much at contributing to collective understanding as to telling others in the field about the interesting thing the authors have done. Our sense that this was so was further supported by the discussion/conclusion sections of these texts, which tend to affirm that the intervention worked but not to point to any broader implications. There is often little attempt in the article to explain why or how this intervention worked in terms of broader principles or more general theories. This failure to shed light on a particular innovation as a type of response to a type of problem led us to question whether we should consider these articles “research articles.” It seemed to us that on some level these texts represent the awkward product of a merging of scholarly and practical orientationsethey take the generic form of a research article and make claims of newness (and systematicity), assuming a scholarly appearance, yet their aims and outcomes were not convincing to us as “research.” We considered requiring that “new knowledge or new perspectives” be understood to mean theoretical knowledge or perspectives. Yet if we excluded these texts, we risked omitting potentially interesting data from our study. Our HPE consultant confirmed their prevalence and noted further that this type of text has also been a focus of concern in debates over ways to improve theoretical contributions in HPE research. Given that our interest in this project lay in identifying generic possibilities for written work that engaged both academics and practitioners, we did not want to eliminate examples of articles that tried to balance theory and practice, even if they did so in what seemed to us less than ideal ways. These “does it work?” articles rhetorically clothed in scholarly forms were a case in point. Our deliberations around this type of article in particular made us keenly aware of our disciplinary biases in identifying the genre of the research article. “Does it work?” is not, on its own, generally considered a compelling research question in most social science fields. However, we decided not to insist on theoretical contributions. Instead we specified that articles would

8

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

not be added to our corpus if they only described practical innovationsdbe they pedagogical strategies (i.e., teaching tips) or something broader like a school-wide curriculum or a country-wide assessment tooldwithout offering a systematic evaluation of them. However, we would include articles that offered systematically derived evidence for the effectiveness of a practical innovation (or, less commonly, for its limited success). 5.2.4. Is potentially new, new? In the journal Medical Education, we came across a regular feature entitled “The Cross-Cutting Edge” in which scholars from outside HPE describe new research in their own fields that they believe might be beneficially applied to issues in HPE. We decided not to include texts published under this heading because their authors stopped short of actually doing the work they described, tending only to speculate about intersections and carry-overs that might be interesting or fruitful. We, therefore, added to our sampling criteria that applications of ideas must actually be carried out and reported upon, not merely identified as being of potential value. Prompted by input from our P of E consultant, we also determined that, in order to be included in our study, thought experiments need to be worked through to a conclusion and, again, not just introduced as a potentially interesting idea. 5.2.5. How much newness is enough? Our RAs working with HPE and ECE journals raised questions about the inclusion of pilot studies or work generating only preliminary resultsdwas there enough new knowledge here to qualify reports of pilot studies and preliminary results as research articles? We felt that, in principle, the answer was yes. However, we also took into consideration journal editors’ judgements; if the text in question was classed as something other than “research article” (or something synonymous such as “original article” or “article” or “original research”), we considered it more carefully before including it, checking further with our field consultants where necessary. 5.2.6. Are syntheses and direct responses new? We initially excluded literature reviews on the grounds that the collection and synthesis of articles to determine what is known and not known about a subject does not produce new knowledge. However, in the HPE literature we came across several examples of systematic reviews. Unlike narrative reviews, which sum up the state of knowledge about a subject, these reviews are guided by a specific research question, and, while the question itself is not necessarily new, the answer generated by a synthesis of articles focused on the same question can be said to be new. As the name implies, a systematic review is, moreover, a highly methodical undertaking in which strict protocols for searching and evaluating studies are spelled out. While, in principle, systematic reviews that met this description fit our criteria for newness (and systematicity), we noticed that the term was occasionally used rather loosely in scholarly publications such that a text labeled “systematic review” at times more closely approximated a narrative state-of-knowledge synthesis and thus did not qualify as either new or systematic. We therefore considered individually all texts categorized by authors or journal editors as a “review” and included only those that systematically answered a specific research question. By contrast, we automatically excluded direct responses to original articles, either in the same or the immediately subsequent issue. These commonly appear in P of E journals, where an original article often engages with another article published some time ago and the direct response is written by the author of that previously published text. One might, of course, posit that all research articles are in effect responses to existing ones and that the status of direct responses in P of E publications should not, therefore, present a difficulty for inclusion. However, “newness” in more typical research articles is cast as filling a communal knowledge gap, whereas “newness” in direct responses is on the order of clarifying and/or correcting claims, which we considered insufficient to count as “research.” 5.3. How do we recognize “systematicity” in conceptual work and innovation reports? Inquiry processes in empirical studies are generally described in fairly straightforward fashion, and broadly recognized processes are described in methodological texts. This makes systematicity relatively easy to assess in many of the articles in our corpus. There were two types of article, however, where systematicity was more challenging to determine: those reporting conceptual work (which includes almost all articles in P of E), and those reporting on practical innovations (of which there were a substantial number in HPE). Our selection criteria set out guidelines for addressing both, and below we elaborate on the challenges they presented in our study. 5.3.1. Systematicity in conceptual work Our P of E consultant pointed out that philosophers make a distinction between knowledge (based on empirical warrants) and perspectives (based on argumentation/reasoning); this distinction we easily incorporated into our selection criteria. We also readily assented to our consultant's proposal to sidestep positivist biases rejecting anything not empirically verifiable as “mere” opinion and to think of all scholarship, including empirical research, in terms of warranted claims. In this way, empirical evidence becomes just one form of warrant; careful and explicit argumentation that has rigorously considered potential counter-claims is another. However, neither the distinction between knowledge and perspectives nor the emphasis

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

9

on warranted claims addressed the problem of how to distinguish a bona fide research article based on “systematic” nonempirical work from commentaries, editorials, review essays, and the like. In non-empirical studies, there are, of course, no methodology sections that help distinguish systematic work from merely impressionistic observation. (This is not to say, of course, that the presence of a methodology section ensures a properly systematic study was carried out.) Moreover, in scholarly fields where conceptual work is the norm, there are usually no resources paralleling qualitative and quantitative handbooks that allow outsiders to determine what constitutes a careful and rigorous process of inquiry. While there may be publications discussing what constitutes a good philosophical argument or literary critique, such discussion is unlikely to be able to offer guidance that is as straightforward as that which might be found in a research methods text (though, again, this is not to suggest qualitative or quantitative methodological principles translate easily into lab or field). Whether a particular argument or critique goes deep enough, whether there has been sufficient consideration of counter claims, whether presuppositions were adequately examined … such questions are ultimately difficult to answer with the same kind of specificity as questions that can be addressed with reference to, for example, pvalues or intercoder reliability. For this reason, then, our criteria suggest a combination of checks. The two most readily carried out are (a) to look for messages about the argument contained in the text, such as forecasting the structure of the paper at the outset or drawing attention to steps in reasoning (e.g., “if we accept that … then it follows that …”) and (b) to check headings and placement to see what category the journal's editors have assigned the text. With respect to messages about the argument, it is important to note that their absence is not necessarily evidence of a lack of systematicity. Our P of E consultant noted that the presence of such messages in her subfield tended to vary with the philosophical traditions in which writers worked. Conversely, the presence of messages about the argument does not automatically signal sufficient systematicity. The adequacy of an argument will depend to a great extent on the community that produces and consumes the text. Hence, looking at headings and placement in a journal can also be considered an important check; if the editors deem the text a “research article” (or “original article” or some similar term), it might be reasonably assumed it has met communal standards for systematicity. Of course, as we have already indicated, headings are sometimes deployed in inconsistent ways and cannot therefore be considered wholly satisfactory guides on their own. Nevertheless, combining the two checks mentioned herednamely, messages about the argument and editorial headings/placementdplaces us on safer ground. If doubts persist about whether the text in question is sufficiently systematic enough to be considered a “research article,” our criteria suggest that a third check can be made by asking for input from field consultants. As insiders, they are well-placed to determine whether the field's communal standards for differentiating “research” articles have been met in the text. 5.3.2. Systematicity in innovation reports Many articles in the HPE journals involve descriptions and evaluations of some type of practical innovation, usually an intervention designed to solve a practical problemdan assignment or test, a curricular improvement, an assessment instrument, a new education program, etc. In terms of generic form, these texts typically follow the IMRD structure. The intervention is described and evaluated, generally in some systematic way spelled out under a “methods” section. However, such sections are not consistently present, and even when they are, the standards for documenting systematic methods (and results) do not always appear particularly high. While some articles are thorough in reporting how effectiveness was assessed and report results with robust details and/or numbers, others are decidedly weak in this regard, to the point where we wondered whether any claim for systematicity could really be made. Brief, unelaborated references to exit questionnaires or follow-up interviews, and assertions that the intervention had been “generally well-received” or that “we observed substantial increases” did not satisfy us that there had been due attention to method. Yet, we decided that we would have to accept even “vaguely systematic” as a criterion since we couldn't legitimately set standards for another scholarly community based on our own disciplinary backgrounds. Consequently, where articles had been classed as research by the journals in which they were published (in other words, had been labeled as “research article” or something similar), we allowed methodological vagueness. Indeed, even where the journals themselves offered no clues, we chose to err on the side of generosity and include these texts in our corpus. 6. Conclusion This paper addresses an inherent contradiction in genre studies, whereby on the one hand, genre theory rejects the notion that genres are fixed, but, on the other hand, genre research needs genres to be rendered at least relatively stable in order to study them and thereby to develop sound theory. When one sets out to study a particular genre, one necessarily has to fix it, at least temporarily, for the purposes of that inquiry. This raises questions about how to navigate methodologically the tension between stability and changedhow to operationalize, in a theoretically sound way, Schryer's (1993) notion of “stableenough-for-now.” A good deal of genre-based inquiry into the “research article” genre appears to avoid this question, but we argue that it is essential for genre researchers not to assume a priori and implicitly what defines this genre. The criteria used to identify research articles (and, indeed, any genre under study) need to be identified and made explicit. They also need to be both firm enough to make consistent data selection decisions and flexible enough to take into account the variety of article types encountered in scholarly publications.

10

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

We have modeled here a grounded approach to developing such criteria, starting with a prestige genre, the research article, which provides a basis for comparison across scholarly subfields, and then working to establish criteria that both recognize the diverse priorities and practices of these subfields and balance this diversity with the needs of genre research to make cross-field comparisons. For the purposes of our own study, the particular set of criteria discussed here (which, we stress again, are not intended as generally conclusive and final) have allowed for the fact that the research article is a varied and evolving genre and have permitted us to shift away from the relatively conservative and powerful IMRD form. With respect to the tension between research and practice in education, such broadening of the genre constitutes a political move, raising questions about the types of knowledge that are privileged when a relatively narrow form of the research article genre is taken as the default sampling choice. Failure to recognize the full range of research writing has implications not only for sampling methodology in genre research but also for broader epistemological valuationdthat is, for the judgments we make regarding what “knowledge” means and how it should be pursued. Such judgments underpin the types of research our society supports financially and institutionally, of course, but they also factor in complex ways in our practical decisionmaking, in everything from public policy, to professional practice, to personal life. This brings us back to the issues we explore in our larger project on the tension between theory and practice in the research article. The connection between research and its application is far from straightforward, and not merely an analytic problem but also one for practical and critical investigation. Assumptions of the type critiqued by Stokes about a unidirectional link between research and its uptake beyond academic realms have been shown to be problematic (see, for example, Star & Griesemer's (1989) work boundary objects and Jasanoff (2004) and her colleagues' studies of co-production). Similarly, the communicative implications of such assumptions, which tend to cast popularization as something separate from and inferior to the work of science, have also been critiqued (Myers, 2003). Genre-based inquiry, with its dual emphasis on linguistic form and social function, could make a valuable contribution to a deeper understanding of complex theory-practice connections as these are made manifest in the language of research. To this end, however, we argue that genre researchers must attend more explicitly to questions of data selectiondin other words, to the (properly) counter-intuitive yet necessary task of defining the genre under study.

References American Philosophical Association (1996). Statement on research. Proceedings and Addresses of the American Philosophical Association 70(2), 119e121. Bakhtin, M. M. (1986). In M. Holquist, & C. Emerson (Eds.), Speech genres and other late essays. Austin: University of Texas Press. Bazerman, C. (1988). Shaping written knowledge: The genre and activity of the experimental article in science. Madison: University of Wisconsin Press. Bazerman, C., & Prior, P. (2004). What writing does and how it does it: An introduction to analyzing texts and textual practices. Mahwah, NJ: Lawrence Erlbaum. Bhatia, V. K. (2001). Analyzing genre: Some conceptual issues. In M. Hewings (Ed.), Academic writing in context: Implications and applications (pp. 79e92). Birmingham: University of Birmingham Press. Bhatia, V. K. (2004). Worlds of written discourse: A genre-based view. New York: Continuum. Carr, W., & Kemmis, S. (1986). Becoming critical: Education, knowledge and action research. London: Routledge. Cochrane-Smith, M., & Lytle, S. L. (2015). Practitioner inquiry, knowledge, and university culture. In J. J. Loughran (Ed.), International handbook of self-study of teaching and teacher education practices (pp. 601e651). Dordrecht: Springer. Eva, K. W. (2009). Broadening the debate about quality in medical education research. Medical Education, 43(4), 294e296. Gray, B. (2015). Linguistic variation in research articles: When discipline tells only part of the story. Amsterdam: John Benjamins. Gross, A. G., Harmon, J. E., & Reidy, M. S. (2002). Communicating science: The scientific article from the 17th century to the present. Oxford: Oxford UP. Huberman, M. (1996). Moving mainstream: Taking a closer look at teacher research. Language Arts, 73(2), 124e140. Hyland, K. (2004). Disciplinary discourses: Social interactions in academic writing. Ann Arbour: University of Michigan Press. Jasanoff, S. (Ed.). (2004). States of knowledge: The co-production of science and the social order. London: Routledge. Kittle Autry, M. (2013). Genre change online: Open access and the scientific research article genre (Doctoral dissertation). Raleigh: North Carolina State University. Knorr-Cetina, K. (1999). Epistemic cultures: How the sciences make knowledge. Cambridge, MA: Harvard UP. Labaree, D. (1998). Educational researchers: Living with a lesser form of knowledge. Educational Researcher, 27(8), 4e12. Labaree, D. (2004). The trouble with ed schools. New Haven, CT: Yale UP. Li, L.-J., & Ge, G.-C. (2009). Genre analysis: Structural and linguistic evolution of the English-medium medical research article (1985-2004). English for Specific Purposes, 28, 93e104. Martinez, I., Beck, S., & Panza, C. (2009). Academic vocabulary in agriculture research articles: A corpus-based study. English for specific purposes (Vol. 28, pp. 183e198). Miller, C. (1984). Genre as social action. Quarterly Journal of Speech, 70, 151e167. Molino, A. (2010). Personal and impersonal authorial references: A contrastive study of English and Italian linguistics research articles. Journal of English for Academic Purposes, 9(2), 86e101. Myers, G. (2003). Discourse studies of scientific popularisation: Questioning the boundaries. Discourse Studies, 5(2), 265e279. Peck MacDonald, S. (2002). The analysis of academic discourse(s). In E. Barton, & G. Stygall (Eds.), Discourse studies in writing (pp. 115e133). Cresskill, NJ: Hampton Press. Regehr, G. (2010). It's NOT rocket science: Rethinking our metaphors for research in health professions education. Medical Education, 44(1), 31e39. Ruiying, Y., & Allison, D. (2004). Research articles in applied linguistics: Structures from a functional perspective. English for Specific Purposes, 23(3), 264e279. Schon, D. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books. Schryer, C. (1993). Records as genre. Written Communication, 10(2), 200e234. Sheldon, E. (2011). Rhetorical differences in RA introductions written by English L1 and L2 and Castilian Spanish L1 writers. Journal of English for Academic Purposes, 10(4), 238e251. Star, S. L., & Griesemer, J. R. (1989). Institutional ecology, ‘translations’ and boundary objects: Amateurs and professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39. Social Studies of Science, 19(3), 387e420. Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation. Washington DC: Brookings Institution Press. Swales, J. M. (1990). Genre analysis. Cambridge: Cambridge UP.

A. van Enk, K. Power / Journal of English for Academic Purposes 29 (2017) 1e11

11

Swales, J. M. (2002). On models in applied discourse analysis. In C. Candlin (Ed.), Research and practice in professional discourse (pp. 61e77). Hong Kong: City University of Hong Kong Press. Tierney, W. G., & Holley, K. a (2008). Inside Pasteur's quadrant: Knowledge production in a profession. Educational Studies, 34(4), 289e297. Valipouri, L., & Nassaji, H. (2013). A corpus-based study of academic vocabulary in chemistry research articles. Journal of English for Academic Purposes, 12(4), 248e263. Yardley, S. (2014). Lost in translation: Why medical education research must embrace “real-world” complexities. Medical Education, 48(3), 225e227. Dr. van Enk is a Research Associate at the Centre for Health Education Scholarship at the University of British Columbia. She has taught courses on academic writing, discourse analysis, narrative research, adult literacy, and digital literacy. Her current work focuses on relationships between research and practice, particularly the context of in health professional education, and on notions of expertise as these manifest in the discourses of lay, professional, and research communities. Dr. Power is an Instructor with Arts Studies in Research and Writing at the University of British Columbia. She is an applied linguist currently co-editing an interdisciplinary (linguistics and economics) volume on discourses of austerity. She is also Secretary for the International Gender and Language Association. Her research interests include religious discourse, gender/sexuality, economic discourse, and academic writing.