Development of metalanguage for multimodal composing: A case study of an L2 writer’s design of multimedia texts

Journal of Second Language Writing 47 (2020) 100714

Contents lists available at ScienceDirect

Journal of Second Language Writing journal homepage: www.elsevier.com/locate/jslw

Development of metalanguage for multimodal composing: A case study of an L2 writer’s design of multimedia texts

T

Dong-shin Shina,*, Tony Cimaskob, Youngjoo Yic a b c

Dong-shin Shin, Literacy and Second Language Studies, University of Cincinnati, Cincinnati, OH 45221-0022, United States Miami University, United States The Ohio State University, United States

A R T IC LE I N F O

ABS TRA CT

Keywords: Multimodal composing Multimodality L2 writing Systemic functional approach to multimodal discourse analysis Metalanguage Digital multimodal composing

This study investigates a sixth-grade L2 writer’s composition of digital multimodal texts, and his development of the metalanguage of modal and intermodal resources of language and image. Based on a systemic functional approach to multimodal discourse analysis (SF-MDA) and principles of sociosemiotic ethnography, it examines both the student’s composing processes and his composed texts. In the writing center of an inclusive sixth-grade classroom in a U.S. elementary school, the student created multimodal PowerPoint slides explaining the greenhouse effect, and Glogster-based multimedia texts arguing for banning guns. The collected data include the student’s texts, interview responses, and observation notes. Grounded in SF-MDA, the analytical framework draws on codes such as employed modes, modal resources, intermodal relations, and constructed meanings (i.e., ideational, interpersonal, and textual). The findings suggest that a) the student employed linguistic and visual modes in the mediums drawing on concurrence and complementarity intermodal relations; b) the student used non-linguistic resources for the ideational meaning of texts as a primary semiotic mode; and c) the student developed awareness of intermodal relations and metafunctions of sign systems in a non-linear way. The findings contribute valuable insights into understanding L2 learners’ development of metalanguage for multimodal composing.

1. Introduction A growing number of second language (L2) writers engage in multimodal composing and literacy practices that often entail design processes of mixing and remixing different languages, genres, modalities, modes, and styles. Studies demonstrate that multimodal composing and literacy practice allow L2 learners to develop English language proficiency and metalanguage through authentic writing opportunities, diverse text types, and hypertexts organized in non-linear ways (Barton & Potts, 2013; Belcher, 2017; Shin & Cimasko, 2008). Pedagogically, multimodality in L2 writing and literacy instruction allows learners to have expanded views of literacy, engages them in critical literacies, and promotes learners’ language awareness (Yi, Shin, & Cimasko, 2019). These affordances allow learners to develop semiotic knowledge with multiple modes that enable them to more clearly express a wider range of identities, as well as an increased sense of authorial agency (Hafner, 2015; Jiang, 2017; Yi & Angay-Crowder, 2016). In particular, it has been reported that L2 writers’ expression of voice is enhanced, with increased semiotic resources beyond language. A substantial number of studies have investigated the aforementioned multimodal affordances for L2 learners, framed by

⁎

Corresponding author. E-mail addresses: [email protected] (D.-s. Shin), [email protected] (T. Cimasko), [email protected] (Y. Yi).

https://doi.org/10.1016/j.jslw.2020.100714 Received 28 April 2019; Received in revised form 21 January 2020; Accepted 23 January 2020 1060-3743/ © 2020 Elsevier Inc. All rights reserved.

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

multiliteracies and sociocultural theories (New London Group, 1996; Street, 1984). Such examinations have focused on learners’ uses of multimodal texts, explored ways to engage with texts in specific multimodal textual practices, and elucidated the influences and effects of multimodal literacy practices for language and literacy learning and identity construction (Ajayi, 2009; Toohey et al., 2015). Drawing on social semiotics and systemic functional linguistics (SFL) (Halliday & Matthiessen, 2014; Kress, 2003), researchers have recently started to explore multilingual writers’ employment and orchestration of modes/semiotic resources into ensembles in multimodal composing (Shin, 2018; Smith, Pacheco, & de Almeida, 2017). Although this line of research has looked at composing processes and metalanguage development, few studies have examined how students synthesize both modal resources of single modes and intermodal relations across modes in the creation of a multimodal ensemble (Shanahan, 2013). To address this research gap, the current study examines a sixth-grade L2 learner’s composing of multimodal texts that he created using linguistic and visual modes available in multimedia mediums. In particular, we will highlight the learner’s uses of modal and intermodal resources of language and image as well as his metalanguage development in multimodal composing by looking at both its processes and produced texts, drawing on principles of sociosemiotic ethnography (Iedema, 2001; Prior, 2013). The study will expand the existing research on multilingual writers’ multimodal composing through an analysis of modal and intermodal resources of language and image, and through a methodological combination of composing process analysis and textual analysis of the student’s composed product. The present research specifically sought to answer following questions: 1. How did an L2 learner orchestrate language and images into multimodal texts with multimedia tools? 2. How did the learner use intermodal relations between language and image, and how did he develop the metalanguage of the semiotic systems? 2. Theoretical framework 2.1. Systemic functional approach to multimodal discourse analysis This study draws on a systemic functional approach to multimodal discourse analysis (SF-MDA) (Jewitt, Bezemer, & O’Halloran, 2016; Kress & van Leeuwen, 2001; Kress, 2003; Unsworth, 2006). As the term suggests, SF-MDA couples SFL with multimodal discourse analysis (MDA) and entails conducting multimodal discourse analyses in accordance with systemic functional principles. Both SFL and MDA are based on social semiotics (Kress & van Leeuwen, 2001; Prior, 2005), which conceptualizes meaning-making as a social and cultural practice. MDA describes the interaction of multiple modes that are “socially made and culturally shared” semiotic resources (e.g., language, images, sound) for the creation of meaning (Kress, 2010, p.104). Using a social semiotics approach, MDA examines how writers (re)design their texts by orchestrating available meaning-making resources into a multimodal whole, making authorial decisions appropriately for specific audiences and purposes. Writers’ agency in designing is closely interconnected with their understanding of the apt relations between employed modes and intended meanings, as well as the synaesthetic semiosis of composing multimodal ensembles (Kress, 2003, 2010), showing that “meanings presented in two or more co-present semiotic modes, e.g. the visual/pictorial and oral/linguistic, combine in such a way that new forms of meaning may obtain” (Nelson, 2006, p. 59). Focusing on metafunctions of language, SFL explicates how a text is constructed within the contexts of culture and situation. The context of culture is associated with a genre, whereas the context of situation is related to an instantiation of the genre. A specific text realizes three register variables that reflect the context of situation—field, tenor, and mode—and texts differ from one another due to their own different combinations of these variables. The field represents the ideas of a text being addressed, tenor represents the relationship between the individuals involved, and mode represents the organization of the ideas within the medium of communication (Halliday & Matthiessen, 2014; Schleppegrell, 2004). Hence, any situation of communication involves exchanging ideas, establishing an interpersonal relationship, and organizing ideas to facilitate communication. The field, tenor, and mode variables of a situation correspond to three metafunctions of language—ideational, interpersonal, and textual—that serve to construct the same three respective meanings of a text. According to SFL, a text construes ideational, interpersonal, and textual meanings. The ideational meaning represents the experience and the world, and is realized through a transitivity system that explicates the participants, processes, and circumstances of a text. The process types include material, mental, verbal, behavioral, existential, and relational. To explicate processes that have SFLspecific meanings, material process (e.g., act, make) describes concrete actions, behavioral process (e.g., look at, smile) explains physiological and psychological behaviors, and relational process (e.g., be, become) covers relationships between two elements. The interpersonal meaning describes different social roles, identities, and relationships construed by language use, and is construed through lexico-grammatical resources such as mood, modality, and appraisal system. Mood types, which include declarative, interrogative, and imperative grammar structures, are intertwined with speech functions such as statement, question, offer, and command that are introduced by Halliday (Eggins, 2004, p. 145). Modality involves modalization (e.g., might, sometimes) to express possibility or usual occurrence as well as modulation (e.g., should, willing to) to express obligation and inclination. Appraisal system describes three interacting elements such as attitude, engagement, and graduation within a linguistic framework that respectively explain “feelings,” “sourcing attitudes and the play of voices around opinions in the discourse,” and “grading phenomena” (Martin & White, 2005, p. 35). Textual meaning explains how the information is organized within a medium of communication, and its meaning is realized through the theme and rheme structure (Eggins, 2004). Theme is a point of departure for a message, whereas rheme offers new information about the point of departure. Theme has major systems that involve choices of type of theme, marked versus unmarked themes, and predicated versus unpredicated themes (Eggins, 2004, p. 299). The types of theme include topical themes with transitivity 2

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

elements, interpersonal themes, and textual themes. Theme markedness explains atypical themes based on the conflation of the theme constituent with mood and transitivity elements (e.g., “In New York” in the clause, “In New York, December is cold”), while theme predication entails use of an additional clausal element (e.g., It is… that…). Kress and van Leeuwen (2006) conceptualized similar semiotic metafunctions for analysis of the visual mode and its communicational grammar: representational, interactive, and compositional, which respectively represent ideational, interpersonal, and textual meanings of the linguistic mode. The representational/ideational function deals with material reality to explain how visual resources representing transitivity construct ideas in communication, and the interactive/interpersonal function relates to social reality, explicating how evaluative and attitudinal meanings are constructed through interpersonal visual resources (e.g., gaze, credibility of image, tone of color). The compositional/textual function explains how a text is organized through visual meaning-making resources including the layout, placement, and relative salience of images. Visual representation (Kress & van Leeuwen, 2006) includes the narrative structure for designing social actions and the conceptual structure for designing social constructs. That is, the narrative structure represents doing and happening in the visuals, while the conceptual structure represents participants in terms of social constructs. The conceptual structure involves the “classificatory,” “analytical,” and “symbolical” sub-structures (p. 59). The “classificatory” elaborates the taxonomy among participants focusing on subordinate and superordinate relations, and the “analytical” illustrates participants in terms of a part–whole structure focusing on spatiality and timeline. The “symbolical” structure describes a participant’s meaning or identity through the symbolic attributive for representational salience and the symbolic suggestive for generalized essence (p. 105). In analyzing meanings of a multimodal text, SF-MDA focuses on “the organizational principles of a system and less on the sign maker” (Shanahan, 2013, p. 198). It notes that modes/semiotic resources are “the sets of system choices” and that “each semiotic resource has its own systems of meaning, units of analysis, and structure” (Jewitt et al., 2016, p. 36). Thus, systems of meaning for language and image are described differently from each other. Language has its semiotic system of lexico-grammatical resources, whereas image has its semiotic system of visual resources. All modes have “potentials for meaning, though differently with different modes,” and each mode has ideational, interpersonal, and textual functions (Kress, 2010, p. 104). In any given communication “several modes are always used together, in modal ensembles” (Kress, 2010, p. 28), and jointly construct the three metafunctions across modes through semiotic interactions in the contexts of culture and situation (Jewitt et al., 2016, p. 46). As such, SF-MDA involves looking at both separate modes and intermodal relations among the modes in describing the meanings of a multimodal ensemble.

2.2. Intermodal relations between language and image Intermodal relations contribute to constructing the ideational, interpersonal, and textual meanings of a multimodal text. In order to understand the participant’s multimodal composing in the current study, it is necessary to explore intermodal relations between language and image (Painter & Martin, 2011). Regarding the ideational meaning that language and image create across modes within synaesthetic semiosis, Unsworth (2006) presents three types of relations—concurrence, complementarity, and connection. Concurrence between language and image explains how one mode specifies or describes the meaning of the other without adding any new information as “ideational equivalence,” taking the forms of redundancy, exposition, instantiation, or homospatiality. Concurrence shows redundant information across the modes, although it is not “a simple inter-modal duplication of meaning” (p. 60). Redundancy and exposition explain the same information with different levels of generality, instantiation relates to image embodying language and vice versa, and homospatiality depicts meaning construction of two modes “in one spatially bonded homogenous entity” (p. 61). Complementarity describes how a new element is added by either language or image in relation to augmentation and divergence of ideational meaning. Ideational augmentation explains that both modes offers additional meanings to one another, whereas ideational divergence provides opposing ideational content to the other mode. Lastly, connection is composed of two ideational types, projection and conjunction. Projection explains how quoting or reporting speech and thoughts are inscribed within the intermodal links, and conjunction shows how conjunctive relations of time, place, and cause are conveyed. In terms of constructing interpersonal meanings, images may require a viewer’s participation through the gaze of a represented participant at the viewer, which is known as demand, whereas images without such a gaze offer information to the viewer (Painter, Martin, & Unsworth, 2013, p. 18). The images evoke feelings, co-articulate attitudes with verbiage, and convey the modality of truth or credibility through use of real images representing the natural world. The semiotic complex between image and language jointly constructs evaluative meanings and co-articulate attitudes through close-up and eye-level images that juxtapose language choices (Painter et al., 2013, p. 137). For textual meanings, images utilize layout resources and framing. In organizing given and new information, familiar information is located in the given position on the left, and unfamiliar and technical information in the new position on the right (Kress & van Leeuwen, 2006). The ideal and real information utilizes the structure that arranges abstract, general information at the top and concrete, specific information at the bottom. In terms of spatial relationship, image and language are organized in a complementary layout that organizes them into separately demarked spaces, or an integrated layout that treats image and language as a unified entity (Painter et al., 2013, pp. 93–98). As such, visual mode in multimodal composition intermodally construes its textual meaning in interconnected relations with the linguistic mode.

3

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

3. Method 3.1. Context This study was conducted in a sixth-grade classroom of Greenville Intermediate School (all place and participant names are pseudonyms), which is located in a rural area in the northeast United States. The students of Greenville Intermediate were predominantly from economically challenged backgrounds, given that about 30 percent of its students qualified for free and reducedprice lunches during the school year when the present study was conducted. The school had a computer lab equipped with thirty-five computers, and all classes could use it for their instructional activities throughout the academic year. The Greenville Intermediate School followed a state-mandated curriculum framework that directly aligns with the Common Core State Standards (CCSS) (National Governors Association Center for Best Practices, Council of Chief State School Officers, 2010). The sixth-grade class had 18 students, including nine girls and nine boys. Four of the 18 students in the sixth-grade classroom were classified as requiring Individual Education Plans (IEPs) due to their status as English learners or special needs students. In addition to the computer lab, the sixth-grade class had four computers, two iPads, a SmartBoard, and an Elmo document camera for the teacher to use. The sixth-grade class’ daily schedule started with individual literacy and mathematics activities as morning work, before the regular daily classes in content areas including English language arts (ELA), mathematics, science, and social studies. 3.2. Participant Although we observed all the students of the sixth-grade class, in the current paper, we introduce one focal student’s composing process by adopting a case study method (Merriam, 2009). By examining a single student’s multimodal composing in depth, we aim to contextually understand the meaning-making process in which he was engaged, rather than finding general patterns of bi/multilingual learners’ multimodal composing practices. The focal student for this study was Michael, a Ukrainian child whose family immigrated to the States when he was a kindergartener. Michael spoke Ukrainian with family members at home as his first language (L1), and English for school work and for social interactions with peers as his L2. He possessed limited literacy skill in Ukrainian compared to his speaking skill, but his English reading and writing proficiencies were at grade level. In addition to being an L2 learner, we selected Michael for the study because he enjoyed computer-related work (e.g., online class activities, computer games), and his computer skill was good enough to carry out the multimodal composing task. The classroom teacher, Laurie Johnson, was interested in instructional technologies and used multimedia authoring tools in literacy activities with her students. For instance, Ms. Johnson utilized Edmodo, a collaborative web-based platform with pedagogical features for connecting students and sharing ideas, in her instruction across content areas throughout the school year. She delivered to her colleagues in the school a workshop on how to use Edmodo. She was pursuing her master’s degree in a language and literacy education program with which the first author was affiliated, and took courses about linguistically and culturally diverse learners’ emergent literacy and about multiliteracies. Ms. Johnson conducted a curriculum project for master’s study that incorporated Web 2.0 technologies into the curriculum and instruction of content areas, and invited the first author to her classroom. The author provided support for the teacher to design, deliver, and report on the curriculum project. 3.3. Curricular units The study was based on two curricular units in the content areas of ELA and science: expository writing from November to December on the greenhouse effect, and argumentative writing from January to April on banning guns. Drawing on a thematic approach between ELA and science, Ms. Johnson designed an expository writing unit to investigate and explain the greenhouse effect and global climate change, referring to Next Generation Science Standards (NGSS Lead States, 2013). She developed an ELA curricular unit of argumentative writing on banning guns by adopting a permeable curriculum approach (Dyson, 1993) that supports students in using their interests and funds of knowledge as valuable meaning-making resources (Moll, Amanti, Neff, & Gonzalez, 1992). According to CCSS, the 6th graders were required to write “informative/explanatory texts to examine a topic and convey ideas, concepts, and information” and “arguments to support claims.” Expository writing was meant to help students “write through the selection, organization, and analysis of relevant content,” whereas the argument genre was taught with the goal of enabling students to express their claims with clear reasons and relevant evidence (National Governors Association Center for Best Practices, Council of Chief State School Officers, 2010, n. p.). Ms. Johnson incorporated multimodal composing and digital design into the curricular units by using multimedia authoring tools (i.e., PowerPoint, Glogster, and Edmodo) for producing and publishing texts. While interacting, collaborating, and sharing ideas on the class Edmodo website, students created multimodal expository and argumentative texts using PowerPoint as well as Glogster, an interactive online multimedia platform for composing and publishing multimedia posters. To teach students how to produce multimodal texts in the mandated academic genres, the teacher employed an SFL-informed genre pedagogy (Gebhard, 2019; Harman, 2018; Rose & Martin, 2012; Troyan, Sembiante, & King, 2019), making references to writing workshops (Calkins, 2013) that the sixth-grade class also used for its ELA curriculum. She scaffolded students on how to compose multimodal texts through a teachinglearning cycle with three main stages: deconstruction, joint construction, and independent construction. These stages were designed with their own structure and different kinds of literacy support, and made up a cycle that provided multiple entry points and back and forth movements between stages for students, depending on their needs. Deconstruction. This stage was created to orient students to the curricular unit. The teacher built shared fields of the expository 4

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

and argumentative texts that the students were expected to write during this stage, in addition to checking students’ understanding of concepts that had previously been introduced. This stage entailed an analysis of model texts that focused on specific language features of the texts, to build a shared metalanguage that students could draw upon in the joint construction stage. Ms. Johnson used mentor texts from books (e.g., Rosa Parks) and resources from the Internet (e.g., YouTube video of President Obama’s Back to School Speech, NASA Climate Kids Website) to teach the purpose and function of expository and argumentative writing. After reading the mentor texts, the students participated in close reading activities in Edmodo such as sharing information about the greenhouse effect, finding textual evidence for argumentative prompts, and assessing uses of nonlinguistic modes in multimodal texts. In this stage, the students built shared common background knowledge for expository and argumentative writing, while simultaneously familiarizing themselves with new technologies for multimodal text productions. Joint construction. This stage focused on teacher-led collaborative writing. Drawing on shared knowledge from the deconstruction stage, students jointly composed multimodal expository texts on the greenhouse effect in PowerPoint and multimodal arguments on banning guns in Glogster. The co-construction of the multimodal texts allowed the students to see what semiotic resources were available in PowerPoint and Glogster, and how the multimodal resources realized the ideational, interpersonal, and textual meanings of the expository and argumentative texts. Concerning multimodal composing, this step was meant to further show how various linguistic and non-linguistic modes create a multimodal ensemble. For instance, Ms. Johnson explained the meanings that words and images could create intermodally through co-elaborating, complementing, or connecting relationships. Like the deconstruction stage, the joint construction stage was designed to prepare students for successful individual composing in the independent construction stage. Independent construction. After building knowledge about multimodal ensembles by co-creating an expository text and an argumentative text, students began to write their own expository texts on the greenhouse effect and argumentative essays on banning guns. For expository writing, the students directly composed PowerPoint slides without drafting in notebooks, but for their multimodal arguments, they composed their first draft in notebooks before creating their final drafts in Glogster. Throughout their composing process, they exchanged feedback on each other’s ideas, selected modes, and syntheses of modes on the Edmodo site and in face-to-face conferences with the teacher. When the students completed their texts, they posted them on the Edmodo site and critically reflected on their own texts. 3.4. Data sources and analysis Adopting the principles of qualitative study (Merriam, 2009), multiple domains of data were collected, including the student’s written texts, feedback exchanges in Edmodo, interview data, and instructional materials. We collected electronic copies of the PowerPoint slides and the Glogster texts that were posted in the class Edmodo. We took observation fieldnotes of the students’ composing processes of the texts and class interactions. The student’s written and multimodal texts were used for the primary examination of student multimodal composing, while fieldnotes, feedback exchanges in Edmodo, interview data, and instructional materials provided contextual information about the student’s designing processes. We conducted two 15-minute formal interviews for both writing curricular units: the first occurred at the beginning of the project to gather information about Michael’s computer uses and skills, and the second was a formal exit interview to ask about his multimodal composing experiences, including opportunities and challenges. In addition, we held five informal interviews with Michael during the curricular units to ask questions about his designing processes, focusing on modal resources, intermodal relations, and synaesthetic orchestration of all employed modes. We conducted informal interviews about the designing process, two for the expository writing unit and three for the argumentative writing unit. Each of the informal interviews lasted for about 20 minutes. Both the formal and informal interviews were audiorecorded and transcribed. Through multiple domains of data, we aimed to understand Michael’s composing process with multimodal resources and his understanding of modal resources and intermodal relations in multimodal composing over time as he produced multimodal expository and argumentative texts. We conducted domain analysis and triangulation by employing multiple methods of data collection (e.g., observation, interviewing, documentation), cross-checking multiple sources of data, and comparing multiple investigators’ analyses, in addition to member checking with the participant, to secure the validity of our data analysis (Denzin & Lincoln, 2003, p. 8; Merriam, 2009, pp. 215–216). By coupling SF-MDA (Jewitt et al., 2016; Unsworth, 2006) with sociosemiotic ethnography (Iedema, 2001; Prior, 2013), we analyzed both Michael’s composing processes and his composed texts. Our analysis was initiated by an examination of the composing processes of his multimodal texts, with his class interactions as a backdrop. The analytical framework we used included codes such as employed modes, created meanings, rhetorical choices, and perceived audiences. The unit of analysis to investigate Michael’s composing processes was each genre text that was bounded by its contexts of composing. Upon completion of an analysis of composing processes, we turned our attention to the composed multimodal texts by drawing on SF-MDA; our first step was examining the modes (i.e., language, image, space, and color) that Michael employed in the expository and argumentative texts. Given that space and color were inseparable from linguistic and visual modes in multimodal composing within PowerPoint and Glogster, linguistic and visual modes were examined as two main modes, rather than counting spatial placements and visual attributes (e.g., color, size) as separate modes (Martin & Rose, 2008). The analysis involved examinations of the semiotic systems of linguistic and visual modes in relation to ideational, interpersonal, and textual metafunctions that create respective meanings of the text (see Table 1 for semiotic systems of language and image). For language, we looked at transitivity systems for ideational meaning, lexico-grammatical resources such as mood, modality and appraisal systems for interpersonal meaning, and uses of theme and rheme for textual meaning. For the narrative and conceptual structures of visual representation in images, we examined visual happenings associated with transitivity for ideational meaning; credibility of images, use of gaze, light, 5

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

Table 1 SF-MDA Systems of Language and Image. Adapted from Jewitt et al. (2016) Mode

Metafunction

System

Explanation

Language

Ideational Interpersonal Textual

Participant, process, and circumstance Mood, modality, and appraisal network Theme and rheme

Representing reality Enacting social relations Organizing information

Image

Ideational Interpersonal Textual

Narrative and conceptual structures of Visual representation for participant, process, and circumstance Gaze, light, ambience of color, and credibility of images Layout, proportion, and alignment

Representing reality Enacting social relations Organizing information

and ambience of color for interpersonal meaning; and layout, proportion, and alignment for textual meaning. After considering the semiotic system of each separate mode, we then investigated the intermodality between the linguistic and visual complex. For the intermodal analysis, semiotic combinations of words and images were identified, and then intermodal relations for ideational, interpersonal, and textual meanings. For the ideational meaning, we looked at concurrence, complementarity, and connection relations. We identified jointly constructed feelings and relationships for interpersonal meaning, and relative placements and layouts for textual meaning. Finally, we turned our attention to how employed modes synaesthetically constructed the multimodal ensembles with the ideational, interpersonal, and textual meanings that were operating simultaneously. 4. Findings We present Michael’s orchestration of the linguistic and visual modes, and his understanding of modal resources and intermodal relations of semiotic systems of language and image, by examining his composing processes and produced texts (see online supplementary material for clause-level SFL analysis of language of expository and argumentative texts). 4.1. Composing expository multimodal text Composing process. The expository text, composed of four slides (see Fig. 1), explained the greenhouse effect, its effects on the earth, and its causes through linguistic and visual modes. We observed Michael constructing the first slide as a title page by adding a text box with the title “The Greenhouse Effect,” and then an image of an earth globe within a greenhouse that he found online to illustrate the title and multiply its meaning. The second slide (the upper right slide in Fig. 1) described the greenhouse effect with both words and images (e.g., globe, sun, atmosphere, space). Michael’s composing processes involved making a black background to depict outer space, and then inserting a gray gas ring as the atmosphere and an image of the globe that he found online. To explain the greenhouse effect from an authoritative stance, he made an effort to accurately depict natural phenomena. After adding a circle for the sun at the top left of the second slide, he inserted arrows to show the direction of the sun’s heat and two text boxes that explain the greenhouse effect with scientific terms that are domain-specific, according to CCSS. The linguistic explanations (i.e., “Atmosphere that is filled with CO2 and other gases traps heat and makes the earth warm” and “CO2 gas in atmosphere blocks sun being radiated into space after hitting the earth”) contained in the text boxes were an appropriation of what Ms. Johnson and the class discussed during the co-construction stage. That is, he emulated the wording of the class PowerPoint slides, although he not only omitted the definite article the in front of the words sun and atmosphere, but also conveyed scientifically incorrect information (i.e., “CO2 gas in atmosphere blocks sun being radiated into space after hitting the earth.”). In creating the third and fourth slides (left and right slides at the bottom in Fig. 1), we observed Michael first searching online for images1 of extreme weather and CO2 gas generation, inserting titles (i.e., “What does the greenhouse effect do to the earth?” and “What causes greenhouse gases?”) in the text boxes, and then adding the pictures he had gathered. Fig. 1 below presents his first draft of an expository PowerPoint text on the greenhouse effect. When he posted the first draft of his expository text on the class Edmodo for a feedback activity, he received a comment from Christopher, a reading group member: “Put arrows in the atmosphere like mine.” In response to Christopher’s comment, by transforming Christopher’s straight red arrows, Michael added straight and curved red arrows inside the gray gas ring in the second slide (the upper right slide in Fig. 2) of his revised text to show trapped heat in the atmosphere. He made changes on the first, third, and fourth slides by inserting his name and changing the pictures. Regarding the change of images in the third and fourth slides, Michael explained his revision of the first draft in an interview: Researcher: Why did you change from a sick earth image [in third slide] to this one [the picture of the earth in the green house]? Michael: Because the greenhouse picture is better. Mrs. Johnson told me this slide is about greenhouse effect. Researcher: Yeah. The title is “What does the greenhouse effect do to the earth?” You also changed the color of the gas ring. Why did you change it? Michael: Because it’s gas [I changed the color from orange to gray].

1

Michael’s use of images available online qualifies as fair use. 6

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

Fig. 1. Michael’s first draft of an expository PowerPoint text on greenhouse effect.

As noted in the above excerpt, Michael continued to make an effort to increase the accuracy and credibility of his explanation in revising the first draft. Through this composing process, Michael orchestrated linguistic and visual modes into a synaesthetic multimodal ensemble. The expository text explains the causes of the greenhouse effect and its effects on the earth, forming authoritative and formal relationships with readers through multimodal texts in PowerPoint, as seen in Fig. 2. In the next section, we shift our focus from the composing process to the product of composition by presenting the meanings of his expository PowerPoint text. We first analyze his use of individual semiotic modes (i.e., language and image) and then his use of intermodal relations and metafunctions among semiotic systems. The analysis in the following section will refer to the revised text displayed in Fig. 2. Use of modal resources of single mode. Words that Michael used across all four slides created ideational, interpersonal, and textual meanings. The first slide constructed its ideational meaning with a participant of a nominal group, “The Greenhouse Effect.” On the second slide, the language construed its ideational meaning through a transitivity system that entails participants (i.e., CO2 gas and atmosphere), material (i.e., block and trap) and relational processes (i.e., is), and circumstance (i.e., in atmosphere). The two sentences explained the greenhouse effect in declarative statements that contributed to formal relationships with readers. Both sentences introduced topical, unmarked, and unpredicated themes (i.e., “CO2 gas in atmosphere” and “Atmosphere that is filled with CO2 and other gases”) as a point of departure for what followed. The third and fourth slides introduced answers to the questions “What does the greenhouse effect do to the earth?” and “What causes greenhouse gases?” that Ms. Johnson posed to the class. The third slide title realized its ideational meaning through transitivity with a participant (i.e., the greenhouse effect) and a material process (i.e., do). The fourth slide title had a participant (i.e., what) and a causative relational process (i.e., causes). These title sentences employed the interrogative mood with “What,” and functioned as a proposition demanding information about the greenhouse effect for interpersonal meaning. Both sentences construed their textual meaning with a topical theme, “What.” Image, as a principle semiotic resource for all slides, has its own semiotic system for ideational, interpersonal, and textual metafunctions of a text. The first title slide constructed its ideational meaning representing the greenhouse effect with an image of the earth within a greenhouse that was spatially placed in the center of the page. The ideational meaning of the second slide was realized through the narrative structure for the transitivity system that included participants (i.e., sun and gas), action processes (material process in language) (i.e., yellow arrows to represent sunlight radiating toward the earth, and red arrows to represent the movement 7

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

Fig. 2. Michael’s revised expository PowerPoint text on the greenhouse effect.

of trapped heat) (Kress & van Leeuwen, 2006, p. 63), and circumstances (i.e., black background as space and gas ring as the atmosphere). These semiotic resources construed the narrative structure that describes the visual happenings and actions of the greenhouse effect. Regarding the conceptual structure of representation, symbolic and analytical structures were employed. The symbolic suggestive structure portrayed general features of the earth, and the analytical structure spatially depicted inter-image relations (e.g., gray gas ring and red arrows inside the ring) in relation to a whole image of the greenhouse effect. With regard to interpersonal meaning, the detailed scientific information about trapped heat (i.e., curved red arrows), the different colors of the drawings to illustrate elements of the greenhouse effect (e.g., CO2 gas ring, the sun, the earth, heat), and the blackness of space in the background were employed in an attempt to enhance the credibility of the text. The textual meaning was realized by the spatial composition that placed the earth and the trapped heat in the center and the sun at the top left corner, according to the importance of information. The third slide realized its ideational meaning through a narrative structure with a transitivity system that included an image of the earth in a greenhouse as a participant, images of extreme weather as values, and arrows as conversion processes (causative relational process in language) showing inter-event relations between the greenhouse effect and natural catastrophes (Kress & van Leeuwen, 2006, pp. 68–69). In terms of conceptual structure of representation, the symbolic attributive structure of specific images of catastrophes around the world (e.g., the burning globe, drought-stricken land) heightened the sense of danger from the intensifying greenhouse effect. For interpersonal meaning, photos of actual extreme weather catastrophes contributed to the credibility of the information in the text as a “reflection of the fidelity of representation with the natural world” (Unsworth, 2006, p. 67), and textual meaning was realized by organizing an image of the earth in a greenhouse in the center and arranging six weather images that have similar salience in terms of size and placement around the drawing of the earth. The fourth slide realized its ideational meaning—that factories, cars, a cow, and a refrigerator generate greenhouse gases—through the narrative structure of visual representation. Images (e.g., factory smog, a refrigerator) are participants, and a gray gas ring is an ascribed value in a causative relation. For conceptual structure of representation, the symbolic attributive structure for images accentuating the smog from factories and cars represents the seriousness of pollution. Photos of real pollution sources (e.g., image of factory smog) constructed credibility for interpersonal meaning (Kress & van Leeuwen, 2006; Unsworth, 2006), and its textual meaning was realized by spatial composition that put images conveying key information in the center (e.g., images of sources for greenhouse gas) inside the gas ring. 8

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

Use of intermodal relations and metafunctions between modes. The semiotic resources that Michael used for his multimodal explanation include intermodal relations between language and image. The language-image complex jointly constructed the ideational, interpersonal, and textual meanings of the multimodal text that explained the greenhouse effect and its causes. The language (i.e., “The Greenhouse Effect”) and image (i.e., the earth in a greenhouse) on the first slide presented the title, exhibiting ideational equivalence in a concurrence relationship that duplicated the ideational meaning without adding any new information. The language and image combination co-constructed a distant and formal stance without ambient visual effects for warmth, and organized a title text box and an image in a complementary layout by putting the title box above the image in a demarcated part of the slide. On the second slide, the intermodal relations constructed the ideational meaning of the text, by instantiating what the words conveyed regarding the greenhouse effect in the images and vice versa, in concurrence. As concurrence could entail some redundancy across modes rather than a simple intermodal duplication of meaning (Unsworth, 2006, p. 60), images (i.e., sun, arrow, globe, gray gas ring, and black background) and words constructed the ideational meaning by providing additional information. In particular, images explained the interconnections among the sun, atmosphere, and the earth, whereas the words in text boxes added information about specific gases (e.g., CO2) filling the atmosphere. In terms of interpersonal meaning, the language and image complex coconstrued a formal relationship for scientific specificity with scientific details. To this end, the text employed technical and disciplinary words (e.g., atmosphere, CO2, space), and detailed graphic images describing trapped heat and a gray ring for greenhouse gases. For the textual meaning of multimodal exposition, the language and image jointly formed a multimodal organization in an integrated layout, putting the images diagonally in the middle as a primary mode and arranging words around the images as a unified feature (Painter et al., 2013). On the third slide, the intermodal relation between language and image formed a complementarity relationship and augmented the ideational meaning of the text. The words (i.e., “What does the greenhouse effect do to the earth?”) presented a question and the images (e.g., drought-stricken field, flooded city, burning globe) provided answers to the question. For interpersonal meaning, the language-image complex enhanced the mostly accurate yet emotionally potent stance through use of a scientific question, photos, and a drawing of disastrous consequences resulting from extreme climate change; the images have comparable semiotic salience in terms of degrees of affect, realism in depiction, quantification, and size of image. The textual meaning of the language-image complex was created in a complementary layout, in that Michael arranged the words in a title box at the top and images in a main message box below. The language and image on the fourth slide realized its ideational meaning, forming a complementarity relation for ideational augmentation. That is, words (i.e., “What causes greenhouse gases?”) and images (e. g., photo of emissions from factories, drawing of polluting gas from cars) jointly constructed the ideational meaning in an interdependent relation as a question and an answer. The scientific accuracy and the interpersonal affect were constructed by combining a scientific question, a photo of real factory smoke, and symbolic drawings of sources for CO2 gas (e.g., polluting gas from cars). The textual meaning of the language-image complex was created in a complementary layout that arranged the words and images in separate demarcated spaces. 4.2. Composing argumentative multimodal text Composing process. Michael composed a multimodal argument for banning guns. He started to compose by brainstorming ideas about the selected topic in face-to-face and Edmodo class activities. When he posted the thesis of his argument, “I think guns should be illegal except for hunting and wars,” on the class Edmodo site, the teacher and peers provided him with feedback comments such as, “What are your reasons for banning guns?” and “Why do you want to have exceptions for banning guns?” When students started to compose their own argumentative essays, Ms. Johnson instructed the students to create a graphic organizer, saying that the organizer would help them to “write better.” Michael drew a graphic organizer with statements such as “What our topic is about” and “Valid Reasons” by appropriating the graphic organizer that the class created during the co-construction stage. Drawing on the comments, he filled the organizer and handwrote his argumentative essay by using only the linguistic mode, with the thesis and valid reasons such as making America safe and making him “feel much safer” (see online supplementary material for Michael’s handwritten argumentative essay). Michael initiated his remediating process by adding a US map as a background and drew a graphic organizer with the statements, “What our topic is about” and “Valid Reasons,” which he used in the graphic organizer for the first handwritten draft. After creating a graphic organizer, Michael revised his initial handwritten argument into a five-paragraph essay by deleting some of the content (e.g., types of guns). After he typed the five paragraphs as valid reasons, he included images that he found through Google to describe the linguistic essay. In an interview with Michael after completing the first draft of the Glogster text, he stated reasons for this change and for his inclusion of the visual mode in the Gloster text: Researcher: You deleted some sentences from the first draft. Why did you delete them? Michael: Because I said them before. I don’t need them again. Researcher: Right. you put the thesis statement in the first sentence. You added six pictures in your draft. Where did you find them? Michael: Google. Researcher: Why did you want to put these images in the essay? Michael: Because they explain what I wrote. Michael used visual resources (e.g., images, graphics) to depict the monomodal alphabetic essay. As stated in the last interaction with the researcher, he used images to illustrate what he wrote in the linguistic mode for his Glogster text. 9

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

Fig. 3. Michael’s first multimodal argumentative text.

When the students exchanged feedback on the first drafts of their Glogster texts, Ms. Johnson commented regarding Michael’s Gloster draft (see Fig. 3), “Images should not be used as an illustration of your writing.” His classmate suggested deleting the sentence “What our text is about” in the top right corner by commenting, “You don’t need what our topic is about.” Drawing on the received feedback comments, he revised the initial draft by deleting two images (i.e., hunting and war pictures) and the unnecessary notes, and rearranging images. Through this revising process, Michael construed the final draft of his multimodal argumentative text (see Fig. 4). Use of modal resources of single mode. Michael’s Glogster-mediated multimodal argumentative essay included linguistic and visual modes. Words as a main mode created the ideational, interpersonal, and textual meanings of his multimodal argumentative text with its own semiotic system (e.g., transitivity, mood, modality, theme and rheme). He constructed the ideational meaning with

Fig. 4. Michael’s revised multimodal argumentative text. 10

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

mental process (i.e., think), relational process (i.e., be), and a propositional circumstance (i.e., except for hunting and wars), to maintain the thesis, “I think guns should be illegal except for hunting and wars,” as a thinker. Stating the reasons for his thesis, he utilized material processes (e.g., “go,” “kill,” “make”) and presented participants as actors to show that those who possess guns illegally will “go to jail” for a lifetime, whereas those who keep his proposed regulation will contribute to making him “feel much safer.” Through use of mental process (e.g., “feel”), he emphasized the thesis statement. In other words, he repeated his claim as a thinker. Michael construed the interpersonal meaning of the importance and necessity for banning guns, drawing on declarative mood and modulation (i.e., “should”). The use of a modulated modality “should” realized a high-level imperative relationship with readers. He also used multiple topical, unmarked, and unpredicated themes (e.g., “I”, “guns,” “someone,” “America”) for the textual meaning of his argument. The visual mode had its own semiotic system that realized ideational, interpersonal, and textual meanings. The images (i.e., nogun signs, smiling children in front of a school bus, and USA flag and map) construed the ideational meaning with the narrative structure of banning guns in the US for a safe America and for happy children. For the conceptual structure of representation, the symbolic attributive structure depicted an image of smiling children in front of a school bus, and the symbolic suggestive structure represented the general features of guns. Regarding interpersonal meaning, the two no-gun sign images conveyed an imperative statement to readers for banning guns as well as his personal stance toward guns. The images construed the textual meaning with a backgrounded image (i.e., U.S. map) and foregrounded the images that carried key information around the text box. Use of intermodal relations and metafunctions between modes. The linguistic and visual mode complex constructed intermodally ideational, interpersonal, and textual meanings. For the ideational meaning of banning guns, the language and image complex represented primarily a concurrence relationship in the first draft of the Glogster text (Fig. 3), but the revised text employed a complementarity relationship. In the first draft, Michael employed the visual mode as an instantiation of the linguistic mode, by using images primarily as an illustration of words, although the images suggested additional meanings about hunting and warfare (e.g., hunter, types of gun). While revising the first draft in response to Ms. Johnson’s feedback, Michael deleted the images of hunting and war to focus his argumentative essay on banning guns for America’s safety. In an interview with him after completing a revision, he explained as follows: Researcher: Your first draft had six images, and you deleted two of them. Do you like these four? Michael: Yes. Researcher: Why did you delete the hunting and war images? Michael: I don’t need those pictures. I talked about guns for hunting and wars in my writing. Researcher: I see. As Mrs. Johnson said, images should not be used only to illustrate texts. Right? Michael: Yes. Researcher: You moved the school bus and the “save the children” images to the left. Why did you put these two in the same side? Michael: I wanted to put them together because they are about school and children. Researcher: Oh. You made a statement about school safety on the left side. What about the right side? Michael: America. Researcher: Oh. I see. You are mentioning school safety with these pictures on the left and America’s safety with the picture on the right side. As seen in Michael’s answer to a question about deleting images from the first draft of his Glogster text, he deleted the hunting and war images, comparing them with the words that were written in the text box. His attempt to construct the ideational meaning in a complementarity relation made use of the visual and linguistic modes of the revised text in a way that enabled one mode to add new information to the other. The words expressed his argument—banning guns except for hunting and wars so that “I would feel much safer”—while images (e.g., no-gun sign with the save the children text, smiling children) thematically provided the information about whom “I” indexed. The image-word complex co-constructed the ideational meaning of the text forming ideational augmentation. For interpersonal meaning, the semiotic combination between language and image construed an authoritative relationship with readers in conveying Michael’s argumentative proposition. Michael employed an imperative mood with his use of a modulated modality “should” for the linguistic mode, and the images of no-gun signs with a red-cross negative symbol for the visual mode. The semiotic combination of the “Save the Children” text and image intensified intermodally a strong imperative stance for banning guns to make America safe. In addition, the sentence “I would feel much safer” and the image of smiling children in front of a school bus co-constructed the urgency of banning guns to make America safe for children in a high level of emotional appeal. In constructing the textual meaning, Michael displayed the linguistic mode as a main mode by putting a text box in the center of the page, while placing images around the box. The intermodal organization of images and words was based on a complementary layout that placed both modes in demarcated spaces on the page (Painter et al., 2013). 5. Discussion The findings show that Michael was able to produce multimodal ensembles orchestrating linguistic and visual modes among the available modes in PowerPoint and Glogster. He utilized the visual mode for ideational meaning to construct the main ideas of the texts, and employed concurrence and complementarity relations between language and image. These findings provide insights to Michael’s developing metalanguage for multimodal composing. We critically review his metalanguage development, focusing on mode choices and intermodal resources. The multimedia tools, PowerPoint and Glogster, offered a range of meaning-making resources from word, image, and music to 11

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

video for writers to compose texts. However, the linguistic and visual modes provided more affordances for Michael than other modes for his creation of expository and argumentative texts. When he was asked about his choice of language and image over other modes, he stated, “I want to write like books,” and “They have texts and pictures.” In light of this, Michael’s semiotic choices for composing within the new multimedia were shaped by the dimensions of older print media. He adopted a conventional print-based writing norm that has been valued in schools, even when he was encouraged to use new non-traditional modes and motivated to write. This kind of composition involving both old and new media is referred to as a “return” stance and commonly evidenced in new literacy practices (Leander, 2009, p. 147). For instance, Ware’s study of English learners’ multimedia literacy (Ware, 2008) corroborates our findings in that the students produced PowerPoint multimodal texts with language and images for information display and summary, employing writing norms within the print medium, despite their appreciation of multimedia composing in multimedia mediums. The return stance regarding meaning-making practices in a newer medium (e.g., Glogster) was also instantiated in Michael’s multimodal composing, given that he was using features of more familiar media (e.g., books) while remediating texts into newer media (Bolter & Grusin, 1999). While employing linguistic and visual modes as main meaning-making resources, Michael utilized non-linguistic visual resources to convey the main ideas of a multimodal ensemble. For his expository writing, he distributed meanings of the PowerPoint slides about the greenhouse effect across linguistic and visual modes with a growing understanding of the modal and intermodal resources of the modes. Michael utilized images as a primary mode to create the ideational meaning of the text beyond the interpersonal one on the third and fourth slides. As such, he did not privilege linguistic resources as the carriers of ideational meaning over visual modes, avoiding uses of images mainly as an interactive hook for interpersonal meaning, which is often evidenced in children’s multimodal texts (Shanahan, 2013; Shin, 2018). Michael showed developing awareness of the semiotic systems of linguistic and visual modes, which could become his metalanguage of the communicative potentials of semiotic systems, leveraging his L2 writing development (Belcher, 2017). The intermodal relations that Michael used varied depending on the mediums of the expository and argumentative texts. In the PowerPoint-mediated expository text, specifically for the third and fourth slides, the language and image complex constructed the ideational meaning, with additional information provided by each mode that was complementary to what was offered in the other mode. In case of the medium of Glogster, the linguistic and visual semiotic combination provided redundant information across the modes. When we asked Michael why he put new information in both words and images into the expository PowerPoints text, unlike the argumentative Glogster text, Michael stated “I don’t know. PowerPoints are like that.” As the statement suggests, Michael’s multimodal composing was closely associated with his perceptions of a medium of composition and its genre text. Michael created his PowerPoint and Glogster texts by following the existing texts in those mediums to which he was exposed during the deconstruction and joint-construction stages and in other classes. In this, Cimasko and Shin (2017) have shown that when learners are introduced to composing in new mediums, they tend to emulate the previously known genre of writing within the medium. Michael’s composing processes showed that he was developing the metalanguage of multimodality, vacillating in a non-linear way between composing a multimodal ensemble and making an illustration. In particular, this was evidenced in his construction of ideational meanings in both texts. In his expository writing, he constructed meanings in heightened intermodal awareness between words and images using a complementarity relation, which highlights the synaesthetic nature among the modes of a multimodal ensemble (Kress, 2003). That is, he used language to ask questions (i.e., “What does the greenhouse effect do to the earth?” and “What causes greenhouse gases?”) and effectively used a variety of images (e.g., extreme weather, smog) to answer them, offering complementary meanings with those provided in the other mode for ideational augmentation. However, in the subsequent argumentative essay, Michael used images (e.g., no-gun signs) as an illustration of words drawing on a concurrence intermodal relation, despite Ms. Johnson’s advice that “images should not be an illustration of words,” which he was reminded of throughout both units. Michael’s employment of a complementarity intermodal relation between words and images was just an instantiation rather than a deliberate choice from his developed metalanguage for multimodal composing. 6. Conclusion This study investigated the multimodal composing of Michael, a sixth-grade L2 writer, focusing on modal resources and intermodal relations of language and image. Findings of the study provide the field of L2 writing with timely insights into the development of metalanguage for multimodal composing, helping educators to be better prepared for teaching students how to comprehend and compose multimodal texts for effective communication for the 21st century. In addition, the study expands existing research methods on multimodal composing by examining multimodal composing processes as well as products with longitudinal ethnographic data through SF-MDA (Jewitt et al., 2016) and sociosemiotic ethnography (Iedema, 2001; Prior, 2013). Given that diverse forms of multimodal texts (e.g., print-based posters, digital slide shows, live oral storytelling) are increasingly used across contexts in school and beyond, it is critical that L2 learners develop multimodal communicative competence to select apt modal resources and orchestrate different modes in various intermodal relations for meaning (Jewitt, 2009; Royce, 2007). The study has implications for future research and pedagogy on multimodal composing. Future research can investigate L2 writers’ composing that (re)mixes a range of genres, modes beyond familiar linguistic and visual ones, and various mediums. Such an investigation would shed light on students’ appropriation of those unfamiliar modes for multimodal-meaning making practices, as well as their understanding of semiotic systems of the modes and intermodal relations across the modes. Moreover, studies can explore translanguaged multimodal composing in which learners flexibly use their first and second languages and dialects with other modes for language learning and/or self-expression. Fluid translanguaging was not evidenced in Michael’s multimodal composing since ELA classes—even bilingual classes (García, 2009)—support textual production in a single target language in most public 12

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

schools in the United States. Future research on translanguaged multimodal composing can explicate how L2 writers become agentive meaning-makers through exploitation of all available semiotic resources. Methodologically, complementing text analyses of SF-MDA with an ethnographic approach, more L2 research can show how L2 writers develop metalanguage for multimodal composing over longer stretches of time, with thick descriptions of composing processes at the micro-textual and the macro-discursive levels. This kind of research will contribute to understanding the complexity of L2 learners’ multimodal composing and their ontogenetic development of multimodal semiotic knowledge over time and across contexts. For effective pedagogy on multimodal composing, it is critical for teachers to keep curricular and instructional practices that promote L2 writers’ development of metalanguage for multimodal composing. Teachers can offer explicit instruction on multimodality to bridge old and new composing practices within various mediums across multiple contexts. In this, they can apply a teaching-learning cycle that progresses from deconstruction through joint construction to independent construction, providing students with scaffolding on various combinations of different modes and their meaning potentials as semiotic systems (Rose & Martin, 2012; Unsworth, 2006). Given that quality multimodal instruction necessitates knowledge of multimodal composing, it is necessary for teachers to develop expertise in evidence-based and well-designed curricula with an open-mind about new writing practices (Yi, 2017). This kind of multimodal composing pedagogy could provide opportunities for L2 learners to develop a level of multimodal communicative competence that is essential to increasingly multilingual and multimodal communications in and out of school. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Appendix A. Supplementary data Supplementary material related to this article can be found in the online version, at https://doi.org/10.1016/j.jslw.2020.100714. References Ajayi, L. (2009). English as a second language learners’ exploration of multimodal texts in a junior high school. Journal of Adolescent & Adult Literacy, 52, 585–595. Barton, D., & Potts, D. (2013). Language learning online as a social practice. TESOL Quarterly, 47, 815–820. Belcher, D. (2017). On becoming facilitators of multimodal composing and digital design. Journal of Second Language Writing, 38(4), 80–85. Bolter, J., & Grusin, R. (1999). Remediation: Understanding new media. Cambridge, MA: MIT Press. Calkins, L. (2013). A guide to the common core writing workshop intermediate grades. Portsmouth, MH: Heinemann. Cimasko, T., & Shin, D. (2017). Multimodal resemiotization and authorial agency in an L2 writing classroom. Written Communication, 34(4), 387–413. Denzin, N. K., & Lincoln, Y. S. (Eds.). (2003). The landscape of qualitative research: Theories and issues(2nd ed.). Thousand Oaks, CA: Sage. Dyson, A. H. (1993). Social worlds of children learning to write. New York, NY: Teachers College Press. Eggins, S. (2004). An introduction to systemic functional linguistics. London, UK: Bloomsbury. García, O. (2009). Bilingual education in the 21st century: A global perspective. Malden/Oxford: Wiley/Blackwell. Gebhard, M. (2019). Teaching and researching ELLs’ disciplinary literacies: Systemic functional linguistics in action in the context of U.S. school reform. New York, NY: Routledge. Hafner, C. (2015). Remix culture and English language teaching: The expression of learner voice in digital multimodal compositions. TESOL Quarterly, 49(3), 486–509. Halliday, M. A. K., & Matthiessen, C. M. I. M. (2014). An introduction to functional grammar. London, UK: Hodder Arnold. Harman, R. (Ed.). (2018). Bilingual learners and social equity: Critical approach to systemic functional linguistics. New York, NY: Springer. Iedema, R. (2001). Resemiotization. Semiotica, 137(1), 23–39. Jewitt, C. (Ed.). (2009). The Routledge handbook of multimodal analysis. London, UK: Routledge. Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing multimodality. New York, NY: Routledge. Jiang, L. (2017). The affordances of digital multimodal composing for EFL learning. ELT Journal, 71(4), 413–422. Kress, G. (2003). Literacy in the new media age. London, UK: Routledge. Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London, UK: Routledge. Kress, G., & van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. New York, NY: Oxford University Press. Kress, G., & van Leeuwen, T. (2006). Reading images: A grammar of visual design. London, UK: Routledge. Leander, K. (2009). Composing with old and new media: Toward parallel pedagogy. In V. Carrington, & M. Robinson (Eds.). Digital literacies: Social learning and classroom practices (pp. 147–164). London, UK: Sage. Martin, J. R., & Rose, D. (2008). Genre relations: Mapping culture. London, UK: Equinox. Martin, J. R., & White, P. R. R. (2005). The language of evaluation: Appraisal in English. New York, NY: Palgrave. Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: The Jossey-Bass. Moll, L., Amanti, C., Neff, N., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory Into Practice, 31(2), 132–141. National Governors Association Center for Best Practices, Council of Chief State School Officers (2010). Common core state standards for English language arts and literacy in history/social studies, science, and technical subjects. Washington, DC: Author. Nelson, M. E. (2006). Mode, meaning, and synaesthesia in multimedia L2 writing. Language, Learning, & Technology, 19(2), 56–76. New London Group (1996). A pedagogy of multiliteracies: Designing social features. Harvard Educational Review, 66(1), 60–92. NGSS Lead States (2013). Next generation science standards: For states, by states. Washington, DC: National Academies Press. Painter, C., & Martin, J. R. (2011). Intermodal complementarity: Modelling affordances across image and verbiage in children’s picture books. In F. Yan (Ed.). Studies in functional linguistics and discourse analysis (pp. 132–158). Beijing, China: Education Press of China. Painter, C., Martin, J. R., & Unsworth, L. (2013). Reading visual narratives: Image analysis of children’s picture books. London: Equinox. Prior, P. (2005). Moving multimodality beyond the binaries: A response to Gunther Kress’ “Gains and Losses”. Computers and Composition, 22, 23–30. Prior, P. (2013). Multimodality and ESP research. In B. Paltridge, & S. Starfield (Eds.). The handbook of English for specific purposes (pp. 519–534). Blackwell-Wiley Press. Rose, D., & Martin, J. (2012). Learning to write, reading to learn: Genre, knowledge and pedagogy of the Sydney school. Oakville, CT: Equinox. Royce, T. (2007). Multimodal communicative competence in second language contexts. In T. Royce, & W. Bowcher (Eds.). New directions in the analysis of multimodal discourse (pp. 361–390). New Jersey: Lawrence Erlbaum Associates, Inc.

13

Journal of Second Language Writing 47 (2020) 100714

D.-s. Shin, et al.

Schleppegrell, M. (2004). The language of schooling. Mahwah, NJ: Lawrence Erlbaum Associates. Shanahan, L. E. (2013). Composing “kid-friendly” multimodal text: When conversations, instruction, and signs come together. Written Communication, 30(2), 194–227. Shin, D. (2018). Multimodal mediation and argumentative writing: A case study of a multilingual writer’s metalanguage awareness development. In R. Harman (Ed.). Bilingual learners and social equity: Critical approaches to systemic functional linguistics (pp. 225–242). New York, NY: Springer. Shin, D., & Cimasko, T. (2008). Multimodal design and second language composition: New tools, traditional norms. Computers and Composition, 25(4), 376–395. Smith, B., Pacheco, M., & de Almeida, C. R. (2017). Multimodal codemeshing: Bilingual adolescents’ processes composing across modes and languages. Journal of Second Language Writing, 36, 6–22. Street, B. (1984). Literacy in theory and practice. Cambridge, UK: Cambridge University Press. Toohey, K., Dagenais, D., Fodor, A., Hof, L., Nunez, O., Singh, A., et al. (2015). “That sounds so Cooool”: Entanglements of children, digital tools, and literacy practices. TESOL Quarterly, 49(3), 461–485. Troyan, F. J., Sembiante, S. F., & King, N. (2019). A case for a functional linguistic knowledge base in world language teacher education. Foreign Language Annals, 52(3), 644–669. Unsworth, L. (2006). Towards a metalanguage for multiliteracies education: Describing the meaning-making resources of language-image interactions. English Teaching: Practice & Critique, 5(1), 55–76. Ware, P. (2008). Language learners and multimedia literacy in and after school. Pedagogies: An International Journal, 3, 37–51. Yi, Y. (2017). Establishing multimodal literacy research in the field of L2 writing: Let’s move the field forward. Journal of Second Language Writing, 38(4), 90–91. Yi, Y., & Angay-Crowder, T. (2016). Multimodal pedagogies for teacher education in TESOL. TESOL Quarterly, 50(4), 988–999. Yi, Y., Shin, D., & Cimasko, T. (2019). Multimodal literacies in teaching and learning English in and out of school. In L. de Oliveira (Ed.). The handbook of TESOL in K-12 (pp. 163–177). New York, NY: John Wiley & Sons, Ltd. Dong-shin Shin is an assistant professor in the Literacy and Second Language Studies program of University of Cincinnati. She has been pursuing research into digital literacy, multimodal composing, disciplinary literacy in content areas, and L2 teacher professional development. Her work has appeared in Computers and Composition, Language and Education, Language Learning & Technology, CALICO Journal, The Urban Review, Written Communication, and others. Tony Cimasko is the ESL Composition coordinator in the Department of English at Miami University in Oxford, Ohio, teaching graduate courses on L2 writing theory and pedagogy as well as undergraduate and graduate second language writing courses. His research interests include multimodal composition, professional and pedagogical genres, and feedback practices. His work has been published in the Journal of Second Language Writing, Computers and Composition, English for Specific Purposes, Written Communication, and others. Youngjoo Yi is an associate professor in foreign, second, and multilingual language education at the Ohio State University. Her research interests include linguistically and culturally diverse students’ practice and learning of multilingual and multimodal literacy and their identity construction. She is a co-editor of TESOL Journal, and her work has been published in TESOL Quarterly, Journal of Second Language Writing, Foreign Language Annals, Canadian Modern Language Journal, and others.

14