Using focus groups to investigate study abroad theories and practice

System xxx (2017) 1e11

Contents lists available at ScienceDirect

System journal homepage: www.elsevier.com/locate/system

Using focus groups to investigate study abroad theories and practice Paula Winke Michigan State University, United States

a r t i c l e i n f o

a b s t r a c t

Article history: Received 10 September 2017 Accepted 11 September 2017 Available online xxx

In this paper, I review 10 qualitative and mixed-methods research articles on study abroad that employed focus groups. I look specifically at the quality of the focus group reporting, which I evaluated by comparing the methods reported on in the studies against focus group reporting criteria described by Krueger and Casey (2015). I generally found that applied linguists and international education researchers who investigate study abroad have yet to take full advantage of focus groups. Most studies lacked clear focus group descriptions. Missing was information on the directions, who participated, who the moderator was, what the script or questions were, and when the focus groups were held. On the other hand, the 10 studies collectively demonstrate the strong potential of focus groups to help applied linguists better understand theories and hypotheses about study abroad. In particular, focus groups have an advantage in helping researchers understand how students develop interculturality during study abroad, and how various social factors, such as social media use and host-family contact, impact study abroad experiences and language learning. Study abroad researchers should learn more about focus groups and their specific advantages to increase study robustness, and to build a stronger theoretical basis for study abroad's continued use. © 2017 Elsevier Ltd. All rights reserved.

Keywords: Qualitative methods Focus groups Study abroad

1. Introduction Focus groups are a tool used by social and behavioral scientists to understand opinions, motivations, attitudes, and thought processes that underlie behaviors surrounding certain societal objects or specific events (Krueger & Casey, 2015; Stewart & Shamdasani, 2015). More broadly, a focus group may be viewed as a special type of interactive group discussion (Ho, 2012). This method for qualitative data collection within applied linguistics has grown and expanded in the field recently, especially when the societal object or event can be identified as having a large impact or influence on behaviors (including learning behaviors), such as study abroad (DeKeyser, 2014). Study abroad researchers have successfully used focus groups to understand students' choices and decisions regarding
& Cots, 2016); how study abroad changes study abroad (Doyle, Gendall, Meyer, Hoek, & Tait, 2010; Foster, 2014; Martin-Rubio students' perceptions on culture, learning, and the world (Bacon, 2002; Ching, Lien, & Chao, 2014; Forsey, Broomhall, & Davis, 2012; McLeod & Wainwright, 2009; Medina, Hathaway, & Pilonieta, 2015); and how study abroad contributes to the language-development process (Holmes, Bavieri, Ganassin, & Murphy, 2016; Zamastil-Vondrova, 2005). But thus far, there has been no succinct review of how focus groups have been (and can be) used to support and enhance applied linguists’

E-mail address: [email protected]. https://doi.org/10.1016/j.system.2017.09.018 0346-251X/© 2017 Elsevier Ltd. All rights reserved.

Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

2

P. Winke / System xxx (2017) 1e11

research on study abroad. This review is necessary because both qualitative study abroad research and focus group methodology are of age. How well the data collection tool (focus groups) is used by researchers investigating this robust educational phenomenon (study abroad) needs empirical investigation so that applied linguists can check whether they are using the tool well, and whether they are using it to its full potential. In this paper, I first review focus groups in the general tradition of social science research because focus groups are a data collection tool that applied linguists have borrowed from the social science fields of marketing and business management. Second, I provide the procedures of this descriptive research report in which I and a graduate research assistant reviewed 10 recent applied linguistics or international education research studies in which the authors used focus groups to investigate various aspects of study abroad. (She and I coded the 10 empirical research studies for focus group reporting quality.) Third, I summarize the quality of the focus-group reporting in the studies. And finally, I outline the strengths of these papers, and also review how the researchers could have given better information on their focus group designs and procedures. 2. Focus groups as a research tradition Focus groups have been used for a long time in both public and private sector research, and they have a particularly strong basis in the behavioral sciences (Stewart & Shamdasani, 2015) and in marketing research (Parker & Tritter, 2006; Smithson, 2000). What is unique with a focus group, as opposed to series of one-on-one interviews or even a group interview, is that a focus group uses group interaction as a direct data collection method (Kitzinger, 1994; Stewart & Shamdasani, 2015, pp. 10e11). 2.1. Definition of focus groups in social science The following descriptions encompass the nature of focus groups in social and behavioral science research: 1. A focus group is a “special type of group in terms of purpose, size, composition and procedures. The purpose of a focus group is to listen and gather information. It is a way to better understand how people feel of think about an issue, product, or service. Focus groups are used to gather opinions” (Krueger & Casey, 2015, p. 2). 2. A focus group has “the primary aim of describing and understanding perceptions, interpretations, and beliefs of a select population to gain understanding of a particular issue from the perspectives of the group's participants” (Kahn & Manderson, 1992, p. 57). Individuals who share specific characteristics in common are recruited for focus groups, such as all individuals participating in the same summer study abroad program (Krueger & Casey, 2015). One defining characteristic of a focus group is the list of open-ended questions that were carefully designed to explore a topic about which little is known (Krueger, 1998; Stewart & Shamdasani, 2015). For example, if one of the questions the researcher has about the program is whether homestays have advantages over dormitory living, some of the open-ended questions during the focus group may zero in on this topic by asking the participants to “talk about their living situations during the study abroad program,” or to “discuss where they would advise new students to live during the same study abroad program next year.” The object is not for the group to come to a consensus or make decisions on which is better or should be offered in the future, but rather the goal is for the researcher to be able to use the discussion (or transcript of it) to gauge individuals’ perceptions on the topic, so that the researcher can understand the issues in a nuanced way, only making decisions or recommendations if that is the goal of the larger research project (Krueger & Casey, 2015; Morgan & Kreuger, 1993). 2.2. General parameters for describing focus groups Focus groups in the larger fields of social and applied research have a rather defined structure. Krueger and Casey (2015) outlined major features of a robust focus group for applied research. The definitions and parameters were and are prescriptive in nature, and some researchers have commented that not all focus groups will adhere to all specifications (Turney & Pocknee, 2005): The suggestions are just that; the criteria will not always apply to all focus group research (Morgan & Kreuger, 1993). Nevertheless, Krueger and Casey (2015) recommended that a focus group should (1) be comprised of people (plural) who will give the researcher the type of information needed, (2) be serial or sequential (involve several groups) to avoid generalizations influenced by outlying views, (3) involve a group of people with shared experiences (who are homogeneous, as defined by parameters established by the researcher) but who do not know the other group members too well, which will encourage uninhibited expression on a topic or experience, (4) be used for robust data collection that (5) produces data that are extremely qualitative in nature, and (6) be a planned and focused group discussion of the topic at hand by having a trained moderator who uses prepared scripts of the guidelines for participation and the open-ended questions. The parameters for focus groups outlined by Krueger and Casey (2015, pp. 6e8), and commented on by Turney and Pocknee (2005, pp. 36e39) are summarized in Table 1. In the final column of Table 1 is a list of ways in which authors can report on the focus group parameters in their research studies. For example, researchers should document and disclose the number of people who participated in each of the focus Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

P. Winke / System xxx (2017) 1e11

3

Table 1 Descriptions of focus group parameters and reporting suggestions for focus group researchers. Parameter

Description

Empirical research reporting suggestions





2. Focus groups are serial Multiple or repeated groups optimize pattern detection and
or sequential. increase the reliability and validity of results. Multiple groups ward off findings from outliers (generalizations stemming from extreme views).
1. Focus groups involve people (plural).

The group of people is small enough to share insights, but large enough so the group can discuss a variety of views.

3. Participants are homogeneous and unfamiliar w/one another.

The participants come from (or are stakeholders in) the social
phenomenon or object of study. They share a relationship with the study object. Unfamiliarity with each other allows
them to speak uninhibitedly.


4. Focus groups are for The participants do not make decisions, recommendations, or
robust data collection. reach a consensus. Focus groups have a narrow purpose and
focus on attitudes and perceptions. 5. Focus groups make use Focus groups obtain qualitative insights into attitudes, of qualitative data. perceptions, and opinions. Discussions lead to saturation points, where the flow of new information ceases.





6. A focus group is a focused discussion.


Participants enter to contribute views. There should be no distractions, no diversions. The moderator poses topics, observes, listens, and inserts probes. Participants interact and share their views with each other (not with the moderator).


The number of focus groups When and where they were conducted How long they lasted (& # of words) The number of people in each focus group In repeated focus groups (e.g., pre-study abroad, post study abroad), whether the participants were the same or different In cross-sectional focus group, whether the participants were from the same homogeneous group or if they had different backgrounds/characteristics Full description of participants (ages, learning experiences, etc.) and their relationship with the object of study Information on whether they know each other, and if yes, how and for how long For complex relationships, a map, showing who is familiar with whom Information on the trained moderator who conducted the focus group(s) Reproduction of the script of the open-ended questions outlining the focus group topic or topics for exploration The type of qualitative study (i.e., if mixed: concurrent embedded, sequential transformative), determined ahead of time The qualitative data analysis plan, determined ahead of time List of the “focus group rules” that inform the participants that responding or withholding is voluntary (to set a noncoercive, democratic environment) Transcript evidence that the moderator had little steering or intervention to do (The interactive discussion was amongst the participants.)

Notes. The parameters and descriptions are summaries from Kruger's 1988, 1994 work, distilled by Turney & Pocknee, 2005 review (pp. 36e39) of Kruger's work. I added the reporting suggestions.

group sessions, and report where and when they were conducted, and outline how long they lasted. If the author is extremely exact, he or she could also indicate the number of words spoken across the focus groups or by the subgroups within the focus groups, and how many of those words were spoken by the participants as compared to the focus group's moderator. For
n and Bülow (2003) described a focus group session in which the topic was contributors to workplace burnout. example, Hyde They had three different high-burnout-rate professions represented in the focus group: general practitioners (physicians), teachers, and social workers. The researchers provided a table (p. 310) containing the number of participants who belonged to each of the three professions, and then noted the distribution of the focus group discourse across the three groups in terms of the length of conversation time (in minutes), number of words, number of turns, average number of words per turn, number of pauses between turns, sum of the length of pauses between turns (in seconds), and average length of pauses between turns (in seconds). Data analysis tools like NVivo (http://www.qsrinternational.com/nvivo-product) and ATLAS.ti (http://atlasti. com/) are invaluable in providing detailed data reports. Not all studies need extremely exact focus-group language descriptions, but it is important that researchers who use focus groups know how data collected from them are used and how the findings are (or can be) described and displayed in empirical research studies. Morgan and Kreuger (1993) reminded focus group researchers, however, to “always keep the fundamentally qualitative purpose of focus groups firmly in mind” (p. 11); that is, they warned that quantifying focus group data is only occasionally useful. 2.3. General procedures for conducting focus groups Researchers in social science and applied linguistics have provided practical suggestions on how to set up and conduct focus group sessions. For example, within the field of applied linguistics, Ho (2012) outlined the following steps and procedures, which I elaborate on by referring to other researchers in social science who have also made similar suggestions: 2.3.1. Select the respondents The participant composition is crucial for focus group investigations (Flores & Alonso, 1995). Ho (2012) recommended inviting “information rich” focus-group participants who have excellent insight on the topic (Ho was referencing an earlier edition of Krueger and Casey’s 2015 work), and also trying to assemble as diverse a group as possible (mixed genders, ages, etc.) from within the otherwise homogeneous population to capture the possible range of varying views. Ho suggested “six to twelve” (p. 2) people; more recently, Krueger and Casey (2015, p. 2) recommended groups with 5 to 10 people, with no less Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

4

P. Winke / System xxx (2017) 1e11


n and Bülow (2003) summarized prior theorists' work (Goffman, 1961; than 4 and no more than 16 at any given time. Hyde Harre, 1984) stating that the participants can theoretically be (a) an aggregation of individuals who share social features or experiences (e.g., students who have returned from study abroad), (b) a small group that has shared values or who conform to a certain set of norms (e.g., teachers who believe study abroad is good), or (c) a focused gathering of people who agree to discuss a single topic with great focus (e.g., educators brought together to talk about study abroad at the college level). 2.3.2. Prepare the “interview guide” or script The interview guide should comprise a series of open-ended questions that set the agenda for the group's discussion. Ho (2012) noted that general, unstructured questions usually come first, and more specific questions, if present, should come later once the group discussion has had time to develop. Ho recommended researchers to pilot the interview guide to determine if the wording is appropriate and easily understood. Delving deeply into the topic of creating robust focus group questions, Krueger (1998), in the third volume out of six in a Sage Publications series on focus group interviewing, recommended that researchers should phrase and sequence questions carefully. Krueger explained that researchers should create probe and follow-up questions to delve deeper into the discussions and to uncover unplanned discussions that lie underneath the general topic area. The guide or script can be supplemented with (or even fully comprised of) pictures, movie clips, or any type of prompt that can be used to generate a discussion among the participants. Researchers need not be limited by text-only questioning. 2.3.3. Select the focus group moderator The moderator plays a key role in the success of the focus group. He or she establishes “the common communicative
n & Bülow, 2003, p. 311) and should be empathic, courteous, and regard the focus group participants as ground” (Hyde knowledgeable and wise (Krueger & Casey, 2015, p. 104). He or she should respectfully facilitate a discussion among the participants and allow the discussion to flow. As Krueger and Casey explained, it is more important to have the right moderator than to have the lead researcher on the study be the moderator: Thus, the lead researcher should have the moderator understand the study's main research questions and be involved in the preparation and piloting of the interview guide so that he or she is extremely familiar and comfortable with the topic and understands fully the information needed. In other words, the moderator “must understand enough about the topic to know what type of information will be most useful to the study” (Krueger & Casey, 2015, p. 104). During the discussion, the moderator can “encourage different members to speak,” (Smithson, 2000, p. 108), especially if certain individuals are silent but appear to want to speak. Parker and Tritter (2006) noted that the moderator (or facilitator, as they called the role), could be a pair of individuals, or the moderator could be accompanied by an observer who records supplemental (observational) data during the discussion. There are different approaches to selecting a suitable moderator: one who is from a background similar to the focus group participants (Smithson, 2000), a professional trained in focus group procedures (Ho, 2012), or an individual from the research team. The latter option is particularly warranted “when there is a real need for a moderator who has a detailed familiarity with either the project goals or the participants' point of view” (Morgan & Kreuger, 1993, p. 3). 2.3.4. Determine the setting for the focus group Researchers have decisions to make regarding where the focus group will be conducted and how the participants will be arranged in the room (it is important to note, however, that a growing number of focus groups are being conducted online). Recommendations for room configuration include setting individuals in a circle or semi-circle to facilitate the group discussion with the individuals wearing numbers going sequentially around the room or nametags with their participant number or a pseudonym. The purpose of using numbers or pseudonyms is to preserve the confidentiality of participants and to make data transcription easier. In recent focus groups that I was involved in facilitating, we trained the focus group participants to say their participant number when they started to talk. This was particularly helpful because we had large numbers of participants (20e35) and only audio-recorded the sessions. While this number-stating practice helped with transcription, we wondered if this initial conversation requirement affected the free-flow of discussion during the focus group. One possible solution to this would be to videotape the focus group discussion for later transcription and analysis. As mentioned above, in lieu of face-to-face meetings, researchers can conduct focus groups using internet-based groupdiscussion software such as Blackboard (http://blackboard.com/), Skype (https://www.skype.com), or Zoom (https://zoom. us/), which have internal recording options (audio or video-based) that are convenient for researchers to use. Online focus groups continue to gain momentum as an efficient and cost-effective medium for social research (Finch, Lewis, & Turley, 2014; Murray, 1997; Stewart & Shamdasani, 2015; Turney & Pocknee, 2005). The benefits of online focus groups are many; two notable benefits are the cost reduction and ease of logistics. The latter is particularly critical if, for example, study abroad students are located far apart from one another in the host country, and if the moderator is not in-country with the students. Tuttas (2015) outlined some potential reasons that virtual focus groups may be less ideal; that is, some marketing professionals and academic researchers worry about the effect the online format has on the focus group atmosphere and dynamics; furthermore, they are skeptical that the moderator can maintain a role similar to that he or she would have in a faceto-face context. Tuttas countered those criticisms and proposed that today's technology is advanced enough to warrant robust online focus groups at the lower end of the size spectrum (she ran groups online with 5 students). Ho (2012) recommended follow-up questionnaires be mailed or emailed to the participants (be they online or not) to gather their feedback or thoughts

Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

P. Winke / System xxx (2017) 1e11

5

they had upon the completion of the focus group session. Such recommendations are outlined by Krueger and Casey (2015) as well. 2.3.5. Analyze the focus group data Analyzing focus group data is as diverse and multidimensional as investigating any type of qualitative data. Researchers first need to identify the research approach they will use (e.g., a pure qualitative study, or a mixed-methods study, see Creswell, 2014). It is beyond the scope of this article to discuss all of the methods possible, thus I refer readers to some of the major methods to analyze focus group data (Krueger & Casey, 2015). Examples of appropriate genres for analyzing focus €rnyei, 2007), in which the theories come from the group data include the following: Grounded Theory (Charmaz, 2014; Do data themselves; Content Analysis (Krippendorff, 2013), in which the open-ended responses are coded and classified using either emergent or pre-established schema; and Conversation Analysis, in which the dialogue is intensively investigated in an attempt to understand the individuals’ relationships with other group discussants and position on the topic at hand (Kasper & Wagner, 2014). What is most important is that the researchers have a plan on how to analyze the data before the data are collected (Krueger & Casey, 2015). The researchers should also explicitly name their method for data analysis in their paper (as called for by Creswell, 2014), and also describe (with sufficient detail for replication) the data collection procedures and the data analysis procedures in the methods section of the paper (Krueger & Casey, 2015). In sum, it is important that the readers understand what was done with the focus group data that were collected because such transparency will help lead to reproducibility and a greater understanding of how to interpret the results. On a broader note, maintaining a transparent stance is in keeping with recent calls (e.g., De Costa, 2016) to carry out applied linguistic research in an ethical manner. As mentioned above, researchers do not have to follow all of Krueger and Casey's (2015) parameters for focus groups (see Table 1); rather, they have many choices about how to analyze data from focus group sessions. Yet to be described, however, are the parameters applied linguists have followed or used to investigate study abroad research questions, what methods they have used for data collection, and what they have done to analyze focus group discussion data. That is the purpose of the present study; the guiding research questions are the following. 1. According to Krueger and Casey's (2015) parameters, what is the level of focus group reporting quality in research on study abroad? 2. How detailed are focus group procedures in research on study abroad? (To what extent do applied linguists report how they conducted their focus groups?) 3. How are data from focus groups on study abroad analyzed? (What qualitative data analysis methods do the applied linguists use?)

3. Method 3.1. Studies on study abroad that use focus groups For this research, I and a graduate research assistant conducted a search to find research articles on study abroad in which the researchers employed focus group sessions. Thus, we had two main search terms: study abroad; and focus groups. We first looked through applied linguistics and international studies indexes available at the university library, and second, we looked through Google Scholar. We limited our search to published research articles in peer-reviewed journals, and we further limited our search to include papers published in the last 15 years (2002e2017). Altogether, we found 10 articles on study abroad and focus groups. The 10 studies are listed in Table 2. I categorized them according to their general study abroad topic, of which there were three.

3.2. Procedure This is a descriptive study in which no statistical analyses were performed. Rather, I and the graduate research assistant read through the 10 studies and coded them on focus group reporting quality using the 6 parameters in Table 1 and two additional criteria: (7) do the authors provide a description of the focus group procedures for transparency and reproducibility purposes?; and (8), do the authors describe how the data were analyzed for reliability, replication, and generalizability purposes?

4. Results The results are displayed in tabular format in Table 2. The table contains an overview of the 10 articles, and the “X's” indicate a “yes” in that a study adhered to at least the 6 focus group parameters (listed in Table 1) and the two extra focus group quality areas introduced above (7, procedures reported; 8, data analysis methods reported). Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

6

P. Winke / System xxx (2017) 1e11

Table 2 Empirical studies on study abroad that described focus group data and indicators of the authors’ focus group reporting quality. Topic

Authors & Year

Journal

Title

1. Choices and decisions during SA

Doyle et al., 2010

J.of Studies in International Ed.

Foster, 2014

J. of Research in International Ed.


& Cots, 2016 Martin-Rubio

Language and Intercultural Communication International Educator

An investigation of factors associated with student participation in study abroad Student destination choices in higher education: Exploring attitudes of Brazilian students to study in the UK Englishisation at a global space: Students and staff making sense of language choices Good faith or hard data? Justifying short-term programs Interculturality and the study abroad experience: Students' learning from the IEREST materials Learning the rules: Language development and cultural adjustment during study abroad Developing a scale to measure situational changes in shortterm study abroad programs Broadening the mind? Australian student reflections on the experience of overseas study Researching the study abroad experience How preservice teachers' study abroad experiences lead to changes in their perceptions of ELLs

Focus Group Quality Indicator 1

2. How SA affects SLA processes

3. How SA changes students' perceptions and selfperceptions

Zamastil-Vondrova, 2005 Holmes et al., 2016

Language and Intercultural Communication

Bacon, 2002

Foreign Language Annals

Ching et al., 2014

International J. of Research Studies in Ed.

Forsey et al., 2012

J. of Studies in International Ed.

McLeod & Wainwright, 2009

J. of Studies in International Ed. Frontiers: The Interdisciplinary Journal of Study Abroad

Medina et al., 2015

2

3

4

X

X

X

X

X

X

X

X

6

7

8

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

5 X

X

X X

X

X

Notes. SA ¼ Study abroad.“X” indicates that the authors at least minimally reported on this aspect of focus group quality, while a blank indicates the authors did not report on this aspect of quality at all.

4.1. Level of focus group quality The 10 studies outlined in Table 2 varied in terms of focus group reporting quality beyond their adherence to the eight parameters mentioned above; they also varied on how focus groups were used within the larger research data collection context. (See Table 3.) Out of the 10 studies, focus groups data were used as the sole means of data collection for only two
& Cots, 2016; McLeod & Wainwright, 2009). Four of the papers were qualitative studies that used focus studies (Martin-Rubio group sessions in additional to other data collection techniques such as observations, written reflections, and journals (Bacon, 2002; Holmes et al., 2016; Medina et al., 2015; Zamastil-Vondrova, 2005). Three studies employed a mixed methods approach, in which surveys were administered to a large sample, and a small subset of them participated in focus groups (Doyle et al., 2010; Forsey et al., 2012; Foster, 2014). In these three latter studies, the focus group data were used to supplement the quantitative findings. One study appeared to be unique: Ching et al. (2014) reported on a qualitative focus group study within a larger quantitative (survey-based) study. They used five focus groups with 8e9 international college students each (40 total) who were studying in Taiwan to generate topics that they later used to create a Likert-scale survey to investigate students' perceived difficulties in “situational changes” (p. 57), such as participating in class discussions, shopping, using mass transit, and adapting to the local etiquette while studying abroad. When these different uses of focus groups within the research are considered, one can understand why the reporting practices may also differ; that is, less space is devoted to detailed reporting on how the focus groups were conducted and the data analyzed when focus groups are not the main focus of the study. Nevertheless, the information could be presented in a supplemental document online, which most journals now allow. Within the categories of how focus groups are used (Table 3), certain studies rise to the top in terms of focus group
and Cots's (2016) study are the best examples of focus group reporting quality. Overall, Foster's (2014) study and Martin-Rubio research reporting in the set of 10 articles examined in this study. Foster used two focus groups as part of a mixed-methods study to investigate students' choices about whether to study abroad. Across three universities in Brazil, 117 students responded to a questionnaire on why they may or may not study abroad in the United Kingdom. The two focus groups were with a small subset of the study participants: one involved 9 undergraduate students at one of the three universities, and the Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

P. Winke / System xxx (2017) 1e11

7

Table 3 Examples of how focus groups have been used within research on study abroad. Method

Description

Studies

1. Mixed (quantitative and qualitative)

The study comprises a survey; a subset of survey takers participate in focus groups; the focus group data is used to bolster survey findings.


Doyle et al., 2010
Forsey et al., 2012
Foster, 2014

2. Qualitative

Focus groups were used to develop topics for survey questions in the larger study, thus the work in preparing for the quantitative study was qualitative.


Ching et al., 2014

Focus groups are one of many various qualitative data-collection methods employed in the study.







The only method for data collection are focus groups.


& Cots, 2016
Martin-Rubio
McLeod & Wainwright, 2009

Bacon, 2002 Holmes et al., 2016 Medina et al., 2015 Zamastil-Vondrova, 2005

other contained 14 graduate students at another. She briefly described that the focus group sessions were transcribed using Audio Notetaker (https://www.sonocent.com/), then a “thematic analysis” (p. 156) was conducted with the aid of NVivo. Foster then divided her results section into questionnaire and focus group findings, making the results and following discussion sections easy to understand. The article did not include information on the moderator; that is, was the moderator the same person for each focus group, or was the moderator Foster herself? Furthermore, the scripts used to guide the discussions
n & were not provided. The data from the focus groups were not broken down by individual speaker (as, for example, Hyde Bülow, 2003; broke down their data), which made it difficult to discern how well the focus group data represented the views of all of the focus group participants. Foster used the NVivo software, so a breakdown of speaker topics and frequency could have been extracted from the software and included in her paper.
and Cots's (2016) study was one of two (out of the 10) that used focus group data as the only source of data Martin-Rubio collection to investigate the research questions, and their study had a higher reporting quality. They probed into “language choice in university mobility programs” (p. 403). They conducted two focus groups with 12 bilingual Catalan/Spanish students before they went abroad to study for a year in Denmark, and 6 of those same 12 upon return. A third focus group was
conducted with one lecturer and two administrative staff members of the study abroad program in Denmark. Martin-Rubio and Cots (2016) went into great detail to outline exactly who was in each focus group session (each was identified by a gender-revealing pseudonym), and exactly how long each focus group lasted (96, 42, and 79 min, respectively). The researchers indicated the student focus groups were audio and video recorded, while the faculty/staff focus group was only audio recorded. They detailed the students' areas of study, and exactly who moderated each focus group session. The researchers alluded to the focus group script that the moderators used to generate the discussions, but pointed out that discussants were allowed to raise new issues and questions. The authors wrote that they used “Membership Categorization Analysis (MCA)” to analyze the focus group data after transcription. This is a type of emergent thematic analysis in which the research “focuses on the categorizations that the participants mobilize and negotiate with co-participants in the focus-group sessions in order to 'make sense' of their language choice in favor of using English and practically ignoring Danish” (p. 406). The three main themes of the script were: language
and Cots demonstrated through anlearning and use; intercultural competence; and European citizenship. Martin-Rubio notated and coded excerpts of the focus group speech that languages and language-use choice was a type of cultural capital, meaning that the study abroad students decided, in effect, how much they would invest into learning Danish based on their perceived estimation of how valuable of a commodity the Danish language was to them. On the other end of the spectrum, Doyle et al. (2010), Forsey et al. (2012), Bacon (2002), and Zamastil-Vondrova (2005) provided the least amount of information about the focus groups they conducted for their research. Both Doyle et al. Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

8

P. Winke / System xxx (2017) 1e11

(2010) and Forsey et al. (2012) had mixed-method studies very similar to Foster's (2014), but they differed from Foster's study in terms of the level of information about the focus groups given. Doyle et al. (2010) used focus groups to investigate New Zealand students’ perceptions on study abroad. Doyle et al. surveyed 1, 368 first and second year undergraduate students on the subject of why they would or would not study abroad; subsequent and supplemental focus groups involved 8 boys at a public high school and 10 girls at a private high school (respectively in their own focus groups, but the exact number of focus groups at each school was not reported on) who were asked to discuss (a) their motivations for studying abroad, (b) factors that impacted their positive or negative view of study abroad, (c) their expectations of or experience of studying abroad, and, (d) if they had studied abroad before, how they viewed the experience impacted them personally and academically. Although the authors did use the qualitative information to describe student motivations for and barriers to studying abroad, there were limitations regarding how the researchers reported on the focus groups. First, the exact number of focus groups was not reported (readers will assume there were two), but more notably, there was no analysis section or information on how the data were coded. In Forsey et al.’s (2012) mixed-methods study, the quantitative portion was a survey given to 219 undergraduates regarding their perceptions of study abroad programs, and the qualitative element involved a focus group with 14 students regarding their perceptions and actual experiences of their study abroad programs upon return to their home institution. These discussions were named “semi-structured in-depth interviews” and later in the paper reduced to “interviews”. Who the moderator was and whether or not he or she was trained was not reported in the study; furthermore, the authors did report on the number of focus groups or participated in each of them. Finally, no information was provided regarding the focus group data analysis. Bacon's (2002) study was a well done, in-depth ethnographic case study in which one person's study abroad experience was described. And that one person also participated in focus groups. The student was a native speaker of English studying for an academic year in Mexico, and several types of data were collected to triangulate the participant's experiences: three 1-h interviews, three 45-min writing samples, and two focus groups with three similar students also participating in the same study-abroad program. In the paper there is no information on the focus groups. The author of the article mentioned in the notes section that the data were drawn from a larger study with four students in the same study abroad program, and that this particular student was selected for a more in-depth analysis because “she [was] representative of many students who [had] what would be considered an adequate level of language to be able to profit from her study abroad experience. The other students in the study did not have as high an initial proficiency” (p. 646). While the study was informative and interesting, the reporting on the focus groups was confusing and lacking. For example, quotes presented in the results section were not always explicitly qualified as to from which source they originated, whether from the focus groups or the interviews. 4.2. Focus group procedures None of the 10 studies had any information the focus group procedures (i.e., how the room was set up, where the focus groups where, or when, exactly, they were conducted), detailed what the focus group directions to the participants were, or provided the focus group script in an appendix or in a supplemental file. 4.3. Focus-group data analysis The researchers of 5 out of the 10 studies explicitly stated what method they used to analyze the data from the focus group sessions. Foster (2014) transcribed the focus groups using Audio Notetaker, and then applied a “thematic analysis” (p. 156)
and Cots (2016) used “Membership Categorization Analysis using the NVivo software. As mentioned above, Martin-Rubio (MCA)” (p. 406) to analyze the focus group data after transcription. In the paper, they wrote that they were attempting to apply MCA to the study abroad field, something that had not been done before. Holmes et al. (2016) provided the most detail. They described that the data were collected during the program, and then analyzed using “thematic analysis” (p. 456), but they stopped short of detailing how the data were coded and by whom. No coding reliability estimates were reported, and the process of ensuring inter-rater reliability were not mentioned. Indeed, most researchers simply reported that the data were coded or that themes emerged, and readers were left to imagine how exactly these operations were performed. 5. Discussion By looking at the data in Table 2, one can conclude that focus groups have not reached their full potential in applied linguistics and international education research on study abroad. In general, critical information regarding focus group participant descriptions, as well as methodological and procedural specifications, were omitted from the surveyed articles. In the 10 papers reviewed here, it appears authors were not following recommended guidelines for reporting on focus groups: data collection and analyses were not explained, and the results were presented without clear information on the source from which the data came (Doyle et al., 2010; Holmes et al., 2016). Regular reporting of from which sources the data came when the results are presented would be extremely helpful in applied linguistics studies on study abroad, because readers should understand that focus group data are necessarily different than information gained from journal entries, one-on-one interview data, or from written responses on questionnaires. Focus group data are influenced and enhanced by the discussionoriented, social mediation of the focus group session. Incorporating quotes from focus groups together with quotes from Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

P. Winke / System xxx (2017) 1e11

9

reflective essays, journals, or one-on-one interviews may be stripping away some of the social-structure information that the focus group data hold. 5.1. Study abroad researchers must report how they conducted their focus groups None of the studies reviewed in this paper contained discussions by the researchers on any difficulties in conducting the focus group sessions or analyzing the data. Parker and Tritter (2006) noted (referring to Kitzinger, 1994) that focus groups, as
n compared to interviews, often include conversational speech that is incomplete or interrupted. They summarized how Hyde and Bülow (2003) reported that focus groups generate data that pertains to both individual and group-level information, attitudes, and views, and that it can be extremely difficult to disentangle the individual-level data from the group-level data. In several of the studies reviewed in this paper, the focus group data were used to back up what was found from a quantitative survey given to a larger group of students, and in general the quotes presented all supported those quantitative data findings. However, focus groups “have the potential to generate unexpected and unpredictable outcomes, both in terms of the data gathered and the complexities of the research process as a whole” (Parker & Tritter, 2006, p. 34). The challenge for all researchers employing the focus group data collection method is to tap into both the general, expected themes that affirm the numbers of a mixed-methods research design, but also allow for the unexpected richness and variety that could be uncovered through open-ended, focus group data collection. 5.2. Study abroad researchers must theorize on the purposes of focus group discussions Another concern is whether focus groups in study abroad research are always being used at the right time. Morgan and Kreuger (1993) stressed that a “match between the researcher's topic of interest and the participants' ability to discuss those topics is essential for successful focus groups” (p. 10). Two of the studies in Table 2 used focus groups with students who had
and Cots (2016) investigated “language choice in not yet studied abroad to understand aspects of study abroad. Martin-Rubio university mobility programs” (p. 403), and conducted a focus group with 12 students before they went abroad; 6 of those same 12 partook in a second focus group upon return. It is not clear if the first (pre-study-abroad) focus group gave the authors any useful information. Doyle et al. (2010) conducted two focus groups, one with 8 high school boys, and one with 10 high school girls, and asked in the focus groups about (a) their motivations toward study abroad, (b) factors that impact their positive or negative view of study abroad, and (c) their expectations of studying abroad at the college level. However, Morgan and Kreuger (1993) warned that when focus participants have too little involvement in the topic, then focus groups may be of little use. They wrote that “social scientists seldom gain much by asking participants to report on things that are beyond their
and Cots's (2016) study and in Doyle et al.’s (2010) study if experience” (p. 13). It would have been helpful in Martin-Rubio they had indicated exactly from which focus groups any quotes or themes came, because then one would be able to see if the pre-study-abroad focus groups were of value. 5.3. More study abroad research on with robust focus groups is needed These objections aside, all 10 of the studies demonstrated that focus groups are helpful in understanding the social, cultural, linguistics, and programmatic complexities related to language learning, intellectual development, and intercultural exploration during study abroad. More study abroad research with focus groups is needed. Holmes et al.'s (2016) study is perhaps the best expression of the considerable need of applied linguists to harness the full power of focus groups to investigate issues in study abroad. The research of Holmes et al.'s (2016) study was centered on how students use their time in a specific type of program abroad to “become global and critical cultural intercultural citizens” (p. 453), a research area rarely tapped, and best detailed through qualitative methods. In their study, which included participant journals, written and oral reflections, observations of the classes, questionnaire responses, and focus groups (also called group discussions and workshops in the paper, which is unfortunately methodologically confusing), Holmes et al. (2016) investigated how students understood and viewed their developing intercultural perspectives while abroad. Despite a lack of transparency and limitations in data reporting, Holmes et al. (2016) skillfully demonstrated how focus groups can be used to investigate complex human growth and development inherent to study abroad. They underscored the importance of social networks for the learning outcomes of study abroad because through social networks students develop interculturality (Dervin & Layne, 2013; Dervin, 2014; Risager & Dervin, 2014). Study abroad program directors need to understand how social networks are established and sustained during study abroad to help students foster them. Focus groups are robust qualitative instruments that can be used to shape an informed critical perspective on the social networks that are present in study abroad. Holmes et al. (2016) pointed to the need for researchers to understand how study abroad shapes students’ “developing sense of citizenship” and how students differentiate “'cultural' from 'intercultural,' and how learning another language contributes to identity construction” (p. 466). Their study alludes to the high value of focus groups in understanding these relatively unexplored, socially-constructed complexities that are part of study abroad. DeKeyser (2010) succinctly pointed out that because study abroad programs are mostly small in size, they are ripe for the application of rich qualitative tools. He noted that from a methodological standpoint, researchers are in the position to “narrow the gap between the quantitative and qualitative research traditions” (p. 81). DeKeyser referenced Lafford (2007, p. 749) who stated that in second language acquisition, researchers have yet to thoroughly integrate both “macro-level Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

10

P. Winke / System xxx (2017) 1e11

phenomena” (meaning quantitative data) with “micro-level phenomena” (that is, data that are more social in nature, and collected qualitatively). Nevertheless, with robust planning and documentation of focus group discussions, study abroad researchers can do this. As described by DeKeyser (2014), how individual differences such as student motivation contribute to advanced language learning during study abroad perhaps must be investigated qualitatively, not quantitatively. This is because study abroad involves mostly homogenous students: They are all highly motivated. Thus Likert-scale questions on motivation won't be very informative. But, the nuances of motivation within study abroad participants learning language at the advanced level or beyond can be robustly investigated qualitatively through observations, interviews, and focus groups. In this context, focus groups may allow for a rather large number of participants (which would help with reliability and validity), when compared to other purely qualitative data collection methods. Bacon (2002) as well wrote that qualitative approaches to data collection, such as the approach she took (a longitudinal, ethnographic approach with focus groups and other data collection techniques) “provides a rich and multilayered view of cultural adjustment” (p. 645) which is something quantitative data, arguably, cannot document. To access complex issues involved with study abroad, such as cultural adjustment and the social and societal benefits of study abroad, robust focus group data collection methods are needed. And the authors of such work need not be constrained in their reporting; journals now allow for supplemental materials that do not count toward the word limit. Thus, a strong suggestion is that study abroad researchers who use focus groups in any capacity should present the methods (i.e., directions and script) and procedures in a supplemental file. Doing so will help readers understand the focus group data collection techniques, which will help them interpret the data and understand the generalizability of the results. A second suggestion is that study abroad researchers harness the power of online focus groups. Such data collection techniques have come of age in social science research, and study abroad contexts are ripe for their application. 5.4. Applied linguists need to learn more about focus groups Applied linguists must turn to the experts in focus group methodology (Krueger & Casey, 2015; Stewart & Shamdasani, 2015; Turney & Pocknee, 2005) to learn more. For example, Krueger and Casey's (2015) 5th edition book on focus group methods has essential chapters on how to (a) develop strategies for asking questions, (b) develop moderating skills, (c) conduct focus groups with young people, (d) and carry out international and cross-cultural focus groups. This information is particularly important for study abroad researchers who want to use focus groups because there is a need in applied linguistics research to better understand how interculturality is developed during study abroad, and how that development affects language learning and motivation to learn to the advanced levels of proficiency. One of the major difficulties in following students longitudinally during study abroad is access to the students while they are abroad. Technology-mediated focus groups (see Tuttas, 2015) are an inexpensive and expedient way to tap into students' developing perceptions while they are actively engaging in studying abroad. In conclusion, study abroad researchers should improve their skills of focus group facilitation and use focus groups more frequently. The way to start may be to try out a focus group within a purely qualitative study on study abroad by using focus groups alongside other qualitative methods, as Bacon (2002), Holmes et al. (2016), and Medina et al. (2015) did. Through practice, study abroad researchers will gain the ability to conduct full-on focus group
and Cots (2016) did. New research is inquiries into the social and intercultural aspects of study abroad, like Martin-Rubio needed on how students gain linguistic, cultural, and social skills during study abroad. The path showing how to do such research qualitatively with focus groups has been paved. Focus groups may help researchers understand newer questions, such as how social media affects learning while abroad (and whether target-language social media habits learned while abroad sustain learning post-study abroad), and how studying abroad helps individual students motivate themselves to learn a language to an advanced, superior, or distinguished level of proficiency. Author's note Paula Winke is an associate professor in the Second Language Studies Ph.D. Program at Michigan State University. Acknowledgments I would like to thank Ms. Karolina Achirri, Dr. Jessica Fox, and an anonymous System reviewer for their help with different parts of this paper. Any mistakes in the paper are my own. References Bacon, S. M. (2002). Learning the rules: Language development and cultural adjustment during study abroad. Foreign Language Annals, 35(6), 637e646. http://doi.org/10.1111/j.1944-9720.2002.tb01902.x. Charmaz, K. (2014). Constructing grounded theory. Los Angeles: Sage. Ching, G. S., Lien, W., & Chao, P. (2014). Developing a scale to measure the situational changes in short-term study abroad programs. International Journal of Research Studies in Education, 3(5), 53e71. http://doi.org/10.5861/ijrse.2014.771. Creswell, J. (2014). A concise introduction to mixed methods research. Thousand Oaks, CA: SAGE Publications. De Costa, P. I. (Ed.). (2016). Ethics in applied Linguistics: Language researcher narratives. New York: Routledge. DeKeyser, R. (2010). Monitoring processes in Spanish as a second language during a study abroad program. Foreign Language Annals, 43(1), 80e92. http:// doi.org/10.1111/j.1944-9720.2010.01061.x.

Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018

P. Winke / System xxx (2017) 1e11

11

DeKeyser, R. M. (2014). Research on language development during study abroad. In C. Perez-Vidal (Ed.), Language acquisition in study abroad and formal instruction contexts (pp. 313e325). Amsterdam: Benjamins. http://doi.org/10.1075/aals.13.16ch13. Dervin, F. (2014). Exploring “new” interculturality online. Language and Intercultural Communication, 14(2), 191e206. http://doi.org/10.1080/14708477.2014. 896923. Dervin, F., & Layne, H. (2013). A guide to interculturality for international and exchange students: An example of hospitality? Journal of Multicultural Discourses, 8(1), 1e19. http://doi.org/10.1080/17447143.2012.753896. € rnyei, Z. (2007). Research methods in applied Linguistics: Quantitative, qualitative, and mixed methodologies. Oxford: Oxford University Press. Do Doyle, S., Gendall, P., Meyer, L. H., Hoek, J., Tait, C., McKenzie, L., et al. (2010). An investigation of factors associated with student participation in study abroad. Journal of Studies in International Education, 14(5), 471e490. http://doi.org/10.1177/1028315309336032. Finch, H., Lewis, J., & Turley, C. (2014). Focus groups. In J. Ritchie, J. Lewis, C. McNaughton Nicholls, & R. Ormston (Eds.), Qualitative research practice: A guide for social science students & researchers (2nd ed., pp. 211e242). Los Angeles: Sage Publications. Flores, J. G., & Alonso, G. C. (1995). Using focus groups in educational research: Exploring teachers' perspectives on educational change. Evaluation Review, 19(1), 84e101. https://doi.org/10.1177/0193841X9501900104. Forsey, M., Broomhall, S., & Davis, J. (2012). Broadening the mind? Australian student reflections on the experience of overseas study. Journal of Studies in International Education, 16(2), 128e139. http://doi.org/10.1177/1028315311407511. Foster, M. (2014). Student destination choices in higher education: Exploring attitudes of Brazilian students to study in the United Kingdom. Journal of Research in International Education, 13(2), 149e162. http://doi.org/10.1177/1475240914541024. Goffman, E. (1961). Asylums. New York: Doubleday. Harre, R. (1984). Some reflections on the concept of “social representation”. Social Research: An International Quarterly, 51(4), 927e938. Available at: http:// www.jstor.org/stable/40970971. Ho, D. G. E. (2012). Focus groups. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics (pp. 1e7). Hoboken, New Jersey: Wiley-Blackwell. http://doi. org/10.1002/9781405198431.wbeal0418. Holmes, P., Bavieri, L., Ganassin, S., & Murphy, J. (2016). Interculturality and the study abroad experience: Students' learning from the IEREST materials. Language and Intercultural Communication, 16(3), 452e469. http://doi.org/10.1080/14708477.2016.1168054.
n, L.-C., & Bülow, P. (2003). Who's talking: Drawing conclusions from focus groups-Some methodological considerations. International Journal of Social Hyde Research Methodology, 6(4), 305e321. http://doi.org/10.1080/13645570210124865. Kahn, M. E., & Manderson, L. (1992). Focus groups in tropical diseases research. Health Policy and Planning, 7(1), 56e66. Available at: https://search.proquest. com/docview/1303104111?accountid¼12598. Kasper, G., & Wagner, J. (2014). Conversation analysis in applied linguistics. Annual Review of Applied Linguistics, 34, 171e212. http://doi.org/10.1017/ S0267190514000014. Kitzinger, J. (1994). The methodology of focus groups: The importance of interaction between research participants. Sociology of Health & Illness, 16(1), 103e121. http://doi.org/10.1111/1467-9566.ep11347023. Krippendorff, K. (2013). Content Analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage Publications. Krueger, R. A. (1998). Developing questions for focus Groups: Focus group kit 3. Thousand Oaks, CA: Sage Publications. Krueger, R. A., & Casey, M. A. (2015). Focus groups: A practical guide for applied research (5th ed.). Thousand Oaks, CA: Sage Publications. Lafford, B. a (2007). Second language acquisition reconceptualized? The impact of Firth and Wagner (1997). The Modern Language Journal, 91(5), 735e756. http://doi.org/10.1111/j.1540-4781.2007.00666.x.
, X., & Cots, J. M. (2016). Englishisation at a global space: Students and staff making sense of language choices. Language and Intercultural Martin-Rubio Communication, 16(3), 402e417. http://doi.org/10.1080/14708477.2016.1168051. McLeod, M., & Wainwright, P. (2009). Researching the study abroad experience. Journal of Studies in International Education, 13(1), 66e71. http://doi.org/10. 1177/1028315308317219. Medina, A. L., Hathaway, J. I., & Pilonieta, P. (2015). How preservice teachers' study abroad experiences lead to changes in their perceptions of English language learners. Frontiers: The Interdisciplinary Journal of Study Abroad, 25, 73e91. Retrieved from http://0-search.ebscohost.com.umhblib.umhb.edu/ login.aspx?direct¼true&db¼eric&AN¼EJ1071106&site¼ehost-live&scope¼cite. Morgan, D. L., & Kreuger, R. A. (1993). When to use focus groups and why. In D. L. Morgan (Ed.), Successful focus Groups: Advancing the state of the art (pp. 3e19). Newbury Park, CA: Sage Publications. Murray, P. J. (1997). Using virtual focus groups in qualitative research. Qualitative Health Research, 7(4), 542e549. https://doi.org/10.1177/ 104973239700700408. Parker, A., & Tritter, J. (2006). Focus group method and methodology: Current practice and recent debate. International Journal of Research & Method in Education, 29(1), 23e37. http://doi.org/10.1080/01406720500537304. Risager, K., & Dervin, F. (2014). Introduction. In K. Risager, & F. Dervin (Eds.), Researching identity and interculturality (pp. 1e28). New York: Routledge. Smithson, J. (2000). Using and analysing focus groups: Limitations and possibilities. International Journal of Social Research Methodology, 3(2), 103e119. http://doi.org/10.1080/136455700405172. Stewart, D. W., & Shamdasani, P. N. (2015). Focus Groups: Theory and practice (3rd ed.). Thousand Oaks, CA: Sage Publications. Turney, L., & Pocknee, C. (2005). Virtual focus groups: New frontiers in research. International Journal of Qualitative Methods, 4(2), 32e43. Available at: http:// journals.sagepub.com/doi/full/10.1177/160940690500400203. Tuttas, C. A. (2015). Lessons learned using web conference technology for online focus group interviews. Qualitative Health Research, 25(1), 122e133. http:// doi.org/10.1177/1049732314549602. Zamastil-Vondrova, K. (2005). Good faith or hard data? Justifying short-term programs. International Educator, 14(1), 44e49. Available at: https://www. questia.com/article/1P3-1387655611/good-faith-or-hard-data-justifying-short-term-programs.

Please cite this article in press as: Winke, P., Using focus groups to investigate study abroad theories and practice, System (2017), https://doi.org/10.1016/j.system.2017.09.018