EFL education: Learners’ views from secondary- and tertiary-level perspectives

Studies in Educational Evaluation 62 (2019) 61–71

Contents lists available at ScienceDirect

Studies in Educational Evaluation journal homepage: www.elsevier.com/locate/stueduc

Evaluating supplementary and mainstream ESL/EFL education: Learners’ views from secondary- and tertiary-level perspectives

T

⁎

Art Tsanga, , Daniel Funga, Alice Hoi Ying Yaub a b

The Chinese University of Hong Kong, Hong Kong The University of Hong Kong, Hong Kong

A R T I C LE I N FO

A B S T R A C T

Keywords: Supplementary education Mainstream/daytime education English as a foreign/second language English for specific purposes Secondary and tertiary education Learners’ perspectives

English as a second/foreign language education (ESL/EFL) in the supplementary setting is under-researched despite its prevalence worldwide. This quantitative study investigated the effects of supplementary and mainstream ESL/EFL education on eleven facets from learners’ secondary-level and tertiary-level perspectives. In addition, we examined learners’ perception of the usefulness of supplementary and mainstream classes in equipping them for the use of English for different purposes in the tertiary setting. 203 participants studying at two tertiary institutions in Hong Kong completed a 72-item questionnaire. The MANOVAs conducted revealed significant main effects of education settings (i.e. supplementary/mainstream) and education levels (i.e. secondary-level/tertiary-level) as well as interaction effects of these variables on learners’ perception. There were also significant differences in learners’ views of the usefulness of the two education settings in preparing them for the use of English for specific purposes at tertiary level. This article culminates with a discussion of these findings and implications.

1. Introduction Supplementary education1 literally refers to the kinds of education learners receive in addition to normal daytime (i.e. mainstream) schooling. Such a mode of education comes in multifarious types – conducted individually or in small to large groups, held after school hours and/or during weekends/public holidays, and in terms of content, related or unrelated to school and examination syllabi (Bray, 2013). Notwithstanding the great variations, there are three central characteristics of supplementary education as summarized by Yung (2016) which are, (a) “Privateness – tutoring…provided by individuals or organizations in exchange for a fee,” (b) “Academic subjects – typically national and second languages, mathematics, …leisure and/or personal development such as music, art and sports are excluded,” and (c) “Supplementation – Shadow education supplements the provision of schools…either for remedial or enrichment purposes” (p. 26). Supplementary education, in this study, is defined as the type of schooling illustrating all these three features as stated by Yung (2016). This mode of education has been gaining in prevalence and drawing greater attention among researchers in recent years (Bray & UNESCO, 2009; Hamid, Khan, & Islam, 2017; Yung, 2015; Zhan, Bray, Wang,

Lykins, & Kwo, 2013). It prevails not only in Asia (Bray & UNESCO, 2009; Lee, 2010), but also in many corners of the world such as Australia, Europe, and America (Bray & UNESCO, 2009; Forsey, 2013; Ireson & Rushforth, 2011; Iveta, 2010; Smyth, 2008). The popularity of supplementary education is substantiated by the higher number of students taking supplementary classes, which are mostly on examined subjects (Zhan et al., 2013), prior to examination periods (Bray, 2013). The primary motive for students’ enrollment in supplementary education courses alongside their formal school education, especially in the East Asian context, is the pursuit of upward social mobility which hinges greatly on academic achievements (Bray & Lykins, 2012; Gao, 2008). Among all the subjects in supplementary education, English as a second/foreign language (ESL/EFL) is often the most popular (Lee, 2010; Yung, 2016; Zhan et al., 2013). This is most attributable to the fact that ESL/EFL is a core subject in many non-English speaking countries (e.g. China, Japan, Malaysia, and Vietnam) (Nunan, 2003). In many places in East Asia, for instance, English proficiency assessment is often employed as a screening tool for further education especially when competing for more prestigious institutions and for job opportunities (Ross, 2008). Another possible reason for its popularity is the

⁎

Corresponding author. E-mail addresses: [email protected] (A. Tsang), [email protected] (D. Fung), [email protected] (A.H.Y. Yau). 1 Some synonymous expressions include private education (Lee, 2010), private tuition (Smyth, 2008), private tutoring (Zhan et al., 2013) and shadow education (Bray, 1999). In this article, these terms are used synonymously with supplementary education. https://doi.org/10.1016/j.stueduc.2019.04.004 Received 14 July 2018; Received in revised form 3 April 2019; Accepted 5 April 2019 Available online 04 May 2019 0191-491X/ © 2019 Elsevier Ltd. All rights reserved.

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

are also quite mixed. Hamid, Sussex, and Khan (2009) conducted a mixed-methods study in Bangladesh. Although a correlation between participation in supplementary English classes and English proficiency existed, the link was rather weak; as for qualitative findings, the interviewees were generally positive towards this mode of education in terms of interest and its usefulness in English enhancement and for examinations. Similar findings were reported in a more recent study on learners’ perception by Hamid et al. (2017). In this study, the learners perceived that the supplementary classes were helpful for English improvement. It is also noteworthy that 82% of the 572 respondents reported taking supplementary English classes in the Hamid et al. (2017) study, which is an increase of 7% reported by Hamid et al. (2009). In contrast to the learners’ positive views mentioned above, Yung (2015) determined that interviewees, who were year-one undergraduates and who received private tutoring in English when they were at secondary schools, held “ambivalent and paradoxical attitudes” (p. 707). On the one hand, the learners acknowledged the instrumental value of these supplementary English sessions for their performance in examinations; on the other hand, they noticed an absence of educational value with regard to the development of language proficiency and cultivation of interest. Similar criticisms of over-emphasis on examinations but lack of linguistic development by high-school students were reported by Chung (2013) via a study conducted from Taiwan. Interestingly, however, Lee (2010) reported that not all learners during their pre-university schooling took supplementary English lessons merely for the sake of examinations and/or entering universities; some received training in general language and oral communication skills.

importance of English as a lingua franca in international and intercultural communication. In light of the above, ESL/EFL in supplementary education is a research-worthy topic; the dearth of studies in this area warrants more investigation. While there are many ways of approaching the topic (e.g. examining the correlation between supplementary education and learners’ academic achievements), we explored learners’ perceptions of the effectiveness of supplementary as well as mainstream ESL/EFL education. As scholars such as Brown (2009) have commented, in the realms of language acquisition and applied linguistics, research into students’ perceptions and “trends in second language acquisition (SLA) scholarship seem to indicate that learners’ beliefs and perceptions might be more central to effective second language (L2) acquisition than previously thought” (p. 47). For instance, learners’ perceptions have significant impacts on their achievement (Williams & Burden, 1997) and implications for language education such as efficacy in instruction and learning (Bloemert, Paran, Jansen, & van de Grift, 2017; Brown, 2009; Yung, 2016). Therefore, students’ perspectives of the relative effectiveness of mainstream and supplementary ESL/EFL education is an area worthy of investigation. With a view to providing greater insights, the differentiation between learners’ perceptions from the perspective of secondary and tertiary settings was also examined. 1.1. Actual and perceived effectiveness of supplementary education on academic achievement Before reviewing the literature concerning supplementary ESL/EFL education, it is worthwhile to first discuss supplementary education in general. Despite the popularity of supplementary education globally, research in this area is still very scarce to date; hence also there is a relatively limited amount of relevant literature citable in this article. Supplementary education is often examination-oriented in nature (e.g. Kwok, 2004; Yung, 2015, 2016), and researchers have focused on learners’ perceptions (e.g. Forsey, 2013; Zhan et al., 2013) and the link between supplementary education and academic achievements (e.g. Smyth, 2008). However, the findings seem to be quite mixed. Zhan et al. (2013) surveyed 1624 students in 16 secondary schools in Hong Kong to measure their perceptions of the effectiveness of supplementary education. It was found that the students generally regarded supplementary lessons as “most effective at improving examination grades, confidence in examinations, revision skills, and learning strategies” (p. 503). However, such positive findings were not in sync with a small-scale study by Forsey (2013), who conducted semistructured interviews with 10 students in Australia. The interviewees’ attitudes were mixed in terms of the contribution of supplementary lessons on their academic grades. Smyth (2008) conducted a quantitative study with 4709 students in 112 secondary schools in Ireland. Taking various potential confounding variables such as socio-economic status and attitudes towards academic studies into account, Smyth (2008) demonstrated, via multilevel regression and propensity score matching techniques, that no significant differences exist between the level of involvement in supplementary education and academic performance. In contrast, Lee (2010), in a study based in South Korea, found that the participants who received EFL supplementary education scored higher in English examinations than those who did not engage in tutoring. The mixed results from the aforementioned researchers imply that more investigation into learners’ perceptions and actual relationships between supplementary education and achievement is of value.

1.3. ESL/EFL mainstream education Mainstream education is provided by “both public and private schools that teach an officially recognized curriculum” (Bray, Zhan, Lykins, Wang, & Kwo, 2014, p. 24). This mode of education is understood as compulsory education in places where school attendance is mandatory. Due to the great variety of this form of education globally, we only provide a brief overview of mainstream education in Hong Kong, the context of the study. In Hong Kong, at present, ESL/EFL mainstream education starts from primary one and ends at secondary six, a total of 12 years. As stated in English language education key learning area curriculum guide (Curriculum Development Council, 2017), mainstream schools should provide a holistic education (e.g. facilitating learners’ development of all four skills, namely reading, writing, listening and speaking) to students at all levels. The official suggested lesson time allocation for primary and secondary level per year is around 450 h and 430 h respectively (Curriculum Development Council, 2017). To the best of our knowledge, there are hardly any existing reports or studies investigating the exact content, teaching approaches, homework, and so forth in mainstream ESL/EFL lessons. However, it is apparent to us that mainstream schools incorporate all four skills in their syllabi though they may give different extents of emphasis to different skills. Depending on a number of factors such as teachers’ teaching style and school culture, English classes can range from teacher-centred to student-centred, accuracy-oriented to fluencyoriented lessons. 1.4. ESL/EFL at the tertiary level Aside from ESL/EFL supplementary and mainstream education, another primary focus of the present study (see Table 1) was on learners’ perspectives of ESL/EFL in the context of transition from the secondary to tertiary level (the rationale behind the present study will be expounded later). A brief literature review in this area is therefore provided here. There is very little research on the comparison of ESL/EFL learners’ views, experiences, and adaptation as they are promoted from

1.2. Learners’ perceptions of ESL/EFL supplementary education As noted by Yung (2015) and Hamid et al. (2017), there is also a paucity of studies on ESL/EFL supplementary education. Yung and Hamid et al. pointed out that only a few studies hitherto on supplementary education are directly germane to ESL/EFL contexts. Overall, findings in the literature of learners’ views towards ESL/EFL tutoring 62

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

Table 1 Independent and dependent variables of the two RQs. Research question (RQ)

Independent variables

Dependent variables

Statistical test

RQ One: General English learning experiences of supplementary and mainstream classes at secondary- and tertiary-level studies

Types of classes (supplementary vs mainstream English classes) Education level (secondary-level vs tertiary-level perceptions) Types of classes (supplementary vs mainstream English classes)

11 different aspects of English learning experiences (see Table 2 below)

2 × 2 repeated measures MANOVA

14 different purposes of English use at tertiary-level studies (see Table 3 below)

One-way repeated measures MANOVA

RQ Two: Equipping students in using English for different purposes at tertiary-level studies

scores). To the best of our knowledge, not any study has comprehensively established the differences in the perceived effects between mainstream and supplementary classes on improving learners’ multifarious facets in ESL/EFL (e.g. reading, writing, vocabulary and many other areas) as well as their perceived abilities in using English in specific situations at the tertiary level. This motivated us to explore further in these areas to illuminate our understanding. Fourth, as supplementary education comes in various formats (e.g. one-to-one, small-group) and delivery mechanisms, which leads to inconclusive and at times contradictory results (Bray, 2014), we thought that larger-scale quantitative studies should be particularly useful for drawing relatively more definitive conclusions. While studies such as Yung (2015) and Lee (2010) investigated learners’ experiences in depth and discussed their intriguing findings, the low number of participants (n = 14 in Yung, 2015; n = 43 in Lee, 2010) may yield rather limited implications. Taking the abovementioned caveats by Bray (2014) and the discrepancies among the studies (e.g. learners’ attitudes as discussed) into consideration, studies with larger sample sizes may well help elucidate this research area in greater depth. We acknowledge that in a relatively large-scale study as this, it is difficult to generalize the vast diversity of the quality and quantity of supplementary education students receive. In fact, this applies to research on mainstream education where a substantial number of variables (e.g. pedagogical approaches, classroom settings, school culture) may affect one’s findings. Nevertheless, the overall patterns generated from such heterogeneity in learners’ supplementary (and mainstream) education experiences and relatively large sample size in the present study have implications for our understanding of the general contours of both educational settings. We endeavoured to address the following research questions: 1a. What are the differences in learners’ perceptions of the effects of the secondary-level supplementary and mainstream English education on their general English learning experiences? 1b. What are the changes, if any, in learners’ perceptions prior to and after studying at a tertiary institution? 2. From a learners’ perspective, how useful are the supplementary and mainstream secondary English sessions in equipping them for the use of English for different purposes2 in the tertiary setting?

secondary to tertiary level in the literature. Nonetheless, research on other subjects such as mathematics (i.e. investigating mathematics education in the secondary-tertiary transition e.g. Gueudet, 2008; Hoyles, Newman, & Noss, 2001) has attracted much greater attention than ESL/EFL. This is somewhat surprising in view of the considerable number of ESL/EFL learners internationally pursuing their studies at tertiary institutions where English is used as the main medium of instruction and assessment. There are, however, a small number of researchers who have provided insights into tertiary students’ language needs and challenges. A large-scale study was carried out by Evans and Green (2007), who surveyed approximately 5000 undergraduates on their experiences of using English for various purposes in the tertiary setting. The results indicated that the respondents were generally ill-equipped for the language needs in tertiary education such as inadequate receptive and productive vocabulary, lack of grammatical competence and fluency, which affected their abilities to express ideas and write assignments. Stephen, Welman, and Jordaan (2004) analyzed the relationship between learners’ English proficiency and their academic achievements. The results illustrated the significant impact of language abilities on academic performance and even led the researchers to conclude that English proficiency is “a prerequisite for academic success” (p. 51). Apart from general English proficiency, a remarkable recurrent theme in the literature is the difficulty learners encounter in English for academic or specific purposes. Cai (2013) discussed the challenges learners in China face in academic writing such as reviewing and critiquing as well as academic phrases and styles. In another large-scale survey by Evans and Morrison (2011), specific areas of challenges for freshmen were identified including comprehending specialist vocabulary and lectures, and adhering to appropriate academic style. In light of the above, we thought it was worthwhile to examine the roles mainstream and supplementary ESL/EFL education play in equipping (or ill-equipping) learners for the use of English in different situations at the tertiary level. As explained, we approached this via learners’ perceptions. 1.5. Rationale and research questions

2. Method

This research was initiated based on four prime purposes. First, we aimed to contribute to the literature in view of the sparsity of research in supplementary education (especially ESL/EFL education) and ESL/ EFL in secondary-tertiary transition. The ubiquity of supplementary classes across the globe and the need for the immense number of ESL/ EFL learners to employ English for various purposes at tertiary level necessitate significantly more scrutiny than the status quo. Second, the choice of our focus – ESL/EFL – is somewhat axiomatic. In addition to its relevance to a substantial number of ESL/EFL students, English proficiency plays a pivotal role in learners’ success in their studies, as discussed above. The importance of English in higher education calls for more light to be shed on as regards the adequacy of ESL/EFL education at the secondary level in preparing learners for their tertiary schooling. Third, many of the existing evaluations of the effectiveness of (ESL/ EFL) supplementary education in the literature are too broad and general (e.g. general academic achievement; language proficiency

In this article, we report on part of the quantitative findings from a larger research project on learners’ opinions on secondary, mainstream, and supplementary schooling. 2.1. Participants A total of 2033 tertiary-level students (Male = 92; Female = 102)4 2

These will be explained below under The Questionnaire. For the questionnaire items corresponding to RQ One, all students provided their responses and so the sample size used for analysis was 203. For the questionnaire items corresponding to RQ Two, however, 16 students had not completed the full set of the questions and so deleting the participants listwise rendered a sample of 187. 4 Nine participants did not state their gender. 3

63

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

similarities. We then fine-tuned the format of the questionnaires based on the respondents’ feedback and suggestions, but no changes were required regarding the content of the drafted questionnaire. Methodologically, we opted to ask the same individuals to reflect on their past and present thoughts rather than investigating two different samples (i.e. secondary- and tertiary-level students’ opinions) since there would be a substantial number of confounding variables if the latter had been adopted (see limitations).

aged between 17 and 23 (mean = 19.19; S.D. = 1.21) in Hong Kong were recruited via convenience sampling. These participants are all native Cantonese students from various disciplines such as Business and Psychology. Prior to participating in this research, they have all had English instruction in their primary and secondary education for at least 12 years (given that all the students completed Secondary 6 and that English is a compulsory subject for all students from Primary 1 to Secondary 6, as mentioned above). Based on their public English examination (taken after completing Secondary 6) scores, their English proficiency was best described as intermediate (i.e. Approximately B1 and B2 levels in the Common European Framework of Reference for Languages (CEFR) or bands 4 to 6.5 in the International English Language Testing System (IELTS)). Most of the students were first and second year students in the tertiary institutions from which we collected our data. The selection criteria for this part of the project included that (a) the participant had experience in supplementary English education at the secondary level and (b) the participant could provide their opinions of supplementary and mainstream English education from the secondary-level and tertiary-level perspectives (see below).

2.3. Procedures The questionnaires were administered in groups outside class time in classrooms with the presence of administrators to guide the participants to complete the questionnaires in a step-by-step manner. As stressed by the participants in the pilot study, clear guidance facilitated the respondents’ completion of the questionnaire, especially the more cognitively challenging parts (i.e. sections one and two). This mode of administering the questionnaires also eliminated potential problems such as low response rates and incomplete questionnaires. The administrators explained the content in each section before the respondents completed the items individually. There was no time limit. Only when every respondent had finished a certain part did the administrators continued with the following section. The administrators were also responsible for addressing questions and providing clarifications for the respondents. The completion time of the entire questionnaire was approximately 30 min and the administrators did not observe any signs of fatigue.

2.2. The questionnaire The questionnaire consisted of 72 five-point5 Likert-type items to measure respondents’ level of agreement. The questionnaire (see Appendix) was divided into three sections with different focuses. In Section 1, the effects of secondary-level supplementary English sessions on learners’ English learning experience at secondary and tertiary level; in Section 2, the effects of secondary-level English sessions in mainstream schools on learners’ English learning experience at secondary and tertiary level; in Section 3, the usefulness of supplementary and mainstream secondary English sessions for learner’s use of English for specific purposes in tertiary settings. Sections 1 and 2 consisted of 11 general aspects of English learning such as grammar, vocabulary, listening, and speaking (the full list of items is provided below in the tables under results and in the Appendix). These 11 items were created and discussed by the authors, all of whom have experience in ESL/EFL education. As for Section 3, the 14 items were constructed by the authors also taking into account authors’ colleagues’ and tertiary-level students’ suggestions. These 14 aspects were thought to be rather comprehensive at capturing the situations in which English, rather than Chinese, the local language, is used at tertiary institutions. The first two sections aimed at addressing the first research question while the third section addressing the second research question. After we scrutinized and were satisfied with the first draft of the questionnaire, we invited three students to complete the questionnaire individually alongside cognitive interviewing, a diagnostic pre-testing tool especially for questionnaires (Collins, 2003). According to Collins, the two prime cognitive interviewing techniques are “think aloud interviewing” and “probing” (p. 235). The first two participants attempted the think-aloud approach but found it quite challenging because the questionnaire was designed to investigate the respondents’ thoughts in the past and the present (i.e. their views towards mainstream and supplementary English education when they were secondary school students and at present as tertiary-level students). We therefore adopted the probing technique and asked all three respondents for their reflection immediately after every (sub-)section in the questionnaire. We had concerns whether the learners would have difficulty discerning their thoughts in the two temporal dimensions, but all three participants stated that they could manage this when ample time was given. The participants also commented that drawing their attention to the focuses of different sections was crucial given their

2.4. Data analysis Aside from descriptive statistical procedures, a 2 × 2 repeated measures MANOVA (with follow-up ANOVAs) were performed to answer the first research question. Such an analysis allowed us to identify the main effects of the type of classes (i.e. supplementary versus mainstream English classes) and the level (i.e. secondary-level versus tertiary-level perceptions), as well as the interaction effect between the two variables across the various aspects of English learning experiences. As for the second research question, a one-way repeated measures MANOVA was performed to contrast supplementary and mainstream English classes regarding their usefulness in equipping students with the English required for different purposes in the tertiary setting. The following table depicts the independent and dependent variables for our two research questions. 3. Results 3.1. General English learning experiences of supplementary and mainstream classes at secondary- and tertiary-level studies [RQ One] The descriptive statistics (with the rating from 1 (strongly disagree) to 5 (strongly agree)) of students’ perceptions on the 11 different aspects of English learning experiences are presented in Table 2. It is shown that descriptively, students perceived that supplementary classes helped them most with their reading, writing, and vocabulary in their secondary-level studies (with a mean rating of 3.80–3.83). In contrast, these classes appeared to be least helpful in developing speaking, motivation, English for work, and English for pleasure (with a mean rating of 3.11–3.24. From a tertiary perspective, not a single area was perceived to be fairly useful (i.e. no mean score close to 4); speaking (mean = 3.17) and motivation (mean = 3.03), were still considered to be the areas least improved. For mainstream classes, at secondary-level, the strengths were thought to lie in their advancement in speaking, writing, and grammar (with a mean rating of 3.44–3.55. However, these classes were limited in developing students’ motivation, confidence, English for work, and English for pleasure (with a mean of 2.85–3.09). As can be seen, at the

5 We thought that it made complete sense if learners could not see any differences between mainstream and supplementary education in certain areas, hence providing the option of neutrality in the five-point scale. See Appendix.

64

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

Table 2 Descriptive statistics of the perceived usefulness of supplementary and mainstream English classes for secondary-level and tertiary-level studies. Measure

Listening Speaking Reading Writing Vocabulary Grammar Motivation Confidence English for work English for pleasure Overall English proficiency

Secondary-level studies

Tertiary-level studies

Supplementary classes Mean (S.D.)

Mainstream classes Mean (S.D.)

Supplementary classes Mean (S.D.)

Mainstream classes Mean (S.D.)

3.45 3.22 3.83 3.80 3.80 3.57 3.13 3.33 3.11 3.24 3.51

3.40 3.44 3.37 3.48 3.38 3.55 2.95 3.09 2.85 3.04 3.32

3.20 3.17 3.42 3.46 3.54 3.47 3.03 3.30 3.26 3.29 3.39

3.32 3.40 3.32 3.40 3.37 3.49 3.13 3.17 3.15 3.18 3.37

(0.96) (1.11) (0.90) (0.83) (0.93) (0.93) (0.95) (0.91) (0.93) (1.00) (0.75)

(1.04) (1.07) (0.97) (0.93) (1.00) (0.98) (1.04) (1.07) (1.02) (1.06) (0.93)

(0.95) (1.08) (0.87) (0.90) (0.93) (0.92) (1.02) (0.99) (1.01) (1.02) (0.77)

(1.00) (1.10) (0.98) (1.00) (1.04) (0.97) (1.04) (0.99) (1.06) (1.03) (0.88)

significant for a number of variables: reading [F (1, 202) = 14.95, p < .001, η2 = 0.07], writing [F(1, 202) = 7.15, p = .008, η2 = 0.03], vocabulary [F(1, 202) = 6.51, p = .011, η2 = 0.03], motivation [F(1, 202) = 8.96, p = .003, η2 = 0.04] and overall English proficiency [F(1, 202) = 4.79, p = .030, η2 = 0.02]. Following up the interaction effects with separate t-tests indicated that students thought that the supplementary classes were significantly better than mainstream English classes in all these five aspects at secondary-level studies. However, at tertiary education, they found that the two types of classes were no longer different in benefiting them in reading, writing, and overall English proficiency Fig. 5. For vocabulary, although the mean differences narrowed (as shown in the converging lines in Fig. 3), there was still a significant effect between the two types of classes. Last, for motivation, supplementary classes were rated significantly better than mainstream classes; however, the opposite was found at tertiary level (see Fig. 4).

secondary level, the mean scores for the areas perceived to be most and least improved through attending mainstream classes are all lower than the corresponding scores under supplementary education except speaking. This suggests that learners generally perceive supplementary education has a more positive effect on their ESL/EFL development than mainstream classes. From the tertiary perspective, same as supplementary classes, the mean scores for mainstream classes cluster around 3.1–3.5. Inferential statistics below will make the differences between the levels and the settings clearer. From the 2 × 2 repeated measures MANOVA conducted, it was found that there was a significant main effect of type of classes: Pillai’s trace, V = 0.30, F(11,192) = 7.41, p < .001, η2 = 0.30, a significant main effect of education level: Pillai’s trace, V = 0.20, F (11,192) = 4.27, p < .001, η2 = 0.20, and a significant interaction effect: Pillai’s trace, V = 0.14, F(11,192) = 2.73, p = .003, η2 = 0.14. Repeated measures ANOVAs were conducted to follow up with the significant results.

3.2. Equipping students in using English for different purposes at tertiarylevel studies [RQ Two]

3.1.1. Supplementary vs mainstream English classes First, regarding the type of classes, there were significant main effects on speaking [F(1, 202) = 6.97, p = .009, η2 = 0.03], reading, F (1, 202) = 22.09, p < .001, η2 = 0.10], writing [F(1, 202) = 7.37, p = .007, η2 = 0.04], vocabulary [F(1, 202) = 17.97, p < .001, η2 = 0.08], confidence [F(1, 202) = 6.53, p = .011, η2 = 0.03], and English for work [F(1, 202) = 7.65, p = .006, η2 = 0.04]. For reading, writing, vocabulary, confidence, and English for work, students perceived that supplementary English classes were significantly more useful. On the other hand, for speaking, mainstream English classes were perceived as more effective.

Descriptive statistics are presented in Table 3. It can be found that in general, supplementary classes were not really helpful in all aspects involving speaking (with a mean rating of 2.86–3.03). However, these classes seemed to be better in developing students’ academic vocabulary, academic writing, academic reading, and reading assessment (with a mean rating of 3.49–3.57). With mainstream English classes, the strengths lie in developing students’ speaking skills in all aspects (with a

3.1.2. Secondary-level vs tertiary-level perceptions Significant main effects were found on listening [F(1, 202) = 8.20, p = .005, η2 = 0.04], reading [F(1, 202) = 20.20, p < .001, η2 = 0.09], writing [F(1, 202) = 15.77, p < .001, η2 = 0.07], vocabulary [F(1, 202) = 5.70, p = .018, η2 = 0.03], and English for work [F (1, 202) = 12.67, p < .001, η2 = 0.06]. For listening, reading, writing, and vocabulary, students used to think that the supplementary and mainstream English classes they had were useful when they were at secondary schools. However, from the perspective of tertiary education, they found these classes significantly less useful in improving the areas of listening, reading, writing, and vocabulary. With regard to English for work, students’ view changed in the other direction. When they had moved to tertiary education, they found that the classes they had were significantly more useful than they had first thought when they were at secondary schools. Fig. 5. Mean scores of the perceived usefulness of supplementary and mainstream English classes on overall English proficiency from secondary-level and tertiary-level perspectives.

3.1.3. Type of classes X education levels As shown in Figures below, interaction effects were found to be 65

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

Table 3 Descriptive statistics of the perceived usefulness of supplementary and mainstream English classes in equipping students for different purposes at tertiary-level studies. English for different purposes

Supplementary classes Mean (S.D.)

Mainstream classes Mean (S.D.)

Ability to listen to English for academic purposes (e.g. in lectures) Ability to listen to English in daily conversation Spoken English in presentations Spoken English in discussion in lectures/tutorials Spoken English for other academic purposes (e.g. speaking to professors, lecturers, etc.) Spoken English in daily conversation (e.g. with non-Chinese speakers) Ability to read academic materials (e.g. textbooks and journal articles) Ability to read assessment-related materials (e.g. marking schemes/assignment instructions) Ability to read emails and notices/posters Academic writing Ability to write non-academic texts (e.g. emails to tutors & notices/posters for clubs) Academic vocabulary (e.g. expressions you use in writing academic essays) Non-academic vocabulary (e.g. conversing with non-Chinese classmates) Overall, these sessions / classes are useful for my overall use of English at tertiary level

3.39 3.31 2.95 3.03 2.86 2.95 3.49 3.56 3.29 3.49 3.20 3.57 3.08 3.41

3.46 3.49 3.61 3.64 3.41 3.41 3.52 3.48 3.63 3.39 3.44 3.31 3.25 3.41

(0.87) (0.90) (1.13) (0.98) (1.03) (1.07) (0.98) (0.99) (0.93) (0.96) (1.02) (1.02) (1.02) (0.88)

(0.97) (0.92) (0.94) (0.87) (0.97) (1.00) (0.87) (0.84) (0.87) (0.93) (0.98) (0.89) (0.92) (0.85)

writing, vocabulary, confidence and English for work. In contrast, the effectiveness of mainstream English classes were always greater than supplementary ones in developing students’ English speaking even after moving to tertiary education. It is unsurprising that learners found supplementary education, highly targeting at improving learners’ examination scores (e.g. Yung, 2015; Zhan et al., 2013), more effective in developing their vocabulary and writing. The much less amount of lesson time in supplementary education (compared with that in mainstream education) is one likely reason that class time is usually allocated to vocabulary, reading, and writing, areas perceived to be more easily improved efficiently. There could be a few reasons for the greater confidence gained in the use of English from supplementary schools. There could well be correlations between learners’ perception of higher gains in proficiency such as vocabulary and writing and their confidence in using English. Another possible reason is the psychological satisfaction from having attended these additional classes. As Bray (2006) comments, learners attend tutorials for fear of being left behind and losing competitive edge. Zhan et al. (2013) also found that students gained confidence in examinations after attending supplementary classes. As many students in secondary schools are only likely to use English in examinations in Hong Kong (rather than, for example, using it as a lingua franca in authentic communication), the boost in confidence in examinations may well have contributed to the greater confidence in using English. Finally, it is easily conceivable that learners regarded mainstream education as more conducive to developing their spoken English than supplementary education does. The finding here is in line with Su (2005) study in which most forms of supplementary English education in Korea did not develop students’ spoken English but focus on other areas such as reading and writing. To the best of our knowledge, in Hong Kong, speaking is hardly ever the focus in many tutorial settings. As for most mainstream schools in Hong Kong, it is common practice that they designate specific lessons for speaking regularly. Having said this, we are aware that mainstream and supplementary ESL/EFL education elsewhere may have different focuses and arrangements. More studies are needed before more definitive conclusions can be drawn. Second, the usefulness of English classes, whether supplementary or mainstream, was discounted in terms of reading, listening, writing, and vocabulary when students moved to tertiary-level studies. English for work, however, was found to take the opposite direction where students started to appreciate more of these classes in enhancing their English for work. The decrement in reading, listening, writing, and vocabulary scores from the tertiary angle is perhaps attributable to the washback effect of examination and the heavily assessment-oriented culture in Hong Kong. Learners, teachers, and even teaching materials attach great importance to the examination syllabi (Biggs, 1998; Chow & Mok-

mean rating of 3.41–3.64), as well as reading in various aspects (mean rating of 3.48–3.63). However, they were not very helpful in areas such as academic writing and vocabulary development in general (with a mean rating of 3.25–3.39). To investigate if there are any significant differences between supplementary and mainstream classes across the various purposes of use of English at tertiary studies, a repeated measures MANOVA was conducted, revealing significant differences: Pillai’s trace, V = 0.43, F (14,173) = 9.41, p < .001, η2 = 0.43. Thus, repeated measures ANOVAs were used to follow up the results and identify which of the purposes were significantly different. It was found that mainstream classes were perceived to be significantly more helpful in a number of aspects: listening to daily conversations [F(1, 186) = 4.92, p = .028, η2 = 0.03], speaking during presentations [F (1, 186) = 42.98, p < .001, η2 = 0.19], speaking during lecture or tutorial discussion [F (1, 186) = 38.64, p < .001, η2 = 0.17], speaking for other academic English purposes such as speaking to professors [F(1, 186) = 32.74, p < .001, η2 = 0.15], speaking in daily conversation [F (1, 186) = 21.24, p < .001, η2 = 0.10], reading emails [F (1, 186) = 17.33, p < .001, η2 = 0.09], writing non-academic texts such as emails or notices for students’ clubs / societies [F (1, 186) = 6.99, p = .009, η2 = 0.04], and non-academic vocabulary [F (1, 186) = 4.64, p = .032, η2 = 0.02]. Supplementary English classes, however, were only significantly more effective in developing students’ academic vocabulary for tertiary studies [F(1, 186) = 10.29, p = .002, η2 = 0.05]. 4. Discussion In this study, we set out to investigate learners’ perspectives of the effects of supplementary and mainstream education on various areas in their ESL/EFL learning. The two research questions are answered below. 4.1. Learners’ perceptions of ELT from four angles: supplementary, mainstream, secondary-level and tertiary-level [RQ One] On the whole from both the secondary- and tertiary-level perspectives, learners rated both supplementary and mainstream classes from somewhat neutral to rather positive in facilitating their development in all eleven areas. Learners’ perception of specific areas in general English learning has been listed above under results and will not be reiterated here. Three key findings can be generalized from the statistical analyses. First, the areas where supplementary English classes were always (i.e. from either a secondary or a tertiary perspective) perceived to be more effective (with statistically significant differences) included reading, 66

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

Cheung, 2004). The widespread phenomenon of washback – “what is examined becomes what is taught” (Choi, 1999, p. 412) is commonplace. Therefore, when recounting secondary-level perspectives, learners could well have based their judgment of improvement on the scope of examinations; however, their definitions of these areas and skills (e.g. listening, writing) changed as they experience differently at tertiary level where the all-important public examination no longer exists and there are multifarious authentic situations in which English is employed (see Table 2). Learners’ views also changed for English for work. This was perceived more positively at tertiary level than at secondary level. We speculate that the learners were uncertain about what English for work entailed when they were in secondary schools, but as there are more opportunities for working part-time and/or engaging in work-related events at the tertiary level, learners realized that the overall experience (i.e. mainstream and supplementary ESL/EFL education) was more conducive to developing their English for work purposes than they had previously thought. Third, taken together the main and interaction effects, the perceived benefits of supplementary classes were generally discounted when students are promoted to tertiary education (as shown in the five Figures above). Supplementary English classes were thought to be significantly more effective than mainstream English classes in developing learners’ reading, writing, and overall English proficiency at secondarylevel studies. Nonetheless, when students progressed to the tertiary level, the perceived usefulness of supplementary classes in these areas was no longer superior to mainstream English classes. A possible reason for supplementary education being seen as statistically significantly more useful in enhancing these three areas than the mainstream education is, again, germane to the requirements in examinations. The learners might well have based their judgment on whether a particular form of education assisted effectively in their performing well in the examinations. It is especially intriguing to note that in reading (Fig. 1), both lines converge from secondary to tertiary levels with a substantial drop in usefulness of supplementary education. This can perhaps be explained by learners’ encounter with the genres at the tertiary level they find less familiar with such as academic texts. In other words, they realized that they were not as well equipped as they had previously thought to read texts at the tertiary level either in supplementary or mainstream secondary education. Similar explanations may also apply to writing (Fig. 2) that learners were not taught much academic writing in both settings at the secondary level.

Fig. 2. Mean scores of the perceived usefulness of supplementary and mainstream English classes on writing from secondary-level and tertiary-level perspectives.

Fig. 3. Mean scores of the perceived usefulness of supplementary and mainstream English classes on vocabulary from secondary-level and tertiary-level perspectives.

Fig. 4. Mean scores of the perceived usefulness of supplementary and mainstream English classes on motivation from secondary-level and tertiary-level perspectives. Fig. 1. Mean scores of the perceived usefulness of supplementary and mainstream English classes on reading from secondary-level and tertiary-level perspectives. 67

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

nature of this under-researched area, definitive conclusions cannot be drawn until more research is conducted. Some limitations of the present research have been highlighted for researchers when conducting future investigations. A number of implications for the stakeholders concerned are presented below. Supplementary classes, notoriously known as cram schools, are not necessarily viewed negatively by the learners themselves. It is interesting to note from Table 2 that overall, learners perceive supplementary education to be as useful as mainstream classes, despite the far less time of contact (to our knowledge and as evident in studies such as Yung (2015), the duration of each session spans usually one to two hours approximately, held weekly). Rather than focusing on the items associated with significant differences, the large number of items associated with non-significant differences is remarkable as they seem to imply the high efficiency in learning promoted by supplementary education. This is definitely noteworthy for mainstream schools and other stakeholders. Having said that, there are two potential concerns with supplementary classes. The first is that tutorial schools do not seem to focus much on non-examined or less important areas in assessments such as speaking. Second, these classes do not seem to promote learners’ linguistic skills (with the sole exception being academic vocabulary) to a significantly greater extent than mainstream schooling in handling different situations where English is used at the tertiary level. From the tertiary angle, there seems to be very little added value of supplementary education as perceived by the learners. This is understandable as the content covered in tutoring is mostly tailored for students to tackle examinations; however, the greater concern here is the possible mismatch between the scope of assessment and authentic use of English at tertiary level. As presented, supplementary education is considered significantly less helpful than mainstream schooling in equipping learners for certain everyday scenarios in section III of the questionnaire such as ‘ability to read emails and notices/posters’ and ‘ability to listen to English in daily conversation’. A highly possible implication here is that the scope of assessments at the secondary level may not be in line with even the basic use of English at the tertiary level. If the learners found the knowledge highly transferrable and there was a greater alignment between assessments and authentic use of English, the fruits of supplementary ESL/EFL education should extend to tertiary education in at least certain areas (e.g. the two examples given above), but unfortunately, this does not seem to be the case. Aside from these significant differences, when considering both forms of schooling together, not one item was rated '4' (meaning ‘agree’) or very close to ‘4’ in Table 3. This implies that learners do not generally find the knowledge they acquired from secondary schooling transferrable to the use of English for different purposes at tertiary settings. Although we acknowledge that the learning goals in secondary schooling may not align closely with those at tertiary level, given the prevalence of tertiary education worldwide, an imperative implication is that tertiary institutions should consider providing more relevant workshops and bridging programs (including what may be considered as basic skills such as reading emails and posters, and spoken English in daily conversation) especially for prospective students so that they are better equipped to use and understand English when they study at these institutions. All in all, although supplementary education is perceived to play an overall important role in ESL/EFL education as mainstream education at the secondary level, mainstream education should not be completely replaced by supplementary education. The participants in Yung (2016) commented that ESL/EFL supplementary education expanded thanks to students’ dissatisfaction with mainstream schooling (e.g. teachers’ lack of examination-orientedness, less effective teaching methods). Indeed, ESL/EFL tutoring is perceived to be more effective as far as examinations and secondary education is concerned. However, as uncovered in this study by the current tertiary students, learners should well be aware that there are certain benefits of mainstream education which

4.2. Learners’ perceptions of the usefulness of supplementary and mainstream ELT for their use of English in the tertiary setting [RQ Two] As regards the specific areas in the use of English in the tertiary setting, the findings suggest that mainstream English classes were on the whole better at helping students cope with the various needs at tertiary-level studies. Supplementary English classes appeared to be helpful only significantly more than mainstream classes in the area of academic vocabulary development. Learners are often taught a wide range of words and expressions in these classes (as revealed by some participants in Yung (2015) and Yung (2016)) and this is one likely reason why they thought supplementary classes facilitate their learning of academic vocabulary more than mainstream schools. Although many mainstream schools also focus on examinations, they tend to adhere much more to curricular guides by official bodies via, for example, incorporating a number of non-assessed areas in their syllabi such as whole-person development, generic skills such as communication skills and so forth. It could well be students’ experiences in these non-assessed areas that led them to view mainstream schools being better at equipping them in far more aspects (seven of these) than the supplementary schools do (only one aspect) for the use of English at the tertiary level. 5. Limitations Although the sample size reached 200 participants, in view of the heterogeneity in the nature of supplementary education (e.g. private, small-group, lecture-type), a larger number of participants would have contributed to greater generalizability of the findings. Future studies may also consider analyzing how the quantity and quality of supplementary education affect learners’ perspectives and actual achievements. As explained, in view of the exploratory nature and dearth of research in this area, it was our intention at this stage to only explore the general perceived effects of the two kinds of schooling, namely mainstream and supplementary education, and not to focus narrowly on the variables within each kind (e.g. frequency of tutoring; teachers’ pedagogical approaches). Also, being quantitative in nature, some interpretation of the findings was based on our knowledge and speculation (this was also partly because there is a dearth of relevant literature). Future research may consider adopting a qualitative or mixedmethods approach in which participants’ views and elaborations are solicited. Additionally, we were well aware that it might not have been methodologically ideal to include the retrospective part of the questionnaire (i.e. what learners thought when they were in secondary schools). We discussed the possibility of comparing and contrasting current secondary-level and tertiary-level students’ views but due to the vast differences in the settings and modes of supplementary education, as well as individual differences (e.g. amount of supplementary education received), we were concerned about the great threat to validity should this method be adopted. We, as well as the students in the pilot study, attempted the questionnaires and we were able to recall our perceptions and distinguish the differences between our current and past views. Therefore, we opted for the retrospective method of investigation. Last, this study has only cast light on learners’ perception of the effects of the two education settings; their actual academic performance can also be collated for analyses in order to compare the actual improvement with learners’ perceived improvement in future research. 6. Conclusion We have reported learners’ perceptions, as well as the various differences in their perceptions, of (a) the mode of education (i.e. mainstream versus supplementary) and (b) the level of education (i.e. secondary-level versus tertiary-level) in the general ESL/EFL context as well as (c) the usefulness of these classes for their specific use of English for different purposes in the tertiary context. Overall, differences were found in all the above comparisons. However, given the exploratory 68

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

extend beyond their secondary schooling.

Appendix A Section 1 of the questionnaire

Section 2 of the questionnaire

Section 3 of the questionnaire

69

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

177). (2nd ed.). Singapore: Eastern Universities Press. Chung, I. F. (2013). Crammed to learn English: What are learners’ motivation and approach? The Asia-Pacific Education Researcher, 22(4), 585–592. Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238. Curriculum Development Council (2017). English language education: Key learning area curriculum guide (primary 1–secondary 6). Retrieved fromhttps://www.edb.gov.hk/ attachment/en/curriculum-development/kla/eng-edu/Curriculum%20Document/ ELE%20KLACG_2017.pdf. Evans, S., & Green, C. (2007). Why EAP is necessary: A survey of Hong Kong tertiary students. Journal of English for Academic Purposes, 6(1), 3–17. Evans, S., & Morrison, B. (2011). Meeting the challenges of English-medium higher education: The first-year experience in Hong Kong. English for Specific Purposes, 30(3), 198–208. Forsey, M. (2013). But did it help you get to university? A qualitative study of supplementary education in Western Australia. In J. Aurini, S. Davies, & J. Dierkes (Eds.). Out of the shadows: The Global Intensification of Supplementary Education (International Perspectives on Education and Society, Volume 22) (pp. 171–189). Bingley, U.K: Emerald. Gao, X. (2008). You had to work hard ʼcause you didn’t know whether you were going to wear shoes or straw sandals!. Journal of Language, Identity and Education, 7(3–4), 169–187. Gueudet, G. (2008). Investigating the secondary–tertiary transition. Educational Studies in Mathematics, 67(3), 237–254. Hamid, M. O., Sussex, R., & Khan, A. (2009). Private tutoring in English for secondary school students in Bangladesh. TESOL Quarterly, 43(2), 281–308. Hamid, M. O., Khan, A., & Islam, M. M. (2017). The spread of private tutoring in English in developing societies: Exploring students’ perceptions. Discourse: Studies in the Cultural Politics of Education, 39(6), 868–886. Hoyles, C., Newman, K., & Noss, R. (2001). Changing patterns of transition from school to university mathematics. International Journal of Mathematical Education in Science and Technology, 32(6), 829–845. Ireson, J., & Rushforth, K. (2011). Private tutoring at transition points in the English education system: Its nature, extent and purpose. Research Papers in Education, 26(1), 1–19. Iveta, S. (2010). Private tutoring in Eastern Europe and Central Asia: Policy choices and

References Biggs, J. (1998). The assessment scene in Hong Kong. In P. Stimpson, & P. Morris (Eds.). Curriculum and assessment for Hong Kong: Two components, one system (pp. 315–324). Hong Kong: Open University of Hong Kong Press. Bloemert, J., Paran, A., Jansen, E., & van de Grift, W. (2017). Students’ perspective on the benefits of EFL literature education. The Language Learning Journal, 1–14. https://doi. org/10.1080/09571736.2017.1298149. Bray, M. (1999). The shadow education system: Private tutoring and its implications for planners. Paris: UNESCO, International Institute for Educational Planning. Bray, M. (2006). Private supplementary tutoring: Comparative perspectives on patterns and implications. Compare: A Journal of Comparative Education, 36(4), 515–530. Bray, M. (2013). Shadow education: Comparative perspectives on the expansion and implications of private supplementary tutoring. Procedia - Social and Behavioral Sciences, 77, 412–420. Bray, M. (2014). The Impact of shadow education on student academic achievement: Why the research is inconclusive and what can be done about it. Asia Pacific Education Review, 15(3), 381–389. Bray, M., & Lykins, C. (2012). Shadow education: Private supplementary tutoring and its implications for policy makers in Asia. Philippines: Asian Development Bank. Bray, M., & UNESCO (2009). Confronting the shadow education system: What government policies for what private tutoring? Paris: United Nations Educational, Scientific and Cultural Organization; International Institute for Educational Planning. Bray, M., Zhan, S., Lykins, C., Wang, D., & Kwo, O. (2014). Differentiated demand for private supplementary tutoring: Patterns and implications in Hong Kong secondary education. Economics of Education Review, 38, 24–37. Brown, A. V. (2009). Students’ and teachers’ perceptions of effective foreign language teaching: A comparison of ideals. The Modern Language Journal, 93(1), 46–60. Cai, L. J. (2013). Students’ perceptions of academic writing: A needs analysis of EAP in China. Language Education in Asia, 4(1), 5–22. Choi, C. C. (1999). Public examinations in Hong Kong. Assessment in Education: Principles, Policy & Practice, 6(3), 405–417. Chow, W. K. A., & Mok-Cheung, H. A. (2004). English language teaching in Hong Kong SAR: Tradition, transition and transformation. In H. W. Kam, & R. Y. L. Wong (Eds.). English language teaching in East Asia today: Changing policies and practices (pp. 150–

70

Studies in Educational Evaluation 62 (2019) 61–71

A. Tsang, et al.

indicator of academic performance at a tertiary institution. SA Journal of Human Resource Management, 2(3), 42–53. Su, S. O. (2005). Globalization and English language education in Korea: Communicative competence in English as an international language through private sector education (Unpublished Doctor of Education dissertation). USA: Arizona State University. Williams, M., & Burden, R. (1997). Psychology for language teachers: A social constructivist approach. Cambridge, England: Cambridge University Press. Yung, K. W. H. (2015). Learning English in the shadows: Understanding Chinese learners’ experiences of private tutoring. TESOL Quarterly, 49(4), 707–732. Yung, K. W. H. (2016). Learning English as a second language in Hong Kong: Understanding the selves of senior secondary students in shadow education. (Unpublished Doctor of Education dissertation). Hong Kong: The University of Hong Kong. Zhan, S., Bray, M., Wang, D., Lykins, C., & Kwo, O. (2013). The effectiveness of private tutoring: Student’s perception in comparison with mainstream schooling in Hong Kong. Asia Pacific Education Review, 14(4), 495–509.

implications. Compare: A Journal of Comparative and International Education, 40(3), 327–344. Kwok, L. Y. P. (2004). Emergence of demand for private supplementary tutoring in Hong Kong: Argument, indicators and implications. Hong Kong Teachers’ Centre Journal, 3, 1–14. Lee, B. (2010). The pre-university English-educational background of college freshmen in a foreign language program: A tale of diverse private education and English proficiency. Asia Pacific Education Review, 11, 69–82. Nunan, D. (2003). The impact of English as a global language on educational policies and practices in the Asia-Pacific region. TESOL Quarterly, 37, 589–613. Ross, S. J. (2008). Language testing in Asia: Evolution, innovation, and policy challenges. Language Testing, 25(1), 5–13. Smyth, E. (2008). The more, the better? Intensity of involvement in private tuition and examination performance. Educational Research and Evaluation, 14(5), 465–476. Stephen, D., Welman, J., & Jordaan, W. (2004). English language proficiency as an

71