- Email: [email protected]
Effectiveness of on-screen pinyin in learning Chinese: An expertise reversal for multimedia redundancy effect
Computers in Human Behavior 27 (2011) 11–15
Contents lists available at ScienceDirect
Computers in Human Behavior journal homepage: www.elsevier.com/locate/comphumbeh
Effectiveness of on-screen pinyin in learning Chinese: An expertise reversal for multimedia redundancy effect Chee Ha Lee *, Slava Kalyuga School of Education, The University of New South Wales, NSW, Australia
a r t i c l e
i n f o
Article history: Available online 31 May 2010 Keywords: Cognitive load Classic Chinese Pinyin Multimedia redundancy effect Expertise reversal effect
a b s t r a c t The reported study is aimed at developing effective techniques for reducing learner cognitive overload while using pinyin (a phonetic system) to learn the Chinese language. According to the multimedia redundancy effect, commonly used concurrent visual presentations of characters and pinyin, as well as their auditory pronunciations might impose an unnecessary extraneous load on learners and thus eliminate any potential benefits of using pinyin. In this study, learning effects of three computer-based presentation techniques (full on-screen pinyin transcription, partial on-screen pinyin transcription, and no on-screen pinyin) were compared for learners with higher and lower levels of prior language proficiency and pinyin knowledge. Results demonstrated the superiority of the partial pinyin condition over the two other conditions for more experienced learners. No differences between the three experimental groups were found for learners with lower levels of prior knowledge. The effectiveness of different pinyin presentation techniques depended on levels of learner prior knowledge (an expertise reversal effect). Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction The basic unit of the Chinese written language is a character. Because of the ideographic nature of the written language, pronunciation information conveyed by characters is very limited. Pinyin represents a modern phonic transcription system that assists in learning pronunciation of characters in Putonghua (the official spoken language of China, also called Mandarin). Pinyin shares the alphabet with English, and diacritical signs are marked above the vowel of a pinyin to indicate the tone of the word. A syllable pronounced in different tones would represent different characters and result in different meaning in the Chinese language (e.g., / ma¯/, /má/, /ma˘/, /mà/). In China, students are usually well trained in reading and pronouncing pinyin before they start learning characters. Pinyin is recommended for assisting students to read independently at early stages of learning. Therefore, characters in the Chinese language textbooks (and more recently, in computer-based materials) for early stage learning are usually transcribed with pinyin and only new words are transcribed with pinyin for later stages (Zhang & Zhu, 2007). It is assumed that young readers could comprehend text composed of a large amount of unfamiliar characters with the assis-
* Corresponding author. Address: School of Education, The University of New South Wales, Sydney, NSW 2052, Australia. Tel.: +61 2 9385 1958; fax: +61 2 9385 1946. E-mail address: [email protected] (C.H. Lee). 0747-5632/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2010.05.005
tance of the pinyin transcription. For example, Shu, Zeng, and Chen (1993) demonstrated that junior primary students performed better on a vocabulary meaning test (involving words familiar to participants in their spoken language) in a characters-with-pinyin experimental condition than in a characters-only condition. Shu and Liu (1994) also found the advantage of using pinyin to help young children in sentence reading. However, in both studies, the pinyin condition did not demonstrate any advantage in test questions that included vocabulary items that were not familiar to the participants in their spoken language. While the pinyin transcription for the words that were familiar to participants in their spoken language allowed them to comprehend the written text, it was not the case for unfamiliar words. Limitations of using pinyin for learners who do not use Mandarin as a daily spoken language (i.e., learners with a Chinese dialect background) were also demonstrated by Xu, Bao, and Xu (1997). In practice, pinyin is often used simultaneously with its auditory pronunciation. Chung (2003) compared a visual pinyin presentation with an auditory pronunciation condition (without pinyin transcription) and demonstrated superiority of the visual pinyin on a pronunciation recall test. This was a surprising result because it had been expected that encoding activities could be enhanced using a dual-modality instructional format (a character with its verbal pronunciation). According to the modality effect in cognitive load theory, presenting a picture (character) with written explanations (pinyin) could be less effective than presenting it with spoken explanations (pronunciation). Chung (2003) suggested that the visual pinyin might have allowed the
12
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
rehearsal of its pronunciation in working memory, while the verbal pronunciation vanished quickly and was difficult to recall. In regards to the effects of these instructional conditions on the meaning recall post-tests, no significant differences were observed. Based on the description of the experimental procedure, it is possible to speculate that participants’ prior skills in reading pinyin could have reduced working memory load and contributed to these results. The study described in this paper investigated the effects of simultaneously presenting visual pinyin and its auditory pronunciation compared to an auditory-only presentation (no-pinyin condition) using relatively more complex learning materials (classical Chinese texts) in a computer-based learning environment. The instruction with visual pinyin included two forms corresponding to different levels of pinyin presence: a full pinyin condition in which all the characters were transcribed with pinyin and a partial pinyin condition in which only potentially new or key characters were transcribed with pinyin. Classical Chinese is the language of ancient Chinese historical, philosophical and poetic texts. Relative to modern Chinese, it is characterized by using significantly fewer characters for expressing ideas. Also, grammar rules and vocabulary in classical Chinese are different from modern Chinese (Faurot, 1995). From the perspective of cognitive load theory, classical Chinese is characterized by high levels of element interactivity for beginners. For students who are not familiar with Mandarin (including participants in this study), the level of intrinsic cognitive load would be further increased. The majority of Hong Kong students use Cantonese (one of the Chinese dialects) for daily communication. Learning Mandarin is recommended for all students from Year 1 to Year 9 levels, and pinyin is used conventionally to accompany characters in Mandarin textbooks and computerbased materials to assist in learning pronunciation. Since Hong Kong students are familiar with English, distinguishing English and pinyin may impose an additional load before students become fluent pinyin readers. McGinnis (1997) suggested the term the can effect to describe the situations when pinyin spellings could be very easily confused with pronunciation of English (e.g., can is pronounced as /tshan/ in pinyin). In addition, students may also require cognitive resources to cross-reference the Mandarin pronunciation of characters with their Cantonese pronunciation. Thus, if pinyin is used with complex learning materials like classical Chinese texts, the level of intrinsic cognitive load could be very high. In this situation, according to the multimedia redundancy effect (Kalyuga, 2000; Kalyuga, Chandler, & Sweller, 1999, 2004; Mayer, Heiser, & Lonn, 2001), a concurrent presentation of Chinese characters (pictorial representations), pinyin (visual verbal representations), and its auditory pronunciation (spoken verbal representations) may impose an unnecessary extraneous load and thus eliminate any potential benefits of using pinyin. The visual pinyin or its auditory equivalent could be regarded as a redundant source of information. Since the auditory equivalent is essential for learners who may not be skillful in reading pinyin during early stages of learning, it is expected that visual pinyin may inhibit learning, especially for learners who are not familiar with spoken Mandarin and not experienced in reading pinyin. However, according to the expertise reversal effect (Kalyuga, 2007; Kalyuga, Ayres, Chandler, & Sweller, 2003), levels of learner prior experience could affect the effectiveness of an instructional technique. It could be predicted that the effectiveness of pinyin might depend on levels of learner prior language proficiency and experience with pinyin. Therefore, possible interactions between instructional formats and learner levels of expertise were also investigated in this study.
2. Methods 2.1. Participants Six classes of 240 Year 8 students (113 females and 127 males; average age 14.04 years, SD = 0.68) from two Hong Kong subsidized secondary schools participated in the experiment. At each school, three classes were selected randomly. 2.2. Materials and procedure The experiment included a pretest phase, a learning phase, and a post-test phase. 2.2.1. Pretest phase The pretest phase was conducted in a normal classroom setting in a single session of 25 min. Participants were first asked to complete a brief questionnaire to collect demographic information including family language background. The pretest was used to evaluate the overall level of participants’ combined prior knowledge of pinyin and Mandarin language. The paper-based, multiple-choice pretest consisted of three questions each of which included 12 items, and participants were instructed by a pre-recorded voice message. The first two questions were used to evaluate familiarity with pinyin and ability to distinguish four tones of Mandarin. Students were requested to match a written pinyin with a pronunciation they heard. The last question evaluated students’ familiarity with pronunciations of some of the characters selected from the learning materials. Students were requested to judge whether a pronunciation they heard corresponding to a character was correct or not. Each response was scored 1 (correct) or 0 (incorrect). The maximum score of 36 could be obtained if all items were answered correctly. The answer options ‘I don’t know if it is correct or incorrect’ or ‘I can’t make a judgment’ were included in all items to prevent participants from guessing their answers. Cronbach’s a of 0.65 (for a modified sample N = 120, see Section 3) indicated a sufficiently high level of internal consistency for the three pretest questions. Following the pretest phase, participants in each school were listed in ascending order according to their pretest scores and alternatively allocated to three experimental conditions (i.e., full pinyin transcription, partial pinyin transcription, and no-pinyin condition). 2.2.2. Learning phase The learning and post-test phases were conducted 1 week later in the language laboratory in a single session of 40 min. Twelve sentences were selected from several classic Chinese text. Using a pre-recorded voice message, the participants were instructed to read and comprehend 12 sentences shown on the computer screen and learn the pronunciation of the characters in Mandarin. They were also told that a short test would immediately follow the learning phase. The learning materials were presented on the computer screen using PowerPoint slides. The presentation used traditional characters and a common Chinese font Kaiti ( ). The visual pinyin transcription in the corresponding experimental condition was presented just above the characters (see Fig. 1). This presentation format corresponded to the format in the Mandarin textbooks and materials used in the two participating schools. The distances between the Chinese characters and pinyin were the same across different PowerPoint slides. The levels of difficulty of the sentences were judged by four experienced Chinese language teachers from the participating schools as appropriate for their Year 8 students (the content and structure of the sentences were not too hard or too easy to comprehend). The time spent on reading each sentence was equalized throughout the three
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
13
for sentence comprehension and pronunciation items to prevent participants from making random choices. The post-test was accompanied by pre-recorded instructions to ensure that all participants were allocated the same amount of time for the test. They were not allowed to go back to the previous questions. The first set of five items was designed to assess comprehension of the learning materials. The measures were based on evaluating the adequacy of the learner constructed text representations after reading the learning materials. In each item, participants were given a sentence written in modern Chinese with meaning presumably corresponding to the meaning conveyed by the learning materials and were asked to judge whether the statement was correct or not. Statements were written according to possible correct or incorrect representations. The adequacy of learner representations was judged by their ability to verify the correctness of the presented statements. The second set of six items was designed to evaluate learners’ recognition of characters. Corresponding six sentences were selected from the learning materials with some characters modified (usually a homophone or a character that shares a similar graphic structure with the correct original one). Participants were asked to circle a wrongly used character (if any) and write down the correct one if applicable. A voice-over pronouncing the sentences was provided. The third set of seven items was designed to evaluate learners’ pronunciation performance. For these items, sentences or phrases were selected from the learning materials. For each item, students were required to judge whether the pronunciation of the target character (presented with a bracket) was correct or not. For the first and third sets of items, each response was scored 1 (correct) or 0 (incorrect). For the second set of items, each response was scored 1 (correct), 0.5 (the correct character was not provided for a correctly identified modified character) or 0 (incorrect). The maximum scores of 5, 6, and 7 could be obtained for measures of sentence comprehension, recognition, and pronunciation correspondingly if all items were answered correctly.
3. Results
Fig. 1. Examples of experimental formats used in the learning phase.
instructional conditions. The learning procedure and the sentence presentation order were also the same. Each sentence (with full, partial, or none pinyin transcription added depending on the experimental condition) was first shown on the computer screen for 3–7 s depending on the length of the sentence. It was then followed by a voice-over pronouncing the sentence three times with 4 s intervals between each reading (the sentence remained on the screen). The PowerPoint slides changed automatically, and the word (The end) was shown on the last slide. 2.2.3. Post-test phase A multiple-choice, paper-based test consisted of three sets of items measuring sentence comprehension, recognition of characters, and pronunciation, respectively. The options ‘I don’t know if it is correct or incorrect’ or ‘I can’t make a judgment’ were provided
The dependent variables under analysis were post-test scores for sentence comprehension, recognition of characters, and pronunciation, while the independent variables were instructional formats (full pinyin, partial pinyin, or no pinyin) and levels of learner expertise (lower and higher level). In order to exclude students with a strong Mandarin speaking background, the participants who had indicated in the questionnaire that they studied in Mainland China and communicated with their family members (native Mandarin speakers) in Mandarin were eliminated from the analysis. Pretest scores for the remaining 178 students were used as indicators of levels of learner prior experience with pinyin and Mandarin language. In order to obtain two groups of learners with distinctively different levels of prior knowledge, the pretest scores were arranged in an ascending order and a median range of about one third of participants (58 students with pretest scores between 21 and 25) was excluded from the analyses. Subsequently, the total number of participants in the modified experimental groups was 120. With the three modified groups (n = 43 for the pinyin group, n = 39 for the partial pinyin group, and n = 38 for the no-pinyin group), analysis of variance (ANOVA) indicated a highly non-significant difference for the pretest scores, F(2, 117) = 0.78, MSE = 36.40 (M = 22.53, SD = 6.54 for the pinyin group; M = 22.44, SD = 5.68 for the partial pinyin group; and M = 22.95, SD = 5.78 for the no-pinyin group; .05 significance level was used throughout the manuscript). Thus, the three groups of participants were at the same level of
14
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
auditory pronunciation, the results demonstrated an interaction between instructional conditions and levels of learner expertise (an expertise reversal effect). The partial pinyin condition outperformed the no-pinyin conditions for more knowledgeable learners, however this superiority was found in learning pronunciation but not in sentence comprehension and recognition of characters. For less knowledgeable learners, there were no differences among the three conditions.
7 full pinyin 6.5
partial pinyin no pinyin
6 5.5
4. Discussion
5 4.5 4 3.5 3 2.5 2 lower prior knowledge level
higher prior knowledge level
Fig. 2. Interaction between instructional formats and levels of learner expertise for post-test scores measuring learning of pronunciation.
prior familiarity with pinyin and Mandarin language, and none of the groups had an initial advantage in the experiment. MANOVA for the three dependent variables indicated no significant main effect for experimental conditions, H = 0.43, F(3, 113) = 1.62, g2p = 0.04. Expectedly, there was a significant main effect for the expertise levels, H = 0.22, F(3, 112) = 8.23, g2p = 0.18, p < .01. Most importantly, MANOVA indicated a significant interaction between instructional conditions and levels of learner expertise, H = 0.08, F(3, 113) = 3.03, g2p = 0.07, p < .05 (see Fig. 2). Means and standard deviations are provided in Table 1. Separate univariate ANOVAs for interactions between instructional conditions and levels of learner expertise for each dependent variable revealed a significant effect for measures of pronunciation learning, F(2, 114) = 4.04, MSE = 2.19, p < .05, g2p = 0.07. There were no significant effects for measures of sentence comprehension, F(2, 114) = 0.20, MSE = 0.94, and recognition of characters, F(2, 114) = 1.97, MSE = 0.99. In regards to simple effects for learning pronunciation, ANOVAs indicated a significant difference between the three instructional formats for higher knowledge level learners, F(2, 54) = 3.67, MSE = 2.69, p < .05, g2p = 0.12, with no simple effect for lower knowledge level learners, F(2, 60) = 0.65, MSE = 1.73. According to Tukey HSD post hoc test, the partial pinyin format outperformed the nopinyin format for the higher knowledge level learners (p < .05). Thus, although no overall multimedia redundancy effect was found in using visual pinyin transcription simultaneously with its
This paper investigated cognitive load aspects of learning the Chinese language using a phonic transcription system called pinyin. Because of the ideographic nature of the Chinese language, it is conventionally taken for granted that pinyin is always useful in providing pronunciation information for Chinese characters and should accompany characters in learning materials for beginner learners. It is also a common practice to present characters with both pinyin and verbal pronunciation instructions concurrently. The multimedia redundancy effect generated by cognitive load theory may render the presentation of pinyin accompanied by its verbal pronunciation an ineffective instructional method. Learners might need to cross-reference the spoken and visual sources of essentially the same information. Mental processing of duplicate pronunciation information might therefore leave insufficient cognitive resources for learning schemas required for comprehension, characters recognition, and learning pronunciation. It is suggested that an instructional format without visual pinyin (i.e., presenting characters and their verbal pronunciation) could be a more effective alternative. The results of the described experiment indicated no overall multimedia redundancy effect. However, an expertise reversal effect was found demonstrating a significant interaction between the effectiveness of alternative instructional techniques in learning pronunciation and levels of learner prior experience. For less experienced learners, no differences were found between the full pinyin transcription condition, partial pinyin transcription condition, and no-pinyin condition. Insufficient knowledge in learner long-term memory may have resulted in a very high level of intrinsic cognitive load that overloaded working memory. For more experienced learners, the partial pinyin condition outperformed the no-pinyin instructional format in learning pronunciation, with no differences between other conditions. The available knowledge structures may have reduced the level of intrinsic load sufficiently to allow processing of the redundant visual pinyin. Mayer and Johnson (2008) have recently demonstrated that the multimedia redundancy principle (according to which graphics and narration are superior to graphics, narration, and on-screen text) may not apply when the on-screen text is a short version of the narration that highlights its key points and is placed next to the corresponding portion of the graphic. The pinyin transcription in the partial pinyin condition was provided only next to potentially new (key) characters to draw the learner’s attention and could be regarded as exactly this case. The above advantage of partial pinyin was not demonstrated for the sentence comprehension and character recognition test tasks
Table 1 Means and standard deviations for post-test scores measuring learning pronunciation for different experimental conditions and levels of learner prior knowledge. Lower prior knowledge level
M SD
Higher prior knowledge level
Full pinyin
Partial pinyin
No pinyin
Full pinyin
Partial pinyin
No pinyin
3.50 1.10
3.09 1.02
3.47 1.78
4.52 1.89
5.47 1.23
4.00 1.67
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
for which no differences were found between the three experimental conditions. This result is in correspondence with some previous studies (Shu et al., 1993; Shu & Liu, 1994) that have suggested that even though pinyin enhanced independent reading and character recognition for young Chinese children (native speakers), it may not be beneficial for children with dialect background when dealing with new vocabulary items that are unfamiliar to learners in their spoken language (Xu et al., 1997). This is the category of participants used in this study. The benefits of pinyin were demonstrated only in learning pronunciation. The conventional believe that pinyin should necessarily benefit learning the Chinese language may not be always correct. The reported study indicates that the effectiveness of pinyin as a means of instructional support may depend on learner prior language proficiency and experience with pinyin, as well as on the level of the provided support (full or partial support). The effect may also apply only to certain aspect of language learning (learning pronunciation of characters). The results highlight the importance of assessing actual levels of learner prior knowledge and tailoring accordingly the levels of pinyin support and presentation formats. The effectiveness of such adaptive computer-based learning environments (including instructional procedures based on learner control over required levels of pinyin support) needs to be investigated in future studies. Acknowledgments The authors would like to thank Yin Ping Chiu, Yee Mei Wan, and the Principals of the participating schools for their support in this study. The authors also thank the technicians and the students for their cooperation.
15
References Chung, K. K. H. (2003). Effects of pinyin and first language words in learning of Chinese characters as a second language. Journal of Behavioral Education, 12, 207–223. Faurot, J. L. (1995). Gateway to the Chinese Classics: A practical guide to literary Chinese. San Francisco, CA: China Books and Periodicals. Kalyuga, S. (2000). When using sound with a text or picture is not beneficial for learning. Australian Journal of Educational Technology, 16, 161–172. Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 16, 161–172. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). Expertise reversal effect. Educational Psychologist, 38, 23–31. Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371. Kalyuga, S., Chandler, P., & Sweller, J. (2004). When redundant on-screen text in multimedia technical instruction can interfere with learning. Human Factors, 46, 567–581. Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93, 187–198. Mayer, R. E., & Johnson, C. I. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380–386. McGinnis, S. (1997). Tonal spelling versus diacritics for teaching pronunciation of Mandarin Chinese. The Modern Language Journal, 81, 228–236. Shu, H., & Liu, B. X. (1994). Pinyin zai xiaoxue dinianji ertong zaoqi yuedu zhong zuoyong de yanjiu (The role of pinyin in early reading for Chinese junior primary school children). Psychological Development and Education, 3, 11–15. Shu, H., Zeng, H. M., & Chen, Z. (1993). Xiaoxue dinianji ertong liyong pinyin xuexi shengzici de shiyan yanjiu (Research on using pinyin in learning vocabulary by junior primary school children). Psychological Development and Education, 1, 18–22. Xu, F., Bao, X. H., & Xu, M. (1997). Pinyin zai fangyan ertong zici xuexi zhong de zuoyong (The role of pinyin in learning vocabulary for Chinese children with dialect background). Chinese Journal of Applied Psychology, 3, 39–44. Zhang, Q., & Zhu, J. L. (Eds.). (2007). Yu Wen (Language). Year 1–Year 6. Nanjing, China: Jiangsu Education Publishing House.
Contents lists available at ScienceDirect
Computers in Human Behavior journal homepage: www.elsevier.com/locate/comphumbeh
Effectiveness of on-screen pinyin in learning Chinese: An expertise reversal for multimedia redundancy effect Chee Ha Lee *, Slava Kalyuga School of Education, The University of New South Wales, NSW, Australia
a r t i c l e
i n f o
Article history: Available online 31 May 2010 Keywords: Cognitive load Classic Chinese Pinyin Multimedia redundancy effect Expertise reversal effect
a b s t r a c t The reported study is aimed at developing effective techniques for reducing learner cognitive overload while using pinyin (a phonetic system) to learn the Chinese language. According to the multimedia redundancy effect, commonly used concurrent visual presentations of characters and pinyin, as well as their auditory pronunciations might impose an unnecessary extraneous load on learners and thus eliminate any potential benefits of using pinyin. In this study, learning effects of three computer-based presentation techniques (full on-screen pinyin transcription, partial on-screen pinyin transcription, and no on-screen pinyin) were compared for learners with higher and lower levels of prior language proficiency and pinyin knowledge. Results demonstrated the superiority of the partial pinyin condition over the two other conditions for more experienced learners. No differences between the three experimental groups were found for learners with lower levels of prior knowledge. The effectiveness of different pinyin presentation techniques depended on levels of learner prior knowledge (an expertise reversal effect). Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction The basic unit of the Chinese written language is a character. Because of the ideographic nature of the written language, pronunciation information conveyed by characters is very limited. Pinyin represents a modern phonic transcription system that assists in learning pronunciation of characters in Putonghua (the official spoken language of China, also called Mandarin). Pinyin shares the alphabet with English, and diacritical signs are marked above the vowel of a pinyin to indicate the tone of the word. A syllable pronounced in different tones would represent different characters and result in different meaning in the Chinese language (e.g., / ma¯/, /má/, /ma˘/, /mà/). In China, students are usually well trained in reading and pronouncing pinyin before they start learning characters. Pinyin is recommended for assisting students to read independently at early stages of learning. Therefore, characters in the Chinese language textbooks (and more recently, in computer-based materials) for early stage learning are usually transcribed with pinyin and only new words are transcribed with pinyin for later stages (Zhang & Zhu, 2007). It is assumed that young readers could comprehend text composed of a large amount of unfamiliar characters with the assis-
* Corresponding author. Address: School of Education, The University of New South Wales, Sydney, NSW 2052, Australia. Tel.: +61 2 9385 1958; fax: +61 2 9385 1946. E-mail address: [email protected] (C.H. Lee). 0747-5632/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2010.05.005
tance of the pinyin transcription. For example, Shu, Zeng, and Chen (1993) demonstrated that junior primary students performed better on a vocabulary meaning test (involving words familiar to participants in their spoken language) in a characters-with-pinyin experimental condition than in a characters-only condition. Shu and Liu (1994) also found the advantage of using pinyin to help young children in sentence reading. However, in both studies, the pinyin condition did not demonstrate any advantage in test questions that included vocabulary items that were not familiar to the participants in their spoken language. While the pinyin transcription for the words that were familiar to participants in their spoken language allowed them to comprehend the written text, it was not the case for unfamiliar words. Limitations of using pinyin for learners who do not use Mandarin as a daily spoken language (i.e., learners with a Chinese dialect background) were also demonstrated by Xu, Bao, and Xu (1997). In practice, pinyin is often used simultaneously with its auditory pronunciation. Chung (2003) compared a visual pinyin presentation with an auditory pronunciation condition (without pinyin transcription) and demonstrated superiority of the visual pinyin on a pronunciation recall test. This was a surprising result because it had been expected that encoding activities could be enhanced using a dual-modality instructional format (a character with its verbal pronunciation). According to the modality effect in cognitive load theory, presenting a picture (character) with written explanations (pinyin) could be less effective than presenting it with spoken explanations (pronunciation). Chung (2003) suggested that the visual pinyin might have allowed the
12
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
rehearsal of its pronunciation in working memory, while the verbal pronunciation vanished quickly and was difficult to recall. In regards to the effects of these instructional conditions on the meaning recall post-tests, no significant differences were observed. Based on the description of the experimental procedure, it is possible to speculate that participants’ prior skills in reading pinyin could have reduced working memory load and contributed to these results. The study described in this paper investigated the effects of simultaneously presenting visual pinyin and its auditory pronunciation compared to an auditory-only presentation (no-pinyin condition) using relatively more complex learning materials (classical Chinese texts) in a computer-based learning environment. The instruction with visual pinyin included two forms corresponding to different levels of pinyin presence: a full pinyin condition in which all the characters were transcribed with pinyin and a partial pinyin condition in which only potentially new or key characters were transcribed with pinyin. Classical Chinese is the language of ancient Chinese historical, philosophical and poetic texts. Relative to modern Chinese, it is characterized by using significantly fewer characters for expressing ideas. Also, grammar rules and vocabulary in classical Chinese are different from modern Chinese (Faurot, 1995). From the perspective of cognitive load theory, classical Chinese is characterized by high levels of element interactivity for beginners. For students who are not familiar with Mandarin (including participants in this study), the level of intrinsic cognitive load would be further increased. The majority of Hong Kong students use Cantonese (one of the Chinese dialects) for daily communication. Learning Mandarin is recommended for all students from Year 1 to Year 9 levels, and pinyin is used conventionally to accompany characters in Mandarin textbooks and computerbased materials to assist in learning pronunciation. Since Hong Kong students are familiar with English, distinguishing English and pinyin may impose an additional load before students become fluent pinyin readers. McGinnis (1997) suggested the term the can effect to describe the situations when pinyin spellings could be very easily confused with pronunciation of English (e.g., can is pronounced as /tshan/ in pinyin). In addition, students may also require cognitive resources to cross-reference the Mandarin pronunciation of characters with their Cantonese pronunciation. Thus, if pinyin is used with complex learning materials like classical Chinese texts, the level of intrinsic cognitive load could be very high. In this situation, according to the multimedia redundancy effect (Kalyuga, 2000; Kalyuga, Chandler, & Sweller, 1999, 2004; Mayer, Heiser, & Lonn, 2001), a concurrent presentation of Chinese characters (pictorial representations), pinyin (visual verbal representations), and its auditory pronunciation (spoken verbal representations) may impose an unnecessary extraneous load and thus eliminate any potential benefits of using pinyin. The visual pinyin or its auditory equivalent could be regarded as a redundant source of information. Since the auditory equivalent is essential for learners who may not be skillful in reading pinyin during early stages of learning, it is expected that visual pinyin may inhibit learning, especially for learners who are not familiar with spoken Mandarin and not experienced in reading pinyin. However, according to the expertise reversal effect (Kalyuga, 2007; Kalyuga, Ayres, Chandler, & Sweller, 2003), levels of learner prior experience could affect the effectiveness of an instructional technique. It could be predicted that the effectiveness of pinyin might depend on levels of learner prior language proficiency and experience with pinyin. Therefore, possible interactions between instructional formats and learner levels of expertise were also investigated in this study.
2. Methods 2.1. Participants Six classes of 240 Year 8 students (113 females and 127 males; average age 14.04 years, SD = 0.68) from two Hong Kong subsidized secondary schools participated in the experiment. At each school, three classes were selected randomly. 2.2. Materials and procedure The experiment included a pretest phase, a learning phase, and a post-test phase. 2.2.1. Pretest phase The pretest phase was conducted in a normal classroom setting in a single session of 25 min. Participants were first asked to complete a brief questionnaire to collect demographic information including family language background. The pretest was used to evaluate the overall level of participants’ combined prior knowledge of pinyin and Mandarin language. The paper-based, multiple-choice pretest consisted of three questions each of which included 12 items, and participants were instructed by a pre-recorded voice message. The first two questions were used to evaluate familiarity with pinyin and ability to distinguish four tones of Mandarin. Students were requested to match a written pinyin with a pronunciation they heard. The last question evaluated students’ familiarity with pronunciations of some of the characters selected from the learning materials. Students were requested to judge whether a pronunciation they heard corresponding to a character was correct or not. Each response was scored 1 (correct) or 0 (incorrect). The maximum score of 36 could be obtained if all items were answered correctly. The answer options ‘I don’t know if it is correct or incorrect’ or ‘I can’t make a judgment’ were included in all items to prevent participants from guessing their answers. Cronbach’s a of 0.65 (for a modified sample N = 120, see Section 3) indicated a sufficiently high level of internal consistency for the three pretest questions. Following the pretest phase, participants in each school were listed in ascending order according to their pretest scores and alternatively allocated to three experimental conditions (i.e., full pinyin transcription, partial pinyin transcription, and no-pinyin condition). 2.2.2. Learning phase The learning and post-test phases were conducted 1 week later in the language laboratory in a single session of 40 min. Twelve sentences were selected from several classic Chinese text. Using a pre-recorded voice message, the participants were instructed to read and comprehend 12 sentences shown on the computer screen and learn the pronunciation of the characters in Mandarin. They were also told that a short test would immediately follow the learning phase. The learning materials were presented on the computer screen using PowerPoint slides. The presentation used traditional characters and a common Chinese font Kaiti ( ). The visual pinyin transcription in the corresponding experimental condition was presented just above the characters (see Fig. 1). This presentation format corresponded to the format in the Mandarin textbooks and materials used in the two participating schools. The distances between the Chinese characters and pinyin were the same across different PowerPoint slides. The levels of difficulty of the sentences were judged by four experienced Chinese language teachers from the participating schools as appropriate for their Year 8 students (the content and structure of the sentences were not too hard or too easy to comprehend). The time spent on reading each sentence was equalized throughout the three
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
13
for sentence comprehension and pronunciation items to prevent participants from making random choices. The post-test was accompanied by pre-recorded instructions to ensure that all participants were allocated the same amount of time for the test. They were not allowed to go back to the previous questions. The first set of five items was designed to assess comprehension of the learning materials. The measures were based on evaluating the adequacy of the learner constructed text representations after reading the learning materials. In each item, participants were given a sentence written in modern Chinese with meaning presumably corresponding to the meaning conveyed by the learning materials and were asked to judge whether the statement was correct or not. Statements were written according to possible correct or incorrect representations. The adequacy of learner representations was judged by their ability to verify the correctness of the presented statements. The second set of six items was designed to evaluate learners’ recognition of characters. Corresponding six sentences were selected from the learning materials with some characters modified (usually a homophone or a character that shares a similar graphic structure with the correct original one). Participants were asked to circle a wrongly used character (if any) and write down the correct one if applicable. A voice-over pronouncing the sentences was provided. The third set of seven items was designed to evaluate learners’ pronunciation performance. For these items, sentences or phrases were selected from the learning materials. For each item, students were required to judge whether the pronunciation of the target character (presented with a bracket) was correct or not. For the first and third sets of items, each response was scored 1 (correct) or 0 (incorrect). For the second set of items, each response was scored 1 (correct), 0.5 (the correct character was not provided for a correctly identified modified character) or 0 (incorrect). The maximum scores of 5, 6, and 7 could be obtained for measures of sentence comprehension, recognition, and pronunciation correspondingly if all items were answered correctly.
3. Results
Fig. 1. Examples of experimental formats used in the learning phase.
instructional conditions. The learning procedure and the sentence presentation order were also the same. Each sentence (with full, partial, or none pinyin transcription added depending on the experimental condition) was first shown on the computer screen for 3–7 s depending on the length of the sentence. It was then followed by a voice-over pronouncing the sentence three times with 4 s intervals between each reading (the sentence remained on the screen). The PowerPoint slides changed automatically, and the word (The end) was shown on the last slide. 2.2.3. Post-test phase A multiple-choice, paper-based test consisted of three sets of items measuring sentence comprehension, recognition of characters, and pronunciation, respectively. The options ‘I don’t know if it is correct or incorrect’ or ‘I can’t make a judgment’ were provided
The dependent variables under analysis were post-test scores for sentence comprehension, recognition of characters, and pronunciation, while the independent variables were instructional formats (full pinyin, partial pinyin, or no pinyin) and levels of learner expertise (lower and higher level). In order to exclude students with a strong Mandarin speaking background, the participants who had indicated in the questionnaire that they studied in Mainland China and communicated with their family members (native Mandarin speakers) in Mandarin were eliminated from the analysis. Pretest scores for the remaining 178 students were used as indicators of levels of learner prior experience with pinyin and Mandarin language. In order to obtain two groups of learners with distinctively different levels of prior knowledge, the pretest scores were arranged in an ascending order and a median range of about one third of participants (58 students with pretest scores between 21 and 25) was excluded from the analyses. Subsequently, the total number of participants in the modified experimental groups was 120. With the three modified groups (n = 43 for the pinyin group, n = 39 for the partial pinyin group, and n = 38 for the no-pinyin group), analysis of variance (ANOVA) indicated a highly non-significant difference for the pretest scores, F(2, 117) = 0.78, MSE = 36.40 (M = 22.53, SD = 6.54 for the pinyin group; M = 22.44, SD = 5.68 for the partial pinyin group; and M = 22.95, SD = 5.78 for the no-pinyin group; .05 significance level was used throughout the manuscript). Thus, the three groups of participants were at the same level of
14
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
auditory pronunciation, the results demonstrated an interaction between instructional conditions and levels of learner expertise (an expertise reversal effect). The partial pinyin condition outperformed the no-pinyin conditions for more knowledgeable learners, however this superiority was found in learning pronunciation but not in sentence comprehension and recognition of characters. For less knowledgeable learners, there were no differences among the three conditions.
7 full pinyin 6.5
partial pinyin no pinyin
6 5.5
4. Discussion
5 4.5 4 3.5 3 2.5 2 lower prior knowledge level
higher prior knowledge level
Fig. 2. Interaction between instructional formats and levels of learner expertise for post-test scores measuring learning of pronunciation.
prior familiarity with pinyin and Mandarin language, and none of the groups had an initial advantage in the experiment. MANOVA for the three dependent variables indicated no significant main effect for experimental conditions, H = 0.43, F(3, 113) = 1.62, g2p = 0.04. Expectedly, there was a significant main effect for the expertise levels, H = 0.22, F(3, 112) = 8.23, g2p = 0.18, p < .01. Most importantly, MANOVA indicated a significant interaction between instructional conditions and levels of learner expertise, H = 0.08, F(3, 113) = 3.03, g2p = 0.07, p < .05 (see Fig. 2). Means and standard deviations are provided in Table 1. Separate univariate ANOVAs for interactions between instructional conditions and levels of learner expertise for each dependent variable revealed a significant effect for measures of pronunciation learning, F(2, 114) = 4.04, MSE = 2.19, p < .05, g2p = 0.07. There were no significant effects for measures of sentence comprehension, F(2, 114) = 0.20, MSE = 0.94, and recognition of characters, F(2, 114) = 1.97, MSE = 0.99. In regards to simple effects for learning pronunciation, ANOVAs indicated a significant difference between the three instructional formats for higher knowledge level learners, F(2, 54) = 3.67, MSE = 2.69, p < .05, g2p = 0.12, with no simple effect for lower knowledge level learners, F(2, 60) = 0.65, MSE = 1.73. According to Tukey HSD post hoc test, the partial pinyin format outperformed the nopinyin format for the higher knowledge level learners (p < .05). Thus, although no overall multimedia redundancy effect was found in using visual pinyin transcription simultaneously with its
This paper investigated cognitive load aspects of learning the Chinese language using a phonic transcription system called pinyin. Because of the ideographic nature of the Chinese language, it is conventionally taken for granted that pinyin is always useful in providing pronunciation information for Chinese characters and should accompany characters in learning materials for beginner learners. It is also a common practice to present characters with both pinyin and verbal pronunciation instructions concurrently. The multimedia redundancy effect generated by cognitive load theory may render the presentation of pinyin accompanied by its verbal pronunciation an ineffective instructional method. Learners might need to cross-reference the spoken and visual sources of essentially the same information. Mental processing of duplicate pronunciation information might therefore leave insufficient cognitive resources for learning schemas required for comprehension, characters recognition, and learning pronunciation. It is suggested that an instructional format without visual pinyin (i.e., presenting characters and their verbal pronunciation) could be a more effective alternative. The results of the described experiment indicated no overall multimedia redundancy effect. However, an expertise reversal effect was found demonstrating a significant interaction between the effectiveness of alternative instructional techniques in learning pronunciation and levels of learner prior experience. For less experienced learners, no differences were found between the full pinyin transcription condition, partial pinyin transcription condition, and no-pinyin condition. Insufficient knowledge in learner long-term memory may have resulted in a very high level of intrinsic cognitive load that overloaded working memory. For more experienced learners, the partial pinyin condition outperformed the no-pinyin instructional format in learning pronunciation, with no differences between other conditions. The available knowledge structures may have reduced the level of intrinsic load sufficiently to allow processing of the redundant visual pinyin. Mayer and Johnson (2008) have recently demonstrated that the multimedia redundancy principle (according to which graphics and narration are superior to graphics, narration, and on-screen text) may not apply when the on-screen text is a short version of the narration that highlights its key points and is placed next to the corresponding portion of the graphic. The pinyin transcription in the partial pinyin condition was provided only next to potentially new (key) characters to draw the learner’s attention and could be regarded as exactly this case. The above advantage of partial pinyin was not demonstrated for the sentence comprehension and character recognition test tasks
Table 1 Means and standard deviations for post-test scores measuring learning pronunciation for different experimental conditions and levels of learner prior knowledge. Lower prior knowledge level
M SD
Higher prior knowledge level
Full pinyin
Partial pinyin
No pinyin
Full pinyin
Partial pinyin
No pinyin
3.50 1.10
3.09 1.02
3.47 1.78
4.52 1.89
5.47 1.23
4.00 1.67
C.H. Lee, S. Kalyuga / Computers in Human Behavior 27 (2011) 11–15
for which no differences were found between the three experimental conditions. This result is in correspondence with some previous studies (Shu et al., 1993; Shu & Liu, 1994) that have suggested that even though pinyin enhanced independent reading and character recognition for young Chinese children (native speakers), it may not be beneficial for children with dialect background when dealing with new vocabulary items that are unfamiliar to learners in their spoken language (Xu et al., 1997). This is the category of participants used in this study. The benefits of pinyin were demonstrated only in learning pronunciation. The conventional believe that pinyin should necessarily benefit learning the Chinese language may not be always correct. The reported study indicates that the effectiveness of pinyin as a means of instructional support may depend on learner prior language proficiency and experience with pinyin, as well as on the level of the provided support (full or partial support). The effect may also apply only to certain aspect of language learning (learning pronunciation of characters). The results highlight the importance of assessing actual levels of learner prior knowledge and tailoring accordingly the levels of pinyin support and presentation formats. The effectiveness of such adaptive computer-based learning environments (including instructional procedures based on learner control over required levels of pinyin support) needs to be investigated in future studies. Acknowledgments The authors would like to thank Yin Ping Chiu, Yee Mei Wan, and the Principals of the participating schools for their support in this study. The authors also thank the technicians and the students for their cooperation.
15
References Chung, K. K. H. (2003). Effects of pinyin and first language words in learning of Chinese characters as a second language. Journal of Behavioral Education, 12, 207–223. Faurot, J. L. (1995). Gateway to the Chinese Classics: A practical guide to literary Chinese. San Francisco, CA: China Books and Periodicals. Kalyuga, S. (2000). When using sound with a text or picture is not beneficial for learning. Australian Journal of Educational Technology, 16, 161–172. Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 16, 161–172. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). Expertise reversal effect. Educational Psychologist, 38, 23–31. Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371. Kalyuga, S., Chandler, P., & Sweller, J. (2004). When redundant on-screen text in multimedia technical instruction can interfere with learning. Human Factors, 46, 567–581. Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93, 187–198. Mayer, R. E., & Johnson, C. I. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380–386. McGinnis, S. (1997). Tonal spelling versus diacritics for teaching pronunciation of Mandarin Chinese. The Modern Language Journal, 81, 228–236. Shu, H., & Liu, B. X. (1994). Pinyin zai xiaoxue dinianji ertong zaoqi yuedu zhong zuoyong de yanjiu (The role of pinyin in early reading for Chinese junior primary school children). Psychological Development and Education, 3, 11–15. Shu, H., Zeng, H. M., & Chen, Z. (1993). Xiaoxue dinianji ertong liyong pinyin xuexi shengzici de shiyan yanjiu (Research on using pinyin in learning vocabulary by junior primary school children). Psychological Development and Education, 1, 18–22. Xu, F., Bao, X. H., & Xu, M. (1997). Pinyin zai fangyan ertong zici xuexi zhong de zuoyong (The role of pinyin in learning vocabulary for Chinese children with dialect background). Chinese Journal of Applied Psychology, 3, 39–44. Zhang, Q., & Zhu, J. L. (Eds.). (2007). Yu Wen (Language). Year 1–Year 6. Nanjing, China: Jiangsu Education Publishing House.