- Email: [email protected]
An analysis of educational informatization level of students, teachers, and parents: In Korea
Computers & Education 56 (2011) 760–768
Contents lists available at ScienceDirect
Computers & Education journal homepage: www.elsevier.com/locate/compedu
An analysis of educational informatization level of students, teachers, and parents: In Korea JaMee Kim a, WonGyu Lee b, * a b
Department of Computer Science Education, Graduate School, Korea University, Anam-dong Seongbuk–gu, Seoul 136-701, Republic of Korea Department of Computer Education, College of Education, Korea University, Anam-dong Seongbuk–gu, Seoul 136-701, Republic of Korea
a r t i c l e i n f o
a b s t r a c t
Article history: Received 12 June 2010 Received in revised form 17 October 2010 Accepted 21 October 2010
Korea is recognized as one of the most advanced countries in terms of informatization. The development of informatization has impacted education, and education informatization has contributed to the improvement of teaching in the classroom. Accordingly, education informatiozation is one of the paramount pedagogical issues in South Korea. This study evaluated the effect of Korea’s education informatization policy qualitatively. The access to education information, competency, utilization and the satisfaction levels of students, teachers and parents, who are the groups most directly affected by education policies, were analyzed. In addition, the level of education informatization, the growth of which had been expected by experts, was measured. The rate of achievement as compared to the level expected by experts was analyzed. According to the results of the analysis, the rate of achievement of all three groups was low, and the education informatization level of parents was the lowest. On the basis of the results of this study, we have suggestions about the kind of effort that is needed to raise the level of education informatization. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Improving classroom teaching Pedagogical issue
1. Introduction In an international evaluation of informatization, Korea was ranked No. 1 in DOI (Digital Opportunity Index; DOI) in 2007, and 9th in NRI (Networked Readiness Index; NRI) in 2008. Korea has maintained a high level of informatization (IPC, 2008). Also, in both the WEF(World Economic Forum; WEF) and IMD (International Institute for Management Development; IMD) evaluations of national competitiveness in the IT category, and in the WEF national competitiveness indexes in the technical preparedness category, Korea is maintaining a high level. Korea is rated as excellent in terms of the number of high-speed Internet subscribers (2nd), the number of Internet users (6th), and the level of IT-related laws (7th) (Jamee & Seungjin, 2009). Development of informatization at the national level has led to developments in education as well. In the area of education, the Ministry of Education, Science & Technology (MEST) devised and instituted the ‘education informatization promotion action plan’ in 1996, and tried to raise the informatization level of the general public in line with the government’s informatization policy. Education informatization has impacted improving classroom teaching. So education informatiozation is one of the pedagogical issues in Korea. As for the education informatization policy of Korea, which has been enforced since 1996, KRW(South Korean Won; KRW) 1.4 trillion was invested in ‘Phase 1 (1999–2001)’ and KRW 3.5 trillion was invested in ‘Phase 2 (2002–2005)’ (Hyesook, 2006; Jungwha, 1996). Due to these investments, the education informatization of Korea reached such a high level that it was recognized the world over in 2005, when Phase 2 of the education informatization policy was completed (Jamee & Seungjin, 2009; Yeonseob, Bongwoon, & Youngrok, 2006). In ‘Phase 3 (2006–2010),’ in order to utilize the existing infrastructure, e-learning and u-learning systems will be established, and the performance and quality management systems for education informatization will be developed and improved (MEST, 2009). “Informatization” is the utilization of information and communication technology to facilitate and promote efficiency in the activities of various fields in society. “Education informaztization,” as used in this study, means the utilization of data processed through electronic methods by using information devices for education purposes in the field of education. In other words, education informatization is the utilization of information technology to facilitate the diversification and improvement of education contents, methods, and forms (MEST & KERIS, 2009; MOPAS, 2009).
* Corresponding author. Tel.: þ82 2 3290 2391; fax: þ82 2 3290 2390. E-mail address: [email protected] (WonGyu Lee). 0360-1315/$ – see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2010.10.018
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
761
In Korea, the evaluation of the performance of the education informatization project began after the conclusion of phase-1 of the project. korea Education Research & Information Service (KERIS), in charge of education informatization, has been measuring the education informatization level of schools in 16 cities and provinces since 2001 in order to ascertain the current status, and it contributes to the decision-making that is necessary for policy-making (Youngae, & Hyesook, 2008). Even though many portions of the nationwide infrastructure related to informatization has already been built, education informatization will be meaningful only when individuals are competent enough to utilize the available information technologies or tools and accept them as systematized knowledge (Seungjin, & Jamee et al, 2007). In other words, only when education informatization is actively utilized can the goal of education informatization be said to have been accomplished in actuality (Jamee, & Seungjin, 2009). The aim of this study was to assess the current education informatization levels of students, teachers and parents in Korea. This study also sought to determine, based on the current levels, what needs to be improved in terms of education informatization. The research questions were 1. What are the expected standards being considered by experts regarding education informatization? 2. What are the current education informatization achievement rates of students, teachers and parents? 3. What items need to be improved in order to advance education informatization for each group? That is, first, “expected standards” means the levels at which the experts consider the education informatization to be fully achieved. Second, the education informatization levels of students, teachers, and parents are measured, and then each group’s achievement rate relative to the expected standards of the experts is assessed. From the achievement rates compared with the actual indices, the education informatization levels of the three groups can be more objectively determined. Third, items requiring improvement are more objectively assessed through portfolio matrix analysis.
2. Theoretical background As for Korea’s education informatization policy, substantial effort has been made to develop detailed indexes for individual areas of the informatization policy in order to evaluate the performance of the education informatization policy through these indexes. This is because education informatization has had significant impact on improving classroom teaching. There have been numerous studies involving elementary schools, middle schools and special education schools as follows. First, comparative studies were conducted at the national level. UNESCO and OECD tried to develop an international comparative index not only for various projects aimed at reducing the international and inter-regional digital divide by means of education informatization and ICT, but also for making education policies. In addition, from 2002 to 2004, OECD conducted studies in 18 countries, including Australia, Canada, the Balkans, India, Indonesia, Japan, the Republic of Korea, Malaysia, New Zealand, the Philippines, Singapore, Slovenia, South Africa, Thailand, Great Britain, USA, Uzbekistan, and Vietnam, to analyze the kinds of indexes related to the utilization of ICT (Junghee et al., 2006). Studies conducted at the national level looked at the ability to use information devices, improvement of Internet accessibility, and selection and utilization of educational materials. With regard to these topics, international organizations like the EU, OECD, and UNESCO (Pejova, 2002) are discussing how to measure ICT competency (ANZIL, 2004; CILIP, 2004; EnIL, 2005). ICT competency contributes to productive activities such as the development of educational courses and the creation of new knowledge for teaching in class. Second, the OECD has been conducting studies every three years since 2000 in 30 OECD-member countries and 11 non-OECD member countries on ICT utilization in PISA (Programme for International Student Assessment: PISA) (OECD, 2005a, 2007). PISA provides the basic data necessary for making educational policies in the countries studied by measuring the competency levels in reading, mathematics and science of 15-year-old students, and by analyzing their links to the background variables that are affecting the competency levels. Third, studies were conducted to analyze the current status of education informatization in different regions or in elementary and middle schools, and to compare the informatization levels among school levels or among schools. When it comes to analyzing the current status of national informatization in different countries or the current status of ICT utilization in each country, there is a report on the performance index development project relating to the educational utilization of ICT which was carried out by the Asia-Pacific Educational Executive Office of UNESCO from 2002 to 2004 (Junghee et al., 2006). Fourth, the following studies on index development and measurement of the education informatization level were conducted in Korea, in order to evaluate education informatization projects. They are “An analysis of level measure of educational informatization in 2006” (Heungju & Hyesook, 2006), “A study on the current status of educational informatization in 2005” (Youngae & Hyesook, 2005), “A study on developing indicators for information and communication technology use in primary and secondary education” (Inwoo et al., 2001), “A study on developing indicators for information and communication technology use in higher education” (Kapsu et al., 2002), and “A study on developing indicators for information and communication technology use in lifelong education” (Heesoo et al., 2003). Other studies include “The research on ICT indicators for special education” (Wonhee, Hyunki, Aehwa, Sungnam, & Jiho, 2004) and “Development of self-diagnostic indicators of ICT in primary and secondary schools” (Sungeun, Cheolil, Hyesook, Seonyong, & Sunghee, 2007). As well, there was an exceptional study on the development of a standard for analysis of the effectiveness of e-learning(Junghee et al., 2006). It focused on primary and secondary education, which is not education informatization, and was confined to e-learning. In the aforementioned studies, the sum of all the indexes did not total 100%, but they interpreted the meaning of each and every index. In other words, they did not analyze which index was superior within a group of similar indexes. They evaluated the infrastructure at the national level or the school level rather than the effect of policies from the perspective of education consumers, and they did not measure the IT levels of individuals. The objectives and targets of the existing studies related to the education informatization index can be summarized, and are shown in Table 1.
762
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
Table 1 A objective, unit, and subject matter of each index. Research This study OECD PISA 2001, 2003 (OECD, 2005a, 2005b, 2007) A comparison of the education informatization levels (KERIS) The self-diagnosis indexes A index of e-learning effectiveness
Objective
Unit
comparison of the education informatization levels of students, teachers, and parents supplementary indexes of the grades of 15 year-old students in OECD countries and non-OECD countries A comparison of the education informatization levels of elementary and secondary education in 16 cities and provinces of Korea the self-diagnosis indexes for analysis of the education informatization of each schools A comparison of the building of the international infrastructure for e-learning and its effects
Person
students, teachers, and parents
Subject matter
Nation
15 year-old students
School
elementary and secondary school in 16 cities and provinces
School
students, teachers, and infrastructure
Nation
students and teachers
3. Study method The contents of indexes and development procedures, how the experts came up with the standards, and weights, the sampling procedure, and the methods and characteristics of the subjects are described here. 3.1. Index development procedure and description The tools developed to make the indexes for diagnosing the education informatization levels of students, teachers, and parents were modified based on a preliminary investigation. The primary tools were modified three times between September 10, 2007 and September 27, 2007. A total of 15 education informatization experts reviewed the tools twice each before the secondary tools were finalized on October 10. The secondary tools improved the statement method according to the roles of the subjects (i.e., students, teachers, or parents), and after two expert meetings, the tertiary tools were finalized on November 2. After the preliminary test and reliability analysis of the tertiary tools were conducted on November 5 and 6, respectively, the final tools were completed. The final tools were applied from November 8 onward, in order to make the indexes. The indexes of this study were designed to contain all the information needed to be able to measure individual levels on the basis of a comparison with the key indexes previously developed in relation to informatization. The education informatization index of this study, developed on the basis of the developmental procedure discussed above, refers to the overall level of education informatization, which puts together the level of individuals’ accessibility to education informatization, the level of usability, and the utilization level. The education informatization index is divided into the accessibility index, the competency index, the utilization index, and the satisfaction index, while the utilization index is classified into quantitative utilization and qualitative utilization. As for each area, the detailed component areas, the concrete measurement elements of the detailed areas and indexes were generated and defined. First, the accessibility index is the basic device requirement for education informatization. It indicates the level of infrastructure (possession of device or possession of wireless internet connection enabled device, etc.) established for an individual to utilize information and communication technology. Second, the competency index is the level of ability necessary to use education information. It indicates an individual’s level of ability to use devices related to education informatization (including the ability to use computers and other information and communication devices as well as the Internet, and the ability to deal with the harmful effects of Internet use). Third, the utilization index refers to the extent to which education information is used. The utilization index is composed of quantitative utilization and qualitative utilization. Quantitative utilization indicates the amount of education information usage (quantitative level), while qualitative utilization indicates the level of direct support in teaching and learning, and includes the outcome gained through quantitative utilization. Fourth, the satisfaction index indicates the levels of direct or indirect forms of positive changes (from teaching and learning, communication and policy aspects) perceived by the users in the field through the utilization of education information. 3.2. Method of standards and weights designation On the basis of the index for diagnosing the level of education informatization, an attempt was made to measure the levels that the education informatization of students, teachers, and parents must reach. In other words, on the basis of the standard for the level of education informatization suggested by the group of experts, this study tried to establish the levels of education informatization suitable for each group. This study used the Angoff and Jaeger method to set up the standards (Angoff, 1971; Jaeger, 1978). To set up the standards, the experts in the group related to education informatization were classified into the elementary school, middle school, high school, and general levels. The opinions of 10 experts at each level were collected during the time period from November 13, 2007 to November 20, 2007. A total of 37 experts (8 experts from elementary schools, 10 experts from middle schools, 9 experts from high schools, and 10 experts from the general group) participated in the opinion collection process. The opinions of the experts were finally analyzed on November 28, and the final standards and weights were determined after a couple of meetings. 3.3. Sampling procedure The sampling of study subjects for education informatization level assessment was done by using the following 3-stage cluster sampling method:
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
763
In the first stage, study regions were chosen in 16 cities and provinces. Based on the populations, the regions were classified into major cities, medium and small cities, and rural townships. In the second stage, based on consideration of the education levels and living environment conditions, the regions were classified into three levels (high, medium, and low). That is, they were categorized based on socioeconomic classes. In the third stage, the subject schools were selected based on consideration of sizes of the student bodies of the primary and middle schools in each region. The assigned number of students, teachers, and parents from the selected schools were surveyed, and the replies were returned via registered mail. The survey was not done via the web to include subjects with low accessibility. In particular, as the accessibility area included considerations such as internet accessibility and the availability of computers, the self-administration method was used to get more objective responses. 3.4. Sampling method The more detailed procedure and method of sampling subjects for diagnosing the education informatization level are as follows: According to the aforementioned procedure, metropolitan cities, cities and provinces were differentiated in accordance with the following standards. First, there are the classifications of major cities, medium and small cities, and rural townships. All seven special and metropolitan cities are classified as major cities. The smaller cities, including provincial capital cities, in the 9 provincial regions were classified as medium and small cities, and the rural district regions were classified as rural townships. Second, within the selected regions, further differentiations - high, medium, and low levels - were made, based on education levels and living environments. This study came up with 144 groups, and then after considering the group sizes, chose 1 to 2 schools in each group. Accordingly, a total of 177 schools were designated for this study. Third, the election criteria for schools were as follows. Students, teachers, and parents from the selected schools were chosen as respondents of the survey. However, 10 parents were selected to ensure at least 5 responses. Students and teachers were surveyed in schools and their responses were collected immediately. As for parents, the questionnaires were given to students to give to their parents, and the questionnaires were collected afterwards. Fourth, subjects were selected as follows. Fourth graders and higher-grade elementary students were selected as subjects, and the subjects were distributed evenly among different grades. The reason why elementary school students older than 4th graders were selected was that the subjects had to perfectly understand the questionnaire. From the finally selected schools, four 4th graders, four 5th graders, three 6th graders, four 7th graders, four 8th graders, three 9th graders, four 10th graders, four 11th graders, and three 12th graders were selected. As far as teachers are concerned, elementary school teachers were selected by grade, and middle and high school teachers were selected by the subjects they teach. When it comes to parents, one parent of each student participating in the survey was selected. 3.5. Characteristics of subjects The survey for the diagnosis of the education informatization level was conducted between November 23, 2007 and December 14, 2007. 1668 students from 166 schools across the country between the 4th grade and 12th grade, 1017 teachers, and 1354 parents participated in the survey. At the 95% confidence level, the sampling error of this survey was 2.4% points for students, 3.1% points for teachers, and 2.7% points for parents. The sample size of the subjects of this study is shown in Table 2. 4. Findings In this section, the experts’ standard for the education informatization level, the education informatization levels of students, teachers and parents, and the improvements based on the portfolio matrix are described. 4.1. Experts’ standard for the education informatization level This study measured the education informatization levels considered by experts to be the expected standards. The expected standards were measured by using the Angoff method and the Jaeger method. That is, the standards were established in such a way that even experts using tools developed to diagnose education informatization levels would agree that education informatization has been sufficiently achieved. Also, the averages of the scores calculated from the two methods were established as the bases of the education informatization levels. The standard scores for the education informatization levels established by experts for students, teacher, and parents are shown in Table 3. As for the standard scores for the overall education informatization indexes established by experts, for individual groups, the standard for teachers was the highest at 80.90 points, 80.64 for students, and 76.52 for parents. As far as the standard scores for individual areas are
Table 2 Number of samples participating in the education informatization level assessment.
elementary school middle school high school total
schools
students
teachers
parents
52 59 55 166
520 585 563 1668
324 360 333 1017
434 469 451 1354
764
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768 Table 3 The standard scores for the education informatization levels established by experts for students, teacher and parents. pollee
education informatization indexes
student teacher parent
80.64 80.90 76.52 accessibility index
student teacher parent
ownership of devices
availability of IT
recency of the operation system
70.95 83.11 76.97
98.65 99.32 99.32
100 100 97.30
88.04 93.72 89.48 competency index
student teacher parent
accessibility element
78.03 81.28 70.88 utilization index
student teacher parent
competency element hardware literacy
software literacy
internet literacy
selfdevelopment
71.96 78.38 65.54
75.03 80.35 70.49
81.39 83.93 74.53
86.49 83.78 –
utilization element quantitative utilization
qualitative utilization
73.74 62.97 63.19
82.07 80.17 77.38
78.79 74.60 71.23
satisfaction index
78.60 79.56 74.96
concerned, accessibility had the highest score (88.04 points), followed by utilization (78.79 points), satisfaction (78.60 points), and competency (78.03 points). Teachers had the highest standard score in accessibility, and their standard score for utilization was the lowest (74.60 points). For the parents, the standard score of accessibility (89.48 points) was the highest and the standard score of competency (70.88 points) was the lowest.
4.2. The education informatization levels of students, teachers, and parents and the rate of achievement The rate of achievement as compared to the overall education informatization indexes and the indexes by area, and the expected values of students, teachers, and parents, are shown in Table 4. The rates of achievement are the ratios of the current levels to the standard scores established by experts, as shown in Table 3. Accordingly, if “0 < rate of achievement < 100,” it means that the current level failed to reach the expectation of experts, and if “rate of achievement S 100,” it means that the level exceeded the expected value. Table 4 The indexes are the current education informatization levels of individual groups and, the rate of achievement. pollee
education informatization indexes(rate of achievement)
student teacher parent
85.89(97.55) 89.80(95.82) 85.33(95.36) accessibility index (rate of achievement)
student teacher parent
85.89(97.55) 89.80(95.82) 85.33(95.36) competency index(rate of achievement)
student teacher parent
56.13(71.94) 62.37(76.74) 43.30(61.08) utilization index (rate of achievement)
student teacher parent
64.56(81.94) 62.42(83.67) 46.66(65.50)
accessibility element(rate of achievement) ownership of devices
availability of IT
recency of the operation system
77.12(108.71) 77.04(92.70) 77.03(100.08)
91.56(92.82) 97.00(97.66) 93.02(93.65)
92.54(92.54) 96.88(96.88) 90.11(92.62)
competency element(rate of achievement) hardware literacy
software literacy
internet literacy
selfdevelopment
60.53(84.12) 53.36(68.08) 38.26(58.38)
60.11(80.11) 63.70(79.28) 41.22(58.48)
70.12(86.16) 70.47(83.96) 49.43(66.32)
16.79 (19.41) 61.21(73.06) –
utilization element (rate of achievement) quantitative utilization
qualitative utilization
66.93(90.77) 66.72(105.9) 49.73(78.69)
63.29(77.11) 60.43(75.38) 44.98(58.13)
satisfaction index (rate of achievement) 61.98(78.85) 62.09(78.04) 50.96(67.98)
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
765
The overall education informatization indexes were measured, and the results showed that teachers’ index was the highest at 66.96 points, followed by that of students (66.46 points), and that of parents (56.54 points). In other words, the overall education informatization index of teachers was higher than that of students by about 0.5 points. As for the rate of achievement as compared to the expected value, the rate of achievement of teachers was 82.77%, so it was higher than that of students (82.40%) as well as that of parents was 73.89%, who scored the lowest among the three groups. For starters, students’ accessibility index was very high, at 85.89 points out of a possible 100 points. Students’ utilization index was 64.56 points, their satisfaction index was 61.98 points, and their competency index was 56.13 points. Their competency index was the lowest. As for the rate of achievement in each area as compared to the expected value, the rate of achievement of the accessibility area with the highest index was the highest (97.55%), and that of the utilization area was 81.94%, followed by 78.85% in the satisfaction area and 71.94% in the competency area. As in the case of students, the accessibility index of teachers was the highest (89.80 points out of a possible 100 points). The highest rate of achievement as compared to the experts’ standard was 95.82%. That of the index of the competency area was 62.37 points, and that rate of achievement was the lowest (76.74%). Parents’ index for the accessibility area was 85.33 points, and the rate of achievement was quite high, at 95.36%. However, the rate of achievement in the satisfaction, utilization and competency areas was lower than 70%. In particular, the education informatization level of parents was relatively lower than that of students or teachers.
4.3. Portfolio matrix analysis of the educational informatization indexes To deduce the items to be considered for improving the overall education informatization indices, a two-dimensional diagram was created using the importance rate and the achievement rate (%) of each element relative to the expected standards. From the portfolio matrix analysis, elements that need to be improved first, based on the current state, were extracted. The matrix is expressed as four quadrants, and each quadrant’s meaning is expressed below: An element in quadrant 1 is an element with both high importance and a high achievement rate. Therefore, an element with high current levels, the maintenance of which is very desirable, falls under the “maintenance and sustainability” category. An element in quadrant 2 is an element with high importance but a low achievement rate (below average). That is, an element most urgently requiring improvement falls under the “intensive improvement” category. An element in quadrant 3 is an element with both low importance and a low achievement rate. Therefore, consideration must be given as to whether such an element is actually needed for measuring the education informatization level, and this element falls under the “passive improvement” category. An element in quadrant 4 is an element with low importance but a high achievement rate. An element that needs to be managed at the current level falls under the “passive management” category. The results of the portfolio matrix analysis for improvement of the education informatization indexes of students, who are the subjects of this study, are shown in Fig. 1. The element found to have high importance and a high achievement rate, based on the results of the portfolio matrix analysis on education informatization of students and falling under the “maintenance and sustainability” category, is the element of computer/Internet
Fig. 1. The results of the portfolio matrix analysis for improvement of the education informatization indexes of students.
766
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
Fig. 2. The results of the portfolio matrix analysis for improvement of the education informatization indexes of teachers.
quantitative utilization. That is, students’ quantitative utilization of computers and the Internet is determined to be sufficient. In contrast, falling under the “intensive improvement” category of high importance but a low achievement rate, are the elements of qualitative utilization, software utilization, and satisfaction. For these three elements, more so than with other elements, ways must be found to increase their achievement rates. From the teachers’ portfolio matrix results, as shown in Fig. 2, the computer/Internet quantitative utilization element was found to fall under the “maintenance and sustainability” category, with high importance and a high achievement rate. Falling under the “intensive improvement” category of high importance but a low achievement rate are the elements of qualitative utilization, software utilization, and satisfaction. Qualitative utilization, in particular, is an element that shows very high importance but a low achievement rate. For the “maintenance and sustainability” and “intensive improvement” category elements, the analysis results of the teachers’ portfolio matrix coincided with those of the students’ results.
Fig. 3. The results of the portfolio matrix analysis for improvement of the education informatization indexes of parents.
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
767
The portfolio matrix analysis results of the importance rate and the achievement rate (%) of the overall education informatization indices for parents are given in Fig. 3. For parents, computer/Internet quantitative utilization and device ownership elements are in the “maintenance and sustainability” category of high importance and a high achievement rate. Placing in the “intensive improvement” category of high importance but a low achievement rate are the elements of qualitative utilization, software utilization, and satisfaction. 5. Discussions Here is a discussion of the results of the analysis of the education informatization levels of students, teachers, and parents. First, in terms of the students’ education informatization level, they scored the highest in the area of accessibility. This result shows that Korea’s IT infrastructure is at the highest level in the world (OECD, 2005b, 2007). However, the students’ competency and utilization levels were low. In particular, the qualitative utilization and software usage were placed as elements requiring “intensive improvement.” When students used their own competency to learn, the impact was shown to be the highest (Ravitz, Mergendoller, & Rush, 2002). In other words, even when the accessibility level is high, if the competency level is low, the utilization of education information is difficult. Therefore, more must be done to increase competency in order for the students’ education information utilization result to show an increase in learning. More effort is needed because the learning effect is high when the students themselves become the agents of learning (Harrison et al., 2003; Papanastasiou, Zembylas, & Charalambos, 2003). Second, in terms of the teachers’ education informatization level, the achievement rate in the accessibility area was high, while the competency area showed a low achievement rate of only 76.74%. This result is in agreement with past findings, that many teachers are passive in developing their own competency (Seongmu, Seungjin, Taesook, Jaesung, & Jamee, 2008). In the utilization area, while the quantitative utilization was high, the qualitative utilization was low. Because they want to receive a diverse array of information concerning their preference, many teachers utilize websites. For this reason, the quantitative utilization level is shown to be high. However, since teachers are not active in utilization that involves producing their own materials or sharing materials (Jaesin, Hyoroon, Taesook, Jaesung, & Jamee, 2009), the qualitative utilization is shown to be low. In other words, this result shows that the teachers have high quantitative utilization, and also that such a high quantitative utilization does not lead to increased qualitative utilization related to teaching and learning. Third, the parents’ education informatization level shows a low rate of achievement compared with those of the students and teachers. The achievement rates of qualitative utilization and software usage elements are lower than 60%. The low utilization rate of the parents may negatively impact the guidance of their children. There is also a high possibility that it may negatively impact the children’s utilization of education information and the Internet (Jamee, Hyeoncheol, Hongrae, & Junghoon, 2009). The reason for this is that parental interest is an important element of children’s study habits (Jaesin et al., 2009; Barker & Wendel, 2001). Since parents play an important role in increasing their children’s utilization of education information, it appears that there is a need to provide education to the parents, in order to improve in the competency or utilization area. 6. Conclusion This study sought to assess the education informatization level of Korea, which possesses a world-class IT infrastructure. In particular, the study attempted to determine the state of education informatization at the personal levels of students, teachers, and parents, and to find out which areas need improvement. From the diagnosis results regarding the current state of education informatization, all three groups showed an achievement rate of over 95% in the “accessibility” area. However, in the “competency,” “utilization”, and “satisfaction” areas, the teacher group showed achievement rates of less than 80%. For the student group, the competency rate was about 71%, showing a lower achievement rate compared with other areas. The parents showed low achievement rates overall, with less than 70% in all three areas. Based on the results, this study proposes the following: The students need to improve qualitative utilization and satisfaction. The teachers need to increase competency in their use of educational programs and application software. The teachers also need to increase the qualitative utilization related to production or sharing of education information, in order to contribute to improvement in teaching and learning. For the parents, since the levels in all areas, except for accessibility, are low, priority should be given to strengthening competency. This should be done because only when competency reaches a higher level can the qualitative utilization for teaching and learning increase. For balanced development of education informatization, this study diagnosed the current level and proposed elements for improvement. The study also discovered that infrastructure expansion alone does not raise the overall education informatization level. In future studies, how the level of each area of education informatization differs depending on student grade or region should be examined through comparisons, and what areas need support should also be investigated. References Angoff, W. H. (1971). Scales, norms, and equivalent score. In R. L. Thorndike (Ed.), Educational measurement (2nd ed). Washington. DC: American Council on Education. Australian and New Zealand Institute for Information Literacy (ANZIL). (2004). Australian and New Zealand information literacy Framework: principles, standards and practice. Accessed August 12, 2005 available at. In Alan Bundy (Ed.), Adelaide, Australian and New Zealand Institute for information Literacy, 2004 (2nd ed) http://www.caul. edu.au/info-literacy/InfoLiteracyFramework.pdf. Barker, K., & Wendel, T. (2001). e-learning Studying Canada’s Virtual Secondary Schools. In SAEE research Series, Vol. 8. Society For The Advancement Of Excellence In Education. Chartered Institute of Library and Information Professionals (CILIP). (2004). A short introduction to information literacy. Retrieved August 12, 2005 from. http://www.cilip. org.uk/professionalguidance/informationliteracy. European Network on Information Literacy (EnIL) (European Union Retrieved August 12, 2005) from http://www.ceris.to.cnr.it/Basili/EnIL/index.html Harrison, C., Comber, C., Fisher, T., Haw, K., Lewin, C., Lunzer, E., et al. (2003). ImpaCT2: The impact of information and communication technologies on pupil learning and attainment. London: DfES. Heesoo, L., Taejung, K., Wongki, S., Sungbae, O., Soonok, J., & Jaesin, S. (2003). A study on developing indicators for information and communication technology use in lifelong education. Korea Education Research & Information Service. Heungju, K., & Hyesook, K. (2006). An analysis of level measure of educational informatization, in 2006. Korea Education Research & Information Service.
768
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
Hyesook, K. (2006). A study of the current status educational informatization. Korea Education Research & Information Service. Informatization Promotion Committee (IPC). (2008). The basic plan of national informatization. IPC. Inwoo, P., Myeongsoon, P., Dosoon, P., Myeonghye, H., Bumseok, K., Heungju, K., & Soojung, J. (2001). A study on developing indicators for information and communication technology use in primary and secondary education. Korea Education Research & Information Service. Jaeger, R. M. (1978). A proposal for setting a standard on the North Carolina High School Competency Test. Paper presented at the spring meeting of the North Carolina Association for Research in Education Chapel Hill. Jaesin, S., Hyoroon, A., Taesook, A., Jaesung, H., & Jamee, K. (2009). An analysis of the current status and effect about EDUNET. Korea Education Research & Information Service. Jamee, K., Hyeoncheol, K., Hongrae, K., & Junghoon, K. (2009). Development of indicator and criteria for level measure of educational informatization of students, teachers, and parents. Journal of the Korean Association of Information Education, 13(2), 145–157. Jamee, K., & Seungjin, L. (2009). An analysis of content and policy about educational informatization. Korea Education Research & Information Service. Junghee, S., Jusung, J., Bokeong, K., Hyeonjin, K., Sangha, L., Jungweon, S., et al. (2006). Development of standard for analysis of effectiveness of E-learning: Focused on primary and secondary education. Korea Education Research & Information Service. Jungwha, S. (1996). An evaluation of educational policy in Korea, A research of education. Education Research in Hongik university. Kapsu, K., Okhwa, L., Seyeoung, C., Sukhee, W., Minjoo, R., & Beomseog, K. (2002). A study on developing indicators for information and communication technology use in higher education. Korea Education Research & Information Service. MEST & KERIS. (2009). 2008 Adapting education to the information age. MEST & KERIS. Ministry of Education, Science and Technology (MEST). (2009). Informatization promotion action plan (draft) of the Ministry of Education, Science and Technology 2009. MEST. Ministry of Public Administration and Security (MOPAS). (2009). The plan of national informatization. MOPAS. OECD. (2005a). Are students ready for a technology-rich world? What PISA studies tell us. OECD. OECD. (2005b). PISA 2003 technical report. Paris: OECD. OECD. (2007). PISA 2006: Science competencies for tomorrow’s world executive summary. OECD. Papanastasiou, E., Zembylas, M., & Charalambos, V. (2003). Can computer use hurt science achievement? The USA results from PISA. Journal of Science and Technology, 12(3). Plenum Publishing Corporation: New York. Pejova, Z. (2002), Information literacy: An issue which requests urgent action in developing countries and countries in transition. White Paper prepared for UNESCO, the U.S. National Commission on Libraries and Information Science, and the National Forum on Information Literacy for use at the Information Literacy Meeting of Experts, Prague, The Czech Republic, 2003. Viewed July 15, 2006 http://www.nclis.gov/libinter/infolitconf&meet/papers/pejova-fullpaper.pdf. Ravitz, J., Mergendoller, J., & Rush, W. (2002, April). Cautionary tales about correlations between student computer use and academic achievement. Paper presented at annual meeting of the American Educational Research Association. New Orleans, LA. Seongmu, J., Seungjin, L., Taesook, A., Jaesung, H., & Jamee, K. (2008). An analysis of the current status and effect about EDUNET. Korea Education Research & Information Service. Seungjin, L., Jamee, K., Hyeoncheol, K., Minha, K., Aenam, K., Injin, J., et al. (2007). Development of index for gap and level of ICT in education to resolve the polarization of ICT in education in elementary and secondary schools. Korea Education Research & Information Service. (KERIS). Sungeun, B., Cheolil, L., Hyesook, K., Seonyong, K., & Sunghee, J. (2007). Development of self-diagnostic indicators of ICT in primary and secondary schools. Korea Education Research & Information Service. Wonhee, P., Hyunki, S., Aehwa, K., Sungnam, K., & Jiho, D. (2004). The research on ICT indicators for special education. Korea Education Research & Information Service. Yeonseob, H., Bongwoon, H., & Youngrok, K. (2006). A study on analysis of ICT in education budget and ways to reform in Korea. Korea Education Research & Information Service. Youngae, K., & Hyesook, K. (2005). A study on the current status of educational informatization, in 2005. Korea Education Research & Information Service. Youngae, K., & Hyesook, K. (2008). A study about level measure of educational informatization, in 2008. Korea Education Research & Information Service.
Contents lists available at ScienceDirect
Computers & Education journal homepage: www.elsevier.com/locate/compedu
An analysis of educational informatization level of students, teachers, and parents: In Korea JaMee Kim a, WonGyu Lee b, * a b
Department of Computer Science Education, Graduate School, Korea University, Anam-dong Seongbuk–gu, Seoul 136-701, Republic of Korea Department of Computer Education, College of Education, Korea University, Anam-dong Seongbuk–gu, Seoul 136-701, Republic of Korea
a r t i c l e i n f o
a b s t r a c t
Article history: Received 12 June 2010 Received in revised form 17 October 2010 Accepted 21 October 2010
Korea is recognized as one of the most advanced countries in terms of informatization. The development of informatization has impacted education, and education informatization has contributed to the improvement of teaching in the classroom. Accordingly, education informatiozation is one of the paramount pedagogical issues in South Korea. This study evaluated the effect of Korea’s education informatization policy qualitatively. The access to education information, competency, utilization and the satisfaction levels of students, teachers and parents, who are the groups most directly affected by education policies, were analyzed. In addition, the level of education informatization, the growth of which had been expected by experts, was measured. The rate of achievement as compared to the level expected by experts was analyzed. According to the results of the analysis, the rate of achievement of all three groups was low, and the education informatization level of parents was the lowest. On the basis of the results of this study, we have suggestions about the kind of effort that is needed to raise the level of education informatization. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Improving classroom teaching Pedagogical issue
1. Introduction In an international evaluation of informatization, Korea was ranked No. 1 in DOI (Digital Opportunity Index; DOI) in 2007, and 9th in NRI (Networked Readiness Index; NRI) in 2008. Korea has maintained a high level of informatization (IPC, 2008). Also, in both the WEF(World Economic Forum; WEF) and IMD (International Institute for Management Development; IMD) evaluations of national competitiveness in the IT category, and in the WEF national competitiveness indexes in the technical preparedness category, Korea is maintaining a high level. Korea is rated as excellent in terms of the number of high-speed Internet subscribers (2nd), the number of Internet users (6th), and the level of IT-related laws (7th) (Jamee & Seungjin, 2009). Development of informatization at the national level has led to developments in education as well. In the area of education, the Ministry of Education, Science & Technology (MEST) devised and instituted the ‘education informatization promotion action plan’ in 1996, and tried to raise the informatization level of the general public in line with the government’s informatization policy. Education informatization has impacted improving classroom teaching. So education informatiozation is one of the pedagogical issues in Korea. As for the education informatization policy of Korea, which has been enforced since 1996, KRW(South Korean Won; KRW) 1.4 trillion was invested in ‘Phase 1 (1999–2001)’ and KRW 3.5 trillion was invested in ‘Phase 2 (2002–2005)’ (Hyesook, 2006; Jungwha, 1996). Due to these investments, the education informatization of Korea reached such a high level that it was recognized the world over in 2005, when Phase 2 of the education informatization policy was completed (Jamee & Seungjin, 2009; Yeonseob, Bongwoon, & Youngrok, 2006). In ‘Phase 3 (2006–2010),’ in order to utilize the existing infrastructure, e-learning and u-learning systems will be established, and the performance and quality management systems for education informatization will be developed and improved (MEST, 2009). “Informatization” is the utilization of information and communication technology to facilitate and promote efficiency in the activities of various fields in society. “Education informaztization,” as used in this study, means the utilization of data processed through electronic methods by using information devices for education purposes in the field of education. In other words, education informatization is the utilization of information technology to facilitate the diversification and improvement of education contents, methods, and forms (MEST & KERIS, 2009; MOPAS, 2009).
* Corresponding author. Tel.: þ82 2 3290 2391; fax: þ82 2 3290 2390. E-mail address: [email protected] (WonGyu Lee). 0360-1315/$ – see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2010.10.018
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
761
In Korea, the evaluation of the performance of the education informatization project began after the conclusion of phase-1 of the project. korea Education Research & Information Service (KERIS), in charge of education informatization, has been measuring the education informatization level of schools in 16 cities and provinces since 2001 in order to ascertain the current status, and it contributes to the decision-making that is necessary for policy-making (Youngae, & Hyesook, 2008). Even though many portions of the nationwide infrastructure related to informatization has already been built, education informatization will be meaningful only when individuals are competent enough to utilize the available information technologies or tools and accept them as systematized knowledge (Seungjin, & Jamee et al, 2007). In other words, only when education informatization is actively utilized can the goal of education informatization be said to have been accomplished in actuality (Jamee, & Seungjin, 2009). The aim of this study was to assess the current education informatization levels of students, teachers and parents in Korea. This study also sought to determine, based on the current levels, what needs to be improved in terms of education informatization. The research questions were 1. What are the expected standards being considered by experts regarding education informatization? 2. What are the current education informatization achievement rates of students, teachers and parents? 3. What items need to be improved in order to advance education informatization for each group? That is, first, “expected standards” means the levels at which the experts consider the education informatization to be fully achieved. Second, the education informatization levels of students, teachers, and parents are measured, and then each group’s achievement rate relative to the expected standards of the experts is assessed. From the achievement rates compared with the actual indices, the education informatization levels of the three groups can be more objectively determined. Third, items requiring improvement are more objectively assessed through portfolio matrix analysis.
2. Theoretical background As for Korea’s education informatization policy, substantial effort has been made to develop detailed indexes for individual areas of the informatization policy in order to evaluate the performance of the education informatization policy through these indexes. This is because education informatization has had significant impact on improving classroom teaching. There have been numerous studies involving elementary schools, middle schools and special education schools as follows. First, comparative studies were conducted at the national level. UNESCO and OECD tried to develop an international comparative index not only for various projects aimed at reducing the international and inter-regional digital divide by means of education informatization and ICT, but also for making education policies. In addition, from 2002 to 2004, OECD conducted studies in 18 countries, including Australia, Canada, the Balkans, India, Indonesia, Japan, the Republic of Korea, Malaysia, New Zealand, the Philippines, Singapore, Slovenia, South Africa, Thailand, Great Britain, USA, Uzbekistan, and Vietnam, to analyze the kinds of indexes related to the utilization of ICT (Junghee et al., 2006). Studies conducted at the national level looked at the ability to use information devices, improvement of Internet accessibility, and selection and utilization of educational materials. With regard to these topics, international organizations like the EU, OECD, and UNESCO (Pejova, 2002) are discussing how to measure ICT competency (ANZIL, 2004; CILIP, 2004; EnIL, 2005). ICT competency contributes to productive activities such as the development of educational courses and the creation of new knowledge for teaching in class. Second, the OECD has been conducting studies every three years since 2000 in 30 OECD-member countries and 11 non-OECD member countries on ICT utilization in PISA (Programme for International Student Assessment: PISA) (OECD, 2005a, 2007). PISA provides the basic data necessary for making educational policies in the countries studied by measuring the competency levels in reading, mathematics and science of 15-year-old students, and by analyzing their links to the background variables that are affecting the competency levels. Third, studies were conducted to analyze the current status of education informatization in different regions or in elementary and middle schools, and to compare the informatization levels among school levels or among schools. When it comes to analyzing the current status of national informatization in different countries or the current status of ICT utilization in each country, there is a report on the performance index development project relating to the educational utilization of ICT which was carried out by the Asia-Pacific Educational Executive Office of UNESCO from 2002 to 2004 (Junghee et al., 2006). Fourth, the following studies on index development and measurement of the education informatization level were conducted in Korea, in order to evaluate education informatization projects. They are “An analysis of level measure of educational informatization in 2006” (Heungju & Hyesook, 2006), “A study on the current status of educational informatization in 2005” (Youngae & Hyesook, 2005), “A study on developing indicators for information and communication technology use in primary and secondary education” (Inwoo et al., 2001), “A study on developing indicators for information and communication technology use in higher education” (Kapsu et al., 2002), and “A study on developing indicators for information and communication technology use in lifelong education” (Heesoo et al., 2003). Other studies include “The research on ICT indicators for special education” (Wonhee, Hyunki, Aehwa, Sungnam, & Jiho, 2004) and “Development of self-diagnostic indicators of ICT in primary and secondary schools” (Sungeun, Cheolil, Hyesook, Seonyong, & Sunghee, 2007). As well, there was an exceptional study on the development of a standard for analysis of the effectiveness of e-learning(Junghee et al., 2006). It focused on primary and secondary education, which is not education informatization, and was confined to e-learning. In the aforementioned studies, the sum of all the indexes did not total 100%, but they interpreted the meaning of each and every index. In other words, they did not analyze which index was superior within a group of similar indexes. They evaluated the infrastructure at the national level or the school level rather than the effect of policies from the perspective of education consumers, and they did not measure the IT levels of individuals. The objectives and targets of the existing studies related to the education informatization index can be summarized, and are shown in Table 1.
762
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
Table 1 A objective, unit, and subject matter of each index. Research This study OECD PISA 2001, 2003 (OECD, 2005a, 2005b, 2007) A comparison of the education informatization levels (KERIS) The self-diagnosis indexes A index of e-learning effectiveness
Objective
Unit
comparison of the education informatization levels of students, teachers, and parents supplementary indexes of the grades of 15 year-old students in OECD countries and non-OECD countries A comparison of the education informatization levels of elementary and secondary education in 16 cities and provinces of Korea the self-diagnosis indexes for analysis of the education informatization of each schools A comparison of the building of the international infrastructure for e-learning and its effects
Person
students, teachers, and parents
Subject matter
Nation
15 year-old students
School
elementary and secondary school in 16 cities and provinces
School
students, teachers, and infrastructure
Nation
students and teachers
3. Study method The contents of indexes and development procedures, how the experts came up with the standards, and weights, the sampling procedure, and the methods and characteristics of the subjects are described here. 3.1. Index development procedure and description The tools developed to make the indexes for diagnosing the education informatization levels of students, teachers, and parents were modified based on a preliminary investigation. The primary tools were modified three times between September 10, 2007 and September 27, 2007. A total of 15 education informatization experts reviewed the tools twice each before the secondary tools were finalized on October 10. The secondary tools improved the statement method according to the roles of the subjects (i.e., students, teachers, or parents), and after two expert meetings, the tertiary tools were finalized on November 2. After the preliminary test and reliability analysis of the tertiary tools were conducted on November 5 and 6, respectively, the final tools were completed. The final tools were applied from November 8 onward, in order to make the indexes. The indexes of this study were designed to contain all the information needed to be able to measure individual levels on the basis of a comparison with the key indexes previously developed in relation to informatization. The education informatization index of this study, developed on the basis of the developmental procedure discussed above, refers to the overall level of education informatization, which puts together the level of individuals’ accessibility to education informatization, the level of usability, and the utilization level. The education informatization index is divided into the accessibility index, the competency index, the utilization index, and the satisfaction index, while the utilization index is classified into quantitative utilization and qualitative utilization. As for each area, the detailed component areas, the concrete measurement elements of the detailed areas and indexes were generated and defined. First, the accessibility index is the basic device requirement for education informatization. It indicates the level of infrastructure (possession of device or possession of wireless internet connection enabled device, etc.) established for an individual to utilize information and communication technology. Second, the competency index is the level of ability necessary to use education information. It indicates an individual’s level of ability to use devices related to education informatization (including the ability to use computers and other information and communication devices as well as the Internet, and the ability to deal with the harmful effects of Internet use). Third, the utilization index refers to the extent to which education information is used. The utilization index is composed of quantitative utilization and qualitative utilization. Quantitative utilization indicates the amount of education information usage (quantitative level), while qualitative utilization indicates the level of direct support in teaching and learning, and includes the outcome gained through quantitative utilization. Fourth, the satisfaction index indicates the levels of direct or indirect forms of positive changes (from teaching and learning, communication and policy aspects) perceived by the users in the field through the utilization of education information. 3.2. Method of standards and weights designation On the basis of the index for diagnosing the level of education informatization, an attempt was made to measure the levels that the education informatization of students, teachers, and parents must reach. In other words, on the basis of the standard for the level of education informatization suggested by the group of experts, this study tried to establish the levels of education informatization suitable for each group. This study used the Angoff and Jaeger method to set up the standards (Angoff, 1971; Jaeger, 1978). To set up the standards, the experts in the group related to education informatization were classified into the elementary school, middle school, high school, and general levels. The opinions of 10 experts at each level were collected during the time period from November 13, 2007 to November 20, 2007. A total of 37 experts (8 experts from elementary schools, 10 experts from middle schools, 9 experts from high schools, and 10 experts from the general group) participated in the opinion collection process. The opinions of the experts were finally analyzed on November 28, and the final standards and weights were determined after a couple of meetings. 3.3. Sampling procedure The sampling of study subjects for education informatization level assessment was done by using the following 3-stage cluster sampling method:
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
763
In the first stage, study regions were chosen in 16 cities and provinces. Based on the populations, the regions were classified into major cities, medium and small cities, and rural townships. In the second stage, based on consideration of the education levels and living environment conditions, the regions were classified into three levels (high, medium, and low). That is, they were categorized based on socioeconomic classes. In the third stage, the subject schools were selected based on consideration of sizes of the student bodies of the primary and middle schools in each region. The assigned number of students, teachers, and parents from the selected schools were surveyed, and the replies were returned via registered mail. The survey was not done via the web to include subjects with low accessibility. In particular, as the accessibility area included considerations such as internet accessibility and the availability of computers, the self-administration method was used to get more objective responses. 3.4. Sampling method The more detailed procedure and method of sampling subjects for diagnosing the education informatization level are as follows: According to the aforementioned procedure, metropolitan cities, cities and provinces were differentiated in accordance with the following standards. First, there are the classifications of major cities, medium and small cities, and rural townships. All seven special and metropolitan cities are classified as major cities. The smaller cities, including provincial capital cities, in the 9 provincial regions were classified as medium and small cities, and the rural district regions were classified as rural townships. Second, within the selected regions, further differentiations - high, medium, and low levels - were made, based on education levels and living environments. This study came up with 144 groups, and then after considering the group sizes, chose 1 to 2 schools in each group. Accordingly, a total of 177 schools were designated for this study. Third, the election criteria for schools were as follows. Students, teachers, and parents from the selected schools were chosen as respondents of the survey. However, 10 parents were selected to ensure at least 5 responses. Students and teachers were surveyed in schools and their responses were collected immediately. As for parents, the questionnaires were given to students to give to their parents, and the questionnaires were collected afterwards. Fourth, subjects were selected as follows. Fourth graders and higher-grade elementary students were selected as subjects, and the subjects were distributed evenly among different grades. The reason why elementary school students older than 4th graders were selected was that the subjects had to perfectly understand the questionnaire. From the finally selected schools, four 4th graders, four 5th graders, three 6th graders, four 7th graders, four 8th graders, three 9th graders, four 10th graders, four 11th graders, and three 12th graders were selected. As far as teachers are concerned, elementary school teachers were selected by grade, and middle and high school teachers were selected by the subjects they teach. When it comes to parents, one parent of each student participating in the survey was selected. 3.5. Characteristics of subjects The survey for the diagnosis of the education informatization level was conducted between November 23, 2007 and December 14, 2007. 1668 students from 166 schools across the country between the 4th grade and 12th grade, 1017 teachers, and 1354 parents participated in the survey. At the 95% confidence level, the sampling error of this survey was 2.4% points for students, 3.1% points for teachers, and 2.7% points for parents. The sample size of the subjects of this study is shown in Table 2. 4. Findings In this section, the experts’ standard for the education informatization level, the education informatization levels of students, teachers and parents, and the improvements based on the portfolio matrix are described. 4.1. Experts’ standard for the education informatization level This study measured the education informatization levels considered by experts to be the expected standards. The expected standards were measured by using the Angoff method and the Jaeger method. That is, the standards were established in such a way that even experts using tools developed to diagnose education informatization levels would agree that education informatization has been sufficiently achieved. Also, the averages of the scores calculated from the two methods were established as the bases of the education informatization levels. The standard scores for the education informatization levels established by experts for students, teacher, and parents are shown in Table 3. As for the standard scores for the overall education informatization indexes established by experts, for individual groups, the standard for teachers was the highest at 80.90 points, 80.64 for students, and 76.52 for parents. As far as the standard scores for individual areas are
Table 2 Number of samples participating in the education informatization level assessment.
elementary school middle school high school total
schools
students
teachers
parents
52 59 55 166
520 585 563 1668
324 360 333 1017
434 469 451 1354
764
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768 Table 3 The standard scores for the education informatization levels established by experts for students, teacher and parents. pollee
education informatization indexes
student teacher parent
80.64 80.90 76.52 accessibility index
student teacher parent
ownership of devices
availability of IT
recency of the operation system
70.95 83.11 76.97
98.65 99.32 99.32
100 100 97.30
88.04 93.72 89.48 competency index
student teacher parent
accessibility element
78.03 81.28 70.88 utilization index
student teacher parent
competency element hardware literacy
software literacy
internet literacy
selfdevelopment
71.96 78.38 65.54
75.03 80.35 70.49
81.39 83.93 74.53
86.49 83.78 –
utilization element quantitative utilization
qualitative utilization
73.74 62.97 63.19
82.07 80.17 77.38
78.79 74.60 71.23
satisfaction index
78.60 79.56 74.96
concerned, accessibility had the highest score (88.04 points), followed by utilization (78.79 points), satisfaction (78.60 points), and competency (78.03 points). Teachers had the highest standard score in accessibility, and their standard score for utilization was the lowest (74.60 points). For the parents, the standard score of accessibility (89.48 points) was the highest and the standard score of competency (70.88 points) was the lowest.
4.2. The education informatization levels of students, teachers, and parents and the rate of achievement The rate of achievement as compared to the overall education informatization indexes and the indexes by area, and the expected values of students, teachers, and parents, are shown in Table 4. The rates of achievement are the ratios of the current levels to the standard scores established by experts, as shown in Table 3. Accordingly, if “0 < rate of achievement < 100,” it means that the current level failed to reach the expectation of experts, and if “rate of achievement S 100,” it means that the level exceeded the expected value. Table 4 The indexes are the current education informatization levels of individual groups and, the rate of achievement. pollee
education informatization indexes(rate of achievement)
student teacher parent
85.89(97.55) 89.80(95.82) 85.33(95.36) accessibility index (rate of achievement)
student teacher parent
85.89(97.55) 89.80(95.82) 85.33(95.36) competency index(rate of achievement)
student teacher parent
56.13(71.94) 62.37(76.74) 43.30(61.08) utilization index (rate of achievement)
student teacher parent
64.56(81.94) 62.42(83.67) 46.66(65.50)
accessibility element(rate of achievement) ownership of devices
availability of IT
recency of the operation system
77.12(108.71) 77.04(92.70) 77.03(100.08)
91.56(92.82) 97.00(97.66) 93.02(93.65)
92.54(92.54) 96.88(96.88) 90.11(92.62)
competency element(rate of achievement) hardware literacy
software literacy
internet literacy
selfdevelopment
60.53(84.12) 53.36(68.08) 38.26(58.38)
60.11(80.11) 63.70(79.28) 41.22(58.48)
70.12(86.16) 70.47(83.96) 49.43(66.32)
16.79 (19.41) 61.21(73.06) –
utilization element (rate of achievement) quantitative utilization
qualitative utilization
66.93(90.77) 66.72(105.9) 49.73(78.69)
63.29(77.11) 60.43(75.38) 44.98(58.13)
satisfaction index (rate of achievement) 61.98(78.85) 62.09(78.04) 50.96(67.98)
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
765
The overall education informatization indexes were measured, and the results showed that teachers’ index was the highest at 66.96 points, followed by that of students (66.46 points), and that of parents (56.54 points). In other words, the overall education informatization index of teachers was higher than that of students by about 0.5 points. As for the rate of achievement as compared to the expected value, the rate of achievement of teachers was 82.77%, so it was higher than that of students (82.40%) as well as that of parents was 73.89%, who scored the lowest among the three groups. For starters, students’ accessibility index was very high, at 85.89 points out of a possible 100 points. Students’ utilization index was 64.56 points, their satisfaction index was 61.98 points, and their competency index was 56.13 points. Their competency index was the lowest. As for the rate of achievement in each area as compared to the expected value, the rate of achievement of the accessibility area with the highest index was the highest (97.55%), and that of the utilization area was 81.94%, followed by 78.85% in the satisfaction area and 71.94% in the competency area. As in the case of students, the accessibility index of teachers was the highest (89.80 points out of a possible 100 points). The highest rate of achievement as compared to the experts’ standard was 95.82%. That of the index of the competency area was 62.37 points, and that rate of achievement was the lowest (76.74%). Parents’ index for the accessibility area was 85.33 points, and the rate of achievement was quite high, at 95.36%. However, the rate of achievement in the satisfaction, utilization and competency areas was lower than 70%. In particular, the education informatization level of parents was relatively lower than that of students or teachers.
4.3. Portfolio matrix analysis of the educational informatization indexes To deduce the items to be considered for improving the overall education informatization indices, a two-dimensional diagram was created using the importance rate and the achievement rate (%) of each element relative to the expected standards. From the portfolio matrix analysis, elements that need to be improved first, based on the current state, were extracted. The matrix is expressed as four quadrants, and each quadrant’s meaning is expressed below: An element in quadrant 1 is an element with both high importance and a high achievement rate. Therefore, an element with high current levels, the maintenance of which is very desirable, falls under the “maintenance and sustainability” category. An element in quadrant 2 is an element with high importance but a low achievement rate (below average). That is, an element most urgently requiring improvement falls under the “intensive improvement” category. An element in quadrant 3 is an element with both low importance and a low achievement rate. Therefore, consideration must be given as to whether such an element is actually needed for measuring the education informatization level, and this element falls under the “passive improvement” category. An element in quadrant 4 is an element with low importance but a high achievement rate. An element that needs to be managed at the current level falls under the “passive management” category. The results of the portfolio matrix analysis for improvement of the education informatization indexes of students, who are the subjects of this study, are shown in Fig. 1. The element found to have high importance and a high achievement rate, based on the results of the portfolio matrix analysis on education informatization of students and falling under the “maintenance and sustainability” category, is the element of computer/Internet
Fig. 1. The results of the portfolio matrix analysis for improvement of the education informatization indexes of students.
766
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
Fig. 2. The results of the portfolio matrix analysis for improvement of the education informatization indexes of teachers.
quantitative utilization. That is, students’ quantitative utilization of computers and the Internet is determined to be sufficient. In contrast, falling under the “intensive improvement” category of high importance but a low achievement rate, are the elements of qualitative utilization, software utilization, and satisfaction. For these three elements, more so than with other elements, ways must be found to increase their achievement rates. From the teachers’ portfolio matrix results, as shown in Fig. 2, the computer/Internet quantitative utilization element was found to fall under the “maintenance and sustainability” category, with high importance and a high achievement rate. Falling under the “intensive improvement” category of high importance but a low achievement rate are the elements of qualitative utilization, software utilization, and satisfaction. Qualitative utilization, in particular, is an element that shows very high importance but a low achievement rate. For the “maintenance and sustainability” and “intensive improvement” category elements, the analysis results of the teachers’ portfolio matrix coincided with those of the students’ results.
Fig. 3. The results of the portfolio matrix analysis for improvement of the education informatization indexes of parents.
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
767
The portfolio matrix analysis results of the importance rate and the achievement rate (%) of the overall education informatization indices for parents are given in Fig. 3. For parents, computer/Internet quantitative utilization and device ownership elements are in the “maintenance and sustainability” category of high importance and a high achievement rate. Placing in the “intensive improvement” category of high importance but a low achievement rate are the elements of qualitative utilization, software utilization, and satisfaction. 5. Discussions Here is a discussion of the results of the analysis of the education informatization levels of students, teachers, and parents. First, in terms of the students’ education informatization level, they scored the highest in the area of accessibility. This result shows that Korea’s IT infrastructure is at the highest level in the world (OECD, 2005b, 2007). However, the students’ competency and utilization levels were low. In particular, the qualitative utilization and software usage were placed as elements requiring “intensive improvement.” When students used their own competency to learn, the impact was shown to be the highest (Ravitz, Mergendoller, & Rush, 2002). In other words, even when the accessibility level is high, if the competency level is low, the utilization of education information is difficult. Therefore, more must be done to increase competency in order for the students’ education information utilization result to show an increase in learning. More effort is needed because the learning effect is high when the students themselves become the agents of learning (Harrison et al., 2003; Papanastasiou, Zembylas, & Charalambos, 2003). Second, in terms of the teachers’ education informatization level, the achievement rate in the accessibility area was high, while the competency area showed a low achievement rate of only 76.74%. This result is in agreement with past findings, that many teachers are passive in developing their own competency (Seongmu, Seungjin, Taesook, Jaesung, & Jamee, 2008). In the utilization area, while the quantitative utilization was high, the qualitative utilization was low. Because they want to receive a diverse array of information concerning their preference, many teachers utilize websites. For this reason, the quantitative utilization level is shown to be high. However, since teachers are not active in utilization that involves producing their own materials or sharing materials (Jaesin, Hyoroon, Taesook, Jaesung, & Jamee, 2009), the qualitative utilization is shown to be low. In other words, this result shows that the teachers have high quantitative utilization, and also that such a high quantitative utilization does not lead to increased qualitative utilization related to teaching and learning. Third, the parents’ education informatization level shows a low rate of achievement compared with those of the students and teachers. The achievement rates of qualitative utilization and software usage elements are lower than 60%. The low utilization rate of the parents may negatively impact the guidance of their children. There is also a high possibility that it may negatively impact the children’s utilization of education information and the Internet (Jamee, Hyeoncheol, Hongrae, & Junghoon, 2009). The reason for this is that parental interest is an important element of children’s study habits (Jaesin et al., 2009; Barker & Wendel, 2001). Since parents play an important role in increasing their children’s utilization of education information, it appears that there is a need to provide education to the parents, in order to improve in the competency or utilization area. 6. Conclusion This study sought to assess the education informatization level of Korea, which possesses a world-class IT infrastructure. In particular, the study attempted to determine the state of education informatization at the personal levels of students, teachers, and parents, and to find out which areas need improvement. From the diagnosis results regarding the current state of education informatization, all three groups showed an achievement rate of over 95% in the “accessibility” area. However, in the “competency,” “utilization”, and “satisfaction” areas, the teacher group showed achievement rates of less than 80%. For the student group, the competency rate was about 71%, showing a lower achievement rate compared with other areas. The parents showed low achievement rates overall, with less than 70% in all three areas. Based on the results, this study proposes the following: The students need to improve qualitative utilization and satisfaction. The teachers need to increase competency in their use of educational programs and application software. The teachers also need to increase the qualitative utilization related to production or sharing of education information, in order to contribute to improvement in teaching and learning. For the parents, since the levels in all areas, except for accessibility, are low, priority should be given to strengthening competency. This should be done because only when competency reaches a higher level can the qualitative utilization for teaching and learning increase. For balanced development of education informatization, this study diagnosed the current level and proposed elements for improvement. The study also discovered that infrastructure expansion alone does not raise the overall education informatization level. In future studies, how the level of each area of education informatization differs depending on student grade or region should be examined through comparisons, and what areas need support should also be investigated. References Angoff, W. H. (1971). Scales, norms, and equivalent score. In R. L. Thorndike (Ed.), Educational measurement (2nd ed). Washington. DC: American Council on Education. Australian and New Zealand Institute for Information Literacy (ANZIL). (2004). Australian and New Zealand information literacy Framework: principles, standards and practice. Accessed August 12, 2005 available at. In Alan Bundy (Ed.), Adelaide, Australian and New Zealand Institute for information Literacy, 2004 (2nd ed) http://www.caul. edu.au/info-literacy/InfoLiteracyFramework.pdf. Barker, K., & Wendel, T. (2001). e-learning Studying Canada’s Virtual Secondary Schools. In SAEE research Series, Vol. 8. Society For The Advancement Of Excellence In Education. Chartered Institute of Library and Information Professionals (CILIP). (2004). A short introduction to information literacy. Retrieved August 12, 2005 from. http://www.cilip. org.uk/professionalguidance/informationliteracy. European Network on Information Literacy (EnIL) (European Union Retrieved August 12, 2005) from http://www.ceris.to.cnr.it/Basili/EnIL/index.html Harrison, C., Comber, C., Fisher, T., Haw, K., Lewin, C., Lunzer, E., et al. (2003). ImpaCT2: The impact of information and communication technologies on pupil learning and attainment. London: DfES. Heesoo, L., Taejung, K., Wongki, S., Sungbae, O., Soonok, J., & Jaesin, S. (2003). A study on developing indicators for information and communication technology use in lifelong education. Korea Education Research & Information Service. Heungju, K., & Hyesook, K. (2006). An analysis of level measure of educational informatization, in 2006. Korea Education Research & Information Service.
768
JaMee Kim, WonGyu Lee / Computers & Education 56 (2011) 760–768
Hyesook, K. (2006). A study of the current status educational informatization. Korea Education Research & Information Service. Informatization Promotion Committee (IPC). (2008). The basic plan of national informatization. IPC. Inwoo, P., Myeongsoon, P., Dosoon, P., Myeonghye, H., Bumseok, K., Heungju, K., & Soojung, J. (2001). A study on developing indicators for information and communication technology use in primary and secondary education. Korea Education Research & Information Service. Jaeger, R. M. (1978). A proposal for setting a standard on the North Carolina High School Competency Test. Paper presented at the spring meeting of the North Carolina Association for Research in Education Chapel Hill. Jaesin, S., Hyoroon, A., Taesook, A., Jaesung, H., & Jamee, K. (2009). An analysis of the current status and effect about EDUNET. Korea Education Research & Information Service. Jamee, K., Hyeoncheol, K., Hongrae, K., & Junghoon, K. (2009). Development of indicator and criteria for level measure of educational informatization of students, teachers, and parents. Journal of the Korean Association of Information Education, 13(2), 145–157. Jamee, K., & Seungjin, L. (2009). An analysis of content and policy about educational informatization. Korea Education Research & Information Service. Junghee, S., Jusung, J., Bokeong, K., Hyeonjin, K., Sangha, L., Jungweon, S., et al. (2006). Development of standard for analysis of effectiveness of E-learning: Focused on primary and secondary education. Korea Education Research & Information Service. Jungwha, S. (1996). An evaluation of educational policy in Korea, A research of education. Education Research in Hongik university. Kapsu, K., Okhwa, L., Seyeoung, C., Sukhee, W., Minjoo, R., & Beomseog, K. (2002). A study on developing indicators for information and communication technology use in higher education. Korea Education Research & Information Service. MEST & KERIS. (2009). 2008 Adapting education to the information age. MEST & KERIS. Ministry of Education, Science and Technology (MEST). (2009). Informatization promotion action plan (draft) of the Ministry of Education, Science and Technology 2009. MEST. Ministry of Public Administration and Security (MOPAS). (2009). The plan of national informatization. MOPAS. OECD. (2005a). Are students ready for a technology-rich world? What PISA studies tell us. OECD. OECD. (2005b). PISA 2003 technical report. Paris: OECD. OECD. (2007). PISA 2006: Science competencies for tomorrow’s world executive summary. OECD. Papanastasiou, E., Zembylas, M., & Charalambos, V. (2003). Can computer use hurt science achievement? The USA results from PISA. Journal of Science and Technology, 12(3). Plenum Publishing Corporation: New York. Pejova, Z. (2002), Information literacy: An issue which requests urgent action in developing countries and countries in transition. White Paper prepared for UNESCO, the U.S. National Commission on Libraries and Information Science, and the National Forum on Information Literacy for use at the Information Literacy Meeting of Experts, Prague, The Czech Republic, 2003. Viewed July 15, 2006 http://www.nclis.gov/libinter/infolitconf&meet/papers/pejova-fullpaper.pdf. Ravitz, J., Mergendoller, J., & Rush, W. (2002, April). Cautionary tales about correlations between student computer use and academic achievement. Paper presented at annual meeting of the American Educational Research Association. New Orleans, LA. Seongmu, J., Seungjin, L., Taesook, A., Jaesung, H., & Jamee, K. (2008). An analysis of the current status and effect about EDUNET. Korea Education Research & Information Service. Seungjin, L., Jamee, K., Hyeoncheol, K., Minha, K., Aenam, K., Injin, J., et al. (2007). Development of index for gap and level of ICT in education to resolve the polarization of ICT in education in elementary and secondary schools. Korea Education Research & Information Service. (KERIS). Sungeun, B., Cheolil, L., Hyesook, K., Seonyong, K., & Sunghee, J. (2007). Development of self-diagnostic indicators of ICT in primary and secondary schools. Korea Education Research & Information Service. Wonhee, P., Hyunki, S., Aehwa, K., Sungnam, K., & Jiho, D. (2004). The research on ICT indicators for special education. Korea Education Research & Information Service. Yeonseob, H., Bongwoon, H., & Youngrok, K. (2006). A study on analysis of ICT in education budget and ways to reform in Korea. Korea Education Research & Information Service. Youngae, K., & Hyesook, K. (2005). A study on the current status of educational informatization, in 2005. Korea Education Research & Information Service. Youngae, K., & Hyesook, K. (2008). A study about level measure of educational informatization, in 2008. Korea Education Research & Information Service.