Out-of-school assistance in the teaching of visual creative programming in the game-based environment – Case study: Poland

Thinking Skills and Creativity 34 (2019) 100593

Contents lists available at ScienceDirect

Thinking Skills and Creativity journal homepage: www.elsevier.com/locate/tsc

Out-of-school assistance in the teaching of visual creative programming in the game-based environment – Case study: Poland

T

⁎

Taras Panskyi , Zdzislawa Rowinska, Sebastian Biedron Lodz University of Technology, Institute of Applied Computer Science, Poland

A R T IC LE I N F O

ABS TRA CT

Keywords: Out-of-school education Creative programming Game-based learning ICT

The paper presents effects of out-of-school teaching of computer science in a visual creative programming course (Scratch) for children aged 9–14, held at the Lodz University of Technology. The research was carrying out during 2016–2018 school years. The study sample consists of 265 primary and secondary students from Lodz Voivodeship (province) in central Poland. The results were obtained from anonymous questionnaires completed by 221 course participants and their parents. The answers confirm that this type of course becomes a new fascinating manner of spending spare time by children. Moreover, quantitative analysis of student’s finals projects also has been performed. In the process of creative programming in the game-based environment, children develop the computational thinking skills, problem-solving strategies, and abstract thinking. Moreover, children are supported by their parents, who notice how important these competences are and how great opportunities they will present for children in future. Authors continue to grow Scratch programming course to democratize access to new technologies and education, preparing future generation for a world in which computational and algorithmic thinking is a central part of problem-solving. Perhaps some of the course participants will continue their study of programming and make it a career for their life.

1. Introduction and motivation Nowadays, the omnipresent informatization of the society forces changes in teaching computer science in all types of schools, starting from primary schools up to universities (Chen et al., 2017). The computer science curriculum in primary schools should reflect the current level of the students’ computer expertise. Most children beginning their school career at the age of 6–7 can use many digital devices, such as tablets, smartphones, or game consoles, e.g. Xbox One or PS4. The operation of those devices is so intuitive for children that students who use those technologies for the first time usually catch up quickly. Correct use of those resources should prepare students to live their lives in the information society and to use modern information and communication technologies (ICT) (Margolis, 2010). In recent years in Poland, most urban primary schools have been equipped with computer rooms with permanent access to Internet; however, the manner of teaching computer science has failed to keep pace with the universal presence of ICT. Studying computer operation, simple graphic applications, text editing devices, calculation sheets, or searching for information on the Internet is certainly necessary but not sufficient. The information obtained from primary schools of the Lodz Voivodeship (1118 children) show that during computer science and programming classes students use the MS Office package (41 per cent), Paint (26 per cent), Scratch (23 per cent), and other applications, such as Gimp, Photoshop, or Movie maker (10 per cent). Data indicate that a great

⁎

Corresponding author at: Stefanowskiego 18/22, 90-537, Lodz, Poland. E-mail addresses: [email protected] (T. Panskyi), [email protected] (Z. Rowinska), [email protected] (S. Biedron).

https://doi.org/10.1016/j.tsc.2019.100593 Received 7 April 2019; Received in revised form 8 August 2019; Accepted 20 August 2019 Available online 21 August 2019 1871-1871/ © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/BY-NC-ND/4.0/).

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

majority of children does not develop the skill of logical and computational thinking which are involved in the study of programming (Hsu & Wang, 2018; van Niekerk & Webb, 2016); those children who have the possibility of learning basics of programming start their studies definitely too late, i.e. at the age of 13 (in many countries e.g. Great Britain, Japan, Lithuania, the Czech Republic, Serbia, Slovakia, Slovenia or Turkey children as young as 6–10 years old start learning programming). To improve the existing situation, in 2016 the Ministry of National Education introduced a new core curriculum for computer science in primary schools (www.ore.edu.pl/nowa-podstawa-programowa). It predicts the following results in future: easy solution of problems in everyday and professional situations involving technology, support of other disciplines in innovation, optimization of processes, more frequent choice of further computer science education in secondary schools, increase in the number of students sitting a computer science maturity exam, increase in the number of computer science students at universities, and satisfaction of the expectations of the labor market. The core curriculum includes an increased number of computer science hours (see next Section). All primary school students are required to begin with computational thinking, supported with a visualization or simulation of algorithmic actions. This way, students make their first steps in a visual programming language. Simple applications steering a robot or a different creature on a screen are developed. They use specific Internet resources safely and legally. Students of higher years, who have already been introduced to computational thinking, develop and expand their expertise and computer science skills by learning fundamental IT notions and solving selected issues using algorithms; they also implement solutions and make their first steps in a text programming language. However, despite the introduction of the core curriculum, the study of algorithmics and programming is still neglected in most primary schools. The success of covering that syllabus depends, to a great extent, on a proper preparation of teachers to teach classes of that type. Computer science in I-III grades is usually taught by early school teachers with no experience in teaching algorithmics or programming. In IV-VIII grades, there are also no well-prepared teachers. Primary school teacher competencies in computer science is described in greater detail in the next Section. To teach classes of that type, it is necessary that a teacher has expertise and skills in algorithms and programming environments, visual as well as other ones, e.g. enabling steering a robot. Therefore, constant improvement of one’s competences related to the subject taught is an essential element of teachers’ work (Worek, Jelonek, & Kocór, 2017). At present, in Poland, it has been noticed that it is necessary to support professional improvement of computer science teachers through training courses organized by Teachers’ Improvement Centres, using EU subsidies or subsidies provided by Polish state institutions. However, the results of changes in the computer science curriculum and improved competences of teachers will not be visible in teachers and candidates for universities until a few years’ time; we are in a transitional period now. A separate issue is the relationship between educational failures in higher education and the condition of the education system at the earlier stages. A high rate of failures in universities in technical courses, in particular in computer science courses, and mediocre opinions on the preparation of candidates for those courses may suggest, that it is an imperfect system of teaching in primary and secondary schools that is responsible for such a situation (Smużewska, Wasilewski, & Antoniwicz, 2015). Undoubtedly, the introduction of obligatory algorithmics and programming classes at the early stage of education will improve the quality of teaching and the university study efficiency ratio. However, the effects of the introduced changes will not be visible until a few years’ time; therefore, out-of-school education remains a temporary form of supporting students in learning programming. A great part of primary school students shows, even now, a great interest in learning programming, which is, unfortunately, absent from their schools. Work-related expectations of parents towards their children also include the need of obtaining that knowledge as early as at the stage of a primary school. Kozłowski and Matczak (2016) show that the profession of a computer scientist and a programmer is one of the most expected for children, followed by medical professions (28.8 per cent); in 2011 it was 6.5 per cent and in 2014 it grew to as much as 10.9 per cent (Brzezińska & Rekosiewicz, 2016). As a result, children and their parents seek other, out-of-school possibilities of learning programming in the form of workshops or training courses (Glušac, Makitan, Karuović, Radosav, & Milanov, 2015; Hawrot, 2015). This work presents effects of out-of-school teaching of computer science in a visual programming course (Scratch) for children aged 9–14, held at the Lodz University of Technology. Computer classes of that type have become a new manner of spending spare time by children, apart from sports and foreign language classes, which have been popular so far. In the process of programming, children develop the skill of computational thinking, problem-solving strategies, and creativity in game-based learning. In the parents’ opinions, the improvement in those computer competences should also translate into a change of the manner in which children use computers, smartphones, or tablets, not only as devices for gaming or browsing the Internet, but also as tools for creating one’s own applications. 2. Current state of primary school informatics in Poland The education system in Poland is governed by Acts of Parliament and ministerial regulations adopted, in particular, by the Minister of National Education, responsible for general and vocational education. A major reform in the primary school education system was initiated in the school year 2016/2017 to strengthen general education as the basis for further personal development of students. The reform will be completed in the school year 2022/2023 (Kolanowska, 2018). Since 2016/2017, the school system at the primary level consists of the 8-year primary school for students aged 6/7–15. Primary education is divided into two ISCED stages: integrated early school education (grades I to III) and primary school education (grades IV to VIII). According to the Polish National Curriculum, all children are required to learn informatics (in Poland “informatics” is equivalent to “computer science”) in primary schools. The history of informatics education in Polish schools in the last 25 years (1965–1990) is described in detail by (Sysło, 2014). Gurbiel, Hardt-Olejniczak, Kolczyk, Krupicka, and Syslo (2005) shows the past reform of the Polish education system, the main objective of which was to integrate ICT into all school subjects. In early school education, informatics (understood as computer science) as a school subject is now called information technology 2

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

education (in Polish: edukacja informatyczna). In grades I-III, information technology education is compulsory and it is integrated with subjects such as Polish language, mathematics, social education, natural sciences, art education, music education etc. The new curriculum provides for the minimum of 1 h weekly allocated to information technology classes. However, the minimum number of hours may be increased at each headmaster’s discretion and may be allocated to activities developing students’ interests, in particular activeness and creativity. Students learn informal meaning of selected notions related to computer science, often in the form of play. Key skills to be developed by information technology education include the use of linear order repeating actions (e.g. arranging patterns, pictures, texts, events in a logical sequence), searching actions (e.g. searching for an object with the indicated features by eliminating those objects that do not meet the criteria), instructions or instruction sequences, tasks, riddles, and puzzles leading to the discovery of algorithmic thinking. Student use computers and functional applications by creating graphic compositions and text documents, helping themselves to learn how to read, write, calculate, and present ideas. In grades IV-VIII, informatics (in Polish: informatyka) is taught for at least an hour per week for five years. The curriculum contains a section on algorithmic thinking, problem solving using computers, computational thinking, and game-based learning. During the informatics classes, children are being prepared to use computers, computer networks, and multimedia tools on an advanced level, create simple hypertext pages, and use programming elements in visual and text languages; they also learn about data protection, intellectual property rights etc. These learning achievements should give them a solid background for using ICT in other subjects. Besides the methodological issue of the new curriculum (Sysło & Kwiatkowska, 2015), research literature reveals that school informatics in Poland is still confronted with a number of problems. Firstly, informatics is still taught as if this subject was just a tool for teaching the usage of specific software systems (spreadsheet, word processor etc.) Hadjerrouit (2009) emphasized that informatics does not provide profound understanding on a deeper conceptual level but is used for memorizing details of software and reproducing information about buttons, menu commands, dialogue boxes, or other interface elements. As a result, students have access to highlevel tools for designing and creating computer applications without any knowledge of fundamentals of informatics such as logic, programming methodology, or algorithmic decomposition (Sysło, 2011). Therefore, students find informatics boring and not challenging enough to choose it as a subject for study in high school (Sysło & Kwiatkowska, 2013). Secondly, the current status of furnishing schools with ICT equipment is still unsatisfactory. Due to a poor development of infrastructure and a lack of equipment and skilled personnel in urban as well as rural areas, computers and Internet resources are not widely used for subjects other than informatics. Moreover, almost all rural communities in Poland have a problem with technological facilities and have not been able to obtain any benefit from the advantages of using ICT. Poland, however, has shown determination in efforts to bridge the digital gap between rural and urban students to meet the needs of the national agenda. Together with the pilot programme (September 2016), the Ministry of National Education and the Ministry of Digitalization undertook many complementary activities aimed at improving the quality of schools' access to the Internet (LAN and WiFi) and modern ICT devices (computers, projectors with interactive whiteboards, presentation clickers, tablets, visualizers, etc.) to achieve the 2019 vision toward creating a knowledgeable society. Nevertheless, rural areas still have to cope with limited infrastructures, incapability to buy ICT equipment, and lack of knowledge on ICT use. According to Plebańska and Tarkowski (2016), approximately 500 schools in Poland are ready to work in the one-to-one computing model. Thirdly, schools slowly adapt to technological and pedagogical changes and, as a result, teaching methods are still based on traditional epistemologies. Teachers still use traditional blackboards and chalks where pointers may be drawn as arrows to describe interactive presentations of selected algorithms. Moreover, some teachers in higher grades IV-VIII write whole programming codes on a blackboard while students should draw them on paper and integrate them in programming software without consciously knowing how or why (Adamaszek, Chrza̧stowski-Wachtel, & Niewiarowska, 2008; Saeli, Perrenet, Jochems, & Zwaneveld, 2011). When students want to run a program on their own, try to understand its behavior, rework and refund it, or create and debug a new program, all they are left with is a compiler and debugger. When in confusion or frustration, they have no full support of their teacher. Sysło and Kwiatkowska (2008) show how students in primary school should begin to learn informatics principles and to start master computational thinking. According to Bates (2015), teachers should use traditional technologies in combination with digital technologies. Finally, the real competencies and qualifications of teachers in informatics are far less than might be expected in primary education. The great majority of informatics teachers have already specialized in different areas e.g. chemistry, physics, sometimes even religious or physical education (Kolczyk, 2008). These teachers have obtained qualification to teach informatics during various courses or after-graduate studies. As a result, they often do not feel confident with the concept of logic or algorithm. They are very cautious and hesitant about teaching problem-solving and programming. Furthermore, teachers have not understood still that personal development is crucial for teaching informatics. Teachers should broaden their computer science knowledge in order to respond to rapid ICT technology changes (Webb et al., 2016). Since teachers play a pivotal role in teaching informatics, their perception of the innovation will strongly influence their students’ thinking. It is still not obvious to teachers that lifelong learning of informatics and ICT starts at the very beginning of formal education in primary school (Syslo & Kwiatkowska, 2005). Moreover, the role of teachers is not only to educate themselves and develop their own professional knowledge but also to promote and integrate innovations in teaching informatics and prepare their students for lifelong learning.

3

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Table 1 Participants of the programming course for children by editions, age and gender. Participants (%) Age group (%)

Age

Total

Boys

Girls

38 (79.17%)

9 years 10 years 11 years 12 years 13 years 14 years

5th edition (2018/2019) 15 (31.00%) 12 (25.00%) 11 (22.92%) 5 (10.42%) 4 (8.33%) 1 (2.08%) 48 (100%)

11 (22.92%) 9 (18.75%) 7 (14.58%) 2 (4.17%) 3 (6.25%) 1 (2.08%) 33 (68.75%)

4 (8.33%) 3 (6.25%) 4 (8.33%) 3 (6.25%) 1 (2.08%) 0 (0.00%) 15 (31.25%)

9 years 10 years 11 years 12 years 13 years 14 years

4th edition (2017/2018) 10 (17.24%) 15 (25.86%) 9 (15.52%) 10 (17.24%) 8 (13.79%) 6 (10.34%) 58 (100%)

7 (12.07%) 12 (20.69%) 5 (8.62%) 8 (13.79%) 6 (10.34%) 5 (8.62%) 43 (74.14%)

3 (5.17%) 3 (5.17%) 4 (6.90%) 2 (3.45%) 2 (3.45%) 1 (1.72%) 15 (25.86%)

9 years 10 years 11 years 12 years 13 years 14 years

3rd edition (2017/2018) 20 (28.17%) 19 (26.76%) 15 (21.13%) 9 (12.68%) 4 (5.63%) 4 (5.63%) 71 (100%)

14 (19.72%) 17 (23.94%) 12 (16.90%) 8 (11.27%) 4 (5.63%) 4 (5.63%) 59 (83.10%)

6 (8.45%) 2 (2.82%) 3 (4.23%) 1 (1.41%) 0 (0.00%) 0 (0.00%) 12 (16.90%)

9 years 10 years 11 years 12 years 13 years 14 years

2nd edition (2016/2017) 4 (8.70%) 13 (28.26%) 8 (17.39%) 8 (17.39%) 9 (19.79%) 4 (8.70%) 46 (100%)

3 (6.52%) 12 (26.09%) 8 (17.39%) 7 (15.22%) 8 (17.37%) 4 (8.70%) 42 (91.30%)

1 1 0 1 1 0 4

(2.17%) (2.17%) (0.00%) (2.17%) (2.17%) (0.00%) (8.70%)

9 years 10 years 11 years 12 years 13 years 14 years

1st edition(2016/2017) 14 (33.33%) 10 (23.81%) 7 (16.67%) 9 (21.43%) 2 (4.76%) 0 (8.70%) 42 (100%)

13 (30.95%) 8 (19.05%) 6 (14.29%) 8 (19.05%) 0 (0.00%) 0 (0.00%) 35 (83.33%)

1 2 1 1 2 0 7

(2.38%) (4.76%) (2.38%) (2.17%) (4.76%) (0.00%) (16.67%)

10 (20.83%)

34 (58.62%)

24 (41.38%)

54 (76.06%)

17 (23.94%)

25 (54.35%)

21 (45.65%)

31 (73.81%)

11 (26.19%)

3. Method 3.1. Participants From the international perspective, the study sample consists of 265 primary and secondary school students from K4 to K9 curriculum standards in the Lodz Voivodeship (province) in central Poland. The sample is a representative focus group consisting of children aged 9 to 14 during the 2016/2017, 2017/2018 and 2018/2019 school years. Participants were recruited from public/ private primary and secondary schools in the Lodz province. Moreover, the programming course was divided to 5 editions. Table 1 shows the conclusions of the descriptive statistics for the programming course divided by editions. We collected data for this sample for 3 years: during the 1 st edition (from September 2016/2017 school year), 42 participants attended the programming course; during the 2nd edition (from May 2016/2017 school year) – 46; during the 3rd edition (from September 2017/2018 school year) – 71; during the 4th edition (from May 2017/2018 school year) – 58, and during the 5th edition (from September 2018/2019 school year) – 48. The number of participants in the programming course increased by 46 per cent from 2016/2017 to 2017/2018, from 88 to 129. The 3rd edition in the 2016/2018 school year had the biggest number of participants (71); the lowest number of children took part in the 1 st edition – 42. The lowest number of participants in the 1 st edition could be explained by and attributed to poor course marketing and a lack of positive reviews from the former participants and their parents. The 3rd edition boasted the highest number of participants that could be related to the new Polish core curriculum 2017/2018 for computer science teaching in all primary and secondary public schools which attracted the parents’ attention to the course. With respect to gender, there were always more boy than girl participants, with 80 and 20 per cent respectively. In the 2nd 4

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Fig. 1. The comparative analysis of participant age in all course editions.

edition, 91.3 per cent of the participants were boys. On the other hand, there was a high number of girls in the 5th edition, almost one third (31.25 per cent) of the total number of participants. The highest number of girls per the age bracket occurred in the 3rd edition (8.45 per cent) in case of 9-year-old girls, while the strikingly opposite situations with no girls took place in the 1 st edition (14-yearolds), the 2nd edition (11- and 14-year-olds), the 3rd edition (13- and 14-year-olds) and the 5th edition (14-year-olds). The lack of girls aged 13 and 14 and plenty of younger girls could be the result of the parents’ aspiration to seek the best start of education for their children, as the importance and the need of children’s early development for their later success. Interestingly, during all the five editions, the number of girl participants gradually increased (except for the gap in the 2nd edition). Hence, visual programming and game-based learning has become one of the popular approaches to learning programming. The linear increase in the number of girls in the programming course shows clearly that girls could be interested in and/or intrigued by programming and game design. Regarding age, we found that the most common participant age in each edition was 9 and 10. A high number of 9- and 10-year-old participants made more than a half (57.14 per cent in the 1 st edition; 54.93 per cent in the 3rd edition; 56 per cent in the 5th edition) of all participants in all the editions. As a general rule, regardless of the edition, the higher the participant age was, the smaller number of participants were recruited to the course. The highest percentage of 13-year-olds participated in the 2nd edition (19.79 per cent) and the highest percentage of 14-year-olds – in the 4th edition (10.34 per cent). Despite this, the percentage of 13- and 14-yearold participants was the lowest in each edition. The reason for such a small interest in a programming course is a great number of various courses, workshops, and extra classes children take part in after school. Some parents are completely confused by the market oversupply of different educational offers. In some cases, parents could not afford sending their children to attend our programming course only because the children attended other classes at the same time. Another important reason is the perception of visual programming by 13- and 14-year-old children as childish and frivolous, with nothing in common with “real” programming. The comparative analysis of participant age in five course editions is shown in Fig. 1. Special attention should be paid to the factor based on the participant age groups 9 to 11 and 12 to 14. In each course edition, the participants were classified as a member of one of two age groups with a year’s gap between age groups. For the younger group (9–11), more time was spent on a clear explanation of lesson materials, while the older – acquires programming knowledge faster. Moreover, the threshold of 11–12 years between groups was not chosen accidentally, but on the basis of computational thinking skills, program-solving concepts and knowledge of participants in different age group. The younger age group in each of the 5 editions was a relative majority, with the biggest share (79.17 per cent) in the 5th edition and the smallest share (54.35 per cent) in the 2nd edition. Accordingly, the maximum and minimum percentage of participants per edition in the older age group was the opposite (maximum 45.65 per cent – 2nd edition, minimum 20.83 per cent – 5th edition). In other editions, the percentage of participants in a certain age group fluctuated between 54–76 per cent for the younger group and 23–45 per cent for the older one. For instance, the age group factor plays an important role as an evident key in understanding and exploring the educational outcomes beyond computer science from the participant perspective as well as in the pedagogical context from a tutor viewpoint. Regarding origin and a place of residence, we can distinguish between participants from urban and rural areas. Most participants in the programming course lived in urban areas (226 participants – 85 per cent of the sample population). In rural areas, only 39 of the participants (15 per cent) enrolled for the course. Five biggest participant clusters in urban areas, i.e. Lodz, Zgierz, Aleksandrow Lodzki, Pabianice and Konstantynow Lodzki, made almost 79 per cent of all course participants. Moreover, according to the participant geographical distribution, the 86 per cent of a sample population lived at a distance of no more than 20 km from the venue of the programming course (city of Lodz). Moreover, it was both impressive and commendable for us that one participant lived at a distance of 117 km (Smardow) from Lodz. That could only show a great desire of the participant, the commitment of the parents, and an excellent tutors’ approach in learning visual programming. However, transport costs and a distance to the venue of the course show that most of participants reside permanently in cities and villages not far (10–20 km) from Lodz. Fig. 2 shows the course participant clusters by place of residence. 5

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Fig. 2. The formation of participant clusters regarding the place of residence.

3.2. Participant registration This chapter shows how to pre-register a participant to the next course edition. New parents can use an online form to begin the pre-registration process for their children. Online pre-registration is tied directly to the university domain and the course web page. Since children are not adolescents, the online course pre-registration may only be done by parents. Parents send in a request to preregister their children by submitting basic information which include participant’s legal first and last name, age, first and last name of a parent (legal representative), email, and phone number. As an optional but not mandatory feature, a parent can submit additional personal information on a child: should we pay attention to something while working with your child (illness, difficulties, interests)? Based on these critical pieces of information, the parent’s request is sent to the site to be approved. The parent will receive a green confirmation message on screen. An email will be sent to the new parent which includes a confirmation of pre-registration, a course edition timetable, and a link back to the course website. The basic principles and the admissibility of personal data use, as stipulated in the EU’s General Data Protection Regulation, must be observed in the online pre-registration application form. The Institute of Applied Computer Science as a personal data controller may collect, process, or use personal data for relevant purposes according to the Data Protection Regulation. Third parties receiving personal data from the institute, i.e. other university units, may process or use such data only for the purposes which it was transmitted for and in accordance with the law. Online pre-registration contains the information about the parent and participant personal data; therefore, the parent should read and accept the Data Protection Regulation. Otherwise, the parent may receive a Form Incomplete message, which will provide optional notes from the site to the parent and re-open the application form. Online pre-registration becomes active six months before the start of the next edition. A month before the start of a new edition, tutors contact the parents. The parents who confirm their child's participation automatically complete the full registration process. However, approximately 6 per cent of the parents resign from their children’s course participation for various reasons. In such a case, another parent can register in their place. The online pre-registration process ends one week before the start of a new edition. After that, parents can still pre-register; however, children will be automatically entered in a reserve list. If a reserve list is created but all the parents confirm their children participation, candidates from the reserve list are promptly excused and informed about the lack of vacancies.

6

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

3.3. Visual programming environment There are many types of visual programming environments for beginner programmers that highlight the distinct benefits to programming behaviors for problem-solving strategies in game-based learning. These environments may assist users in understanding their programs by examining the state (appearance, location) of the visual objects. Visual programming environments reduce the unnecessary syntax difficulties and assist programmers in visualizing the effects. Moreover, they are able to validate a computer program immediately after it has been modified by the user (Chao, 2016). Nevertheless, this study focuses on the exploration of how beginner programmers learn programming by solving the computational problems and if they increase their programming skills. Therefore, the Scratch programming environment has been chosen as a key game-based learning software for this course. The Scratch programming software is the most popular computing language, more than doubling the next closest language (Blockly) according to the Rich et al. (2018). Scratch is a visual programming environment, allowing programmers to create various scripts through a process of dragging-and-dropping command blocks of code then stacking these blocks together to form coding scripts that could become increasingly complex structures (Lye & Koh, 2018). The Scratch programming language allows programmers to learn main programming concepts, such as loops, synchronization, variables, conditionals, operators, broadcasts, and more (Fields, Kafai, & Giang, 2016). Moreover, Scratch.mit.edu is an online massive community where participants can share their computer programs and implemented ideas. Programmers can share and post their animations, games, stories and the interactive art they have made in the Scratch programming environment. 3.4. Course edition organization Course editions occur cyclically twice a school year (autumn and spring edition). The course is held in the Institute of Applied Computer Science at Lodz University of Technology. Each edition consists of 12 sessions, each taking 2 lesson hours (1,5 clock hour). All registered participants of each particular edition are divided into groups of 8–9 participants at the most. The purpose of small groups is to provide additional opportunities for the participants to interact, provide answers, and to have their responses monitored by the tutors. Tutors assign participants to one of two age groups based on the age of the participant, their programming knowledge and on the additional personal information in the registration form. Course sessions are held only on weekends, that is every Saturday and Sunday. Depending on the wishes and requests of the parents, a participant may be classified to a Saturday or Sunday group. The session hour for each group is also chosen on the basis of parents’ requests. If the parents have not specified a suitable session hour, the participant is classified by the tutors to a proper group. For example, the approximative hours for Saturday groups are as follows: 8:30-10:00; 10:30-12:00; 12:30-14:00; 14:30-16:00; 16:30-18:00. As it can be seen, the break between two neighboring groups is 30 min. Participants come to sessions on their own or with their parents. Parents are also obliged to collect their children after the session. Each participant in a group is working on his personal desktop computer place. Before the start of a new edition, the enrolled participants can choose, for convenience, to work on a computer or on the tablet. Each computer place for a particular participant remains the same until the end of the course edition. Two tutors are always engaged in teaching one session and directing the whole group of participants. Both tutors are working cooperatively and teaching the same session topic at the same time. For example, one tutor could present a new topic, and the other interjects with examples, explanation, and extensions of key programming ideas. Both of them can provide strategies to assist participants in remembering, understanding and organizing the presented information better. Each session starts with the previously presented material and offers an opportunity for reinforcement. A flexible grouping arrangement also provides participants with limited prior knowledge of the target content or problems with assimilation of new information with a possibility to bridge the gap in their background knowledge. Both tutors have sufficient pedagogical competencies in using different game-based approaches in a programming course. Tutors are usually Ph.D. students in computer science with the minimum of 5 years of experience in teaching classes in visual programming to children and certified at least at English B2 proficiency level. They have competencies related to planning meaningful session topics and game-based programming activities within the course curriculum. Tutoring competencies also refer to guiding participants in the learning process during the course sessions, including applying motivation techniques, personalizing activities, and regulating the degree of participant commitment with flexibility. Moreover, if a participant works in a self-directed manner, the role of both tutors is to observe the activities and ensure that nobody gets stuck in understanding the material. Every course edition is divided into 12 sessions presented in Table 1. These sessions are organized into five topics (introduction, creative art and stories, mathematics and logic, game creating, final project), as a way for participants to explore different genres of creative expression and form, while developing familiarity and fluency with computational concepts and practices of game-based learning.

Session aims Session 1: Participants are introduced to the Scratch programming environment by viewing a collection of sample projects and engaging in an exploratory creation. Participants build their own first interactive projects by using the Scratch block (motion) with adding, changing or modifying the Sprites and Stages. Session 2: Reinforcement of the previous motion knowledge. Participants learn how to make their projects more interactive by means of Scratch block (Looks). They learn how to express a complex activity using a sequence of simple instructions, creating a Scratch project that involves animation.

7

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al. Session aims

Session 3: Reinforcement of the previous motion and animation knowledge. Participants learn how to create a Scratch project that combines animation and music (block Sound). They learn how to use sequence, loops, events in development, testing and remixing existing projects. Session 4. Participants learn how to develop greater fluency with computational concepts (i.e. parallelism, events) and practices (i.e. iterative and incremental development). They learn the basis of mathematics and geometry by means of understanding the Pan block and creating their own squares, rectangles, triangles (equilateral, right), circles and more complex geometric shapes (stars, pentagrams, polygons). Finally, they learn different approaches to coordinating action within/across shapes using location points x and y. Session 5. Participants lean how to use more complex Control (if-then, wait until, repeat until) and Broadcast block instructions in their self-directed project through the practices of testing and debugging. Session 6. Participants explore computational creation within the genre of games by designing a maze using Scratch block (Sensing). They create the simple or complex game based on tutor instructions (touching the color, sprite, or mouse). Moreover, they diversify their maze with the time limited option (timer, reset timer). Session 7. Participants learn how to use (create, change, delete) independent and dependent variables in their projects. They learn how to use the rest of Sensing block instructions (distance to, video motion, etc.). Moreover, they explore and create simple calculators or “millionaire” games using the above mentioned variables, Operator block, and ask and wait for instructions. Session 8. Participants learn how to use (create, change, modify, and delete) lists in their projects. The learn how to use More Blocks to reduce, organize and optimize their projects space. Finally, they explore the cloning feature that enables them to create the tower defense and arcane-style games. Session 9-11. Participants create their own final project (adventure, racing, puzzle and fighting games or stories and quizzes, etc.). Tutors monitor the work of the group and help with programming problems. Participant are fully responsible for their own individual, unique final projects. Tutors do not force participants to make specific projects but only inspire them with new and fresh ideas. Session 12. Participants’ final project presentation. They present their projects to others in the group and their parents. The necessary condition for completing the programming course is presentation of the final project. Each participant has 12-15 minutes for project presentation which includes the participant introduction, project key idea, and show-how activity.

Participants in each session: 8–9 Tutors per session: 2 Duration of each session (in minutes): 90 Duration of each edition (in minutes): 1080 The course supports learning programming only in the mother tongue (Polish). However, there were attempts to reach out to parents who wanted their children to be taught in English. Tutors understand the importance of developing skills in other languages and are willing to help parents in whatever way they can to promote programming skills development. In such cases, participants are allowed to switch the Scratch interface into the English version; moreover, only the computational terminology (loops, sequence, events, variables, etc.) is described in English and the rest of the material is Polish. In general, there were 2 such cases in all course editions. Certain life events have been associated with the participant session absence. These included illness of a parent or a close relative, traumatic events in everyday life, and prolonged absence from a course because of personal illness. However, it is unknown and unpredictable how these events precipitate episodes of session absence. But if such events occur and a participant cannot attend a session, he/she informs the tutors via e-mail and, with the tutors’ agreement, the participant can attend classes with another group. If the prolonged absence lasts for more than one session, the participant gets an opportunity of distant learning by receiving all the necessary materials from the tutors and doing homework dedicated to the sessions topic.

3.5. Completing the course edition The final project represents a culmination of participants’ new knowledge and provides an opportunity to expand their understanding of a particular problem. The participants’ final project consists in developing and designing their computer “piece of art” on the basis of all the studied material, own preferences, wishes, and vision. According to the course edition syllabus, the last 3 sessions are dedicated to creating the final project. During the first session dedicated to the final project, tutors lead the class through a group brainstorm, discussing aspects of common ideas participants enjoyed. For example, the tutor asks participants: What is your hobby? What is your favorite computer game? What is your favorite school subject? Then, the tutors are able to inspire the participant to start writing a plan of his/her final project. Participants then complete a shortlist and record ideas they will later use to program a project in the computer lab. Participants are encouraged to talk to colleagues sitting nearby and tutors. For those participants who need a challenge, the best way to assist them includes offering them problems and outcomes and encouraging them to work out the steps they need to take to make a successful project. When participants are really confident, they can use more complex computing concepts to apply their skills in project-making. During the last sessions tutors explain the main stages of creating the final project: a logical beginning (project purpose, settings, loading, etc., if the final project is a game – the choice of the protagonist, number of game levels, the difficulty of levels) and a logical ending (summery, acknowledgements, etc., if the final project is a game – the number of earned and spent points, money, lives, tools, etc.). After that, participants are ready to start creating their final projects. It should also be emphasized that the tutors provide round-the-clock access to the cloud, where each participant can save their final project. The cloud storage gives all the participants the opportunity to save, modify, delete, and create crucial parts of their projects. Moreover, each participant has its own login and password to access the cloud storage. The cloud also provides the possibility of remote work at any time and place, which is an additional flexible tool for the final project maintenance and management. The last session is dedicated to the final project presentation (Figs. 3 and 4). The final projects are presented to the rest of the 8

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Fig. 3. The final project presentation by a 9-year-old participant.

Fig. 4. The final project presentation by a 13-year-old participant.

group and parents. During these presentations, participants also explain ideas behind the project and describe the computational concepts they used: variables, loops, events, etc. Our explanatory analysis revealed that younger participants (9–12) make maze-like, quiz-like and football-like games. Those participants also choose realistic or fantasy adventure games. On the other hand, 12- to 14year-old participants produce more project genre variation (arcade games, fight simulations, tactical and popular on-time games like fortnight, counter strike, etc.). When comparing the final projects based on the age, it may be said that 12- to 14-year-old participants are more precise in game-design, flow control and game logic; however, 9- to 12-year-old participants design their projects in interesting and unique ways (sequence of unpredictable events, weird Sprites, and imaginary situations). The absolute majority (89 per cent) of final projects are game-based. Moreover, the rest of final projects are story-based and are developed mostly by girls. The participants’ final project presentation is very useful in four different ways. Firstly, it is an integral part of the participant’s Scratch programming, since the participants use many of different methods (programming, designing graphics) to express a meaningful idea and a problem-solving concept. Secondly, it is a perceptible result of their work, which facilitates reflection and social mediation in the group of colleagues and tutors. Thirdly, the participants’ presentations are a valuable part of data collection for evaluative analysis. Finally, presentations are a means by which participant work is mediated to interested parties outside the school everyday life, i.e. the tutors, the parents or other family members and academic staff. Fig. 5 illustrates a code from the final project by a 9-year-old participant, showing advanced synchronization (using backdrop changes), parallelism (several scripts), flow control (using forever and forever-if blocks), and data representation (operators and variables). However, the same final project is poorly diversified on abstraction (no clones or new blocks defined).

Fig. 5. Sample code from the final project by a 9-year-old participant. 9

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

4. Results and discussion On our last check of the participant final exam presentation, 257 participants passed the course and 8 of them did not pass. The interesting paradox was that the 8 participants who did not present the final project attended all the course sessions except the last one. Moreover, if a participant did not present the final presentation at the last session, they had the opportunity to retake the presentation and to pass the course on a different date. Obviously, tutors want all the participants to do well at the final presentations; moreover, tutors may even be thinking ahead toward getting them a place at one of the advanced programming courses and, finally, a place at the Technical University of Lodz as a candidate for higher education studies. However, 8 participants did not pass the course even with the additional date and tutors could not get in touch with these participants or their parents. After completing the final presentation, the participants receive course certificates signed by the director of Institute of Applied Informatics, who is also the course supervisor. Moreover, the certificates include the marks received by the participants for their presentations. The whole presentation session is a kind of a ‘mock’ exam for the participants. Some of them present their project for the first time in their lives; for others it is a usual and routine matter. However, all participants receive very good marks for their presentations as an award for completing the programming course, which is a symbol of appreciation for their participation in the course as well as their efforts put into preparing and presenting their individual projects. The mark is supposed to encourage them rather than discourage from learning programming further or create an atmosphere of too much competition among the participants. After the presentations, the tutors ask participants and parents or legal representatives to fill in an anonymous questionnaire dedicated to the course edition. Shortly after all the celebrations, the tutors inform the parents about other programming courses held at the Institute of Applied Informatics. One of the courses is called “Robotics” and is based on the Lego Mindstorms EV3 and Lego Wedo edutainment robots. Participants learn the secrets of robotics, mechatronics, engineering and informatics. A graduate is able to design and program the devices using computer technology, i.e. line follower, gyro boy, inspection robot, remote control robot, etc. Another course held at the Institute of Applied Informatics is titled “Arduino” and is based on the Arduino Uno electronics platform. In this course, tutors take care to increase the complexity of projects to a level just within the reach of participants. A graduate is able to design and program Arduino devices with multiple sensors, such as temperature and humidity, light, movement, etc. Furthermore, a graduate is able to control led and led bars using dimmer switches, to output text information on an LCD display, to program a LED matrix, to design and program a radar using a servo motor and an ultrasonic sensor etc. The tutors learn about the final projects participants have completed on a previous level (the Scratch based course) and use this information to plan the Robotics and Arduino courses. Moreover, the programming courses helps participants to provide consistency and continuity from one course to the next. New courses also give tutors the opportunity to keep track with young people gifted in creative programming and problem-solving outcomes over the long term. Finally, the program involves a study during which coursework and field experiences are closely interwoven: programming and robotics or electronics. The focus of these courses is on the strategies that promote interdisciplinary collaboration of different technical disciplines. Furthermore, the participants learn about the applied character of creative programming in engineering problem-based tasks. 4.1. Qualitative analysis The qualitative study used an anonymous questionnaire provided to the participants and parents or legal representatives. The main questions of the questionnaire are presented in Table 2. The questionnaire was provided to the group of participants after the presentation of final projects. The questionnaire is divided into two parts: the first part is for a participant and the second one is for their parent or legal representative. The tutors left several questionnaires printed on the A4 page on a table in the classroom and each person was instructed to complete one. Tutors also emphasized that participation was completely voluntary, and the questionnaire will serve only for improving and modifying the programming course. Respondents took 10 min on average to complete the questionnaire. The key issue was how to distribute and to collect the questionnaires without compromising the anonymity of the survey. The problem was solved by leaving the group participants and their parents in the classroom for 10 min. During this time, the tutors were in the neighboring classroom. Thus, we obtained the information from the respondents without making them known to the tutors. The questionnaire was completed by 221 participants and their parents or legal representatives, i.e. 36 questionnaires were left empty without any answers. The questionnaire shows the participants’ expectations; it also demonstrates the need for such a creative programming course as a good alternative to out-of-school education. Almost all (92 per cent) parents or legal representatives left their mobile phone numbers and e-mails for notifications of new course editions and for other course offers. In our research, 2 participants answered “computer hacker programming” to the question What do you want to study in the next programming course?, which was very interesting and amusing; however, we viewed these answers with a sense of humour. 4.2. Quantitative analysis Despite many efforts aimed at assessment of CT and algorithmic thinking, assessing the learning of CT concepts and algorithmic or logic constructs in a programming environment such as Scratch remains a challenge. However, tools that can assess the above qualities of projects programmed by students do exist. CT formative-iterative tools are aimed at providing feedback to a learner, usually in an automatic way, in order to develop and enhance student’s CT skills (Román-González, Moreno-León, & Robles, 2019). These tools do not assess individuals but products of their learning, usually programming projects. Therefore, authors find and apply 10

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Table 2 Anonymous questionnaire provided to the participants and parents or legal representatives. For the participant Did you enjoy this course? Yes (100%) No (0%) No answer (0%) Did this course meet your expectations? Yes (100%) No (0%) No answer (0%) Which session topic did you enjoy most? Examples of answers: Variables, Pen block, Lists, all. What do you want to study in the next programming course? Examples of answers: advanced Scratch, Python, robotics, C++, Minecraft, Java, Baltie, 3D computer graphics. No answer (16%) For a parent or legal representative How do you evaluate the course preparation: tutors’ knowledge, substantive preparation, course organization? Examples of answers: Excellent, very good, good, great, ok. Bad, poor, not enough (0%) No answer (4%) Do you want to receive notifications of new courses for your children? If so, please provide contact details. Yes (92%) No (0%) No answer (8%)

Dr. Scratch iterative tool (Moreno-León, Robles, & Román-González, 2015) for Scratch projects quantitative assessment. The Scratch online community includes not only children but also developers and researchers. A group of these researchers created Dr. Scratch, a free and open source web application that analyzes projects programmed with Scratch visual programming environment. Dr. Scratch can automatically measure the degree of CT demonstrated in a Scratch project in hand. The score assigned by Dr. Scratch to each project is based on a degree of seven dimensions of CT and algorithmic thinking competences, i.e. abstraction and problem decomposition, logical thinking, synchronization, parallelism, synchronization, algorithmic notions of flow control, user interactivity, and data representation. For each of these dimensions, students can earn 0–3 points; the highest overall score that can be obtained is 21. Thus, projects with up to 7 points are considered as Basic CT competencies, projects between 8 and 14 points are recognized as Developing, and projects with more than 15 points are marked as Master. For quantitative analysis, tutors, who are at the same time cloud storage administrators, download all 251 fin. l projects from the cloud storage and upload each particular project to Dr. Scratch. Fig. 6 shows the findings from a quantitative analysis of 251 fin. l student’s projects. Results show that our participants are best at synchronization (2.66), parallelism (2.62), flow control (2.28), and logic (2.24). These high scores are followed by user interactivity (2.00) and data representation (1.93) with a higher relative gap in abstraction (1.55). The overall mean CT score is 15.28 with standard deviation of 2.05, which definitely indicates a high-level (Master) of CT and algorithmic thinking skills. These results indicate that students are not likely to be as proficient at filtering out what information is necessary to solve problem (abstraction) and are also less likely to demonstrate the ability to prioritize in a way that allows them to solve their problems efficiently (data representation). However, the ability to deal with abstraction and data representation in programming seems to improve with experience; as students gain more proficiency in programming and solid experience in informatics, they become familiar with it and are able to use and integrate new information with greater success and efficiency.

Fig. 6. The overall mean score of 251 fin. l projects for each of the CT dimension. 11

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Fig. 7. Gender differences in overall mean score of 251 fin. l projects for each of the CT dimension.

Gender differences also manifest themselves in CT skills and abilities. The overall mean CT score was 15.67 with standard deviation of 1.21 for girls and 15.46 with standard deviation of 1.90 for boys respectively. Based on the results presented in the Fig. 7, synchronization, parallelism, flow control and user interactivity hardly differ between participants of both genders. At the same time, logic, data representation and abstraction vary greatly. The analysis revealed common computational patterns for synchronization, parallelism, flow control and user interactivity, where gender does not play a significant role. According to Smith (2010), boys are better in areas of math that involve logical and analytical thinking. However, our course has helped a significant number of girls to become equal or even superior in logical thinking. This shows that tutors can level a gender-based disparity in logical reasoning abilities. Moreover, girls are better in data representation (more accurate representation of position, direction, size, color of each Scratch sprite). Girls learn informatics through meticulous and systematic scrutiny of the material in a precise and detailed manner, while boys tend to create large and expansive final projects without paying any additional attention to improving the visualization aspect. On the other hand, boys have a greater tendency toward high programming abstraction and design. To sum up, the authors believe that major gender differences lie in patterns of learning motivation, achievement and willingness than in overall level of CT skills and intelligence. The Fig. 8 shows the mean CT score of 15.05 with standard deviation of 2.29 for participants aged 9–11 and 15.87 with standard deviation of 1.45 for participants aged 12–14. According to the analysis, participants in the older group (9–11) show slightly higher scores in every CT dimension (except flow control). The only difference in final projects is shown in the dimension of flow control, which means that younger students were using “repeat until”, and not just “repeat” as their older colleagues did. Moreover, younger students’ projects use more repetitions blocks (sequences and loops) then those of the older group.

5. Limitations This study has several limitations. We focused on the quality of anonymous questionnaires and students’ final projects.. Despite

Fig. 8. Age group differences in overall mean score of 251 fin. l projects for each of the CT dimension. 12

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

many efforts aimed at the assessment of quantity and quality of the final projects, the evaluation of projects in the Scratch programming environment remains a challenge. Dr. Scratch digital instrument enables the automatic measurement of the degree of computer programming and CT skills in a given Scratch project, s. nevertheless, even Dr. Scratch could not measure some of the key CT elements like debugging, efficiency, recursive thinking, and pattern generalization (Grover & Pea, 2013). Moreover, Dr. Scratch cannot be used in pretest conditions, since it is applied onto final projects after the students have learned basic computer programming and CT concepts. In this research, anonymous questionnaires were used to provide a better understanding of participants’ and parents’ course feedback instead of more intense forms of qualitative methodologies, such as face to face interviews. Nevertheless, Patton (1999) showed that although written responses to open-ended and close-ended questionnaire questions are the most basic and simple form of qualitative data, they do provide more information and clarity to a quantitative analysis. Another limitation may be a relatively low number of participants, i.e. 265 primary and secondary school students. Yet, even though we should be cautious in the interpretation of these findings due to a small sample size, they provide us with relative confidence about the veracity of discovered arguments. And the final point about our sample is that all the participants come from Lodz Voivodeship (province). Moreover, the sample size does not show the situation for all Polish children with regard to developing their level of programming skills or problem-solving strategies. 6. Conclusions The increase in computer science competences (ICT) of the 21st-century society forces quick changes in curricula for computer science in schools and at universities; that, in turn, requires a certain transitory period. Out-of-school programming courses temporarily fill in a gap which is created as a consequence of an insufficient preparation of primary and secondary schools to teach classes in algorithmics and programming. Courses held at the Lodz University of Technology arouse a great interest of children and teenagers, not only from Lodz, but also from further and smaller cities of the Lodz Voivodeship. Such out-of-school education results in not only the improvement of a child’s computer competences, creativity, abstract thinking, or problem-solving strategies, but also in the development of a child’s personality. Children are supported by their parents, who notice how important these competences are and how great opportunities they will present for children in future. Perhaps some of the course participants will continue their study of programming and make it a career for their life. Programmers are, at present, one of the most desired professional groups in the world. However, if children choose a different career path, the obtained skills of computational thinking and programming will still be useful in other areas. From the viewpoint of universities, candidates to technical courses who already have a certain expertise in programming improve the quality of teaching and the level of studying. References Adamaszek, M., Chrza̧stowski-Wachtel, P., & Niewiarowska, A. (2008). VIPER, a student-friendly visual interpreter of pascal. Lecture Notes in Computer Science, 192–203. Bates, A. W. (2015). Teaching in a digital age. Guidelines for designing teaching and learning for a digital age. Victoria: BCcampus. Brzezińska, A., & Rekosiewicz, M. (2016). Plany edukacyjne i zawodowe uczniów ostatnich klas szkół ponadgimnazjalnych a wymiary rozwoju ich tożsamości. Edukacja, 136(1). Chao, P. (2016). Exploring students’ computational practice, design and performance of problem-solving through a visual programming environment. Computers & Education, 95, 202–215. Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162–175. Fields, D., Kafai, Y., & Giang, M. (2016). Coding by choice: A transitional analysis of social participation patterns and programming contributions in the online scratch community. Mass Collaboration and Education, 209–240. Glušac, D., Makitan, V., Karuović, D., Radosav, D., & Milanov, D. (2015). Adolescents’ informal computer usage and their expectations of ICT in teaching – Case study: Serbia. Computers & Education, 81, 133–142. Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43. Gurbiel, E., Hardt-Olejniczak, G., Kolczyk, E., Krupicka, H., & Syslo, M. (2005). Informatics and ICT in polish education system. From Computer Literacy to Informatics Fundamentals, 46–52. Hadjerrouit, S. (2009). Teaching and learning school informatics: A concept-based pedagogical approach. Informatics in Education, 8(2), 227–250. Hawrot, A. (2015). Pozaszkolna pomoc w nauce. Edukacja, 133(2). Hsu, C., & Wang, T. (2018). Applying game mechanics and student-generated questions to an online puzzle-based game learning system to promote algorithmic thinking skills. Computers & Education, 121, 73–88. Kolanowska, E. (2018). The system of education in Poland. Warsaw: Foundation for the Development of the Education System. Kolczyk, E. (2008). Algorithm – Fundamental concept in preparing informatics teachers. Lecture Notes in Computer Science, 265–271. Kozłowski, W., & Matczak, E. (2016). Aspiracje zawodowe rodziców wobec synów i córek. Edukacja, 138(3). Lye, S., & Koh, J. (2018). Case studies of elementary children’s engagement in computational thinking through scratch programming. Computational Thinking in the STEM Disciplines, 227–251. Margolis, J. (2010). Stuck in the shallow end: Education, race, and computing. Cambridge, MA: MIT Press. Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Automatic analysis of scratch projects to assess and foster computational thinking. RED. Revista de Educación a Distancia, 15(46), 1–23. Patton, M. (1999). Qualitative evaluation and research methods. Newbury Park: Sage. Plebańska, M., & Tarkowski, A. (2016). Cyfryzacja polskiej edukacji. Wizja i postulaty. [ebook]. p.14. Available at:Warszawa: Centrum Cyfrowe Projekt: Polska. https:// centrumcyfrowe.pl/wp-content/uploads/2016/07/cyfryzacja-polskiej-edukacji_final.pdf. Rich, P., Browning, S., Perkins, M., Shoop, T., Yoshikawa, E., & Belikov, O. (2018). Coding in K-8: International trends in teaching elementary/primary computing. TechTrends. Román-González, M., Moreno-León, J., & Robles, G. (2019). Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In S. Kong, & H. Abelson (Eds.). Computational thinking education (pp. 79–98). Singapore: Springer.

13

Thinking Skills and Creativity 34 (2019) 100593

T. Panskyi, et al.

Saeli, M., Perrenet, J., Jochems, W., & Zwaneveld, B. (2011). Teaching programming in secondary school: A pedagogical content knowledge perspective. Informatics in Education, 10(1), 73–88. Smith, I. (2010). Boys, girls & learning pocketbook. Alresford: Teachers’ Pocketbooks. Smużewska, M., Wasilewski, K., & Antoniwicz, D. (2015). Niepowodzenia w studiowaniu z perspektywy uczelni i studentów. Edukacja, 135(4). Sysło, M. (2014). The first 25 years of computers in education in Poland: 1965–1990. IFIP advances in information and communication technology266–290. Sysło, M. (2011). Outreach to prospective informatics students. Lecture Notes in Computer Science, 56–70. Sysło, M., & Kwiatkowska, A. (2015). Introducing a new computer science curriculum for all school levels in Poland. Lecture Notes in Computer Science, 141–154. Sysło, M., & Kwiatkowska, A. (2013). Informatics for all High school students. Informatics in schools. Sustainable informatics education for pupils of all ages43–56. Sysło, M., & Kwiatkowska, A. (2008). The challenging face of informatics education in Poland. Lecture Notes in Computer Science, 1–18. Syslo, M., & Kwiatkowska, A. (2005). Informatics versus information technology – How much informatics is needed to use information technology – A school perspective. From Computer Literacy to Informatics Fundamentals, 178–188. van Niekerk, J., & Webb, P. (2016). The effectiveness of brain-compatible blended learning material in the teaching of programming logic. Computers & Education, 103, 16–27. Webb, M., Davis, N., Bell, T., Katz, Y., Reynolds, N., Chambers, D., et al. (2016). Computer science in K-12 school curricula of the 2lst century: Why, what and when? Education and Information Technologies, 22(2), 445–468. Worek, B., Jelonek, M., & Kocór, M. (2017). Czy nauczyciele uczą się przez całe życie? Kompetencje i aktywność edukacyjna nauczycieli. Edukacja, 140(1).

14