- Email: [email protected]
Combining Learning Factories and ICT- based Situated Learning
Available online at www.sciencedirect.com
ScienceDirect Procedia CIRP 54 (2016) 101 – 106
6th CLF - 6th CIRP Conference on Learning Factories
Combining learning factories and ICT- based situated learning Nina Tvenge*, Kristian Martinsen, Sri Sudha Vijay Keshav Kolla NTNU, Faculty of Technolog, Economy and Management, Teknologivn 22 2815 Gjøvik, Norway * Corresponding author. Tel.: +47 40636548; E-mail address: [email protected]
Abstract This paper discusses the combination of learning factories and situated (on-the-job) ICT-based social learning, as well as the didactics of learning factories and ICT-based situated learning. Moreover, a case study at the Lean Lab - learning factory in Norway is described, with results from questionnaires sent participators/students about the learning outcome in the lifelong learning aspect. Three managers where asked to what extent their company benefited from the lean lab workshops. The paper suggests an enhancement of the learning factory with an online course before and after the lean lab workshop session. This can both make the simulations at the learning lab more effective as well as enhance the usefulness of the learning into positive change for the companies. © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license © 2016 The Authors. Published by Elsevier B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility the scientific committee of theConference 6th CIRPonConference on Learning Factories. under responsibility of the of scientifi c committee of the 6th CIRP Learning Factories Peer-review Keywords: Learning factories, manufacturing education, cyber physical systems, lifelong learning, online courses, lean
1 1.1
Introduction
research), or an effective competency development (if used for education and training)”[2].
Learning Factories
Both manufacturing and learning paradigms will be affected by digitalization in the future, with increasing possibilities for data collection, analysis, modeling, simulation, optimization and communication. Future manufacturing systems will be “Cyber Physical Systems” combining computational elements, physical elements, software and humans. Some call this trend the 4th industry revolution, or Industry 4.075. Similarly, technology is an ever more relevant and important factor in teaching and learning in general. There is, however, still a need for social and practical training and technology is not a substitute for this, but a range of different tools that can enhance learning and increase students’ learning space [1]. To meet manufacturing industries’ demand for knowledge and innovations in the age of Industry 4.0, learning factories are established in many educational and industry organisations [2-4]. The main goals of learning factories are either technological and/or organizational innovation (if used for
Figure 1. The Modern Workplace Learning Framework [5]
2212-8271 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the scientific committee of the 6th CIRP Conference on Learning Factories doi:10.1016/j.procir.2016.03.031
102
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
Didactic methods and mind-sets must be a part of a learning factory, as it is for all learning processes, but review shows that there is still a lack of research conducted on learning and teaching methods in these learning facilities. Consortiums have explored and decided on the morphology, and the focus is on practice-oriented learning processes [2, 6-12], but the effects on learning outcome and best didactical approaches are not well mapped yet, although there was increased focus in 2015[13, 14]. And one can also debate whether the learning factories are focusing too much on efficiency, as in reducing production costs, rather than human needs and demands [8]. Learning factories have a natural place in future lifelong learning, but has potential for being a more efficient learning approach if being a part of a broader educational context. One example is the modern workplace learning framework (Figure 1) [15, 16, 5] The framework is based on, amongst others, analyzes of changes in top 100 learning tools during the last nine years. One conclusion from this work was that learning on the web is different from traditional learning in many ways as it is: • • • • • • • • •
continuous on demand takes place in short bursts takes place on the go and in the flow of work social not designed serendipitous autonomous performance-oriented
4000 respondents were asked what type of activities give most value at work, and activities that are autonomous, as in self-organized by individuals and teams and social, meaning learning with and from one another in the workplace were highest rated. Conventional learning factories could be used in all these different practices together with ICT - based workplace learning. 1.2
ICT based situated learning
ICT has a natural place in today’s education and knowledge creation, but the implementation has far from satisfied the involved participants [17]. The ability to collaborate is highly acknowledged and wanted by employers, and teamwork and communication must be facilitated in the future work places [18]. “The 2012 NMC Horizon Report” say that it is a normal expectation that humans will be able to work, learn and study independent of time and space. And the 2014 report states that: “the “social” of learning will emerge even more and that ”collaborative learning” actually is decisive in order to succeed in education in the future.” These trends are continuing in the 2015 report as well [18-20]. ICT-based learning has gone from being closed off and individual focused to being a social learning arena where sharing is essential and the learner’s needs are at the center, not the technology [21]. There are versatile expectations
when it comes to implementation and use of ICT in lifelong learning; • • • • •
increased learning as an effect of access to more data and knowledge more efficient learning learner focused learning activities new learning environments with higher degree of collaboration/cooperation more opportunities for critical thinking and analytical approaches
Other aspects are increased learner motivation /-productivity and autonomy [22]. Services through web 2.0 and web 3.0, also called semantic web, make it possible to share infinite amounts of multi-medial learning resources. More effective and efficient learning processes can be obtained by utilizing ICT, but not without support, reviews show that learning activities as social interactions guided by a teacher, has had the greatest impact on learning outcome, significantly bigger than other methods [23]. Another significant finding is that the more interaction and working with curriculum, the more you learn [24, 25], a mechanism ICT-supported learning can enhance when implementation is well planned. Development of ICT-supported learning is closely linked to the available technology in time, and its affordances, rather than looking at educational research and needs [25]. A Norwegian monitor [26, 27] on use of ICT in teaching and learning in higher education shows that there is a slight increase in usage of technology in education, but there is no increase in well-planned didactical teaching/learning resources. In 2010 it was stated that “in Norway there is a one-sided focus on tool competency and a random approach: a “try and fail”-strategy when it comes to educational technology” And there is not much evidence on this having changed[28]. A review of “all” research on ICT and learning since the millennium [25] found that collaborative learning is efficient if the participants engage at a more advanced taxonomic level. Collaborative learning can lead to a deeper form of learning; ability of critical thinking; understanding, decision making and even longer memory spans. Research has then seemingly started to focus on how to control the learning processes in order to achieve wanted results/ effects [25]. A growing number of social networks and other web 2.0 and 3.0 services can be used for flexible and informal learning and provide access to experts and peers. Workers can thus set up their personal learning environments (PLEs) according to their interests, learning styles and ambitions. This is both an opportunity and a challenge for the individual learner. Large enterprises have the power to develop internal personal learning environments; SME’s have to utilize more or less ready-made solutions [29]. This does not need to be looked upon as a negative; it can give room for more open learning processes. Open educational resources (OER) are freely accessible documents and media resources for teaching, learning, education, assessment and research purposes.
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
1.3
Lifelong learning
How can the future, competitive, sustainable industry attract and support competent and engaged employees that partake in improvement and growth processes? Learning is a basic human need, and lifelong learning is a key element for a good work life. Lifelong learning is defined by the European Commission as "all learning activity undertaken throughout life, with the aim of improving knowledge, skills and competence, within a personal, civic, social and/or employment-related perspective” [30]. According to the USBM consortium [31] lifelong learning is not widely implemented yet, but embraced all over Europe. And "…Workers must strengthen their critical thinking, embrace technology, and become lifelong learners…". “Learning is not only a human activity that is escaping from the classroom but also one that is being recognized as happening more in sites such as the workplace and home where it has always taken place”[32]. Formal learning plays a minor part in the networked workplace, a norm is that about 80% of workplace learning is informal [33]. Informal learning is a broad expression, which describes a “wide range of new approaches to workplace learning…”[16]. Both informal and formal learning are important in the cycle of lifelong learning. Situated learning, i.e. learning in the particular environment where it also is going to be applied, first presented by Lave and Wenger [34] in Communities of Practice. CoP is defined as “tightly-knit groups that have been practicing long enough to have developed into a cohesive community, which provides a sense of belonging, commitment, and shared identity”[35, 34, 36-38]. The idea behind CoP is that the organization or business must organize the workplace so that the employees can learn, act and make decisions based on their own competencies [38]. Hart [39, 15, 16, 5] emphasizes the importance of supporting informal learning rather than “managing” it. 2 2.1
both master- and bachelor levels. The groups can be unified all-from–one-organization, or it can be a mix of different organizations. Each team will be in charge of equal simulated manufacturing line making a wooden house mockup product. The lines resemble a real manufacturing line including a shop floor data collection and visualization (Andon) system. The lean tools such as JIT, Andon, Hijunka, PokaYoke and 5S [40] are important aspects, but the main focus is on productivity, safety and quality. For each team there is a dedicated supervisor from Lean Lab who facilitates the learning process. There is an element of competition between the teams, but also collaboration within and between the teams.
Figure 2. Lean Lab - a full scale simulator
The simulation starts with an introduction to lean and lean tools, and an introduction to the simulator. Each student will have a dedicated workplace throughout the simulations, doing transportation tasks, assembly and quality assurance. Between each round there is time for reflection and planning of improvements that are applied in the next round. The Lean Lab follows thus the experimental learning cycle as described by Pfeiffer et al. [41]
Case study: Lean Lab Norge AS Description of Lean Lab Norge
Lean Lab Norge AS is a learning factory focusing on teaching lean principles through a full-scale simulator. Lean Lab is situated at the Raufoss industry park and was founded in 2009. The owners are a number of industry partners, SINTEF and NTNU. The learning factory falls into Abele et al’s [2] “Industrial application scenario” but it could be argued to be partially in the “Consultancy application scenario” due to the heavy involvement of SINTEF. Lean lab would mainly be placed as “Company funds, single events, course fees, education, manufacturing, cognitive, closed scenario, brownfield, onsite learning” in the described morphology (ibid). Students/participants are working in three or four teams doing one-day simulations at Lean Lab. Most of the groups attending are employees in manufacturing industry, but there are examples of groups from healthcare, public service and other sectors. NTNU is using the Lean Lab for its campus students as high-fidelity training simulations on
Figure 3. Experimental Learning Cycle [41]. Table 1 shows how Lean Lab workshops or courses are set up according to the learning cycle stages from Pfeiffer’s model. In the first round the work instructions are pre-fixed and non-optimum. The teams will typically fail to reach the targets because of unbalance, waiting time, lack of inventory and defects. After first round each team reflect on failures, bottlenecks etc. of the previous round through a dialogue in the group including the supervisor who facilitate the discussion; a guided social interaction [23], giving hints about the lean tools, but the purpose is that the teams come up with clever ideas themselves. Based on their reflections, the teams will rewrite and apply revised work
103
104
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
instructions. This is repeated between the second and the third round where the supervisors also encourage discussions between the different teams. After the end of round three there will be time for reflections and a summary of the achievements during the day. Table 1: Stages of the Lean Lab simulation
Stage Orientation Clarification Experience
Reflection
Generalization Application
2.2
Lean Lab operations Theoretical overview on lean Information about the workshop, rules and procedures. Agenda and overall tasks. Three different simulation runs, 1st run wit pre-set work instructions, 2nd and 3rd after reflection, generalization and adaption of improvements. Analyses of the 1st (and 2nd round), detect bottlenecks, cause of failure etc., and discuss improvement. Discussions within the group and between the groups. Produce revised work instructions Application of improved instructions in next round (2nd & 3rd)
Learning outcome assessmentß
At the end of a Lean Lab - workshop, the students/ participants answer a short questionnaire on their satisfaction. The authors of this paper wished, however, to investigate if the Lean Lab workshops have contributed to changes in the organization: To what extent has the learning factory experience resulted in improvements and change in the organization of the participants? And is the Lean Lab workshop only a “happening” with personal learning outcome, but difficult to convert to practical actions and implementation afterwards in the company? Three managers in companies who had groups of employees on a lean lab workshop in the past where questioned on the long-term results. The selected managers were: • • •
The CEO of a large company manufacturing electronics (circuit boards) The Vice President of supply chains in a large company manufacturing maritime electronics The director of operations in a medium size company manufacturing products to the defense sector.
Typical questions were on to what extent the workshop has resulted in implemented/ improved lean principles in the company, if manufacturing and/or administrative processes has been improved, successful standardization, reduction of lead times, improved productivity and quality. All three managers where positive to the outcome of the Lean Lab workshop, and believed the workshop had contributed to improvements in the company. Establishment
of continuous improvement culture, less sick leave, improvements on safety, quality and productivity (cutting cost) were mentioned as effects. The authors also made a questionnaire based survey among students attending the workshop, where thirteen of 17 students responded. The questionnaire asked for the learning outcome some time after the workshop, and the responses where mainly very positive. A typical answer from the students was that the Lean Lab workshop gave them a better overview and a framework on the lean paradigm, useful as later reference. On the statement “group activities during the workshop enhanced my personal learning”, all except one, agreed. “The workshop offered diverse ways of assessing my learning” gave, on the other hand, more mixed answers as half of the students disagreed or were not satisfied. A couple of comments from the students exemplifying their perceived learning outcome “I compared my understanding of Lean principles before and after workshop. After the workshop I am able to understand the importance of Lean principles in practice” and “It helped me to understand that group work is more important and step-by-step process can help us to do work more effectively”. 2.3
Discussion of the case results
The outcome of the interviews and survey on the (perceived) learning outcomes shows a general satisfaction to the Lean Lab learning outcome. This is the case for the companies investing in one day at Lean Lab (according to the managers questioned), as well as the individual students. The individual perception of the learning outcome and the mainly positive outcome can have several reasons (in addition to that they did really learn a lot): It was a social event, with aspects of (friendly) competition, a sense of achievement. While campus students felt a sense of “real world manufacturing” in spite of being a simulation, this was not the case for the lifelong learning students, as they came from industry or other workplaces. Still they where positive to the simulation and accepted this as a valuable learning experience. The students gave more mixed response to the question about “diverse ways of assessing my learning”, “enough opportunities [..] to find out if I clearly understand lean principles” and ”opportunity to discuss the lean principles with colleagues”. In other words, some of the students did feel the learning was too unidirectional and with less opportunities to discuss and get feedback. The main reason for this is probably the one-dayonly time constraint as well as the necessary rigidity in the direction of the simulation. Moreover can, of course, this type of simulation not cover all parts of the lean paradigm, and aspects such as lean organization/leadership, flexibility/fast changeovers etc. where not covered. Regarding the effect on the improvement in the attending companies, it should be treated with care. It is difficult to distinguish the effect of the workshop from the combined effects of all improvements measures in the company. The decision to attend the Lean Lab workshop is usually not isolated, but a part of a lean improvement program. The
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
workshop is, however, acknowledged to be a valuable method for learning of lean principles. 3
Combining learning factories and online courses
As described, both students attending the Lean Lab and the companies choosing to use the Lean Lab, had predominantly a positive impression of the learning outcome. The social aspect of learning and the way the simulation connects theory with practice and learning-by-doing are probably some of the main reasons for the success. A literature review on “learning factories” and evaluation of “learning outcome”, where the purpose was to find to what extend there are studies made on learning effects of the morphologies and frameworks that are developed for learning factories, show that there is a lack of research on this, with some exceptions [13]. The didactic “what’s” and also “how’s” of learning processes are work well in progress in learning factories, but knowledge on why we do it exactly like this can still be explored more, and most essential; the effect on students’ learning outcome of our teaching methods in learning factories, needs to be further explored. We know what we want to teach and how to do it, but we don’t really know whether these methods are the best or the most efficient. Recent studies of simulation in nursing education show that students do learn and that the theory-practice gap is closing up. And high-fidelity simulations create enthusiasm and inspiration, which are important features in good learning environments [42]. These are mechanisms we believe are recognized in manufacturing simulations as well. But learning is rarely a one-day happening, learning theories show, on the other hand, that learning often needs repetition and maturation and a one-day-only workshop has its pitfalls in this perspective. Marstio and Kivelä found that learning environments that include “social networking services offers an open learning environment supporting both formal learning in educational institutions as well as informal lifelong learning in SMEs”[43]. The authors argue thus that a combination of an online course in combination with the workshop is likely to enhance the learning outcomes, both for the individuals and the companies. This combination would also allow a less unidirectional learning, cover more aspects on lean, give the students more feedback and possibility do discuss and reflect. In short: it can be a way to improve on those areas where the students’ responses were more mixed.
! !
%
%!# %
! !
!
"
Figure 4 shows a possible structure of a combined course. The idea is to combine on-site online learning with the learning factory workshop. Prior to the workshop the students can view recorded lectures, attend online tutoring (synchronous or asynchronous through a blog/forum). The main idea is to prepare the students theoretical background for the workshop. Students will be in different levels and through selection of the lectures etc. can all students be more on level before the workshop. Some of the later lectures can focus on practical issues of the forthcoming workshop to prepare the students. Prior to the workshop there could be a multiple-choice online test or similar, and it could be a prerequisite for attending the workshop that this test is passed. The main reason for this test is to ensue that students are prepared to the workshop. At the workshop day, the before mentioned orientation and clarification can be shortened and just a quick repetition, using for instance student response - systems (SRS) like Kahoot or Socrative, and then reduce the one-tomany teaching and rather focus on potential collaborative social learning aspects of the simulation rounds. This is a somewhat similar approach as the flipped classroom approach [44]. After the workshop the course can continue with reflections and discussions. We propose to challenge the students with a mini-project based in the needs of their own workplace. The idea is lead the students to reflect on how the knowledge gained through the first steps of the course and the workshop can be applied in their own companies. 4
Conclusions
This paper describes a case study of the Lean Lab Norge learning factory at Raufoss industrial park in Norway. The perceived learning outcome for the participants was assessed through discussions and questionnaires. The paper suggest a combination of the social synchronous learning in the Lean lab simulator and an asynchronous online course, where students can both prepare for the simulation beforehand, as well as discus and reflect afterwards. 5
Acknowledgements
The authors wish to thank the Lean Lab Norge for giving access to the simulator and data on participation. Moreover, the authors wish to thank Norwegian Research Council for funding this work through the SFI Manufacturing and Norgesuniversitetet for their support. References [1] Brown, M., 2005, Learning Spaces, in: D. G. Oblinger and J. L. Oblinger (Eds.), Educating the Net Gen,. Boulder, Colorado, p. 21. [2] Abele, E., Metternich, J., Tisch, M., Chryssolouris, G., Sihn, W., ElMaraghy, H., Hummel, V. and Ranz, F., 2015, Learning Factories for Research, Education, and Training. Procedia CIRP 32, 1-6.
Figure 4. Possible structure of the combined online course/learning factory workshop.
[3] Faller, C., Feldmúller, D., 2015, Industry 4.0 learning factory for regional SMEs. Procedia CIRP 32, 88-91.
105
106
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106 Sustainability: A Program Evaluation from Egypt. International
[4] Seitz, K.-F., Nyhuis, P., 2015, Cyber-physical production systems
Journal of Education and Development using ICT 10, 120-141.
combined with logistic models-a learning factory concept for an improved production planning and control. Procedia CIRP 32, 92-97.
[23] Mincu, M. E., 2015, Teacher quality and school improvement: what is the role of research? Oxford Review of Education 41, 253-269.
[5] Hart, J., 2016, 2016: Rethinking workplace learning, Learning in the
[24] Bonesrønning, H. and Opstad, L., 2012, How Much is Students’
Modern Workplace.
College Performance Affected by Quantity of Study? International
[6] Hummel, V., Hyra, K., Ranz, F. and Schuhmacher, J., 2015,
Review of Economics Education 11, 46-63.
Competence Development for the Holistic Design of Collaborative Work Systems in the Logistics Learning Factory. Procedia CIRP 32,
[25] Wasson, B. M., K., 2014, Information and Communications Technology and Learning: State of the Field Review. Knowledge
76-81.
Center for Education., Oslo.
[7] Kim, Y. S., Lee, J. O. and Park, C. W., 2011, A hybrid learning system proposal for PLC wiring training using AR, Proceedings - 2011 5th IEEE
International
Conference
on
E-Learning
in
[26]
Electronics, ICELIE 2011, pp. 90-94.
2011,
Digital
Tilstand,
in:
R.
Fossland,
Gjerdrum
(Ed.),
Norgesuniversitetets skriftserie. Norgesuniversitetet, Tromsø.
Industrial
[27] 2015, Digital tilstand 2014, in: H. Ørnes (Ed.), Norgesuniversitetets skriftserie. Norgesuniversitetet, Tromsø, p. 226.
[8] Kreimeier, D., Morlock, F., Prinz, C., Krückhans, B., Bakir, D. C. and Meier, H., 2014, Holistic Learning Factories – A Concept to Train
[28] Ottestad, G., Kelentrić, M. and Guðmundsdóttir, G. B., Professional
Lean Management, Resource Efficiency as Well as Management and
Digital Competence in Teacher Education. Nordic Journal of Digital Literacy 4.
Organization Improvement Skills. Procedia CIRP 17, 184-188. [9] Matt, D. T., Rauch, E. and Dallasega, P., 2014, Mini-factory – A
[29] Paulsen, M. F., 2009, Successful E-Learning in Small and Medium-
Learning Factory Concept for Students and Small and Medium Sized
Sized Enterprises. European Journal of Open, Distance and ELearning.
Enterprises. Procedia CIRP 17, 178-183. [10] Pittschellis, R., 2015, Multimedia Support for Learning Factories.
[30] 2001, Making an European Area of Lifelong Learning a Reality. European Commission.
Procedia CIRP 32, 36-40.
from:
http://lll-
Learning Factory (aLF) – Method, Implementation and Evaluation.
portal.eadtu.eu/images/files/Manual_Organising_EADTU
24-09-
Procedia CIRP 32, 13-18.
2010.pdf.
[11] Plorin, D., Jentsch, D., Hopf, H. and Müller, E., 2015, Advanced
[12] Rentzos, L., Doukas, M., Mavrikios, D., Mourtzis, D. and Chryssolouris, G., 2014, Integrating Manufacturing Education with Industrial Practice Using Teaching Factory Paradigm: A Construction Equipment Application. Procedia CIRP 17, 189-194. [13] Balve, P. and Albert, M., 2015, Project-based Learning in Production Engineering at the Heilbronn Learning Factory. Procedia CIRP 32, 104-108. [14] Blume, S., Madanchi, N., Böhme, S., Posselt, G., Thiede, S. and Herrmann, C., 2015, Die Lernfabrik – Research-based Learning for Sustainable Production Engineering. Procedia CIRP 32, 126-131. [15] Hart, J., 2014, Social Learning Handbook 2014, The Next Generation of Learning Practicesin the Age of Knowledge Sharing and Collaboration. Centre for Learning and Performance Technologies (C4LPT). [16] Hart, J., 2015, Modern Workplace Learning: a resource book for L&D. Centre for Learning & Performance Technologies,. [17] Kinchin, I., 2012, Avoiding technology-enhanced non-learning. British Journal of Educational Technology 43, 6. [18] Johnson, L., Adams, S., and Cummins, M., 2012, The NMC Horizon Report: 2012 Higher Education Edition. The New Media Consortium, Austin, Texas. [19] Johnson, L., Adams Becker, S., Estrada, V., Freeman, A., 2014, NMC Horizon Report: 2014 Higher Education Edition. The New Media Consortium, Austin, Texas. [20] Johnson, L., Adams Becker, S., Estrada, V., and Freeman, A., 2015, The NMC Horizon Report: 2015 Higher Education Edition. The New Media Consortium, Austin, Texas. [21] Mayer, R. E., 2010, Learning with technology, in: B. F. Dumont Hanna, Istance David (Ed.), Educational Research and Innovation The Nature of Learning: Using Research to Inspire Practice. OECD Publishing, p. 20. [22] Pouezevara, S., Mekhael, S. and Darcy, N., 2014, Planning and Evaluating ICT in Education Programs Using the Four Dimensions of
[31]
USBM.
2010
[cited
2015].
Available
[32] Wilen-Daugenti, T., 2012, Society 3.0: How Technology Is Reshaping Education, Work and Society. ERIC. [33] Cross, J., 2010, Where did the 80% come from?, Informal Learning Blog. Jay Cross / Internet Time Group, Informal Learning Blog. [34] Lave, J., Wenger, E., 1991, Situated learning: Legiti-mate peripheral participation. Cambridge University Press, Cambridge. [35] Brown, J. S., and Duguid, P, 1991, Organizational learning and communities of-practice: toward a unified view of working, learning, and innovation. Organization Science 2, 40-57. [36] Wenger, E., 1988, Communities of Practice; learning meaning and identity. Cambridge University Press., Cambridge. [37] Wenger, E., 2002, Cultivating communities of practice - A quick start-up guide, in: start_up_guide (Ed.). [38] Wenger, E., McDermott, R., & Snyder, W. M., 2002, Cultivating communities of practice: A guide to managing knowledge. . Harvard Business School Press., Boston. [39] Hart, J., 2011, Learning in a Social Organization, in: L. i. a. S. Organization
(Ed.).
Centre
for
Learning
and
Performance
Technologies (C4LPT). [40] Liker, J. K., 2004, The Toyota way : 14 management principles from the world's greatest manufacturer. McGraw-Hill, New York. [41] Pfeiffer, J. W., & Ballew, A. C. , 1988, Design Skills in Human Resource Development. Pfeiffer & Co, San Diego. [42] Tosterud, R., 2015, Simulation used as a learning approach in nursing education : Students’ experiences and validation of evaluation questionnaires. Karlstads universitet, Karlstad, p. 87. [43] Marstio, T. and Kivelä, S., 2014, Technology enhanced learning in a higher education context – Building bridges by student empowerment and regional development. Nordic Journal of Digital Literacy. [44] Bishop, J. L. and Verleger, M. A., 2013, The flipped classroom: A survey of the research, ASEE National Conference Proceedings, Atlanta, GA.
ScienceDirect Procedia CIRP 54 (2016) 101 – 106
6th CLF - 6th CIRP Conference on Learning Factories
Combining learning factories and ICT- based situated learning Nina Tvenge*, Kristian Martinsen, Sri Sudha Vijay Keshav Kolla NTNU, Faculty of Technolog, Economy and Management, Teknologivn 22 2815 Gjøvik, Norway * Corresponding author. Tel.: +47 40636548; E-mail address: [email protected]
Abstract This paper discusses the combination of learning factories and situated (on-the-job) ICT-based social learning, as well as the didactics of learning factories and ICT-based situated learning. Moreover, a case study at the Lean Lab - learning factory in Norway is described, with results from questionnaires sent participators/students about the learning outcome in the lifelong learning aspect. Three managers where asked to what extent their company benefited from the lean lab workshops. The paper suggests an enhancement of the learning factory with an online course before and after the lean lab workshop session. This can both make the simulations at the learning lab more effective as well as enhance the usefulness of the learning into positive change for the companies. © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license © 2016 The Authors. Published by Elsevier B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility the scientific committee of theConference 6th CIRPonConference on Learning Factories. under responsibility of the of scientifi c committee of the 6th CIRP Learning Factories Peer-review Keywords: Learning factories, manufacturing education, cyber physical systems, lifelong learning, online courses, lean
1 1.1
Introduction
research), or an effective competency development (if used for education and training)”[2].
Learning Factories
Both manufacturing and learning paradigms will be affected by digitalization in the future, with increasing possibilities for data collection, analysis, modeling, simulation, optimization and communication. Future manufacturing systems will be “Cyber Physical Systems” combining computational elements, physical elements, software and humans. Some call this trend the 4th industry revolution, or Industry 4.075. Similarly, technology is an ever more relevant and important factor in teaching and learning in general. There is, however, still a need for social and practical training and technology is not a substitute for this, but a range of different tools that can enhance learning and increase students’ learning space [1]. To meet manufacturing industries’ demand for knowledge and innovations in the age of Industry 4.0, learning factories are established in many educational and industry organisations [2-4]. The main goals of learning factories are either technological and/or organizational innovation (if used for
Figure 1. The Modern Workplace Learning Framework [5]
2212-8271 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the scientific committee of the 6th CIRP Conference on Learning Factories doi:10.1016/j.procir.2016.03.031
102
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
Didactic methods and mind-sets must be a part of a learning factory, as it is for all learning processes, but review shows that there is still a lack of research conducted on learning and teaching methods in these learning facilities. Consortiums have explored and decided on the morphology, and the focus is on practice-oriented learning processes [2, 6-12], but the effects on learning outcome and best didactical approaches are not well mapped yet, although there was increased focus in 2015[13, 14]. And one can also debate whether the learning factories are focusing too much on efficiency, as in reducing production costs, rather than human needs and demands [8]. Learning factories have a natural place in future lifelong learning, but has potential for being a more efficient learning approach if being a part of a broader educational context. One example is the modern workplace learning framework (Figure 1) [15, 16, 5] The framework is based on, amongst others, analyzes of changes in top 100 learning tools during the last nine years. One conclusion from this work was that learning on the web is different from traditional learning in many ways as it is: • • • • • • • • •
continuous on demand takes place in short bursts takes place on the go and in the flow of work social not designed serendipitous autonomous performance-oriented
4000 respondents were asked what type of activities give most value at work, and activities that are autonomous, as in self-organized by individuals and teams and social, meaning learning with and from one another in the workplace were highest rated. Conventional learning factories could be used in all these different practices together with ICT - based workplace learning. 1.2
ICT based situated learning
ICT has a natural place in today’s education and knowledge creation, but the implementation has far from satisfied the involved participants [17]. The ability to collaborate is highly acknowledged and wanted by employers, and teamwork and communication must be facilitated in the future work places [18]. “The 2012 NMC Horizon Report” say that it is a normal expectation that humans will be able to work, learn and study independent of time and space. And the 2014 report states that: “the “social” of learning will emerge even more and that ”collaborative learning” actually is decisive in order to succeed in education in the future.” These trends are continuing in the 2015 report as well [18-20]. ICT-based learning has gone from being closed off and individual focused to being a social learning arena where sharing is essential and the learner’s needs are at the center, not the technology [21]. There are versatile expectations
when it comes to implementation and use of ICT in lifelong learning; • • • • •
increased learning as an effect of access to more data and knowledge more efficient learning learner focused learning activities new learning environments with higher degree of collaboration/cooperation more opportunities for critical thinking and analytical approaches
Other aspects are increased learner motivation /-productivity and autonomy [22]. Services through web 2.0 and web 3.0, also called semantic web, make it possible to share infinite amounts of multi-medial learning resources. More effective and efficient learning processes can be obtained by utilizing ICT, but not without support, reviews show that learning activities as social interactions guided by a teacher, has had the greatest impact on learning outcome, significantly bigger than other methods [23]. Another significant finding is that the more interaction and working with curriculum, the more you learn [24, 25], a mechanism ICT-supported learning can enhance when implementation is well planned. Development of ICT-supported learning is closely linked to the available technology in time, and its affordances, rather than looking at educational research and needs [25]. A Norwegian monitor [26, 27] on use of ICT in teaching and learning in higher education shows that there is a slight increase in usage of technology in education, but there is no increase in well-planned didactical teaching/learning resources. In 2010 it was stated that “in Norway there is a one-sided focus on tool competency and a random approach: a “try and fail”-strategy when it comes to educational technology” And there is not much evidence on this having changed[28]. A review of “all” research on ICT and learning since the millennium [25] found that collaborative learning is efficient if the participants engage at a more advanced taxonomic level. Collaborative learning can lead to a deeper form of learning; ability of critical thinking; understanding, decision making and even longer memory spans. Research has then seemingly started to focus on how to control the learning processes in order to achieve wanted results/ effects [25]. A growing number of social networks and other web 2.0 and 3.0 services can be used for flexible and informal learning and provide access to experts and peers. Workers can thus set up their personal learning environments (PLEs) according to their interests, learning styles and ambitions. This is both an opportunity and a challenge for the individual learner. Large enterprises have the power to develop internal personal learning environments; SME’s have to utilize more or less ready-made solutions [29]. This does not need to be looked upon as a negative; it can give room for more open learning processes. Open educational resources (OER) are freely accessible documents and media resources for teaching, learning, education, assessment and research purposes.
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
1.3
Lifelong learning
How can the future, competitive, sustainable industry attract and support competent and engaged employees that partake in improvement and growth processes? Learning is a basic human need, and lifelong learning is a key element for a good work life. Lifelong learning is defined by the European Commission as "all learning activity undertaken throughout life, with the aim of improving knowledge, skills and competence, within a personal, civic, social and/or employment-related perspective” [30]. According to the USBM consortium [31] lifelong learning is not widely implemented yet, but embraced all over Europe. And "…Workers must strengthen their critical thinking, embrace technology, and become lifelong learners…". “Learning is not only a human activity that is escaping from the classroom but also one that is being recognized as happening more in sites such as the workplace and home where it has always taken place”[32]. Formal learning plays a minor part in the networked workplace, a norm is that about 80% of workplace learning is informal [33]. Informal learning is a broad expression, which describes a “wide range of new approaches to workplace learning…”[16]. Both informal and formal learning are important in the cycle of lifelong learning. Situated learning, i.e. learning in the particular environment where it also is going to be applied, first presented by Lave and Wenger [34] in Communities of Practice. CoP is defined as “tightly-knit groups that have been practicing long enough to have developed into a cohesive community, which provides a sense of belonging, commitment, and shared identity”[35, 34, 36-38]. The idea behind CoP is that the organization or business must organize the workplace so that the employees can learn, act and make decisions based on their own competencies [38]. Hart [39, 15, 16, 5] emphasizes the importance of supporting informal learning rather than “managing” it. 2 2.1
both master- and bachelor levels. The groups can be unified all-from–one-organization, or it can be a mix of different organizations. Each team will be in charge of equal simulated manufacturing line making a wooden house mockup product. The lines resemble a real manufacturing line including a shop floor data collection and visualization (Andon) system. The lean tools such as JIT, Andon, Hijunka, PokaYoke and 5S [40] are important aspects, but the main focus is on productivity, safety and quality. For each team there is a dedicated supervisor from Lean Lab who facilitates the learning process. There is an element of competition between the teams, but also collaboration within and between the teams.
Figure 2. Lean Lab - a full scale simulator
The simulation starts with an introduction to lean and lean tools, and an introduction to the simulator. Each student will have a dedicated workplace throughout the simulations, doing transportation tasks, assembly and quality assurance. Between each round there is time for reflection and planning of improvements that are applied in the next round. The Lean Lab follows thus the experimental learning cycle as described by Pfeiffer et al. [41]
Case study: Lean Lab Norge AS Description of Lean Lab Norge
Lean Lab Norge AS is a learning factory focusing on teaching lean principles through a full-scale simulator. Lean Lab is situated at the Raufoss industry park and was founded in 2009. The owners are a number of industry partners, SINTEF and NTNU. The learning factory falls into Abele et al’s [2] “Industrial application scenario” but it could be argued to be partially in the “Consultancy application scenario” due to the heavy involvement of SINTEF. Lean lab would mainly be placed as “Company funds, single events, course fees, education, manufacturing, cognitive, closed scenario, brownfield, onsite learning” in the described morphology (ibid). Students/participants are working in three or four teams doing one-day simulations at Lean Lab. Most of the groups attending are employees in manufacturing industry, but there are examples of groups from healthcare, public service and other sectors. NTNU is using the Lean Lab for its campus students as high-fidelity training simulations on
Figure 3. Experimental Learning Cycle [41]. Table 1 shows how Lean Lab workshops or courses are set up according to the learning cycle stages from Pfeiffer’s model. In the first round the work instructions are pre-fixed and non-optimum. The teams will typically fail to reach the targets because of unbalance, waiting time, lack of inventory and defects. After first round each team reflect on failures, bottlenecks etc. of the previous round through a dialogue in the group including the supervisor who facilitate the discussion; a guided social interaction [23], giving hints about the lean tools, but the purpose is that the teams come up with clever ideas themselves. Based on their reflections, the teams will rewrite and apply revised work
103
104
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
instructions. This is repeated between the second and the third round where the supervisors also encourage discussions between the different teams. After the end of round three there will be time for reflections and a summary of the achievements during the day. Table 1: Stages of the Lean Lab simulation
Stage Orientation Clarification Experience
Reflection
Generalization Application
2.2
Lean Lab operations Theoretical overview on lean Information about the workshop, rules and procedures. Agenda and overall tasks. Three different simulation runs, 1st run wit pre-set work instructions, 2nd and 3rd after reflection, generalization and adaption of improvements. Analyses of the 1st (and 2nd round), detect bottlenecks, cause of failure etc., and discuss improvement. Discussions within the group and between the groups. Produce revised work instructions Application of improved instructions in next round (2nd & 3rd)
Learning outcome assessmentß
At the end of a Lean Lab - workshop, the students/ participants answer a short questionnaire on their satisfaction. The authors of this paper wished, however, to investigate if the Lean Lab workshops have contributed to changes in the organization: To what extent has the learning factory experience resulted in improvements and change in the organization of the participants? And is the Lean Lab workshop only a “happening” with personal learning outcome, but difficult to convert to practical actions and implementation afterwards in the company? Three managers in companies who had groups of employees on a lean lab workshop in the past where questioned on the long-term results. The selected managers were: • • •
The CEO of a large company manufacturing electronics (circuit boards) The Vice President of supply chains in a large company manufacturing maritime electronics The director of operations in a medium size company manufacturing products to the defense sector.
Typical questions were on to what extent the workshop has resulted in implemented/ improved lean principles in the company, if manufacturing and/or administrative processes has been improved, successful standardization, reduction of lead times, improved productivity and quality. All three managers where positive to the outcome of the Lean Lab workshop, and believed the workshop had contributed to improvements in the company. Establishment
of continuous improvement culture, less sick leave, improvements on safety, quality and productivity (cutting cost) were mentioned as effects. The authors also made a questionnaire based survey among students attending the workshop, where thirteen of 17 students responded. The questionnaire asked for the learning outcome some time after the workshop, and the responses where mainly very positive. A typical answer from the students was that the Lean Lab workshop gave them a better overview and a framework on the lean paradigm, useful as later reference. On the statement “group activities during the workshop enhanced my personal learning”, all except one, agreed. “The workshop offered diverse ways of assessing my learning” gave, on the other hand, more mixed answers as half of the students disagreed or were not satisfied. A couple of comments from the students exemplifying their perceived learning outcome “I compared my understanding of Lean principles before and after workshop. After the workshop I am able to understand the importance of Lean principles in practice” and “It helped me to understand that group work is more important and step-by-step process can help us to do work more effectively”. 2.3
Discussion of the case results
The outcome of the interviews and survey on the (perceived) learning outcomes shows a general satisfaction to the Lean Lab learning outcome. This is the case for the companies investing in one day at Lean Lab (according to the managers questioned), as well as the individual students. The individual perception of the learning outcome and the mainly positive outcome can have several reasons (in addition to that they did really learn a lot): It was a social event, with aspects of (friendly) competition, a sense of achievement. While campus students felt a sense of “real world manufacturing” in spite of being a simulation, this was not the case for the lifelong learning students, as they came from industry or other workplaces. Still they where positive to the simulation and accepted this as a valuable learning experience. The students gave more mixed response to the question about “diverse ways of assessing my learning”, “enough opportunities [..] to find out if I clearly understand lean principles” and ”opportunity to discuss the lean principles with colleagues”. In other words, some of the students did feel the learning was too unidirectional and with less opportunities to discuss and get feedback. The main reason for this is probably the one-dayonly time constraint as well as the necessary rigidity in the direction of the simulation. Moreover can, of course, this type of simulation not cover all parts of the lean paradigm, and aspects such as lean organization/leadership, flexibility/fast changeovers etc. where not covered. Regarding the effect on the improvement in the attending companies, it should be treated with care. It is difficult to distinguish the effect of the workshop from the combined effects of all improvements measures in the company. The decision to attend the Lean Lab workshop is usually not isolated, but a part of a lean improvement program. The
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106
workshop is, however, acknowledged to be a valuable method for learning of lean principles. 3
Combining learning factories and online courses
As described, both students attending the Lean Lab and the companies choosing to use the Lean Lab, had predominantly a positive impression of the learning outcome. The social aspect of learning and the way the simulation connects theory with practice and learning-by-doing are probably some of the main reasons for the success. A literature review on “learning factories” and evaluation of “learning outcome”, where the purpose was to find to what extend there are studies made on learning effects of the morphologies and frameworks that are developed for learning factories, show that there is a lack of research on this, with some exceptions [13]. The didactic “what’s” and also “how’s” of learning processes are work well in progress in learning factories, but knowledge on why we do it exactly like this can still be explored more, and most essential; the effect on students’ learning outcome of our teaching methods in learning factories, needs to be further explored. We know what we want to teach and how to do it, but we don’t really know whether these methods are the best or the most efficient. Recent studies of simulation in nursing education show that students do learn and that the theory-practice gap is closing up. And high-fidelity simulations create enthusiasm and inspiration, which are important features in good learning environments [42]. These are mechanisms we believe are recognized in manufacturing simulations as well. But learning is rarely a one-day happening, learning theories show, on the other hand, that learning often needs repetition and maturation and a one-day-only workshop has its pitfalls in this perspective. Marstio and Kivelä found that learning environments that include “social networking services offers an open learning environment supporting both formal learning in educational institutions as well as informal lifelong learning in SMEs”[43]. The authors argue thus that a combination of an online course in combination with the workshop is likely to enhance the learning outcomes, both for the individuals and the companies. This combination would also allow a less unidirectional learning, cover more aspects on lean, give the students more feedback and possibility do discuss and reflect. In short: it can be a way to improve on those areas where the students’ responses were more mixed.
! !
%
%!# %
! !
!
"
Figure 4 shows a possible structure of a combined course. The idea is to combine on-site online learning with the learning factory workshop. Prior to the workshop the students can view recorded lectures, attend online tutoring (synchronous or asynchronous through a blog/forum). The main idea is to prepare the students theoretical background for the workshop. Students will be in different levels and through selection of the lectures etc. can all students be more on level before the workshop. Some of the later lectures can focus on practical issues of the forthcoming workshop to prepare the students. Prior to the workshop there could be a multiple-choice online test or similar, and it could be a prerequisite for attending the workshop that this test is passed. The main reason for this test is to ensue that students are prepared to the workshop. At the workshop day, the before mentioned orientation and clarification can be shortened and just a quick repetition, using for instance student response - systems (SRS) like Kahoot or Socrative, and then reduce the one-tomany teaching and rather focus on potential collaborative social learning aspects of the simulation rounds. This is a somewhat similar approach as the flipped classroom approach [44]. After the workshop the course can continue with reflections and discussions. We propose to challenge the students with a mini-project based in the needs of their own workplace. The idea is lead the students to reflect on how the knowledge gained through the first steps of the course and the workshop can be applied in their own companies. 4
Conclusions
This paper describes a case study of the Lean Lab Norge learning factory at Raufoss industrial park in Norway. The perceived learning outcome for the participants was assessed through discussions and questionnaires. The paper suggest a combination of the social synchronous learning in the Lean lab simulator and an asynchronous online course, where students can both prepare for the simulation beforehand, as well as discus and reflect afterwards. 5
Acknowledgements
The authors wish to thank the Lean Lab Norge for giving access to the simulator and data on participation. Moreover, the authors wish to thank Norwegian Research Council for funding this work through the SFI Manufacturing and Norgesuniversitetet for their support. References [1] Brown, M., 2005, Learning Spaces, in: D. G. Oblinger and J. L. Oblinger (Eds.), Educating the Net Gen,. Boulder, Colorado, p. 21. [2] Abele, E., Metternich, J., Tisch, M., Chryssolouris, G., Sihn, W., ElMaraghy, H., Hummel, V. and Ranz, F., 2015, Learning Factories for Research, Education, and Training. Procedia CIRP 32, 1-6.
Figure 4. Possible structure of the combined online course/learning factory workshop.
[3] Faller, C., Feldmúller, D., 2015, Industry 4.0 learning factory for regional SMEs. Procedia CIRP 32, 88-91.
105
106
Nina Tvenge et al. / Procedia CIRP 54 (2016) 101 – 106 Sustainability: A Program Evaluation from Egypt. International
[4] Seitz, K.-F., Nyhuis, P., 2015, Cyber-physical production systems
Journal of Education and Development using ICT 10, 120-141.
combined with logistic models-a learning factory concept for an improved production planning and control. Procedia CIRP 32, 92-97.
[23] Mincu, M. E., 2015, Teacher quality and school improvement: what is the role of research? Oxford Review of Education 41, 253-269.
[5] Hart, J., 2016, 2016: Rethinking workplace learning, Learning in the
[24] Bonesrønning, H. and Opstad, L., 2012, How Much is Students’
Modern Workplace.
College Performance Affected by Quantity of Study? International
[6] Hummel, V., Hyra, K., Ranz, F. and Schuhmacher, J., 2015,
Review of Economics Education 11, 46-63.
Competence Development for the Holistic Design of Collaborative Work Systems in the Logistics Learning Factory. Procedia CIRP 32,
[25] Wasson, B. M., K., 2014, Information and Communications Technology and Learning: State of the Field Review. Knowledge
76-81.
Center for Education., Oslo.
[7] Kim, Y. S., Lee, J. O. and Park, C. W., 2011, A hybrid learning system proposal for PLC wiring training using AR, Proceedings - 2011 5th IEEE
International
Conference
on
E-Learning
in
[26]
Electronics, ICELIE 2011, pp. 90-94.
2011,
Digital
Tilstand,
in:
R.
Fossland,
Gjerdrum
(Ed.),
Norgesuniversitetets skriftserie. Norgesuniversitetet, Tromsø.
Industrial
[27] 2015, Digital tilstand 2014, in: H. Ørnes (Ed.), Norgesuniversitetets skriftserie. Norgesuniversitetet, Tromsø, p. 226.
[8] Kreimeier, D., Morlock, F., Prinz, C., Krückhans, B., Bakir, D. C. and Meier, H., 2014, Holistic Learning Factories – A Concept to Train
[28] Ottestad, G., Kelentrić, M. and Guðmundsdóttir, G. B., Professional
Lean Management, Resource Efficiency as Well as Management and
Digital Competence in Teacher Education. Nordic Journal of Digital Literacy 4.
Organization Improvement Skills. Procedia CIRP 17, 184-188. [9] Matt, D. T., Rauch, E. and Dallasega, P., 2014, Mini-factory – A
[29] Paulsen, M. F., 2009, Successful E-Learning in Small and Medium-
Learning Factory Concept for Students and Small and Medium Sized
Sized Enterprises. European Journal of Open, Distance and ELearning.
Enterprises. Procedia CIRP 17, 178-183. [10] Pittschellis, R., 2015, Multimedia Support for Learning Factories.
[30] 2001, Making an European Area of Lifelong Learning a Reality. European Commission.
Procedia CIRP 32, 36-40.
from:
http://lll-
Learning Factory (aLF) – Method, Implementation and Evaluation.
portal.eadtu.eu/images/files/Manual_Organising_EADTU
24-09-
Procedia CIRP 32, 13-18.
2010.pdf.
[11] Plorin, D., Jentsch, D., Hopf, H. and Müller, E., 2015, Advanced
[12] Rentzos, L., Doukas, M., Mavrikios, D., Mourtzis, D. and Chryssolouris, G., 2014, Integrating Manufacturing Education with Industrial Practice Using Teaching Factory Paradigm: A Construction Equipment Application. Procedia CIRP 17, 189-194. [13] Balve, P. and Albert, M., 2015, Project-based Learning in Production Engineering at the Heilbronn Learning Factory. Procedia CIRP 32, 104-108. [14] Blume, S., Madanchi, N., Böhme, S., Posselt, G., Thiede, S. and Herrmann, C., 2015, Die Lernfabrik – Research-based Learning for Sustainable Production Engineering. Procedia CIRP 32, 126-131. [15] Hart, J., 2014, Social Learning Handbook 2014, The Next Generation of Learning Practicesin the Age of Knowledge Sharing and Collaboration. Centre for Learning and Performance Technologies (C4LPT). [16] Hart, J., 2015, Modern Workplace Learning: a resource book for L&D. Centre for Learning & Performance Technologies,. [17] Kinchin, I., 2012, Avoiding technology-enhanced non-learning. British Journal of Educational Technology 43, 6. [18] Johnson, L., Adams, S., and Cummins, M., 2012, The NMC Horizon Report: 2012 Higher Education Edition. The New Media Consortium, Austin, Texas. [19] Johnson, L., Adams Becker, S., Estrada, V., Freeman, A., 2014, NMC Horizon Report: 2014 Higher Education Edition. The New Media Consortium, Austin, Texas. [20] Johnson, L., Adams Becker, S., Estrada, V., and Freeman, A., 2015, The NMC Horizon Report: 2015 Higher Education Edition. The New Media Consortium, Austin, Texas. [21] Mayer, R. E., 2010, Learning with technology, in: B. F. Dumont Hanna, Istance David (Ed.), Educational Research and Innovation The Nature of Learning: Using Research to Inspire Practice. OECD Publishing, p. 20. [22] Pouezevara, S., Mekhael, S. and Darcy, N., 2014, Planning and Evaluating ICT in Education Programs Using the Four Dimensions of
[31]
USBM.
2010
[cited
2015].
Available
[32] Wilen-Daugenti, T., 2012, Society 3.0: How Technology Is Reshaping Education, Work and Society. ERIC. [33] Cross, J., 2010, Where did the 80% come from?, Informal Learning Blog. Jay Cross / Internet Time Group, Informal Learning Blog. [34] Lave, J., Wenger, E., 1991, Situated learning: Legiti-mate peripheral participation. Cambridge University Press, Cambridge. [35] Brown, J. S., and Duguid, P, 1991, Organizational learning and communities of-practice: toward a unified view of working, learning, and innovation. Organization Science 2, 40-57. [36] Wenger, E., 1988, Communities of Practice; learning meaning and identity. Cambridge University Press., Cambridge. [37] Wenger, E., 2002, Cultivating communities of practice - A quick start-up guide, in: start_up_guide (Ed.). [38] Wenger, E., McDermott, R., & Snyder, W. M., 2002, Cultivating communities of practice: A guide to managing knowledge. . Harvard Business School Press., Boston. [39] Hart, J., 2011, Learning in a Social Organization, in: L. i. a. S. Organization
(Ed.).
Centre
for
Learning
and
Performance
Technologies (C4LPT). [40] Liker, J. K., 2004, The Toyota way : 14 management principles from the world's greatest manufacturer. McGraw-Hill, New York. [41] Pfeiffer, J. W., & Ballew, A. C. , 1988, Design Skills in Human Resource Development. Pfeiffer & Co, San Diego. [42] Tosterud, R., 2015, Simulation used as a learning approach in nursing education : Students’ experiences and validation of evaluation questionnaires. Karlstads universitet, Karlstad, p. 87. [43] Marstio, T. and Kivelä, S., 2014, Technology enhanced learning in a higher education context – Building bridges by student empowerment and regional development. Nordic Journal of Digital Literacy. [44] Bishop, J. L. and Verleger, M. A., 2013, The flipped classroom: A survey of the research, ASEE National Conference Proceedings, Atlanta, GA.