- Email: [email protected]
An Innovative Technology: Augmented Reality Based Information Systems
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Available online at www.sciencedirect.com Procedia Computer Science 00 (2019) 000–000
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 158 (2019) 407–414
3rd World Conference on Technology, Innovation and Entrepreneurship (WOCTINE) 3rd World Conference on Technology, Innovation and Entrepreneurship (WOCTINE)
An Innovative Technology: Augmented Reality Based Information An Innovative Technology: Augmented Reality Based Information Systems Systems a a
Damla Aslanaa, Bengü Bayyurt Çetinaa, İzzet Gökhan Özbilginaa * Damla Aslan , Bengü Bayyurt Çetin , İzzet Gökhan Özbilgin *
HAVELSAN Inc., Mustafa Kemal Mahallesi 2120 Cad. No:39 Çankaya, Ankara Postcode: 06510, Turkey HAVELSAN Inc., Mustafa Kemal Mahallesi 2120 Cad. No:39 Çankaya, Ankara Postcode: 06510, Turkey
Abstract Abstract In our generation the information systems evolve with new technologies: augmented reality (AR), IoT, artificial intelligence, In our generation the information systems evolveexchange with newbytechnologies: augmentedthat reality (AR), will IoT, be artificial blockchain etc. Anymore they perform information sensors. It is estimated the systems in a stateintelligence, of extreme blockchain etc. Anymore they perform It isknow estimated that the systems willusbewill in abe state of extreme interaction and reach 50 billion devicesinformation connected exchange in Internetbyinsensors. 2020. We that everything around in interaction interaction and 50 billion devices connected in Internet in 2020. know that aroundis us will be instart interaction and they will doreach everything without any need of human interference. ForWe example, wheneverything our dishwasher full, it will to wash and they will dooreverything without of human interference. Forthe example, our full, it will start totowash automatically, when the run out ofany theneed gasoline, our car will drive to nearest when station, ordishwasher even when is a burglar is entered our automatically, or when the run out of theand gasoline, our car to will to the nearest station, life, or even when a burglar is automatical entered to our house, it will automatically be detected be announced thedrive police office. In business the processes will be in house, it will automatically be detected be announced to theand police office. InNext business life, the processesitwill be automatical in maximum level and this technology willand increase productivity efficiency. to mobile technology, is thought that these maximum level information and this technology efficiency. NextAR to also mobile it istothought that these new generation systems will (IS) increase will takeproductivity the biggest and place in our lives. willtechnology, be integrated these systems to new generation information systems will takewill theaugment biggest place in our AR alsoway will thanks be integrated these systems to augment the information in real world.(IS) Humanity its habitat in lives. an innovative to thesetoAR based IS. This augment the information real world. Humanity will related augment its habitat in an way thanks to these AR based This paper surveys the currentin state-of-the-art AR systems with aerospace & innovative defense, industry, education, medical and IS. gaming paper surveys the currentofstate-of-the-art AR innovation systems related with aerospace & defense, industry, and gaming sectors. The connection AR based IS and is explained with a technological insight. education, In additionmedical to international use sectors. The connection AR are based and innovation is explained a technological In addition to international use cases HAVELSAN’s useofcases alsoISgiven that are performed fromwith the aspect of applied insight. open innovation strategy. This strategy cases HAVELSAN’s use are also given that are performed from the aspect of applied open innovation strategy. This strategy is addressed specific to thecases implemented activities of AR based IS. is addressed specific to the implemented activities of AR based IS. © 2019 The Author(s). Published by Elsevier B.V. © 2019 2019 The The Authors. Published Elsevier B.V. © Author(s). Publishedbyby B.V. committee of the 3rd World Conference on Technology, Innovation and Peer-review under responsibility of Elsevier the scientific Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Entrepreneurship Entrepreneurship Entrepreneurship Keywords: new generation information systems, augmented reality, open innovation strategy, technology innovation Keywords: new generation information systems, augmented reality, open innovation strategy, technology innovation
* Corresponding author. Tel.: +90-312-219-5787; fax: +90-312-219-5797. E-mail address:author. [email protected] * Corresponding Tel.: +90-312-219-5787; fax: +90-312-219-5797. E-mail address: [email protected] 1877-0509 © 2019 The Author(s). Published by Elsevier B.V. 1877-0509 2019responsibility The Author(s). Published Elsevier B.V. Peer-review©under of the scientificbycommittee of the 3rd World Conference on Technology, Innovation and Entrepreneurship Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Entrepreneurship
1877-0509 © 2019 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Entrepreneurship 10.1016/j.procs.2019.09.069
408 2
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
1. Introduction Information systems (IS) present refined data and information to the executives to support their decision-making. They organize all the process data in the organizations. With the evolution of the augmented reality technology the construction and visualization of the business processes are easier than ever happened. Therefore, considering this situation as a technology innovation is not exaggeratedness. The nature of the innovation is required a lasting impression in market or society [1]. The reason of why we mention that augmented reality (AR) is a technology innovation is its providing innovation in information technology (IT) area. AR is a revolutionary approach that it is augmenting our world and our daily life with a new cognition and interaction aspect. It might be stated as sixth sense, which is a new layer to understand the events happened around us. As a commercial defense, simulation and ICT company developing software intensive systems with 1800 employees HAVELSAN is applying open innovation policy to develop and present innovative and pioneer AR based IS solutions to the global market. The rest of the paper is organized as follows: the general definition of AR and the HAVELSAN’s main innovation policy are given in first section, the literature review about AR is given in second section, the international use cases of AR based information systems are provided in third section, and finally the evaluation of HAVELSAN’s innovation strategy related with AR based IS is presented in fourth section as conclusion. 1.1. Definition of AR Augmented reality is to add some contents produced by digital devices into the real-world environment. These contents can be live or indirect physical appearances that are enriched with sound, image, graphics or GPS data [2]. This is a technology to expand someone’s reality. It gives an immersive information environment and also interactivity with the surrounding digital area. Its basic mechanism works with the steps: 1- capture and coordinate data digitally, 2- track the reference points of the data, 3- render virtual information with the combined real environment and display. The various types of data can be captured from different types of sensors such as depth, camera, gyroscope, altimeter, barometer, accelerator, pressure, proximity, compass, IMU (Inertial Measurement Unit), GPS (Global Positioning System) receiver, microphone [2]. The networking for the exchange of the data/information can be provided by RFID (Radio Frequency Identification), Bluetooth or Wi-Fi. 1.2. Innovation Policy of HAVELSAN company HAVELSAN has a leading role in meeting the software based strategic needs of our country with national solutions. To achieve this goal, an open innovation strategy is adopted and implemented. This strategy provides to benefit from not only internal ideas, but also external ideas [3]. The ideas are evaluated and matured with a model that includes the internal and external sources [4]. Lots of stakeholders are incorporated into the innovation processes [5, 6]. In this scope, a five-year Technology Roadmap has been prepared for planning the advanced technology development activities. Over 50 strategic R&D projects planned in this Roadmap for technological development in areas such as augmented/mixed reality, artificial intelligence, learning algorithms, cloud computing, big data, autonomous systems and quantum computing. In addition to the strategic R&D activities, an idea assessment system has been initiated that it is a process from idea to the product. This system is still in progress at the Informatics Innovation Center CoZone HAVELSAN Office, in METU Technopolis. HAVELSAN’s Open Innovation Center is the first open innovation center in the defense industry. Within the scope of open innovation activities, technology-oriented solution competitions and hackathons are organized throughout the year for entrepreneurs, students and technology enthusiasts. HAVELSAN also gives great importance to in-house innovation studies. Innovative ideas and studies on future technologies are evaluated in in-house innovative idea campaigns and competitions. Hundreds of ideas are submitted via the innovation management portal by internal and external stakeholders. Those who succeeded start to work at HAVELSAN’s Open Innovation Center. In the center, joint projects were carried out with universities, technology transfer offices and incubation centers and various trainings are given.
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
409 3
HAVELSAN aims to develop global products that will compete in the international arena and considers this development as a national duty beyond its commercial target. To be able to implement innovation and R&D studies gives great importance to productization in order to obtain commercial results. Since the efficiency of the product is dependent on the fact that HAVELSAN's R&D and innovation ecosystem’s abilities, strategic cooperation activities have been initiated with academy to compete with the ecosystems of international companies. 2. Literature Review Sutherland [7] has initiated the AR researches with the fund of DARPA in 1960. He has presented the term of interactive computer graphics. Azuma et al [8]'s article results are based on Azuma's published complementary survey related with AR. The visualization and tracking techniques are given with their problems. Several application examples are provided from different areas such as defense, sports. The hardware and device shortcomings are represented as the technological challenges. The user interface is also given as the other challenge, it is stated that the interface should be designed by taking into account user experiences. The third challenge is stated as social acceptance, to persuade people to wear a device. Krevelen [9] explains the display, tracking and modeling methods technically and gives their limitations. He also discusses AR applications by grouping as personal information systems, industry and military applications, medical applications, AR for entertainment, AR for the office and education training. The limitations of AR technology are listed as portability and outdoor use, tracking and auto calibration, latency, social acceptance, adaptation, depth perception, eyestrain and overload. Mekni et al [10] research available AR applications based on 12 distinct categories such as medical, military, education, robotics. They point out ergonomic hardware, adequate battery life and obtaining accurate information requirements as the challenges. Livingston et al [11] presents a detailed literature review of military AR projects carried out from 1960s to 2011. The importance of situational awareness, user interface (UI) and BARS (Battlefield Augmented Reality System), created for dismounted warfighter, are mentioned. The technological hardware limitations are given as the challenges. Bermejo et al [12] clarifies various augmented reality applications used in different sectors such as tourism, retail, health care and public services. They define the combining of big data and augmented reality (AR) as a must. The visualization of big data is considered as a new business strategy that it provides improvements by analyzing the enormous data driven from the penetration of mobile device, IoT (internet of things) and social media while performing the organization activities. Olshannikova et al [13] emphasize the importance of the integration of AR and virtual reality (VR) with big data because of the limitations of the human perception. These visualization methods are strictly recommended for new generation systems. The challenges faced with applying them are given at the end of the paper. In spite of these difficulties, the integration of big data and mixed reality (MR) approach promises an immersive user experience through tangible and verbal interaction. Harborth [14] has performed a comprehensive literature review about AR technology for IS. He highlights that AR is an area, which is researched for almost 30 years. It is stated that user behavior should remain under investigation, especially for gaming and smartphone browsers. Miller [15] provides the benefits of new technologies for aerospace & defense industry. These new technologies include augmented reality, blockchain, 3D printing, artificial intelligence, IoT, cyber security etc. He concludes that digitalizing business operations and building a value chain ecosystem are transforming the products with an innovative way. Piedimonte et al [16] present several case studies, which are applied in Italian Air Forces. The utility of augmented reality for C4I systems is provided more promising results than traditional ones. Maj [2] specifies that the application of augmented reality as a new technology increase the combat effectiveness. He explains the benefits and challenges of the use of AR systems in detailed by U.S. military. At the tactical level the locations of the enemy targets and friendly forces are displayed for situational awareness by AR. Especially, AR based systems enhance human cognition by representing the complex information into mental understanding. The challenges are user distraction, user adaption, sensor range and connectivity. At the operational level the requirements are
410 4
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
situational awareness and understanding and a directing and leading plan visualization in a shared vision. The concerns are anti-technology culture, dependency to the system for decision-making and the usability of micromanagement. He recommends holistic data management planning, design of intuitive and rugged systems, and new regulations for military occupations for both tactical and operational levels. Ilic et al [17] discuss why AR is essential for IoT systems. It is noticed that AR enables data collection from smartphones and giving information as output, additionally it is convenient for the integration of and the shared interaction with the new sensors and digital biomarkers. 3. Use Cases 3.1. International Use Cases 3.1.1. Aerospace & Defense The AR ensures digital transformation in battlefield. Its integration with C2 and C4ISR systems provide advantage at the tactical and the operational levels [16, 18, 19]. Situational awareness capability that is crucial for commanders and soldiers during a battle can be raised to the higher level thanks to continuous access and rapid dissemination and exchange of the information. The information is generated by collecting from various sensors, cameras and databases such as cameras on command center, troops in the field or on the UAVs, biometric and emotional sensors on soldiers, big data obtained from the historical events. The response times of deploying forces and providing support can be shortened by utilizing AR based IS. The innovative use cases of aerospace and defense can be grouped as below [2, 10, 11, 20, 21]; • Piloting systems: The cockpit applications are starting with Heads-Up-Displays (HUDs) which display digital data within the user’s field-of-view (FOV) from 1960s. The origin of the super cockpit buzzword is based on 1980s. The pilots have sighted the flight data with an augmented manner by their see-through devices. ARVCOP (AR for the Common Operational Picture) project that has been supported by the Office of Naval Research includes the development of the ability to navigate pilots to find the tracked vessels. F-35 Joint Strike Fighter (JSF) Helmet Mounted Display System (HMDS) has been designed and produced. Canada’s Institute for Aerospace Research (NRC-IAR) has developed the helicopter night vision system. • Remote control systems: Ground vehicles weapon control systems have been developed by SAAB. • Vehicle crew systems: DVES (Driver Vision Enhancement System), VCS (Vehicle Camera System), LSAS (Local Situational Awareness System) have been developed for the drivers and the crew members. BAE Systems has designed the similar of the F-35 JSF HMDS for land vehicles. DARPA has developed ULTRA-VIS and ARA’s ARC 4 with Lockheed Martin, Microvision and BAE Systems. Currently, U.S. Army has awarded a contract with Microsoft Hololens headset for Integrated Visual Augmentation System. In addition, smart weapons, which are guided with AR are another issue included in U.S. Army’s agenda. • Battlefield management systems: Common Operating Picture (COP) solutions provide real-time reporting and geographical information for combat forces by gathering data from the operational functions. Also, navigational information is presented by the integration of AR and GPS based mapping. Arcane company is one of the animated terrain presenters. HUNTR (Heads-Up Navigation, Tracking, and Reporting) system has being developed for tactical situational awareness purposes. This system provides assisting information such as the building names, streets, historical details, etc. with AR technology. • Robotics: NASA has being used Microsoft Hololens to explore the Mars surface before the mission that will be performed in 2020 by Rover team. • Technical guidance: It includes the systems both real-time remote assistance and virtual displaying of the technical instructions. Lockheed Martin, BAE Systems, U.S. Army and Naval engineers are some of the practitioners of this technology. • Training: Navy Research Lab, Marine Corps are using AR as a training aid to teach complex combat tasks. Battlefield Augmented Reality System (BARS) project is initiated to train the soldiers for combat scenarios and simulate real-time enemy behavior.
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
411 5
3.1.2. Industry The innovative use cases of industry can be grouped as below [2, 10, 21, 22, 23, 24, 25, 26, 27]; • Design and implementation: The manufacturing steps are efficiently designed and implemented by AR technology. BMW, SAAB, BAE Systems, Boeing, Catterpillar are among the companies which benefit from AR for engineering and maintenance activities. • Automotive: BMW, Mercedes-Benz, Mitsubishi, SAAB have developed new generation AR based windshields to assist the drivers by navigational and other driving elements. • Integration with IoT: Especially, Industry 4.0 is so tempting to utilize AR that the integration of the sensors and AR will provide cost reduction and efficiency. 3.1.3. Education The innovative use cases of education can be grouped as below [10, 26]; • Lesson contents: AR technology can be adapted to each age-range. Science, history, biology classes are found so suitable to take advantage of this technology’s immersive and impressive teaching capabilities. 3.1.4. Medical The innovative use cases of health care can be grouped as below; • 3D Anatomical systems for medical students: The students can rotate the virtual display of the medical scenes and investigate their details interactively [28]. This ability aids the brainstorming and discussions, which is an advantage for permanent teaching and learning, in a shared vision. • Remote assistance: These applications provide remote videoconference platform as a solution for the limited number of medical specialists problem [2, 26]. The expert of the related issue can connect to the operation area remotely in real-time, communicate and assist by using AR object tracking and synthetic drawing capabilities. • Assisting systems to doctors, nurses and patients: They present the AR based visual information to guide about patient vein location [12], patients’ medical workup outcomes such as MRI [26] or ultrasound [10]. In these solutions AR visualizes the images in backend by processing raw data and transforming into meaningful information. For example; the volumetric fetus data are rendered and display with colorful view on the abdomen of the pregnant woman for technician. • Robotics: ARGOS (Augmented Reality through Graphic Overlays on Stereovideo) system prototype has been developed in Japan in 1993 to solve the spatial human-robot communication problem [29]. In this prototype AR is used as a communication method for robot-human interaction. AR based robotics is also preferred for surgical operations such as head-surgery and LPN (Laparoscopic Partial Nephrectomy) to get navigational assistance for the decision-making guidance [30, 31]. 3.1.5. Gaming The innovative use cases of gaming can be grouped as below; • Community and marketing: Pokemon GO that is a good example puts the gamers and the companies together [32]. The game is played in real world, that high interaction level makes it very enjoyable. These community games are so favorable for marketing and sales purposes to create new opportunities for the brands and companies by introducing the products to consumers. • Mobile applications: Snapchat is a perfect instance that is used as an entertainment tool; it works by displaying various images on our real-time camera frame. This application has been used for 8 years. • Visualizing additional information: AR technology is also started to utilize in professional sports races to track the objects and the players in the field [10, 26]. Fox-Trax system provides easy puck tracking for the audience during the match [33].
412 6
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
3.2. HAVELSAN’s Use Cases HAVELSAN has already shown an innovative approach to present AR based IS to the market by supervising the need of the society. In this context according to the open innovation policy the activities below are performed; • The ecosystem companies are assessed by visited their premises. • The development of a product platform is decided with the consensus of the technical and management boards. This platform is an AR framework that includes core development functions of AR based IS as an software development kit (SDK) to service in-house and external customer. • A new business model is created with the stakeholders such as subcontractors, universities, in-house project teams for the development and the dissemination of the AR projects and products. • A marketing plan has been prepared to implement global products and sell them in international market. • The workshops, hackathons and surveys are achieved to raise awareness. • Various POC (Proof-of-Concept) studies are planned with different companies to develop projects and products that are appealing to different sectors which are categorized at previous section of this paper. • EU project application has been planned and been started to search to establish an international partnership. By applying these steps the below AR based R&D projects are in planning/development process; • Military: Battlefield management system, stand-alone situational awareness system, remote soldier motion simulation, ground vehicle visualization system • Industry: Manufacturing procedure visualization • Education: Training material infrastructure with haptics, maintenance training system • Medical: Vein visualization • Gaming: Mobile serious game All of these IS are implemented or modified with HAVELSAN’s Augmented Reality Framework H-ARF. 4. Conclusion AR is a commonly used technology at present. It has wide spread use in various areas all around the world. At the same time it makes possible developing creative solutions for need-based problems. Therefore, it is improving so rapidly day to day. By these characteristics, this technology is inclined to be an innovation area. According to literature review it is observed that AR is a subject, which is searched since 1960s. However, it becomes prevalent especially after the integration with IS. In previous years the main limitations of AR are hardware specific such as the inadequate sensor range, not portable display devices or insufficient processors for rendering. Nowadays, after the development of the AR display devices; tablets, smartphones and HMDs, AR based IS come into our lives intensively. In this paper we have divided each of the five main usage areas of AR based IS into subcategories. It has not been encountered any sub categorization like that in the previous studies. This new categorization is generated by taking into account the usage purposes/use cases of the systems in related field. For instance, the use cases are listed as piloting systems, remote control systems, vehicle crew systems, battlefield management systems, robotics, technical guidance and training for the aerospace & defense area, and the developed projects and applications are given for each of them. The approach of AR based IS innovation of HAVELSAN incorporation’s is presented. One of the biggest 5 simulator firms in the world HAVELSAN has being applied innovation activities step by step for AR based IS development. In this context, several events such as in-house workshop, hackathon and survey has been achieved, R&D projects have been started, and ecosystem companies are involved in the development activities and created business model. The product and project ideas have been evaluated and matured by the consensus of the internal and external boards. And lots of innovative R&D projects are started to implement according to market requirements. One of those is the development of an augmented reality SDK. It provides product platforming, which is one of the open innovation models [34]. Neverthless there are some challenges of open innovation strategy while applying in a technology area. The main ones of them are the organization and management difficulties. However, it is seen that the effects of these
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
413 7
disadvantages can be reduced with efficient resource planning and allocation. The current procedure of the innovation policy has been improved through this lessons learned. In conclusion, we believe that AR based IS issue is so suitable for innovation that HAVELSAN has the best practices with the open innovation policy. It is suggested to research the outcomes of the marketing and business development activities for future work. References [1] Frankelius, Per. (2009) "Questioning two myths in innovation literature." The Journal of High Technology Management Research 20: 40–51. [2] Maj, R. J. Kenny. (2015) “Augmented Reality at the Tactical and Operational Levels of War.” US Naval War College Newport Rhode Island. [3] Chesbrough, H. William. (2003). “Open Innovation: The new imperative for creating and profiting from technology.” Boston: Harvard Business School Press. [4] Berthon, Pierre R., Pitt, Leyland F., McCarthy, Ian, and Kates, Steven M. (2007) "When customers get clever: Managerial approaches to dealing with creative consumers." Business Horizons 50 (1): 39–47. [5] West, Joel, and Lakhani, Karim R. (2008) "Getting clear about communities in open innovation." Industry and Innovation 15 (2): 223–231. [6] Bogers, Marcel, Zobel, Ann-Kristin, Afuah, Allan, Almirall, Esteve, Brunswicker, Sabine, Dahlander, Linus, Frederiksen, Lars, Gawer, Annabelle, and Gruber, Marc (2017) "The open innovation research landscape: established perspectives and emerging themes across different levels of analysis." Industry and Innovation 24 (1): 8–40. [7] Sutherland, Ivan E. (1968) “A head-mounted three dimensional display.” Proceedings of the AFIPS’68 Fall, part I: 757–764. [8] Azuma, Ronald T., Baillot, Yohan, Feiner, Steven K., Julier, Simon, Behringer, Reinhold, and Macintyre, Blair. (2001) “Recent Advances in Augmented Reality.” IEEE Computer Graphics And Applications 34–47. [9] van Krevelen, D. W. F.. (2007) “Augmented reality: Technologies, applications, and limitations.” Department of Computer Science Vrije Universiteit Amsterdam Vrije Universiteit Amsterdam The Netherlands. [10] Mekni, Mehdi, and Lemieux, Andre. (2011) “Augmented reality: Applications, challenges, and future trends.”. [11] Livingston, Mark A., Rosenblum, Lawrence J., Brown, Dennis G., Schmidt, Gregory S., Julier, Simon J., Baillot, Yohan, Swan II, J. Edward, Ai, Zhuming, and Maassel, Paul. (2011) “Military applications of augmented reality.” Naval Research Laboratory Washington D. C.. [12] Bermejo, Carlos, Huang, Zhanpeng, Braud, Tristan, and Hui, Pan. (2014) “When augmented reality meets big data.” System and Media Laboratory Department of Computer Science and Engineering The Hong Kong University of Science and Technology 2017 Conference Paper Sweden. [13] Olshannikova, Ekaterina, Ometov, Aleksandr, Koucheryavy, Yevgeni, and Olsson, Thomas. (2015) “Visualizing big data with augmented and virtual reality: Challenges and research agenda.” Journal of Big Data 2:22. [14] Harborth, David. (2017) “Augmented reality in information systems research: A systematic literature review.” 1-10. [15] Miller, Randy. (2017) “How digital technologies are transforming aerospace and defense.” https://www.ey.com/en_gl/aerospace-defense/howdigital-technologies-are-transforming-aerospace-and-defense Ernst&Young Global Limited UK Accessed Date: April 2019. [16] Piedimonte, Pompeo, and Ullo, S. Liberata. (2018) “Applicability of the mixed reality to maintenance and training processes of C4I systems in italian air force.” 5th IEEE International Workshop on Metrology for AeroSpace (MetroAeroSpace) Rome 559-564. [17] Ilic, Alexander, and Fleisch, Elgar. (2016) “Augmented reality and the internet of things.” Auto-ID Labs White Paper. [18] Ponirakis, Lore-Anne. (2019) “Developing an intermediate augmented reality capability for infantry.” Navy SBIR. [19] National Research Council. (1999) “Realizing the potential of C4I: Fundamental challenges.” Washington, D. C. The National Academies Press 64-129. [20] Haselton, Todd. (2019) “How the army plans to use Microsoft’s high-tech HoloLens goggles on the battlefield.” https://www.cnbc.com/2019/04/06/microsoft-hololens-2-army-plans-to-customize-as-ivas.html Accessed date: June 2019. [21] Randall, David. (2015) “Army heads up situational awareness technologies.” U.S. Army RDECOM. [22] Joshi, Naveen. (2017) “Impact of IoT on augmented and virtual reality.” https://www.allerin.com/blog/impact-of-iot-on-augmented-andvirtual-reality Accessed date: June 2019. [23] Chakraborty, Arijit. (2018) “Supply chain innovation with iot and augmented reality.” IoT Cloud Platform Blog Oracle. [24] Dhingra, Alka. (2018) “How augmented reality will impact businesses in 2018” IoT Zone Digi-key Electronics https://dzone.com/articles/howaugmented-reality-will-impact-businesses-in-20 Accessed date: June 2019. [25] Mutual Mobile. (2018) “Why IoT finally makes augmented reality important.” https://mutualmobile.com/resources/why-iot-finally-makesaugmented-reality-important Accessed date: June 2019. [26] Partner Content. (2018) “12 Hot augmented reality ideas for your business.” https://www.information-age.com/12-hot-augmented-realityideas-for-your-business-123471817/ Accessed date: June 2019. [27] Knaflewski, Jacek. (2019) “10 Revolutionary augmented reality apps that you should know about” IoT Zone Digi-key Electronics https://dzone.com/articles/10-revolutionary-augmented-reality-apps-that-you-s Accessed date: June 2019. [28] Ingram, Methew. (2017) “Medical student are using augmented reality to study patients in 3d.” http://fortune.com/2017/05/03/medicalaugmented-reality/ Accessed date: June 2019. [29] Milgrim, Paul, Zhai, Shumin, Drascic, David, and Grodski, Julius J. (1993) “Applications for augmented reality for human-robot communication.” Proceedings of the 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems Yokohama Japan. [30] W¨orn, H., Aschke, M., and Kahrs, L. Alexander. (2005) “New augmented reality and robotic based methods for head-surgery.” The International Journal of Medical Robotics and Computer Assisted Surgery 1 (3): 49–56, 2005. [31] Su, Li-Ming, Vagvolgyi, B. Peter, Agarwal, Rahul, Reiley, Carol E., Taylor, Russel H., and Hager, Gregory D. (2009) “Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3d-ct to stereoscopic video registration.” Urology 73 (4): 896–900. [32] Statt, Nick. (2016) “‘Pokémon GO’ finally just went live on iOS and android in the united states.” The Verge https://www.theverge.com/2016/7/5/12102432/pokemon-go-release-date-ios-android-app-available Accessed date: June 2019.
414 8
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
[33] Cavallaro, Rick. (1997) “The FoxTrax hockey puck tracking system.” IEEE Computer Graphics and Applications 17 (2): 6–12. [34] Schutte, Corne, and Marais, Stephan (2010) "The development of open innovation models to assist the innovation process." University of Stellenbosch South Africa.
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Available online at www.sciencedirect.com Procedia Computer Science 00 (2019) 000–000
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 158 (2019) 407–414
3rd World Conference on Technology, Innovation and Entrepreneurship (WOCTINE) 3rd World Conference on Technology, Innovation and Entrepreneurship (WOCTINE)
An Innovative Technology: Augmented Reality Based Information An Innovative Technology: Augmented Reality Based Information Systems Systems a a
Damla Aslanaa, Bengü Bayyurt Çetinaa, İzzet Gökhan Özbilginaa * Damla Aslan , Bengü Bayyurt Çetin , İzzet Gökhan Özbilgin *
HAVELSAN Inc., Mustafa Kemal Mahallesi 2120 Cad. No:39 Çankaya, Ankara Postcode: 06510, Turkey HAVELSAN Inc., Mustafa Kemal Mahallesi 2120 Cad. No:39 Çankaya, Ankara Postcode: 06510, Turkey
Abstract Abstract In our generation the information systems evolve with new technologies: augmented reality (AR), IoT, artificial intelligence, In our generation the information systems evolveexchange with newbytechnologies: augmentedthat reality (AR), will IoT, be artificial blockchain etc. Anymore they perform information sensors. It is estimated the systems in a stateintelligence, of extreme blockchain etc. Anymore they perform It isknow estimated that the systems willusbewill in abe state of extreme interaction and reach 50 billion devicesinformation connected exchange in Internetbyinsensors. 2020. We that everything around in interaction interaction and 50 billion devices connected in Internet in 2020. know that aroundis us will be instart interaction and they will doreach everything without any need of human interference. ForWe example, wheneverything our dishwasher full, it will to wash and they will dooreverything without of human interference. Forthe example, our full, it will start totowash automatically, when the run out ofany theneed gasoline, our car will drive to nearest when station, ordishwasher even when is a burglar is entered our automatically, or when the run out of theand gasoline, our car to will to the nearest station, life, or even when a burglar is automatical entered to our house, it will automatically be detected be announced thedrive police office. In business the processes will be in house, it will automatically be detected be announced to theand police office. InNext business life, the processesitwill be automatical in maximum level and this technology willand increase productivity efficiency. to mobile technology, is thought that these maximum level information and this technology efficiency. NextAR to also mobile it istothought that these new generation systems will (IS) increase will takeproductivity the biggest and place in our lives. willtechnology, be integrated these systems to new generation information systems will takewill theaugment biggest place in our AR alsoway will thanks be integrated these systems to augment the information in real world.(IS) Humanity its habitat in lives. an innovative to thesetoAR based IS. This augment the information real world. Humanity will related augment its habitat in an way thanks to these AR based This paper surveys the currentin state-of-the-art AR systems with aerospace & innovative defense, industry, education, medical and IS. gaming paper surveys the currentofstate-of-the-art AR innovation systems related with aerospace & defense, industry, and gaming sectors. The connection AR based IS and is explained with a technological insight. education, In additionmedical to international use sectors. The connection AR are based and innovation is explained a technological In addition to international use cases HAVELSAN’s useofcases alsoISgiven that are performed fromwith the aspect of applied insight. open innovation strategy. This strategy cases HAVELSAN’s use are also given that are performed from the aspect of applied open innovation strategy. This strategy is addressed specific to thecases implemented activities of AR based IS. is addressed specific to the implemented activities of AR based IS. © 2019 The Author(s). Published by Elsevier B.V. © 2019 2019 The The Authors. Published Elsevier B.V. © Author(s). Publishedbyby B.V. committee of the 3rd World Conference on Technology, Innovation and Peer-review under responsibility of Elsevier the scientific Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Entrepreneurship Entrepreneurship Entrepreneurship Keywords: new generation information systems, augmented reality, open innovation strategy, technology innovation Keywords: new generation information systems, augmented reality, open innovation strategy, technology innovation
* Corresponding author. Tel.: +90-312-219-5787; fax: +90-312-219-5797. E-mail address:author. [email protected] * Corresponding Tel.: +90-312-219-5787; fax: +90-312-219-5797. E-mail address: [email protected] 1877-0509 © 2019 The Author(s). Published by Elsevier B.V. 1877-0509 2019responsibility The Author(s). Published Elsevier B.V. Peer-review©under of the scientificbycommittee of the 3rd World Conference on Technology, Innovation and Entrepreneurship Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Entrepreneurship
1877-0509 © 2019 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 3rd World Conference on Technology, Innovation and Entrepreneurship 10.1016/j.procs.2019.09.069
408 2
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
1. Introduction Information systems (IS) present refined data and information to the executives to support their decision-making. They organize all the process data in the organizations. With the evolution of the augmented reality technology the construction and visualization of the business processes are easier than ever happened. Therefore, considering this situation as a technology innovation is not exaggeratedness. The nature of the innovation is required a lasting impression in market or society [1]. The reason of why we mention that augmented reality (AR) is a technology innovation is its providing innovation in information technology (IT) area. AR is a revolutionary approach that it is augmenting our world and our daily life with a new cognition and interaction aspect. It might be stated as sixth sense, which is a new layer to understand the events happened around us. As a commercial defense, simulation and ICT company developing software intensive systems with 1800 employees HAVELSAN is applying open innovation policy to develop and present innovative and pioneer AR based IS solutions to the global market. The rest of the paper is organized as follows: the general definition of AR and the HAVELSAN’s main innovation policy are given in first section, the literature review about AR is given in second section, the international use cases of AR based information systems are provided in third section, and finally the evaluation of HAVELSAN’s innovation strategy related with AR based IS is presented in fourth section as conclusion. 1.1. Definition of AR Augmented reality is to add some contents produced by digital devices into the real-world environment. These contents can be live or indirect physical appearances that are enriched with sound, image, graphics or GPS data [2]. This is a technology to expand someone’s reality. It gives an immersive information environment and also interactivity with the surrounding digital area. Its basic mechanism works with the steps: 1- capture and coordinate data digitally, 2- track the reference points of the data, 3- render virtual information with the combined real environment and display. The various types of data can be captured from different types of sensors such as depth, camera, gyroscope, altimeter, barometer, accelerator, pressure, proximity, compass, IMU (Inertial Measurement Unit), GPS (Global Positioning System) receiver, microphone [2]. The networking for the exchange of the data/information can be provided by RFID (Radio Frequency Identification), Bluetooth or Wi-Fi. 1.2. Innovation Policy of HAVELSAN company HAVELSAN has a leading role in meeting the software based strategic needs of our country with national solutions. To achieve this goal, an open innovation strategy is adopted and implemented. This strategy provides to benefit from not only internal ideas, but also external ideas [3]. The ideas are evaluated and matured with a model that includes the internal and external sources [4]. Lots of stakeholders are incorporated into the innovation processes [5, 6]. In this scope, a five-year Technology Roadmap has been prepared for planning the advanced technology development activities. Over 50 strategic R&D projects planned in this Roadmap for technological development in areas such as augmented/mixed reality, artificial intelligence, learning algorithms, cloud computing, big data, autonomous systems and quantum computing. In addition to the strategic R&D activities, an idea assessment system has been initiated that it is a process from idea to the product. This system is still in progress at the Informatics Innovation Center CoZone HAVELSAN Office, in METU Technopolis. HAVELSAN’s Open Innovation Center is the first open innovation center in the defense industry. Within the scope of open innovation activities, technology-oriented solution competitions and hackathons are organized throughout the year for entrepreneurs, students and technology enthusiasts. HAVELSAN also gives great importance to in-house innovation studies. Innovative ideas and studies on future technologies are evaluated in in-house innovative idea campaigns and competitions. Hundreds of ideas are submitted via the innovation management portal by internal and external stakeholders. Those who succeeded start to work at HAVELSAN’s Open Innovation Center. In the center, joint projects were carried out with universities, technology transfer offices and incubation centers and various trainings are given.
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
409 3
HAVELSAN aims to develop global products that will compete in the international arena and considers this development as a national duty beyond its commercial target. To be able to implement innovation and R&D studies gives great importance to productization in order to obtain commercial results. Since the efficiency of the product is dependent on the fact that HAVELSAN's R&D and innovation ecosystem’s abilities, strategic cooperation activities have been initiated with academy to compete with the ecosystems of international companies. 2. Literature Review Sutherland [7] has initiated the AR researches with the fund of DARPA in 1960. He has presented the term of interactive computer graphics. Azuma et al [8]'s article results are based on Azuma's published complementary survey related with AR. The visualization and tracking techniques are given with their problems. Several application examples are provided from different areas such as defense, sports. The hardware and device shortcomings are represented as the technological challenges. The user interface is also given as the other challenge, it is stated that the interface should be designed by taking into account user experiences. The third challenge is stated as social acceptance, to persuade people to wear a device. Krevelen [9] explains the display, tracking and modeling methods technically and gives their limitations. He also discusses AR applications by grouping as personal information systems, industry and military applications, medical applications, AR for entertainment, AR for the office and education training. The limitations of AR technology are listed as portability and outdoor use, tracking and auto calibration, latency, social acceptance, adaptation, depth perception, eyestrain and overload. Mekni et al [10] research available AR applications based on 12 distinct categories such as medical, military, education, robotics. They point out ergonomic hardware, adequate battery life and obtaining accurate information requirements as the challenges. Livingston et al [11] presents a detailed literature review of military AR projects carried out from 1960s to 2011. The importance of situational awareness, user interface (UI) and BARS (Battlefield Augmented Reality System), created for dismounted warfighter, are mentioned. The technological hardware limitations are given as the challenges. Bermejo et al [12] clarifies various augmented reality applications used in different sectors such as tourism, retail, health care and public services. They define the combining of big data and augmented reality (AR) as a must. The visualization of big data is considered as a new business strategy that it provides improvements by analyzing the enormous data driven from the penetration of mobile device, IoT (internet of things) and social media while performing the organization activities. Olshannikova et al [13] emphasize the importance of the integration of AR and virtual reality (VR) with big data because of the limitations of the human perception. These visualization methods are strictly recommended for new generation systems. The challenges faced with applying them are given at the end of the paper. In spite of these difficulties, the integration of big data and mixed reality (MR) approach promises an immersive user experience through tangible and verbal interaction. Harborth [14] has performed a comprehensive literature review about AR technology for IS. He highlights that AR is an area, which is researched for almost 30 years. It is stated that user behavior should remain under investigation, especially for gaming and smartphone browsers. Miller [15] provides the benefits of new technologies for aerospace & defense industry. These new technologies include augmented reality, blockchain, 3D printing, artificial intelligence, IoT, cyber security etc. He concludes that digitalizing business operations and building a value chain ecosystem are transforming the products with an innovative way. Piedimonte et al [16] present several case studies, which are applied in Italian Air Forces. The utility of augmented reality for C4I systems is provided more promising results than traditional ones. Maj [2] specifies that the application of augmented reality as a new technology increase the combat effectiveness. He explains the benefits and challenges of the use of AR systems in detailed by U.S. military. At the tactical level the locations of the enemy targets and friendly forces are displayed for situational awareness by AR. Especially, AR based systems enhance human cognition by representing the complex information into mental understanding. The challenges are user distraction, user adaption, sensor range and connectivity. At the operational level the requirements are
410 4
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
situational awareness and understanding and a directing and leading plan visualization in a shared vision. The concerns are anti-technology culture, dependency to the system for decision-making and the usability of micromanagement. He recommends holistic data management planning, design of intuitive and rugged systems, and new regulations for military occupations for both tactical and operational levels. Ilic et al [17] discuss why AR is essential for IoT systems. It is noticed that AR enables data collection from smartphones and giving information as output, additionally it is convenient for the integration of and the shared interaction with the new sensors and digital biomarkers. 3. Use Cases 3.1. International Use Cases 3.1.1. Aerospace & Defense The AR ensures digital transformation in battlefield. Its integration with C2 and C4ISR systems provide advantage at the tactical and the operational levels [16, 18, 19]. Situational awareness capability that is crucial for commanders and soldiers during a battle can be raised to the higher level thanks to continuous access and rapid dissemination and exchange of the information. The information is generated by collecting from various sensors, cameras and databases such as cameras on command center, troops in the field or on the UAVs, biometric and emotional sensors on soldiers, big data obtained from the historical events. The response times of deploying forces and providing support can be shortened by utilizing AR based IS. The innovative use cases of aerospace and defense can be grouped as below [2, 10, 11, 20, 21]; • Piloting systems: The cockpit applications are starting with Heads-Up-Displays (HUDs) which display digital data within the user’s field-of-view (FOV) from 1960s. The origin of the super cockpit buzzword is based on 1980s. The pilots have sighted the flight data with an augmented manner by their see-through devices. ARVCOP (AR for the Common Operational Picture) project that has been supported by the Office of Naval Research includes the development of the ability to navigate pilots to find the tracked vessels. F-35 Joint Strike Fighter (JSF) Helmet Mounted Display System (HMDS) has been designed and produced. Canada’s Institute for Aerospace Research (NRC-IAR) has developed the helicopter night vision system. • Remote control systems: Ground vehicles weapon control systems have been developed by SAAB. • Vehicle crew systems: DVES (Driver Vision Enhancement System), VCS (Vehicle Camera System), LSAS (Local Situational Awareness System) have been developed for the drivers and the crew members. BAE Systems has designed the similar of the F-35 JSF HMDS for land vehicles. DARPA has developed ULTRA-VIS and ARA’s ARC 4 with Lockheed Martin, Microvision and BAE Systems. Currently, U.S. Army has awarded a contract with Microsoft Hololens headset for Integrated Visual Augmentation System. In addition, smart weapons, which are guided with AR are another issue included in U.S. Army’s agenda. • Battlefield management systems: Common Operating Picture (COP) solutions provide real-time reporting and geographical information for combat forces by gathering data from the operational functions. Also, navigational information is presented by the integration of AR and GPS based mapping. Arcane company is one of the animated terrain presenters. HUNTR (Heads-Up Navigation, Tracking, and Reporting) system has being developed for tactical situational awareness purposes. This system provides assisting information such as the building names, streets, historical details, etc. with AR technology. • Robotics: NASA has being used Microsoft Hololens to explore the Mars surface before the mission that will be performed in 2020 by Rover team. • Technical guidance: It includes the systems both real-time remote assistance and virtual displaying of the technical instructions. Lockheed Martin, BAE Systems, U.S. Army and Naval engineers are some of the practitioners of this technology. • Training: Navy Research Lab, Marine Corps are using AR as a training aid to teach complex combat tasks. Battlefield Augmented Reality System (BARS) project is initiated to train the soldiers for combat scenarios and simulate real-time enemy behavior.
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
411 5
3.1.2. Industry The innovative use cases of industry can be grouped as below [2, 10, 21, 22, 23, 24, 25, 26, 27]; • Design and implementation: The manufacturing steps are efficiently designed and implemented by AR technology. BMW, SAAB, BAE Systems, Boeing, Catterpillar are among the companies which benefit from AR for engineering and maintenance activities. • Automotive: BMW, Mercedes-Benz, Mitsubishi, SAAB have developed new generation AR based windshields to assist the drivers by navigational and other driving elements. • Integration with IoT: Especially, Industry 4.0 is so tempting to utilize AR that the integration of the sensors and AR will provide cost reduction and efficiency. 3.1.3. Education The innovative use cases of education can be grouped as below [10, 26]; • Lesson contents: AR technology can be adapted to each age-range. Science, history, biology classes are found so suitable to take advantage of this technology’s immersive and impressive teaching capabilities. 3.1.4. Medical The innovative use cases of health care can be grouped as below; • 3D Anatomical systems for medical students: The students can rotate the virtual display of the medical scenes and investigate their details interactively [28]. This ability aids the brainstorming and discussions, which is an advantage for permanent teaching and learning, in a shared vision. • Remote assistance: These applications provide remote videoconference platform as a solution for the limited number of medical specialists problem [2, 26]. The expert of the related issue can connect to the operation area remotely in real-time, communicate and assist by using AR object tracking and synthetic drawing capabilities. • Assisting systems to doctors, nurses and patients: They present the AR based visual information to guide about patient vein location [12], patients’ medical workup outcomes such as MRI [26] or ultrasound [10]. In these solutions AR visualizes the images in backend by processing raw data and transforming into meaningful information. For example; the volumetric fetus data are rendered and display with colorful view on the abdomen of the pregnant woman for technician. • Robotics: ARGOS (Augmented Reality through Graphic Overlays on Stereovideo) system prototype has been developed in Japan in 1993 to solve the spatial human-robot communication problem [29]. In this prototype AR is used as a communication method for robot-human interaction. AR based robotics is also preferred for surgical operations such as head-surgery and LPN (Laparoscopic Partial Nephrectomy) to get navigational assistance for the decision-making guidance [30, 31]. 3.1.5. Gaming The innovative use cases of gaming can be grouped as below; • Community and marketing: Pokemon GO that is a good example puts the gamers and the companies together [32]. The game is played in real world, that high interaction level makes it very enjoyable. These community games are so favorable for marketing and sales purposes to create new opportunities for the brands and companies by introducing the products to consumers. • Mobile applications: Snapchat is a perfect instance that is used as an entertainment tool; it works by displaying various images on our real-time camera frame. This application has been used for 8 years. • Visualizing additional information: AR technology is also started to utilize in professional sports races to track the objects and the players in the field [10, 26]. Fox-Trax system provides easy puck tracking for the audience during the match [33].
412 6
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
3.2. HAVELSAN’s Use Cases HAVELSAN has already shown an innovative approach to present AR based IS to the market by supervising the need of the society. In this context according to the open innovation policy the activities below are performed; • The ecosystem companies are assessed by visited their premises. • The development of a product platform is decided with the consensus of the technical and management boards. This platform is an AR framework that includes core development functions of AR based IS as an software development kit (SDK) to service in-house and external customer. • A new business model is created with the stakeholders such as subcontractors, universities, in-house project teams for the development and the dissemination of the AR projects and products. • A marketing plan has been prepared to implement global products and sell them in international market. • The workshops, hackathons and surveys are achieved to raise awareness. • Various POC (Proof-of-Concept) studies are planned with different companies to develop projects and products that are appealing to different sectors which are categorized at previous section of this paper. • EU project application has been planned and been started to search to establish an international partnership. By applying these steps the below AR based R&D projects are in planning/development process; • Military: Battlefield management system, stand-alone situational awareness system, remote soldier motion simulation, ground vehicle visualization system • Industry: Manufacturing procedure visualization • Education: Training material infrastructure with haptics, maintenance training system • Medical: Vein visualization • Gaming: Mobile serious game All of these IS are implemented or modified with HAVELSAN’s Augmented Reality Framework H-ARF. 4. Conclusion AR is a commonly used technology at present. It has wide spread use in various areas all around the world. At the same time it makes possible developing creative solutions for need-based problems. Therefore, it is improving so rapidly day to day. By these characteristics, this technology is inclined to be an innovation area. According to literature review it is observed that AR is a subject, which is searched since 1960s. However, it becomes prevalent especially after the integration with IS. In previous years the main limitations of AR are hardware specific such as the inadequate sensor range, not portable display devices or insufficient processors for rendering. Nowadays, after the development of the AR display devices; tablets, smartphones and HMDs, AR based IS come into our lives intensively. In this paper we have divided each of the five main usage areas of AR based IS into subcategories. It has not been encountered any sub categorization like that in the previous studies. This new categorization is generated by taking into account the usage purposes/use cases of the systems in related field. For instance, the use cases are listed as piloting systems, remote control systems, vehicle crew systems, battlefield management systems, robotics, technical guidance and training for the aerospace & defense area, and the developed projects and applications are given for each of them. The approach of AR based IS innovation of HAVELSAN incorporation’s is presented. One of the biggest 5 simulator firms in the world HAVELSAN has being applied innovation activities step by step for AR based IS development. In this context, several events such as in-house workshop, hackathon and survey has been achieved, R&D projects have been started, and ecosystem companies are involved in the development activities and created business model. The product and project ideas have been evaluated and matured by the consensus of the internal and external boards. And lots of innovative R&D projects are started to implement according to market requirements. One of those is the development of an augmented reality SDK. It provides product platforming, which is one of the open innovation models [34]. Neverthless there are some challenges of open innovation strategy while applying in a technology area. The main ones of them are the organization and management difficulties. However, it is seen that the effects of these
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
413 7
disadvantages can be reduced with efficient resource planning and allocation. The current procedure of the innovation policy has been improved through this lessons learned. In conclusion, we believe that AR based IS issue is so suitable for innovation that HAVELSAN has the best practices with the open innovation policy. It is suggested to research the outcomes of the marketing and business development activities for future work. References [1] Frankelius, Per. (2009) "Questioning two myths in innovation literature." The Journal of High Technology Management Research 20: 40–51. [2] Maj, R. J. Kenny. (2015) “Augmented Reality at the Tactical and Operational Levels of War.” US Naval War College Newport Rhode Island. [3] Chesbrough, H. William. (2003). “Open Innovation: The new imperative for creating and profiting from technology.” Boston: Harvard Business School Press. [4] Berthon, Pierre R., Pitt, Leyland F., McCarthy, Ian, and Kates, Steven M. (2007) "When customers get clever: Managerial approaches to dealing with creative consumers." Business Horizons 50 (1): 39–47. [5] West, Joel, and Lakhani, Karim R. (2008) "Getting clear about communities in open innovation." Industry and Innovation 15 (2): 223–231. [6] Bogers, Marcel, Zobel, Ann-Kristin, Afuah, Allan, Almirall, Esteve, Brunswicker, Sabine, Dahlander, Linus, Frederiksen, Lars, Gawer, Annabelle, and Gruber, Marc (2017) "The open innovation research landscape: established perspectives and emerging themes across different levels of analysis." Industry and Innovation 24 (1): 8–40. [7] Sutherland, Ivan E. (1968) “A head-mounted three dimensional display.” Proceedings of the AFIPS’68 Fall, part I: 757–764. [8] Azuma, Ronald T., Baillot, Yohan, Feiner, Steven K., Julier, Simon, Behringer, Reinhold, and Macintyre, Blair. (2001) “Recent Advances in Augmented Reality.” IEEE Computer Graphics And Applications 34–47. [9] van Krevelen, D. W. F.. (2007) “Augmented reality: Technologies, applications, and limitations.” Department of Computer Science Vrije Universiteit Amsterdam Vrije Universiteit Amsterdam The Netherlands. [10] Mekni, Mehdi, and Lemieux, Andre. (2011) “Augmented reality: Applications, challenges, and future trends.”. [11] Livingston, Mark A., Rosenblum, Lawrence J., Brown, Dennis G., Schmidt, Gregory S., Julier, Simon J., Baillot, Yohan, Swan II, J. Edward, Ai, Zhuming, and Maassel, Paul. (2011) “Military applications of augmented reality.” Naval Research Laboratory Washington D. C.. [12] Bermejo, Carlos, Huang, Zhanpeng, Braud, Tristan, and Hui, Pan. (2014) “When augmented reality meets big data.” System and Media Laboratory Department of Computer Science and Engineering The Hong Kong University of Science and Technology 2017 Conference Paper Sweden. [13] Olshannikova, Ekaterina, Ometov, Aleksandr, Koucheryavy, Yevgeni, and Olsson, Thomas. (2015) “Visualizing big data with augmented and virtual reality: Challenges and research agenda.” Journal of Big Data 2:22. [14] Harborth, David. (2017) “Augmented reality in information systems research: A systematic literature review.” 1-10. [15] Miller, Randy. (2017) “How digital technologies are transforming aerospace and defense.” https://www.ey.com/en_gl/aerospace-defense/howdigital-technologies-are-transforming-aerospace-and-defense Ernst&Young Global Limited UK Accessed Date: April 2019. [16] Piedimonte, Pompeo, and Ullo, S. Liberata. (2018) “Applicability of the mixed reality to maintenance and training processes of C4I systems in italian air force.” 5th IEEE International Workshop on Metrology for AeroSpace (MetroAeroSpace) Rome 559-564. [17] Ilic, Alexander, and Fleisch, Elgar. (2016) “Augmented reality and the internet of things.” Auto-ID Labs White Paper. [18] Ponirakis, Lore-Anne. (2019) “Developing an intermediate augmented reality capability for infantry.” Navy SBIR. [19] National Research Council. (1999) “Realizing the potential of C4I: Fundamental challenges.” Washington, D. C. The National Academies Press 64-129. [20] Haselton, Todd. (2019) “How the army plans to use Microsoft’s high-tech HoloLens goggles on the battlefield.” https://www.cnbc.com/2019/04/06/microsoft-hololens-2-army-plans-to-customize-as-ivas.html Accessed date: June 2019. [21] Randall, David. (2015) “Army heads up situational awareness technologies.” U.S. Army RDECOM. [22] Joshi, Naveen. (2017) “Impact of IoT on augmented and virtual reality.” https://www.allerin.com/blog/impact-of-iot-on-augmented-andvirtual-reality Accessed date: June 2019. [23] Chakraborty, Arijit. (2018) “Supply chain innovation with iot and augmented reality.” IoT Cloud Platform Blog Oracle. [24] Dhingra, Alka. (2018) “How augmented reality will impact businesses in 2018” IoT Zone Digi-key Electronics https://dzone.com/articles/howaugmented-reality-will-impact-businesses-in-20 Accessed date: June 2019. [25] Mutual Mobile. (2018) “Why IoT finally makes augmented reality important.” https://mutualmobile.com/resources/why-iot-finally-makesaugmented-reality-important Accessed date: June 2019. [26] Partner Content. (2018) “12 Hot augmented reality ideas for your business.” https://www.information-age.com/12-hot-augmented-realityideas-for-your-business-123471817/ Accessed date: June 2019. [27] Knaflewski, Jacek. (2019) “10 Revolutionary augmented reality apps that you should know about” IoT Zone Digi-key Electronics https://dzone.com/articles/10-revolutionary-augmented-reality-apps-that-you-s Accessed date: June 2019. [28] Ingram, Methew. (2017) “Medical student are using augmented reality to study patients in 3d.” http://fortune.com/2017/05/03/medicalaugmented-reality/ Accessed date: June 2019. [29] Milgrim, Paul, Zhai, Shumin, Drascic, David, and Grodski, Julius J. (1993) “Applications for augmented reality for human-robot communication.” Proceedings of the 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems Yokohama Japan. [30] W¨orn, H., Aschke, M., and Kahrs, L. Alexander. (2005) “New augmented reality and robotic based methods for head-surgery.” The International Journal of Medical Robotics and Computer Assisted Surgery 1 (3): 49–56, 2005. [31] Su, Li-Ming, Vagvolgyi, B. Peter, Agarwal, Rahul, Reiley, Carol E., Taylor, Russel H., and Hager, Gregory D. (2009) “Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3d-ct to stereoscopic video registration.” Urology 73 (4): 896–900. [32] Statt, Nick. (2016) “‘Pokémon GO’ finally just went live on iOS and android in the united states.” The Verge https://www.theverge.com/2016/7/5/12102432/pokemon-go-release-date-ios-android-app-available Accessed date: June 2019.
414 8
Damla Aslan et al. / Procedia Computer Science 158 (2019) 407–414 Aslan, Bayyurt Çetin, Özbilgin / Procedia Computer Science 00 (2019) 000–000
[33] Cavallaro, Rick. (1997) “The FoxTrax hockey puck tracking system.” IEEE Computer Graphics and Applications 17 (2): 6–12. [34] Schutte, Corne, and Marais, Stephan (2010) "The development of open innovation models to assist the innovation process." University of Stellenbosch South Africa.