- Email: [email protected]
Implementing mixed reality in automotive life cycle engineering: A visual analytics based approach
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Available online atonline www.sciencedirect.com Available at www.sciencedirect.com Procedia CIRP 00 (2018) 000–000 Procedia CIRP 00 (2018) 000–000
ScienceDirect ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia CIRP 00 (2017) 000–000 Procedia CIRP 80 (2019) 717–722 www.elsevier.com/locate/procedia
26th 26th CIRP CIRP Life Life Cycle Cycle Engineering Engineering (LCE) (LCE) Conference Conference
Implementing mixed reality in life CIRP Design Conference, May 2018, Nantes, France engineering: Implementing28th mixed reality in automotive automotive life cycle cycle engineering: A visual analytics based analytics based approach approach A new methodologyAtovisual analyze the functional and physical architecture of a,* a a a a Alexander Kaluza Max Juraschek Büth Cerdas Herrmann a, Lennart a, Christoph existing products assembly oriented product family identification Alexander Kaluzaa,*,, for Maxan Juraschek , Lennart Bütha,, Felipe Felipe Cerdas , Christoph Herrmanna a a
Institute of Machine Tools and Production Technology (IWF), Sustainable Manufacturing & Life Cycle Engineering Research Group, Technische Universität Institute of Machine Tools and Production Technology (IWF), Sustainable & Life CycleGermany Engineering Research Group, Technische Universität Braunschweig, Langer Kamp Manufacturing 19b, 38106 Braunschweig, Braunschweig, Langer Kamp 19b, 38106 Braunschweig, Germany
Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat
* Corresponding Tel.:Supérieure +49-531-391-7170; fax: +49-531-391-5842. E-mail address: Écoleauthor. Nationale d’Arts et Métiers, Arts et Métiers ParisTech, [email protected] EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France * Corresponding author. Tel.: +49-531-391-7170; fax: +49-531-391-5842. E-mail address: [email protected]
* Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: [email protected]
Abstract Abstract Supporting engineers with insights on product- or process related environmental impacts requires comprehensive interpretations of complex LCA Supporting engineers with insights on product- or process related environmental impacts requires comprehensive interpretations of complex LCA Abstract models and results. Those rely on large amounts of data as well as different engineering models, both within the technosphere and at the interface models and results. Those rely on large amounts of data as well as different engineering models, both within the technosphere and at the interface to the ecosphere. Methods from visual analytics can support the interpretation of LCA results leading to a better integration of environmental to the ecosphere. Methods from visual analytics can support the interpretation of LCA results unbroken. leading toDue a better integration of environmental Inconstraints today’s business environment, theInteractivity trend towards more product varietyofand to this development, the need of during decision-making. is an essential element the customization visual analyticsisprocess. In this regard, mixed reality technologies constraints during decision-making. Interactivity is an essential element of the visual analytics process. In this regard, mixed reality technologies can increase interactivityproduction and might lead to a emerged better andtofaster of the knowledge hidden behindTothedesign data. and optimize production agile and reconfigurable systems cope understanding with various products and product families. can increase interactivity and might lead to a better and faster understanding of the knowledge hidden behind the data. A case study applying a mixedthe reality solution within a visual analytics-based approachare is introduced. It addresses conceptual design stage in systems as well as to choose optimal product matches, product analysis methods needed. Indeed, most ofthethe known methods aim to A case study applying a mixed reality solution within a visual analytics-based approach is introduced. It addresses the conceptual design stage in the engineering of future vehicle generations. Based on theDifferent case study, potentials andhowever, barriers may of mixed reality inin an automotive life cycle analyze a product or one product family on the physical level. product families, differ largely terms of the number and the engineering of future vehicle generations. Based on the case study, potentials and barriers of mixed reality in an automotive life cycle engineering context areThis discussed. Subsequently, an approach for evaluation with to capabilities in decision support isfor proposed. nature of components. fact impedes an efficient comparison and choice of respect appropriate product family combinations the production engineering context are discussed. Subsequently, an approach for evaluation with respect to capabilities in decision support is proposed. © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license system. AThe new methodology is proposed to analyze existing products in view ofunder their the functional and physical architecture. The aim is to cluster © 2019 Authors. Published by Elsevier B.V. This is an open access article CC BY-NC-ND license © 2019 The Authors. Publishedoriented by Elsevier B.V.families This is an access articleofunder theassembly CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/). these products in new assembly product foropen the optimization existing lines and the creation of future reconfigurable (http://creativecommons.org/licenses/by-nc-nd/3.0/). (http://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering (LCE) Conference. assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering (LCE) Conference. a functional analysis is performed. Moreover, a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the Keywords:between Mixed reality; Automotive analytics similarity product familiesLCE; by Visual providing design support to both, production system planners and product designers. An illustrative Keywords: Mixed reality; Automotive LCE; Visual analytics example of a nail-clipper is used to explain the proposed methodology. An industrial case study on two product families of steering columns of thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach. Introduction (foreground system) or due to the surrounding background ©1. The Authors. Published by Elsevier B.V. 1.2017 Introduction (foreground system) or due to the surrounding background systems, e.g. climatic Validated models are Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.conditions. systems, e.g. climatic conditions. Validated models are
Life Cycle Engineering (LCE) aims at designing the “[…] required and the underlying data collection and processing is Life Cycle Engineering (LCE) aims at designing the “[…] required and the underlying data collection and processing is concepts, usually time and resource intensive. As a second major product life cycle through choices about product concepts, usually time and resource intensive. As a second major structure, materials and processes […]” with Life Cycle challenge, the interpretation of LCA studies tends to be structure, materials and processes […]” with Life Cycle challenge, the interpretation of LCA studies tends to be Assessment (LCA) being “[…] the tool that visualizes the incoherent with the actual needs of engineers and decision Assessment (LCA) being “[…] the tool that visualizes the incoherent with the actual needs of engineers and decision environmental and resource consequences of these choices” makers [2]. This is because LCA results are rather difficult to environmental and resource consequences of these choices” makers [2]. This is because LCA results are rather difficult to 1.[1]. Introduction of the product range manufactured and/or LCA represents a well-established and standardized understand, e.g. dueand to characteristics multi-dimensionality and complex [1]. LCA represents a well-established and standardized understand, e.g. due to multi-dimensionality and complex assembled in this system. In this context, challenge in methodology for the quantitative evaluation of environmental interdependencies [3]. Another reason the are main different scopes. methodology for the quantitative evaluation of environmental interdependencies [3]. Another reason are different scopes. Due to the fast development in the domain of modelling and analysis is now not only to cope with single impacts of products or processes. However, LCA-based LCE While product- or manufacturing-driven engineers focus their impacts of products or However, LCE While productor manufacturing-driven engineers focus their communication anprocesses. ongoing trenddecision of LCA-based digitization products, a limited product or existing families, often fails to and provide adequate support and for actions within a distinct arearange of influence, e.g. product the design of one often fails to provide adequate decision support for actions within a distinct area of influence, e.g. the design of one digitalization, manufacturing enterprises are facing important but also to be able analyze compare to define engineering, e.g. when a comparison between several options component, LCEtocalls forand a tolife cycle products perspective that engineering, e.g. when a comparison between several options component, LCE calls for a life cycle perspective that challenges in today’s continuing new product families. can beand observed that classicalproducts existing needs to be made inmarket productenvironments: development.a Two major encompasses differentItstages, multiple associated needs totowards be made in product development. Two major encompasses different stages, and multipleofassociated products tendency reduction of product development times and product families are regrouped in function clients or features. challenges are faced. The first one lies in applying LCA in a and processes [4]. An interpretation limited to a specific scope challengesproduct are faced. The first one lies in applying LCA in a and processes [4]. An interpretation to a specific shortened lifecycles. In addition, there is and an increasing However, assembly oriented product limited families hardly toscope find. prospective manner to design future products their life could impede holistic feedback aboutareenvironmental prospective manner to design future products and their life could impede holistic feedback about environmental demand of customization, being at the same time in a global On the product family level, products differ mainly in two cycles. This typically requires evaluating a larger number of consequences of engineering choices. cycles. Thiswith typically requiresallevaluating aworld. larger This number of consequences of engineering choices. competition competitors over the trend, main characteristics: (i) the number of components and (ii) the scenarios. Environmental impacts could vary due to changing To overcome these challenges, an approach has been scenarios. Environmental impacts could varymacro due to to changing To overcome these challenges, electrical, an approach has been which is inducing the development from micro type of components (e.g. mechanical, electronical). engineering parameters of the studied product itself suggested to support decision making processes in LCE by engineering parameters of the studied product itself suggested support decision making mainly processes in products LCE by markets, results in diminished lot sizes due to augmenting Classicaltomethodologies considering single product varieties (high-volume to low-volume production) [1]. or solitary, already existing product families analyze the 2212-8271 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license To cope with this variety as wellB.V. as This to be able structure on a physical level (components level) which 2212-8271 © 2019 Theaugmenting Authors. Published by Elsevier is an opentoaccessproduct article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/). (http://creativecommons.org/licenses/by-nc-nd/3.0/). identify possible optimization potentials in the existing causes difficulties regarding Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering (LCE) Conference. an efficient definition and Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering Conference. production system, it is important to have a precise knowledge comparison of(LCE) different product families. Addressing this doi:10.1016/j.procir.2017.04.009 product life cycleDesign through choices about product Keywords: Assembly; method; Family identification
doi:10.1016/j.procir.2017.04.009
2212-8271 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) 2212-8271 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of scientific the scientific committee theCIRP 26thDesign CIRP Conference Life Cycle 2018. Engineering (LCE) Conference. Peer-review under responsibility of the committee of the of 28th 10.1016/j.procir.2019.01.078
718 2
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
means of a visual analytics (VA) cycle [5]. VA aims at a human-centered knowledge building through a combination of automated data analyses and visual exploration. Key characteristics comprise the integration of data, models and visualizations [6]. One technological concept for visualization in VA is Mixed Reality (MR). MR technology is increasingly reaching a state of productiveness that might lead to significant improvements in engineering processes [7]. As summarized by Lindgren et al., embodiment provided through MR powerfully supports cognition processes in engineering [8]. Thus, MR is increasingly applied in manufacturing contexts, e.g. [9]. LCE might leverage from MR technologies manifold. For example, an earlier study identified a high potential for contribution analyses of environmental impacts through a joint visualization of physical products and digital models [10]. However, there are no coherent methods yet regarding the employment of MR technologies in VA for LCE and only few case studies. This paper presents a potential implementation of MR to support LCE activities within an automotive engineering setting. 2. Background 2.1. LCE in automotive engineering design The enrichment of methods in engineering design through life cycle thinking represents an established research approach also known as “ecodesign” or “sustainable design”. From the state of research as well as practical applications McAloone and Pigosso summarize that “[…] a careful and systematic approach to integrated analysis (LCA) and synthesis (ecodesign) is optimal in order to achieve environmentally enhanced solutions […]” [11]. To enable an LCA-based LCE, approaches that streamline the LCA method and/or tightly integrate LCA steps into engineering tools are developed [11]. Examples are tailored LCA tools that are integrated into CAD or PLM systems, e.g. [12]. Other approaches target improved visualizations of LCA results that enable a facilitated interpretation in engineering, e.g. [3]. Automotive LCE further requires coping with technological innovation. One example is the engineering of lightweight vehicle structures. LCE challenges are built around the introduction of new materials or manufacturing processes, including joining techniques, as well as in the selection of appropriate technologies [13,14]. One example is the tradeoff between manufacturing and use stage for electric vehicles, that beyond embodied emissions and weight reductions strongly depends on regional and interindividual factors [15]. 2.2. Visualization in VA-based LCE Engineering problems in general are mathematical representations of physical phenomena [16]. Visualizations in engineering thus aim at depicting those mathematical models to enable the understanding and derivation of actions. Within LCE, one focus for engineering visualization is the interplay of engineering problems within the technosphere, e.g. the effect of increased vehicle mass on engine design in the use stage. Another focus is the relation of engineering decisions to environmental impacts (ecosphere). Within a model-based
LCE, the considered system and its sub-systems are described as a set of mathematical models based on quantified material and energy flows, e.g. shown for electric vehicles in [17]. As reviewed by Ramanujan et al., the number of VA tools in LCE is still very limited [18], but gains increased attention in research. At the interface of design engineering and LCA, the shapeSIFT tool targets the multi-dimensional exploration of a design space based on a repository of materials, manufacturing, functions and shapes. The tool has shown benefits in eco-conscious engineering design by narrowing down options and enabling the interpretation of changes in environmental impacts through parameter variation [19]. The challenge in designing VA tools in specific decision contexts becomes apparent from the development of QuestVis. The tool provides full insight into complex impacts and interdependencies of sustainability-related planning options. However, the initial goal of fostering behavior change was hindered due to the broad information space that itself required teaching [20]. In consequence, Munzner et al. developed a nested model for visualization design and validation [21]. It highlights the importance of domain problem characterization over subsequent steps in visualization design (see Figure 1). The domain problem characterization states that the vocabulary and the exact problem of the target audience needs to be clearly understood. The abstraction translates domain specific tasks and data models to a computer science based description. Both stages together form the foundation of the actual visualization design that consist of the mutual interdependent visual encoding and interaction technique design. The algorithm design includes operative tasks in realization [21]. Ramanujan et al. derived general tasks and design patterns for VA in sustainable design that can be applied to develop respective prototypes [22]. In the sense of the nested model the identified tasks (overview, zoom, filter, details on demand, relate, history, extract) could be understood as expressions of the operation abstraction design, while the design patterns provide LCE-specific encoding and interaction technique designs.
Figure 1: Nested model for visualization design by Munzner et al. [21]
2.3. Mixed Reality in LCE MR is often used interchangeably with the terms Augmented Reality (AR), Augmented Virtuality (AV) or Virtual Reality (VR). Here, we follow the definition stating that when digital and real objects are combined in one application, this application becomes a MR application [10]. Based on Milgram [23] real objects have an actual objective existence, whereas virtual objects exist in essence or effect, but not formally or actually. According to Juraschek et al. [10], based on Azuma [24], an MR application has to be interactive in real time and has to be registered in three dimensions.
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
MR can be illustrated as a two-dimensional continuum stretching from full reality to full virtuality on each side as illustrated in Figure 2 [23]. Between these two exclusive forms of human-interactable environments, MR applications can be allocated according to their share of real (physical) and digital objects. If an application has a higher share of real objects, it will be allocated left from the mid-point of the continuum and thus classified as AR. In the opposite case, an application with a higher share of virtual elements is located in AV. Furthermore, implementation concepts, as for instance immersive VR or tangible interfaces, can be classified in the continuum. Together with the allocation of the respective MR devices the reality-virtuality continuum is a comprehensive framework that can support to analyze, design and implement MR applications.
Figure 2: Reality-virtuality continuum from [23] extended based on [10] with MR capabilities and resulting target area for current study
MR applications can encompass several key capabilities. In the context of LCE, these include displaying virtual and/or real objects, a relation to real objects, interactiveness, instructiveness, immediate and remote collaboration as well as the implementation of temporal scenarios and multidimensional visualizations [10]. These capabilities can also be allocated in the reality-virtuality continuum. Displaying real objects and interacting with them for instance, requires the incorporation of real elements into an application and can formally not be achieved with an occluded head-mounted display (HMD), for example. In such isolated display devices, real objects can only be displayed if they have been sampled and thus digitized [23]. The allocation of key capabilities within the continuum can lead to faster and more coherent design processes for MR applications. 3. Approach and case study This study aims at exploring the potentials of MR through a practically implemented VA approach in LCA-based LCE. The engineering of automotive lightweight components serves as an exemplary case. Thereby, the following steps, based on the nested model, as shown in Figure 1 [21], as well as previous studies on VA and MR in LCE [5,10], are pursued: 1. Domain problem characterization 2. Identification of suitable MR options 3. Abstraction design, visual encoding, interaction technique design and implementation 4. Validation of the MR prototype
719 3
3.1. Domain problem characterization This study follows an LCE approach in the automotive conceptual design stage in general and lightweight body structures in specific. The goal is to design eco-efficient components that should carry lower environmental impacts over the entire life cycle compared to reference designs while meeting technical performance criteria [13]. The design of automotive components is driven by the overall vehicle development. Thus, constraints from the surrounding vehicle, e.g. space or joining technologies, limit the solution space for concept alternatives [25]. In current practice, environmental impacts of body component concepts are assessed upstream from series development and released for different vehicles based on condensed results [25]. However, as reviewed in [14], complex interdependencies on potential environmental impacts call for an in line approach within a specific vehicle development instead. One example are lightweight parts for electric vehicles, where environmental burdens from manufacturing could only be compensated for specific scenarios of vehicle use [15]. Figure 3 explains the setting with two or more domain experts (engineering design and LCA) and one superior project manager. The project manager coordinates and prepares decisions for product concepts based on performance indicators (technical, economic and environmental). The upper part of Figure 3 (A, B1-3, C) illustrates the status quo in component development. Design engineering proposes a set of concept alternatives based on given requirements (B1). This includes different geometries and material combinations [25]. In this case, the example of a roof reinforcement structure is considered. Three different geometries of the cross section (full shape, u-shape, reinforced u-shape) are analyzed that could be either manufactured applying steel or aluminum alloys or carbon fiber reinforced plastics. In addition, rib structures from glass-filled polyamide could be applied as a reinforcement. Technical parameters like wall thickness can be influenced by engineering design. This results in a decision between concept alternatives. Even though many concept alternatives fail due to weak mechanical properties or by violating boundary conditions, several alternatives pass this stage and would be handed over to an LCA expert (B2) [13]. The LCA expert then models several life cycle scenarios for the alternatives. This takes into account different fore- and background systems, e.g. vehicle lifetimes, or electricity production [13]. As a result, project managers need to interpret reports from the domain experts for the concept alternatives with respect to specific assumptions and scenarios (B3). In reality the complexity exceeds the illustrated situation as further disciplines are involved, e.g. development of other parts or focus on specific life cycle stages. The implemented case study now aims at improving the described process through applying a VA-based MR approach as illustrated in the lower part of Figure 3. The approach brings together information from the involved disciplines and their domain-specific tools and makes them available within an explorative environment. In the current case, this encompasses the analytical computation of mechanical performance for different product concepts (BN1). The computation allows a
720 4
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
variation of engineering parameters, e.g. wall thickness or shape. Inventory datasets for determining environmental impacts of different life cycle stages are linked to those engineering models (BN2), as e.g. environmental impacts from manufacturing are directly influenced by choices in design engineering. Beyond that, project managers can explore the design space including different vehicles and lifetime scenarios. This helps to determine if efforts from raw materials and manufacturing stages could be compensated over the vehicle lifetime or not. Finally, a favorable concept alternative can be selected through different KPI (BN3). 3.2. Identification of suitable MR options As explained in section 2.3, the reality-virtuality continuum can support the identification of suitable implementation concepts for the design of MR applications. The previously described user journey serves as a task definition. Subsequently, the MR key capabilities required for the outlined task and subtasks are identified and highlighted in the continuum following the approach described in [10]. The area in which all required capabilities overlap is the target area for the implementation approach and the starting point for the MR application design (Figure 2). As different design alternatives shall be compared that are not yet available as physical prototype or manufactured product, the capability of displaying virtual objects is required. At the same time the comparison with physically available objects should be possible, thus displaying real objects is also a desired capability. Within this product-driven approach, the relation to these real products, i.e. objects, is a desired capability. An application that supports decision processes based on LCA results should allow interaction with data. Furthermore, the application can provide support by giving background information and instruct users through the decision process. LCA results depend on several parameters of different dimensions (e.g. temporal or spatial). Overall, six key MR capabilities were identified for the present goal. From the resulting target area (spatial AR, holographic spatial AR, semi-immersive VR), a holographic spatial AR approach was chosen. The Microsoft HoloLens serves as an implementation platform as it was the best available technology meeting the required capabilities.
3.3. Abstraction design, visual encoding, interaction technique design and implementation The developed application consists of different objects that enable an embodiment of concept alternatives as well as the interaction with underlying data on technical performance, weight and associated environmental impacts. While a roof reinforcement structure represents the sample case, the implementation of other body parts would be possible in parallel following the same workflow and toolchain. Figure 4 shows the design concept of the developed MR application. According to the classification by Ramanujan et al. [22], the application integrates the following design patterns (Table 1): Table 1: Tasks (T) and design patterns (P) according to [22] that are realized within the MR prototype Tasks (T) T: Overview T: Zoom T: Relate
Design patterns (P) Indicator overviews Eco-prominence Eco-persistence Multiscale design exploration (regarding life cycle stages) Linking indicators through the life cycle
The application is currently designed for one user representing a cross-domain project manager and decisionmaker. The implementation consists of certain defining items. Several concept alternatives can be visually explored as 3D models on a virtual table (a). A direct comparison to physical products is enabled (b). Concept alternatives can be selected and are then displayed within a 3D vehicle model to explore potential technical boundaries (c). A user interface enables the iterative adaption of geometries, material alternatives and design parameters (d). In the case of metals this encompasses different alloys, whereas fiber-reinforced plastics e.g. vary in the layer structure of fibers. Drivetrains (combustion, electric vehicles) can be selected to represent potential applications. Each concept alternative can be evaluated regarding mechanical performance (stiffness and strength), weight as well as environmental impacts from raw materials provision, manufacturing and use. A potential break-even between alternatives is highlighted. (e). Iteration enables to understand the interrelations between material and geometry choices and potential environmental impacts.
Figure 3: Conventional workflow in automotive LCE and concurrent approach based on MR and VA
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
721 5
Table 2: Threats and suggested validation methods (V) from [21] and transfer to current study (C)
Figure 4: Design concept and realization of the MR application
The implementation utilizes the game engine Unity as development environment [26]. Within Unity, several libraries are applied. First, vehicle CAD-data (Catia) is prepared for the MR device through reducing model complexity, e.g. holes or roundings. This is necessary to realize a sufficient frame rate when running the application from HoloLens without external computing power. Second, pie and line charts are implemented and adapted for through the third party “Graph and Chart” library [27]. The application uses a structured database that has been realized as a set of csv files that can be directly updated from expert tools in LCA (Umberto LCA+) and design engineering (analytical design tool, Matlab). Material information, geometry (wall thickness, width, height) and component weight were used as main interfaces. As the aim of this study was to create a prototype that can be iterated to enhance the quality with respect to data/ operation abstraction and encoding/ interaction technique design, live interfaces to the expert tools have not been realized. Thus, a manipulation of parameters exceeding pre-calculated inputs is not possible now. A major step towards an industrial implementation would lie in a tight integration of the workflow into respective IT systems. The Unity application requires regular updates to most recent data. PLM systems could provide geometry files and material information that can be exported via scripts. In parallel, LCA data, e.g. specific footprints of manufacturing, need to be updated and contextualized regularly. 4. Validation The MR prototype was validated following the layers of the nested model. Munzner et al. identified main threats and validation goals for each of the layers [21]. On that basis, Table 2 points out derived actions for the current study. The domain problem characterization is built up on previous studies [13,14,25]. In [14] the complexity in LCE of future automotive structures has been highlighted and the need for innovative design tools was expressed. However, the targeted decision situation is currently limited to research-driven activities, as an integration into series development would require an adaption of formalized business processes [25]. Thus, actual adoption rates could not be tested. However, the prototype is made available within a jointly operated research environment between industry and academia. The algorithm design validation in terms of performance showed positive results, as the prototype performs fast at the current stage, but computational complexity will significantly increase with further models covering different stages of the vehicle life cycle, as e.g. shown in [17].
Domain problem characterization Threat: wrong problem V: interview target users C: problem definition based on [13,14,25] V: observe adoption rates C: medium-term goal, exceeding scope Data/ operation abstraction design Threat: bad data operation/ abstraction V: test on target users, collect C: anecdotal evidence from user test anecdotal evidence of utility V: field study, document human C: medium-term goal, exceeding scope usage of applied system Encoding/ interaction technique Threat: ineffective encoding/ interaction technique V: justify technique C: first impressions from user test V: qualitative/ quantitative C: medium-term goal, exceeding scope result image analysis V: lab study, measure human C: qualitative evidence from user test time/ errors for operation Algorithm design Threat: slow algorithm C: implementation currently with low V: analyze computational computational complexity; needs to be complexity regarded in the course of industrial V: measure system time/ implementation projects memory
Anecdotal evidence with regard to “data/ operation abstraction design” and “encoding/ interaction technique” was collected in a user test including 10 experts in industrial research and engineering of automotive components. The test asked to identify an eco-efficient concept alternative through iteration. Feedback was collected using a questionnaire, including structured and open questions. Due to the small sample size, the findings represent impressions and starting points for further evolution. Initially, all test users were introduced to the MR device as inexperienced users need to get acquainted to the mixture of reality and virtuality at first (5 to 10 minutes per user). Although it is one of the most powerful available holographic HMD, the first HoloLens still has a limited field of vision and user interactions. A test of human time and errors is thus not beneficial for the current stage. However, all users were able to perform the required operations (average time 10 minutes) and evaluated that the app in general was easy to use (Table 3).
Table 3: Feedback from user test, sample size: 10, only positive formulations listed, red line represents average
The visual representation and interaction through 3D models, sliders, drop-down menus and graphs overall turned out to be useful for the given task. Most test users noted its utility for understanding interfaces of the component to the vehicle and finally solving the task. However, within the visual representation of concept selection, users desired a more interactive manipulation of the 3D models and underlying
722 6
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
parameters. Reported shortcomings within the workflow include options to save evaluated parameter combinations as a reference point. This limits the effectiveness of the explorative activity, as omitted alternatives cannot be documented in a structured way. In addition, users asked for more details to understand the underlying engineering models. This leads to the conclusion that interrelations between design attributes and resulting environmental impacts are not transparent to the desired degree and trust in the results might be limited. Further, several users reported that it is difficult to identify if an alternative is a global optimum within the decision space. 5. Summary and Outlook The current research presents an approach to introduce MR in a VA-based LCE process. The design of eco-efficient lightweight vehicle components serves as a sample case. Domain experts showed great interest in generating a common understanding between component design engineering and LCE, as the topic is gaining importance. The embodiment provided through MR was evaluated positively. However, the prototype showed shortcomings in promoting decisions and misses potentially useful design patterns. First steps for future research include improving overall usability, implementing the aforementioned design patterns and exploring suitable methods from decision sciences to handle conflicting development goals. To enable an experience closer to industrial practice, the functionality needs to be extended to analyze multiple parts at the same time. Another major step would be a near real time coupling of the MR visualization to tools from LCA and engineering design. This would exceed the current focus of reviewing interdependent results towards an actual collaborative engineering setting. Future advanced MR solutions thereby enable a richer exploration of design and life cycle scenarios. Acknowledgements The results published in this paper are based on the project MultiMaK2 aiming at developing engineering tools towards designing eco-efficient vehicle components. This research and development project is funded by the German Federal Ministry of Education and Research (BMBF) within the Research Campus Open Hybrid LabFactory and managed by the Project Management Agency Karlsruhe (PTKA). The authors are responsible for the content of this publication. References [1] Alting L. Life Cycle Engineering and Design. CIRP Ann - Manuf Technol 1995; 44: 569–580. [2] Laurin L, Amor B, Bachmann TM, Bare J, Koffler C, Genest S, Preiss P, Pierce J, Satterfield B, Vigon B. Life cycle assessment capacity roadmap: decision-making support using LCA. Int J LCA 2016; 21: 443–447. [3] Cerdas F, Kaluza A, Erkisi-Arici S, Böhme S, Herrmann C. Improved visualization in LCA through the application of cluster heat maps. Procedia CIRP 2017; 61: 732–737. [4] Hauschild MZ, Herrmann C, Kara S. An Integrated Framework for Life Cycle Engineering. Procedia CIRP 2017; 61: 2–9. [5] Kaluza A, Gellrich S, Cerdas F, Thiede S, Herrmann C. Life Cycle
Engineering Based on Visual Analytics. Procedia CIRP 2018; 69: 37–42. [6] Keim DA, Mansmann F, Stoffel A, Ziegler H. Visual Analytics. In: Encycl Database Syst. Springer; 2009. p. 27–37. [7] Wang X. Editorial visualization in engineering. Vis Eng. 2014; 2: 1–2. [8] Lindgren R, Johnson-Glenberg M. Emboldened by Embodiment: Six Precepts for Research on Embodied Learning and Mixed Reality. Educ Res 2013; 42: 445–452. [9] Makris S, Karagiannis P, Koukas S, Matthaiakis A. Augmented reality system for operator support in human – robot collaborative assembly. CIRP Ann - Manuf Technol 2016; 65: 61-64. [10] Juraschek M, Büth L, Cerdas F, Kaluza A, Thiede S, Herrmann C. Exploring the Potentials of Mixed Reality for Life Cycle Engineering. Procedia CIRP 2018; 69: 638-643. [11] McAloone TC, Pigosso DCA. Ecodesign Implemtation and LCA. In: Hauschild MZ, Rosenbaum RK, S.I. Olsen SI, editors. Life Cycle Assessment. Cham: Springer International Publishing; 2018. p. 545–576. [12] Gaha R, Yannou B, Benamara A. A new eco-design approach on CAD systems. Int J Precis Eng Manuf. 2014; 15: 1443–1451. [13] Kaluza A, Kleemann S, Fröhlich T, Herrmann C, Vietor T. Concurrent Design & Life Cycle Engineering in Automotive Lightweight Component Development. Procedia CIRP 2017; 66: 16–21. [14] Herrmann C, Dewulf W, Hauschild M, Kaluza A, Kara S, Skerlos S. Life cycle engineering of lightweight structures. CIRP Ann - Manuf Technol 2018; 67: 651–672. [15] Egede P. Environmental Assessment of Lightweight Electric Vehicles. Cham: Springer International Publishing; 2017. [16] Li W, Nee AYC, Ong SK. A State-of-the-Art Review of Augmented Reality in Engineering Analysis and Simulation. Multimodal Technologies Interact 2017; 1(3):17. [17] Cerdas F, Thiede S, Herrmann C. Integrated Computational Life Cycle Engineering - Application to the case of electric vehicles. CIRP Ann Manuf Technol 2018; 67: 25–28. [18] Ramanujan D, Bernstein WZ, Chandrasegaran SK, Ramani K. Visual Analytics Tools for Sustainable Lifecycle Design: Current Status, Challenges, and Future Opportunities. J Mech Des 2017; 139: 111415. [19] Ramanujan D, Bernstein WZ, Kulkarni Devadatta JT, Ramani K. shapeSIFT: Evaluating InfoVis Tools for Eco-Conscious Design. 21st Design for Manufacturing and the Life Cycle Conference 2016. [20] Munzner T, Barsky A, Williams M. Reflections on QuestVis: A Visualization System for an Environmental Sustainability Model. Dagstuhl Follow 2011; 2: 259. [21] Munzner T. A nested model for visualization design and validation. IEEE Trans Vis Comput Graph. 2009; 15: 921–928. [22] Ramanujan D, Bernstein WZ, Ramani K. Design patterns for visualization-based tools in sustainable product design. 22nd Design for Manufacturing and the Life Cycle Conference 2017; DETC2017-68054. [23] Milgram P, Kishino F. A Taxonomy of Mixed Reality Visual-Displays. IEICE Trans Inf Syst 1994; E77d: 1321–1329. [24] Azuma RT. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 1997; 6(4): 355–385. [25] Kaluza A, Kleemann S, Broch F, Herrmann C, Vietor T. Analyzing decision-making in automotive design towards life cycle engineering for hybrid lightweight components. Procedia CIRP 2016; 50: 825–830. [26] Unity Technologies, Unity, unity3d.com. [27] Graph And Chart, Prosource Labs, assetstore.unity.com/packages/tools/gui/graph-and-chart-78488.
ScienceDirect ScienceDirect
Available online atonline www.sciencedirect.com Available at www.sciencedirect.com Procedia CIRP 00 (2018) 000–000 Procedia CIRP 00 (2018) 000–000
ScienceDirect ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia CIRP 00 (2017) 000–000 Procedia CIRP 80 (2019) 717–722 www.elsevier.com/locate/procedia
26th 26th CIRP CIRP Life Life Cycle Cycle Engineering Engineering (LCE) (LCE) Conference Conference
Implementing mixed reality in life CIRP Design Conference, May 2018, Nantes, France engineering: Implementing28th mixed reality in automotive automotive life cycle cycle engineering: A visual analytics based analytics based approach approach A new methodologyAtovisual analyze the functional and physical architecture of a,* a a a a Alexander Kaluza Max Juraschek Büth Cerdas Herrmann a, Lennart a, Christoph existing products assembly oriented product family identification Alexander Kaluzaa,*,, for Maxan Juraschek , Lennart Bütha,, Felipe Felipe Cerdas , Christoph Herrmanna a a
Institute of Machine Tools and Production Technology (IWF), Sustainable Manufacturing & Life Cycle Engineering Research Group, Technische Universität Institute of Machine Tools and Production Technology (IWF), Sustainable & Life CycleGermany Engineering Research Group, Technische Universität Braunschweig, Langer Kamp Manufacturing 19b, 38106 Braunschweig, Braunschweig, Langer Kamp 19b, 38106 Braunschweig, Germany
Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat
* Corresponding Tel.:Supérieure +49-531-391-7170; fax: +49-531-391-5842. E-mail address: Écoleauthor. Nationale d’Arts et Métiers, Arts et Métiers ParisTech, [email protected] EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France * Corresponding author. Tel.: +49-531-391-7170; fax: +49-531-391-5842. E-mail address: [email protected]
* Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: [email protected]
Abstract Abstract Supporting engineers with insights on product- or process related environmental impacts requires comprehensive interpretations of complex LCA Supporting engineers with insights on product- or process related environmental impacts requires comprehensive interpretations of complex LCA Abstract models and results. Those rely on large amounts of data as well as different engineering models, both within the technosphere and at the interface models and results. Those rely on large amounts of data as well as different engineering models, both within the technosphere and at the interface to the ecosphere. Methods from visual analytics can support the interpretation of LCA results leading to a better integration of environmental to the ecosphere. Methods from visual analytics can support the interpretation of LCA results unbroken. leading toDue a better integration of environmental Inconstraints today’s business environment, theInteractivity trend towards more product varietyofand to this development, the need of during decision-making. is an essential element the customization visual analyticsisprocess. In this regard, mixed reality technologies constraints during decision-making. Interactivity is an essential element of the visual analytics process. In this regard, mixed reality technologies can increase interactivityproduction and might lead to a emerged better andtofaster of the knowledge hidden behindTothedesign data. and optimize production agile and reconfigurable systems cope understanding with various products and product families. can increase interactivity and might lead to a better and faster understanding of the knowledge hidden behind the data. A case study applying a mixedthe reality solution within a visual analytics-based approachare is introduced. It addresses conceptual design stage in systems as well as to choose optimal product matches, product analysis methods needed. Indeed, most ofthethe known methods aim to A case study applying a mixed reality solution within a visual analytics-based approach is introduced. It addresses the conceptual design stage in the engineering of future vehicle generations. Based on theDifferent case study, potentials andhowever, barriers may of mixed reality inin an automotive life cycle analyze a product or one product family on the physical level. product families, differ largely terms of the number and the engineering of future vehicle generations. Based on the case study, potentials and barriers of mixed reality in an automotive life cycle engineering context areThis discussed. Subsequently, an approach for evaluation with to capabilities in decision support isfor proposed. nature of components. fact impedes an efficient comparison and choice of respect appropriate product family combinations the production engineering context are discussed. Subsequently, an approach for evaluation with respect to capabilities in decision support is proposed. © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license system. AThe new methodology is proposed to analyze existing products in view ofunder their the functional and physical architecture. The aim is to cluster © 2019 Authors. Published by Elsevier B.V. This is an open access article CC BY-NC-ND license © 2019 The Authors. Publishedoriented by Elsevier B.V.families This is an access articleofunder theassembly CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/). these products in new assembly product foropen the optimization existing lines and the creation of future reconfigurable (http://creativecommons.org/licenses/by-nc-nd/3.0/). (http://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering (LCE) Conference. assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering (LCE) Conference. a functional analysis is performed. Moreover, a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the Keywords:between Mixed reality; Automotive analytics similarity product familiesLCE; by Visual providing design support to both, production system planners and product designers. An illustrative Keywords: Mixed reality; Automotive LCE; Visual analytics example of a nail-clipper is used to explain the proposed methodology. An industrial case study on two product families of steering columns of thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach. Introduction (foreground system) or due to the surrounding background ©1. The Authors. Published by Elsevier B.V. 1.2017 Introduction (foreground system) or due to the surrounding background systems, e.g. climatic Validated models are Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.conditions. systems, e.g. climatic conditions. Validated models are
Life Cycle Engineering (LCE) aims at designing the “[…] required and the underlying data collection and processing is Life Cycle Engineering (LCE) aims at designing the “[…] required and the underlying data collection and processing is concepts, usually time and resource intensive. As a second major product life cycle through choices about product concepts, usually time and resource intensive. As a second major structure, materials and processes […]” with Life Cycle challenge, the interpretation of LCA studies tends to be structure, materials and processes […]” with Life Cycle challenge, the interpretation of LCA studies tends to be Assessment (LCA) being “[…] the tool that visualizes the incoherent with the actual needs of engineers and decision Assessment (LCA) being “[…] the tool that visualizes the incoherent with the actual needs of engineers and decision environmental and resource consequences of these choices” makers [2]. This is because LCA results are rather difficult to environmental and resource consequences of these choices” makers [2]. This is because LCA results are rather difficult to 1.[1]. Introduction of the product range manufactured and/or LCA represents a well-established and standardized understand, e.g. dueand to characteristics multi-dimensionality and complex [1]. LCA represents a well-established and standardized understand, e.g. due to multi-dimensionality and complex assembled in this system. In this context, challenge in methodology for the quantitative evaluation of environmental interdependencies [3]. Another reason the are main different scopes. methodology for the quantitative evaluation of environmental interdependencies [3]. Another reason are different scopes. Due to the fast development in the domain of modelling and analysis is now not only to cope with single impacts of products or processes. However, LCA-based LCE While product- or manufacturing-driven engineers focus their impacts of products or However, LCE While productor manufacturing-driven engineers focus their communication anprocesses. ongoing trenddecision of LCA-based digitization products, a limited product or existing families, often fails to and provide adequate support and for actions within a distinct arearange of influence, e.g. product the design of one often fails to provide adequate decision support for actions within a distinct area of influence, e.g. the design of one digitalization, manufacturing enterprises are facing important but also to be able analyze compare to define engineering, e.g. when a comparison between several options component, LCEtocalls forand a tolife cycle products perspective that engineering, e.g. when a comparison between several options component, LCE calls for a life cycle perspective that challenges in today’s continuing new product families. can beand observed that classicalproducts existing needs to be made inmarket productenvironments: development.a Two major encompasses differentItstages, multiple associated needs totowards be made in product development. Two major encompasses different stages, and multipleofassociated products tendency reduction of product development times and product families are regrouped in function clients or features. challenges are faced. The first one lies in applying LCA in a and processes [4]. An interpretation limited to a specific scope challengesproduct are faced. The first one lies in applying LCA in a and processes [4]. An interpretation to a specific shortened lifecycles. In addition, there is and an increasing However, assembly oriented product limited families hardly toscope find. prospective manner to design future products their life could impede holistic feedback aboutareenvironmental prospective manner to design future products and their life could impede holistic feedback about environmental demand of customization, being at the same time in a global On the product family level, products differ mainly in two cycles. This typically requires evaluating a larger number of consequences of engineering choices. cycles. Thiswith typically requiresallevaluating aworld. larger This number of consequences of engineering choices. competition competitors over the trend, main characteristics: (i) the number of components and (ii) the scenarios. Environmental impacts could vary due to changing To overcome these challenges, an approach has been scenarios. Environmental impacts could varymacro due to to changing To overcome these challenges, electrical, an approach has been which is inducing the development from micro type of components (e.g. mechanical, electronical). engineering parameters of the studied product itself suggested to support decision making processes in LCE by engineering parameters of the studied product itself suggested support decision making mainly processes in products LCE by markets, results in diminished lot sizes due to augmenting Classicaltomethodologies considering single product varieties (high-volume to low-volume production) [1]. or solitary, already existing product families analyze the 2212-8271 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license To cope with this variety as wellB.V. as This to be able structure on a physical level (components level) which 2212-8271 © 2019 Theaugmenting Authors. Published by Elsevier is an opentoaccessproduct article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/). (http://creativecommons.org/licenses/by-nc-nd/3.0/). identify possible optimization potentials in the existing causes difficulties regarding Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering (LCE) Conference. an efficient definition and Peer-review under responsibility of the scientific committee of the 26th CIRP Life Cycle Engineering Conference. production system, it is important to have a precise knowledge comparison of(LCE) different product families. Addressing this doi:10.1016/j.procir.2017.04.009 product life cycleDesign through choices about product Keywords: Assembly; method; Family identification
doi:10.1016/j.procir.2017.04.009
2212-8271 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) 2212-8271 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of scientific the scientific committee theCIRP 26thDesign CIRP Conference Life Cycle 2018. Engineering (LCE) Conference. Peer-review under responsibility of the committee of the of 28th 10.1016/j.procir.2019.01.078
718 2
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
means of a visual analytics (VA) cycle [5]. VA aims at a human-centered knowledge building through a combination of automated data analyses and visual exploration. Key characteristics comprise the integration of data, models and visualizations [6]. One technological concept for visualization in VA is Mixed Reality (MR). MR technology is increasingly reaching a state of productiveness that might lead to significant improvements in engineering processes [7]. As summarized by Lindgren et al., embodiment provided through MR powerfully supports cognition processes in engineering [8]. Thus, MR is increasingly applied in manufacturing contexts, e.g. [9]. LCE might leverage from MR technologies manifold. For example, an earlier study identified a high potential for contribution analyses of environmental impacts through a joint visualization of physical products and digital models [10]. However, there are no coherent methods yet regarding the employment of MR technologies in VA for LCE and only few case studies. This paper presents a potential implementation of MR to support LCE activities within an automotive engineering setting. 2. Background 2.1. LCE in automotive engineering design The enrichment of methods in engineering design through life cycle thinking represents an established research approach also known as “ecodesign” or “sustainable design”. From the state of research as well as practical applications McAloone and Pigosso summarize that “[…] a careful and systematic approach to integrated analysis (LCA) and synthesis (ecodesign) is optimal in order to achieve environmentally enhanced solutions […]” [11]. To enable an LCA-based LCE, approaches that streamline the LCA method and/or tightly integrate LCA steps into engineering tools are developed [11]. Examples are tailored LCA tools that are integrated into CAD or PLM systems, e.g. [12]. Other approaches target improved visualizations of LCA results that enable a facilitated interpretation in engineering, e.g. [3]. Automotive LCE further requires coping with technological innovation. One example is the engineering of lightweight vehicle structures. LCE challenges are built around the introduction of new materials or manufacturing processes, including joining techniques, as well as in the selection of appropriate technologies [13,14]. One example is the tradeoff between manufacturing and use stage for electric vehicles, that beyond embodied emissions and weight reductions strongly depends on regional and interindividual factors [15]. 2.2. Visualization in VA-based LCE Engineering problems in general are mathematical representations of physical phenomena [16]. Visualizations in engineering thus aim at depicting those mathematical models to enable the understanding and derivation of actions. Within LCE, one focus for engineering visualization is the interplay of engineering problems within the technosphere, e.g. the effect of increased vehicle mass on engine design in the use stage. Another focus is the relation of engineering decisions to environmental impacts (ecosphere). Within a model-based
LCE, the considered system and its sub-systems are described as a set of mathematical models based on quantified material and energy flows, e.g. shown for electric vehicles in [17]. As reviewed by Ramanujan et al., the number of VA tools in LCE is still very limited [18], but gains increased attention in research. At the interface of design engineering and LCA, the shapeSIFT tool targets the multi-dimensional exploration of a design space based on a repository of materials, manufacturing, functions and shapes. The tool has shown benefits in eco-conscious engineering design by narrowing down options and enabling the interpretation of changes in environmental impacts through parameter variation [19]. The challenge in designing VA tools in specific decision contexts becomes apparent from the development of QuestVis. The tool provides full insight into complex impacts and interdependencies of sustainability-related planning options. However, the initial goal of fostering behavior change was hindered due to the broad information space that itself required teaching [20]. In consequence, Munzner et al. developed a nested model for visualization design and validation [21]. It highlights the importance of domain problem characterization over subsequent steps in visualization design (see Figure 1). The domain problem characterization states that the vocabulary and the exact problem of the target audience needs to be clearly understood. The abstraction translates domain specific tasks and data models to a computer science based description. Both stages together form the foundation of the actual visualization design that consist of the mutual interdependent visual encoding and interaction technique design. The algorithm design includes operative tasks in realization [21]. Ramanujan et al. derived general tasks and design patterns for VA in sustainable design that can be applied to develop respective prototypes [22]. In the sense of the nested model the identified tasks (overview, zoom, filter, details on demand, relate, history, extract) could be understood as expressions of the operation abstraction design, while the design patterns provide LCE-specific encoding and interaction technique designs.
Figure 1: Nested model for visualization design by Munzner et al. [21]
2.3. Mixed Reality in LCE MR is often used interchangeably with the terms Augmented Reality (AR), Augmented Virtuality (AV) or Virtual Reality (VR). Here, we follow the definition stating that when digital and real objects are combined in one application, this application becomes a MR application [10]. Based on Milgram [23] real objects have an actual objective existence, whereas virtual objects exist in essence or effect, but not formally or actually. According to Juraschek et al. [10], based on Azuma [24], an MR application has to be interactive in real time and has to be registered in three dimensions.
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
MR can be illustrated as a two-dimensional continuum stretching from full reality to full virtuality on each side as illustrated in Figure 2 [23]. Between these two exclusive forms of human-interactable environments, MR applications can be allocated according to their share of real (physical) and digital objects. If an application has a higher share of real objects, it will be allocated left from the mid-point of the continuum and thus classified as AR. In the opposite case, an application with a higher share of virtual elements is located in AV. Furthermore, implementation concepts, as for instance immersive VR or tangible interfaces, can be classified in the continuum. Together with the allocation of the respective MR devices the reality-virtuality continuum is a comprehensive framework that can support to analyze, design and implement MR applications.
Figure 2: Reality-virtuality continuum from [23] extended based on [10] with MR capabilities and resulting target area for current study
MR applications can encompass several key capabilities. In the context of LCE, these include displaying virtual and/or real objects, a relation to real objects, interactiveness, instructiveness, immediate and remote collaboration as well as the implementation of temporal scenarios and multidimensional visualizations [10]. These capabilities can also be allocated in the reality-virtuality continuum. Displaying real objects and interacting with them for instance, requires the incorporation of real elements into an application and can formally not be achieved with an occluded head-mounted display (HMD), for example. In such isolated display devices, real objects can only be displayed if they have been sampled and thus digitized [23]. The allocation of key capabilities within the continuum can lead to faster and more coherent design processes for MR applications. 3. Approach and case study This study aims at exploring the potentials of MR through a practically implemented VA approach in LCA-based LCE. The engineering of automotive lightweight components serves as an exemplary case. Thereby, the following steps, based on the nested model, as shown in Figure 1 [21], as well as previous studies on VA and MR in LCE [5,10], are pursued: 1. Domain problem characterization 2. Identification of suitable MR options 3. Abstraction design, visual encoding, interaction technique design and implementation 4. Validation of the MR prototype
719 3
3.1. Domain problem characterization This study follows an LCE approach in the automotive conceptual design stage in general and lightweight body structures in specific. The goal is to design eco-efficient components that should carry lower environmental impacts over the entire life cycle compared to reference designs while meeting technical performance criteria [13]. The design of automotive components is driven by the overall vehicle development. Thus, constraints from the surrounding vehicle, e.g. space or joining technologies, limit the solution space for concept alternatives [25]. In current practice, environmental impacts of body component concepts are assessed upstream from series development and released for different vehicles based on condensed results [25]. However, as reviewed in [14], complex interdependencies on potential environmental impacts call for an in line approach within a specific vehicle development instead. One example are lightweight parts for electric vehicles, where environmental burdens from manufacturing could only be compensated for specific scenarios of vehicle use [15]. Figure 3 explains the setting with two or more domain experts (engineering design and LCA) and one superior project manager. The project manager coordinates and prepares decisions for product concepts based on performance indicators (technical, economic and environmental). The upper part of Figure 3 (A, B1-3, C) illustrates the status quo in component development. Design engineering proposes a set of concept alternatives based on given requirements (B1). This includes different geometries and material combinations [25]. In this case, the example of a roof reinforcement structure is considered. Three different geometries of the cross section (full shape, u-shape, reinforced u-shape) are analyzed that could be either manufactured applying steel or aluminum alloys or carbon fiber reinforced plastics. In addition, rib structures from glass-filled polyamide could be applied as a reinforcement. Technical parameters like wall thickness can be influenced by engineering design. This results in a decision between concept alternatives. Even though many concept alternatives fail due to weak mechanical properties or by violating boundary conditions, several alternatives pass this stage and would be handed over to an LCA expert (B2) [13]. The LCA expert then models several life cycle scenarios for the alternatives. This takes into account different fore- and background systems, e.g. vehicle lifetimes, or electricity production [13]. As a result, project managers need to interpret reports from the domain experts for the concept alternatives with respect to specific assumptions and scenarios (B3). In reality the complexity exceeds the illustrated situation as further disciplines are involved, e.g. development of other parts or focus on specific life cycle stages. The implemented case study now aims at improving the described process through applying a VA-based MR approach as illustrated in the lower part of Figure 3. The approach brings together information from the involved disciplines and their domain-specific tools and makes them available within an explorative environment. In the current case, this encompasses the analytical computation of mechanical performance for different product concepts (BN1). The computation allows a
720 4
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
variation of engineering parameters, e.g. wall thickness or shape. Inventory datasets for determining environmental impacts of different life cycle stages are linked to those engineering models (BN2), as e.g. environmental impacts from manufacturing are directly influenced by choices in design engineering. Beyond that, project managers can explore the design space including different vehicles and lifetime scenarios. This helps to determine if efforts from raw materials and manufacturing stages could be compensated over the vehicle lifetime or not. Finally, a favorable concept alternative can be selected through different KPI (BN3). 3.2. Identification of suitable MR options As explained in section 2.3, the reality-virtuality continuum can support the identification of suitable implementation concepts for the design of MR applications. The previously described user journey serves as a task definition. Subsequently, the MR key capabilities required for the outlined task and subtasks are identified and highlighted in the continuum following the approach described in [10]. The area in which all required capabilities overlap is the target area for the implementation approach and the starting point for the MR application design (Figure 2). As different design alternatives shall be compared that are not yet available as physical prototype or manufactured product, the capability of displaying virtual objects is required. At the same time the comparison with physically available objects should be possible, thus displaying real objects is also a desired capability. Within this product-driven approach, the relation to these real products, i.e. objects, is a desired capability. An application that supports decision processes based on LCA results should allow interaction with data. Furthermore, the application can provide support by giving background information and instruct users through the decision process. LCA results depend on several parameters of different dimensions (e.g. temporal or spatial). Overall, six key MR capabilities were identified for the present goal. From the resulting target area (spatial AR, holographic spatial AR, semi-immersive VR), a holographic spatial AR approach was chosen. The Microsoft HoloLens serves as an implementation platform as it was the best available technology meeting the required capabilities.
3.3. Abstraction design, visual encoding, interaction technique design and implementation The developed application consists of different objects that enable an embodiment of concept alternatives as well as the interaction with underlying data on technical performance, weight and associated environmental impacts. While a roof reinforcement structure represents the sample case, the implementation of other body parts would be possible in parallel following the same workflow and toolchain. Figure 4 shows the design concept of the developed MR application. According to the classification by Ramanujan et al. [22], the application integrates the following design patterns (Table 1): Table 1: Tasks (T) and design patterns (P) according to [22] that are realized within the MR prototype Tasks (T) T: Overview T: Zoom T: Relate
Design patterns (P) Indicator overviews Eco-prominence Eco-persistence Multiscale design exploration (regarding life cycle stages) Linking indicators through the life cycle
The application is currently designed for one user representing a cross-domain project manager and decisionmaker. The implementation consists of certain defining items. Several concept alternatives can be visually explored as 3D models on a virtual table (a). A direct comparison to physical products is enabled (b). Concept alternatives can be selected and are then displayed within a 3D vehicle model to explore potential technical boundaries (c). A user interface enables the iterative adaption of geometries, material alternatives and design parameters (d). In the case of metals this encompasses different alloys, whereas fiber-reinforced plastics e.g. vary in the layer structure of fibers. Drivetrains (combustion, electric vehicles) can be selected to represent potential applications. Each concept alternative can be evaluated regarding mechanical performance (stiffness and strength), weight as well as environmental impacts from raw materials provision, manufacturing and use. A potential break-even between alternatives is highlighted. (e). Iteration enables to understand the interrelations between material and geometry choices and potential environmental impacts.
Figure 3: Conventional workflow in automotive LCE and concurrent approach based on MR and VA
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
721 5
Table 2: Threats and suggested validation methods (V) from [21] and transfer to current study (C)
Figure 4: Design concept and realization of the MR application
The implementation utilizes the game engine Unity as development environment [26]. Within Unity, several libraries are applied. First, vehicle CAD-data (Catia) is prepared for the MR device through reducing model complexity, e.g. holes or roundings. This is necessary to realize a sufficient frame rate when running the application from HoloLens without external computing power. Second, pie and line charts are implemented and adapted for through the third party “Graph and Chart” library [27]. The application uses a structured database that has been realized as a set of csv files that can be directly updated from expert tools in LCA (Umberto LCA+) and design engineering (analytical design tool, Matlab). Material information, geometry (wall thickness, width, height) and component weight were used as main interfaces. As the aim of this study was to create a prototype that can be iterated to enhance the quality with respect to data/ operation abstraction and encoding/ interaction technique design, live interfaces to the expert tools have not been realized. Thus, a manipulation of parameters exceeding pre-calculated inputs is not possible now. A major step towards an industrial implementation would lie in a tight integration of the workflow into respective IT systems. The Unity application requires regular updates to most recent data. PLM systems could provide geometry files and material information that can be exported via scripts. In parallel, LCA data, e.g. specific footprints of manufacturing, need to be updated and contextualized regularly. 4. Validation The MR prototype was validated following the layers of the nested model. Munzner et al. identified main threats and validation goals for each of the layers [21]. On that basis, Table 2 points out derived actions for the current study. The domain problem characterization is built up on previous studies [13,14,25]. In [14] the complexity in LCE of future automotive structures has been highlighted and the need for innovative design tools was expressed. However, the targeted decision situation is currently limited to research-driven activities, as an integration into series development would require an adaption of formalized business processes [25]. Thus, actual adoption rates could not be tested. However, the prototype is made available within a jointly operated research environment between industry and academia. The algorithm design validation in terms of performance showed positive results, as the prototype performs fast at the current stage, but computational complexity will significantly increase with further models covering different stages of the vehicle life cycle, as e.g. shown in [17].
Domain problem characterization Threat: wrong problem V: interview target users C: problem definition based on [13,14,25] V: observe adoption rates C: medium-term goal, exceeding scope Data/ operation abstraction design Threat: bad data operation/ abstraction V: test on target users, collect C: anecdotal evidence from user test anecdotal evidence of utility V: field study, document human C: medium-term goal, exceeding scope usage of applied system Encoding/ interaction technique Threat: ineffective encoding/ interaction technique V: justify technique C: first impressions from user test V: qualitative/ quantitative C: medium-term goal, exceeding scope result image analysis V: lab study, measure human C: qualitative evidence from user test time/ errors for operation Algorithm design Threat: slow algorithm C: implementation currently with low V: analyze computational computational complexity; needs to be complexity regarded in the course of industrial V: measure system time/ implementation projects memory
Anecdotal evidence with regard to “data/ operation abstraction design” and “encoding/ interaction technique” was collected in a user test including 10 experts in industrial research and engineering of automotive components. The test asked to identify an eco-efficient concept alternative through iteration. Feedback was collected using a questionnaire, including structured and open questions. Due to the small sample size, the findings represent impressions and starting points for further evolution. Initially, all test users were introduced to the MR device as inexperienced users need to get acquainted to the mixture of reality and virtuality at first (5 to 10 minutes per user). Although it is one of the most powerful available holographic HMD, the first HoloLens still has a limited field of vision and user interactions. A test of human time and errors is thus not beneficial for the current stage. However, all users were able to perform the required operations (average time 10 minutes) and evaluated that the app in general was easy to use (Table 3).
Table 3: Feedback from user test, sample size: 10, only positive formulations listed, red line represents average
The visual representation and interaction through 3D models, sliders, drop-down menus and graphs overall turned out to be useful for the given task. Most test users noted its utility for understanding interfaces of the component to the vehicle and finally solving the task. However, within the visual representation of concept selection, users desired a more interactive manipulation of the 3D models and underlying
722 6
Alexander Kaluza et al. / Procedia CIRP 80 (2019) 717–722 Kaluza et al. / Procedia CIRP 00 (2019) 000–000
parameters. Reported shortcomings within the workflow include options to save evaluated parameter combinations as a reference point. This limits the effectiveness of the explorative activity, as omitted alternatives cannot be documented in a structured way. In addition, users asked for more details to understand the underlying engineering models. This leads to the conclusion that interrelations between design attributes and resulting environmental impacts are not transparent to the desired degree and trust in the results might be limited. Further, several users reported that it is difficult to identify if an alternative is a global optimum within the decision space. 5. Summary and Outlook The current research presents an approach to introduce MR in a VA-based LCE process. The design of eco-efficient lightweight vehicle components serves as a sample case. Domain experts showed great interest in generating a common understanding between component design engineering and LCE, as the topic is gaining importance. The embodiment provided through MR was evaluated positively. However, the prototype showed shortcomings in promoting decisions and misses potentially useful design patterns. First steps for future research include improving overall usability, implementing the aforementioned design patterns and exploring suitable methods from decision sciences to handle conflicting development goals. To enable an experience closer to industrial practice, the functionality needs to be extended to analyze multiple parts at the same time. Another major step would be a near real time coupling of the MR visualization to tools from LCA and engineering design. This would exceed the current focus of reviewing interdependent results towards an actual collaborative engineering setting. Future advanced MR solutions thereby enable a richer exploration of design and life cycle scenarios. Acknowledgements The results published in this paper are based on the project MultiMaK2 aiming at developing engineering tools towards designing eco-efficient vehicle components. This research and development project is funded by the German Federal Ministry of Education and Research (BMBF) within the Research Campus Open Hybrid LabFactory and managed by the Project Management Agency Karlsruhe (PTKA). The authors are responsible for the content of this publication. References [1] Alting L. Life Cycle Engineering and Design. CIRP Ann - Manuf Technol 1995; 44: 569–580. [2] Laurin L, Amor B, Bachmann TM, Bare J, Koffler C, Genest S, Preiss P, Pierce J, Satterfield B, Vigon B. Life cycle assessment capacity roadmap: decision-making support using LCA. Int J LCA 2016; 21: 443–447. [3] Cerdas F, Kaluza A, Erkisi-Arici S, Böhme S, Herrmann C. Improved visualization in LCA through the application of cluster heat maps. Procedia CIRP 2017; 61: 732–737. [4] Hauschild MZ, Herrmann C, Kara S. An Integrated Framework for Life Cycle Engineering. Procedia CIRP 2017; 61: 2–9. [5] Kaluza A, Gellrich S, Cerdas F, Thiede S, Herrmann C. Life Cycle
Engineering Based on Visual Analytics. Procedia CIRP 2018; 69: 37–42. [6] Keim DA, Mansmann F, Stoffel A, Ziegler H. Visual Analytics. In: Encycl Database Syst. Springer; 2009. p. 27–37. [7] Wang X. Editorial visualization in engineering. Vis Eng. 2014; 2: 1–2. [8] Lindgren R, Johnson-Glenberg M. Emboldened by Embodiment: Six Precepts for Research on Embodied Learning and Mixed Reality. Educ Res 2013; 42: 445–452. [9] Makris S, Karagiannis P, Koukas S, Matthaiakis A. Augmented reality system for operator support in human – robot collaborative assembly. CIRP Ann - Manuf Technol 2016; 65: 61-64. [10] Juraschek M, Büth L, Cerdas F, Kaluza A, Thiede S, Herrmann C. Exploring the Potentials of Mixed Reality for Life Cycle Engineering. Procedia CIRP 2018; 69: 638-643. [11] McAloone TC, Pigosso DCA. Ecodesign Implemtation and LCA. In: Hauschild MZ, Rosenbaum RK, S.I. Olsen SI, editors. Life Cycle Assessment. Cham: Springer International Publishing; 2018. p. 545–576. [12] Gaha R, Yannou B, Benamara A. A new eco-design approach on CAD systems. Int J Precis Eng Manuf. 2014; 15: 1443–1451. [13] Kaluza A, Kleemann S, Fröhlich T, Herrmann C, Vietor T. Concurrent Design & Life Cycle Engineering in Automotive Lightweight Component Development. Procedia CIRP 2017; 66: 16–21. [14] Herrmann C, Dewulf W, Hauschild M, Kaluza A, Kara S, Skerlos S. Life cycle engineering of lightweight structures. CIRP Ann - Manuf Technol 2018; 67: 651–672. [15] Egede P. Environmental Assessment of Lightweight Electric Vehicles. Cham: Springer International Publishing; 2017. [16] Li W, Nee AYC, Ong SK. A State-of-the-Art Review of Augmented Reality in Engineering Analysis and Simulation. Multimodal Technologies Interact 2017; 1(3):17. [17] Cerdas F, Thiede S, Herrmann C. Integrated Computational Life Cycle Engineering - Application to the case of electric vehicles. CIRP Ann Manuf Technol 2018; 67: 25–28. [18] Ramanujan D, Bernstein WZ, Chandrasegaran SK, Ramani K. Visual Analytics Tools for Sustainable Lifecycle Design: Current Status, Challenges, and Future Opportunities. J Mech Des 2017; 139: 111415. [19] Ramanujan D, Bernstein WZ, Kulkarni Devadatta JT, Ramani K. shapeSIFT: Evaluating InfoVis Tools for Eco-Conscious Design. 21st Design for Manufacturing and the Life Cycle Conference 2016. [20] Munzner T, Barsky A, Williams M. Reflections on QuestVis: A Visualization System for an Environmental Sustainability Model. Dagstuhl Follow 2011; 2: 259. [21] Munzner T. A nested model for visualization design and validation. IEEE Trans Vis Comput Graph. 2009; 15: 921–928. [22] Ramanujan D, Bernstein WZ, Ramani K. Design patterns for visualization-based tools in sustainable product design. 22nd Design for Manufacturing and the Life Cycle Conference 2017; DETC2017-68054. [23] Milgram P, Kishino F. A Taxonomy of Mixed Reality Visual-Displays. IEICE Trans Inf Syst 1994; E77d: 1321–1329. [24] Azuma RT. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 1997; 6(4): 355–385. [25] Kaluza A, Kleemann S, Broch F, Herrmann C, Vietor T. Analyzing decision-making in automotive design towards life cycle engineering for hybrid lightweight components. Procedia CIRP 2016; 50: 825–830. [26] Unity Technologies, Unity, unity3d.com. [27] Graph And Chart, Prosource Labs, assetstore.unity.com/packages/tools/gui/graph-and-chart-78488.