Computers in Industry 101 (2018) 41–50
Contents lists available at ScienceDirect
Computers in Industry journal homepage: www.elsevier.com/locate/compind
Using virtual reality to support the product’s maintainability design: Immersive maintainability verification and evaluation system
T
⁎
Ziyue Guoa,b,c, Dong Zhoua,b,c, , Jiayu Chena,b,c, Jie Genga,b,c, Chuan Lva,b,c, Shengkui Zengb,c a
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, 37th, Xueyuan Road, Haidian District, Beijing, 100191, PR China School of Reliability and Systems Engineering, Beihang University, 37th, Xueyuan Road, Haidian District, Beijing, 100191, PR China c State Key Defense Science and Technology Laboratory on Reliability and Environmental Engineering, Beijing, 10019, PR China b
A R T I C LE I N FO
A B S T R A C T
Keywords: Virtual reality Maintainability Product Design
Maintainability is an important characteristic of complex products. Good maintainability design can save substantial costs and reduce incidents and accidents throughout the product life cycle. Therefore, maintainability should be given full consideration in the early design stages. However, the current desktop tools are not effective to aid in maintainability design. Virtual reality (VR), a state-of-the-art of computer science, generates a virtual environment in which the user can have intuitive feelings and interact with virtual objects. In this paper, an immersive maintainability verification and evaluation system (IMVES) based on virtual reality is proposed. The goal of the system is to develop a cost-effective, rapid and precise method to improve the maintainability design in the early design stages. IMVES enables the user to interact with maintenance objects and conduct immersive simulations. Based on the developed methods, the data generated during simulation can be gathered to analyze the maintainability status. A case study applying IMVES into an aero-engine project is presented to demonstrate the effectiveness and feasibility of the system. Compared with desktop-based methods, the IMVES could provide a more efficient way to conduct a maintenance simulation. Furthermore, the credibility and objectivity of the maintainability evaluation are also improved.
1. Introduction As the major capital goods that underpin manufacturing, services, trade and distribution, complex products play a critical role in modern industrial and economic progress [1]. These products greatly enhance the efficiency of human activities, whether for civil or military purposes. With a high use ratio and rapid technical improvement, these complex products generally are large scale and highly integrated, leading to the maintenance and support of these products becoming a big issue. Maintainability is an important built-in characteristic of complex products. From a high-level point of view, the maintainability can be defined as “the probability that a failed item can be restored to an operational effective condition within a given period of time” [2]. From a more operable perspective, maintainability refers to measures or steps taken during the product design phase to include features that will increase the ease of maintenance and ensure that the product will have minimum downtime and life cycle support costs when used in a field environments [3]. The issue to improve the maintainability is becoming increasingly more important than ever before because of the alarmingly high costs of operation and support and the incidents and
⁎
accidents caused by incomprehensive maintainability design [4]. Good maintainability design enables the maintenance process to be conducted efficiently. Furthermore, good maintainability design can reduce costs throughout the product life cycle and reduce the incidents and accidents. Poor maintainability design will lead to the frequent replacement of components and long downtime of products; this outcome will lead to significant economic loss. The study shows that nearly 70% of the total product life cycle costs are determined in the early design stages [5], in which maintenance costs occupy a large part. and For example, the industry in the United States spends over $300 billion on plant maintenance and operations [6]. The annual cost of maintaining a military jet aircraft is approximately $1.6 million. Approximately 11% of the total operating cost for an aircraft is spent on maintenance activities [7]. Therefore, the cost of maintenance will be greatly reduced during the service stage if the product has a rational maintainability design, which will put great value on both users and enterprises. Another significant issue that needs significant attention is that the maintainability design is strongly associated with personnel security, especially for the fields that have a high degree of potential for accidents such as aviation, nuclear and mining industries. In civil
Corresponding author at: Beihang University, 37th, Xueyuan Road, Haidian District, Beijing, 100191, PR China. E-mail address:
[email protected] (D. Zhou).
https://doi.org/10.1016/j.compind.2018.06.007 Received 23 June 2017; Received in revised form 1 May 2018; Accepted 26 June 2018 Available online 04 July 2018 0166-3615/ © 2018 Elsevier B.V. All rights reserved.
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
behind the whole design schedule. However, as the replacement, digital mockup (DMU) still has some efficiency problems. Among the industry sectors (including aerospace, aircraft and high-speed rail), mature simulation tools such as DELMIA (Digital Enterprise Lean Manufacturing Interactive Application) and JACK (a human modeling and simulation tool) have been widely used. Although these tools realize the visualization of the DMU and simulation of the maintenance process, this kind of maintainability design method produces unsatisfactory effects. The following major factors account for such low efficiency:
• Current design-aiding tools generally consist of desktop software. Graph 1. Comparison of the death toll due to various accident and incident factors.
•
aviation, maintenance is a crucial routine function in day-to-day operations. We investigated air crashes worldwide from 2001 to 2012. The incident and accident causes are classified as flight crew (FC), airplane (AI), ATC & Navigation (ATC&N), weather (WE), maintenance (MA), external factors (EF), landing/takeoff (L/F), unexpected result (UR), security (SE), cargo occurrences (CAOs), fire (FI), unknown (UN) and collisions (CO). Graph 1 shows the death toll caused by incidents and accidents, showing that the airplane is the second leading factor causing deaths (1539 deaths), and maintenance is the third leading factor in deaths (1220 deaths). Maintainability design has a strong correlation with both maintenance and airplane factors. Comprehensive maintainability design in early design stages of airplanes enables the maintainer to conduct the maintenance task in an efficient and comfortable way. The maintenance errors will be substantially reduced, meaning that the damages resulting from the maintenance errors will be kept to a minimum or totally avoided. Regarding the significance of the maintainability of complex products, the maintainability design should be conducted more effectively in the early design stage. VR is a state-of-the-art of computer science. In many fields, VR has played effective roles both for academic research and industrial application [8]. In this paper, an immersive maintainability verification and evaluation system (IMVES) is proposed. The dual aims of IMVES are to verify the feasibility of applying VR to assist maintainability design and further, to develop a timely and effective approach to help the designer make better design decisions in early design stages. IMVES is purposefully designed to generate and conduct an immersive maintenance simulation, in which the designer can interact with virtual objects according to preplanned procedures, and the data related to the maintenance process can be collected to make further maintainability evaluations. IMVES has a general framework, which means it can be used in different design situations after proper modification. In addition to the description of the system architecture and functional components of the IMVES, this paper also describes two cases in which IMVES was used to help the designers to make design decisions for an aero-engine and a helicopter in the early design stages. Based on the analyses of the evaluation results by IMVES and later physical verification, the paper thus highlights that VR is feasible for practical product design and an effective approach to help designers make reasonable decisions.
•
These tools can only display the DMU in a static way. Nevertheless, maintenance is a typical example of human-machine interaction, which is a dynamic process. A desktop-based view cannot oversee the design shortcomings that are exposed during the dynamic process, although these desktop tools can make some simulations where a virtual human can be involved. In other words, these tools still are “two-dimensional” to a certain extent. The desktop-based simulation has a low production efficiency, running counter to the speed requirement in new product development, which leads to maintainability design in a superficial way. The desktop-based simulation is commonly based on animation (the main aspect is key frame control). The elaboration level of the simulation depends on the quantity and accuracy of the key frames. To make the simulation delicate enough to reflect the real maintenance process, the designer must adjust the key frame many times. This kind of production method costs time and strength. A representative case is that, in one of our prior projects, we produced a simulation to simulate the process of pulling off a cable plug in satellite capture. The whole simulation play time is just approximately two minutes and thirty seconds. Despite knowing the Human Task Simulation module of DELMIA well, we still spent two whole days to produce the simulation to make it as precise as possible. For the common engineer, it is a heavy burden added to their design task. Traditional maintainability design methods rely too much on experience, which has considerable subjectivity. This issue is exposed throughout the desktop-based maintainability design process. The traditional maintainability design method is mainly simulationbased. Because designers cannot be involved in the maintenance process, they are just “cartoon-makers”, and this lack of involvement in the maintenance process will lead to a situation in which, given the identical maintenance task, different designers will produce different simulations. Well-experienced designers can make a precise simulation that is close to the real maintenance process. Nevertheless, the simulation made by a less experienced designer may have large deviations, which lead to errors in the subsequent evaluation.
The potentials of VR could bring innovation to the current maintainability design method. With the assistance of VR, we can quickly duplicate a dynamic virtual environment that could be seen as the image of the real maintenance case. In the appropriate environment, the user, rather than a virtual human, can interact with the virtual environment. The advantages VR bring to virtual maintenance simulation are obvious: 1) The DMU can interact with the real user. 2) The simulation could be conducted rapidly because of the fundamental change in the simulation method. 3) The accuracy of the simulation could be greatly improved.
2. The issues with current maintenance design methodsmotivation analysis
3. Related work Maintainability is a significant part of the design stages. For some large complex products, especially the weapons, maintainability design is even a compulsory design content that needs to be carefully conducted. At present, the industry design department has phased out expensive mental mockups, which will cause the maintainability to lag
VR is the state-of-the-art in computer science. Using advanced display and interaction technologies, VR enables the user to have sensual perceptions and interact with the virtual object. Virtual maintenance (VM) serves an important role in maintainability design and 42
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
Fig. 1. Architecture of IMVES.
importantly, usable to help in product design, support decision making and bring innovations. At the end of the paper, the authors also note the challenges before VR can shine brilliantly in industry. There are also several studies on applying VR in routine maintenance work and maintenance training. In paper [23], the authors proposed the Virtual and Augmented Reality Integrated Development Lab (VARID-Lab), a remote handling (RH) system, to help in the inspection and maintenance of the tokamak so that the operator can complete these processes in a safe environment. VARID-Lab has been successfully used in phases of RH development and operation. Paper [24] proposed a 3D workforce training system for energized high-voltage power line maintenance (ALEn3D AT). This work aims to provide a tool to learn the methodology and regulations related to live line maintenance in a risk-free environment. The results showed that workforce training using ALEn3D AT has a positive effect in both theoretical and practical evaluations. The authors also note that ALEn3D AT does not substitute for the live high-voltage line training and field practice because it is a high-risk task. As to the VR applications in maintainability design, there seem to be no pertinent and explicit studies on it. The existing applications using VR on related design work are simple utilization of VR concerning a single aspect such as evaluating visibility or reachability. However, the maintainability design is a systematic effort which consists of many aspects. Feasible evaluation methods are also scarce when the traditional maintainability method comes into the novel VR technology. However [25], notes the positive impact of VR on product design: immersive environment engages the designer more actively in design and provides better support to the design decision. In summary, we have reason to believe that VR technology can help maintainability design, one of the motivations for the preparation of this paper.
maintenance training of the product. VM is defined as the application of virtual simulation-based engineering, which enables engineers to evaluate, analyze, and plan the assembly of mechanical systems [9]. VM should satisfy several functional requirements [10,11]. Generally speaking, VM has gone through two stages. The first stage is a desktopbased simulation. In this stage, maintenance simulations are performed by desktop software such as DELMIA and JACK. Extensive literature on VM methods and applications is presented. Kevin J. Abshire reported how VM is applied in the design of the Lockheed Martin Tactical Aircraft System’s (LMTAS) F-16 program in [12]. In this paper, he summed up the benefits of VM for the product design. He noted that VM can prevent costly redesign and accommodate shortened schedules better than metal mockups. The visualization of VM provides a better exchange of information for engineers. Nanyang Technological University developed a Virtual Maintenance Simulation Desktop System for virtual prototypes, which integrated a new disassembly sequence planning technology and optimization algorithms [13]. Sanchez proposed a novel ergonomic postural assessment method (NERPA) to assess the ergonomics of a workplace and demonstrated the effectiveness of the proposed method in DELMIA [14]. Virtual simulation and analysis software represented by DELMIA and JACK have been applied extensively in maintainability design, analysis and verification [15–18]. VR technology opens the second stage of VM. VR has been widely used in varying disciplines and fields such as medical science [19], education [20], surgery training [21], and the nuclear industry [22]. Academic and industrial communities have studied virtual reality for decades because of its distinctive characteristics. The paper [8] presents a systematic and complete survey about the industrial application of VR in product design and manufacturing. The authors interviewed 18 companies using VR in practical product designs and reached out to 35 VR laboratories. Sixty-two people from multiple disciplines were encouraged to share their perspectives and experiences regarding VR applications. There are many successful cases as well as existing challenges. With the great strides of VR hardware, VR applications and the associated decision making are given unique attention. The paper draws the conclusion that VR is becoming mature, stable, and, most
4. Overview of the IMVES The architecture of IMVES is shown in Fig. 1. Based on the work process, IMVES can be divided into three modules: immersive simulation module, supervision and management module and evaluation 43
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
characteristics of the product, usually without kinetic characteristics. Virtual scene data are the data that describe the whole maintenance scene, including the information of the maintenance object, maintenance tools, maintenance procedure and the maintenance personnel. Virtual scene data are generated when the designer finishes the simulation preparation. Interaction data and motion capture data are the data produced when the simulation personnel have interactions and behavior in the virtual environment. These data record the interaction and motion details during the maintenance simulation such as the interference situation when opening the maintenance window, and the motion data when making a posture. Anthropometric data describe the body size of the maintenance personnel and can be adjusted if the size of the personnel changes.
module. Among these modules, the data is transferred by data channel based on TCP/IP and virtual-reality peripheral network (VRPN) protocol. The main function of the immersive simulation module is to carry out immersive maintenance simulation. The cave automatic virtual environment (CAVE) provides the visualization of the VM environment. Tracked 3D glasses could track the change in the view and provide the corresponding 3D vision. Then, with the aid of a joystick, the user can interact with the maintenance objects. The functions include navigation, disassembly, sectioning, etc. In the process of simulation, the interaction and behavior data can be recorded and transferred to an evaluation module for the following maintainability evaluation. The supervision and management module is designed to achieve two functional goals. Before the immersive simulation, this module is responsible for the simulation scene development and initiating service. In the process of simulation, this module can monitor the whole simulation process. According to the real-time requirements, the supervisor can adjust the simulation scene or interact with the virtual environment collaboratively with simulator personnel. The evaluation module provides the function to evaluate the product maintainability. Based on the evaluation methodologies and interaction data generated in the simulation, the evaluation module can evaluate the product maintainability from the aspects of visibility, reachability, maintenance space, ergonomic analysis, maintenance time and maintenance safety. More detailed description will be given in the following part.
4.2. Hardware realization To realize all functions of IMVES, several hardware facilities are required to support the operation, from integrated systems to single devices. The selection of these devices is challenging because inappropriate device combinations may cause an incompatible result. Fig. 3 shows the part of the IMVES. CAVE [26] is selected as the visualization environment to conduct the immersive maintenance simulation. In IMVES, we use a four-channel CAVE. For the maintenance process, CAVE provides a wider field of vision than other head-mounted displays, which can provide more visual information to the user. Furthermore, in CAVE, the user can observe the scene without cumbersome display devices (only a pair of glasses is required), which lightens the burden of the user and ensures the accuracy of the ergonomic evaluation. In addition, the user can move a certain distance to interact with the virtual environment in CAVE, which greatly improves the authenticity of the simulation. We use the optical motion capture system to collect the motion data generated in the immersive simulation. Eight cameras have a 60 ° field of view and a high resolution and have the ability to track very high-speed motion at up to 960 frames per second, which can cover the whole environment and collect exact motion data. We use a joystick and active three-dimensional (3D) glasses as interaction tools. The function of the joystick is to enable the design to interact with the virtual environment such as disassembling objects, marking points and measuring distance in a virtual maintenance environment. The achievements of the high stability and efficiency of IMVES require powerful graphic processing and computing abilities. Five high performance workstations were selected to provide the computing service for IMVES. These workstations make up a high-performance computer cluster (HPC-Cluster). One is the master computer, and the others are rendering computers. The actual performance demonstrates they are feasible and reliable to execute the graphic processing and computing tasks.
4.1. Function-based working process Two main functions of IMVES are immersive simulation and evaluation, which are necessary to complete the whole maintainability design process. Based on the purpose, the hierarchy for the use of IMVES is organized into seven levels, as shown in Fig. 2. IMVES is an open platform. The friendly usability design of IMVES makes the users not limited only to the professional: the customer and maintainer can also participate. The attendance of the customer and maintainer means that the practical maintenance experience is brought to the design stage, which is of great significance to the maintainability design. To operate IMVES in a more effective way, three interfaces are set. The immersive simulation interface and the supervision and management interface interlink with each other. Only the supervisor grants the display and interaction authority to the immersive simulation interface so that the immersive simulation can be conducted. In addition, the supervisor is responsible for development, management and supervision of the simulation. Until the abovementioned work is finished by supervision and management terminals, the user of the immersive simulation interface can conduct the simulation. With the help of tracked glasses, the joystick and the tracking suit, the user can make immersive and interactive simulations in the virtual environment. Some preliminary sensory evaluations can be realized by the simulation functions. The evaluation interface provides the access to make comprehensive evaluation of the maintainability of the product. Maintainability evaluation software is the core of the interface. With the assistance of simulation personnel, the evaluation personnel can make maintainability evaluations term by term during the immersive simulation. When the simulation is finished, a general evaluation reports that the product maintainability will be generated automatically by the evaluation software. Several evaluation methods have been realized and embedded into this software. As mentioned above, one of the advantages of employing VR to make immersive and interactive simulations is that abundant data can be collected. These data are highly valuable for the follow-up evaluation and verification. The main data types are digital prototype data, virtual scene data, interaction data, anthropometric data and motion capture data. These data are the basis of the simulation and the inputs to the evaluation. The digital prototype data contain the geometric
5. Key elements Immersive maintenance simulation enables the user to interact with virtual objects in a more natural way, which entails that an immersionoriented system is required to develop, supervise and manage the immersive simulation. Several methods have been developed to realize that goal. In IMVES, these methods could be classified as general methods and dedicated methods. General methods mean the methods that realize the basic functions of a VR system. For IMVES, these methods are: 1) DMU import and lightweight treating, 2) multi-channel stereo display, 3) device and operation binding, and 4) constraint and physics management. Fig. 4 shows the administrative user interface of IMVES.
• DMU import and lightweight treating: IMVES supports local import and online import of DMU. Local import refers to importing a DMU
44
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
Fig. 2. Hierarchy of IMVES.
•
from a local computer. Online import refers to importing a DMU directly from an opened 3D modeling software. Traditional DMU contains a mass of dimension parameters resulting in a huge data volume, which seriously affects the simulation efficiency. Therefore, after the DMU is imported, the lightweight treatment will be conducted to reduce model data size, making the model suitable for immersive maintenance simulation. Multi-channel stereo display: Traditional 3D modeling software is displayed on the flat screen, which does not support multi-channel stereo display. However, to carry out immersive simulation, stereo display is of the essence to offer the user realistic visual input. Stereo display works in a PC-Cluster architecture in IMVES. The nodes of the PC-Cluster are divided into one management node and several rendering nodes. The management node is responsible for the
•
• 45
calculation and distribution of the simulation task. The rendering nodes receive the tasks assigned by the management node and rendering graphic. Device and operation binding: As to a maintenance simulation and verification, the main interaction commands are navigation, dismounting, sectioning, distance measurement and marking feature points. Based on the VRPN protocol, these interaction commands are bounded with the joystick controller. When the user intends to interact with a virtual object, the user just needs to move to analog or press the button to trigger the corresponding command such as wandering in a virtual environment and dismounting parts. Constraint and physics management: For the interactive VM simulation, the lack of physical authenticity is a very significant issue. Whether the motions of virtual objects follow the actual kinematics
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
Fig. 3. Hardware of the simulation environment.
Fig. 4. User interface of management and rendering nodes of IMVES. 46
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
has great influence on the authenticity, immersion and interactivity of simulation. IMVES has the ability to define physical properties for DMU to prevent object penetration between two objects when collision occurs. Constraints of kinematics assigned to the virtual objects make sure the movements of virtual objects are close to the real objects. When the setup of the physical and kinematic configuration is complete, real kinetic characteristics of virtual objects will be activated in the interactive simulation. Dedicated methods refer to the methods that are specially developed in IMVES. These methods realize the functions in which IMVES has the ability to conduct maintainability verification and evaluation. 5.1. Motion data processing The significance of using motion capture in IMVES is dual. On the one hand, motion capture can provide tracking and positioning functions to the interaction commands and users. Therefore, the commands can be accurately enacted on the object, and the user can move naturally in the virtual environment. On the other hand, the captured motion data of the user behaviors are further utilized to evaluate the ergonomic status and control the virtual human. Because the motion data captured by optical motion capture system cannot be utilized directly, we developed a rapid motion data processing method in IMVES. The purpose of this method is to transform the format of the motion data captured by sensors into a specified format that can be utilized. For ease of description, we take the case of how the motion data captured by the motion capture system are utilized to drive the manikin in DELMIA as an example to show the processing step of the method. Step 1: A user wearing a motion capture suit moves naturally or has actions in the virtual environment. Each unique LED on the suit transmits a different frequency infrared ray, and it is captured by cameras. After the calculation, the 3D coordinates of each LED in the virtual environment can be derived. Step 2: By the VRPN Server (it is provided by PhaseSpace Inc., sending captured motion data to the outside), we can get the raw motion data, which contains the 3D coordinates of all LEDs that are fixed on user’s body. Step 3: Because of the differences between the data formats, the raw motion data cannot be used directly to control the virtual human in DELMIA. Therefore, in this step, we need to transfer the format of the raw data into the format which can be received by DELMIA. The raw data are transmitted in the format of the user datagram protocol (UDP) socket and received by proposed method. Step 4: The data in last step are temporarily saved as the current frame. The format of the raw data can be converted into the format that DELMIA can use. The converted data are temporarily stored in two arrays, namely, opv array and iAxisComponentsArray array. This step can be further subdivided into a) calculating the direction and position of the human body, b) calculating the posture of each body segment (excluding the fingers), and c) calculating the posture of each knuckle. Step 5: DELMIA provides Automation API to make secondary development. Based on the interface, we can call functions of Automation API to assign the action data for the virtual human. Two main functions we need are SetPostureValues and SetPosition. The former can manipulate the posture of the virtual human, and its parameter is the opv array. The latter can manipulate the orientation and position of virtual human, and its parameter is iAxisComponentsArray array. After the assignments, the actions of the virtual person in DELMIA are consistent with the user actions. The above steps are a general workflow of the motion data processing method. To achieve this round, some additional work needs to be accomplished. For example, a parameter configuration is needed to match the identification number of the active LEDs with body parts. Fig. 5 is a complete cycle of the data processing method.
Fig. 5. A complete cycle of the development data processing method.
5.2. Maintainability evaluation method One of the advantages of using VR to carry out maintenance simulation is that much of the human-involved data is generated in the process of interaction. As for IMVES, the generated data could be utilized to evaluate the maintainability of the product. Integrating these data with valid evaluation methods, some aspects of maintainability of the product could be evaluated quantitatively and precisely, rather than qualitatively and experientially. 5.2.1. Accessibility For practical maintenance activities, accessibility is quite a significant factor. Whether the maintainer has good vision on an object, an unobstructed corridor to approach the object and capacious space to perform maintenance operations has a great influence on maintenance efficiency. In IMVES, visibility can be analyzed based on collaboration between the user and the software. When the user moves around or adjusts the viewpoint, the changes in eye view can be tracked and uploaded to the evaluation software. Visibility can then be evaluated based on the visual cone and criteria. Table 1 shows the criteria for visibility. Reachability is evaluated by the envelope ball. When the user in IMVES performs an action, the corresponding envelope ball of the upper limb is generated based on motion data and anthropometric data. For maintenance space, a quantitative analysis method for maintenance space has been proposed. Maintenance is a combination of a series of basic Table 1 Criteria for visibility. Score
Description
Level
5 4
Target is in the optimum visual range, excellent Target is in the maximum visual range but can be changed into the optimum by posture adjustment, good Target is in the invisible range but can be changed into the optimum by posture adjustment, neutral Target is in the invisible range but can be changed into the maximum by posture adjustment, neutral Target is in the invisible range and cannot be changed into the maximum by posture adjustment, poor
I II
3 2 1
47
III IV V
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
of the engine. Based on the investigation, we analyzed the maintenance tasks for the engine and determined the virtual simulation implementation in IMVES. The whole implementation includes:
therbligs. When the maintainer performs a maintenance motion such as push, screw or twist, the motion data can be captured. These data can be used to calculate the constraint swept volume vcsv, which represents the real swept volume constrained by the spatial arrangement of the product. Then, based on the maintenance tools and anthropometric data, the expected swept volume vesv of each motion can be calculated, which represents the optimal space in which the maintainer can comfortably complete the maintenance task. Finally, with consideration of the operation range of the upper limb, the maintenance space could be quantitatively evaluated. A detailed description of the theoretical basis can be found in [27].
• Setup of the virtual simulation environment: Some preparation work is performed in DELMA. The simulation will be conducted in IMVES. • Processing of the DMU: The DMU of aero-engine is treated as •
5.2.2. Maintenance time and ergonomic analysis A VR-based method to predict maintenance time and analysis ergonomics is integrated into IMVES. In this method, the whole maintenance task is decomposed into several subtasks. Giving consideration to machine operations, the human actions are also decomposed into a number of basic therbligs. After the analyses of the maintenance task and resource, the maintenance simulation can be conducted. A corrective measurement time method (MTM) is proposed to predict maintenance time. Ergonomics analysis can be conducted based on an incident matrix. A detailed description of the theoretical basis can be found in [28].
• •
5.2.3. Maintenance safety Maintenance safety is a serious issue that receives much attention. Renewal of products, a large quantity of maintenance work and high utilization ratio wear out the maintainer, which increases the possibility of safety incidents. In IMVES, we developed a maintenance safety evaluation model. This model consists of an evaluation element, evaluation criteria, and evaluation method. The model considers the virtual reality environment for the maintenance simulation and takes full advantage of the data provided by the immersive interactive simulation. The user can perform static and dynamic checks in the environment to provide information to the safety evaluation model. A detailed description of the theoretical basis can be found in [29].
lightweight. Furthermore, key connecting pieces are attached with actual kinematical constraints, which makes the mechanical motion close to real. Virtual maintainer and maintenance resources: The body size of the virtual maintainer is determined. Males under the 50th percentile are chosen for the simulation. The maintenance resources such as tools, ladders and cranes are also determined. Determination of the maintainability evaluation items: The evaluation of the maintainability will be carried out from six aspects: visibility, reachability, maintenance space, maintenance safety, error prevention and ergonomics. Maintenance operation procedures: Based on the maintenance manuals and the experience of the maintainers at the maintenance depot, the maintenance process and operations in the virtual simulation are determined.
According to the determined implementation, the VM simulation, evaluation and verification of the aero-engine are performed in IMVES. Fig. 7 shows parts of the verification in IMVES. The disassembly task of the key components such as the generator, carburetor and fire detector are conducted. A total of 46 design defects were discovered by IMVES. These problems were classified into three levels: Level I. The maintenance task cannot be completed due to the existence of these problems that must be solved; Level II. The resolution of such maintenance problems will lead to other changes such as increased weight or reduced performance or function. A further trade-off analysis is needed to determine whether and how to perform the improvement measures. Level III. These problems will not influence the maintenance task. However, solving such problems will improve the maintainability of the product. Whether to solve these problems depends on the situation. Graph 2 shows the maintainability design problems found by IMVES. These problems focus mainly on the poor accessibility and exhausting posture limitation caused by the installation site and spatial arrangement. All these problems and the following evaluations have been fed back to the design department. After the physical verification, most of the maintainability defects found by IMVES really existed in the practical maintenance process. The main reasons that a few problems found by IMVES do not exist are that the corresponding parts of the DMU are incomplete, and the maintenance operations are not accurately expressed in the simulation. In the previous cooperation with industry enterprises, we found that the maintainability work is performed in an inefficient and superficial
6. Application and discussion In this section, a case of using IMVES will be presented to describe how IMVES executes functions. The goal of applying IMVES in this case is to evaluate and verify the existing design of a certain type of aeroengine, find design flaws, return the flaws back to the design department and finally improve the maintainability of the upgraded type. Fig. 6 shows the physical prototype of the engine. Sufficient investigation of the product maintainability status is the first step to implement IMVES efficiently. To get the comprehensive and fine-grained details, we achieved an in-depth communication with the designers from the manufacturer and maintainers from maintenance depot on the issues of maintainability design and practical maintenance
Fig. 6. Physical prototype of the aero-engine. 48
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
Fig. 7. Maintainability verification of the aero-engine in IMVES.
and objectively. Compared to the traditional maintainability design method, IMVES has the advantages of the accuracy and efficiency on the simulation and credibility and quantification on the evaluation. However, IMVES also exhibits some issues that need to be further studied. First, the IMVES is simulation-based. The issue that simulation brings is that the maintainability problems found by IMVES need to be further confirmed on the physical mockup. Table 2 is an excerpt of the problem confirmation report. The problems in the report were found by IMVES in a helicopter maintainability design project. The report shows that not all the problems found by IMVES can be verified on the physical mockup, even though IMVES already has a good ability to find problems. Second, the interaction method with the virtual objects in IMVES still resorts to the devices. Using VR glasses to track eye view does not have much influence on the maintenance simulation. However, using a joystick to simulate the practical maintenance operations remains open to improvement, because the way to manipulate a joystick is difficult to express all common maintenance operations, which will lead to reduction of the authenticity. Third, IMVES is a quite complex system. To achieve all the functions, it requires extensive hardware and software resources for support. Five high performance workstations, four industrial-grade digital light processing (DLP) projectors and a series of sensors are employed to build the hardware environment, and several sets developed software work with each other.
Graph 2. Statistics of the maintainability problems found by IMVES.
way. Therefore, the proposition of this paper is aimed to verify the feasibility of VR in maintainability design and develop an effective approach by which the maintainability design work of the complex product could be carried out efficiently in the early design stages. IMVES could be implemented in the early design stages, no matter whether the product is totally new or an improved version. The maintenance simulation and verification could be carried out rapidly and precisely as long as the implementation and DMU are prepared. Then, with utilization of several maintainability evaluation methods, the IMVES could evaluate the maintainability of the product quantitatively Table 2 Excerpts of the problem feedback report. Problem Description
Whether to Adopt
Reason/Improvement Measurement
The installation site of the windshield wiper motor is inconvenient for maintenance. The cockpit microswitch cable wears easily.
No
Front sliding closure should be strengthened so that it can be stepped on.
No
The four screws of the cabin door pulley can easily scratch the passengers. The oil level of the brake sensor cannot be checked.
Yes Yes
The distance between the first and second step of the left trailing beam is too long. It is inconvenient for a maintainer whose height is less than 175 cm.
No
The hatch cover of the equipment cabin is not easily aligned with the keyhole on the door frame. The transparent hose of the joint between the hydraulic tank and hydraulic pump is not fixed. It has been loose. The fire extinguisher is installed on the maintenance platform of the auxiliary power unit (APU), making it impossible to maintain the APU. The space for the maintenance of the electric plating primer is narrow. The activity angle for an open spanner is only 18°. Collision with the transmission shaft and fire extinguisher may occur.
Yes Yes
The windshield wiper motor is installed below the cockpit skeleton. Due to restrictions of the installation site, it cannot be optimized. The microswitch cable is moved to the inside cockpit door and moved to the fuselage from the door spindle. First, a support box is needed for the maintainer to step on the sliding closure, which will add weight. Second, according to the current maintenance plan, the maintenance needs can be met without stepping on the closure. Add a protective cover over the screw to avoid wounding. Transparent material will be used for the pedal dust cover to make it convenient to check the hydraulic oil level. It cannot be changed because of the fuselage structure. Additionally, the inspector is unskilled. According to the experienced ground crew, the distance between the two steps meets the requirements of climbing. After analysis, it is a manufacturing defect from the corresponding factory. An electronic cabinet lock is adopted as an alternative solution. Tube clamps are added to fix the transparent hose.
Yes
Move the fire extinguishing system back 517 mm along the course line.
No
In the verification of the physical prototype, a fixed wrench is used. The maintenance space is sufficient. No collision occurs.
Yes
49
Computers in Industry 101 (2018) 41–50
Z. Guo et al.
These factors definitely enhance the difficulty to play the complete functions of IMVES.
[6] C.J. Latino, Hidden Treasure: Eliminating Chronic Failures Can Cut Maintenance Cost up to 60%, Reliability Center, Hopewell, VA, 1999. [7] U.D. Kumar, New trends in aircraft reliability and maintenance measures, J. Qual. Maint. Eng. 5 (1999) 287–295. [8] L.P. Berg, J.M. Vance, Industry use of virtual reality in product design and manufacturing: a survey, Virtual Real. (2017) 1–17. [9] S. Jayaram, U. Jayaram, Y. Wang, H. Tirumali, K. Lyons, P. Hart, VADE: a virtual assembly design environment, Comput. Graphics Appl. IEEE 19 (1999) 44–50. [10] J. Vora, S. Nair, A.K. Gramopadhye, A.T. Duchowski, B.J. Melloy, B. Kanki, Using virtual reality technology for aircraft visual inspection training: presence and comparison studies, Appl. Ergon. 33 (2002) 559–570. [11] P.J. Glor, E.S. Boyle, Design evaluation for personnel, training and human factors (DEPTH), Proceedings. Annual Reliability and Maintainability Symposium, 1993, IEEE, 1993, pp. 18–25. [12] K.J. Abshire, M.K. Barron, Virtual maintenance real-world applications within virtual environments, Proceedings. Reliability and Maintainability Symposium, 1998, (1998), pp. 132–137. [13] J.-R. Li, L.P. Khoo, S.B. Tor, Desktop virtual reality for maintenance training: an object oriented prototype system (V-REALISM), Comput. Ind. 52 (2003) 109–125. [14] A. Sanchez-Lite, M. Garcia, R. Domingo, S.M. Angel, Novel Ergonomic Postural Assessment Method (NERPA) Using Product-Process Computer Aided Engineering for Ergonomic Workplace Design vol. 8, (2013) e72703. [15] Anonymous, Maintaining focus: routine fixes and servicing are receiving greater emphasis in the design of the latest fighter aircraft, Mech. Eng. 123 (2001) 76–78. [16] Anonymous, Gearing up for JSF assembly: simulating ERGO man at work, Tool Prod. 69 (2003) 20–23. [17] K.J. Abshire, M.K. Barron, Virtual maintenance real-world applications within virtual environments, Proceedings. Annual Reliability and Maintainability Symposium, 1998, IEEE, 1998, pp. 132–137. [18] L.B.S. Gatto, A.C.A. Mól, I.J.A.L.D. Santos, C.A.F. Jorge, A.P. Legey, Virtual simulation of a nuclear power plant’s control room as a tool for ergonomic evaluation, Prog. Nucl. Energy 64 (2013) 8–15. [19] L. Zhang, J. Wade, D. Bian, J. Fan, A. Swanson, A. Weitlauf, Z. Warren, N. Sarkar, Cognitive load measurement in a virtual reality-based driving system for Autism intervention, IEEE Trans. Affective Comput. 8 (2017) 176. [20] S.G. Izard, J.A.J. Méndez, P.R. Palomera, Virtual Reality Educational Tool for Human Anatomy, (2017), p. 41. [21] T. Huber, M. Paschold, C. Hansen, T. Wunderling, H. Lang, W. Kneist, New dimensions in surgical training: immersive virtual reality laparoscopic simulation exhilarates surgical staff, Surg. Endosc. (2017) 1–6. [22] M.H.D. Silva, A.C.D.E. Santo, E.R. Marins, A.P.L.D. Siqueira, D.M. Mol, A.C.D.A. Mol, Using virtual reality to support the physical security of nuclear facilities, Prog. Nucl. Energy 78 (2015) 19–24. [23] P. Dutta, N. Rastogi, K.K. Gotewal, Virtual reality applications in remote handling development for tokamaks in India, Fusion Eng. Des. 118 (2017) 73–80. [24] A.A. García, I.G. Bobadilla, G.A. Figueroa, M.P. Ramírez, J.M. Román, Virtual reality training system for maintenance and operation of high-voltage overhead power lines, Virtual Real. 20 (2016) 27–40. [25] L.P. Berg, J.M. Vance, An industry case study: investigating early design decision making in virtual reality, J. Comput. Inf. Sci. Eng. (2017) 17. [26] C. Cruz-Neira, D.J. Sandin, T.A. Defanti, R.V. Kenyon, J.C. Hart, The CAVE: audio visual experience automatic virtual environment, Commun. ACM 35 (1992) 64–72. [27] D. Zhou, X. Jia, C. Lv, L. Kang, Using the swept volume to verify maintenance space in virtual environment, Assembly Autom. 34 (2014) 192–203. [28] Z. Dong, K. Le, C. Lv, A virtual reality-based maintenance time measurement methodology for complex products, Assembly Autom. 33 (2013) 221–230. [29] J. Geng, D. Zhou, C. Lv, Z. Wang, A modeling approach for maintenance safety evaluation in a virtual maintenance environment, Cad Comp. Aided Des. 45 (2013) 937–949.
7. Closing thoughts VR technology has seen impressive advancements in recent years. VR technology shows great potential for both the civil purposes and the industrial purposes. This paper reports a VR-based system to assist product design in the early design stages. Through interacting with the virtual prototypes in the immersive environment, the designer can have a more explicit comprehension of the practical maintenance scenario of the product even if the product has not come into service. Compared with traditional VR systems, IMVES not only enables users to “look around and click” in the virtual environment but also gathers the interactive data related to maintenance processes, further utilized by embedded algorithms to make quantitative and qualitative maintainability evaluations. A confirmatory application of IMVES on an aeroengine maintainability improvement project is presented. Although there still exists several issues that need to be studied further, the results demonstrate that VR-based IMVES is an effective and novel approach to assist in product design, which can help the designer make more reasonable decisions in the early design stages. The methods to build IMVES could also be leveraged as a reference to enlighten the practical VR applications for product design in the future. Acknowledgments The authors would like to thank all of the MSc students at Beihang University who have contributed to this research: Naidong Zhang, Xu Jia, Zhiyi He and Chunhui Guo. Appendix A. Supplementary data Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.compind.2018.06.007. References [1] M. Hobday, H. Rush, J. Tidd, Innovation in complex products and system, Res. Policy 29 (2004) 793–804. [2] R.F. Stapelberg, Handbook of Reliability, Availability, Maintainability and Safety in Engineering Design, Springer, United Kingdom, 2009. [3] Engineering Design Handbook: Maintainability Engineering Theory and Practice, Department of Defense, Washington, DC, 1976. [4] B.S. Dhillon, Maintainability, Maintenance and Reliability for Engineers, Crc Press, 2006. [5] J. Rix, S. Haas, J. Teixeira, Virtual Prototyping, Virtual Environments and the Product Design Process, Ifip Chapman & Hall, 1995.
50