- Email: [email protected]
Designing an immersive virtual reality interface for layout planning
Journal of Materials Processing Technology 107 (2000) 425±430
Designing an immersive virtual reality interface for layout planning B. Korves*, M. Loftus School of Manufacturing and Mechanical Engineering, The University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
Abstract This paper discusses the reasons why manufacturing layout planning is considered to be an appropriate new area of virtual reality (VR) utilisation; develops a framework for a VR-based layout planning tool and reports on a study comparing the use of immersive VR to a monitor-based system for detecting layout design ¯aws. The evaluation of the proposed framework has been conducted in a study, which did not have provision for an interactive alteration of the layout. The aim of the study was to compare an immersive system and a monitorbased VR system for workplace analysis. Participants were asked to investigate a workplace environment, which included three serious layout design ¯aws (tool arrangement, visibility, and tool location) and give their assessment about potential improvements. # 2000 Elsevier Science B.V. All rights reserved. Keywords: Virtual reality; HMD; Layout planning; Manufacturing cells
1. Introduction Virtual reality (VR), unlike some new technologies, never had to struggle for recognition. Ever since the BBC's 9 O'Clock News featured a short item on VR on 19 January 1993, including a VR model of a jet engine using a head mounted display (HMD), there has been a lot of attention and enthusiasm for this technology. Recently, however, there has been a change, with virtual reality being buzzwords that no longer captures the imagination. VR has become a negative term, standing for something that has promised so much but has not yet delivered. This is due partly to exaggerated expectations and partly to a lack of careful investigation where to apply VR to achieve a noticeable and quanti®able bene®t. At present, research in VR is directed at ®nding quality applications, where VRs proven bene®ts of superior viewing and interaction capabilities outweigh the shortcomings of the technology. 2. Background Layout planning is not a new problem in the manufacturing environment. Decisions between the alternative layout scenarios are traditionally based on de®ned features, such as travel frequency, distance travelled and the physical attributes of the parts, devices and operators [1]. As research has * Corresponding author. E-mail address: [email protected] (B. Korves).
shown, this procedure proves to be satisfactory on the ®rst stage of planning, the general layout design, or the so-called block layout [2]. Unfortunately, such data is less practical to compile when the exact position and orientation of equipment has to be determined in the detailed layout. Current approaches for arranging machinery in a factory, e.g. the formation of manufacturing cells (MCs), has highlighted this problem and the need for a new planning tool [3,4]. MCs aim to combine the ef®ciency of a line-layout (typical for automotive assembly) with the ¯exibility of a functional layout (lathe department, milling department, assembly department, etc.) by forming clusters or cells. These cells generally consist of a few machines, which, in conjunction, complete several processing stages of a part family. They are operated by one or a small number of operators. Recent studies on the implementation of MCs indicate the importance of arranging equipment within the MC and this has often been underestimated causing problems in companies [5,6]. The studies suggest that a lack of operator participation during the planning stage is a weakness in current layout planning procedures. Some features, which could be bene®cial to the layout of a cell, may be obvious to an experienced operator, but could be unknown to the layout planner. The authors believe that these differences could be attributable to the lack of adequate tools to support intra-cellular layout planning. At present, whilst inter-cellular layout planning is done using classical methods (pen and paper, or computer-aided systems), the intra-cellular layout is mostly done through the intuition of the planner. An interactive visual tool for analysing layout scenarios for
0924-0136/00/$ ± see front matter # 2000 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 4 - 0 1 3 6 ( 0 0 ) 0 0 7 1 7 - 2
426
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
MCs by shop¯oor personnel and the layout planner would be desirable. This is where VR has the potential to add value by including the operator in the planning loop. 3. Previous work Banerjee et al. [7,8] at the University of Illinois created a viewing platform for a model shop¯oor running on a CAVE system, where aspects such as illumination and noise level could be altered and investigated. The ®ndings of the research emphasised the importance of the perception of spatial depth for the success of this application. However, a lack of statistical data to support the results was noted. In a desktop system created by Kreitler et al. [9] at the University of Washington in Seattle, large parts or new equipment were manoeuvred through a crowded shop¯oor. It was found that even though custom-made menus can help to simplify the interaction with the tool, a fully immersive system with 3D interaction was required. Again, no statistical data were given. The School of Manufacturing and Mechanical Engineering at The University of Birmingham is in the process of developing an HMD-based interactive layout planning tool, speci®cally aimed at planning MCs [10]. The planning framework includes a prototype of an immersive interface to choose equipment from a database, position it on the shop¯oor and arrange the working environment to the user's preference. Under ¯oor provision can be modelled, which can give the user feedback about ¯oor support for speci®c machines, minimum safety distances between equipment, and the minimum space for walkways. An expert system to support user guidance for general aspects on MC design will also be included later. At the present stage, the model includes standard shop¯oor equipment, which can be selected from an immersive menu and placed on a model shop¯oor. The functionality of the machines includes the axes travel and loading/unloading procedures. Equipment can be moved on the shop¯oor according to realistic behaviour. Auditory and textural feedback is given, when the user of the tool violates prede®ned constraints. 4. Experimentation The experimental task was to assess a given workplace and give feedback about potential problems with the proposed layout. In particular, the investigation was aimed at collecting statistical data supporting (or not) the choice of an HMD system for visual workplace analysis compared with a monitor-based system. The virtual environment (VE) that had to be assessed by the participants consisted of a single operator automotive engine assembly station within a closed loop conveyor. A platen on the conveyor carried the engine to the workplace, where the sump was assembled. The
Fig. 1. Picture of the workplace.
participants were confronted with a similar set-up to the one shown in Fig. 1. The picture shows a platen with the engine, a small table with the engine sump and tools on the right, the conveyor control buttons, and the bolt-feeder suspended above the conveyor. The engine had to be released via the conveyor control buttons, before the operator picks up the oil pan, and places it on the top of the engine block. The pan would jump into position, triggered by a ``proximity snapping'' technique, which is activated when an object is close to its ®nal position. This compensates for the lack of haptic feedback during the assembly process by automatically locking an object into the right position once it has been lined up close enough to its assembly position. Once the oil pan was assembled the participants had two alternative ways to fasten it with 14 bolts. The ®rst way was to pick up a bolt on the table and put it into one of the bolt-holes. When the bolt was close enough to a hole it snapped into position and a new bolt appeared in the hand of the user. The second way of fastening the bolts was via the suspended bolt-feeder. It had to be picked by the users, moved to the bolt-holes where the bolts were automatically inserted in the holes and tightened. The participants were divided into two groups. One group were given a description of the workplace and asked to perform the oil pan assembly, using the HMD and a 3D mouse. They were not able to ``¯y'' around the VE but had to physically walk, with the positioning tracking system providing the data for updating the viewpoint of the users. They were also not able to move their viewpoint to a location, which would have the user in reality standing ``inside'' a solid object, e.g. the table, conveyor, or engine block. This was done by placing wooden tables at positions where the virtual objects appeared in the VE. The second group of participants were asked to perform the same task in the VE interacting via a monitor and standard mouse. Here the participants had full freedom to manoeuvre around the VE. At the end of the experiment the subjects were asked to ®ll in a brief questionnaire.
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430 Table 1 Flaw detection ranking system Hinted question ``0'' ``1'' ``2'' ``3'' ``4''
Open question
The ®rst part of this questionnaire was concerned with the participants' background in manufacturing engineering and experience of immersive VR. The second part was aimed at ®nding out about the participants subjective experiences with the VE and comments about how to improve the layout if necessary. The latter questions were speci®cally aimed at the known problem areas of the workplace layout. Major problems inherent in the design were the following: the bolt-feeder was extremely high; the bolt-holes on the far side of the engine could not be seen once the oil pan was in place; the table was too far away from the workstation. Independent variables included the viewing device (HMD or monitor), manufacturing experience, and experience in using immersive VR. The two dependent variables were workplace appraisal (``1'' very poor to ``5'' very good), and ¯aw detection (``0''±``4''). The ¯aw detection ranking system is shown in Table 1. The ``'' indicates that the user detected the problem area, but did not clearly specify the exact problem, and ``'' indicates that the problem was clearly identi®ed and speci®ed. The appraisal rating was to be given without any speci®c guidelines on what to base the subjective impression. This was thought to be a closely related scenario to asking for feedback from shop¯oor operators using a layout design tool. At ®rst the operators would come to a subjective judgement similar to the general appraisal of a layout. The expert system to guide the operator through the layout analysis would be comparable to the second focus of this experiment, i.e. the detection of speci®c layout ¯aws. Four hypotheses were formed prior to the experimentation: 1. The better the participants are able to visualise the proposed workplace, the more sceptical they will be with their appraisal, due to more design ¯aws being detected. 2. The HMD is the superior viewing device for a human operator orientated task; therefore HMD users will show a higher ¯aw detection ratio and lower appraisal ratings. 3. Experience with immersive VR is very important for fully utilising the capabilities of HMD applications. Hence, experienced users will be better at identifying the weak points in a design. 4. Experience in manufacturing will be more bene®cial for
427
monitor users, because the more experienced the shop¯oor operators the better they can investigate on speci®c features of the design. Hypothesis 1 is based purely on common sense and Hypothesis 3 is based on experiences with novice users of immersive VR. Hypothesis 2 is one of the key elements of this investigation. Past experience has shown that sometimes users get confused when working on a monitor-based VR system, because they can no longer tell which viewpoint they have chosen (how high is the viewpoint, which direction and what angle with the horizon does it have). This becomes even more severe when visibility and reach issues for a human operator are at the centre of the VR application. Finally, Hypothesis 4 was based on the belief that experienced manufacturing users can still effectively check a proposed workplace layout using the monitor. The model was run on an SGI Indigo2 Maximum Impact with a VR4 HMD, 3D mouse, 20 in. monitor, and standard mouse. Participants' background ranged from years of work experience at an automotive manufacturer and 50 h experience with immersive VR to novice users of VR with no experience in manufacturing. 5. Results It was the aim of this study to investigate the usefulness of immersive VR for analysing workplace layout, and clarify the in¯uence of previous user experience in VR and manufacturing. Fig. 2a and b shows charts of the results of the
Fig. 2. (a) Workplace appraisal in relation to manufacturing experience for HMD users and monitor users; (b) workplace appraisal in relation to experience with immersive VR for HMD users and monitor users.
428
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
Fig. 3. (a) Flaw detection percentage in relation to manufacturing experience for HMD users and monitor users; (b) ¯aw detection percentage in relation to immersive VR experience for HMD users and monitor users.
questionnaire. These include general workplace appraisal, both for the group using the HMD and the group using the monitor in relation to their manufacturing background and experience with immersive VR. While the VE looked appealing at ®rst sight, it contained the three major layout ¯aws as outlined previously. Workplace appraisal overall was signi®cantly higher using the monitor, but highly variable mean 3:9; t 3:19; d:f: 8. People using the HMD were generally more sceptical about the layout mean 2:6; t 3:14; d:f: 8. The lines in the charts represent the linear trend lines. The results of the second part of the questionnaire are shown in Fig. 3a and b. These are much more signi®cant than the previous ones. Flaw detection cannot be separated from overall appraisal of a workplace. Naturally, the more ¯aws users identi®ed, the less likely they would be to give a good appraisal rating. This was thought to be a closely related scenario to working with an expert system as part of a future intra-cellular layout planning tool. The signi®cant result of this part of the experimentation is the enormous difference between the ¯aw detection percentage using the HMD (62% mean with S.D. of 21%) and monitor (17% mean with S.D. of 10%).
effect on the HMD users (all equally sceptical) it did have a small in¯uence on the monitor user group. This effect was not signi®cant enough, though, to accept Hypothesis 4 without further tests. Experience with the use of immersive VR is generally positive for the performance of participants (HMD and monitor). The more experience they had, the more critical they became. This effect is stronger for HMD users. On any level of VR experience the HMD users were generally more sceptical with their appraisal than monitor users. The interpretation of the ¯aw detection in relation to the viewing device also supports the hypotheses. The experiment clearly identi®es the bene®t of an HMD for this type of human operator orientated task. One subject using the monitor did not identify a single ¯aw, not even after hinted questions. While this could be an odd sample, the trend is apparent. Paradoxically, manufacturing experience in HMD users supported their ability to spot the ¯aws, whereas it was a negative factor for monitor users. A brief correlation between the individual manufacturing experience and the individual VR experience of the participants showed a positive correlation for both HMD users and monitor users. In general, the more manufacturing experience a participant claimed to have in the questionnaire, the more immersive VR experience he or she also possessed. The positive correlation between experience in both manufacturing and immersive VR on the one hand and performance on the other for the HMD users (the standard viewing device for the proposed framework) clearly supports Hypotheses 2 and 3. A comparison with the data in Fig. 2a and b also supports Hypothesis 1. With the ¯aw detection rate being so much higher, the low workplace appraisal for HMD users is a logical consequence. Overall, experience in the use of immersive VR had a bene®cial effect on the accuracy of workplace analysis (more ¯aws spotted, generally more layout critique given). 7. Conclusion
6. Discussion
In general the experimentation supports the expected correlation formulated in the ®rst three hypotheses. The connection between the workplace appraisal and the viewing device was clearly shown. The correlation between the manufacturing experience and workplace appraisal though was much less signi®cant than expected. Immersive VR experience was bene®cial but not dependent on the viewing device. The HMD was generally regarded as more realistic. Hypothesis 4 could not be supported nor clearly rejected by this experiment. Three statements can be formulated as the outcome of the tests:
The results of the analysis for the in¯uence of the viewing device on the workplace appraisal support Hypothesis 4. While the manufacturing experience had absolutely no
1. The parameters of immersive VR experience and manufacturing experience will have to be considered as independent variables in future experiments.
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
2. Immersive VR has to be thoroughly introduced, and the users should have a good understanding of the virtual environment. 3. HMDs can have a crucial impact on the success of certain applications. The direct ¯aw detection performance of the participants showed a much greater difference with respect to the viewing device than the general workplace appraisal suggested. The superiority of the HMD over the monitor can be increased by both experiences in manufacturing and immersive VR. This gives an indication about the performance of an immersive system in standard layout planning procedures within a company, where users would generally have a good knowledge of the shop¯oor operations and would gradually become more familiar with immersive VR. These ®ndings do suggest that acquiring an immersive VR system would make a great difference and therefore could decide over the success or failure of the application. 8. Current research The reported study was conducted in a wider context, which is outlined in Fig. 4. It shows the various stages of development in relation to interactivity and system feedback. During the ®rst stage, the pre-design phase, the goals and the desired functionality of the layout planning tool were established based on literature surveys and analysis of existing tools [10]. This study now was part of the following stage, the layout perception stage. The VR tool was neither interactive nor did it give any system feedback. The model used in this study had interactive features, but not in the way this tool ®nally aims to provide, where the MC layout can be altered freely by the user. It also had no system feedback. More recent work has focused on the design of 3D interaction in an immersive VR environment. Initially the standard 3D interaction interface of the VR software was examined. Here the user has to immerse the representation of their hand into the computer generated object and while
429
pressing one button move the object along the six degrees of freedom (6 d.f.) movement of their hand. The need for further development became apparent. There are various ways of improving the 3D interactive interface and the ®nal metaphor has to correspond with the speci®c characteristics of the task [11±13]. Generally, during layout design the user has to arrange fairly large objects (e.g. machines) on the shop¯oor, which is mostly a 2D plane. It was found that the relatively limited ®eld of view (FOV) of the HMD and the jitters induced by the tracking system caused problems. Pilot users found it hard to position equipment correctly on the ¯oor, without part of the objects disappearing through it. Therefore, the set-up was changed and objects were restricted to 3 d.f. only allowing them to slide on the ¯oor and rotate around the vertical axes. While this alteration stopped machines from disappearing through the ¯oor, the limited FOV remained a problem. This arises because it is necessary to immerse the hand into an object, in order to manipulate it the users had to get very close to the machines in the virtual environment, which then blocked their view. However, the users also reported that they could not always see where they were positioning the selected object in relation to other objects. A way of remote interaction had to be found. The ®rst implementation of a remote interactive solution included a long ``broom stick'' attached to the objects for interaction from a comfortable distance. The users had to reach for the stick and move the equipment to the desired location. This proved unsatisfactory, because the interaction became unrealistic and the constraints of the objects on the shop¯oor made the grabbing of the pole very dif®cult. It had to be picked up with the hand in a certain position, otherwise rotation of the machine was nearly impossible. At present, two new interaction metaphors are being tested. One is similar to an interaction described by Mine et al. [12] where the user can select objects from a distance using a `ray-gun'. Another interface is closely related to the work of Stoakley et al. [14] where the user can interact with the environment via an immersive ``world in miniature'' (WIM). These interfaces are under development and the results will be reported in a comparative study. The pilot tests for the 3D interaction have already been conducted with an industrial partner, and the entire tool will ®nally be tested in the industrial trial phase where all interactive features are incorporated. References
Fig. 4. Development stages for the layout planning tool in relation to interactivity and feedback.
[1] E.S. Grajo, Strategic layout planning and simulation for lean manufacturing, in: Proceedings of the Winter Simulation Conference, 1996, pp. 564±568. [2] J. Hilger, Design of manufacturing cells: the inter-cell traf®c, in: Proceedings of the ASME Conference on Design, Analysis, and Control of Manufacturing Cells, PED-53, 1991, pp. 197±204. [3] J.L. Burbidge, The Introduction of Group Technology, Heinemann, London, 1975.
430
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
[4] G. Harhalakis, I. Minis, Cell formation for minimal traf®c under capacity constraints, in: Proceedings of the ASME Conference on Design, Analysis, and Control of Manufacturing Cells, PED-53, 1991, pp. 83±98. [5] U. WemmerloÈv, D.J. Johnson, Cellular manufacturing at 46 user plants: implementation experiences and performance improvements, Int. J. Prod. Res. 35 (1) (1997) 29±49. [6] U. WemmerloÈv, N.L. Hyer, Cellular manufacturing in the US industry: a survey of users, Int. J. Prod. Res. 27 (1989) 1511±1530. [7] A. Banerjee, P. Banerjee, S. Mehrotra, An enabling environment for inputting qualitative information in manufacturing systems layout design, in: Proceedings of the VR in Manufacturing Research and Education Symposium, UIC, The University of Illinois, Chicago, IL, October 7±8, 1996. [8] P. Banerjee, Factory models using virtual reality, 1995. http:// www_ivri.me.uic.edu/ivri/index.html. [9] M. Kreitler, J. Heim, R. Smith, Virtual environments for design and analysis of production facilities, in: Proceedings of the Working Conference on Managing Concurrent Manufacturing to Improve Industrial Performance, University of Washington, 1995.
[10] B. Korves, M. Loftus, The application of immersive virtual reality for layout planning of manufacturing cells, in: Proceedings of the 14th National Conference on Manufacturing Research, Derby, 1998, pp. 303±308. [11] I. Poupyrev, S. Weghorst, M. Billinghurst, T. Ichikawa, Egocentric object manipulation in virtual environments: empirical evaluation of interaction techniques, in: Proceedings of Eurographics'98, Vol. 17, No. 3, 1998. [12] M.R. Mine, F.P. Brooks, C.H. Sequin, Moving objects in space: exploiting proprioception in virtual-environment interaction, in: Proceedings of SIGGRAPH'97, Los Angeles, CA, 1997. [13] D.A. Bowman, L.F. Hodges, An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments, in: Proceedings of the Symposium on Interactive 3D Graphics, ACM, New York, 1997, pp. 35±38. [14] R. Stoakley, M.J. Conway, R. Pausch, Virtual reality on a WIM: interactive worlds in miniature, in: Proceedings of CHI'95, 1995, pp. 265±272.
Designing an immersive virtual reality interface for layout planning B. Korves*, M. Loftus School of Manufacturing and Mechanical Engineering, The University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
Abstract This paper discusses the reasons why manufacturing layout planning is considered to be an appropriate new area of virtual reality (VR) utilisation; develops a framework for a VR-based layout planning tool and reports on a study comparing the use of immersive VR to a monitor-based system for detecting layout design ¯aws. The evaluation of the proposed framework has been conducted in a study, which did not have provision for an interactive alteration of the layout. The aim of the study was to compare an immersive system and a monitorbased VR system for workplace analysis. Participants were asked to investigate a workplace environment, which included three serious layout design ¯aws (tool arrangement, visibility, and tool location) and give their assessment about potential improvements. # 2000 Elsevier Science B.V. All rights reserved. Keywords: Virtual reality; HMD; Layout planning; Manufacturing cells
1. Introduction Virtual reality (VR), unlike some new technologies, never had to struggle for recognition. Ever since the BBC's 9 O'Clock News featured a short item on VR on 19 January 1993, including a VR model of a jet engine using a head mounted display (HMD), there has been a lot of attention and enthusiasm for this technology. Recently, however, there has been a change, with virtual reality being buzzwords that no longer captures the imagination. VR has become a negative term, standing for something that has promised so much but has not yet delivered. This is due partly to exaggerated expectations and partly to a lack of careful investigation where to apply VR to achieve a noticeable and quanti®able bene®t. At present, research in VR is directed at ®nding quality applications, where VRs proven bene®ts of superior viewing and interaction capabilities outweigh the shortcomings of the technology. 2. Background Layout planning is not a new problem in the manufacturing environment. Decisions between the alternative layout scenarios are traditionally based on de®ned features, such as travel frequency, distance travelled and the physical attributes of the parts, devices and operators [1]. As research has * Corresponding author. E-mail address: [email protected] (B. Korves).
shown, this procedure proves to be satisfactory on the ®rst stage of planning, the general layout design, or the so-called block layout [2]. Unfortunately, such data is less practical to compile when the exact position and orientation of equipment has to be determined in the detailed layout. Current approaches for arranging machinery in a factory, e.g. the formation of manufacturing cells (MCs), has highlighted this problem and the need for a new planning tool [3,4]. MCs aim to combine the ef®ciency of a line-layout (typical for automotive assembly) with the ¯exibility of a functional layout (lathe department, milling department, assembly department, etc.) by forming clusters or cells. These cells generally consist of a few machines, which, in conjunction, complete several processing stages of a part family. They are operated by one or a small number of operators. Recent studies on the implementation of MCs indicate the importance of arranging equipment within the MC and this has often been underestimated causing problems in companies [5,6]. The studies suggest that a lack of operator participation during the planning stage is a weakness in current layout planning procedures. Some features, which could be bene®cial to the layout of a cell, may be obvious to an experienced operator, but could be unknown to the layout planner. The authors believe that these differences could be attributable to the lack of adequate tools to support intra-cellular layout planning. At present, whilst inter-cellular layout planning is done using classical methods (pen and paper, or computer-aided systems), the intra-cellular layout is mostly done through the intuition of the planner. An interactive visual tool for analysing layout scenarios for
0924-0136/00/$ ± see front matter # 2000 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 4 - 0 1 3 6 ( 0 0 ) 0 0 7 1 7 - 2
426
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
MCs by shop¯oor personnel and the layout planner would be desirable. This is where VR has the potential to add value by including the operator in the planning loop. 3. Previous work Banerjee et al. [7,8] at the University of Illinois created a viewing platform for a model shop¯oor running on a CAVE system, where aspects such as illumination and noise level could be altered and investigated. The ®ndings of the research emphasised the importance of the perception of spatial depth for the success of this application. However, a lack of statistical data to support the results was noted. In a desktop system created by Kreitler et al. [9] at the University of Washington in Seattle, large parts or new equipment were manoeuvred through a crowded shop¯oor. It was found that even though custom-made menus can help to simplify the interaction with the tool, a fully immersive system with 3D interaction was required. Again, no statistical data were given. The School of Manufacturing and Mechanical Engineering at The University of Birmingham is in the process of developing an HMD-based interactive layout planning tool, speci®cally aimed at planning MCs [10]. The planning framework includes a prototype of an immersive interface to choose equipment from a database, position it on the shop¯oor and arrange the working environment to the user's preference. Under ¯oor provision can be modelled, which can give the user feedback about ¯oor support for speci®c machines, minimum safety distances between equipment, and the minimum space for walkways. An expert system to support user guidance for general aspects on MC design will also be included later. At the present stage, the model includes standard shop¯oor equipment, which can be selected from an immersive menu and placed on a model shop¯oor. The functionality of the machines includes the axes travel and loading/unloading procedures. Equipment can be moved on the shop¯oor according to realistic behaviour. Auditory and textural feedback is given, when the user of the tool violates prede®ned constraints. 4. Experimentation The experimental task was to assess a given workplace and give feedback about potential problems with the proposed layout. In particular, the investigation was aimed at collecting statistical data supporting (or not) the choice of an HMD system for visual workplace analysis compared with a monitor-based system. The virtual environment (VE) that had to be assessed by the participants consisted of a single operator automotive engine assembly station within a closed loop conveyor. A platen on the conveyor carried the engine to the workplace, where the sump was assembled. The
Fig. 1. Picture of the workplace.
participants were confronted with a similar set-up to the one shown in Fig. 1. The picture shows a platen with the engine, a small table with the engine sump and tools on the right, the conveyor control buttons, and the bolt-feeder suspended above the conveyor. The engine had to be released via the conveyor control buttons, before the operator picks up the oil pan, and places it on the top of the engine block. The pan would jump into position, triggered by a ``proximity snapping'' technique, which is activated when an object is close to its ®nal position. This compensates for the lack of haptic feedback during the assembly process by automatically locking an object into the right position once it has been lined up close enough to its assembly position. Once the oil pan was assembled the participants had two alternative ways to fasten it with 14 bolts. The ®rst way was to pick up a bolt on the table and put it into one of the bolt-holes. When the bolt was close enough to a hole it snapped into position and a new bolt appeared in the hand of the user. The second way of fastening the bolts was via the suspended bolt-feeder. It had to be picked by the users, moved to the bolt-holes where the bolts were automatically inserted in the holes and tightened. The participants were divided into two groups. One group were given a description of the workplace and asked to perform the oil pan assembly, using the HMD and a 3D mouse. They were not able to ``¯y'' around the VE but had to physically walk, with the positioning tracking system providing the data for updating the viewpoint of the users. They were also not able to move their viewpoint to a location, which would have the user in reality standing ``inside'' a solid object, e.g. the table, conveyor, or engine block. This was done by placing wooden tables at positions where the virtual objects appeared in the VE. The second group of participants were asked to perform the same task in the VE interacting via a monitor and standard mouse. Here the participants had full freedom to manoeuvre around the VE. At the end of the experiment the subjects were asked to ®ll in a brief questionnaire.
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430 Table 1 Flaw detection ranking system Hinted question ``0'' ``1'' ``2'' ``3'' ``4''
Open question
The ®rst part of this questionnaire was concerned with the participants' background in manufacturing engineering and experience of immersive VR. The second part was aimed at ®nding out about the participants subjective experiences with the VE and comments about how to improve the layout if necessary. The latter questions were speci®cally aimed at the known problem areas of the workplace layout. Major problems inherent in the design were the following: the bolt-feeder was extremely high; the bolt-holes on the far side of the engine could not be seen once the oil pan was in place; the table was too far away from the workstation. Independent variables included the viewing device (HMD or monitor), manufacturing experience, and experience in using immersive VR. The two dependent variables were workplace appraisal (``1'' very poor to ``5'' very good), and ¯aw detection (``0''±``4''). The ¯aw detection ranking system is shown in Table 1. The ``'' indicates that the user detected the problem area, but did not clearly specify the exact problem, and ``'' indicates that the problem was clearly identi®ed and speci®ed. The appraisal rating was to be given without any speci®c guidelines on what to base the subjective impression. This was thought to be a closely related scenario to asking for feedback from shop¯oor operators using a layout design tool. At ®rst the operators would come to a subjective judgement similar to the general appraisal of a layout. The expert system to guide the operator through the layout analysis would be comparable to the second focus of this experiment, i.e. the detection of speci®c layout ¯aws. Four hypotheses were formed prior to the experimentation: 1. The better the participants are able to visualise the proposed workplace, the more sceptical they will be with their appraisal, due to more design ¯aws being detected. 2. The HMD is the superior viewing device for a human operator orientated task; therefore HMD users will show a higher ¯aw detection ratio and lower appraisal ratings. 3. Experience with immersive VR is very important for fully utilising the capabilities of HMD applications. Hence, experienced users will be better at identifying the weak points in a design. 4. Experience in manufacturing will be more bene®cial for
427
monitor users, because the more experienced the shop¯oor operators the better they can investigate on speci®c features of the design. Hypothesis 1 is based purely on common sense and Hypothesis 3 is based on experiences with novice users of immersive VR. Hypothesis 2 is one of the key elements of this investigation. Past experience has shown that sometimes users get confused when working on a monitor-based VR system, because they can no longer tell which viewpoint they have chosen (how high is the viewpoint, which direction and what angle with the horizon does it have). This becomes even more severe when visibility and reach issues for a human operator are at the centre of the VR application. Finally, Hypothesis 4 was based on the belief that experienced manufacturing users can still effectively check a proposed workplace layout using the monitor. The model was run on an SGI Indigo2 Maximum Impact with a VR4 HMD, 3D mouse, 20 in. monitor, and standard mouse. Participants' background ranged from years of work experience at an automotive manufacturer and 50 h experience with immersive VR to novice users of VR with no experience in manufacturing. 5. Results It was the aim of this study to investigate the usefulness of immersive VR for analysing workplace layout, and clarify the in¯uence of previous user experience in VR and manufacturing. Fig. 2a and b shows charts of the results of the
Fig. 2. (a) Workplace appraisal in relation to manufacturing experience for HMD users and monitor users; (b) workplace appraisal in relation to experience with immersive VR for HMD users and monitor users.
428
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
Fig. 3. (a) Flaw detection percentage in relation to manufacturing experience for HMD users and monitor users; (b) ¯aw detection percentage in relation to immersive VR experience for HMD users and monitor users.
questionnaire. These include general workplace appraisal, both for the group using the HMD and the group using the monitor in relation to their manufacturing background and experience with immersive VR. While the VE looked appealing at ®rst sight, it contained the three major layout ¯aws as outlined previously. Workplace appraisal overall was signi®cantly higher using the monitor, but highly variable mean 3:9; t 3:19; d:f: 8. People using the HMD were generally more sceptical about the layout mean 2:6; t 3:14; d:f: 8. The lines in the charts represent the linear trend lines. The results of the second part of the questionnaire are shown in Fig. 3a and b. These are much more signi®cant than the previous ones. Flaw detection cannot be separated from overall appraisal of a workplace. Naturally, the more ¯aws users identi®ed, the less likely they would be to give a good appraisal rating. This was thought to be a closely related scenario to working with an expert system as part of a future intra-cellular layout planning tool. The signi®cant result of this part of the experimentation is the enormous difference between the ¯aw detection percentage using the HMD (62% mean with S.D. of 21%) and monitor (17% mean with S.D. of 10%).
effect on the HMD users (all equally sceptical) it did have a small in¯uence on the monitor user group. This effect was not signi®cant enough, though, to accept Hypothesis 4 without further tests. Experience with the use of immersive VR is generally positive for the performance of participants (HMD and monitor). The more experience they had, the more critical they became. This effect is stronger for HMD users. On any level of VR experience the HMD users were generally more sceptical with their appraisal than monitor users. The interpretation of the ¯aw detection in relation to the viewing device also supports the hypotheses. The experiment clearly identi®es the bene®t of an HMD for this type of human operator orientated task. One subject using the monitor did not identify a single ¯aw, not even after hinted questions. While this could be an odd sample, the trend is apparent. Paradoxically, manufacturing experience in HMD users supported their ability to spot the ¯aws, whereas it was a negative factor for monitor users. A brief correlation between the individual manufacturing experience and the individual VR experience of the participants showed a positive correlation for both HMD users and monitor users. In general, the more manufacturing experience a participant claimed to have in the questionnaire, the more immersive VR experience he or she also possessed. The positive correlation between experience in both manufacturing and immersive VR on the one hand and performance on the other for the HMD users (the standard viewing device for the proposed framework) clearly supports Hypotheses 2 and 3. A comparison with the data in Fig. 2a and b also supports Hypothesis 1. With the ¯aw detection rate being so much higher, the low workplace appraisal for HMD users is a logical consequence. Overall, experience in the use of immersive VR had a bene®cial effect on the accuracy of workplace analysis (more ¯aws spotted, generally more layout critique given). 7. Conclusion
6. Discussion
In general the experimentation supports the expected correlation formulated in the ®rst three hypotheses. The connection between the workplace appraisal and the viewing device was clearly shown. The correlation between the manufacturing experience and workplace appraisal though was much less signi®cant than expected. Immersive VR experience was bene®cial but not dependent on the viewing device. The HMD was generally regarded as more realistic. Hypothesis 4 could not be supported nor clearly rejected by this experiment. Three statements can be formulated as the outcome of the tests:
The results of the analysis for the in¯uence of the viewing device on the workplace appraisal support Hypothesis 4. While the manufacturing experience had absolutely no
1. The parameters of immersive VR experience and manufacturing experience will have to be considered as independent variables in future experiments.
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
2. Immersive VR has to be thoroughly introduced, and the users should have a good understanding of the virtual environment. 3. HMDs can have a crucial impact on the success of certain applications. The direct ¯aw detection performance of the participants showed a much greater difference with respect to the viewing device than the general workplace appraisal suggested. The superiority of the HMD over the monitor can be increased by both experiences in manufacturing and immersive VR. This gives an indication about the performance of an immersive system in standard layout planning procedures within a company, where users would generally have a good knowledge of the shop¯oor operations and would gradually become more familiar with immersive VR. These ®ndings do suggest that acquiring an immersive VR system would make a great difference and therefore could decide over the success or failure of the application. 8. Current research The reported study was conducted in a wider context, which is outlined in Fig. 4. It shows the various stages of development in relation to interactivity and system feedback. During the ®rst stage, the pre-design phase, the goals and the desired functionality of the layout planning tool were established based on literature surveys and analysis of existing tools [10]. This study now was part of the following stage, the layout perception stage. The VR tool was neither interactive nor did it give any system feedback. The model used in this study had interactive features, but not in the way this tool ®nally aims to provide, where the MC layout can be altered freely by the user. It also had no system feedback. More recent work has focused on the design of 3D interaction in an immersive VR environment. Initially the standard 3D interaction interface of the VR software was examined. Here the user has to immerse the representation of their hand into the computer generated object and while
429
pressing one button move the object along the six degrees of freedom (6 d.f.) movement of their hand. The need for further development became apparent. There are various ways of improving the 3D interactive interface and the ®nal metaphor has to correspond with the speci®c characteristics of the task [11±13]. Generally, during layout design the user has to arrange fairly large objects (e.g. machines) on the shop¯oor, which is mostly a 2D plane. It was found that the relatively limited ®eld of view (FOV) of the HMD and the jitters induced by the tracking system caused problems. Pilot users found it hard to position equipment correctly on the ¯oor, without part of the objects disappearing through it. Therefore, the set-up was changed and objects were restricted to 3 d.f. only allowing them to slide on the ¯oor and rotate around the vertical axes. While this alteration stopped machines from disappearing through the ¯oor, the limited FOV remained a problem. This arises because it is necessary to immerse the hand into an object, in order to manipulate it the users had to get very close to the machines in the virtual environment, which then blocked their view. However, the users also reported that they could not always see where they were positioning the selected object in relation to other objects. A way of remote interaction had to be found. The ®rst implementation of a remote interactive solution included a long ``broom stick'' attached to the objects for interaction from a comfortable distance. The users had to reach for the stick and move the equipment to the desired location. This proved unsatisfactory, because the interaction became unrealistic and the constraints of the objects on the shop¯oor made the grabbing of the pole very dif®cult. It had to be picked up with the hand in a certain position, otherwise rotation of the machine was nearly impossible. At present, two new interaction metaphors are being tested. One is similar to an interaction described by Mine et al. [12] where the user can select objects from a distance using a `ray-gun'. Another interface is closely related to the work of Stoakley et al. [14] where the user can interact with the environment via an immersive ``world in miniature'' (WIM). These interfaces are under development and the results will be reported in a comparative study. The pilot tests for the 3D interaction have already been conducted with an industrial partner, and the entire tool will ®nally be tested in the industrial trial phase where all interactive features are incorporated. References
Fig. 4. Development stages for the layout planning tool in relation to interactivity and feedback.
[1] E.S. Grajo, Strategic layout planning and simulation for lean manufacturing, in: Proceedings of the Winter Simulation Conference, 1996, pp. 564±568. [2] J. Hilger, Design of manufacturing cells: the inter-cell traf®c, in: Proceedings of the ASME Conference on Design, Analysis, and Control of Manufacturing Cells, PED-53, 1991, pp. 197±204. [3] J.L. Burbidge, The Introduction of Group Technology, Heinemann, London, 1975.
430
B. Korves, M. Loftus / Journal of Materials Processing Technology 107 (2000) 425±430
[4] G. Harhalakis, I. Minis, Cell formation for minimal traf®c under capacity constraints, in: Proceedings of the ASME Conference on Design, Analysis, and Control of Manufacturing Cells, PED-53, 1991, pp. 83±98. [5] U. WemmerloÈv, D.J. Johnson, Cellular manufacturing at 46 user plants: implementation experiences and performance improvements, Int. J. Prod. Res. 35 (1) (1997) 29±49. [6] U. WemmerloÈv, N.L. Hyer, Cellular manufacturing in the US industry: a survey of users, Int. J. Prod. Res. 27 (1989) 1511±1530. [7] A. Banerjee, P. Banerjee, S. Mehrotra, An enabling environment for inputting qualitative information in manufacturing systems layout design, in: Proceedings of the VR in Manufacturing Research and Education Symposium, UIC, The University of Illinois, Chicago, IL, October 7±8, 1996. [8] P. Banerjee, Factory models using virtual reality, 1995. http:// www_ivri.me.uic.edu/ivri/index.html. [9] M. Kreitler, J. Heim, R. Smith, Virtual environments for design and analysis of production facilities, in: Proceedings of the Working Conference on Managing Concurrent Manufacturing to Improve Industrial Performance, University of Washington, 1995.
[10] B. Korves, M. Loftus, The application of immersive virtual reality for layout planning of manufacturing cells, in: Proceedings of the 14th National Conference on Manufacturing Research, Derby, 1998, pp. 303±308. [11] I. Poupyrev, S. Weghorst, M. Billinghurst, T. Ichikawa, Egocentric object manipulation in virtual environments: empirical evaluation of interaction techniques, in: Proceedings of Eurographics'98, Vol. 17, No. 3, 1998. [12] M.R. Mine, F.P. Brooks, C.H. Sequin, Moving objects in space: exploiting proprioception in virtual-environment interaction, in: Proceedings of SIGGRAPH'97, Los Angeles, CA, 1997. [13] D.A. Bowman, L.F. Hodges, An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments, in: Proceedings of the Symposium on Interactive 3D Graphics, ACM, New York, 1997, pp. 35±38. [14] R. Stoakley, M.J. Conway, R. Pausch, Virtual reality on a WIM: interactive worlds in miniature, in: Proceedings of CHI'95, 1995, pp. 265±272.