- Email: [email protected]
Optimising procedures in manufacturing systems
Industrial Ergonomics ELSEVIER
International Journal of Industrial Ergonomics 17 (1996) 43-51
Optimising procedures in manufacturing systems Philip Marsden, Mark Green Human Reliabili~ Associates Ltd, I, School House, Higher Lane Dalton, Wigan, Lancs, WN8 7RP. UK
Received 10 July 1994; revised 22 September 1994
Abstract
A significant amount of time and effort in manufacturing operations is directed toward the development of procedures intended to ensure that work activities are carried out in a safe and efficient manner. Despite the often high level of investment, however, it appears that there are a number of problem areas associated with procedure use where the systems frequently fail to meet their intended objectives. The cost of failures in a procedure system can vary quite widely dependent upon the nature of the operation. In most cases the consequences of failure have only a negligible effect on the overall performance of a company and can be measured in terms of production hours. Occasionally, however, the consequences of failure can be catastrophic and result in significant damage to people, plant and the environment. The purpose of this paper is twofold: The first objective is to provide a preliminary review of common problems which are associated with the catastrophic failure of procedure systems in a manufacturing environment. The second aim is to consider some of the ways that a human factors approach might be used to help prevent the more serious failures associated with the use of procedures in industrial settings.
Relevance to industry
A ~eliable and well-managed system for procedure design and use is an important component in a safe and efficient manufacturing process. Poorly designed procedures may be misinterpreted or even ignored by management and operational staff, which in turn can result in major incidents involving human casualties and production loss. Keywords: Procedures; Training; Accidents; Human factors
I. Introduction
T h e safe a n d efficient o p e r a t i o n o f any m a n u f a c t u r i n g system is highly d e p e n d e n t u p o n the existence o f a g r e e d m e t h o d s of work, w h e r e b y a c o m p a n y specifies how it w a n t s c e r t a i n w o r k activities to be p e r f o r m e d ( G r e e n a n d M a r s d e n , 1992; M a r s d e n a n d W h i t t i n g t o n , 1993). T h e m o s t
usual way of c o m m u n i c a t i n g such m e t h o d s to e m p l o y e e s is via w r i t t e n instructions or p r o c e d u r e s (Folley, 1975; Swezey, 1987). D e f i n e d b r o a d l y t h e m a j o r f u n c t i o n o f a p r o c e d u r e is to e n s u r e t h a t all tasks a r e c a r r i e d o u t in a safe, efficient a n d c o n t r o l l e d m a n n e r a n d this a p p l i e s e s p e c i a l l y to o p e r a t i o n s t h a t involve a significant hazard component.
0169-8141/96/$15.00 © 1996 Elsevier Science B.V. All rights reserved SSDI 0169-8141(94)00102-2
44
P. Marsden, M. Green/International Journal of lndustrial Ergonomics 17 (1996) 43-51
Problems associated with the use of procedures in an industrial situation have been reviewed on a number of occasions. In the nuclear industry, for example, Michael Morgenstern and his colleagues (Morgenstern et al., 1987) concluded that the vast majority of procedures reviewed failed to attain a classification considered "minimally acceptable". Similar conclusions were reached by Zach (1980) following a study of control room operating procedures. In this investigation it was found that procedures tended to be inadequate both in terms of the amount of information they contain and the way the information is presented. Finally, in a seminal study of maintenance errors reported to the U.S. Nuclear Regulatory Commission during the period 1975-1978, Brune and Weinstein (1982) concluded that procedural deficiencies were implicated in more than half of employee errors. Consequently, the authors suggested that the general quality of maintenance procedures constituted a major source of vulnerability in industrial operations of all types. Each of the studies reported above raise serious doubts about the way that procedure systems are designed and operated in an industrial situation. The purpose of the present paper is to identify some of the ways that procedure systems can go wrong. Following this, we shall examine what, if anything, can be done to improve the general quality of procedure provision in a manufacturing environment. The account draws heavily upon the experience of dealing with procedures in the context of Nuclear Power Plant (NPP) operation, and on recent developments made within the field of human factors engineering. Despite the emphasis on NPP operations it is the view of the authors that there are many lessons to be learned from a consideration of procedure failures in a NPP domain that will be directly relevant to the design of procedure systems in all sectors of industry.
2. Overview of the a r g u m e n t
In essence, the theme of this paper is that there are three traditional problem areas associated with the use of procedures in a manufactur-
ing environment. Described in overview we propose that procedure systems tend to fail for three basic reasons: - Because of weaknesses inherent in the way that procedures are prepared; - Because people frequently fail to perform tasks in the way that procedures prescribe; - Because systems within the organizational infrastructure that should provide support for procedure systems tend to be poorly structured and operated. Human factors investigations have been active in each of the above areas and have produced general guidelines that can be used to address each problem type.
3. Procedure preparation
In the afternoon, on the 10th of March 1983, an engineer working at Quad Cities prepared a note explaining the procedure for performing a shutdown scheduled to be undertaken later that night by the evening shift. The shift supervisor incorrectly interpreted the procedure as instructing him to insert control rods in an order that was the reverse of that required under normal circumstances. Prior to the arrival of the night shift supervisor, the evening shift operators incorrectly installed 33 rods. In order to do this, the shift team needed to by-pass a safety critical system (the rod worth minimiser, or RWM) designed specifically to prevent improper rod insertion. Following shift hand over, the night shift supervisor initially restored the integrity of this system. Because the system prevented further rods to be inserted into the core in the manner specified, however, the decision was taken that the RMW was faulty and the systems was again by-passed. A further 10 rods were wrongly inserted into the core, and the reactor was shut down without incident. A subsequent investigation concluded that the near-miss incident had created the basic conditions for a rod drop accident and imposed a $150,000 fine on the owning company, Commonwealth Edison. Many reported incidents involving procedures occur as a result of weaknesses inherent in the
45
P. Marsden, M. Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
way that procedures are prepared. The near-miss accident cited above, for example, would seem to provide an interesting case study of how procedures can sometimes fail when they are prepared by "domain experts" who are familiar with plant and process theory but who have little idea regarding how the system works on a day-to-day level. Indeed, it is by no means uncommon for procedures prepared by highly qualified technical personnel working in isolation to be incomplete, incorrect a n d / o r generally unrealistic with regard to the task they describe. There are at least three important reasons why domain experts may not be the group of people best qualified to prepare new work procedures. First, because product specialists tend to perform a quality control function in most organizations, they are rarely aware of the specific details which lie behind the way that operators actually fulfil their work roles. In fact, as more than one study has shown (e.g., Marsden and Green, 1991), they may not even be aware of recent modifications made to plant and equipment and it is not uncommon for procedures to be written by experts which incorporate instructions for plant and equipment no longer in service. The second problem with using a domain expert to prepare work procedures arises as a result of the process of acquiring expertise. We now know, for example, that the essential paradox of expertise is that the more a person comes to understand a particular job or work role, the less able they are to verbalise their expert knowledge (Rector et al., 1985). This means that an expert will often prepare an instruction which fails to include important details. It is precisely for this reason that the discipline of knowledge engineering was formed in the early 1980's (Hayes-Roth et al., 1983). Finally, domain experts tend on the whole to be unfamiliar with information presentation methods which have been developed by human factors specialists to help optimise the information take-up of the end user. Consequently, experts frequently produce documentation which is poorly formatted and the failure to provide userfriendly procedures can effectively render the instruction unusable as an operator aid.
~[ Step I ~1 "t Task Analysis
Step 2 Select format Step 5 Quality assure Step 3 Draft procedure Step 6 Monitor/review effectiveness
Step 5 Approve procedure
Fig. 1. A development process for optimising procedure quality. (Based on Embrey, 1986).
One way to reduce the likelihood of preparing deficient procedures involves the specification of a formal procedure development process such as the one proposed by Embrey (Embrey, 1986). He argued that optimal procedure development requires a six-stage process incorporating two design iterations. A graphic representation of the development process discussed by Embrey is provided in Fig. 1. Each of these stages are discussed further below.
3.1. Step 1: Task analysis In this scheme, the first step in procedure preparation involves carrying out a systematic analysis of the task using a formal method such as Hierarchical Task Analysis (HTA). In essence, HTA is methodology for breaking tasks down into sub-goals and specifying the way these goals are to be combined in order to achieve the overall objective of the task. At the highest level of analysis, tasks are defined in terms of work plans which take the form of sub-goal sequences (e.g. 1. Set up the gland water system: 2. Establish ACW supply, etc.). At the lowest level, each sub-goal is described in terms of sequences of object-action pairings such as "1. Open valve PVI01", "2. Close valve PSV 54". There are two major advantages to be gained by using a method such as HTA for procedures development. First, it forces
46
P. Marsden, M. Green /International Journal of Industrial Ergonomics 17 (1996) 43-51
the expert to make what He or She knows verbally explicit and this permits the accuracy and completeness of operational knowledge to be tested objectively prior to implementation as a procedure. Second, it provides an opportunity to involve end users in procedure preparation and this can help encourage feelings of ownership.
3.2. Step 2: Procedure preparation Once the task has been formally analysed, the next step in the process is the selection of an appropriate format. Several factors may need to be taken into consideration when choosing a format for the procedure, but two of the most important considerations are task complexity and user familiarity. As a rule, as the operators' familiarity with a task increases and the task complexity decreases, the more likely it is that operators will need a Job Performance Aid (JPA) which uses a checklist format. Conversely, as a task complexity increases and familiarity diminishes, then a much greater level of operator support will be required, and this would best be provided by using a step-by-step procedures format. Alternative procedural formats may be required for jobs involving more than one person (e.g. playscript formats, etc., Berry, 1981)
3.3. Step 3: Preparation of draft procedure The third stage of development involves preparing a draft document based upon task elements determined in step 1, presented in the format determined in step 2. In preparing the draft document, several decisions may need to be made regarding the design of the layout of the procedure. Once a layout has been devised, Embrey suggests that it should be optimised by checking for consistency with human factors guidelines which specify best-practice for information-presentation.
3.4. Step 4." Quality assurance Prior to approving the procedure for general use, several tests of the draft procedure will need to be undertaken and these should be performed
by suitably qualified inspectors. Wherever possible, testing should take place in the actual operating environment. Embrey suggests that it is desirable that a series of end-user consultations is also undertaken during this phase of development to ensure that the proposed method of work is consistent with current practices. All results should be fully documented and this information can be used to make modifications to the draft procedure. Incremental refinement of the procedure may necessitate several design iteration between steps 1 to 4.
3.5. Step 5: Approving the procedure for general use In the fifth stage, the procedure is approved for general use. New procedures will often need to have special controls a n d / o r limitations placed on their use such that only certain qualified personnel would be allowed to perform the procedure on the first few occasions. This qualification would be especially relevant to tasks involving a significant hazard component.
3.6. Step 6." Long-term maintenance The final aspect of Embrey's (1986) proposed process involves the long-term maintenance and updating of the procedure over the lifetime of the system. He suggests that routine checks of the effectiveness of the procedure be made periodically to assure the long-term applicability and usability of the procedure over time.
4. Non-compliance with written instructions
On the 18th of September, 1988 at Stade PWR in the Lower Saxony region of West Germany, an electrical failure caused a valve to close in one of four main steam lines. As a direct result of the closure other valves in the remaining three lines automatically closed, blocking all four lines. The automatic reactor protection system at this point should have scrammed the reactor and turbine. Contrary to procedures however, operators overrode the system and reopened the valves in an attempt to keep the reactor running. The valves
P. Marsden, M.
Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
quickly closed for a second time, and the system scrammed. Opening and closing of the steam line valves in rapid succession led to a violent pressure wave in the main steam lines causing a
maximum displacement of 20 cm. This was more than double the acceptable limit and an investigation concluded that the actions of the operators brought the steam lines close to failure.
BFP/3A/06 ] OBJECTIVE" COMMISSION AUXILIARY LUBRICATION OIL SYSTEM
'or H~d-~s
I
-I I Removeauthorir~=d]
Hold-Offs
Note isolation
number
I
..... °1 Establishlub. tank level
Jnform operator Ensureoil flow present
I SIGHT: I GLASS
$ >0.8 bar.~
~Y~s
I
Monitorfilter J~ pressuredrop J~
-[
drop <40kpa?J ~YES Aux. lub. oll ~tem now commi~oned
I
Selectother filter
47
Inform operator I
Reportto
sh~ ofrce
Fig. 2. Objective: Commission auxiliary lubrication oil system.
48
P. Marsden, M. Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
N o n - c o m p l i a n c e with p r o c e d u r e s f e a t u r e s p r o m i n e n t l y in statistics covering a c c i d e n t a n d n e a r miss reports. I n a survey o f 180 significant
BFP/3A/06
]
n u c l e a r events c i t e d by G r e e n (1991), for example, m o r e t h a n 18% o f all a c c i d e n t s a r e a t t r i b u t e d in s o m e way to the failure o f p e r s o n n e l to follow
OBJECTIVE"
COMMISSION AUXILIARY LUBRICATION OIL SYSTEM
|
:::::
: :
:::
:.
: :
SUPPLEMENTARY INFORMATI O N l. Check system for 'Hold-Offs' l. l l.x~cate Hold-Offs on this system 1.2 Remove authorised Hold-Offs
1.2 Hold-Offsare authorisedbyProofof Isolationnumber providedin BFP/3A/01 1 . 3 N O (a) Note Proof of Isolation number
1.3 Are all Hold-Offs removed?
0,) R ~ to shiftom,,e YES Proceedwithnextstep
2. Prepare lubricating oil system for start up 2.1 Close filter vents 2.2 Select A or B filter 2.3 Read lubricating oil level in tank 2.4 Is oil level greater than 0.2m?
2 . 4 N O (a) Notelevel (b) Inform operator YES
Proceed with next step
3. Start Auxiliary lubricating oil pump ~. 1 Ensure oil flow present in sight glass 4. Monitor pump pressure 4. l Is pump pressure greater than 0.8 bar?
4. At Io~ eonla'ollmnel
4"INO Inform operator
YES ~
withnextstep
5. Monitor filter pressure drop across filters 5.1 Is pressure drop across filters greater than 40 kpa?
~lNO Pro,,~ withnm step YES (c) g / ~Inform P"~m ~ o~mor (d) ~ t
step 3. I - - i f OK proceed
Auxiliary lubricating oil system now commissioned
Fig. 3. Objective: Commission auxiliary lubrication oil system.
P. Marsden, M. Green/International Journal of lndustrial Ergonomics 17 (1996) 43-51
procedures. Similarly, Brune and Weinstein (1982) found failure to comply with the prescriptions contained within procedures contributed significantly to procedure-related performance errors in activities involving the maintenance, test and calibration of equipment. From the point of view of the user, it is possible to distinguish between two broad classes of non-compliant behaviour: intentional non compliance (e.g. routine violations, unsafe acts, etc.) and unintentional non-compliance (e.g. a procedure was used but an action was omitted, a procedure was used but an action was performed incorrectly, etc.). Of these two types, intentional noncompliance is potentially the most serious, and it is this aspect of behaviour that we are primarily concerned with here. In attempting to provide an answer to the question "Why don't people follow procedures?", it is useful to begin by considering what users typically think about procedures. As part of a study of employee attitudes reported by Zach (1980), a large group of operators were asked to say why they thought procedures were needed. Interestingly, the most conspicuous answer was that procedures were required primarily for the purpose of regulatory inspections and little mention was made of the fact that they serve as operator aids. In fact, procedures were commonly viewed as the bane of the operator's life and were perceived to devaluate the work role. In particular, operators reported that working with procedures makes work much less rewarding and the job more difficult than it would otherwise be. These few comments provide an important insight into why people sometimes fail to comply with a procedure. Procedural systems are typically viewed by operators as a system of work control designed essentially to protect the company in the event of an accident. Consequently, operators report that they have little confidence in the usefulness of procedures as usable work aids. Unfortunately, this view is not without some foundation. When pressed, most operators are able to recall occasions were non-compliance with a procedure was sanctioned (either implicitly or explicitly) by supervisors or others in the organisation. This tended to occur particularly when
49
production pressures were high. Inconsistencies such as these tend to undermine a compliance policy and leave areas of ambiguity regarding when procedures are to be followed. What changes would operators like to see made to procedures? When this question was put to operators in two studies carried out by Zach (1980) and Seminara et al. (1976) answers tended to be concerned largely with the ongoing debate between the relative merits of "lock-step" (i.e. high compliance) versus guidance (low compliance) procedures. The greater proportion of answers indicated a preference for guidance documentation which places much more reliance on the personal initiative and experience of the user. Consequently, operators expressed a desire to see more concise forms of procedure which described tasks only in terms of important details. Users also indicated a preference for flow diagram (as opposed to text-based) procedures. They suggested that flow make it easier to identify key elements of the task quickly and to relate specific actions to overall objectives. On this issue, however, it was acknowledged that there is always some loss of information involved in the use of flow diagrams for complex text. Examples of a procedure specified using both a flow chart and text-based presentation methods are provided in Fig. 2 and Fig. 3. Finally, the operators indicated that they would like more formal training relating to procedures of all types. Training with procedures in mind is an important issue in its own right and will be considered more fully in the following section. The major point which is being made here is that none of the above changes are unreasonable provided that adequate user support is made available. Moreover, being seen attempting to address these problems could help ameliorate the potential threat of serious incidents arising from the failure of users to comply with a written instruction.
5. Procedures infrastructure
On February llth, 1981, at Tennessee Valley Sequoyah I plant, a unit operator instructed a
P. Marsden, M. Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
50
recently qualified employee to check the containment spray valve and ensure that it was closed. Instead of closing the valve the operator opened it, and as a direct result 40,000 gallons of primary cooling water was released into the containment area, along with a further 65,000 gallons of water held in a storage unit. Eight workers were contaminated as a result of the accident which involved low grade radioactive materials. The reactor was fortunately in a cold shutdown condition at the time of the incident thus preventing the occurrence of a more serious accident. The final source of vulnerability to be considered here deals with those situations where the procedures themselves are not directly at fault, but rather, where there has been a failure of some type in the system which should provide some support for the major user groups. Of particular concern are those incidents that occur because the operator is inadequately trained to use the procedure, or where problems arise because of a failure to monitor the effectiveness of procedures. The most common scenario for a failure of this latter kind is the situation where a procedure is not revised following an operational failure of some type. The basic design philosophy proposed by Human Reliability Associates to counter problems of this type advocates the development of a procedural system comprised of three interrelated elements: A Procedures database (or inventory), a training function and a system for monitoring and updating. These three elements and their interrelationships are shown in overview in Fig. 4.
/
"
/MONITORING/ "~
\\T .... g
Training\ \ needs Prog. . . . . ~
Aod,t//
trails// Feedback-driven ~/tell ..... t
; PROCEDURES "~
Fig. 4. Minimum structural requirements for an effective procedures infrastructure. (Source: Green and Marsden, 1992).
In this type of system, procedures are explicitly related to training in so far as they are developed specifically with training needs in mind. For example, analysis of the structure of the procedure will imply particular training requirements and this information can be used to formulate explicit training strategies. Similarly, the procedures themselves can be used as a primary source of training materials and these are presented to trainees as part of a structured training programme. Employee qualification is based upon methods of testing in which the trainee must demonstrate his or her competence relative to particular sets of procedures. Procedures are also formally linked in the system to a monitoring function. In this case it is proposed that quality points be built in to procedures that lay down a clear audit trail each time the procedure is used. The audit trail can be periodically examined and checked for compliance, and reviews of procedure quality should be used to modify the procedures in a process of feedback-driven refinement. There are several important points which follow from the implementation of a procedures infrastructure of the type proposed. First, it is a mistake to think of procedures as finalised static documents once they are written. Rather, they should be thought of as "living documents" which always exist in a state of latest revision. People should always be encouraged to challenge the prescriptions contained within procedures with a view to upgrading safety or production targets. Second, employee training must work for the benefit of both the company and employee alike. Such objectives are best served by the use of proactive training methods involving a blend of on-the-job versus formal training methods which deliver both technical (relating to the process) and behavioural (relating to task skills) knowledge. It is not sufficient to assume competence based upon the principle of time served. Rather, formal methods of competence testing should be employed which require the trainee to demonstrate his or her skills relative to current procedures. Finally, the process of rapid prototyping and feedback-driven refinement is more productive when the monitoring process is implemented
P. Marsden, M. Green / International Journal of Industrial Ergonomics 17 (1996) 43-51 in a blame-free environment. People should always be encouraged to report their mistakes and near-misses so that procedural implications can be evaluated and error-tolerant solutions designed in an attempt to minimise the likelihood of catastrophic system failure.
6.
Conclusions
There are three traditional problems areas that are traditionally associated with the use of procedures in manufacturing system. In particular, it was proposed that procedure systems frequently fail to attain their desired objectives: Due to weaknesses that are inherent within the preparation process; Because people fail to comply with an established procedure; Because systems which should provide support for procedures are often poorly designed and implemented. It was suggested that improvements in each of these areas can be made by: Adopting a formal development process; Listening to the views of users and involving them in methods of work control; Developing a strong procedures infrastructure which explicitly relates procedures to a training and monitoring function. It is not suggested that implementing each of these recommendations will guarantee that future procedure provision will be perfect. Experience suggests, however, that they will lead to significant improvements being made in each of the above areas should the principles set out in this paper be applied.
51
References
Berry, E., 1981. How to get users to follow procedures. IEEE Transactions on Professional Communication, 25: 21-26, 4. Brune, R.L. and Weinstein, M., 1982. Procedures evaluation checklist for maintenance, test and calibration procedures used in the nuclear industry. NUREG CR-1369. Sandia National Laboratories, Albuquerque, NM. Embrey, D., 1986. Guidelines for the Preparation of Procedures for the High Reliability of Ultrasonic Inspection Tests. Human Reliability Associates, Dalton, Wigan. Folley Jnr., J.D., 1975. Research issues in JPA technology development. Paper presented at the SALT-Improved Information Aids, LMI Conference on Arlington, VA. Green, M., 1991. Root Cause Analysis. Human Reliability Associates, Dalton, Wigan. Green, M. and Marsden, P., 1992. The design of effective procedures for industry. 1992 Ergonomics Society Annual Conference. University of Aston, Birmingham. Hayes-Roth, F., Waterman, D.A. and Lenat, D.B., 1983. Building Expert Systems. Addison-Wesley, London. Marsden, P. and Whittington, C., 1993. Work instruction provision: Traditional problems, human factors solutions. Paper presented at IBC Conference Human Factors in Nuclear Safety, 23-24 April 1994, London. Marsden, P. and Green, M., 1991. Boiler feedwater pump start up: A case study in procedure preparation. Human Reliability Associates, Dalton, Wigan. Morgenstern, M.H., Barnes, V.E., McGuire., Radford, L.R. and Wheeler, W.A., 1987. Study of operating procedures in nuclear power plants. NUREG CR-3968. Battelle Human Affairs Research Centre, Seattle, WA. Rector, A.L., Newton, P.D. and Marsden, P., 1985. What kind of an expert does a system need? In: P. Johnson and S, Cook. (Eds.), People and Computers: Designing the Interface. Cambridge University Press, Cambridge. Seminara, J.l., Gonzalez, W.R. and Parson, S.G,, 1976. Human factors review of nuclear power plant control room design. Electric Power Research Institute Report NP-309 Chapter 17, November, 1976. Swezey, R.W., 1987. Design of job aids and procedure writing. In: G. Salvendy (Ed.), Handbook of Human Factors. John Wiley, New York. Zach, S.E., 1980. Control room operating procedures: Content and format. Proceedings of the Human Factors Society Annual General Meeting, 1980.
International Journal of Industrial Ergonomics 17 (1996) 43-51
Optimising procedures in manufacturing systems Philip Marsden, Mark Green Human Reliabili~ Associates Ltd, I, School House, Higher Lane Dalton, Wigan, Lancs, WN8 7RP. UK
Received 10 July 1994; revised 22 September 1994
Abstract
A significant amount of time and effort in manufacturing operations is directed toward the development of procedures intended to ensure that work activities are carried out in a safe and efficient manner. Despite the often high level of investment, however, it appears that there are a number of problem areas associated with procedure use where the systems frequently fail to meet their intended objectives. The cost of failures in a procedure system can vary quite widely dependent upon the nature of the operation. In most cases the consequences of failure have only a negligible effect on the overall performance of a company and can be measured in terms of production hours. Occasionally, however, the consequences of failure can be catastrophic and result in significant damage to people, plant and the environment. The purpose of this paper is twofold: The first objective is to provide a preliminary review of common problems which are associated with the catastrophic failure of procedure systems in a manufacturing environment. The second aim is to consider some of the ways that a human factors approach might be used to help prevent the more serious failures associated with the use of procedures in industrial settings.
Relevance to industry
A ~eliable and well-managed system for procedure design and use is an important component in a safe and efficient manufacturing process. Poorly designed procedures may be misinterpreted or even ignored by management and operational staff, which in turn can result in major incidents involving human casualties and production loss. Keywords: Procedures; Training; Accidents; Human factors
I. Introduction
T h e safe a n d efficient o p e r a t i o n o f any m a n u f a c t u r i n g system is highly d e p e n d e n t u p o n the existence o f a g r e e d m e t h o d s of work, w h e r e b y a c o m p a n y specifies how it w a n t s c e r t a i n w o r k activities to be p e r f o r m e d ( G r e e n a n d M a r s d e n , 1992; M a r s d e n a n d W h i t t i n g t o n , 1993). T h e m o s t
usual way of c o m m u n i c a t i n g such m e t h o d s to e m p l o y e e s is via w r i t t e n instructions or p r o c e d u r e s (Folley, 1975; Swezey, 1987). D e f i n e d b r o a d l y t h e m a j o r f u n c t i o n o f a p r o c e d u r e is to e n s u r e t h a t all tasks a r e c a r r i e d o u t in a safe, efficient a n d c o n t r o l l e d m a n n e r a n d this a p p l i e s e s p e c i a l l y to o p e r a t i o n s t h a t involve a significant hazard component.
0169-8141/96/$15.00 © 1996 Elsevier Science B.V. All rights reserved SSDI 0169-8141(94)00102-2
44
P. Marsden, M. Green/International Journal of lndustrial Ergonomics 17 (1996) 43-51
Problems associated with the use of procedures in an industrial situation have been reviewed on a number of occasions. In the nuclear industry, for example, Michael Morgenstern and his colleagues (Morgenstern et al., 1987) concluded that the vast majority of procedures reviewed failed to attain a classification considered "minimally acceptable". Similar conclusions were reached by Zach (1980) following a study of control room operating procedures. In this investigation it was found that procedures tended to be inadequate both in terms of the amount of information they contain and the way the information is presented. Finally, in a seminal study of maintenance errors reported to the U.S. Nuclear Regulatory Commission during the period 1975-1978, Brune and Weinstein (1982) concluded that procedural deficiencies were implicated in more than half of employee errors. Consequently, the authors suggested that the general quality of maintenance procedures constituted a major source of vulnerability in industrial operations of all types. Each of the studies reported above raise serious doubts about the way that procedure systems are designed and operated in an industrial situation. The purpose of the present paper is to identify some of the ways that procedure systems can go wrong. Following this, we shall examine what, if anything, can be done to improve the general quality of procedure provision in a manufacturing environment. The account draws heavily upon the experience of dealing with procedures in the context of Nuclear Power Plant (NPP) operation, and on recent developments made within the field of human factors engineering. Despite the emphasis on NPP operations it is the view of the authors that there are many lessons to be learned from a consideration of procedure failures in a NPP domain that will be directly relevant to the design of procedure systems in all sectors of industry.
2. Overview of the a r g u m e n t
In essence, the theme of this paper is that there are three traditional problem areas associated with the use of procedures in a manufactur-
ing environment. Described in overview we propose that procedure systems tend to fail for three basic reasons: - Because of weaknesses inherent in the way that procedures are prepared; - Because people frequently fail to perform tasks in the way that procedures prescribe; - Because systems within the organizational infrastructure that should provide support for procedure systems tend to be poorly structured and operated. Human factors investigations have been active in each of the above areas and have produced general guidelines that can be used to address each problem type.
3. Procedure preparation
In the afternoon, on the 10th of March 1983, an engineer working at Quad Cities prepared a note explaining the procedure for performing a shutdown scheduled to be undertaken later that night by the evening shift. The shift supervisor incorrectly interpreted the procedure as instructing him to insert control rods in an order that was the reverse of that required under normal circumstances. Prior to the arrival of the night shift supervisor, the evening shift operators incorrectly installed 33 rods. In order to do this, the shift team needed to by-pass a safety critical system (the rod worth minimiser, or RWM) designed specifically to prevent improper rod insertion. Following shift hand over, the night shift supervisor initially restored the integrity of this system. Because the system prevented further rods to be inserted into the core in the manner specified, however, the decision was taken that the RMW was faulty and the systems was again by-passed. A further 10 rods were wrongly inserted into the core, and the reactor was shut down without incident. A subsequent investigation concluded that the near-miss incident had created the basic conditions for a rod drop accident and imposed a $150,000 fine on the owning company, Commonwealth Edison. Many reported incidents involving procedures occur as a result of weaknesses inherent in the
45
P. Marsden, M. Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
way that procedures are prepared. The near-miss accident cited above, for example, would seem to provide an interesting case study of how procedures can sometimes fail when they are prepared by "domain experts" who are familiar with plant and process theory but who have little idea regarding how the system works on a day-to-day level. Indeed, it is by no means uncommon for procedures prepared by highly qualified technical personnel working in isolation to be incomplete, incorrect a n d / o r generally unrealistic with regard to the task they describe. There are at least three important reasons why domain experts may not be the group of people best qualified to prepare new work procedures. First, because product specialists tend to perform a quality control function in most organizations, they are rarely aware of the specific details which lie behind the way that operators actually fulfil their work roles. In fact, as more than one study has shown (e.g., Marsden and Green, 1991), they may not even be aware of recent modifications made to plant and equipment and it is not uncommon for procedures to be written by experts which incorporate instructions for plant and equipment no longer in service. The second problem with using a domain expert to prepare work procedures arises as a result of the process of acquiring expertise. We now know, for example, that the essential paradox of expertise is that the more a person comes to understand a particular job or work role, the less able they are to verbalise their expert knowledge (Rector et al., 1985). This means that an expert will often prepare an instruction which fails to include important details. It is precisely for this reason that the discipline of knowledge engineering was formed in the early 1980's (Hayes-Roth et al., 1983). Finally, domain experts tend on the whole to be unfamiliar with information presentation methods which have been developed by human factors specialists to help optimise the information take-up of the end user. Consequently, experts frequently produce documentation which is poorly formatted and the failure to provide userfriendly procedures can effectively render the instruction unusable as an operator aid.
~[ Step I ~1 "t Task Analysis
Step 2 Select format Step 5 Quality assure Step 3 Draft procedure Step 6 Monitor/review effectiveness
Step 5 Approve procedure
Fig. 1. A development process for optimising procedure quality. (Based on Embrey, 1986).
One way to reduce the likelihood of preparing deficient procedures involves the specification of a formal procedure development process such as the one proposed by Embrey (Embrey, 1986). He argued that optimal procedure development requires a six-stage process incorporating two design iterations. A graphic representation of the development process discussed by Embrey is provided in Fig. 1. Each of these stages are discussed further below.
3.1. Step 1: Task analysis In this scheme, the first step in procedure preparation involves carrying out a systematic analysis of the task using a formal method such as Hierarchical Task Analysis (HTA). In essence, HTA is methodology for breaking tasks down into sub-goals and specifying the way these goals are to be combined in order to achieve the overall objective of the task. At the highest level of analysis, tasks are defined in terms of work plans which take the form of sub-goal sequences (e.g. 1. Set up the gland water system: 2. Establish ACW supply, etc.). At the lowest level, each sub-goal is described in terms of sequences of object-action pairings such as "1. Open valve PVI01", "2. Close valve PSV 54". There are two major advantages to be gained by using a method such as HTA for procedures development. First, it forces
46
P. Marsden, M. Green /International Journal of Industrial Ergonomics 17 (1996) 43-51
the expert to make what He or She knows verbally explicit and this permits the accuracy and completeness of operational knowledge to be tested objectively prior to implementation as a procedure. Second, it provides an opportunity to involve end users in procedure preparation and this can help encourage feelings of ownership.
3.2. Step 2: Procedure preparation Once the task has been formally analysed, the next step in the process is the selection of an appropriate format. Several factors may need to be taken into consideration when choosing a format for the procedure, but two of the most important considerations are task complexity and user familiarity. As a rule, as the operators' familiarity with a task increases and the task complexity decreases, the more likely it is that operators will need a Job Performance Aid (JPA) which uses a checklist format. Conversely, as a task complexity increases and familiarity diminishes, then a much greater level of operator support will be required, and this would best be provided by using a step-by-step procedures format. Alternative procedural formats may be required for jobs involving more than one person (e.g. playscript formats, etc., Berry, 1981)
3.3. Step 3: Preparation of draft procedure The third stage of development involves preparing a draft document based upon task elements determined in step 1, presented in the format determined in step 2. In preparing the draft document, several decisions may need to be made regarding the design of the layout of the procedure. Once a layout has been devised, Embrey suggests that it should be optimised by checking for consistency with human factors guidelines which specify best-practice for information-presentation.
3.4. Step 4." Quality assurance Prior to approving the procedure for general use, several tests of the draft procedure will need to be undertaken and these should be performed
by suitably qualified inspectors. Wherever possible, testing should take place in the actual operating environment. Embrey suggests that it is desirable that a series of end-user consultations is also undertaken during this phase of development to ensure that the proposed method of work is consistent with current practices. All results should be fully documented and this information can be used to make modifications to the draft procedure. Incremental refinement of the procedure may necessitate several design iteration between steps 1 to 4.
3.5. Step 5: Approving the procedure for general use In the fifth stage, the procedure is approved for general use. New procedures will often need to have special controls a n d / o r limitations placed on their use such that only certain qualified personnel would be allowed to perform the procedure on the first few occasions. This qualification would be especially relevant to tasks involving a significant hazard component.
3.6. Step 6." Long-term maintenance The final aspect of Embrey's (1986) proposed process involves the long-term maintenance and updating of the procedure over the lifetime of the system. He suggests that routine checks of the effectiveness of the procedure be made periodically to assure the long-term applicability and usability of the procedure over time.
4. Non-compliance with written instructions
On the 18th of September, 1988 at Stade PWR in the Lower Saxony region of West Germany, an electrical failure caused a valve to close in one of four main steam lines. As a direct result of the closure other valves in the remaining three lines automatically closed, blocking all four lines. The automatic reactor protection system at this point should have scrammed the reactor and turbine. Contrary to procedures however, operators overrode the system and reopened the valves in an attempt to keep the reactor running. The valves
P. Marsden, M.
Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
quickly closed for a second time, and the system scrammed. Opening and closing of the steam line valves in rapid succession led to a violent pressure wave in the main steam lines causing a
maximum displacement of 20 cm. This was more than double the acceptable limit and an investigation concluded that the actions of the operators brought the steam lines close to failure.
BFP/3A/06 ] OBJECTIVE" COMMISSION AUXILIARY LUBRICATION OIL SYSTEM
'or H~d-~s
I
-I I Removeauthorir~=d]
Hold-Offs
Note isolation
number
I
..... °1 Establishlub. tank level
Jnform operator Ensureoil flow present
I SIGHT: I GLASS
$ >0.8 bar.~
~Y~s
I
Monitorfilter J~ pressuredrop J~
-[
drop <40kpa?J ~YES Aux. lub. oll ~tem now commi~oned
I
Selectother filter
47
Inform operator I
Reportto
sh~ ofrce
Fig. 2. Objective: Commission auxiliary lubrication oil system.
48
P. Marsden, M. Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
N o n - c o m p l i a n c e with p r o c e d u r e s f e a t u r e s p r o m i n e n t l y in statistics covering a c c i d e n t a n d n e a r miss reports. I n a survey o f 180 significant
BFP/3A/06
]
n u c l e a r events c i t e d by G r e e n (1991), for example, m o r e t h a n 18% o f all a c c i d e n t s a r e a t t r i b u t e d in s o m e way to the failure o f p e r s o n n e l to follow
OBJECTIVE"
COMMISSION AUXILIARY LUBRICATION OIL SYSTEM
|
:::::
: :
:::
:.
: :
SUPPLEMENTARY INFORMATI O N l. Check system for 'Hold-Offs' l. l l.x~cate Hold-Offs on this system 1.2 Remove authorised Hold-Offs
1.2 Hold-Offsare authorisedbyProofof Isolationnumber providedin BFP/3A/01 1 . 3 N O (a) Note Proof of Isolation number
1.3 Are all Hold-Offs removed?
0,) R ~ to shiftom,,e YES Proceedwithnextstep
2. Prepare lubricating oil system for start up 2.1 Close filter vents 2.2 Select A or B filter 2.3 Read lubricating oil level in tank 2.4 Is oil level greater than 0.2m?
2 . 4 N O (a) Notelevel (b) Inform operator YES
Proceed with next step
3. Start Auxiliary lubricating oil pump ~. 1 Ensure oil flow present in sight glass 4. Monitor pump pressure 4. l Is pump pressure greater than 0.8 bar?
4. At Io~ eonla'ollmnel
4"INO Inform operator
YES ~
withnextstep
5. Monitor filter pressure drop across filters 5.1 Is pressure drop across filters greater than 40 kpa?
~lNO Pro,,~ withnm step YES (c) g / ~Inform P"~m ~ o~mor (d) ~ t
step 3. I - - i f OK proceed
Auxiliary lubricating oil system now commissioned
Fig. 3. Objective: Commission auxiliary lubrication oil system.
P. Marsden, M. Green/International Journal of lndustrial Ergonomics 17 (1996) 43-51
procedures. Similarly, Brune and Weinstein (1982) found failure to comply with the prescriptions contained within procedures contributed significantly to procedure-related performance errors in activities involving the maintenance, test and calibration of equipment. From the point of view of the user, it is possible to distinguish between two broad classes of non-compliant behaviour: intentional non compliance (e.g. routine violations, unsafe acts, etc.) and unintentional non-compliance (e.g. a procedure was used but an action was omitted, a procedure was used but an action was performed incorrectly, etc.). Of these two types, intentional noncompliance is potentially the most serious, and it is this aspect of behaviour that we are primarily concerned with here. In attempting to provide an answer to the question "Why don't people follow procedures?", it is useful to begin by considering what users typically think about procedures. As part of a study of employee attitudes reported by Zach (1980), a large group of operators were asked to say why they thought procedures were needed. Interestingly, the most conspicuous answer was that procedures were required primarily for the purpose of regulatory inspections and little mention was made of the fact that they serve as operator aids. In fact, procedures were commonly viewed as the bane of the operator's life and were perceived to devaluate the work role. In particular, operators reported that working with procedures makes work much less rewarding and the job more difficult than it would otherwise be. These few comments provide an important insight into why people sometimes fail to comply with a procedure. Procedural systems are typically viewed by operators as a system of work control designed essentially to protect the company in the event of an accident. Consequently, operators report that they have little confidence in the usefulness of procedures as usable work aids. Unfortunately, this view is not without some foundation. When pressed, most operators are able to recall occasions were non-compliance with a procedure was sanctioned (either implicitly or explicitly) by supervisors or others in the organisation. This tended to occur particularly when
49
production pressures were high. Inconsistencies such as these tend to undermine a compliance policy and leave areas of ambiguity regarding when procedures are to be followed. What changes would operators like to see made to procedures? When this question was put to operators in two studies carried out by Zach (1980) and Seminara et al. (1976) answers tended to be concerned largely with the ongoing debate between the relative merits of "lock-step" (i.e. high compliance) versus guidance (low compliance) procedures. The greater proportion of answers indicated a preference for guidance documentation which places much more reliance on the personal initiative and experience of the user. Consequently, operators expressed a desire to see more concise forms of procedure which described tasks only in terms of important details. Users also indicated a preference for flow diagram (as opposed to text-based) procedures. They suggested that flow make it easier to identify key elements of the task quickly and to relate specific actions to overall objectives. On this issue, however, it was acknowledged that there is always some loss of information involved in the use of flow diagrams for complex text. Examples of a procedure specified using both a flow chart and text-based presentation methods are provided in Fig. 2 and Fig. 3. Finally, the operators indicated that they would like more formal training relating to procedures of all types. Training with procedures in mind is an important issue in its own right and will be considered more fully in the following section. The major point which is being made here is that none of the above changes are unreasonable provided that adequate user support is made available. Moreover, being seen attempting to address these problems could help ameliorate the potential threat of serious incidents arising from the failure of users to comply with a written instruction.
5. Procedures infrastructure
On February llth, 1981, at Tennessee Valley Sequoyah I plant, a unit operator instructed a
P. Marsden, M. Green/International Journal of Industrial Ergonomics 17 (1996) 43-51
50
recently qualified employee to check the containment spray valve and ensure that it was closed. Instead of closing the valve the operator opened it, and as a direct result 40,000 gallons of primary cooling water was released into the containment area, along with a further 65,000 gallons of water held in a storage unit. Eight workers were contaminated as a result of the accident which involved low grade radioactive materials. The reactor was fortunately in a cold shutdown condition at the time of the incident thus preventing the occurrence of a more serious accident. The final source of vulnerability to be considered here deals with those situations where the procedures themselves are not directly at fault, but rather, where there has been a failure of some type in the system which should provide some support for the major user groups. Of particular concern are those incidents that occur because the operator is inadequately trained to use the procedure, or where problems arise because of a failure to monitor the effectiveness of procedures. The most common scenario for a failure of this latter kind is the situation where a procedure is not revised following an operational failure of some type. The basic design philosophy proposed by Human Reliability Associates to counter problems of this type advocates the development of a procedural system comprised of three interrelated elements: A Procedures database (or inventory), a training function and a system for monitoring and updating. These three elements and their interrelationships are shown in overview in Fig. 4.
/
"
/MONITORING/ "~
\\T .... g
Training\ \ needs Prog. . . . . ~
Aod,t//
trails// Feedback-driven ~/tell ..... t
; PROCEDURES "~
Fig. 4. Minimum structural requirements for an effective procedures infrastructure. (Source: Green and Marsden, 1992).
In this type of system, procedures are explicitly related to training in so far as they are developed specifically with training needs in mind. For example, analysis of the structure of the procedure will imply particular training requirements and this information can be used to formulate explicit training strategies. Similarly, the procedures themselves can be used as a primary source of training materials and these are presented to trainees as part of a structured training programme. Employee qualification is based upon methods of testing in which the trainee must demonstrate his or her competence relative to particular sets of procedures. Procedures are also formally linked in the system to a monitoring function. In this case it is proposed that quality points be built in to procedures that lay down a clear audit trail each time the procedure is used. The audit trail can be periodically examined and checked for compliance, and reviews of procedure quality should be used to modify the procedures in a process of feedback-driven refinement. There are several important points which follow from the implementation of a procedures infrastructure of the type proposed. First, it is a mistake to think of procedures as finalised static documents once they are written. Rather, they should be thought of as "living documents" which always exist in a state of latest revision. People should always be encouraged to challenge the prescriptions contained within procedures with a view to upgrading safety or production targets. Second, employee training must work for the benefit of both the company and employee alike. Such objectives are best served by the use of proactive training methods involving a blend of on-the-job versus formal training methods which deliver both technical (relating to the process) and behavioural (relating to task skills) knowledge. It is not sufficient to assume competence based upon the principle of time served. Rather, formal methods of competence testing should be employed which require the trainee to demonstrate his or her skills relative to current procedures. Finally, the process of rapid prototyping and feedback-driven refinement is more productive when the monitoring process is implemented
P. Marsden, M. Green / International Journal of Industrial Ergonomics 17 (1996) 43-51 in a blame-free environment. People should always be encouraged to report their mistakes and near-misses so that procedural implications can be evaluated and error-tolerant solutions designed in an attempt to minimise the likelihood of catastrophic system failure.
6.
Conclusions
There are three traditional problems areas that are traditionally associated with the use of procedures in manufacturing system. In particular, it was proposed that procedure systems frequently fail to attain their desired objectives: Due to weaknesses that are inherent within the preparation process; Because people fail to comply with an established procedure; Because systems which should provide support for procedures are often poorly designed and implemented. It was suggested that improvements in each of these areas can be made by: Adopting a formal development process; Listening to the views of users and involving them in methods of work control; Developing a strong procedures infrastructure which explicitly relates procedures to a training and monitoring function. It is not suggested that implementing each of these recommendations will guarantee that future procedure provision will be perfect. Experience suggests, however, that they will lead to significant improvements being made in each of the above areas should the principles set out in this paper be applied.
51
References
Berry, E., 1981. How to get users to follow procedures. IEEE Transactions on Professional Communication, 25: 21-26, 4. Brune, R.L. and Weinstein, M., 1982. Procedures evaluation checklist for maintenance, test and calibration procedures used in the nuclear industry. NUREG CR-1369. Sandia National Laboratories, Albuquerque, NM. Embrey, D., 1986. Guidelines for the Preparation of Procedures for the High Reliability of Ultrasonic Inspection Tests. Human Reliability Associates, Dalton, Wigan. Folley Jnr., J.D., 1975. Research issues in JPA technology development. Paper presented at the SALT-Improved Information Aids, LMI Conference on Arlington, VA. Green, M., 1991. Root Cause Analysis. Human Reliability Associates, Dalton, Wigan. Green, M. and Marsden, P., 1992. The design of effective procedures for industry. 1992 Ergonomics Society Annual Conference. University of Aston, Birmingham. Hayes-Roth, F., Waterman, D.A. and Lenat, D.B., 1983. Building Expert Systems. Addison-Wesley, London. Marsden, P. and Whittington, C., 1993. Work instruction provision: Traditional problems, human factors solutions. Paper presented at IBC Conference Human Factors in Nuclear Safety, 23-24 April 1994, London. Marsden, P. and Green, M., 1991. Boiler feedwater pump start up: A case study in procedure preparation. Human Reliability Associates, Dalton, Wigan. Morgenstern, M.H., Barnes, V.E., McGuire., Radford, L.R. and Wheeler, W.A., 1987. Study of operating procedures in nuclear power plants. NUREG CR-3968. Battelle Human Affairs Research Centre, Seattle, WA. Rector, A.L., Newton, P.D. and Marsden, P., 1985. What kind of an expert does a system need? In: P. Johnson and S, Cook. (Eds.), People and Computers: Designing the Interface. Cambridge University Press, Cambridge. Seminara, J.l., Gonzalez, W.R. and Parson, S.G,, 1976. Human factors review of nuclear power plant control room design. Electric Power Research Institute Report NP-309 Chapter 17, November, 1976. Swezey, R.W., 1987. Design of job aids and procedure writing. In: G. Salvendy (Ed.), Handbook of Human Factors. John Wiley, New York. Zach, S.E., 1980. Control room operating procedures: Content and format. Proceedings of the Human Factors Society Annual General Meeting, 1980.