- Email: [email protected]
Anatomy of an incident—Multiple failure of safety systems under stress
JCHAS-958; No of Pages 6
RESEARCH ARTICLE
Anatomy of an incident— Multiple failure of safety systems under stress Most laboratories are mostly safe most of the time. It has been said that laboratory safety does not follow Murphy’s law — what can go wrong will usually not go wrong, and so flawed and incomplete safety systems are maintained until too late.1 Unusual, high-stress situations can reveal the flaws and inadequacies at the cost of an incident. Here, I describe one such incident: how a day of electrical shut down in the laboratory, a continuous alarm, the presence of non-staff contractors, language barriers and lack of clear channels of communication lead to a failure to properly evacuate and properly investigate the source of the alarm. The development of new procedures and systems proofed against these problems is also described, as well as their implementation.
By Hugo Schmidt INTRODUCTION
Funded by the Singapore National Research Foundation, the Cambridge Centre for Advanced Research and Education in Singapore (Cambridge CARES) is a subsidiary of the University of Cambridge. It is located on the National University of Singapore’s Campus for Research Excellence and Technological Enterprise Campus (CREATE), which hosts several such co-operative initiatives. Cambridge CARES leases the top, seventh floor of CREATE’s RESEARCH Tower for the CARES chemical engineering laboratories. While CARES manages its own laboratories, the overall infrastructure of CREATE is maintained by the CREATE Building Management Office (BMO). This complex structure of responsibilities lead to the incident on the 18th of February, 2017. Gas leaks are a main concern of safety management, as asphyxiant, toxic, and explosive gases are in use throughout the two CARES laboratories. Between them, both laboratories Hugo Schmidt is affiliated with Cambridge CARES, Singapore, Singapore (E-mail: [email protected]. uk).
1871-5532 https://doi.org/10.1016/j.jchas.2017.10.005
have 111 gas sensors, and triggering a sensor leads to a loud gas alarm. To perform annual maintenance, CREATE BMO shut down the electricity on the entire CREATE RESEARCH Tower. The CREATE BMO engaged contractors to carry out this annual servicing. This had been communicated to all research groups working in CREATE, and CARES had instructed its workers to not visit its laboratories on this day. In response to the electrical shut down, the RESEARCH tower switched to the emergency backup that maintains a minimal level of energy for certain functions, including the gas alarms. At 11:00 a.m., a gas alarm was triggered. However, the contractors present at the CARES laboratories did not evacuate the premises, even after repeated urgings. Flaws in laboratory safety procedures often go unnoticed for long periods of time.1 This near-miss incident was the result of the breakdown of several systems. First, the contractors had not been properly briefed, on what to do in the event of an alarm. Second, there was a confusion because of chronic alarm caused by the electrical shut down. Third, during the alarm, language barriers made communicating with the contractors difficult. Fourth, there were problems with the procedural structures to deal with such an unusual situation.
In this paper, I describe the incident and the analysis that was applied. I discuss the multiple causal factors that lead to the near-miss incident, and the corrective measures undertaken to ensure that such a problem does not arise again. The results provide the basis for how to handle such a situation, including guidelines on a procedure with redundancies for handling contractor presence. Description of Incident
Cambridge CARES maintains two laboratories on level 7 of the CREATE RESEARCH building. To minimize gas risk, all compressed gas cylinders are stored in one of two gas manifold rooms. Each gas manifold room is secured with a four-hour rated fire door as well as gas alarms. The total air volume in each gas manifold room is changed twenty times an hour. Flammable and explosive gases are further contained in gas cylinder cabinets, each rated two hours fire resistant. Gas flow from the cylinders to the laboratories is controlled by multiple valves, including a solenoid valve that shuts automatically in the event of a gas alarm. The opening and closing of the solenoid valves is controlled by a gas control panel in the corridor outside of the laboratories. Gas alarms caused by breaches in the flammable gas co-axial lines are indicated by a pair of lights specific to each line,
ã 2017 Published by Elsevier Inc. on behalf of Division of Chemical Health and Safety of the American Chemical Society.
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
1
JCHAS-958; No of Pages 6
Figure 1. (a) Outline of laboratory 1 of CARES showing location of gas manifold rooms, (b) Gas manifold room, (c) Gas control panel, (d) beacon lights.
and other gas alarms are indicated by a code displayed on the central display (see Figure 1a–c). The gas alarm has two stages, depending on severity of gas leak: yellow, requiring a general shut down of experiments and investigation of the leak, and red, requiring an immediate evacuation. A yellow gas alarm is triggered by the presence of 5% of a given gas’ dangerous level, and the red alarm by 10%. For example, the yellow alarm would trigger in response to 5% of the Lower Explosive Limit (LEL) of an explosive gas such as ethylene, and the red alarm would trigger at 10% of the LEL. The red alarm is louder than the yellow alarm and continuous, and the difference in alarms is further indicated by beacon lights prominently placed around the lab that show exactly which stage of the alarm has been reached (see Figure 1d). The Incident occurred on the 18th February, 2017. To conduct annual maintenance, the Building Management Office (BMO) had shut down electricity throughout the RESEARCH Tower. The lab had previously been shut down, and the building was secured. Contractors were on site to conduct maintenance and testing that can only be done when the electrical systems are shut down. The lab manager was on site, but in the CREATE office that is in the CREATE Tower, rather than the RESEARCH Tower. At 11:00 a.m., the lab manager received a call from the BMO that a gas alarm had sounded 2
at level 7. This was the only way the lab manager could have received this information, as the usual Building Management System Alarm Notification had been deactivated, along with the rest of the equipment at level 7. Arriving at Level 7, the contractors were unsure what the alarm meant. Despite repeated urgings, the contractors refused to evacuate. The contractors maintained that the BMO had informed them that the alarm was only a side-effect of the power shut-down. The lab manager called the BMO, and determined that the BMO only referred to the standard ‘power-outage alarm’, not the gas alarm. The lab manager returned and tried again to persuade the contractors to evacuate. Despite some initial resistance, after some discussion, the contractors agreed to evacuate, leaving behind the lab manager and the representatives of the BMO trying to resolve the situation. No census was taken, and it was unclear how many contractors were on site. To make sure that no casualties were present in the laboratory, the lab manager donned the Self Contained Breathing Apparatus (SCBA) and investigated both laboratories to see if any unconscious casualties were present. No casualties were present. The compressed gas cylinders had already been sealed by the automatic closing of the solenoid valves. To ensure further safety, the lab manager sealed all gas supplies at the cylinders
themselves, before returning to the gas panel outside of the laboratory. The gas panel was opened, but it proved impossible to reset the alarm, as the password was not available. It was also not possible to determine which gas sensor had been triggered, as it was indicated by a code on the LED display. The premises were evacuated, and the security staff contacted to ensure that no one would enter the building until it had been deemed safe. The gas safety contractor was called and sent to CARES. The BMO staff left, and the lab manager awaited the contractor’s arrival. On arriving, the gas safety contractor and lab manager proceeded to the laboratories to investigate the source of the alarm. After investigation, the gas safety contractor concluded that there was no significant risk and that the alarm was likely the result of sensor malfunction. The alarm was deactivated, and the sensor investigated, discovering a faulty part that had triggered the alarm. The contractor safely removed the part. In summary, the unusual situation stressed sequential safeguards beyond breaking point. Had the alarm sounded under normal working hours, evacuation would have been the normal response. Similarly, had the electrical system not been shut down, there would not have been a chronic alarm condition that confused the issue of the gas alarm. Causal Factors
To determine the causes that lead to the failure of evacuation and the difficulty in responding, I carried out a cause-and-effect analysis2 (Figure 2). Management Factors
The contractors were misled by the briefing provided by the BMO, leading them to conclude that the gas alarm was simply the result of electrical power-down. The CARES Lab Manager was present in the office rather than the laboratory. Neither Cambridge CARES nor the BMO properly instructed the contractors as to the importance of evacuation.
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
JCHAS-958; No of Pages 6
Figure 2. Failure to evacuate cause and effect diagram.
Neither Cambridge CARES nor the CREATE BMO required contractor management to designate a representative who would take instruction from CARES’ lab manager. BMO and CARES representatives were unsure as to who was present on the Level 7. The Emergency Response Plan (ERP) was unclear on whether to contact the Site Controller in this instance. There was no clearly defined laboratory shut-down procedure that covered both the unplugging of all equipment and the shutdown of all gases, with a designated verification that the shutdown had been completed.
There was no way to use the gas panel due to an absence of the password. Furthermore, it was difficult to determine which gas sensor had triggered the alarm.
A language barrier existed between the Lab Manager and many contractors (Table 1).
Equipment Factors
DISCUSSION
During the alarm, the lifts were still operational and there was no way to indicate – other than the alarm – that the level was off limits. The gas control panel lacked a clear placing of the number for the gas safety contractor. The gas control panel lacked a clear description of which alarm code corresponded to which sensor. There were no signs indicating that an alarm mandated evacuation. Manpower Factors
Engineering Factors
The volume of gases, and their potential spread was not known. Nor was their likely persistence under different conditions known. The SCBA was located in the Mechanical and Engineering Room, which was locked. In the absence of the lab manager, or another designated responsible person, it would have been inaccessible. The loud alarm made communication with the BMO staff difficult
The contractor staff did not listen to the Lab Manager’s instructions. No leader of the contractors was designated. There was no clear understanding of how many people were present on level 7. Chain of responsibility and authority was unclear with the use of contractors, especially contractors not commissioned by CARES. There was no communication between CARES and contractors following evacuation.
Extreme cases may make bad law, but they do make very good case studies. Systemic failures that occur when a system (such as the lab safety and health procedures) is stressed due to unusual circumstances reveal failure points in the system that are not apparent in usual operations. As shown in the case study, the incident arose through the sequential failure of multiple systems. The risk of the lab manager searching the lab for casualties unsupported was caused first by the failure to properly secure all gas cylinders in advance of the lab shutdown, which meant that the possibility of a gas leak was serious; second, by the lack of a list of contractors and a lack of verbal confirmation that the evacuation was complete, which meant that the absence of casualties could not be assumed; third, by the lack of provision for back-up to the lab manager in the event of a necessary SCBA use. Any of these would have decreased the risk.
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
3
JCHAS-958; No of Pages 6
Table 1. Failure Points and Corrective Actions.
Failure Point
Corrective Action Proposed
Responsibility
Failure to brief all contractors and contractor management in advance.
CARES shall develop a contractor procedure that stipulates that all contractor management must be informed in advance that evacuation on hearing an alarm is mandatory, as is evacuation in response to direction by the lab manager or other CARES staff. Signs shall be posted instructing all staff and contractors that evacuation in the event of a red gas alarm/second stage fire alarm is mandatory. Signs shall be prepared in English, Malay, Tamil and Standard Mandarin. CARES shall request contractor management to provide CARES with a list of the staff sent to work at CARES premises, designating one worker as an official representative who shall form point-of-contact. The representative shall be required to speak conversational English. Further, the CARES Lab Manager or other designated responsible person shall brief all contractors on their arrival, taking a roll call to make sure that the list provided by the contractor management is accurate. These requirements will be communicated to all contractors that will work in CARES or at the 7th level of the CREATE RESEARCH Tower, regardless of whether they have been contracted by CARES or by the CREATE BMO. These requirements will be communicated by both CARES and the CREATE BMO.
CARES Lab Manager.
Contractors would not listen to instruction by lab manager. Chain of responsibility and authority was unclear.
No communication between contractors and CARES following evacuation.
Language barrier between contractors and CARES Lab Manager. Failure to distinguish between a chronic alarm triggered by power outage, and an actual alarm requiring evacuation. Difficult to know whom to contact in this situation.
Communication difficulties due to alarm. CARES Lab Manager present in office rather than lab. BMO unsure who was present at level 7.
Self Contained Breathing Apparatus in locked mechanical room.
4
CARES Lab Manager.
CARES Lab Manager. Building Management Office
The BMO shall inform CARES and contractors if there is a chronic alarm, and which alarm is meant. The BMO shall also say that all other alarms should be treated as real, with full evacuation. CARES will repeat these instructions to all contractors when they arrive at the laboratories.
Building Management Office CARES Lab Manager
All induction materials, signs etc. shall be updated to include the name and contact details of the gas safety contractor directly responsible for the gas alarm. The Emergency Response Plan shall be similarly updated to state that the gas safety contractor is to be contacted in the event of a gas alarm. Contact person and contact number shall be posted prominently next to the gas panel. Given the loudness of the alarm, evacuation shall proceed first. All supplementary questions to be posed to BMO, CARES management etc. only after the building is clear. This shall be made clear to all contractors. CARES lab manager shall be present in the laboratory for all servicing that must be conducted.
CARES Lab Manager
The BMO shall be requested to have a full list of all contractors who are at level 7, and present this list to CARES, bearing in mind this may include contactors unconnected to CARES who are working on the roof infrastructure. All contractors will be required to sign in and out on entering the CARES labs. One of the two SCBA shall be placed in the CARES office to be easily accessible if the M&E room is inaccessible.
CARES Lab Manager Building Management Office CARES Lab Manager.
Building Management Office
CARES Lab Manager CARES Lab Manager
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
JCHAS-958; No of Pages 6
Table 1 (Continued )
Failure Point
Corrective Action Proposed
Responsibility
Lab Manager investigated site without backup.
A backup shall be in place in any situation where the use of the SCBA is necessary. If CARES staff are not present, a representative of the Building Management Office should be present. Lab air conditioning systems shall be updated to include a purge capacity. The purge capacity should increase the removal of air to maximum, dispensing dangerous gas. The password for the gas panel shall be placed within the gas panel, next to the manual. The gas panel can be opened only with the key that either the CARES lab manager or the gas contractor has. In the event of lab manger absence, the keys are passed to a designated substitute lab manager. Spare copies of the key are lodged with the program manager. CARES shall investigate the possibility of barriers or traffic cones to indicate that the level is off-limits, along with signage indicating the same. The feasibility of more substantial barriers shall be investigated. Lifts should be grounded in the event of an alarm.
CARES Lab Manager
Problem with dissipating gas. Password for gas panel unknown.
No way to secure laboratory.
The Emergency Response Plan was unclear as to whether to contact Site Controller in event of a non-CARES evacuation. Lack of clear lab shutdown procedure. Unsure of potential gas spread.
Power cut created unusual conditions.
CARES Lab Manager shall meet with CARES Site Controller to establish the correct course of action in this event.
CARES Lab Manager shall develop a comprehensive procedure for the shut-down of the laboratory and its reactivation. CARES shall investigate the volume of gases that can be potentially released, as well as the time that it would take to clear, both under normal operations and in event of powershut down. BMO shall thoroughly debrief all contractors and CARES on how the infrastructure will behave in this condition.
The failure of the contractors to evacuate, was caused first by the absence of adequate briefing prior to beginning work; second, by a failure to distinguish the chronic alarm from the real alarm; third, by a lack of a designated contact person and a clear chain of responsibility which made the language barrier an obstacle to the evacuation. If any of these elements had been corrected for, evacuation would have occurred as normal. This is a problem that can affect all laboratories, but laboratories in multi-lingual societies (such as Singapore) which are the result of inter-institutional cooperation (such as CARES) are particularly vulnerable to it. The most fundamental problem was the failure of the contractors to be
briefed that evacuation was necessary on hearing the alarm. Any chronic alarm that was to be disregarded needed to be clearly identified and described as an exception. To prevent a repeat of this in the future, a redundant briefing system is designed, where the contractors are briefed in advance by their management on request of both CARES and the CREATE BMO, on arrival by the BMO, and on reaching the laboratory by the CARES lab manager. The failure of any individual contractor to take the initiative in leading the evacuation is an example of the “bystander effect”. When no one individual has been designated to take the lead, the natural tendency is for everyone to wait for someone else to make
Building Management Office CARES Lab Manager
CARES Lab Manager
Building Management Office CARES Lab Manager
CARES Lab Manager
CARES Lab Manager and CARES team.
Building Management Office.
the first move. It is therefore of great importance to designate one person from a contractor group who will be the point of contact in an incident. This person may or may not have a formal position of responsibility within his organization, but this is not important, as long as his responsibility is clear to both him and to the other contractors. Further, having such a person as a designated point of contact allows the problem of language barriers to be minimized. If care is taken to select a multi-lingual person as point of contact, he can relay the lab manager’s instructions to the other contractors, ensuring that they are fulfilled. Finally, such a point of contact is important even after evacuation has been effected. In the case of a real
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
5
JCHAS-958; No of Pages 6
emergency, emergency response personnel need to know if there is anyone left behind in the laboratories. Since non-lab staff cannot be kept on site until the response is over, at least a verbal confirmation that all contractors have been evacuated is necessary. Warning signs and similar provisions of information can never substitute for clear communication between all parties; however, such signs can provide an important auxiliary to communication. Hence the placing of signs around the laboratory informing all occupants that evacuation in the case of an alarm is mandatory. These signs need to be in the four local languages, to ensure effectiveness. The incident also revealed the necessity of a detailed procedure for shutting down the laboratory and rendering all aspects safe. During the shutdown, the main concern had been the safety of
6
the equipment from any power-surge on reactivation of the building’s power. The importance of closing all cylinders had been forgotten. To prevent such a situation in the future, a detailed procedure is necessary that describes how the lab may be shut down and made safe, and then be reactivated. Had such a procedure been in place, the concern over a gas leak would have been reduced. In summary, by establishing redundant safety systems, as well as clear lines of communication and clear chains of responsibility, it is possible to guard the safety of all persons in the laboratory (whether employee or contractor) and the safe operation of the laboratory, even in the event of a highly unusual situation. An implementation of these principles is described, following the analysis of a multi-system failure.
Postscript
Since this writing, there was a second gas alarm that occurred during a similar servicing at level 7 of the RESEARCH tower. Thanks to the methods implemented here, the incident was not repeated, and evacuation and inspection proceeded swiftly and safely.
REFERENCES 1. Hendershot, D. C. Do you believe in Murphy’s Law? J. Chem. Health Saf. 2014, 21, 28. 2. Cournoyer, M. E.; Trujillo, S.; Lawton, C. M.; Land, W. M.; Schreibeer, S. B. Anatomy of an incident. J. Chem. Health Saf. 2016, 23, 40–48.
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
RESEARCH ARTICLE
Anatomy of an incident— Multiple failure of safety systems under stress Most laboratories are mostly safe most of the time. It has been said that laboratory safety does not follow Murphy’s law — what can go wrong will usually not go wrong, and so flawed and incomplete safety systems are maintained until too late.1 Unusual, high-stress situations can reveal the flaws and inadequacies at the cost of an incident. Here, I describe one such incident: how a day of electrical shut down in the laboratory, a continuous alarm, the presence of non-staff contractors, language barriers and lack of clear channels of communication lead to a failure to properly evacuate and properly investigate the source of the alarm. The development of new procedures and systems proofed against these problems is also described, as well as their implementation.
By Hugo Schmidt INTRODUCTION
Funded by the Singapore National Research Foundation, the Cambridge Centre for Advanced Research and Education in Singapore (Cambridge CARES) is a subsidiary of the University of Cambridge. It is located on the National University of Singapore’s Campus for Research Excellence and Technological Enterprise Campus (CREATE), which hosts several such co-operative initiatives. Cambridge CARES leases the top, seventh floor of CREATE’s RESEARCH Tower for the CARES chemical engineering laboratories. While CARES manages its own laboratories, the overall infrastructure of CREATE is maintained by the CREATE Building Management Office (BMO). This complex structure of responsibilities lead to the incident on the 18th of February, 2017. Gas leaks are a main concern of safety management, as asphyxiant, toxic, and explosive gases are in use throughout the two CARES laboratories. Between them, both laboratories Hugo Schmidt is affiliated with Cambridge CARES, Singapore, Singapore (E-mail: [email protected]. uk).
1871-5532 https://doi.org/10.1016/j.jchas.2017.10.005
have 111 gas sensors, and triggering a sensor leads to a loud gas alarm. To perform annual maintenance, CREATE BMO shut down the electricity on the entire CREATE RESEARCH Tower. The CREATE BMO engaged contractors to carry out this annual servicing. This had been communicated to all research groups working in CREATE, and CARES had instructed its workers to not visit its laboratories on this day. In response to the electrical shut down, the RESEARCH tower switched to the emergency backup that maintains a minimal level of energy for certain functions, including the gas alarms. At 11:00 a.m., a gas alarm was triggered. However, the contractors present at the CARES laboratories did not evacuate the premises, even after repeated urgings. Flaws in laboratory safety procedures often go unnoticed for long periods of time.1 This near-miss incident was the result of the breakdown of several systems. First, the contractors had not been properly briefed, on what to do in the event of an alarm. Second, there was a confusion because of chronic alarm caused by the electrical shut down. Third, during the alarm, language barriers made communicating with the contractors difficult. Fourth, there were problems with the procedural structures to deal with such an unusual situation.
In this paper, I describe the incident and the analysis that was applied. I discuss the multiple causal factors that lead to the near-miss incident, and the corrective measures undertaken to ensure that such a problem does not arise again. The results provide the basis for how to handle such a situation, including guidelines on a procedure with redundancies for handling contractor presence. Description of Incident
Cambridge CARES maintains two laboratories on level 7 of the CREATE RESEARCH building. To minimize gas risk, all compressed gas cylinders are stored in one of two gas manifold rooms. Each gas manifold room is secured with a four-hour rated fire door as well as gas alarms. The total air volume in each gas manifold room is changed twenty times an hour. Flammable and explosive gases are further contained in gas cylinder cabinets, each rated two hours fire resistant. Gas flow from the cylinders to the laboratories is controlled by multiple valves, including a solenoid valve that shuts automatically in the event of a gas alarm. The opening and closing of the solenoid valves is controlled by a gas control panel in the corridor outside of the laboratories. Gas alarms caused by breaches in the flammable gas co-axial lines are indicated by a pair of lights specific to each line,
ã 2017 Published by Elsevier Inc. on behalf of Division of Chemical Health and Safety of the American Chemical Society.
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
1
JCHAS-958; No of Pages 6
Figure 1. (a) Outline of laboratory 1 of CARES showing location of gas manifold rooms, (b) Gas manifold room, (c) Gas control panel, (d) beacon lights.
and other gas alarms are indicated by a code displayed on the central display (see Figure 1a–c). The gas alarm has two stages, depending on severity of gas leak: yellow, requiring a general shut down of experiments and investigation of the leak, and red, requiring an immediate evacuation. A yellow gas alarm is triggered by the presence of 5% of a given gas’ dangerous level, and the red alarm by 10%. For example, the yellow alarm would trigger in response to 5% of the Lower Explosive Limit (LEL) of an explosive gas such as ethylene, and the red alarm would trigger at 10% of the LEL. The red alarm is louder than the yellow alarm and continuous, and the difference in alarms is further indicated by beacon lights prominently placed around the lab that show exactly which stage of the alarm has been reached (see Figure 1d). The Incident occurred on the 18th February, 2017. To conduct annual maintenance, the Building Management Office (BMO) had shut down electricity throughout the RESEARCH Tower. The lab had previously been shut down, and the building was secured. Contractors were on site to conduct maintenance and testing that can only be done when the electrical systems are shut down. The lab manager was on site, but in the CREATE office that is in the CREATE Tower, rather than the RESEARCH Tower. At 11:00 a.m., the lab manager received a call from the BMO that a gas alarm had sounded 2
at level 7. This was the only way the lab manager could have received this information, as the usual Building Management System Alarm Notification had been deactivated, along with the rest of the equipment at level 7. Arriving at Level 7, the contractors were unsure what the alarm meant. Despite repeated urgings, the contractors refused to evacuate. The contractors maintained that the BMO had informed them that the alarm was only a side-effect of the power shut-down. The lab manager called the BMO, and determined that the BMO only referred to the standard ‘power-outage alarm’, not the gas alarm. The lab manager returned and tried again to persuade the contractors to evacuate. Despite some initial resistance, after some discussion, the contractors agreed to evacuate, leaving behind the lab manager and the representatives of the BMO trying to resolve the situation. No census was taken, and it was unclear how many contractors were on site. To make sure that no casualties were present in the laboratory, the lab manager donned the Self Contained Breathing Apparatus (SCBA) and investigated both laboratories to see if any unconscious casualties were present. No casualties were present. The compressed gas cylinders had already been sealed by the automatic closing of the solenoid valves. To ensure further safety, the lab manager sealed all gas supplies at the cylinders
themselves, before returning to the gas panel outside of the laboratory. The gas panel was opened, but it proved impossible to reset the alarm, as the password was not available. It was also not possible to determine which gas sensor had been triggered, as it was indicated by a code on the LED display. The premises were evacuated, and the security staff contacted to ensure that no one would enter the building until it had been deemed safe. The gas safety contractor was called and sent to CARES. The BMO staff left, and the lab manager awaited the contractor’s arrival. On arriving, the gas safety contractor and lab manager proceeded to the laboratories to investigate the source of the alarm. After investigation, the gas safety contractor concluded that there was no significant risk and that the alarm was likely the result of sensor malfunction. The alarm was deactivated, and the sensor investigated, discovering a faulty part that had triggered the alarm. The contractor safely removed the part. In summary, the unusual situation stressed sequential safeguards beyond breaking point. Had the alarm sounded under normal working hours, evacuation would have been the normal response. Similarly, had the electrical system not been shut down, there would not have been a chronic alarm condition that confused the issue of the gas alarm. Causal Factors
To determine the causes that lead to the failure of evacuation and the difficulty in responding, I carried out a cause-and-effect analysis2 (Figure 2). Management Factors
The contractors were misled by the briefing provided by the BMO, leading them to conclude that the gas alarm was simply the result of electrical power-down. The CARES Lab Manager was present in the office rather than the laboratory. Neither Cambridge CARES nor the BMO properly instructed the contractors as to the importance of evacuation.
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
JCHAS-958; No of Pages 6
Figure 2. Failure to evacuate cause and effect diagram.
Neither Cambridge CARES nor the CREATE BMO required contractor management to designate a representative who would take instruction from CARES’ lab manager. BMO and CARES representatives were unsure as to who was present on the Level 7. The Emergency Response Plan (ERP) was unclear on whether to contact the Site Controller in this instance. There was no clearly defined laboratory shut-down procedure that covered both the unplugging of all equipment and the shutdown of all gases, with a designated verification that the shutdown had been completed.
There was no way to use the gas panel due to an absence of the password. Furthermore, it was difficult to determine which gas sensor had triggered the alarm.
A language barrier existed between the Lab Manager and many contractors (Table 1).
Equipment Factors
DISCUSSION
During the alarm, the lifts were still operational and there was no way to indicate – other than the alarm – that the level was off limits. The gas control panel lacked a clear placing of the number for the gas safety contractor. The gas control panel lacked a clear description of which alarm code corresponded to which sensor. There were no signs indicating that an alarm mandated evacuation. Manpower Factors
Engineering Factors
The volume of gases, and their potential spread was not known. Nor was their likely persistence under different conditions known. The SCBA was located in the Mechanical and Engineering Room, which was locked. In the absence of the lab manager, or another designated responsible person, it would have been inaccessible. The loud alarm made communication with the BMO staff difficult
The contractor staff did not listen to the Lab Manager’s instructions. No leader of the contractors was designated. There was no clear understanding of how many people were present on level 7. Chain of responsibility and authority was unclear with the use of contractors, especially contractors not commissioned by CARES. There was no communication between CARES and contractors following evacuation.
Extreme cases may make bad law, but they do make very good case studies. Systemic failures that occur when a system (such as the lab safety and health procedures) is stressed due to unusual circumstances reveal failure points in the system that are not apparent in usual operations. As shown in the case study, the incident arose through the sequential failure of multiple systems. The risk of the lab manager searching the lab for casualties unsupported was caused first by the failure to properly secure all gas cylinders in advance of the lab shutdown, which meant that the possibility of a gas leak was serious; second, by the lack of a list of contractors and a lack of verbal confirmation that the evacuation was complete, which meant that the absence of casualties could not be assumed; third, by the lack of provision for back-up to the lab manager in the event of a necessary SCBA use. Any of these would have decreased the risk.
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
3
JCHAS-958; No of Pages 6
Table 1. Failure Points and Corrective Actions.
Failure Point
Corrective Action Proposed
Responsibility
Failure to brief all contractors and contractor management in advance.
CARES shall develop a contractor procedure that stipulates that all contractor management must be informed in advance that evacuation on hearing an alarm is mandatory, as is evacuation in response to direction by the lab manager or other CARES staff. Signs shall be posted instructing all staff and contractors that evacuation in the event of a red gas alarm/second stage fire alarm is mandatory. Signs shall be prepared in English, Malay, Tamil and Standard Mandarin. CARES shall request contractor management to provide CARES with a list of the staff sent to work at CARES premises, designating one worker as an official representative who shall form point-of-contact. The representative shall be required to speak conversational English. Further, the CARES Lab Manager or other designated responsible person shall brief all contractors on their arrival, taking a roll call to make sure that the list provided by the contractor management is accurate. These requirements will be communicated to all contractors that will work in CARES or at the 7th level of the CREATE RESEARCH Tower, regardless of whether they have been contracted by CARES or by the CREATE BMO. These requirements will be communicated by both CARES and the CREATE BMO.
CARES Lab Manager.
Contractors would not listen to instruction by lab manager. Chain of responsibility and authority was unclear.
No communication between contractors and CARES following evacuation.
Language barrier between contractors and CARES Lab Manager. Failure to distinguish between a chronic alarm triggered by power outage, and an actual alarm requiring evacuation. Difficult to know whom to contact in this situation.
Communication difficulties due to alarm. CARES Lab Manager present in office rather than lab. BMO unsure who was present at level 7.
Self Contained Breathing Apparatus in locked mechanical room.
4
CARES Lab Manager.
CARES Lab Manager. Building Management Office
The BMO shall inform CARES and contractors if there is a chronic alarm, and which alarm is meant. The BMO shall also say that all other alarms should be treated as real, with full evacuation. CARES will repeat these instructions to all contractors when they arrive at the laboratories.
Building Management Office CARES Lab Manager
All induction materials, signs etc. shall be updated to include the name and contact details of the gas safety contractor directly responsible for the gas alarm. The Emergency Response Plan shall be similarly updated to state that the gas safety contractor is to be contacted in the event of a gas alarm. Contact person and contact number shall be posted prominently next to the gas panel. Given the loudness of the alarm, evacuation shall proceed first. All supplementary questions to be posed to BMO, CARES management etc. only after the building is clear. This shall be made clear to all contractors. CARES lab manager shall be present in the laboratory for all servicing that must be conducted.
CARES Lab Manager
The BMO shall be requested to have a full list of all contractors who are at level 7, and present this list to CARES, bearing in mind this may include contactors unconnected to CARES who are working on the roof infrastructure. All contractors will be required to sign in and out on entering the CARES labs. One of the two SCBA shall be placed in the CARES office to be easily accessible if the M&E room is inaccessible.
CARES Lab Manager Building Management Office CARES Lab Manager.
Building Management Office
CARES Lab Manager CARES Lab Manager
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
JCHAS-958; No of Pages 6
Table 1 (Continued )
Failure Point
Corrective Action Proposed
Responsibility
Lab Manager investigated site without backup.
A backup shall be in place in any situation where the use of the SCBA is necessary. If CARES staff are not present, a representative of the Building Management Office should be present. Lab air conditioning systems shall be updated to include a purge capacity. The purge capacity should increase the removal of air to maximum, dispensing dangerous gas. The password for the gas panel shall be placed within the gas panel, next to the manual. The gas panel can be opened only with the key that either the CARES lab manager or the gas contractor has. In the event of lab manger absence, the keys are passed to a designated substitute lab manager. Spare copies of the key are lodged with the program manager. CARES shall investigate the possibility of barriers or traffic cones to indicate that the level is off-limits, along with signage indicating the same. The feasibility of more substantial barriers shall be investigated. Lifts should be grounded in the event of an alarm.
CARES Lab Manager
Problem with dissipating gas. Password for gas panel unknown.
No way to secure laboratory.
The Emergency Response Plan was unclear as to whether to contact Site Controller in event of a non-CARES evacuation. Lack of clear lab shutdown procedure. Unsure of potential gas spread.
Power cut created unusual conditions.
CARES Lab Manager shall meet with CARES Site Controller to establish the correct course of action in this event.
CARES Lab Manager shall develop a comprehensive procedure for the shut-down of the laboratory and its reactivation. CARES shall investigate the volume of gases that can be potentially released, as well as the time that it would take to clear, both under normal operations and in event of powershut down. BMO shall thoroughly debrief all contractors and CARES on how the infrastructure will behave in this condition.
The failure of the contractors to evacuate, was caused first by the absence of adequate briefing prior to beginning work; second, by a failure to distinguish the chronic alarm from the real alarm; third, by a lack of a designated contact person and a clear chain of responsibility which made the language barrier an obstacle to the evacuation. If any of these elements had been corrected for, evacuation would have occurred as normal. This is a problem that can affect all laboratories, but laboratories in multi-lingual societies (such as Singapore) which are the result of inter-institutional cooperation (such as CARES) are particularly vulnerable to it. The most fundamental problem was the failure of the contractors to be
briefed that evacuation was necessary on hearing the alarm. Any chronic alarm that was to be disregarded needed to be clearly identified and described as an exception. To prevent a repeat of this in the future, a redundant briefing system is designed, where the contractors are briefed in advance by their management on request of both CARES and the CREATE BMO, on arrival by the BMO, and on reaching the laboratory by the CARES lab manager. The failure of any individual contractor to take the initiative in leading the evacuation is an example of the “bystander effect”. When no one individual has been designated to take the lead, the natural tendency is for everyone to wait for someone else to make
Building Management Office CARES Lab Manager
CARES Lab Manager
Building Management Office CARES Lab Manager
CARES Lab Manager
CARES Lab Manager and CARES team.
Building Management Office.
the first move. It is therefore of great importance to designate one person from a contractor group who will be the point of contact in an incident. This person may or may not have a formal position of responsibility within his organization, but this is not important, as long as his responsibility is clear to both him and to the other contractors. Further, having such a person as a designated point of contact allows the problem of language barriers to be minimized. If care is taken to select a multi-lingual person as point of contact, he can relay the lab manager’s instructions to the other contractors, ensuring that they are fulfilled. Finally, such a point of contact is important even after evacuation has been effected. In the case of a real
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005
5
JCHAS-958; No of Pages 6
emergency, emergency response personnel need to know if there is anyone left behind in the laboratories. Since non-lab staff cannot be kept on site until the response is over, at least a verbal confirmation that all contractors have been evacuated is necessary. Warning signs and similar provisions of information can never substitute for clear communication between all parties; however, such signs can provide an important auxiliary to communication. Hence the placing of signs around the laboratory informing all occupants that evacuation in the case of an alarm is mandatory. These signs need to be in the four local languages, to ensure effectiveness. The incident also revealed the necessity of a detailed procedure for shutting down the laboratory and rendering all aspects safe. During the shutdown, the main concern had been the safety of
6
the equipment from any power-surge on reactivation of the building’s power. The importance of closing all cylinders had been forgotten. To prevent such a situation in the future, a detailed procedure is necessary that describes how the lab may be shut down and made safe, and then be reactivated. Had such a procedure been in place, the concern over a gas leak would have been reduced. In summary, by establishing redundant safety systems, as well as clear lines of communication and clear chains of responsibility, it is possible to guard the safety of all persons in the laboratory (whether employee or contractor) and the safe operation of the laboratory, even in the event of a highly unusual situation. An implementation of these principles is described, following the analysis of a multi-system failure.
Postscript
Since this writing, there was a second gas alarm that occurred during a similar servicing at level 7 of the RESEARCH tower. Thanks to the methods implemented here, the incident was not repeated, and evacuation and inspection proceeded swiftly and safely.
REFERENCES 1. Hendershot, D. C. Do you believe in Murphy’s Law? J. Chem. Health Saf. 2014, 21, 28. 2. Cournoyer, M. E.; Trujillo, S.; Lawton, C. M.; Land, W. M.; Schreibeer, S. B. Anatomy of an incident. J. Chem. Health Saf. 2016, 23, 40–48.
Journal of Chemical Health & Safety, May/June 2017
Please cite this article in press as: Schmidt, H2., Anatomy of an incident— Multiple failure of safety systems under stress. J. Chem. Health Safety (2017), https://doi.org/10.1016/j.jchas.2017.10.005