- Email: [email protected]
Three-dimensional incident management
INCIDENT MANAGEMENT
Three-dimensional incident management Wendy Goucher, Security empowerment consultant, Idrach Ltd. I grew up in a household where incident management was the norm. My dad was a construction site agent, and about the first thing that had to be arranged wherever we lived was a phone. It was in the hall, so that it could be heard all over the house and in the garden. Long phone calls when dad was not at work were frowned on, especially at critical times such as bonfire night when the incidence of, well incidents, was fairly high. So I was used to the phone calls that prompted his swift exit out of the door, along with big sigh that mum would give as she put his meal back onto of a pan of boiling water in the hope it would stay warm, without drying out. That was why he developed his love for gravy; it rehydrated meals that had sat like that for several hours while he was called out. I never thought that, many years later; a series of similar situations would lead to a new area of interest and a change in career.
When work calls Having had about 12 years of being a Navy officer’s wife and dealing with the demands on time and loyalty it entailed, I found the transition to being the wife of a civilian information security professional, especially when he went to work for a large financial corporation, less of a welcome relief than I expected. True, there was not the long term separation, but that was almost easier than the times when he was home, but on call into the night. One way I dealt with this ‘mistress’ in our lives which took so much of his attention, was to discuss his work with him. It was because of that that I became interested in the profession. The fact that many incidents had a human element which was either cause or effect (and sometimes both) intrigued me. Ultimately 16
Computer Fraud & Security
it intrigued me more than my current career, and a change had to happen.
“I heard of one person whose young child got so fed up of their parent’s phone ringing at important moments they dropped it in the bath.” What these experiences did was give me a huge insight into the three-dimensional nature of on-call incident management. On one level, I find the demands easier now. As I a child we were ‘chained’ to the house when there was a chance dad might be required. When my husband headed the incident response team we could go out because he had a pager, a phone, and a BlackBerry, so he was rarely out of touch. On the other hand the sight of him on the edge of a family event trying to find a quiet corner to have an incident meeting on the phone became commonplace, and it was wearing after a while. Earlier, I referred to the incident job as a ‘mistress’ and at times it felt like that because it was an entity in our lives which demanded instant and undivided attention. It could be argued that it was well paid, and that would be true. That did not remove the feelings for the supporting families though. I heard of one person whose young child got so fed up of their parent’s phone ringing at important moments they dropped it in the bath. To see increased payments for out of hours incident cover as a fair and total exchange is missing the problems that these demands can have on
Wendy Goucher
the family and on the individual concerned. If the manager in charge of the team fails to consider these stresses and demands then they could find staff are unable to stay in the team for an extended time.
Why incident response should matter to you So, why should that worry you? Does your organisation even have an incident response team? Maybe it does, but unofficially only at this point. Well the need is becoming more prevalent. Gene Schultz, who was the founder of the U.S. Department of Energy Incident Advisory Capability Team said: “Incident response is now a necessary component of a successful computer and network security life cycle that includes countermeasures, detection, and response.” It should also be remembered that incident response is not universally unattractive. Especially in situations where the incident is likely to be complex and time consuming, a successful team member is likely to ‘enjoy’ the adrenalin of the pressured situation with the need to unravel both the problems and the solution. With a person such as this their family is unlikely to be able to provide that ‘thrill’. Terry Gudaitis, in his chapter concerning the Human side of Incident Response, says that incident responders like the initial excitement of a case. “Firefighters and police officers know that their family lives and personal lives can suffer dramatically due to their intense work schedule,” he admitted. One of my dad’s bosses believed that it was essential to try and get the wives of his site managers together for a social event at least once a year. He found that, February 2010
INCIDENT MANAGEMENT in meeting these ladies, he could begin to get a feeling for their tolerance and support for the job and its demands. It also meant that he was not just a faceless man whose fault it was that we had to spend another sunny Saturday at home in case the pouring of the concrete flooring went wrong. He was ‘Peter’ and he had shown respect for the feelings and positions of the wives by entertaining them. I realise that people’s family situations have a tendency to be more complicated nowadays, but contact and respect is still important. Being the friends and family of someone on incident management duty might seem less important, but that rather depends on what the business is. It could be a chemical plant, the network supporting a major utility, a stock market or a health service network. The fallout from the failure of any of these would be significant, and possibly even fatal, so maybe we need to think seriously about the whole picture.
“He was not just a faceless man whose fault it was that we had to spend another sunny Saturday at home in case the pouring of the concrete flooring went wrong”
What do the background support team do? Physical support: When the call comes at three in the morning, it might well be a simple matter of making the hot caffeine-rich drink to help to kickstart the duty staff member. I have also found that acting as a casual personal assistant in the finding of pen, paper, and even getting the spare mobile phone battery charging, can all help. Gudaitis talks about the fact that responders often become so involved in the incident that they neglect to eat or even drink properly, and therefore dehydration as well as the effects of lack of proper nutrition and sleep can cause significant problems and reduce the effectiveness, and recovery, of the team. Who can help with this February 2010
will depend on where the responder is operating, either at home or work, but proper preparation should be addressed. Psychological support: When this was my problem, I found it meant that I tried not to moan until the incident was complete and fielded the children and other visitors and interruptions whenever possible. This might seem trivial, but removing non-incident stress can be a real help to the situation and make it easier for the responder to unwind after the event is over.
What can the organisational team do? First and foremost the organisation needs to respect and support the responder. In his situation he is largely referring to the sort of incident that would mean that the responder has to be on site to handle the situation. That entails food, quiet, ‘timeout’ rooms where staff can take themselves away from the fraught atmosphere and possibly rest for a short time. After the event, especially if it was particularly difficult or time consuming, recognising that the responder needs to recover might lead to a little flexibility about in-office working hours for a couple of days. This could reduce longer term stress issues as the staff member gets a chance to recover from the adrenalin high of the incident. The extent to which this is necessary will depend very much on the situation. Some incidents can include long hours of boredom as systems are rebooted or software reinstalled. Clearly this would require less recovery. It will be clear from the beginning of this piece that the majority of my experience in incident response has been from the position of backstage support. I feel that, in many cases, organisations ignore or fail to respect that part of their staff ’s lives to the detriment of the effectiveness of the organisation as a whole, and certainly of their IR teams. I am not suggesting that the incident manager needs to know and understand all their staff and in turn their home lives and pressures. However, they should show
some respect for them. One petrochemical company who employed a colleague’s husband a few years ago insisted that, in addition to out of hours payment, the family or the staff member got ‘experience’ vouchers. These were expected to be used with the family, so that the whole family that had been inconvenienced by the work got the benefit. Certainly my colleague found that the children were much more tolerant of their dad missing a football match when it meant they did a bit of white water rafting in the summer. This was a non-intrusive way of respecting the team and possibly attracting, and retaining the best members of your incident response team. At the end of the day, and to be totally business-focused about it, looking after the three-dimensional incident response team means that you are operating in a more cost-effective and efficient way. If the IR team has an average retention of two years, and the team is eight people, then you looking at being in a state of near constant ‘training up’. Bruce Tuckman talks about all teams needing to go through the stages of ‘Forming, Storming and Norming’ before they are ‘Performing’ at their optimum. If staff are changing every two to three months, then this time of peak performance is common and the team is likely to rely heavily on the experience and knowledge of the core staff. This, in turn can mean that they are under greater pressure as they are consulted even when they are not ‘on call’. So, get your policies and procedures well designed to suit both your organizational operational needs and the responders. Matthew Pemble, who was incident response manager for RBS, said that IR policies need to be useable by the tired, and possibly emotional, in the middle of the night after a 18 hour working day. Train and rehearse and analyse and improve and learn from mistakes, both yours and any others you can hear about. However, if you want to have a world-class incident response team then design your management process to cover all three dimensions.
Computer Fraud & Security
17
Three-dimensional incident management Wendy Goucher, Security empowerment consultant, Idrach Ltd. I grew up in a household where incident management was the norm. My dad was a construction site agent, and about the first thing that had to be arranged wherever we lived was a phone. It was in the hall, so that it could be heard all over the house and in the garden. Long phone calls when dad was not at work were frowned on, especially at critical times such as bonfire night when the incidence of, well incidents, was fairly high. So I was used to the phone calls that prompted his swift exit out of the door, along with big sigh that mum would give as she put his meal back onto of a pan of boiling water in the hope it would stay warm, without drying out. That was why he developed his love for gravy; it rehydrated meals that had sat like that for several hours while he was called out. I never thought that, many years later; a series of similar situations would lead to a new area of interest and a change in career.
When work calls Having had about 12 years of being a Navy officer’s wife and dealing with the demands on time and loyalty it entailed, I found the transition to being the wife of a civilian information security professional, especially when he went to work for a large financial corporation, less of a welcome relief than I expected. True, there was not the long term separation, but that was almost easier than the times when he was home, but on call into the night. One way I dealt with this ‘mistress’ in our lives which took so much of his attention, was to discuss his work with him. It was because of that that I became interested in the profession. The fact that many incidents had a human element which was either cause or effect (and sometimes both) intrigued me. Ultimately 16
Computer Fraud & Security
it intrigued me more than my current career, and a change had to happen.
“I heard of one person whose young child got so fed up of their parent’s phone ringing at important moments they dropped it in the bath.” What these experiences did was give me a huge insight into the three-dimensional nature of on-call incident management. On one level, I find the demands easier now. As I a child we were ‘chained’ to the house when there was a chance dad might be required. When my husband headed the incident response team we could go out because he had a pager, a phone, and a BlackBerry, so he was rarely out of touch. On the other hand the sight of him on the edge of a family event trying to find a quiet corner to have an incident meeting on the phone became commonplace, and it was wearing after a while. Earlier, I referred to the incident job as a ‘mistress’ and at times it felt like that because it was an entity in our lives which demanded instant and undivided attention. It could be argued that it was well paid, and that would be true. That did not remove the feelings for the supporting families though. I heard of one person whose young child got so fed up of their parent’s phone ringing at important moments they dropped it in the bath. To see increased payments for out of hours incident cover as a fair and total exchange is missing the problems that these demands can have on
Wendy Goucher
the family and on the individual concerned. If the manager in charge of the team fails to consider these stresses and demands then they could find staff are unable to stay in the team for an extended time.
Why incident response should matter to you So, why should that worry you? Does your organisation even have an incident response team? Maybe it does, but unofficially only at this point. Well the need is becoming more prevalent. Gene Schultz, who was the founder of the U.S. Department of Energy Incident Advisory Capability Team said: “Incident response is now a necessary component of a successful computer and network security life cycle that includes countermeasures, detection, and response.” It should also be remembered that incident response is not universally unattractive. Especially in situations where the incident is likely to be complex and time consuming, a successful team member is likely to ‘enjoy’ the adrenalin of the pressured situation with the need to unravel both the problems and the solution. With a person such as this their family is unlikely to be able to provide that ‘thrill’. Terry Gudaitis, in his chapter concerning the Human side of Incident Response, says that incident responders like the initial excitement of a case. “Firefighters and police officers know that their family lives and personal lives can suffer dramatically due to their intense work schedule,” he admitted. One of my dad’s bosses believed that it was essential to try and get the wives of his site managers together for a social event at least once a year. He found that, February 2010
INCIDENT MANAGEMENT in meeting these ladies, he could begin to get a feeling for their tolerance and support for the job and its demands. It also meant that he was not just a faceless man whose fault it was that we had to spend another sunny Saturday at home in case the pouring of the concrete flooring went wrong. He was ‘Peter’ and he had shown respect for the feelings and positions of the wives by entertaining them. I realise that people’s family situations have a tendency to be more complicated nowadays, but contact and respect is still important. Being the friends and family of someone on incident management duty might seem less important, but that rather depends on what the business is. It could be a chemical plant, the network supporting a major utility, a stock market or a health service network. The fallout from the failure of any of these would be significant, and possibly even fatal, so maybe we need to think seriously about the whole picture.
“He was not just a faceless man whose fault it was that we had to spend another sunny Saturday at home in case the pouring of the concrete flooring went wrong”
What do the background support team do? Physical support: When the call comes at three in the morning, it might well be a simple matter of making the hot caffeine-rich drink to help to kickstart the duty staff member. I have also found that acting as a casual personal assistant in the finding of pen, paper, and even getting the spare mobile phone battery charging, can all help. Gudaitis talks about the fact that responders often become so involved in the incident that they neglect to eat or even drink properly, and therefore dehydration as well as the effects of lack of proper nutrition and sleep can cause significant problems and reduce the effectiveness, and recovery, of the team. Who can help with this February 2010
will depend on where the responder is operating, either at home or work, but proper preparation should be addressed. Psychological support: When this was my problem, I found it meant that I tried not to moan until the incident was complete and fielded the children and other visitors and interruptions whenever possible. This might seem trivial, but removing non-incident stress can be a real help to the situation and make it easier for the responder to unwind after the event is over.
What can the organisational team do? First and foremost the organisation needs to respect and support the responder. In his situation he is largely referring to the sort of incident that would mean that the responder has to be on site to handle the situation. That entails food, quiet, ‘timeout’ rooms where staff can take themselves away from the fraught atmosphere and possibly rest for a short time. After the event, especially if it was particularly difficult or time consuming, recognising that the responder needs to recover might lead to a little flexibility about in-office working hours for a couple of days. This could reduce longer term stress issues as the staff member gets a chance to recover from the adrenalin high of the incident. The extent to which this is necessary will depend very much on the situation. Some incidents can include long hours of boredom as systems are rebooted or software reinstalled. Clearly this would require less recovery. It will be clear from the beginning of this piece that the majority of my experience in incident response has been from the position of backstage support. I feel that, in many cases, organisations ignore or fail to respect that part of their staff ’s lives to the detriment of the effectiveness of the organisation as a whole, and certainly of their IR teams. I am not suggesting that the incident manager needs to know and understand all their staff and in turn their home lives and pressures. However, they should show
some respect for them. One petrochemical company who employed a colleague’s husband a few years ago insisted that, in addition to out of hours payment, the family or the staff member got ‘experience’ vouchers. These were expected to be used with the family, so that the whole family that had been inconvenienced by the work got the benefit. Certainly my colleague found that the children were much more tolerant of their dad missing a football match when it meant they did a bit of white water rafting in the summer. This was a non-intrusive way of respecting the team and possibly attracting, and retaining the best members of your incident response team. At the end of the day, and to be totally business-focused about it, looking after the three-dimensional incident response team means that you are operating in a more cost-effective and efficient way. If the IR team has an average retention of two years, and the team is eight people, then you looking at being in a state of near constant ‘training up’. Bruce Tuckman talks about all teams needing to go through the stages of ‘Forming, Storming and Norming’ before they are ‘Performing’ at their optimum. If staff are changing every two to three months, then this time of peak performance is common and the team is likely to rely heavily on the experience and knowledge of the core staff. This, in turn can mean that they are under greater pressure as they are consulted even when they are not ‘on call’. So, get your policies and procedures well designed to suit both your organizational operational needs and the responders. Matthew Pemble, who was incident response manager for RBS, said that IR policies need to be useable by the tired, and possibly emotional, in the middle of the night after a 18 hour working day. Train and rehearse and analyse and improve and learn from mistakes, both yours and any others you can hear about. However, if you want to have a world-class incident response team then design your management process to cover all three dimensions.
Computer Fraud & Security
17