- Email: [email protected]
Analysing and modelling train driver performance
ARTICLE IN PRESS
Applied Ergonomics 36 (2005) 671–680 www.elsevier.com/locate/apergo
Analysing and modelling train driver performance Ronald W. McLeoda,, Guy H. Walkerb, Neville Moraya a
Nickleby HFE Ltd., 16, Waterside Avenue, Glasgow, G77 6TJ, UK Brunel University, School of Engineering & Design, Uxbridge UB8 3PH, UK
b
Received 25 November 2004; accepted 25 May 2005
Abstract Arguments for the importance of contextual factors in understanding human performance have been made extremely persuasively in the context of the process control industries. This paper puts these arguments into the context of the train driving task, drawing on an extensive analysis of driver performance with the Automatic Warning System (AWS). The paper summarises a number of constructs from applied psychological research thought to be important in understanding train driver performance. A ‘situational model’ is offered as a framework for investigating driver performance. The model emphasises the importance of understanding the state of driver cognition at a specific time (‘Now’) in a specific situation and a specific context. r 2005 Elsevier Ltd. All rights reserved. Keywords: Train driver performance; Automatic warning system; Models of human performance; Human reliability; Cognitive engineering
1. Introduction The work reported in this paper was carried out as part of a study for the Rail Safety and Standards Board (RSSB). The aim was to understand and assess the risks of driver unreliability associated with extended uses of the Automatic Warning System (AWS) on the UK rail network. 1.1. The Automatic Warning System (AWS) The purpose of railway signalling is to ensure safety and regulate traffic. Safety is ensured by maintaining sufficient headway separation between trains, given the relatively long distances required to slow them down. In practice this is achieved by displaying lights (combined with other signals) at the side of the track to communicate to the driver the need to slow down or stop. In the UK, the AWS also provides an audible alert of the state of the upcoming signal supported by an inCorresponding author. Tel.: +44 141 616 3104.
E-mail address: [email protected] (R.W. McLeod). 0003-6870/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.apergo.2005.05.006
cab visual reminder (known as the ‘sunflower’) once the warning has been acknowledged by the driver. If the driver fails to acknowledge the audible warning, a ‘full service’ brake application is made automatically. As the driver can override the automatic brake application, maintenance of safety still depends on the reliability of the driver. The issue for human factors is that, despite what appears to be an extremely powerful and compelling attention-getting device, combined with a highly visible visual reminder, there has been a significant and recurring incidence of AWS failing to prevent experienced train drivers passing signals at danger (SPADs). According to Vaughan (2000) in describing the original AWS system adopted by British Rail in 1936, ‘No-one ever thought that the driver would ignore the warning. It was utterly taken for granted that a driver would always take notice of the signal’ (p.12). In connection with the crash at Watford Junction in 1996, Hall (1999) wrote that ‘It seems inconceivable that a driver can acknowledge receiving two warnings and yet take no action to apply the brake to stop the train at the red signal, yet it happens’. (p.97).
ARTICLE IN PRESS 672
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
AWS was originally intended to support ‘semaphore’ signalling communicating two possible instructions to the driver: (a) stop at this signal, (b) proceed to the next signal. In the UK, semaphore signalling has nowadays been largely superceded by modern coloured light signalling. This system uses coloured lights to indicate up to four different states: green (meaning it is safe to proceed to the next signal), double yellow (expect to stop at the signal after the next one), single yellow (expect to stop at the next signal), red (stop at this signal). Coloured light signalling supports increased traffic density by providing earlier warning of the state of upcoming signals (and therefore more efficient headway separation) for drivers of high-speed trains. In the modern AWS system however, the audible warning still only discriminates between the original two signal states: ‘clear’ (indicated by a bell or simulated chime at around 1200 Hz) sounds if the driver is approaching a proceed, or green signal aspect; ‘warning’ (indicated by a steady alarm or horn sound at around 800 Hz) sounds if the signal is showing any other aspect apart from green. In addition to this use of AWS to support signalling, use of the system has in recent years been extended to draw the driver’s attention to a number of non-signal related events such as emergency, temporary or permanent speed restrictions, and certain level crossings. These are what is known as ‘Extended’ uses of AWS. In summary, while it appears relatively straightforward conceptually, the ways in which ‘standard’ (as opposed to Extended) AWS is used in the UK introduces a number of cognitive complexities for the driver. These include:
AWS does not fully discriminate between the possible states giving rise to the audible alert. The system depends on the driver correctly interpreting which of more than one possible condition the audible alert refers to, and therefore what behaviour is appropriate. There are situations where the in-cab visual reminder can refer not to the immediate past signal, but to a signal some time prior to that. The time period over which any AWS indication is ‘active’ in conveying information about the track ahead can vary from a few seconds to possibly many minutes.
1.2. Existing human factors approaches to understanding train driver cognition Train driver performance is dominated by cognitive and perceptual factors. Interaction with AWS is only one element in what can be a demanding perceptual and cognitive workload. While the context of the study was based on understanding possible driver unreliability
with AWS, the work reported here is thought to be relevant to understanding many aspects of cognitive performance in train driving. Much of the existing human factors research on train driver performance is concerned with SPADs and draws on hierarchically based forms of task analysis, based on relatively simple models of human decision-making and information processing (see McLeod et al., 2003a, for review). From the perspective of cognitive psychology and cognitive engineering, however, information-processing based approaches are thought to be fundamentally inappropriate for understanding real-time cognition, even for tasks that, superficially at least, appear as relatively well-defined and constrained as train driving. Specifically, information-processing based approaches are thought not to be able to capture the contextual and situated nature of human performance. Vicente (1999) offers a compelling critique of traditional approaches to task analysis. Similarly, Hollnagel (1998) provides a detailed critique of human reliability estimation techniques from the perspective of cognitive engineering. Although there are doubts about whether the techniques they propose (Cognitive Work Analysis (CWA) in the case of Vicente and the Cognitive Reliability and Error Analysis Method (CREAM) in the case of Hollnagel) overcome these limitations for the train driver context, the underlying arguments are very powerful. Little if any of the existing human factors research into train driving has tried to understand the mechanisms by which contextual and situational factors influence driver performance (other than in the very limited sense of seeking to identify factors likely to influence or ‘shape’ driver performance). The relative simplicity of the psychological basis of the existing research, and the continuing risk of SPAD incidents, suggests that a more comprehensive, psychologically richer understanding of the train driving task is needed.
2. Recent thinking in the psychology of real-world human performance There is a large and well-respected body of thinking and research that focuses on understanding human cognition and performance in the real-world contexts in which it occurs. Recognising the psychological complexity of the AWS problem, a number of ideas from this wider knowledge base were reviewed. These go beyond the immediate literature on human factors of AWS. The aim was to draw on thinking about cognitive performance in real-world tasks that seemed relevant to understanding the psychological context of train drivers’ use of AWS. The following sections very briefly summarise some of these areas. A more detailed review,
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
with full reference list, is included in McLeod et al. (2003a). Strategic behaviour: The importance of strategy and strategic behaviour in human performance has been recognised, at least in the research literature, for many years. In particular, strategy is essential in helping the human maintain performance in situations of stress, very high or very low workload or when subject to other influences (usually referred to as ‘Performance Shaping Factors’, PSFs). The concept of strategy has rarely however been used explicitly to try to understand or explain human behaviour and performance in realworld situations. An exception is recent research by Merat et al. (2002) into train driver eye movements that appears to provide some insight into strategic behaviour in train drivers. Situation awareness: Situation awareness is possibly the most widely cited construct in understanding human performance in complex real-time systems. Not surprisingly, it is also considered a key psychological construct in safe train driving. Loss of situation awareness leads to what Sarter and Woods (1997) have described as automation ‘surprises’. In connection with the Purley rail crash in 1989, Vaughan (2000) describes ‘Driver Morgan of the Littlehampton train [..] completely at a loss to understand what had happened’ (p. 91). Vaughan notes that driver Morgan must have acknowledged and cancelled the two previous AWS warnings leading up to the red signal, but failed to initiate an appropriate braking maneuver until it was too late. A wide range of factors can potentially influence the train drivers’ understanding or belief about the current state of the world and therefore how an AWS alarm is interpreted. These include: the nature of the alarm (bell or horn); visibility of signals and magnets on the track ahead; the driver’s interpretation of the nature of the preceding alarm; expectations about the current location; the train speed to be achieved, and by what time or by what location it is to be achieved. Changes in any of these factors can potentially lead to a change in the drivers’ interpretation of a particular AWS alarm. In combination, and when put into the wider context including factors such as the driver’s level of fatigue, emotional state, values and belief system, they have the potential to cause the driver to misinterpret what an AWS alarm refers to, and what action to take. Situated behaviour: The term ‘situated behaviour’ seeks to take understanding of human performance out of the laboratory context, and to ground it in the realworld environment in which it occurs. The common themes are that the context and particularly the situation at the time have an extremely powerful influence on how the human performs in real-world tasks.
673
The work of Rasmussen, Pedersen and Goodstein (1995), Vicente (1999) and many others on the analysis of cognitive work1 takes the view that it is not possible to understand human performance without locating it within organisational and technological constraints. They demonstrate why traditional normative analysis approaches are not well suited to understanding complex socio-technical systems because they underestimate (if they capture them at all) the contextdependent aspects of human performance. Distributed cognition: Ecological-based research such as much of the varied research into distributed cognition (e.g. Hutchins, 1995) emphasises the critical role that the situation and environment play in cognitive performance. The distributed cognition concept recognises that cognitive performance draws on, and is supported by many artefacts of the organisation and surrounding environment. It is, for example, clear that the placement of a prominent AWS magnet in the middle of the track can help to cue the driver in advance of a warning sounding. The driver is also likely to take advantage of many other environmental cues. These might include TPWS antennas, hot axle box detectors, other incidental AWS sounds (such as a relay’s clicking) and a myriad of other track/trackside artefacts. Cognitive control modes and subjectively available time: Hollnagel (1998) has developed a model of cognitive performance (COCOM) that recognises and seeks to account for the critical role of context in cognition. His ideas of cognitive control and subjectively available time seem potentially important in understanding train driver behaviour. The knowledge and experience of a route that drivers develop over time (what is known as ‘Route knowledge’) supports anticipation and future-orientated behaviour. Route knowledge allows the driver to think ahead, and helps control the allocation of cognitive and perceptual resources based on expectations about the future. It also helps the driver in spotting and interpreting cues and other information. In the COCOM framework, the effect of route knowledge might be to increase subjectively available time, thereby allowing the driver greater cognitive control over performance. Ecological optics: Following the lead of Gibson (1979), there is a very large body of knowledge about the ways in which humans, as well as many other species, make use of information directly available from movement of the eyes of an observer in a visual field. A sizeable body of this work has concentrated on how information directly (i.e. without requiring cognition) available from what is known as the ‘optic flow’ is used to control the timing and co-ordination of movement 1 Note that this approach is concerned with the analysis of cognitive work: it is not cognitive task analysis.
ARTICLE IN PRESS 674
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
through the world. The information is derived directly from pattern and motion perception, and guides action without the need for thought, calculation, or conscious decision-making. This model of the visual control of movement has been shown to apply to many areas, including walking, running, driving cars, and flight (both by humans and birds). From the perspective of ecological perception, the train driver is in a relatively unique and paradoxical situation. Seated at the front of a fast moving vehicle, the driver is on the one hand enclosed in an extremely powerful and compelling optic flow. But, apart from a very few situations (such as approaching buffers), the information that their senses would normally draw on to control movement is of no relevance to their driving task. The driver has no control over directional movement, and control of speed is entirely mediated by cognitive processes such as perception of signs, and route knowledge. Whether and how the driver balances the (cognitive) use of the speedometer and the (direct) use of the optic flow in estimating train speed is not known. The implications—if any—of this paradoxical situation for driver performance and reliability are not known, and, so far as we are aware, have never been investigated. It is however possible that it could, for example, contribute to the ‘driving without attention’ (DWAM) phenomenon (discussed below). Attention: There are at least two quite distinct meanings of the word ‘attention’ relevant to the study of train driver performance. Both have been intensively investigated for many years in laboratory and practical situations. The first is often called ‘vigilance’, and is typical of ‘watch-keeping’ tasks, where an observer tries to detect signals that arrive at infrequent intervals during which nothing much happens. Often the vigilance decrement can be avoided by an occasional message to the observer that requires an answer. The experimental Pro-Active AWS system (Dore, 1998) provided a means to achieve this in requiring, effectively, an ‘answer’ from the driver as to the signal aspect ahead. The vigilance decrement is also dependent on the extent of trust that the driver has in AWS. The occurrence of a string of frequent signals that do not require a response could cause the vigilance decrement to increase. The second meaning of attention is in ‘dynamic selective attention’. In real situations, particularly in the case of vision (though also in hearing), attention tends to be directed to one source of information at a time. To some extent attention can be shared between two or more tasks, although performance will usually be degraded. It is interesting that although evidence given to the Southall Inquiry maintained that drivers would always be looking at signals before the AWS sounded, in their
study of driver eye movements, Merat et al. (2002) found that on a significant proportion of signals the contrary was true. As predicted by quantitative attention models, attention was elsewhere and was attracted by the sound of the AWS. Driving without attention: There is considerable interest in psychological research and among safety professionals in the concepts of DWAM, and ‘lookingwithout-seeing’. These notions may have relevance to the AWS context, although no specific literature related to train driving has been identified. Allen (1965) wrote that, ‘In modern multi-aspect signalling there is a tendency to shorten the signal sections. [y] When traffic is dense through multi-aspect territory, a train will often come upon a quick succession of double or single yellow aspects. At every one the driver will get an AWS warning, which he has to cancel. In such conditions, the risk of frequent canceling action degenerating into a reflex action that loses its full meaning for the driver is a real one’ (p. 332). In their extensive review of Human Factors in road traffic safety, Dewar and Olsen (2002) consider the possible reasons for the large number of driving accidents associated with car driver inattention, ‘highway hypnosis’ and ‘DWAM’. They refer to the state as being associated with monotonous, uneventful driving, where a lack of novelty promotes automatic responses. Trust in automation: A growing body of research has been investigating the role that trust and confidence in technology plays in influencing the way people interact with technology-based systems. If a system is regarded as untrustworthy, it will tend not to be used correctly. Likewise, if an unreliable system is trusted then it will tend to be used even though it malfunctions. Despite the importance of trust in technology, it remains relatively ill-defined and only partly understood psychologically. The dynamics of trust in technology have been dealt with by Lee and Moray (1992) and Parasuraman and Riley (1997) among others in connection with supervisory control and automated systems. They define the dynamics of trust around predictability, dependability and faith. In simplistic terms, trust is established by consistent and desirable behaviour. This relies on the observability of system behaviour, as well as the individual actively sampling it. As trust is gained, the focus shifts away from observing specific behaviours towards assessing the global disposition of the device (it is said to be ‘dependable’). Over time, trust serves to reduce the effort expended in checking the system to ensure it is performing as expected. In a questionnaire survey of 277 drivers conducted as part of the AWS study (McLeod et al., 2003b), a substantial proportion reported issues suggesting drivers know they cannot always trust an AWS warning.
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
Simple heuristics and recognition-primed decisionmaking: Perhaps most intriguing, is work reported by Gigerenzer and Todd (1999) on what they term ‘simple heuristics’. Gigerenzer and Todd and their colleagues take the view that complex human performance in the real world cannot possibly involve the rational cognitive strategies assumed by much experimental psychology. Understanding decisions in the real world requires different, more psychologically plausible explanations. The work on simple heuristics focuses on what they term ‘fast and frugal heuristics—simple rules for making decisions with realistic mental resources’. The notion of simple heuristics is similar to the wellestablished concept of recognition-primed decisionmaking (see for example, Klein, 1989, 2003). Instead of emphasising heuristics and rules, Klein’s model of decision-making is based on the ability of humans to pattern-match. Repeated experiences are unconsciously linked together to form a pattern. Links between features in past and current situations enable typical features of situations to be extracted, and typical responses generated (Klein, 1989). This model is similar to Reason’s (1990) well-known GEMS model of human error. Whether or not ‘simple heuristics’ and/or ‘recognition-primed decisions’ are important in determining train driver performance with AWS remains to be seen. It might seem surprising if, in some form, they were not. 2.1. Summary of psychological thinking So how does consideration of this variety of psychological constructs help in understanding train driver performance? Two general observations seem relevant. The first is that, however keen the human factors professional might be for tools and methods that can be applied systematically across domains, the fact is that the psychological basis of human performance is exceedingly complex. Generic information-processing models of the human operator can provide useful generalisations. They have value in systems engineering, and in seeking to predict asymptotic limits of human performance. Even relatively simple models can have value as engineering tools by providing structure and drawing attention to possible limitations in the performance of humans in systems. However, as a means of understanding why specific individuals on a particular day in a particular set of circumstances behaved (or failed to behave) in a particular way, or predicting how an individual might behave in a given set of unexpected circumstances, such models have little to offer. The second general observation is that if understanding what happened in an incident, or what might happen, is important, the characteristics of the context
675
and situation at the time the behaviour of interest occurred or is expected to occur are paramount.
3. A situational model of train driver performance with AWS In summary then, train driver performance with AWS needs to be understood in terms of the context and situation at the time a signal is intended to influence driver behaviour. Existing approaches to analysing and modelling driver performance (based largely on information-processing models, and hierarchical task decompositions) are not able to capture these key contextual and situational factors. Based on consideration of the areas of psychological thinking outlined above, and drawing on insight from discussion with drivers, and consideration of incident records, a possible framework was developed to underpin assessment of the risk of loss of human reliability associated with extended AWS. More generally, the framework may be of value in any approach to understanding the cognitive aspects of driver behaviour. The aim was to provide a framework to help identify possible states of cognition at the time of interaction with an AWS signal (or any other perceptual/cognitive task) that could lead to loss of driver reliability. While the model is not directly aligned with any one of the psychological concepts summarised earlier, it draws on important aspects of many of them. In particular, and critically, the emphasis in the situational model is on the time domain. This reflects the fact that many of the circumstances and situational factors involved are inherently time-dependent, or time-limited. These include:
The role of time in controlling the safe passage of a train (such as the time for which signals are visible, and the time involved in accelerating or braking). Events or features which are only true at particular locations on a track. Expectations or beliefs a driver might hold because of the way in which events have evolved over time prior to an event (due to the spatial relationship between signals and speed restrictions, say, or because of the relative speed and distance between trains further down the track).
Fig. 1 illustrates a ‘situational model’ of driver behaviour. The model is based on the following assumptions (each of these is represented as an element on the figure).
Driver performance is based on knowledge, skills and expectations gained from training and previous driving experience. As well as explicit knowledge of
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
676
“NOW”
Perceive
Act Act
Decide
Driving Strategy
Allocation of Attention
Speed change
Immediate Priorities Spot track party
Next signal
Next speed limit
State of track ahead Sound of last alarm Current speed limit
Sight of last signal
Current World Model Current meaning of sunflower
Track events
Progress against timetable Immediate route history
AWS false alarm rate
Recent route history “Route Knowledge”
Expectations
Knowledge and Experience Driving Discipline
Organisational culture
Procedures, etc
Fig. 1. Situational model of driver performance in interacting with AWS.
the rules, procedures, and disciplines governing professional driving, this knowledge base includes recent experience of the route being driven (e.g. knowing the next signal has been at caution on every approach over the past fortnight). It will also include such things as a mental representation of the expected AWS false alarm and other failure rates based on the individual driver’s past experience. The driver holds a mental representation of the current state of the world as it affects the current driving activity. This includes elements such as the current speed limit, state of the track (e.g. are track-side workers in the vicinity?) and awareness of significant features in the route over a psychologically relevant timescale ahead. It will also hold awareness of the immediate history (e.g. ‘The last six caution signals have cleared before I reached them because I am part of a queue of trains following the same route, so the next one is likely to clear before I reach it too’). Two elements are particularly important in determining the driver’s cognitive state at any specific moment
(‘Now’). These are the immediate priority and the expectation of what the world will be like in the next few moments. The immediate priority might be to reduce speed, or to ensure every individual in a trackside party can be seen. Expectation might cover the expected location of the next signal, or the point at which a speed limit is expected to come into force. Both immediate priorities and expectations are continually changing and updating as the train progresses along the route. Allocation of the driver’s attentional resources are largely determined by the immediate priority and expectations. Responses to events in the outside world are determined by a driving strategy which can vary from moment to moment, within reasonably broad constraints. The nature of the driving strategy, and especially how they vary, will depend to a large extent on the drivers training, experience and confidence.
The situational model therefore focuses on trying to establish key aspects of the state of cognition at a
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
specific time in a specific situation and a specific context (what is shown on the framework as ‘Now’). The emphasis is on the immediate situation facing the driver, and the immediately preceding history. Longer term knowledge and experience is clearly important, but only in so far as it shapes or determines the state of cognition at the moment of behaviour. The situational model shown in Fig. 1 is aligned against an example of the type of simple informationprocessing model of human performance (PerceiveDecide-Act) widely used in previous train driver research. Aligning these approaches seeks to illustrate that, despite the limitations outlined earlier, the simple information-processing approach can have some value in understanding driver performance (particularly from a system design point of view). To be of any value in understanding real-time behaviour however, the information-processing approach must be viewed in the context of the situational factors that determine actual performance in a real-world, real-time, environment. Finally, by its nature the situational approach can represent no more than a ‘snapshot’ in time. Details of what matters at any moment, and how factors interact will change from moment to moment and from situation to situation. A given driver at a given signal on a given day may have a quite different cognitive state to the same driver, driving the same train, on the same route, at the same time though on a different day if the events leading up to interaction with the signal are sufficiently different.
677
4. Application of the situational model McLeod et al. (2003b) used the situational model of train driver behaviour as the basis for an assessment of the level of risk associated with extended uses of AWS. The aim was not to attempt to quantify the absolute level of driver unreliability with Extended AWS. Rather, and in common with established safety management practice, the assessment sought to provide confidence that the level of risk of driver unreliability was ‘as low as is reasonably practical’ (ALARP). The situational model provided the framework for carrying out this risk assessment, following the three stages described below.
4.1. Define and model scenarios Based on structured interviews with train drivers, and drawing on knowledge of real SPAD incidents, 20 scenarios representing a wide range of typical encounters with extended uses of AWS signals were developed. Each scenario was analysed based on the target and actual speed profile, the sequence of signals, and encounters with AWS warnings. Table 1 summarises 14 of these scenarios, and Fig. 2 illustrates a graphical representation of one of the scenarios. The scenario develops from left to right, with the encounter with the extended AWS warning occuring between points A and B.
Table 1 Brief description of the 20 AWS usage scenarios Brief description Incremental increases in line speed that do not reflect actual train acceleration. PSR and attendant AWS redundant, and in close proximity to signal. Driver already braking for signal when AWS received for PSR. Driver responds to line speed increases instead of AWS indications for cautionary signal. Driver is braking appropriately in response to AWS for TSR, but not appropriately for upcoming danger signal. TSR commencement is beyond signal at danger. Four aspect signalling interspersed with PSR’s. Driver is distracted by guard communication while receiving AWS warnings in quick succession for signal and PSR. Interrupted view of facing signals on bi-directional line, driver expects certain aspect(s) to be displayed. AWS does not discriminate between signals at caution or danger. Multiple AWS warnings due to trains bunching in rear of ESR. Repetitive AWS cancellations with no action required as train speed is already low. Closely spaced AWS magnets for PSR and signal. Driver receives horn but sunflower remains black, or, driver receives continuous horn activation requiring one cancellation, not two. Sunflower left activated after departing from AWS-fitted route onto an unfitted route. TSR in place with AWS, sunflower remains active for TSR and last fitted signal. Driver receives AWS at distant signal. Driver subsequently departs from station with sunflower active, proceeds through successive stop signals, but last in sequence is showing danger. Driver reads through to signal on wrong line. Correct AWS indication is assumed to be a Code 2 fault. Single line working with unsuppressed magnets. Initial reaction to AWS warnings is that they are redundant even though they may apply to an unexpected stop aspect. Signal located in tunnel. AWS coincides with having to sound horn and other train approaching. Competing priorities (view of signal and responding to AWS versus safety of track workers). Confusing TSR layout. Driver cancelling AWS just to keep the brakes off due to insufficient time to attribute individual warnings to specific signage.
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
678
Speed profile
Point B
Point A
90 70 60 Line Speed 40 Train Speed Physical Characteristics
Green
Green
Green
Green
Yellow
90
70 60
60
Signal
Warning Speed Board Sign
AWS 1
AWS 1 2
Red 60
Station
Station
AWS 3
AWS
AWS 4
Restricted visibility
AWS
5
6
Horn
Horn
AWS 7
AWS Indications Visual Audible
Bell
Bell
Bell
Bell
Horn
Situational Factors 1-Route expectancy 8-No action needed
2-Unexpected AWS
6-Ambiguous referents
9-Opposite action appopriate 18-pre-commitment
Fig. 2. Graphical representation of an Extended AWS scenario.
4.2. Identify situational factors Using the situational model as a guide, each scenario was then assessed to identify factors that might be important influences on the driver’s state of mind leading up to the time the AWS signal is encountered. Specifically, this looked for factors thought likely to be important in the drivers current world model, priorities and expectations (see Fig. 1) immediately before the AWS signal was encountered (i.e. the cognitive ‘Now’). Table 2 summarises 18 situational factors extracted from the 20 scenarios analysed. From the analysis of the 20 scenarios, four of these factors were identified most frequently; expectations about the route immediately ahead, uncertainty over what a warning refers to, ambiguous AWS referents (where the sunflower can refer to more than one thing), and situations where, for whatever reason, no action is needed in response to a warning. 4.3. Identify potential conflicts In the final stage of the assessment, potential conflicts between the various situational factors identified as influencing the driver’s state of cognition at the point of encounter with the Extended AWS signal (‘Now’) were
considered. A very simple influence model was used as a guide for making judgements about whether the driver’s reliability might be impaired due to conflicts between the driver’s likely model of the world, expectations about the immediate route ahead, and current priorities.
5. Conclusions The study forming the basis for this paper sought to understand the nature of driver reliability with the AWS, and to estimate the likely change in driver reliability if the use of AWS is further extended. Simple information-processing based models of driver performance were considered inadequate as a basis for making this assessment: specifically, they are thought not to be able to adequately capture the contextual and situated nature of human performance. The situational model of train driver performance, and the associated method for assessing human reliability described above are neither comprehensive, nor rigorous. They rely heavily on a competent and experienced analyst being able to apply complex psychological constructs to real-world situations. The strength of the method however is in focusing attention on the importance of a richer understanding of
Table 2 Situational factors affecting risk with Extended AWS (Identified in from analysis of 20 scenarios)
Route expectancy
Drivers’ expectations about the route or diagram (from route knowledge, WONs/PONs or immediate history) conflict with actual state of the route/ diagram. The AWS warning was not expected. AWS (audible or sunflower) expected but did not occur. The driver feels under time pressure to maintain line speed or progress, e.g. running behind schedule, conscious of heavy traffic behind or ahead. Driver’s attention may be distracted, for example, by phone/ radio, passing trains, track-side activity, entering/ leaving tunnels, etc. There is more than one indication current simultaneously, or it is not clear what caused the last AWS warning, i.e. the sunflower refers to more than one warning. The warning (horn or active sunflower) can be disregarded (such as unsuppressed magnets). The driver does not need to take any action to comply with the meaning of the warning, e.g. already running at or below required speed and/or route knowledge informs the driver that action can be postponed. Though the current behaviour is at odds with implications of the warning, the behaviour is legitimate in the circumstances (e.g. continues to accelerate despite TSR, as new speed restriction is still higher than current train speeds). Competing tasks or multiple priorities while an AWS indication is active. The driver experiences uncertainty about the train’s location in relation to the AWS referent, whether a warning is genuine or what the warning refers to, e.g. confusing TSR implementation and signage, and/or unusually long braking distances provided. Where there are long periods of monotony or repetitive simple work, e.g. prolonged high speed running under green aspects, or repetitive cancellation of AWS. The driver receives a large quantity of warnings in quick succession, e.g. on the approach to ESR’s interspersed with cautionary signal aspects. Magnets are closely spaced, and warnings can be cancelled with one button push/release. The sunflower is active, but can not be cancelled as successive signals are not AWS fitted, e.g. leaving an AWS-fitted line with the sunflower active onto an unfitted line, or receiving a warning at a semaphore distant signal for the last in a series of stop signals. The meaning of a particular AWS warning changes, e.g. after a long period where the same warning means one thing (e.g. 70 mph TSR) it changes to mean something different (e.g. 30 mph TSR). The physical ergonomics of the AWS cab equipment are poor, e.g. poor equipment placement, visibility, characteristics of audible alerts etc. Where the driver is already committed to a course of action after receiving an AWS warning, e.g. already braking for a station when approaching a cautionary signal.
Unexpected AWS AWS absent Time pressure Distractions Ambiguous referents Redundant AWS No action needed Opposite action appropriate Multiple priorities Uncertainty Monotony Quantity Multiple cancellations Sunflower not cancelled. Change in meaning Ergonomics Pre-commitment
ARTICLE IN PRESS
Description
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
Title
679
ARTICLE IN PRESS 680
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
the complex psychological factors involved in real-time human performance. However keen the human factors professional might be for tools and methods that can be applied systematically across domains, the fact is that the psychological basis of human performance is exceedingly complex. In the hands of suitably experienced and competent practitioners, both the situational model of driver behaviour, and the method used to assess driver reliability with AWS could have wider application in understanding loss of human reliability in safety critical systems. If understanding what happened in an incident, or what might happen, is important, the characteristics of the context and situation at the time the behaviour of interest occurred or is expected to occur are paramount. References Allen, G.F., 1965. British Rail After Beeching. Ian Allen, London. Dewar, R.E., Olsen, P., 2002. Human Factors in Traffic Safety. Criterion Press. Dore, D.M., (1998). The pro-active automatic warning system (PAWS). Report to the new measures working group. Gibson, J.J., 1979. The Ecological Approach to Visual Perception. Houghton Mifflin, Boston. Gigerenzer, G., Todd, P.M., 1999. Simple Heuristics That Make Us Smart. Oxford University Press, Oxford. Hall, S., 1999. Hidden Dangers: Railway Safety in the Era of Privatisation. Ian Allen, Shepperton, Surrey.
Hollnagel, E., 1998. Cognitive Reliability and Error Analysis Method. Elsevier, London. Hutchins, E., 1995. Cognition in the Wild. MIT Press, CAE. Klein, G., 1989. Recognition-primed decisions. Adv. Man-Machine Systems Res. 5, 47–92. Klein, G., 2003. Intuition at Work. Currency Doubleday, New York. Lee, J., Moray, N., 1992. Trust, control strategies and allocation of function in human–machine systems. Ergonomics 35 (10), 1243–1270. McLeod, R.W., Walker, G., Moray, N., (2003a). Extended AWS study: Review of the knowledge base. Nickleby HFE Ltd, B/C271/ FD.8. McLeod, R.W., Walker, G., Moray, N., Love, G., (2003b). Project summary report: driver reliability with extended AWS. Nickleby HFE Ltd, B/C271/FD.5. Merat, N., Mills, A., Bradshaw, M., Everatt, J., Groeger, J., 2002. Allocation of attention among train drivers. In: McCabe, P.T. (Ed.), Contemporary Ergonomics 2002. Taylor and Francis, London. Parasuraman, R., Riley, V., 1997. Humans and automation: use, misuse, disuse, abuse. Human Factors 39 (2), 230–253. Rasmussen, J., Pedersen, A-M., Goodstein, L., 1995. Cognitive Engineering: Concepts and Applications. Wiley, New York. Reason, J., 1990. Human Error. Cambridge University Press, Cambridge. Sarter, N.B., Woods, D.D., 1997. Team play with a powerful a nd independent agent: operational experiences and automation surprises on the Airbus A-320. Human Factors 39 (4), 553–569. Vaughan, A., 2000. Tracks to Disaster. Ian Allen, Shepperton, Surrey. Vicente, K.J., 1999. Cognitive Work Analysis: Towards Safe, Productive and Healthy Computer-Based Work. Lawrence Erlbaum, Marwah, NI.
Applied Ergonomics 36 (2005) 671–680 www.elsevier.com/locate/apergo
Analysing and modelling train driver performance Ronald W. McLeoda,, Guy H. Walkerb, Neville Moraya a
Nickleby HFE Ltd., 16, Waterside Avenue, Glasgow, G77 6TJ, UK Brunel University, School of Engineering & Design, Uxbridge UB8 3PH, UK
b
Received 25 November 2004; accepted 25 May 2005
Abstract Arguments for the importance of contextual factors in understanding human performance have been made extremely persuasively in the context of the process control industries. This paper puts these arguments into the context of the train driving task, drawing on an extensive analysis of driver performance with the Automatic Warning System (AWS). The paper summarises a number of constructs from applied psychological research thought to be important in understanding train driver performance. A ‘situational model’ is offered as a framework for investigating driver performance. The model emphasises the importance of understanding the state of driver cognition at a specific time (‘Now’) in a specific situation and a specific context. r 2005 Elsevier Ltd. All rights reserved. Keywords: Train driver performance; Automatic warning system; Models of human performance; Human reliability; Cognitive engineering
1. Introduction The work reported in this paper was carried out as part of a study for the Rail Safety and Standards Board (RSSB). The aim was to understand and assess the risks of driver unreliability associated with extended uses of the Automatic Warning System (AWS) on the UK rail network. 1.1. The Automatic Warning System (AWS) The purpose of railway signalling is to ensure safety and regulate traffic. Safety is ensured by maintaining sufficient headway separation between trains, given the relatively long distances required to slow them down. In practice this is achieved by displaying lights (combined with other signals) at the side of the track to communicate to the driver the need to slow down or stop. In the UK, the AWS also provides an audible alert of the state of the upcoming signal supported by an inCorresponding author. Tel.: +44 141 616 3104.
E-mail address: [email protected] (R.W. McLeod). 0003-6870/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.apergo.2005.05.006
cab visual reminder (known as the ‘sunflower’) once the warning has been acknowledged by the driver. If the driver fails to acknowledge the audible warning, a ‘full service’ brake application is made automatically. As the driver can override the automatic brake application, maintenance of safety still depends on the reliability of the driver. The issue for human factors is that, despite what appears to be an extremely powerful and compelling attention-getting device, combined with a highly visible visual reminder, there has been a significant and recurring incidence of AWS failing to prevent experienced train drivers passing signals at danger (SPADs). According to Vaughan (2000) in describing the original AWS system adopted by British Rail in 1936, ‘No-one ever thought that the driver would ignore the warning. It was utterly taken for granted that a driver would always take notice of the signal’ (p.12). In connection with the crash at Watford Junction in 1996, Hall (1999) wrote that ‘It seems inconceivable that a driver can acknowledge receiving two warnings and yet take no action to apply the brake to stop the train at the red signal, yet it happens’. (p.97).
ARTICLE IN PRESS 672
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
AWS was originally intended to support ‘semaphore’ signalling communicating two possible instructions to the driver: (a) stop at this signal, (b) proceed to the next signal. In the UK, semaphore signalling has nowadays been largely superceded by modern coloured light signalling. This system uses coloured lights to indicate up to four different states: green (meaning it is safe to proceed to the next signal), double yellow (expect to stop at the signal after the next one), single yellow (expect to stop at the next signal), red (stop at this signal). Coloured light signalling supports increased traffic density by providing earlier warning of the state of upcoming signals (and therefore more efficient headway separation) for drivers of high-speed trains. In the modern AWS system however, the audible warning still only discriminates between the original two signal states: ‘clear’ (indicated by a bell or simulated chime at around 1200 Hz) sounds if the driver is approaching a proceed, or green signal aspect; ‘warning’ (indicated by a steady alarm or horn sound at around 800 Hz) sounds if the signal is showing any other aspect apart from green. In addition to this use of AWS to support signalling, use of the system has in recent years been extended to draw the driver’s attention to a number of non-signal related events such as emergency, temporary or permanent speed restrictions, and certain level crossings. These are what is known as ‘Extended’ uses of AWS. In summary, while it appears relatively straightforward conceptually, the ways in which ‘standard’ (as opposed to Extended) AWS is used in the UK introduces a number of cognitive complexities for the driver. These include:
AWS does not fully discriminate between the possible states giving rise to the audible alert. The system depends on the driver correctly interpreting which of more than one possible condition the audible alert refers to, and therefore what behaviour is appropriate. There are situations where the in-cab visual reminder can refer not to the immediate past signal, but to a signal some time prior to that. The time period over which any AWS indication is ‘active’ in conveying information about the track ahead can vary from a few seconds to possibly many minutes.
1.2. Existing human factors approaches to understanding train driver cognition Train driver performance is dominated by cognitive and perceptual factors. Interaction with AWS is only one element in what can be a demanding perceptual and cognitive workload. While the context of the study was based on understanding possible driver unreliability
with AWS, the work reported here is thought to be relevant to understanding many aspects of cognitive performance in train driving. Much of the existing human factors research on train driver performance is concerned with SPADs and draws on hierarchically based forms of task analysis, based on relatively simple models of human decision-making and information processing (see McLeod et al., 2003a, for review). From the perspective of cognitive psychology and cognitive engineering, however, information-processing based approaches are thought to be fundamentally inappropriate for understanding real-time cognition, even for tasks that, superficially at least, appear as relatively well-defined and constrained as train driving. Specifically, information-processing based approaches are thought not to be able to capture the contextual and situated nature of human performance. Vicente (1999) offers a compelling critique of traditional approaches to task analysis. Similarly, Hollnagel (1998) provides a detailed critique of human reliability estimation techniques from the perspective of cognitive engineering. Although there are doubts about whether the techniques they propose (Cognitive Work Analysis (CWA) in the case of Vicente and the Cognitive Reliability and Error Analysis Method (CREAM) in the case of Hollnagel) overcome these limitations for the train driver context, the underlying arguments are very powerful. Little if any of the existing human factors research into train driving has tried to understand the mechanisms by which contextual and situational factors influence driver performance (other than in the very limited sense of seeking to identify factors likely to influence or ‘shape’ driver performance). The relative simplicity of the psychological basis of the existing research, and the continuing risk of SPAD incidents, suggests that a more comprehensive, psychologically richer understanding of the train driving task is needed.
2. Recent thinking in the psychology of real-world human performance There is a large and well-respected body of thinking and research that focuses on understanding human cognition and performance in the real-world contexts in which it occurs. Recognising the psychological complexity of the AWS problem, a number of ideas from this wider knowledge base were reviewed. These go beyond the immediate literature on human factors of AWS. The aim was to draw on thinking about cognitive performance in real-world tasks that seemed relevant to understanding the psychological context of train drivers’ use of AWS. The following sections very briefly summarise some of these areas. A more detailed review,
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
with full reference list, is included in McLeod et al. (2003a). Strategic behaviour: The importance of strategy and strategic behaviour in human performance has been recognised, at least in the research literature, for many years. In particular, strategy is essential in helping the human maintain performance in situations of stress, very high or very low workload or when subject to other influences (usually referred to as ‘Performance Shaping Factors’, PSFs). The concept of strategy has rarely however been used explicitly to try to understand or explain human behaviour and performance in realworld situations. An exception is recent research by Merat et al. (2002) into train driver eye movements that appears to provide some insight into strategic behaviour in train drivers. Situation awareness: Situation awareness is possibly the most widely cited construct in understanding human performance in complex real-time systems. Not surprisingly, it is also considered a key psychological construct in safe train driving. Loss of situation awareness leads to what Sarter and Woods (1997) have described as automation ‘surprises’. In connection with the Purley rail crash in 1989, Vaughan (2000) describes ‘Driver Morgan of the Littlehampton train [..] completely at a loss to understand what had happened’ (p. 91). Vaughan notes that driver Morgan must have acknowledged and cancelled the two previous AWS warnings leading up to the red signal, but failed to initiate an appropriate braking maneuver until it was too late. A wide range of factors can potentially influence the train drivers’ understanding or belief about the current state of the world and therefore how an AWS alarm is interpreted. These include: the nature of the alarm (bell or horn); visibility of signals and magnets on the track ahead; the driver’s interpretation of the nature of the preceding alarm; expectations about the current location; the train speed to be achieved, and by what time or by what location it is to be achieved. Changes in any of these factors can potentially lead to a change in the drivers’ interpretation of a particular AWS alarm. In combination, and when put into the wider context including factors such as the driver’s level of fatigue, emotional state, values and belief system, they have the potential to cause the driver to misinterpret what an AWS alarm refers to, and what action to take. Situated behaviour: The term ‘situated behaviour’ seeks to take understanding of human performance out of the laboratory context, and to ground it in the realworld environment in which it occurs. The common themes are that the context and particularly the situation at the time have an extremely powerful influence on how the human performs in real-world tasks.
673
The work of Rasmussen, Pedersen and Goodstein (1995), Vicente (1999) and many others on the analysis of cognitive work1 takes the view that it is not possible to understand human performance without locating it within organisational and technological constraints. They demonstrate why traditional normative analysis approaches are not well suited to understanding complex socio-technical systems because they underestimate (if they capture them at all) the contextdependent aspects of human performance. Distributed cognition: Ecological-based research such as much of the varied research into distributed cognition (e.g. Hutchins, 1995) emphasises the critical role that the situation and environment play in cognitive performance. The distributed cognition concept recognises that cognitive performance draws on, and is supported by many artefacts of the organisation and surrounding environment. It is, for example, clear that the placement of a prominent AWS magnet in the middle of the track can help to cue the driver in advance of a warning sounding. The driver is also likely to take advantage of many other environmental cues. These might include TPWS antennas, hot axle box detectors, other incidental AWS sounds (such as a relay’s clicking) and a myriad of other track/trackside artefacts. Cognitive control modes and subjectively available time: Hollnagel (1998) has developed a model of cognitive performance (COCOM) that recognises and seeks to account for the critical role of context in cognition. His ideas of cognitive control and subjectively available time seem potentially important in understanding train driver behaviour. The knowledge and experience of a route that drivers develop over time (what is known as ‘Route knowledge’) supports anticipation and future-orientated behaviour. Route knowledge allows the driver to think ahead, and helps control the allocation of cognitive and perceptual resources based on expectations about the future. It also helps the driver in spotting and interpreting cues and other information. In the COCOM framework, the effect of route knowledge might be to increase subjectively available time, thereby allowing the driver greater cognitive control over performance. Ecological optics: Following the lead of Gibson (1979), there is a very large body of knowledge about the ways in which humans, as well as many other species, make use of information directly available from movement of the eyes of an observer in a visual field. A sizeable body of this work has concentrated on how information directly (i.e. without requiring cognition) available from what is known as the ‘optic flow’ is used to control the timing and co-ordination of movement 1 Note that this approach is concerned with the analysis of cognitive work: it is not cognitive task analysis.
ARTICLE IN PRESS 674
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
through the world. The information is derived directly from pattern and motion perception, and guides action without the need for thought, calculation, or conscious decision-making. This model of the visual control of movement has been shown to apply to many areas, including walking, running, driving cars, and flight (both by humans and birds). From the perspective of ecological perception, the train driver is in a relatively unique and paradoxical situation. Seated at the front of a fast moving vehicle, the driver is on the one hand enclosed in an extremely powerful and compelling optic flow. But, apart from a very few situations (such as approaching buffers), the information that their senses would normally draw on to control movement is of no relevance to their driving task. The driver has no control over directional movement, and control of speed is entirely mediated by cognitive processes such as perception of signs, and route knowledge. Whether and how the driver balances the (cognitive) use of the speedometer and the (direct) use of the optic flow in estimating train speed is not known. The implications—if any—of this paradoxical situation for driver performance and reliability are not known, and, so far as we are aware, have never been investigated. It is however possible that it could, for example, contribute to the ‘driving without attention’ (DWAM) phenomenon (discussed below). Attention: There are at least two quite distinct meanings of the word ‘attention’ relevant to the study of train driver performance. Both have been intensively investigated for many years in laboratory and practical situations. The first is often called ‘vigilance’, and is typical of ‘watch-keeping’ tasks, where an observer tries to detect signals that arrive at infrequent intervals during which nothing much happens. Often the vigilance decrement can be avoided by an occasional message to the observer that requires an answer. The experimental Pro-Active AWS system (Dore, 1998) provided a means to achieve this in requiring, effectively, an ‘answer’ from the driver as to the signal aspect ahead. The vigilance decrement is also dependent on the extent of trust that the driver has in AWS. The occurrence of a string of frequent signals that do not require a response could cause the vigilance decrement to increase. The second meaning of attention is in ‘dynamic selective attention’. In real situations, particularly in the case of vision (though also in hearing), attention tends to be directed to one source of information at a time. To some extent attention can be shared between two or more tasks, although performance will usually be degraded. It is interesting that although evidence given to the Southall Inquiry maintained that drivers would always be looking at signals before the AWS sounded, in their
study of driver eye movements, Merat et al. (2002) found that on a significant proportion of signals the contrary was true. As predicted by quantitative attention models, attention was elsewhere and was attracted by the sound of the AWS. Driving without attention: There is considerable interest in psychological research and among safety professionals in the concepts of DWAM, and ‘lookingwithout-seeing’. These notions may have relevance to the AWS context, although no specific literature related to train driving has been identified. Allen (1965) wrote that, ‘In modern multi-aspect signalling there is a tendency to shorten the signal sections. [y] When traffic is dense through multi-aspect territory, a train will often come upon a quick succession of double or single yellow aspects. At every one the driver will get an AWS warning, which he has to cancel. In such conditions, the risk of frequent canceling action degenerating into a reflex action that loses its full meaning for the driver is a real one’ (p. 332). In their extensive review of Human Factors in road traffic safety, Dewar and Olsen (2002) consider the possible reasons for the large number of driving accidents associated with car driver inattention, ‘highway hypnosis’ and ‘DWAM’. They refer to the state as being associated with monotonous, uneventful driving, where a lack of novelty promotes automatic responses. Trust in automation: A growing body of research has been investigating the role that trust and confidence in technology plays in influencing the way people interact with technology-based systems. If a system is regarded as untrustworthy, it will tend not to be used correctly. Likewise, if an unreliable system is trusted then it will tend to be used even though it malfunctions. Despite the importance of trust in technology, it remains relatively ill-defined and only partly understood psychologically. The dynamics of trust in technology have been dealt with by Lee and Moray (1992) and Parasuraman and Riley (1997) among others in connection with supervisory control and automated systems. They define the dynamics of trust around predictability, dependability and faith. In simplistic terms, trust is established by consistent and desirable behaviour. This relies on the observability of system behaviour, as well as the individual actively sampling it. As trust is gained, the focus shifts away from observing specific behaviours towards assessing the global disposition of the device (it is said to be ‘dependable’). Over time, trust serves to reduce the effort expended in checking the system to ensure it is performing as expected. In a questionnaire survey of 277 drivers conducted as part of the AWS study (McLeod et al., 2003b), a substantial proportion reported issues suggesting drivers know they cannot always trust an AWS warning.
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
Simple heuristics and recognition-primed decisionmaking: Perhaps most intriguing, is work reported by Gigerenzer and Todd (1999) on what they term ‘simple heuristics’. Gigerenzer and Todd and their colleagues take the view that complex human performance in the real world cannot possibly involve the rational cognitive strategies assumed by much experimental psychology. Understanding decisions in the real world requires different, more psychologically plausible explanations. The work on simple heuristics focuses on what they term ‘fast and frugal heuristics—simple rules for making decisions with realistic mental resources’. The notion of simple heuristics is similar to the wellestablished concept of recognition-primed decisionmaking (see for example, Klein, 1989, 2003). Instead of emphasising heuristics and rules, Klein’s model of decision-making is based on the ability of humans to pattern-match. Repeated experiences are unconsciously linked together to form a pattern. Links between features in past and current situations enable typical features of situations to be extracted, and typical responses generated (Klein, 1989). This model is similar to Reason’s (1990) well-known GEMS model of human error. Whether or not ‘simple heuristics’ and/or ‘recognition-primed decisions’ are important in determining train driver performance with AWS remains to be seen. It might seem surprising if, in some form, they were not. 2.1. Summary of psychological thinking So how does consideration of this variety of psychological constructs help in understanding train driver performance? Two general observations seem relevant. The first is that, however keen the human factors professional might be for tools and methods that can be applied systematically across domains, the fact is that the psychological basis of human performance is exceedingly complex. Generic information-processing models of the human operator can provide useful generalisations. They have value in systems engineering, and in seeking to predict asymptotic limits of human performance. Even relatively simple models can have value as engineering tools by providing structure and drawing attention to possible limitations in the performance of humans in systems. However, as a means of understanding why specific individuals on a particular day in a particular set of circumstances behaved (or failed to behave) in a particular way, or predicting how an individual might behave in a given set of unexpected circumstances, such models have little to offer. The second general observation is that if understanding what happened in an incident, or what might happen, is important, the characteristics of the context
675
and situation at the time the behaviour of interest occurred or is expected to occur are paramount.
3. A situational model of train driver performance with AWS In summary then, train driver performance with AWS needs to be understood in terms of the context and situation at the time a signal is intended to influence driver behaviour. Existing approaches to analysing and modelling driver performance (based largely on information-processing models, and hierarchical task decompositions) are not able to capture these key contextual and situational factors. Based on consideration of the areas of psychological thinking outlined above, and drawing on insight from discussion with drivers, and consideration of incident records, a possible framework was developed to underpin assessment of the risk of loss of human reliability associated with extended AWS. More generally, the framework may be of value in any approach to understanding the cognitive aspects of driver behaviour. The aim was to provide a framework to help identify possible states of cognition at the time of interaction with an AWS signal (or any other perceptual/cognitive task) that could lead to loss of driver reliability. While the model is not directly aligned with any one of the psychological concepts summarised earlier, it draws on important aspects of many of them. In particular, and critically, the emphasis in the situational model is on the time domain. This reflects the fact that many of the circumstances and situational factors involved are inherently time-dependent, or time-limited. These include:
The role of time in controlling the safe passage of a train (such as the time for which signals are visible, and the time involved in accelerating or braking). Events or features which are only true at particular locations on a track. Expectations or beliefs a driver might hold because of the way in which events have evolved over time prior to an event (due to the spatial relationship between signals and speed restrictions, say, or because of the relative speed and distance between trains further down the track).
Fig. 1 illustrates a ‘situational model’ of driver behaviour. The model is based on the following assumptions (each of these is represented as an element on the figure).
Driver performance is based on knowledge, skills and expectations gained from training and previous driving experience. As well as explicit knowledge of
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
676
“NOW”
Perceive
Act Act
Decide
Driving Strategy
Allocation of Attention
Speed change
Immediate Priorities Spot track party
Next signal
Next speed limit
State of track ahead Sound of last alarm Current speed limit
Sight of last signal
Current World Model Current meaning of sunflower
Track events
Progress against timetable Immediate route history
AWS false alarm rate
Recent route history “Route Knowledge”
Expectations
Knowledge and Experience Driving Discipline
Organisational culture
Procedures, etc
Fig. 1. Situational model of driver performance in interacting with AWS.
the rules, procedures, and disciplines governing professional driving, this knowledge base includes recent experience of the route being driven (e.g. knowing the next signal has been at caution on every approach over the past fortnight). It will also include such things as a mental representation of the expected AWS false alarm and other failure rates based on the individual driver’s past experience. The driver holds a mental representation of the current state of the world as it affects the current driving activity. This includes elements such as the current speed limit, state of the track (e.g. are track-side workers in the vicinity?) and awareness of significant features in the route over a psychologically relevant timescale ahead. It will also hold awareness of the immediate history (e.g. ‘The last six caution signals have cleared before I reached them because I am part of a queue of trains following the same route, so the next one is likely to clear before I reach it too’). Two elements are particularly important in determining the driver’s cognitive state at any specific moment
(‘Now’). These are the immediate priority and the expectation of what the world will be like in the next few moments. The immediate priority might be to reduce speed, or to ensure every individual in a trackside party can be seen. Expectation might cover the expected location of the next signal, or the point at which a speed limit is expected to come into force. Both immediate priorities and expectations are continually changing and updating as the train progresses along the route. Allocation of the driver’s attentional resources are largely determined by the immediate priority and expectations. Responses to events in the outside world are determined by a driving strategy which can vary from moment to moment, within reasonably broad constraints. The nature of the driving strategy, and especially how they vary, will depend to a large extent on the drivers training, experience and confidence.
The situational model therefore focuses on trying to establish key aspects of the state of cognition at a
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
specific time in a specific situation and a specific context (what is shown on the framework as ‘Now’). The emphasis is on the immediate situation facing the driver, and the immediately preceding history. Longer term knowledge and experience is clearly important, but only in so far as it shapes or determines the state of cognition at the moment of behaviour. The situational model shown in Fig. 1 is aligned against an example of the type of simple informationprocessing model of human performance (PerceiveDecide-Act) widely used in previous train driver research. Aligning these approaches seeks to illustrate that, despite the limitations outlined earlier, the simple information-processing approach can have some value in understanding driver performance (particularly from a system design point of view). To be of any value in understanding real-time behaviour however, the information-processing approach must be viewed in the context of the situational factors that determine actual performance in a real-world, real-time, environment. Finally, by its nature the situational approach can represent no more than a ‘snapshot’ in time. Details of what matters at any moment, and how factors interact will change from moment to moment and from situation to situation. A given driver at a given signal on a given day may have a quite different cognitive state to the same driver, driving the same train, on the same route, at the same time though on a different day if the events leading up to interaction with the signal are sufficiently different.
677
4. Application of the situational model McLeod et al. (2003b) used the situational model of train driver behaviour as the basis for an assessment of the level of risk associated with extended uses of AWS. The aim was not to attempt to quantify the absolute level of driver unreliability with Extended AWS. Rather, and in common with established safety management practice, the assessment sought to provide confidence that the level of risk of driver unreliability was ‘as low as is reasonably practical’ (ALARP). The situational model provided the framework for carrying out this risk assessment, following the three stages described below.
4.1. Define and model scenarios Based on structured interviews with train drivers, and drawing on knowledge of real SPAD incidents, 20 scenarios representing a wide range of typical encounters with extended uses of AWS signals were developed. Each scenario was analysed based on the target and actual speed profile, the sequence of signals, and encounters with AWS warnings. Table 1 summarises 14 of these scenarios, and Fig. 2 illustrates a graphical representation of one of the scenarios. The scenario develops from left to right, with the encounter with the extended AWS warning occuring between points A and B.
Table 1 Brief description of the 20 AWS usage scenarios Brief description Incremental increases in line speed that do not reflect actual train acceleration. PSR and attendant AWS redundant, and in close proximity to signal. Driver already braking for signal when AWS received for PSR. Driver responds to line speed increases instead of AWS indications for cautionary signal. Driver is braking appropriately in response to AWS for TSR, but not appropriately for upcoming danger signal. TSR commencement is beyond signal at danger. Four aspect signalling interspersed with PSR’s. Driver is distracted by guard communication while receiving AWS warnings in quick succession for signal and PSR. Interrupted view of facing signals on bi-directional line, driver expects certain aspect(s) to be displayed. AWS does not discriminate between signals at caution or danger. Multiple AWS warnings due to trains bunching in rear of ESR. Repetitive AWS cancellations with no action required as train speed is already low. Closely spaced AWS magnets for PSR and signal. Driver receives horn but sunflower remains black, or, driver receives continuous horn activation requiring one cancellation, not two. Sunflower left activated after departing from AWS-fitted route onto an unfitted route. TSR in place with AWS, sunflower remains active for TSR and last fitted signal. Driver receives AWS at distant signal. Driver subsequently departs from station with sunflower active, proceeds through successive stop signals, but last in sequence is showing danger. Driver reads through to signal on wrong line. Correct AWS indication is assumed to be a Code 2 fault. Single line working with unsuppressed magnets. Initial reaction to AWS warnings is that they are redundant even though they may apply to an unexpected stop aspect. Signal located in tunnel. AWS coincides with having to sound horn and other train approaching. Competing priorities (view of signal and responding to AWS versus safety of track workers). Confusing TSR layout. Driver cancelling AWS just to keep the brakes off due to insufficient time to attribute individual warnings to specific signage.
ARTICLE IN PRESS R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
678
Speed profile
Point B
Point A
90 70 60 Line Speed 40 Train Speed Physical Characteristics
Green
Green
Green
Green
Yellow
90
70 60
60
Signal
Warning Speed Board Sign
AWS 1
AWS 1 2
Red 60
Station
Station
AWS 3
AWS
AWS 4
Restricted visibility
AWS
5
6
Horn
Horn
AWS 7
AWS Indications Visual Audible
Bell
Bell
Bell
Bell
Horn
Situational Factors 1-Route expectancy 8-No action needed
2-Unexpected AWS
6-Ambiguous referents
9-Opposite action appopriate 18-pre-commitment
Fig. 2. Graphical representation of an Extended AWS scenario.
4.2. Identify situational factors Using the situational model as a guide, each scenario was then assessed to identify factors that might be important influences on the driver’s state of mind leading up to the time the AWS signal is encountered. Specifically, this looked for factors thought likely to be important in the drivers current world model, priorities and expectations (see Fig. 1) immediately before the AWS signal was encountered (i.e. the cognitive ‘Now’). Table 2 summarises 18 situational factors extracted from the 20 scenarios analysed. From the analysis of the 20 scenarios, four of these factors were identified most frequently; expectations about the route immediately ahead, uncertainty over what a warning refers to, ambiguous AWS referents (where the sunflower can refer to more than one thing), and situations where, for whatever reason, no action is needed in response to a warning. 4.3. Identify potential conflicts In the final stage of the assessment, potential conflicts between the various situational factors identified as influencing the driver’s state of cognition at the point of encounter with the Extended AWS signal (‘Now’) were
considered. A very simple influence model was used as a guide for making judgements about whether the driver’s reliability might be impaired due to conflicts between the driver’s likely model of the world, expectations about the immediate route ahead, and current priorities.
5. Conclusions The study forming the basis for this paper sought to understand the nature of driver reliability with the AWS, and to estimate the likely change in driver reliability if the use of AWS is further extended. Simple information-processing based models of driver performance were considered inadequate as a basis for making this assessment: specifically, they are thought not to be able to adequately capture the contextual and situated nature of human performance. The situational model of train driver performance, and the associated method for assessing human reliability described above are neither comprehensive, nor rigorous. They rely heavily on a competent and experienced analyst being able to apply complex psychological constructs to real-world situations. The strength of the method however is in focusing attention on the importance of a richer understanding of
Table 2 Situational factors affecting risk with Extended AWS (Identified in from analysis of 20 scenarios)
Route expectancy
Drivers’ expectations about the route or diagram (from route knowledge, WONs/PONs or immediate history) conflict with actual state of the route/ diagram. The AWS warning was not expected. AWS (audible or sunflower) expected but did not occur. The driver feels under time pressure to maintain line speed or progress, e.g. running behind schedule, conscious of heavy traffic behind or ahead. Driver’s attention may be distracted, for example, by phone/ radio, passing trains, track-side activity, entering/ leaving tunnels, etc. There is more than one indication current simultaneously, or it is not clear what caused the last AWS warning, i.e. the sunflower refers to more than one warning. The warning (horn or active sunflower) can be disregarded (such as unsuppressed magnets). The driver does not need to take any action to comply with the meaning of the warning, e.g. already running at or below required speed and/or route knowledge informs the driver that action can be postponed. Though the current behaviour is at odds with implications of the warning, the behaviour is legitimate in the circumstances (e.g. continues to accelerate despite TSR, as new speed restriction is still higher than current train speeds). Competing tasks or multiple priorities while an AWS indication is active. The driver experiences uncertainty about the train’s location in relation to the AWS referent, whether a warning is genuine or what the warning refers to, e.g. confusing TSR implementation and signage, and/or unusually long braking distances provided. Where there are long periods of monotony or repetitive simple work, e.g. prolonged high speed running under green aspects, or repetitive cancellation of AWS. The driver receives a large quantity of warnings in quick succession, e.g. on the approach to ESR’s interspersed with cautionary signal aspects. Magnets are closely spaced, and warnings can be cancelled with one button push/release. The sunflower is active, but can not be cancelled as successive signals are not AWS fitted, e.g. leaving an AWS-fitted line with the sunflower active onto an unfitted line, or receiving a warning at a semaphore distant signal for the last in a series of stop signals. The meaning of a particular AWS warning changes, e.g. after a long period where the same warning means one thing (e.g. 70 mph TSR) it changes to mean something different (e.g. 30 mph TSR). The physical ergonomics of the AWS cab equipment are poor, e.g. poor equipment placement, visibility, characteristics of audible alerts etc. Where the driver is already committed to a course of action after receiving an AWS warning, e.g. already braking for a station when approaching a cautionary signal.
Unexpected AWS AWS absent Time pressure Distractions Ambiguous referents Redundant AWS No action needed Opposite action appropriate Multiple priorities Uncertainty Monotony Quantity Multiple cancellations Sunflower not cancelled. Change in meaning Ergonomics Pre-commitment
ARTICLE IN PRESS
Description
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
Title
679
ARTICLE IN PRESS 680
R.W. McLeod et al. / Applied Ergonomics 36 (2005) 671–680
the complex psychological factors involved in real-time human performance. However keen the human factors professional might be for tools and methods that can be applied systematically across domains, the fact is that the psychological basis of human performance is exceedingly complex. In the hands of suitably experienced and competent practitioners, both the situational model of driver behaviour, and the method used to assess driver reliability with AWS could have wider application in understanding loss of human reliability in safety critical systems. If understanding what happened in an incident, or what might happen, is important, the characteristics of the context and situation at the time the behaviour of interest occurred or is expected to occur are paramount. References Allen, G.F., 1965. British Rail After Beeching. Ian Allen, London. Dewar, R.E., Olsen, P., 2002. Human Factors in Traffic Safety. Criterion Press. Dore, D.M., (1998). The pro-active automatic warning system (PAWS). Report to the new measures working group. Gibson, J.J., 1979. The Ecological Approach to Visual Perception. Houghton Mifflin, Boston. Gigerenzer, G., Todd, P.M., 1999. Simple Heuristics That Make Us Smart. Oxford University Press, Oxford. Hall, S., 1999. Hidden Dangers: Railway Safety in the Era of Privatisation. Ian Allen, Shepperton, Surrey.
Hollnagel, E., 1998. Cognitive Reliability and Error Analysis Method. Elsevier, London. Hutchins, E., 1995. Cognition in the Wild. MIT Press, CAE. Klein, G., 1989. Recognition-primed decisions. Adv. Man-Machine Systems Res. 5, 47–92. Klein, G., 2003. Intuition at Work. Currency Doubleday, New York. Lee, J., Moray, N., 1992. Trust, control strategies and allocation of function in human–machine systems. Ergonomics 35 (10), 1243–1270. McLeod, R.W., Walker, G., Moray, N., (2003a). Extended AWS study: Review of the knowledge base. Nickleby HFE Ltd, B/C271/ FD.8. McLeod, R.W., Walker, G., Moray, N., Love, G., (2003b). Project summary report: driver reliability with extended AWS. Nickleby HFE Ltd, B/C271/FD.5. Merat, N., Mills, A., Bradshaw, M., Everatt, J., Groeger, J., 2002. Allocation of attention among train drivers. In: McCabe, P.T. (Ed.), Contemporary Ergonomics 2002. Taylor and Francis, London. Parasuraman, R., Riley, V., 1997. Humans and automation: use, misuse, disuse, abuse. Human Factors 39 (2), 230–253. Rasmussen, J., Pedersen, A-M., Goodstein, L., 1995. Cognitive Engineering: Concepts and Applications. Wiley, New York. Reason, J., 1990. Human Error. Cambridge University Press, Cambridge. Sarter, N.B., Woods, D.D., 1997. Team play with a powerful a nd independent agent: operational experiences and automation surprises on the Airbus A-320. Human Factors 39 (4), 553–569. Vaughan, A., 2000. Tracks to Disaster. Ian Allen, Shepperton, Surrey. Vicente, K.J., 1999. Cognitive Work Analysis: Towards Safe, Productive and Healthy Computer-Based Work. Lawrence Erlbaum, Marwah, NI.