- Email: [email protected]
Experiments on Air Traffic Controller's Representation as Approach to MMS-Design
Copyright e IFAC Integrated Systems Engineering, Baden-Baden, Germany, 1994
EXPERIMENTS ON AIR TRAFFIC CONTROLLER'S REPRESENTATION AS APPROACH TO MMS-DESIGN T. BIERWAGEN and H. HELBING Berlin University of Technology, Department of Psychology, DFG-Projekt 'Flugsiche"mg als MMS ', Dovestr. 1-5, 10587 Berlin , Germany
Abstract: Due to rising automation in the field of air traffic control, the importance of knowledge about mental information processing is discussed. Based on a brief review of the discussion on mental models, a working definition of the term is given . As part of this definition , the structure of information representation is investigated by an experiment , following the priming paradigm and carried out with experienced controllers. The experiment is reported in detail , including a discussion of the results. The results are set into the broader context of implications for man-machine-system design in air traffic control. Key Words: Man-machine systems; mental models; air-traffic control; cognitive systems; human factors; simulation; automation
1. INTRODUCTION
2. MENTAL MODELS
In the domain of air traffic control (ATC) with its highly integrated and safety-critical automated systems human factors gain importance. In this respect, designing the future man-machine-system (MMS) in ATC needs basic knowledge about mental information processing of the operators. Such knowledge allows a description of mental models of controllers. Although the meaning of the term 'mental model' seem to vary significantly, some common aspects can be stated. That is on the one hand the use of static system knowledge and on the other hand the use of actual, dynamic data and strategies to transform these data. For a description of strategies of information intake, processing and mental models the representation of data has to be investigated as a basic step .
Reviewing literature one can find, that the term 'mental model' is fairly popular in scientific community. Psychologists as well as human factors engineers use it in different domains. This again leads to very different ideas of what a mental model is. In fact , there is a need of definition on what a 'mental model' is and what indeed it is not . Going back to literature, it has to be stated, that a common definition is not available for this term. On the contrary, people use their ideosyncratic definitions. These definitions are domain-specific and user-specific. Domain-specific because cases are handled , which are implied by the domain the model was developed in. Generalizations are omitted. They are user-specific because the background of the person using this model for scientific or design purposes comes in: human factors engineers tend to use other terms and definitions than cognitive psychologists.
The report gives a brief review of the discussion of the term 'mental model ' together with the author's ideas on that topic. Based on this, an experiment regarding the structure of representation in the field of ATC as one fundamental element of a mental model is reported. Conclusions are drawn to further implications on man-machinesystem-design in future ATC-systems. The work reported was carried out in an interdisciplinary research project at Berlin University of Technology, named "En-route Controller's Representation (EnCoRe)" .
Undoubtably several influences on the term have to be considered . Mental models usually apply to very complex systems. This implies a specific kind of 'weakness' in defining exactly their way of working , their contents or their structure. Norman (1983) states, that they are incomplete, unstable, non-exclusive, unscientific and inconsistent. They often are context-sensitive as well, and a general abstraction from any context comes along with
327
losses in accuracy.
total set of schemata instantiated at the time." (cited from Wilson & Rutherford , 1989, p. 624) .
2. 1. Mental Models in th e field of MMS-design
Wilson & Rutherford (1989) propose three different kinds of models for further use in human factors research : the designer 's conceptual model , the user 's conceptual model and the user 's mental model. Whilst the conceptual models consist of ideas on a model, the user's mental model refers to a description of the user's internal representation in terms of cognitive psychology. Main differences in the definitions of mental models are pointed out by Wilson & Rutherford as well: " . .. whereas in psychology the attempt is to describe mental processes, in human factors it is the product of such processes that is of concern." (p . 622).
The term mental model seems to be based on Craik, as cited from Wilson & Rutherford (1989) : " ... the originator of the mental model notion , the Scottish psychologist Keneth Craik . . . " (p . 621) . TofHer stated the common expression , that " every person carries within his head a mental model of the world - a subjective representation of external reality" (p . 621). One of the frequently cited cognitive psychologists in the field of mental models is Johnson-Laird (1983) . He defines mental models as to " . . . enable individuals to make inferences and predictions, to understand phenomena, to decide what action to take and to control its execution ... " (p. 397) . He provides a typology of mental models with the main distinction between conceptual and physical mental models, with more widespread categories in the second level. He stresses the quality of 'running' a mental model and the relational structure of the representation. His basic idea is that of effective procedures as foundation of any mental process.
In summary no clear and commonly shared understanding of the scientific construct 'mental model' is arising at the horizon. Several problems like different aims in using mental models in psychology and human factors , psychological discussions on the format of representation (analogue versus propositional) and suitability of different concepts (,mental model ', 'schemata', 'picture', 'frame') to describe mental information processing need further investigation . A clear definition of the term 'mental model ' has to be given, otherwise interdisciplinary cooperation on related topics will continue to imply misunderstandings.
Norman (1983) introduces the distinction between a target system, a conceptual model, a scientists conceptualization and a system image. If the conceptual model (the target system representation from the designer's point of view) meets the demands of being exact, complete and consistent, the user will have a very good system image of the process. This for the user leads to an useful mental model.
2.2. Mental Models in EnCoRe Project
Due to the problems discussed above, the authors formed a (not really closed) working-definition of 'mental model' in the domain of ATC tasks. Mental models are dynamic subjective conceptions of any subject matter. They deal with data, using strategies that are controlled by creative or inferential processes. Looking at the data, there are two kinds of data to be distinguished: static data, established in long term memory, and dynamic data, handled in short term memory. In the context of ATC, static or quasi-static data can be identified as system knowledge like airspace structure, working regulations, aircraft performance data, physical constraints like the existence of radars , navigation aids , airports etc. Dynamic data are divided in low-frequency and high-frequency data. The border between them in the project EnCoRe is lined out anywhere close to 60 minutes. Low-frequency dynamic data are data like status of navigation aids or runways, temporary restricted areas, weather data etc. The most important data for the controller's actions are highfrequency data like information displayed on the radar screen, flight progress strips in use , coordination with adjacent sectors or centers, voice com-
Rouse & Morris suggest two dimensions of mental models: the level of behavioural discretion and the nature of model manipulation . These dimensions correspond to two different experimental methods. Problems with low level of behavioural discretion are to be assessed with inferential methods, whereas problems of an explicit nature of model manipulation are to be assessed by verbal methods. They " ... conclude that mental models are not necessarily computational models." (cited from Wilson & Rutherford, 1989, p . 619). Approaching from ATC, Whitfield & Jackson (1982) proposed the term 'picture' as a kind of mental model. This could be understood as an internal representation of the traffic situation and a set of rules to evaluate mentally possible actions prior to their execution. The concept is based mainly on the dynamic radar screen data used by controllers to handle traffic. In opposite to this Rumelhart ". .. described a mental model as the 328
increasing 'black box' nature of systems, the ... complexity of control . .. mean that in large part the mental models that operators develop are in the hands of designers" (p. 627). In fact, systems should be designed for the user, and therefore a deep understanding of the user's mental processes is essential for successful syst.em design. A way of describing these mental information processing is the terminology of a mental model. If one does not want to design technology for technology, but for man, mental models should be the starting point to fit technology as much to them as possible.
munication with the pilots etc. The sum of these data is contained in the term 'picture' as it is used in the domain of ATC. In contrast to static data, dynamic data is prepared and presented technically, and therefore this kind of data is subject to automation . Strategies inhering in a mental model arise from selection and, more important, from training and growing experience during work. Normally they are not easily to verbalize by controllers, but principally they are codifyable. They can be expressed in "IF ... THEN ... " rules and are put together to a more or less complete set of rules. With regard to Rasmussen (1986), they are the short-cuts in his 'decision ladder'. They can be described in terms of schemata fairly good. This fact leads to the statement, that schemata are a fundamental part of mental models.
Taking into account that, together with other aspects like a shift from (process-oriented) process management tasks towards (interface-oriented) information management tasks, mental models will lead to systems with a greater reliablility and validity and with less semantic errors produced in the design process.
Summing up different kinds of data and strategies, or schemata, does not justify a new hypothetical construct like a mental model. Another element has to be added. This element is the creative or inferential component. Based on strategies dynamic data are interpreted and - if necessary, e. g. in case of a conOict - solutions are generated. The set of rules is reviewed in search of an identical situation. If there is none, similar situations are to be identified. The rules related to this similar situation are applied and, in case of success, memorized and added to the set of rules. The core of this infering process in the domain of ATC is the ability to predict future situations. Based on data and strategies, the controllers interpret data by checking future systems states to detect conOicting aircrafts. In case of conOicts detected, they check different solutions by mentally checking the quality of the solution in order to choose the best one. This third component of a mental model is one of the main reasons why man should be kept in the loop. It has to be examined very carefully, if it is senseful to automate these processes with regard to safety and efficiency.
.I 3. PRIMING EXPERIMENT Dealing with the structure of a mental model, one point of interest is the mental representation of high-frequency dynamic data (picture). Different ways of structuring are possible. One could imagine parameters as sorting criteria as well as the (analogue) position or any kind of relation between two or more aircrafts. The relational representation shows the advantage of goal-relevance: the main goal is to detect and to avoid conOicts. If there is any sufficient conflict indicator (like distance between aircrafts), the relation of this parameter regarding pairs of aircrafts would be an unambigous indicator of a conflict, and this way of representation should be more efficient in terms of problem-solving. But speaking of more than 4-5 aircrafts under control, a complete relational representation is impossible because of the number of relations to be regarded. The experiment reported is guided by another idea: The focus are objects with attributes (parameters) attached. One ofthese parameters may indicate a conflict (see also Falzon, 1982), depending on parameters of further objects. By this, one parameter may define also a group of conflicting objects. In cognitive psychology this fact is known as chunking or clustering of information. This way of representation seems to be more efficient in terms of memory-usage.
So in contrast to Dubois & Gaussin (1993) the terms 'mental representation' and 'mental model' are not interchangeable. Rather, mental representation is a fundamental part of a mental model, which, in addition, allows for creating unique solutions.
2.9. Benefits of a Mental Model
Bringing together the advantages of efficient selection of information and efficient problem-solving suggests to use both approaches in combination. Due to the specific needs one of the representational form could be used: object-attributerepresentation for handling bulks of information,
Picking up the discussion of designer and user model, it was stated, that they have to be congruent. This convergence can be gained in two different ways: the designer can fit his ideas to the user, or VIce versa. Wilson & Rutherford state, that "the 329
and relational representation, if fast problemsolving is needed. A similar approach was suggested by Leroux (1993) .
C.1I11", FlIghtJe.,.' . CI. .red . Fllghtlenl
/ 230
-0
3.1. Method
DLH211 11713113
.
Speed
Vertical Spe.cl Indicator
e
e
e
e .
Fig. 1: Radar Target
In order to test the object-attribute structure of the picture of air traffic controllers, an ATCsimulation system was developed, and lab experiments were performed. Specific parameters of aircrafts (like 'flightlevel') in dependency oflateral distance ('far away ' from each other, or 'nearby') were varied systematically. According to the parameters 'flightlevel '1, 'vertical speed component,2, 'ETO'3 and 'heading'4, two aircrafts were in conflict or not. In analogy to the priming paradigm, reported by McNeill (1966) , any memory access on aircrafts established in the same cluster is expected to be significantly faster than for two aircrafts belonging to different clusters. Specific to the idea of priming is the activation of clusters prior to the query. The simulation was stopped, one aircraft (prime) was highlighted, then the callsign of another aircraft was presented on the cleared screen . The subjects were asked wether this second aircraft (target) was under control in the simulation at that time. The variation of parameters (e. g. both aircrafts same 'flightlevel' or not) was perfomed implicitly. The subjects were not told about this fact. Response times were recorderd and analyzed.
stem called 'EnCoRe-PLUS' (EnCoRe - programmable airspace simulation) is a PC-based simulation of a radar controller working position. ATCspecific data as well as experiment-specific data can be determined in setup-files . The system presents a normal secondary-radar display supplemented by electronical flight progress strips (callsign and route information only). This image can be varied according to experimental aims. Airspace structure data can be displayed if needed . To perform interactive scenarios with online changes in flight data, a ghost-pilot system can be connected to the system. An intercom for verbal communication between controller and pilot, including tape recording, is available. Every event, especially all kinds of interaction with the system concerning traffic flow or experimental issues, are recorded in a logfile. Computer-aided data analysis is supported. The presentation of radar targets (see Fig. 1) is similar to existing ATC-systems. The primary target is the rhombus. Attached is a label with further alphanumerical information (callsign, flightlevel, speed, on demand a vertical speed indicator and cleared flightlevel) , a velocity leader and an artificial afterglow, presenting four positions in distances of 10 seconds.
Hypothesis were as follows: A. Response time and error rate are expected to be smaller, if prime and target share the same 'flightlevel' and 'vertical speed component'. No differences are expected for the parameters 'ETO' and 'heading'. B. Response time and error rate are different depending on the lateral separation (presentation distance) of prime and target . Shorter response times and less errors are expected in case of nearby presented aircrafts.
The experiments were performed in the 'Bremen Siid Radar 2'-sector of the lower airspace in Germany. This piece of airspace was selected taking into account usual traffic structure as w~ll as the controllers' familiarity with the airspace. The experiment was non-interactive. Presentation of the relevant parameters differed : 'flightlevel' was presented as alphanumeric information, 'vertical speed component' as symbolic information (up- or down-arrow), 'heading' as symbolic information as well (map display) . 'ETO' had to be assessed by the controllers.
The experiment was carried out with 25 experienced controllers from Berlin Air Route Traffic Control Center. The average overall experience was 10 years. All participants were volunteers.
The priming queries were presented every 30-40 seconds, embedded in a running simulation. The answers (yes/no) were given by specified keys on the computer keyboard. Maximum allowed response time was 5000 ms, then simulation restarted. Two different kinds of distractor-requests were included in the experiment to prevent subjects from simply learning callsigns. Questionaires were used successfully to show the validity of distractor tasks.
3.2. Experimental Simulation System
Due to the demands of performing cognitivepsychological experiments in the domain of ATC, a new simulation facility was build up. The sylaltitude, in steps of 100 ft climb or deacent 3 Estimated Time Over the next Fix {given in degree from O· to 359· 2 indicating
330
each other were found (p ::; .01) . 2100 -
-- - -
--- .. _-
-
2000
-
100%
3 ...f- Discussion
- 80%
1 g()()
:::: K 1SJOII 1500
vac
~O
The transfer of the priming paradigm to the domain of ATe brought about some unexpected insights. Obviously priming works in this kind of environment . Significant differen ces between experimental conditions were found .
- 20% HOG
FL
non-.qua'
O'lb
Fig. 2: Response Time and Error Rate (medians)
The basic hypothesis was not confirmed. Response times for targets with primed conflictrelevant identical parameters were not always significantly shorter than for other pairs of aircrafts. Two reasons are to be discussed : First , the priming paradigm has been designed for 'lexical priming' . The scenario and the constraints of priming in ATC-areas are much more complex, so that effects of primes might be hidden by other influences . Second, priming usually works on pre~tt~ntive processes , wheras here prime and target mdlcate a conflict risk , which activates decisions. Therefore a delayed response is plausible.
Data preparation started from original data of the most busy days in Germany in 1991 and 1993. Data were modified according to the parameters to be varied .
3.3. Results For data analysis 24 subjects were considered . One was skipped because of obvious problems with the instructions. The median of five replications was calculated. Response times are not influenced by effects of fatigue or attentional shifts.
Response time and error rate are low, if two aircrafts are far away from each other and both are in climb/descent. This indicates, that vertical speed component in the condition 'far away ' possibly is a grouping factor for clustering.
In correspondence with hypothesis A the parameter 'vertical speed component' shows the shortest ~esponse times and a low error rate (see Fig. 2) , mdependent from lateral distance. Contrary to the hypothesis the parameter 'flightlevel ' shows the longest response times , but the lowest error r~te . The p.arameter 'ETO ' shows low response tImes , but hIgh error rates. This could be explained by the fact, that the parameter 'ETO ' was not presented directly. The last parameter, 'heading ', corresponds to the hypothesis: Response times are high, with an average error rate. Looking at the control condition where none of the parameters corresponded between prime and target , shows an ambigous result . In these cases the controllers had low response times and average error rates.
Contrary to that , if two aircrafts are on the same flightlevel (parameter 'flightlevel') or at the same time at the same place (parameter 'ETO ' ), there seems to be an effect in the condition 'nearby ' . Because both parameter together represent a 4dimensional position , they may indicate a conflict. An analysis of speed-accuracy trade-offs shows that in the condition 'nearby' controllers tend t~ be more accurate and slower. The consequences of an unsolved conflict nearby are much more 'expensive' than in the condition 'far away ': in this case correspondence of these parameters implies danger. One can draw the conclusion, that same ~ig~tlevel and same ETO in the condition 'nearby ' mdlcate a relational mental representation .
The basic hypothesis of shorter response times for primes and targets with equal paramet.ers (in contrast to the control condition 'not equal ') is not significant. An analysis of variance (2 x 4 MANOVA) of response times and error rates indicated no main effect for 'parameter' or 'lateral distance'. Only the control condition ('not equal ') was significant concerning 'lateral distance ' (p ~ .045) . So far hypothesis A and B are not confirmed .
The results of the priming experiments led to a new conception of the controllers' representation . Furthermore, references to experimental procedures that may reveal more directly the controller's strategies to select and to group information were indicated .
In a non parametric analysis of error rate , highly significant effects on the variable 'lateral distance ' are detected (p ~ .0022) . The parameter 'vertical speed component' showed less errors for the condition 'far away' than for 'nearby' , whereas for corresponding 'flightlevel' and 'ETO' significantly more errors for pairs of aircrafts far away from
4. IMPLICATIONS ON MMS-DESIGN IN ATC Automation of ATC-systems is necessary for an increase of efficiency and capacity while maintaining its safety standards. Besides integrating se331
veral different systems, a significant shift in the working methods of controllers will take place.
6. ACKNOWLEDGEMENTS The research was founded by Deutsche Forschungsgemeinschaft (DFG) in 1992 to 1994, titled "Flugsicherung als Mensch-Maschine-System" , coded Ey 4/16-1. The authors thank Klaus Eyferth, Harald Kolrep, Cornelia Niessen, Norbert Wolff and Stefan Husmann. Klaus Eyferth and Harald Kolrep contributed substantially with critical remarks. The Deutsche Flugsicherung GmbH (DFS) and Eurocontrol supported the work significantly.
Therefore a cooperative concept of future air traffic management is needed, as demanded e. g. from German ATC-authority DFS with its CATMACconcept. But the term 'cooperative' has to be interpreted not only in technological sense, but also in the way of man-machine-interaction . This cannot mean to fit man to the machine. Instead, the controllers' strategies have to be mirrored in technology. Starting point of process automation should be an understanding of mental processes of the operator. In close cooperation between computer sciences, engineering and cognitive psychology, automation concepts have to grow.
7. REFERENCES
So far, there has been little research on mental models in ATC. A clear conception of what is going on in the operator 's mind could result in a formal, computer-implemented model. This model could finally help to predict consequences of fut.ure automatization and to structure the training of air traffic controllers.
Bainbridge, L. (1987). Ironies of Automation . In: New Technology and Human Error (Rasmussen, J ., Duncan , K. &. Leplat , J., Eds.), pp. 2il-283 . Wiley, Chichester. Dubois. M. &. Gaussin, J. (1993) . How to Fit the Man-Machine Interface and Mental Models in the Operators. In: Verification and Validation of Complex Sy!tem!; Human factor! /$!ue! (Wise, J . A., Hopkin, V. D. &. Stager, P., Eds.), pp. 381397, NATO-ASI F 110. Springer, Berlin. Falzon, P. (1982) . Display Structures: Compatibility with the Operator Mental Representation and reasoning Process. Proc. 2nd Eurpean An. Conf. on Human Deci!ion Making and Manual Control, pp. 297-305. Johnson-Laird, P. N. (1983). Mental Model!. Harvard University Press, Cambridge MASS. Kluwe, R. H. &. Haider, H. (1990) . Modelle zur internen Reprasentation komplexer technischer Systeme. Sprache und Kognition, 27, pp. 619-629. Leroux. M. (1993). The Role of Verification and Validation in the Design Process of Knowledge Based Components of Air Traffic Control Systems. In: Verification and Validation of Complex Sy!tem! ; Human factor! I!!ue! (Wise, J. A., Hopkin , V . D. &. Stager, P. , Eds.), pp. 357-373, NATO-ASI F 110. Springer, Berlin. McNeill (1966). A study of word association. J. of Verbal Learning, Verbal Behaviour, 5. Norman, D. A. (1983) . Some Observations on Mental Models. In: Mental Model! (Gentner, D. &. Stevens, A. L. , Eds.), pp. 7-14 . Earlbaum, Hillsdale NJ . Rasmussen, J . (1986). Information Proces!ing and Human-Machine Interaction . Elsevier, Amsterdam. Whitfield , D. &. Jackson , A. (1982) . The Air Traffic Controller's Picture as an Example of Mental Model. In: Proc . of the IFAC Conf. on Analy!is , De!ign and Evaluation of Man-Machine Sy!tems (Johannsen, G. &. Rijnsdorp, J . E., Eds.), pp. 4552. Pergamon, London . Wilson, J . R. &. Rutherford, A. (1989). Mental Models: Theory and application in Human Factors. Human Factor!, 31, pp. 617-634.
The experiment reported shows, that a clustering of information helps the controller to identify conflicts . There are indications on a very effective relational representation in cases of conflict risk . Both kinds of representation should be supported by new display tools.
5. SUMMARY The problems of automation arising in the field of ATC were outlined. Part of a solution is the concept of mental models. A review of theories on mental models pointed out clearly the n{'('d for further research and clear definitions and t.erminology in this interdisciplinary field . A terminological suggestion for mental models in a sp<'cific domain was given together with the advantages of this definition . An experiment investigating parts of the mental model was reported . The basic priming paradigm was applied successfully to the domain of ATC, although some problems occured . The results of the experiment indicated a clustering of information in case of two aircrafts presented far away from each other. The chunking parameter is the 'vertical speed component '. In case of two aircrafts presented next to each other, there are indications for a relational representation . Implications on man-machine-system design in the domain of ATC and applications of the experimental results are discussed .
332
EXPERIMENTS ON AIR TRAFFIC CONTROLLER'S REPRESENTATION AS APPROACH TO MMS-DESIGN T. BIERWAGEN and H. HELBING Berlin University of Technology, Department of Psychology, DFG-Projekt 'Flugsiche"mg als MMS ', Dovestr. 1-5, 10587 Berlin , Germany
Abstract: Due to rising automation in the field of air traffic control, the importance of knowledge about mental information processing is discussed. Based on a brief review of the discussion on mental models, a working definition of the term is given . As part of this definition , the structure of information representation is investigated by an experiment , following the priming paradigm and carried out with experienced controllers. The experiment is reported in detail , including a discussion of the results. The results are set into the broader context of implications for man-machine-system design in air traffic control. Key Words: Man-machine systems; mental models; air-traffic control; cognitive systems; human factors; simulation; automation
1. INTRODUCTION
2. MENTAL MODELS
In the domain of air traffic control (ATC) with its highly integrated and safety-critical automated systems human factors gain importance. In this respect, designing the future man-machine-system (MMS) in ATC needs basic knowledge about mental information processing of the operators. Such knowledge allows a description of mental models of controllers. Although the meaning of the term 'mental model' seem to vary significantly, some common aspects can be stated. That is on the one hand the use of static system knowledge and on the other hand the use of actual, dynamic data and strategies to transform these data. For a description of strategies of information intake, processing and mental models the representation of data has to be investigated as a basic step .
Reviewing literature one can find, that the term 'mental model' is fairly popular in scientific community. Psychologists as well as human factors engineers use it in different domains. This again leads to very different ideas of what a mental model is. In fact , there is a need of definition on what a 'mental model' is and what indeed it is not . Going back to literature, it has to be stated, that a common definition is not available for this term. On the contrary, people use their ideosyncratic definitions. These definitions are domain-specific and user-specific. Domain-specific because cases are handled , which are implied by the domain the model was developed in. Generalizations are omitted. They are user-specific because the background of the person using this model for scientific or design purposes comes in: human factors engineers tend to use other terms and definitions than cognitive psychologists.
The report gives a brief review of the discussion of the term 'mental model ' together with the author's ideas on that topic. Based on this, an experiment regarding the structure of representation in the field of ATC as one fundamental element of a mental model is reported. Conclusions are drawn to further implications on man-machinesystem-design in future ATC-systems. The work reported was carried out in an interdisciplinary research project at Berlin University of Technology, named "En-route Controller's Representation (EnCoRe)" .
Undoubtably several influences on the term have to be considered . Mental models usually apply to very complex systems. This implies a specific kind of 'weakness' in defining exactly their way of working , their contents or their structure. Norman (1983) states, that they are incomplete, unstable, non-exclusive, unscientific and inconsistent. They often are context-sensitive as well, and a general abstraction from any context comes along with
327
losses in accuracy.
total set of schemata instantiated at the time." (cited from Wilson & Rutherford , 1989, p. 624) .
2. 1. Mental Models in th e field of MMS-design
Wilson & Rutherford (1989) propose three different kinds of models for further use in human factors research : the designer 's conceptual model , the user 's conceptual model and the user 's mental model. Whilst the conceptual models consist of ideas on a model, the user's mental model refers to a description of the user's internal representation in terms of cognitive psychology. Main differences in the definitions of mental models are pointed out by Wilson & Rutherford as well: " . .. whereas in psychology the attempt is to describe mental processes, in human factors it is the product of such processes that is of concern." (p . 622).
The term mental model seems to be based on Craik, as cited from Wilson & Rutherford (1989) : " ... the originator of the mental model notion , the Scottish psychologist Keneth Craik . . . " (p . 621) . TofHer stated the common expression , that " every person carries within his head a mental model of the world - a subjective representation of external reality" (p . 621). One of the frequently cited cognitive psychologists in the field of mental models is Johnson-Laird (1983) . He defines mental models as to " . . . enable individuals to make inferences and predictions, to understand phenomena, to decide what action to take and to control its execution ... " (p. 397) . He provides a typology of mental models with the main distinction between conceptual and physical mental models, with more widespread categories in the second level. He stresses the quality of 'running' a mental model and the relational structure of the representation. His basic idea is that of effective procedures as foundation of any mental process.
In summary no clear and commonly shared understanding of the scientific construct 'mental model' is arising at the horizon. Several problems like different aims in using mental models in psychology and human factors , psychological discussions on the format of representation (analogue versus propositional) and suitability of different concepts (,mental model ', 'schemata', 'picture', 'frame') to describe mental information processing need further investigation . A clear definition of the term 'mental model ' has to be given, otherwise interdisciplinary cooperation on related topics will continue to imply misunderstandings.
Norman (1983) introduces the distinction between a target system, a conceptual model, a scientists conceptualization and a system image. If the conceptual model (the target system representation from the designer's point of view) meets the demands of being exact, complete and consistent, the user will have a very good system image of the process. This for the user leads to an useful mental model.
2.2. Mental Models in EnCoRe Project
Due to the problems discussed above, the authors formed a (not really closed) working-definition of 'mental model' in the domain of ATC tasks. Mental models are dynamic subjective conceptions of any subject matter. They deal with data, using strategies that are controlled by creative or inferential processes. Looking at the data, there are two kinds of data to be distinguished: static data, established in long term memory, and dynamic data, handled in short term memory. In the context of ATC, static or quasi-static data can be identified as system knowledge like airspace structure, working regulations, aircraft performance data, physical constraints like the existence of radars , navigation aids , airports etc. Dynamic data are divided in low-frequency and high-frequency data. The border between them in the project EnCoRe is lined out anywhere close to 60 minutes. Low-frequency dynamic data are data like status of navigation aids or runways, temporary restricted areas, weather data etc. The most important data for the controller's actions are highfrequency data like information displayed on the radar screen, flight progress strips in use , coordination with adjacent sectors or centers, voice com-
Rouse & Morris suggest two dimensions of mental models: the level of behavioural discretion and the nature of model manipulation . These dimensions correspond to two different experimental methods. Problems with low level of behavioural discretion are to be assessed with inferential methods, whereas problems of an explicit nature of model manipulation are to be assessed by verbal methods. They " ... conclude that mental models are not necessarily computational models." (cited from Wilson & Rutherford, 1989, p . 619). Approaching from ATC, Whitfield & Jackson (1982) proposed the term 'picture' as a kind of mental model. This could be understood as an internal representation of the traffic situation and a set of rules to evaluate mentally possible actions prior to their execution. The concept is based mainly on the dynamic radar screen data used by controllers to handle traffic. In opposite to this Rumelhart ". .. described a mental model as the 328
increasing 'black box' nature of systems, the ... complexity of control . .. mean that in large part the mental models that operators develop are in the hands of designers" (p. 627). In fact, systems should be designed for the user, and therefore a deep understanding of the user's mental processes is essential for successful syst.em design. A way of describing these mental information processing is the terminology of a mental model. If one does not want to design technology for technology, but for man, mental models should be the starting point to fit technology as much to them as possible.
munication with the pilots etc. The sum of these data is contained in the term 'picture' as it is used in the domain of ATC. In contrast to static data, dynamic data is prepared and presented technically, and therefore this kind of data is subject to automation . Strategies inhering in a mental model arise from selection and, more important, from training and growing experience during work. Normally they are not easily to verbalize by controllers, but principally they are codifyable. They can be expressed in "IF ... THEN ... " rules and are put together to a more or less complete set of rules. With regard to Rasmussen (1986), they are the short-cuts in his 'decision ladder'. They can be described in terms of schemata fairly good. This fact leads to the statement, that schemata are a fundamental part of mental models.
Taking into account that, together with other aspects like a shift from (process-oriented) process management tasks towards (interface-oriented) information management tasks, mental models will lead to systems with a greater reliablility and validity and with less semantic errors produced in the design process.
Summing up different kinds of data and strategies, or schemata, does not justify a new hypothetical construct like a mental model. Another element has to be added. This element is the creative or inferential component. Based on strategies dynamic data are interpreted and - if necessary, e. g. in case of a conOict - solutions are generated. The set of rules is reviewed in search of an identical situation. If there is none, similar situations are to be identified. The rules related to this similar situation are applied and, in case of success, memorized and added to the set of rules. The core of this infering process in the domain of ATC is the ability to predict future situations. Based on data and strategies, the controllers interpret data by checking future systems states to detect conOicting aircrafts. In case of conOicts detected, they check different solutions by mentally checking the quality of the solution in order to choose the best one. This third component of a mental model is one of the main reasons why man should be kept in the loop. It has to be examined very carefully, if it is senseful to automate these processes with regard to safety and efficiency.
.I 3. PRIMING EXPERIMENT Dealing with the structure of a mental model, one point of interest is the mental representation of high-frequency dynamic data (picture). Different ways of structuring are possible. One could imagine parameters as sorting criteria as well as the (analogue) position or any kind of relation between two or more aircrafts. The relational representation shows the advantage of goal-relevance: the main goal is to detect and to avoid conOicts. If there is any sufficient conflict indicator (like distance between aircrafts), the relation of this parameter regarding pairs of aircrafts would be an unambigous indicator of a conflict, and this way of representation should be more efficient in terms of problem-solving. But speaking of more than 4-5 aircrafts under control, a complete relational representation is impossible because of the number of relations to be regarded. The experiment reported is guided by another idea: The focus are objects with attributes (parameters) attached. One ofthese parameters may indicate a conflict (see also Falzon, 1982), depending on parameters of further objects. By this, one parameter may define also a group of conflicting objects. In cognitive psychology this fact is known as chunking or clustering of information. This way of representation seems to be more efficient in terms of memory-usage.
So in contrast to Dubois & Gaussin (1993) the terms 'mental representation' and 'mental model' are not interchangeable. Rather, mental representation is a fundamental part of a mental model, which, in addition, allows for creating unique solutions.
2.9. Benefits of a Mental Model
Bringing together the advantages of efficient selection of information and efficient problem-solving suggests to use both approaches in combination. Due to the specific needs one of the representational form could be used: object-attributerepresentation for handling bulks of information,
Picking up the discussion of designer and user model, it was stated, that they have to be congruent. This convergence can be gained in two different ways: the designer can fit his ideas to the user, or VIce versa. Wilson & Rutherford state, that "the 329
and relational representation, if fast problemsolving is needed. A similar approach was suggested by Leroux (1993) .
C.1I11", FlIghtJe.,.' . CI. .red . Fllghtlenl
/ 230
-0
3.1. Method
DLH211 11713113
.
Speed
Vertical Spe.cl Indicator
e
e
e
e .
Fig. 1: Radar Target
In order to test the object-attribute structure of the picture of air traffic controllers, an ATCsimulation system was developed, and lab experiments were performed. Specific parameters of aircrafts (like 'flightlevel') in dependency oflateral distance ('far away ' from each other, or 'nearby') were varied systematically. According to the parameters 'flightlevel '1, 'vertical speed component,2, 'ETO'3 and 'heading'4, two aircrafts were in conflict or not. In analogy to the priming paradigm, reported by McNeill (1966) , any memory access on aircrafts established in the same cluster is expected to be significantly faster than for two aircrafts belonging to different clusters. Specific to the idea of priming is the activation of clusters prior to the query. The simulation was stopped, one aircraft (prime) was highlighted, then the callsign of another aircraft was presented on the cleared screen . The subjects were asked wether this second aircraft (target) was under control in the simulation at that time. The variation of parameters (e. g. both aircrafts same 'flightlevel' or not) was perfomed implicitly. The subjects were not told about this fact. Response times were recorderd and analyzed.
stem called 'EnCoRe-PLUS' (EnCoRe - programmable airspace simulation) is a PC-based simulation of a radar controller working position. ATCspecific data as well as experiment-specific data can be determined in setup-files . The system presents a normal secondary-radar display supplemented by electronical flight progress strips (callsign and route information only). This image can be varied according to experimental aims. Airspace structure data can be displayed if needed . To perform interactive scenarios with online changes in flight data, a ghost-pilot system can be connected to the system. An intercom for verbal communication between controller and pilot, including tape recording, is available. Every event, especially all kinds of interaction with the system concerning traffic flow or experimental issues, are recorded in a logfile. Computer-aided data analysis is supported. The presentation of radar targets (see Fig. 1) is similar to existing ATC-systems. The primary target is the rhombus. Attached is a label with further alphanumerical information (callsign, flightlevel, speed, on demand a vertical speed indicator and cleared flightlevel) , a velocity leader and an artificial afterglow, presenting four positions in distances of 10 seconds.
Hypothesis were as follows: A. Response time and error rate are expected to be smaller, if prime and target share the same 'flightlevel' and 'vertical speed component'. No differences are expected for the parameters 'ETO' and 'heading'. B. Response time and error rate are different depending on the lateral separation (presentation distance) of prime and target . Shorter response times and less errors are expected in case of nearby presented aircrafts.
The experiments were performed in the 'Bremen Siid Radar 2'-sector of the lower airspace in Germany. This piece of airspace was selected taking into account usual traffic structure as w~ll as the controllers' familiarity with the airspace. The experiment was non-interactive. Presentation of the relevant parameters differed : 'flightlevel' was presented as alphanumeric information, 'vertical speed component' as symbolic information (up- or down-arrow), 'heading' as symbolic information as well (map display) . 'ETO' had to be assessed by the controllers.
The experiment was carried out with 25 experienced controllers from Berlin Air Route Traffic Control Center. The average overall experience was 10 years. All participants were volunteers.
The priming queries were presented every 30-40 seconds, embedded in a running simulation. The answers (yes/no) were given by specified keys on the computer keyboard. Maximum allowed response time was 5000 ms, then simulation restarted. Two different kinds of distractor-requests were included in the experiment to prevent subjects from simply learning callsigns. Questionaires were used successfully to show the validity of distractor tasks.
3.2. Experimental Simulation System
Due to the demands of performing cognitivepsychological experiments in the domain of ATC, a new simulation facility was build up. The sylaltitude, in steps of 100 ft climb or deacent 3 Estimated Time Over the next Fix {given in degree from O· to 359· 2 indicating
330
each other were found (p ::; .01) . 2100 -
-- - -
--- .. _-
-
2000
-
100%
3 ...f- Discussion
- 80%
1 g()()
:::: K 1SJOII 1500
vac
~O
The transfer of the priming paradigm to the domain of ATe brought about some unexpected insights. Obviously priming works in this kind of environment . Significant differen ces between experimental conditions were found .
- 20% HOG
FL
non-.qua'
O'lb
Fig. 2: Response Time and Error Rate (medians)
The basic hypothesis was not confirmed. Response times for targets with primed conflictrelevant identical parameters were not always significantly shorter than for other pairs of aircrafts. Two reasons are to be discussed : First , the priming paradigm has been designed for 'lexical priming' . The scenario and the constraints of priming in ATC-areas are much more complex, so that effects of primes might be hidden by other influences . Second, priming usually works on pre~tt~ntive processes , wheras here prime and target mdlcate a conflict risk , which activates decisions. Therefore a delayed response is plausible.
Data preparation started from original data of the most busy days in Germany in 1991 and 1993. Data were modified according to the parameters to be varied .
3.3. Results For data analysis 24 subjects were considered . One was skipped because of obvious problems with the instructions. The median of five replications was calculated. Response times are not influenced by effects of fatigue or attentional shifts.
Response time and error rate are low, if two aircrafts are far away from each other and both are in climb/descent. This indicates, that vertical speed component in the condition 'far away ' possibly is a grouping factor for clustering.
In correspondence with hypothesis A the parameter 'vertical speed component' shows the shortest ~esponse times and a low error rate (see Fig. 2) , mdependent from lateral distance. Contrary to the hypothesis the parameter 'flightlevel ' shows the longest response times , but the lowest error r~te . The p.arameter 'ETO ' shows low response tImes , but hIgh error rates. This could be explained by the fact, that the parameter 'ETO ' was not presented directly. The last parameter, 'heading ', corresponds to the hypothesis: Response times are high, with an average error rate. Looking at the control condition where none of the parameters corresponded between prime and target , shows an ambigous result . In these cases the controllers had low response times and average error rates.
Contrary to that , if two aircrafts are on the same flightlevel (parameter 'flightlevel') or at the same time at the same place (parameter 'ETO ' ), there seems to be an effect in the condition 'nearby ' . Because both parameter together represent a 4dimensional position , they may indicate a conflict. An analysis of speed-accuracy trade-offs shows that in the condition 'nearby' controllers tend t~ be more accurate and slower. The consequences of an unsolved conflict nearby are much more 'expensive' than in the condition 'far away ': in this case correspondence of these parameters implies danger. One can draw the conclusion, that same ~ig~tlevel and same ETO in the condition 'nearby ' mdlcate a relational mental representation .
The basic hypothesis of shorter response times for primes and targets with equal paramet.ers (in contrast to the control condition 'not equal ') is not significant. An analysis of variance (2 x 4 MANOVA) of response times and error rates indicated no main effect for 'parameter' or 'lateral distance'. Only the control condition ('not equal ') was significant concerning 'lateral distance ' (p ~ .045) . So far hypothesis A and B are not confirmed .
The results of the priming experiments led to a new conception of the controllers' representation . Furthermore, references to experimental procedures that may reveal more directly the controller's strategies to select and to group information were indicated .
In a non parametric analysis of error rate , highly significant effects on the variable 'lateral distance ' are detected (p ~ .0022) . The parameter 'vertical speed component' showed less errors for the condition 'far away' than for 'nearby' , whereas for corresponding 'flightlevel' and 'ETO' significantly more errors for pairs of aircrafts far away from
4. IMPLICATIONS ON MMS-DESIGN IN ATC Automation of ATC-systems is necessary for an increase of efficiency and capacity while maintaining its safety standards. Besides integrating se331
veral different systems, a significant shift in the working methods of controllers will take place.
6. ACKNOWLEDGEMENTS The research was founded by Deutsche Forschungsgemeinschaft (DFG) in 1992 to 1994, titled "Flugsicherung als Mensch-Maschine-System" , coded Ey 4/16-1. The authors thank Klaus Eyferth, Harald Kolrep, Cornelia Niessen, Norbert Wolff and Stefan Husmann. Klaus Eyferth and Harald Kolrep contributed substantially with critical remarks. The Deutsche Flugsicherung GmbH (DFS) and Eurocontrol supported the work significantly.
Therefore a cooperative concept of future air traffic management is needed, as demanded e. g. from German ATC-authority DFS with its CATMACconcept. But the term 'cooperative' has to be interpreted not only in technological sense, but also in the way of man-machine-interaction . This cannot mean to fit man to the machine. Instead, the controllers' strategies have to be mirrored in technology. Starting point of process automation should be an understanding of mental processes of the operator. In close cooperation between computer sciences, engineering and cognitive psychology, automation concepts have to grow.
7. REFERENCES
So far, there has been little research on mental models in ATC. A clear conception of what is going on in the operator 's mind could result in a formal, computer-implemented model. This model could finally help to predict consequences of fut.ure automatization and to structure the training of air traffic controllers.
Bainbridge, L. (1987). Ironies of Automation . In: New Technology and Human Error (Rasmussen, J ., Duncan , K. &. Leplat , J., Eds.), pp. 2il-283 . Wiley, Chichester. Dubois. M. &. Gaussin, J. (1993) . How to Fit the Man-Machine Interface and Mental Models in the Operators. In: Verification and Validation of Complex Sy!tem!; Human factor! /$!ue! (Wise, J . A., Hopkin, V. D. &. Stager, P., Eds.), pp. 381397, NATO-ASI F 110. Springer, Berlin. Falzon, P. (1982) . Display Structures: Compatibility with the Operator Mental Representation and reasoning Process. Proc. 2nd Eurpean An. Conf. on Human Deci!ion Making and Manual Control, pp. 297-305. Johnson-Laird, P. N. (1983). Mental Model!. Harvard University Press, Cambridge MASS. Kluwe, R. H. &. Haider, H. (1990) . Modelle zur internen Reprasentation komplexer technischer Systeme. Sprache und Kognition, 27, pp. 619-629. Leroux. M. (1993). The Role of Verification and Validation in the Design Process of Knowledge Based Components of Air Traffic Control Systems. In: Verification and Validation of Complex Sy!tem! ; Human factor! I!!ue! (Wise, J. A., Hopkin , V . D. &. Stager, P. , Eds.), pp. 357-373, NATO-ASI F 110. Springer, Berlin. McNeill (1966). A study of word association. J. of Verbal Learning, Verbal Behaviour, 5. Norman, D. A. (1983) . Some Observations on Mental Models. In: Mental Model! (Gentner, D. &. Stevens, A. L. , Eds.), pp. 7-14 . Earlbaum, Hillsdale NJ . Rasmussen, J . (1986). Information Proces!ing and Human-Machine Interaction . Elsevier, Amsterdam. Whitfield , D. &. Jackson , A. (1982) . The Air Traffic Controller's Picture as an Example of Mental Model. In: Proc . of the IFAC Conf. on Analy!is , De!ign and Evaluation of Man-Machine Sy!tems (Johannsen, G. &. Rijnsdorp, J . E., Eds.), pp. 4552. Pergamon, London . Wilson, J . R. &. Rutherford, A. (1989). Mental Models: Theory and application in Human Factors. Human Factor!, 31, pp. 617-634.
The experiment reported shows, that a clustering of information helps the controller to identify conflicts . There are indications on a very effective relational representation in cases of conflict risk . Both kinds of representation should be supported by new display tools.
5. SUMMARY The problems of automation arising in the field of ATC were outlined. Part of a solution is the concept of mental models. A review of theories on mental models pointed out clearly the n{'('d for further research and clear definitions and t.erminology in this interdisciplinary field . A terminological suggestion for mental models in a sp<'cific domain was given together with the advantages of this definition . An experiment investigating parts of the mental model was reported . The basic priming paradigm was applied successfully to the domain of ATC, although some problems occured . The results of the experiment indicated a clustering of information in case of two aircrafts presented far away from each other. The chunking parameter is the 'vertical speed component '. In case of two aircrafts presented next to each other, there are indications for a relational representation . Implications on man-machine-system design in the domain of ATC and applications of the experimental results are discussed .
332