- Email: [email protected]
A predictive man-machine environment for training and evaluating control operators
EngngApplic.Arttf. lntell. Vol.5, No. 5, pp. 441-450, 1992
0952-1976/92$5.00+ 0.00 Copyright© 1992PergamonPressLtd
Printed in Great Britain. All rights reserved
Contributed Paper
A Predictive Man-Machine Environment for Training and Evaluating Control Operators NICHOLAS V. FINDLER Arizona State University, Tempe CONGYUN LUO Arizona State University, Tempe MIN-HUI KUO Arizona State University, Tempe
After a brief discussion of decision support systems and the problems of Air Traffic Control (A TC), the design and implementation of a Predictive Man-Machine Environment (PMME) are described that can in general be used in the automatic training and evaluation of control operators, as well as serve as the basis of their routine planning and decision-making activity. PMME works with two connected workstations, one of which displays the essential features of the Current World and the other those of the Extrapolated World. The latter predicts the future consequences of the control operator's tentative decisions made in responding to the Current World. If the operator is satisfied with the consequences, he finalizes his decisions; otherwise, he modifies the tentative decision and, time permitting, goes through this cycle as many times as necessary. A simulated air traffic control environment for the PMME has also been implemented and two sets of experiments in it performed with tasks of equal complexity--one with and the other without the extrapolation facility. A statistically significant improvement has been shown in the A TC operations that rely on the extrapolation facility. It can be said in general that the PMME increases the effectiveness and efficiency of human judgmental processes by extending their range, capabilities and speed. Keywords: Man-machine environment, automatic training and evaluation of control operators. • Delicate judgements are needed concerning which environmental and control variables are relevant and what their current values are. The task of decision making under uncertainty refers to the fact that the currently available knowledge about the world is partial, imprecise and possibly even inconsistent. (See Chaps 1-3 of Ref. 1).
INTRODUCTION
The increasingly high rate of automation of a multitude of operations has raised the demands on the human element left in the control loop. The difficulties of the human controller are due to many factors, the most frequent of which are the following: • The knowledge base needed for a decision is too large for a human to access or even to make conceptual use of. • To arrive at a satisfactory solution requires a large amount of processing of the available information. • There is a time pressure to perform the computations (because the environment may continually change) and/or to obtain the solution (because certain actions must be performed before a certain point of time).
It is, therefore, important to devise techniques that would enhance the quality and shorten the length of the training process of control operators. It would also be necessary to implement some objective measures for evaluating their performance and use these as information feedback for self-correcting control actions. ON DECISION SUPPORT SYSTEMS
Correspondence should be sent to: Professor N. V. Findler, Department of Computer Science and Engineering, and Artificial Intelligence Laboratory, Arizona State University Tempe, AZ 85287-5406, U.S.A. 441
Decision support systems 2-5 (DSSs) are first discussed before the actual work accomplished is described. A DSS is a computer-based information system that helps a user make decisions by providing him with the necessary information in an easily understandable
442
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
form. The User is situated in an interactive environment and usually faces a menu-driven front-end. The system accesses a database to locate the necessary data, applies appropriate mathematical and/or statistical models, and displays the information needed at the user's terminal. Based on the information provided, the user can explore different alternatives in order to arrive at a decision. A more formal definition of DSSs is: 5
Decision support systems allow the decision maker to combine personal judgment with computer output in a user-machine process. Such systems are capable of soloing various types of problems (structured, semistructured, and unstructured) and offer query capabilities to provide information requested. As deemed appropriate, they use quantitative models as well as database elements for problem solving. From. an enhanced perspective, decision support systems are an integral part of the decision maker's approach to problem solving, which stress a broad view of the organization by employing the "management by perception" principle. Although the main objective of a DSS is to help in selecting correct decisions, it offers far more possibilities. The various activities involved include problem definition and structuring; data collection, filtering and fusion; choice generation and selection; prioritization and routing; planning and deciding; explaining and forecasting. These steps may be taken before, during or after the decision-making cycle. One can conclude that the important DSS factors are accessibility, flexibility, facilitation, learning, interaction, and ease of use. A few additional issues are noted: • Some DSSs focus on supporting decision making for effective planning and control in terms of finding and solving managerial problems at all levels. • DSSs give support to the user but usually do not replace him. Humans' qualitative models and other non-tangible considerations enhance the whole decision-making process. • DSSs encompass not only the user and the computer but also various models and the languages and databases of modelling and simulation--which all contribute to the decision-making process. • A DSS is an important tool when a decisionmaking process references some uncertain factors or involves many conditions, constraints, and consequences to be considered simultaneously. ON AIR TRAFFIC CONTROL Air traffic control (ATC) has evolved from flag waving, when the major concern was to avoid collision with the ground and trees, to modern computer-based
systems aiding the control of thousands of commercial, military and small private planes. 6-9 Current systems have the capabilities of detecting the airplane's ID, speed and location, and predicting potential traffic conflicts before their occurrence. Future generations of ATC systems will be built on space and satellite technology. Communication, navigation, and surveillance will be accomplished from earth orbit and will cover the entire globe. ATC is a complex topic and it is not possible to give even a brief outline of its major aspects in this paper. It suffices to say that A T C is performed in areas designated to contain instrument-flight-rules operations during portions of the terminal operation and while transiting between the terminal and the en route environments. (The other operations are under visualflight-rules 8 when the pilot, and not the controller, is responsible for the adherence to the rules.) Area Navigation Routes include two types of airways. The low-altitude airways are designated from 1200 ft above ground level up to 17,999 ft. The high-altitude or jet airways are designated at or above 18,000 ft. Separation is the essence of air traffic safety. It is provided by establishing approved minimum longitudinal, lateral, and vertical distances between adjacent aircraft. Longitudinal separation is the spacing between two aircraft at the same altitude and on the same route. Lateral separation is also between aircraft at the same altitude but on different routes. On low-altitude airways (less than 18,000ft), the minimum horizontal separation is 3 miles if the airplane is within 40 miles of the radar antenna and 5 miles if it is outside the 40 mile range. In the high-altitude case (above 18,000 ft), the separation ranges from 5 to 10 miles according to various factors such as speed and weather conditions. Vertical separation is established by assignment to different altitudes or flight levels. The minimum vertical separation is 1000ft for those airplanes whose altitudes are less than 29,000ft, and 2000ft above 29,000 ft. Finally, the separation between airplanes and obstructions is defined as follows: if the airplane is at least 3 miles from an obstruction and at least 1000 ft above it, a minimum 3-mile distance from it must be maintained until the obstruction has been passed. If the airplane is less than 3 miles from the obstruction and its altitude is at least 1000 ft above it, the minimum lateral distance of 3 miles must be reached and maintained until it passes the obstruction. An event may be characterized as the product of interactions, among aircraft, airspace, facilities, and ATC operations. There are several types of events. The types of events that are associated with a single aircraft include conflict, violation, clearance request, and flight status. The event type associated with multiple aircraft is conflict. A conflict is defined as a situation in which two or more planes get too close to each other, or one plane gets too close to geographical obstructions. A violation involves only one aircraft that violates some
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
REAL.WORLD ..,Instructions
J"
COMPUTER
.~,,,,o~/
'
\
o
p cup,,~,rr
0 ~
Fig. 1. The predictive man-machine environment for decision making and planning. flight parameters, such as the landing angle. Takeoff clearance, approach clearance, and landing clearance are clearance request events. Initial contact, filing flight plans and missing the approach are events concerning the flight status. Activities are defined as top-level sequences of m a n machine interactions which respond to a group of closely related events. Accordingly, a typical flight goes through the following terminal and en route operational activities: a jet is waiting at the end of the runway and the captain informs the departure control that he is ready for takeoff; the controller replies from the control tower by issuing the takeoff clearance and departure instructions; the jet leaves the ground and an electronically generated "data tag" (containing information about the jet's flight number, ground speed and altitude) is shown on the radar scope; as the flight approaches the boundary of the en route control sector, the terminal controller hands off control to the en route controller; when the flight is close to its destination, the pilot asks for a clearance for landing; the flight holds in a waiting pattern until the landing clearance is received; finally, the data tag disappears from the radar scope after the plane has landed. The present authors have been interested for some years in applying AI methodology to this intellectually and economically challenging area. 1.10-15
443
Environment) update the representation of the affected objects in the computer. The updating can be controlled in three different ways: • at regular points of time, following a userdefined frequency (time-driven updating); • whenever one of the set of user-specified events takes place, an update is triggered (event-driven updating); • on user's command, regardless of how long a time has elapsed since the last update or whether certain conditions in the environment are satisfied (user-driven updating). The user makes tentative decisions on the basis of the Current World displayed, say, at time to and transmits them to the computer. He then specifies a time point in the future, tl, at which he wants to see the expected consequences of the tentative decisions. This extrapolation of the status of the world is c o m p u t e d - - o n the basis of the computer-resident m o d e l - - t h r o u g h periodic (user-specified) time increments up to the final time point ft. However, if during the time period between to and tl, a conflict situation occurs, as defined by the user again, the calculation is interrupted, and the conflict situation with the corresponding time i s displayed on the second unit called Extrapolated World. If no conflict has occurred between the time points in question, the relevant features of the world will reflect the permissible consequences of the tentative decisions on the second screen. If the user is satisfied with the status of the resulting world, he finalizes the tentative decisions and informs the computer accordingly. Otherwise, he makes a different set of tentative decisions and goes through the above cycle as many times as necessary, time permitting. THE SIMULATED AIR TRAFFIC CONTROL ENVIRONMENT
First, some terminology needs to be established. To set up any simulated environment for the PMME, one has to define a set of objects inhabiting it. The conditions that affect the objects can be either constant over time (e.g. the location of a mountain range) or THE DESIGN OF THE P M M E subject to change (e.g. current weather). Objects can also affect each other (e.g. two planes with an intersectRelying on the authors' previous work concerning a ing flight path). An object passes through different simpler environment* (see Refs 12 and 16, and Chap. 8 phases in its interaction with the environment. A phase of Ref. 1), the design of the P M M E was done according can be considered as a distinct stage in the lifetime of to the flow of information and control shown in Fig. 1. the o b j e c t - - t h e period of its sojourn in the environThe Current World displays the essential features of ment. During a phase, the object may perform certain the Real World. Events from the Real World (or, as in functions toward achieving a subgoal (or, finally, a this case, from the Simulated Air Traffic Control goal). This achievement of the subgoal places the * In this early work, the user was not able to define the ATC object into another phase (N.B. a plane passes through environment--the location and the number of airports, airways, eight major phases in its flight: preparation for take-off, radio beacons, obstructions, etc. were fixed. Also, the environ- take-off, climb, cruise, descent, approach, landing and ment included only en route operations and no near-terminal ones. Consequently, the performance measures of the control operations taxiing). The overall goal is, of course, the safe and did not cover those that are relevant near an airport. timely flight between the origin and the destination.
444
N I C H O L A S V. F I N D L E R et al.: A P R E D I C T I V E M A N - M A C H I N E E N V I R O N M E N T
Associated with each object is a set of attributes which determine the constant and varying properties of the object--including the phase it is in. The number and the type of the attributes, in general, depend on the domain of application. The user must define the functions that can alter the values of the attributes every time a user-specified incremental parameter (usually time) value elapses. There is a trade-off here between "too large" and "too small" a triggering value for the incremental parameter. In the former case, more computation has to be performed but there is a lesser chance that a critical conflict can get by unnoticed in between two attribute updatings. In the latter case, the opposite is true. It is important to point out that the attributes of the objects must change independently from each other in the Current World and in the Extrapolated World. This separation of attribute scopes is a basic requirement when causal relations between decisions and consequences are sought. A large knowledge base needs to be prepared for any non-trivial domain of application. For example, in case of the Simulated Air Traffic Control Environment (SATCE), it must include • the physical characteristics of several airports (length and location of airstrips; height and location of potential obstructions, such as mountains and towers; direction of radio beacons; etc.) • the physical characteristics of the participating planes (symbolic notation for the manufacture/ model, maximum initial landing speed, approach angle, approach speed, descent angle, descent speed, maximum and average cruising speed, climb angle, take-off speed, turning radius, fuel capacity, fuel consumption in different phases, etc.); • dictionary and syntax needed in communicating between pilots and the air traffic controller; • a program to interpret the above communication; • links between controller instructions and the functions that make attribute value changes accordingly. The literature on simulation techniques approaches is, of course, very extensive. 17'i8
and
THE IMPLEMENTATION OF THE PMME
The PMME runs on two workstations, each with a separate control unit, user interface, communication unit and graphics unit. Workstation I has been assigned to display the Current World containing simulated air traffic scenarios. Workstation II displays the Extrapolated World obtained through the Extrapolator which predicts future situations. Figure 2 illustrates the organizational structure of the different components.
User Interface I I
I
Simulator
I
I
I
Graphics Unit I
]
I Graphics Unit II I
Workstation
I
Extrapolator
Workstation
[
II
Fig. 2. The organization structure of the PMME.
The Control Unit I is the main driver of Workstation I. It controls the operations of the User Interface I, the Air Traffic Simulator, the Communication Unit I and the Graphics Display Unit I. The User Interface I is the interface between the user and the Current World. It is menu-driven and the user can define tt.rough it the geographical properties of the control sector, input the simulation parameters (the length of the simulation run and the frequency of updating the world image), request statistical reports, and interact in general concerning his decision-making activity. The Air Traffic Simulator* generates the images of each aircraft in the control sector and updates the dynamic information of these aircraft. (Static information about the aircraft includes its model, flight number and destination. Dynamic information consists of its speed, altitude, location, flight direction, available fuel and flight status.) The Communication Unit I handles the communication between the two workstations. When the Extrapolator is enabled, it writes the Current World information to a disk file and sends an activating signal to the Extrapolator. Also, it receives and processes the user's final decision from Workstation II. The Graphics Display Unit 1 displays the Current World as a horizontal or vertical projection (see Figs 3 and 4). The user has further options to choose from, such as showing the trace of an airplane in either projection (see Fig. 5), zooming into or out of an area of interest, changing the area of interest (see Fig. 6), inquiring about a certain airplane's data, issuing commands to an airplane, as well as deciding whether to do extrapolation. The Control Unit H is the main driver of Workstation II. It governs the operations in it between the User Interface II, the Extrapolator, the Communication Unit II, and the Graphics Display Unit II. The User Interface H is the interface between the user and the Extrapolated World. It acts as a front-end, * If this arrangement is employed in actual ATC operations, this unit is to be replaced by digitized input from radar sensors.
NICHOLAS V. FINDLER et
al.:
A PREDICTIVE MAN-MACHINE ENVIRONMENT
445
I J~] FileView-DCL Command H-display V-display Change area Zoom Planes info Finalize opt Movement tracer Go
,408 9 30 7
Airplane aa1615 and Airplane sw408 conflict
nw1686 dc9 35 235
III I/
I
Stop Task
I
Fig. 3. The current world, horizontal projection.
FileView-DCL Command
[]J~'-I
17:27:37
45000.0 / aa1615
nw1686 dc9 35 235
/ 7 2 7 30
220
H-display V-display Change area Zoom Planes onfo Issue command Time ( I'o"~/0ff ) Movement tracer Extrapolation opt Go nw383 17:35 0 scottsdal aa787 17:37 0 phoenix
J
\
aa787 737 0
nw383
o \
/dc9o
"~
J Hold Output
~
0 Feet
0
I Fig. 4. The current world, vertical projection.
EMI 5:5-C
J
Stop Task
I
'~
446
N I C H O L A S V. F I N D L E R et al.: A P R E D I C T I V E M A N - M A C H I N E E N V I R O N M E N T
i~
FileView-DCL Command
I~1~-----II
1:0:20
45000.0
H-display V-display Change area Zoom Planes onfo Issue command Time (Ionl/off) Movement tracer Extrapolation opt Go
Tracing vertical Movement of aa787 •
•
Stop tracing? Please enter y or n
/
aa787 737 15 3O0
0 Feet
I .o,oo=u, I
I s'o0Task I
Fig. 5. The trace of an airplane's flight path, vertical projection.
H-display V-display Change area Zoom Planes info Issue command Time (on/I'~'1) Movement tracer Extrapolation opt Go
,408 9 30 7
LLC (0.0 0.0) (40 40)
W ~oo 55
"Allarea Enter New LLC & W d when done Input = (x, y, w) I Fig. 6. Changing the area of interest, horizontal projection.
Stop Task
I
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT allowing the user to make tentative and final decisions, and to specify the extrapolation parameters (the length and time increments of the extrapolation process). The Extrapolator calculates the situation in the Extrapolated World, starting from the situation in the Current World. It also detects conflicts and warns the user about them through the User Interface II. The Communication Unit H receives messages from Workstation I, such as "the geographical properties file is ready to be read" and "start the extrapolation". Once available, it writes the controller's final decision to a disk file and informs Workstation I thereof. The Graphics Display Unit H, displays the Extrapolated World as a horizontal (see Fig. 7) or vertical projection. As with the Graphics Display Unit I, the user is offered the options of showing the trace of an airplane in either projection, zooming into and out of an area of interest, changing the area of interest, and inquiring about a specific airplane. Finally, there is an independent computational component not included in the system diagram on Fig. 2, the Eoaluation Module. It serves to measure and compare the efficiency and the effectiveness of the controller with and without the Extrapolator facility. The module contains the specification of experiments and a set of metrics used in the evaluation. The simulated radar scopes are patterned after those in use at the T R A C O N centers of the Federal Aviation Administration. The displays present a map which is
447
approx 100 miles in radius and contain a "control section", an area for which the present controller is responsible. The map is centered over the location of the radar dish that generates the display. The circular grid lines are shown five miles apart. The location of the airways, radio beacons, mountain peaks and other relevant landmarks are also indicated in a standard symbolic format. In real-life, A T C displays produce only a blip on the screen for an individual aircraft while a so-called transponder on the aircraft continually sends messages to superimpose additional display information as needed. This information on the aircraft's identification, ground speed, altitude and other data is presented in a block connected with a line to the exact location of the aircraft on the screen. The display in our work simulates this final result. The simulated environment has been kept as realistic as possible but the displays are also enhanced with certain additional facilities that are not yet available with real-life radar scopes. Summing these up, use has been made of the color capabih'ties of the display units to make the identification of various groups of symbols easier. The user is also given the ability to center the display over any part of the map, and to zoom it in and out as necessary to obtain a more-detailed view of potential problem areas. There is additionally a system clock, a text window for presenting the status of the system and the airplanes, and a menu display to make
5 :)
H-display V-display Change area Zoom Planes info Finalize opt Movement tracer Go
,408 9 30 7
Airplane aa 1615 and Airplane sw408 conflict
lw1686 :1c9 35 .>35
I
Stop Task
Fig. 7. The extrapolated world, horizontal projection.
I
448
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
P M M E more powerful and flexible. The menu system allows the user to select or change any of the simulation parameters which are not associated with the behavior of the airplanes. Through the menu system, one can vary the time parameters of the simulation, change the color of a group of symbols, or alter the center and scale of the map, as noted above. The next innovation concerning a second display mode has also been provided, wherein the vertical profile of an airplane's movement can be viewed. The display in this mode is centered on the target aircraft and the viewing plane is rotated so that the target is always moving perpendicular to the controller's line of sight. Although the current A T C radar scopes do not yet have this capability either, the facility was provided to enhance the man-machine interface in certain critical situations. Also, the user can request in this system a trace o f an aircraft's trajectory in either projection. The internal structure of the program handles each aircraft as an independent object, complete with its own performance and behavioral characteristics. ~9'2° The airplane objects carry information about their flight plan, commands previously issued to them by the controller, a full description of their current status, and a pointer to a generic block of information (an information structure of the frame type) containing the performance characteristics of a class of airplanes (e.g. DC-9, Boeing 747, etc.). This way, any environment can be described in a high-level manner, in terms of the properties of the aircraft in that area. The data structures used allow the user to manipulate system time easily. To update the environment to a new time, a set of operations has to be performed on the representation of each airplane. Extrapolation is accomplished by taking the current state of the environment and then repeatedly updating it by the time increment parameter until the specified time is reached and the display is triggered. (In this, it is assumed that no conflict has occurred. Otherwise, the extrapolation process is interrupted, warning sound and light signals are given, and the conflict in question is displayed at that time point.) Further, the environments can be saved periodically to allow the controller the possibility of an "instant replay". Another novel addition to the system is a versatile reminder system. The controller is able to describe a variety of anticipated situations using the menu system. When the program senses that a predefined situation is about to occur, it alerts the controller with a tone and a message with reference to the triggering condition. This facility may save the controller valuable time by providing retrieval cues for previously generated plans. In setting up the PMME, the system allows the user to define the geographical properties of the environment, such as the area of the control sector, the location of the airports, waiting stacks, airways, ranges of the mountains, towers, and radio beacons. The user can also adjust the level of task difficulty--
that is, the range* of traffic density (upper and lower limits of the number of aircraft) and the ratio between simulated and real time (to suit the needs of student controllers). The geographical properties can be stored in a file to be retrieved at later training sessions. One can thus evaluate the improvement in the user's performance while the level of difficulty is kept constant. THE EXPERIMENTS AND THEIR EVALUATION
In order to evaluate the P M M E concept, two groups of experiments were designed and performed. The objective was to compare the efficiency and effectiveness of human planning and decision making in a complex, real-time problem d o m a i n - - w i t h and without the aid of the interactive extrapolation facility. The task specified for the subjects was en route and near-terminal A T C under light, moderate and heavy traffic conditions. The performance of the subjects was automatically evaluated in each experiment. The primary quality measures were the number of airspace conflicts and violations which occurred during the session. The secondary measures related to the efficiency with which the aircraft were routed through the subject's control sector. These include the following: • The number of instructions issued--the smaller it is the better the control strategy is, other factors being equal. (Reason: pilots prefer fewer, effective and efficient instructions.) • The number of extrapolations r e q u e s t e d - same as above. (Reason: extrapolation takes time away from concentrating on the situation, particularly under time-stressed conditions.) • The number of conflicts--for all aircraft in a conflict situation during the session, updated at the end of every internal time increment. • The average "off-distance"--the difference in miles between the lengths of each aircraft's requested (usually straight-line) route and the sum of the distance actually travelled through the air traffic controller's sector plus the distance from the point it left the sector to the plane's destination. • The average "climb/descend distance" per aircraft--the total distance in feet each aircraft was told to climb and descend during its flight through the sector, plus the difference between its requested altitude and its altitude upon leaving the sector. • The average delay of departures and arrivals per aircraft--difference between actual and scheduled time points. • The ratio between the simulated and real-time. * The actual number and the location of the planes participating in the experiment at hand are within the ranges specified and are determined by means of random number generators.
NICHOLAS V. FINDLER et
al.:
A PREDICTIVE MAN-MACHINE ENVIRONMENT
449
Table 1. Summaryof experimental results. With extrapolation Air traffic density
Light
Average no. of planes Average no. of commands Average no. of extrapolations Average no. of conflicts Average no. of violations Average off-distance Average climb/descent-distance Average departure/arrival delay Average simulated/real time
10.9 8.5 5.0 1.9 1.4 0.13 0.27 1.0 1.21
Moderate H e a v y L i g h t Moderate Heavy 15.2 11.2 7.5 4.0 0.8 0.10 0.12 1.4 1.39
The faster this user-selected number approached one, the more effective the controller's learning process was. As noted before, the program randomly generated the number of participating aircraft, their flight plans and certain other characteristics. For the benefit of objective comparisons, the task complexity was the same for the corresponding experiments--30 with and 30 without the extrapolation facility. Each session lasted 20 min of simulated time. The time increment was 60 s in the Current World and 30 s in the Extrapolated World. Each group of 30 sessions was further divided into three sets with light, moderate and heavy air traffic d e n s i t y - - a b o u t 10, 15 and 20 airplanes, respectively. The results are summarized in Table 1 and the effect of the extrapolation is shown in Table 2. The statistical analysis of the significance of these figures is summarized in the Appendix. CONCLUSIONS The experimental results clearly indicate that in the task environment chosen, the P M M E is an effective teaching tool, and the subjects' performance with the extrapolation facility was significantly better than without it. (There is also an extraneous and small contributory factor to note. Since the two display units are connected to the same hard disk memory, the Current World environment runs somewhat faster in real-time when the extrapolator is disabled, which makes the controller's task a little more difficult. However, the Table 2. The changes with extrapolation Air traffic density
Light
Average no. of planes Average no. of commands Average no. of extrapolations Average no. of conflicts Average no. of violations Average off-distance Average climb/descent-distance Average departure/arrival delay Average simulation/real time
-+ 12% -- 41% - 36% - 32% - 31% - 17% + 32%
Moderate Heavy -+ 8% -- 57% - 60% - 62% - 71% - 26% + 30%
Without extrapolation
-+ 13% -- 28% - 33% - 29% - 40% - 9% + 42%
20.5 16.1 10.2 9.1 2.6 0.22 0.31 2.1 2.42
10.3 7.6 0.0 3.2 2.2 0.19 0.39 1.2 0.82
15.4 10.4 0.0 9.4 2.0 0.26 0.41 1.9
0.97
20.5 14.3 0.0 12.7 3.9 0.31 0.52 2.3 1.41
difference in speed was small enough to state unequivocally that an accurate evaluation of the usefulness of the P M M E has been obtained.) The results were especially impressive in reducing the number of conflicts when the air traffic density is moderate (57% less conflicts and 60% less violations with the extrapolator enabled). In general, the number of conflicts, violations, average off-distance, average climb/descend distance, and average arrival/departure delay were all reduced significantly. However, the number of commands used increased to some extent. This was expected because some of the commands were made to avoid the potential conflicts warned about by the extrapolator. The simulated and real-time ratios also increased when the extrapolator was used, particularly under heavy traffic conditions. (This, to a large extent, was due to the time-consuming w a y - - v i a the k e y b o a r d - - i n which the subjects communicated with the simulated pilots). Although the use of the extrapolation facility improved the subjects' performance, it was found that the facility can also be abused and lead to undesirable outcomes as follows: • After having gained some experience in the use of the extrapolator, it is easy to become complacent about the control task when no conflicts occur within the time interval between the current and the extrapolated time. • If extrapolations are continually requested for short time intervals, the major part of one's attention is drawn away from the Current World to the Extrapolated W o r l d - - a g a i n resulting in a possible oversight of conflicts in the immediate future. • In "crisis" situations in which many decisions must be made in a short period of time (e.g. several conflicts are imminent), extrapolation can often be more of a hindrance than help. When facing such a confusing situation, a seemingly safe thing to do is to run the extrapolation into the near future. However, this action may result in wasting time during which effective directives should have been issued to resolve the conflicts.
450
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
In spite of the above caveats, PMME is a useful, general-purpose environment for real-time decision making and planning. It should, however, be relied upon only as an auxiliary source of information--the planner and decision maker should always focus his attention on the unfolding events in the Current World. The PMME can be improved in several of its aspects. First, low-level mouse programming utilities would strengthen the graphics user interface. The student controller should be able to select different viewing areas and retrieve information about an aircraft via mouse positioning instead of keyboard typing. Second, SATCE should be able to generate more-realistic scenarios, such as those based on Poisson rather than uniform distributions and incorporating also unexpected events (emergency landing, hijacking attempts, sudden changes in the weather, etc.). Summing up, the PMME can in general be used to improve trainees' analytical skills, long-range planning ability and diagnostic capabilities, so that they can balance objectives, associate methods with results, convert objectives--specified as the desired state of the environment--to dynamic process models, utilize resources to minimize risks, and contrast costs with benefits. Several other applications seem also possible, both for automated training and evaluation as well as for routine control operations. These range from investment portfolio management to command and control tasks in different conflict situations. Acknowledgements--The authors are grateful for many useful ideas obtained from T. W. Bickmore, R. F. Cromp and N. Mazur, and for the subjects' efforts and enthusiasm. The DEC Workstation 3100s used for this project were obtained through the Digital Equipment Corporation External Research Grant No. 158 and 774.
REFERENCES 1. Findler N. V. Constributions to a Computer-Based Theory of Strategies. Springer, New York (1990). 2. Andriole S. J. Handbook of Decision Support Systems. Tab Books, Blue Ridge Summit, PA (1989). 3. Hawgood J. and Humphreys P. Effective Decision Support Systems. Gower Technical Press, Aldershot (1987). 4. Ross S. C., Penlesky R. J. and Doney L. D. Developing and Using Decision Support Applications. West Publishing Company, St Paul, MN (1988). 5. Thierauf R. J. User-Oriented Decision Support Systems Accent on Problem Finding. Prentice Hall, Englewood Clifs, NJ (1988).
6. Brenlove M. S. The Air Traffic System. Iowa State University Press, Cedar Falls, IA (1987).
7. Federal Aviation Administration Air Traffic Control Specialist. 8. 9. 10.
11. 12. 13.
14. 15. 16.
17. 18. 19. 20.
Federal Aviation Administration Announcement No. FAA/ATC-008, GS-2152-7 (1988). Federal Aviation Regulations (FAR). Garrison K. Flying in Congested Airspace: A Private Pilot's Guide. Tab Books, Blue Ridge Summit, PA (1989). Findler N. V. Some artificial intelligence contributions to air traffic control. Proc. Fourth Jerusalem Conf. on Information Technology, pp. 470-475 (1984). Findler N. V. The role of strategies in air traffic control. Proc. the Eighteenth Hawaii Int. Conf. on System Sciences, Vol. I, pp. 522529, Honolulu, HI (1985). Findler N. V. Air traffic control: A challenge for artificial intelligence. A1 Expert Mag. 2, 59-66 (1987). Findler N. V. and Lo R. An examination of distributed planning in the world of air traffic control. J. Distributed parallel Processing 3, 411-431 (1986). Also reprinted in Bond A. and Gasser L. (Eds) Readings in Distributed Artificial Intelligence. Morgan Kaufmann, Los Altos, CA (1988). Findler N. V. and Lo R. A distributed artificial intelligence approach to air traffic control, lEE Proc. Part D, Control Theory Applic. 138, 515-524 (1991). Findler N. V., Bickmore T. W. and Cromp R. F. A generalpurpose man-machine environment with special reference to air traffic control. Int. J. Man-Machine Stud. 23, 587-603 (1985). Findler N. V. A prototype of a man-machine environment for the study of air traffic control. In Encyclopedia of Computer Science and Technology (Edited by Kent A. and Williams J. G.), Vol. 19. Marcel Dekker, New York (1988). Korn G. A. Interactive Dynamic System Simulation. McGraw-Hill, New York (1989). Siegel A. I. and Wolf J. J. Man-Machine Simulation Models. Wiley, New York (1969). Hearn D. and Baker M. P. Computer Graphics. Prentice-Hall, Englewood Cliffs, NJ (1986). Rogers D. F. Procedural Elements for Computer Graphics. McGraw-Hill, New York (1985).
APPENDIX The Statistical Analysis of the Results First, assume that the mean of the number of (a) conflicts or (b) violations per session with and without extrapolation follows a normal distribution with a finite mean and variance. Let the means with and without extrapolation b e / h and/~2, respectively. One can state the null and the alternative hypothesis as follows:
Ho:/AI =f12, and H~:/z~
0952-1976/92$5.00+ 0.00 Copyright© 1992PergamonPressLtd
Printed in Great Britain. All rights reserved
Contributed Paper
A Predictive Man-Machine Environment for Training and Evaluating Control Operators NICHOLAS V. FINDLER Arizona State University, Tempe CONGYUN LUO Arizona State University, Tempe MIN-HUI KUO Arizona State University, Tempe
After a brief discussion of decision support systems and the problems of Air Traffic Control (A TC), the design and implementation of a Predictive Man-Machine Environment (PMME) are described that can in general be used in the automatic training and evaluation of control operators, as well as serve as the basis of their routine planning and decision-making activity. PMME works with two connected workstations, one of which displays the essential features of the Current World and the other those of the Extrapolated World. The latter predicts the future consequences of the control operator's tentative decisions made in responding to the Current World. If the operator is satisfied with the consequences, he finalizes his decisions; otherwise, he modifies the tentative decision and, time permitting, goes through this cycle as many times as necessary. A simulated air traffic control environment for the PMME has also been implemented and two sets of experiments in it performed with tasks of equal complexity--one with and the other without the extrapolation facility. A statistically significant improvement has been shown in the A TC operations that rely on the extrapolation facility. It can be said in general that the PMME increases the effectiveness and efficiency of human judgmental processes by extending their range, capabilities and speed. Keywords: Man-machine environment, automatic training and evaluation of control operators. • Delicate judgements are needed concerning which environmental and control variables are relevant and what their current values are. The task of decision making under uncertainty refers to the fact that the currently available knowledge about the world is partial, imprecise and possibly even inconsistent. (See Chaps 1-3 of Ref. 1).
INTRODUCTION
The increasingly high rate of automation of a multitude of operations has raised the demands on the human element left in the control loop. The difficulties of the human controller are due to many factors, the most frequent of which are the following: • The knowledge base needed for a decision is too large for a human to access or even to make conceptual use of. • To arrive at a satisfactory solution requires a large amount of processing of the available information. • There is a time pressure to perform the computations (because the environment may continually change) and/or to obtain the solution (because certain actions must be performed before a certain point of time).
It is, therefore, important to devise techniques that would enhance the quality and shorten the length of the training process of control operators. It would also be necessary to implement some objective measures for evaluating their performance and use these as information feedback for self-correcting control actions. ON DECISION SUPPORT SYSTEMS
Correspondence should be sent to: Professor N. V. Findler, Department of Computer Science and Engineering, and Artificial Intelligence Laboratory, Arizona State University Tempe, AZ 85287-5406, U.S.A. 441
Decision support systems 2-5 (DSSs) are first discussed before the actual work accomplished is described. A DSS is a computer-based information system that helps a user make decisions by providing him with the necessary information in an easily understandable
442
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
form. The User is situated in an interactive environment and usually faces a menu-driven front-end. The system accesses a database to locate the necessary data, applies appropriate mathematical and/or statistical models, and displays the information needed at the user's terminal. Based on the information provided, the user can explore different alternatives in order to arrive at a decision. A more formal definition of DSSs is: 5
Decision support systems allow the decision maker to combine personal judgment with computer output in a user-machine process. Such systems are capable of soloing various types of problems (structured, semistructured, and unstructured) and offer query capabilities to provide information requested. As deemed appropriate, they use quantitative models as well as database elements for problem solving. From. an enhanced perspective, decision support systems are an integral part of the decision maker's approach to problem solving, which stress a broad view of the organization by employing the "management by perception" principle. Although the main objective of a DSS is to help in selecting correct decisions, it offers far more possibilities. The various activities involved include problem definition and structuring; data collection, filtering and fusion; choice generation and selection; prioritization and routing; planning and deciding; explaining and forecasting. These steps may be taken before, during or after the decision-making cycle. One can conclude that the important DSS factors are accessibility, flexibility, facilitation, learning, interaction, and ease of use. A few additional issues are noted: • Some DSSs focus on supporting decision making for effective planning and control in terms of finding and solving managerial problems at all levels. • DSSs give support to the user but usually do not replace him. Humans' qualitative models and other non-tangible considerations enhance the whole decision-making process. • DSSs encompass not only the user and the computer but also various models and the languages and databases of modelling and simulation--which all contribute to the decision-making process. • A DSS is an important tool when a decisionmaking process references some uncertain factors or involves many conditions, constraints, and consequences to be considered simultaneously. ON AIR TRAFFIC CONTROL Air traffic control (ATC) has evolved from flag waving, when the major concern was to avoid collision with the ground and trees, to modern computer-based
systems aiding the control of thousands of commercial, military and small private planes. 6-9 Current systems have the capabilities of detecting the airplane's ID, speed and location, and predicting potential traffic conflicts before their occurrence. Future generations of ATC systems will be built on space and satellite technology. Communication, navigation, and surveillance will be accomplished from earth orbit and will cover the entire globe. ATC is a complex topic and it is not possible to give even a brief outline of its major aspects in this paper. It suffices to say that A T C is performed in areas designated to contain instrument-flight-rules operations during portions of the terminal operation and while transiting between the terminal and the en route environments. (The other operations are under visualflight-rules 8 when the pilot, and not the controller, is responsible for the adherence to the rules.) Area Navigation Routes include two types of airways. The low-altitude airways are designated from 1200 ft above ground level up to 17,999 ft. The high-altitude or jet airways are designated at or above 18,000 ft. Separation is the essence of air traffic safety. It is provided by establishing approved minimum longitudinal, lateral, and vertical distances between adjacent aircraft. Longitudinal separation is the spacing between two aircraft at the same altitude and on the same route. Lateral separation is also between aircraft at the same altitude but on different routes. On low-altitude airways (less than 18,000ft), the minimum horizontal separation is 3 miles if the airplane is within 40 miles of the radar antenna and 5 miles if it is outside the 40 mile range. In the high-altitude case (above 18,000 ft), the separation ranges from 5 to 10 miles according to various factors such as speed and weather conditions. Vertical separation is established by assignment to different altitudes or flight levels. The minimum vertical separation is 1000ft for those airplanes whose altitudes are less than 29,000ft, and 2000ft above 29,000 ft. Finally, the separation between airplanes and obstructions is defined as follows: if the airplane is at least 3 miles from an obstruction and at least 1000 ft above it, a minimum 3-mile distance from it must be maintained until the obstruction has been passed. If the airplane is less than 3 miles from the obstruction and its altitude is at least 1000 ft above it, the minimum lateral distance of 3 miles must be reached and maintained until it passes the obstruction. An event may be characterized as the product of interactions, among aircraft, airspace, facilities, and ATC operations. There are several types of events. The types of events that are associated with a single aircraft include conflict, violation, clearance request, and flight status. The event type associated with multiple aircraft is conflict. A conflict is defined as a situation in which two or more planes get too close to each other, or one plane gets too close to geographical obstructions. A violation involves only one aircraft that violates some
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
REAL.WORLD ..,Instructions
J"
COMPUTER
.~,,,,o~/
'
\
o
p cup,,~,rr
0 ~
Fig. 1. The predictive man-machine environment for decision making and planning. flight parameters, such as the landing angle. Takeoff clearance, approach clearance, and landing clearance are clearance request events. Initial contact, filing flight plans and missing the approach are events concerning the flight status. Activities are defined as top-level sequences of m a n machine interactions which respond to a group of closely related events. Accordingly, a typical flight goes through the following terminal and en route operational activities: a jet is waiting at the end of the runway and the captain informs the departure control that he is ready for takeoff; the controller replies from the control tower by issuing the takeoff clearance and departure instructions; the jet leaves the ground and an electronically generated "data tag" (containing information about the jet's flight number, ground speed and altitude) is shown on the radar scope; as the flight approaches the boundary of the en route control sector, the terminal controller hands off control to the en route controller; when the flight is close to its destination, the pilot asks for a clearance for landing; the flight holds in a waiting pattern until the landing clearance is received; finally, the data tag disappears from the radar scope after the plane has landed. The present authors have been interested for some years in applying AI methodology to this intellectually and economically challenging area. 1.10-15
443
Environment) update the representation of the affected objects in the computer. The updating can be controlled in three different ways: • at regular points of time, following a userdefined frequency (time-driven updating); • whenever one of the set of user-specified events takes place, an update is triggered (event-driven updating); • on user's command, regardless of how long a time has elapsed since the last update or whether certain conditions in the environment are satisfied (user-driven updating). The user makes tentative decisions on the basis of the Current World displayed, say, at time to and transmits them to the computer. He then specifies a time point in the future, tl, at which he wants to see the expected consequences of the tentative decisions. This extrapolation of the status of the world is c o m p u t e d - - o n the basis of the computer-resident m o d e l - - t h r o u g h periodic (user-specified) time increments up to the final time point ft. However, if during the time period between to and tl, a conflict situation occurs, as defined by the user again, the calculation is interrupted, and the conflict situation with the corresponding time i s displayed on the second unit called Extrapolated World. If no conflict has occurred between the time points in question, the relevant features of the world will reflect the permissible consequences of the tentative decisions on the second screen. If the user is satisfied with the status of the resulting world, he finalizes the tentative decisions and informs the computer accordingly. Otherwise, he makes a different set of tentative decisions and goes through the above cycle as many times as necessary, time permitting. THE SIMULATED AIR TRAFFIC CONTROL ENVIRONMENT
First, some terminology needs to be established. To set up any simulated environment for the PMME, one has to define a set of objects inhabiting it. The conditions that affect the objects can be either constant over time (e.g. the location of a mountain range) or THE DESIGN OF THE P M M E subject to change (e.g. current weather). Objects can also affect each other (e.g. two planes with an intersectRelying on the authors' previous work concerning a ing flight path). An object passes through different simpler environment* (see Refs 12 and 16, and Chap. 8 phases in its interaction with the environment. A phase of Ref. 1), the design of the P M M E was done according can be considered as a distinct stage in the lifetime of to the flow of information and control shown in Fig. 1. the o b j e c t - - t h e period of its sojourn in the environThe Current World displays the essential features of ment. During a phase, the object may perform certain the Real World. Events from the Real World (or, as in functions toward achieving a subgoal (or, finally, a this case, from the Simulated Air Traffic Control goal). This achievement of the subgoal places the * In this early work, the user was not able to define the ATC object into another phase (N.B. a plane passes through environment--the location and the number of airports, airways, eight major phases in its flight: preparation for take-off, radio beacons, obstructions, etc. were fixed. Also, the environ- take-off, climb, cruise, descent, approach, landing and ment included only en route operations and no near-terminal ones. Consequently, the performance measures of the control operations taxiing). The overall goal is, of course, the safe and did not cover those that are relevant near an airport. timely flight between the origin and the destination.
444
N I C H O L A S V. F I N D L E R et al.: A P R E D I C T I V E M A N - M A C H I N E E N V I R O N M E N T
Associated with each object is a set of attributes which determine the constant and varying properties of the object--including the phase it is in. The number and the type of the attributes, in general, depend on the domain of application. The user must define the functions that can alter the values of the attributes every time a user-specified incremental parameter (usually time) value elapses. There is a trade-off here between "too large" and "too small" a triggering value for the incremental parameter. In the former case, more computation has to be performed but there is a lesser chance that a critical conflict can get by unnoticed in between two attribute updatings. In the latter case, the opposite is true. It is important to point out that the attributes of the objects must change independently from each other in the Current World and in the Extrapolated World. This separation of attribute scopes is a basic requirement when causal relations between decisions and consequences are sought. A large knowledge base needs to be prepared for any non-trivial domain of application. For example, in case of the Simulated Air Traffic Control Environment (SATCE), it must include • the physical characteristics of several airports (length and location of airstrips; height and location of potential obstructions, such as mountains and towers; direction of radio beacons; etc.) • the physical characteristics of the participating planes (symbolic notation for the manufacture/ model, maximum initial landing speed, approach angle, approach speed, descent angle, descent speed, maximum and average cruising speed, climb angle, take-off speed, turning radius, fuel capacity, fuel consumption in different phases, etc.); • dictionary and syntax needed in communicating between pilots and the air traffic controller; • a program to interpret the above communication; • links between controller instructions and the functions that make attribute value changes accordingly. The literature on simulation techniques approaches is, of course, very extensive. 17'i8
and
THE IMPLEMENTATION OF THE PMME
The PMME runs on two workstations, each with a separate control unit, user interface, communication unit and graphics unit. Workstation I has been assigned to display the Current World containing simulated air traffic scenarios. Workstation II displays the Extrapolated World obtained through the Extrapolator which predicts future situations. Figure 2 illustrates the organizational structure of the different components.
User Interface I I
I
Simulator
I
I
I
Graphics Unit I
]
I Graphics Unit II I
Workstation
I
Extrapolator
Workstation
[
II
Fig. 2. The organization structure of the PMME.
The Control Unit I is the main driver of Workstation I. It controls the operations of the User Interface I, the Air Traffic Simulator, the Communication Unit I and the Graphics Display Unit I. The User Interface I is the interface between the user and the Current World. It is menu-driven and the user can define tt.rough it the geographical properties of the control sector, input the simulation parameters (the length of the simulation run and the frequency of updating the world image), request statistical reports, and interact in general concerning his decision-making activity. The Air Traffic Simulator* generates the images of each aircraft in the control sector and updates the dynamic information of these aircraft. (Static information about the aircraft includes its model, flight number and destination. Dynamic information consists of its speed, altitude, location, flight direction, available fuel and flight status.) The Communication Unit I handles the communication between the two workstations. When the Extrapolator is enabled, it writes the Current World information to a disk file and sends an activating signal to the Extrapolator. Also, it receives and processes the user's final decision from Workstation II. The Graphics Display Unit 1 displays the Current World as a horizontal or vertical projection (see Figs 3 and 4). The user has further options to choose from, such as showing the trace of an airplane in either projection (see Fig. 5), zooming into or out of an area of interest, changing the area of interest (see Fig. 6), inquiring about a certain airplane's data, issuing commands to an airplane, as well as deciding whether to do extrapolation. The Control Unit H is the main driver of Workstation II. It governs the operations in it between the User Interface II, the Extrapolator, the Communication Unit II, and the Graphics Display Unit II. The User Interface H is the interface between the user and the Extrapolated World. It acts as a front-end, * If this arrangement is employed in actual ATC operations, this unit is to be replaced by digitized input from radar sensors.
NICHOLAS V. FINDLER et
al.:
A PREDICTIVE MAN-MACHINE ENVIRONMENT
445
I J~] FileView-DCL Command H-display V-display Change area Zoom Planes info Finalize opt Movement tracer Go
,408 9 30 7
Airplane aa1615 and Airplane sw408 conflict
nw1686 dc9 35 235
III I/
I
Stop Task
I
Fig. 3. The current world, horizontal projection.
FileView-DCL Command
[]J~'-I
17:27:37
45000.0 / aa1615
nw1686 dc9 35 235
/ 7 2 7 30
220
H-display V-display Change area Zoom Planes onfo Issue command Time ( I'o"~/0ff ) Movement tracer Extrapolation opt Go nw383 17:35 0 scottsdal aa787 17:37 0 phoenix
J
\
aa787 737 0
nw383
o \
/dc9o
"~
J Hold Output
~
0 Feet
0
I Fig. 4. The current world, vertical projection.
EMI 5:5-C
J
Stop Task
I
'~
446
N I C H O L A S V. F I N D L E R et al.: A P R E D I C T I V E M A N - M A C H I N E E N V I R O N M E N T
i~
FileView-DCL Command
I~1~-----II
1:0:20
45000.0
H-display V-display Change area Zoom Planes onfo Issue command Time (Ionl/off) Movement tracer Extrapolation opt Go
Tracing vertical Movement of aa787 •
•
Stop tracing? Please enter y or n
/
aa787 737 15 3O0
0 Feet
I .o,oo=u, I
I s'o0Task I
Fig. 5. The trace of an airplane's flight path, vertical projection.
H-display V-display Change area Zoom Planes info Issue command Time (on/I'~'1) Movement tracer Extrapolation opt Go
,408 9 30 7
LLC (0.0 0.0) (40 40)
W ~oo 55
"Allarea Enter New LLC & W d when done Input = (x, y, w) I Fig. 6. Changing the area of interest, horizontal projection.
Stop Task
I
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT allowing the user to make tentative and final decisions, and to specify the extrapolation parameters (the length and time increments of the extrapolation process). The Extrapolator calculates the situation in the Extrapolated World, starting from the situation in the Current World. It also detects conflicts and warns the user about them through the User Interface II. The Communication Unit H receives messages from Workstation I, such as "the geographical properties file is ready to be read" and "start the extrapolation". Once available, it writes the controller's final decision to a disk file and informs Workstation I thereof. The Graphics Display Unit H, displays the Extrapolated World as a horizontal (see Fig. 7) or vertical projection. As with the Graphics Display Unit I, the user is offered the options of showing the trace of an airplane in either projection, zooming into and out of an area of interest, changing the area of interest, and inquiring about a specific airplane. Finally, there is an independent computational component not included in the system diagram on Fig. 2, the Eoaluation Module. It serves to measure and compare the efficiency and the effectiveness of the controller with and without the Extrapolator facility. The module contains the specification of experiments and a set of metrics used in the evaluation. The simulated radar scopes are patterned after those in use at the T R A C O N centers of the Federal Aviation Administration. The displays present a map which is
447
approx 100 miles in radius and contain a "control section", an area for which the present controller is responsible. The map is centered over the location of the radar dish that generates the display. The circular grid lines are shown five miles apart. The location of the airways, radio beacons, mountain peaks and other relevant landmarks are also indicated in a standard symbolic format. In real-life, A T C displays produce only a blip on the screen for an individual aircraft while a so-called transponder on the aircraft continually sends messages to superimpose additional display information as needed. This information on the aircraft's identification, ground speed, altitude and other data is presented in a block connected with a line to the exact location of the aircraft on the screen. The display in our work simulates this final result. The simulated environment has been kept as realistic as possible but the displays are also enhanced with certain additional facilities that are not yet available with real-life radar scopes. Summing these up, use has been made of the color capabih'ties of the display units to make the identification of various groups of symbols easier. The user is also given the ability to center the display over any part of the map, and to zoom it in and out as necessary to obtain a more-detailed view of potential problem areas. There is additionally a system clock, a text window for presenting the status of the system and the airplanes, and a menu display to make
5 :)
H-display V-display Change area Zoom Planes info Finalize opt Movement tracer Go
,408 9 30 7
Airplane aa 1615 and Airplane sw408 conflict
lw1686 :1c9 35 .>35
I
Stop Task
Fig. 7. The extrapolated world, horizontal projection.
I
448
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
P M M E more powerful and flexible. The menu system allows the user to select or change any of the simulation parameters which are not associated with the behavior of the airplanes. Through the menu system, one can vary the time parameters of the simulation, change the color of a group of symbols, or alter the center and scale of the map, as noted above. The next innovation concerning a second display mode has also been provided, wherein the vertical profile of an airplane's movement can be viewed. The display in this mode is centered on the target aircraft and the viewing plane is rotated so that the target is always moving perpendicular to the controller's line of sight. Although the current A T C radar scopes do not yet have this capability either, the facility was provided to enhance the man-machine interface in certain critical situations. Also, the user can request in this system a trace o f an aircraft's trajectory in either projection. The internal structure of the program handles each aircraft as an independent object, complete with its own performance and behavioral characteristics. ~9'2° The airplane objects carry information about their flight plan, commands previously issued to them by the controller, a full description of their current status, and a pointer to a generic block of information (an information structure of the frame type) containing the performance characteristics of a class of airplanes (e.g. DC-9, Boeing 747, etc.). This way, any environment can be described in a high-level manner, in terms of the properties of the aircraft in that area. The data structures used allow the user to manipulate system time easily. To update the environment to a new time, a set of operations has to be performed on the representation of each airplane. Extrapolation is accomplished by taking the current state of the environment and then repeatedly updating it by the time increment parameter until the specified time is reached and the display is triggered. (In this, it is assumed that no conflict has occurred. Otherwise, the extrapolation process is interrupted, warning sound and light signals are given, and the conflict in question is displayed at that time point.) Further, the environments can be saved periodically to allow the controller the possibility of an "instant replay". Another novel addition to the system is a versatile reminder system. The controller is able to describe a variety of anticipated situations using the menu system. When the program senses that a predefined situation is about to occur, it alerts the controller with a tone and a message with reference to the triggering condition. This facility may save the controller valuable time by providing retrieval cues for previously generated plans. In setting up the PMME, the system allows the user to define the geographical properties of the environment, such as the area of the control sector, the location of the airports, waiting stacks, airways, ranges of the mountains, towers, and radio beacons. The user can also adjust the level of task difficulty--
that is, the range* of traffic density (upper and lower limits of the number of aircraft) and the ratio between simulated and real time (to suit the needs of student controllers). The geographical properties can be stored in a file to be retrieved at later training sessions. One can thus evaluate the improvement in the user's performance while the level of difficulty is kept constant. THE EXPERIMENTS AND THEIR EVALUATION
In order to evaluate the P M M E concept, two groups of experiments were designed and performed. The objective was to compare the efficiency and effectiveness of human planning and decision making in a complex, real-time problem d o m a i n - - w i t h and without the aid of the interactive extrapolation facility. The task specified for the subjects was en route and near-terminal A T C under light, moderate and heavy traffic conditions. The performance of the subjects was automatically evaluated in each experiment. The primary quality measures were the number of airspace conflicts and violations which occurred during the session. The secondary measures related to the efficiency with which the aircraft were routed through the subject's control sector. These include the following: • The number of instructions issued--the smaller it is the better the control strategy is, other factors being equal. (Reason: pilots prefer fewer, effective and efficient instructions.) • The number of extrapolations r e q u e s t e d - same as above. (Reason: extrapolation takes time away from concentrating on the situation, particularly under time-stressed conditions.) • The number of conflicts--for all aircraft in a conflict situation during the session, updated at the end of every internal time increment. • The average "off-distance"--the difference in miles between the lengths of each aircraft's requested (usually straight-line) route and the sum of the distance actually travelled through the air traffic controller's sector plus the distance from the point it left the sector to the plane's destination. • The average "climb/descend distance" per aircraft--the total distance in feet each aircraft was told to climb and descend during its flight through the sector, plus the difference between its requested altitude and its altitude upon leaving the sector. • The average delay of departures and arrivals per aircraft--difference between actual and scheduled time points. • The ratio between the simulated and real-time. * The actual number and the location of the planes participating in the experiment at hand are within the ranges specified and are determined by means of random number generators.
NICHOLAS V. FINDLER et
al.:
A PREDICTIVE MAN-MACHINE ENVIRONMENT
449
Table 1. Summaryof experimental results. With extrapolation Air traffic density
Light
Average no. of planes Average no. of commands Average no. of extrapolations Average no. of conflicts Average no. of violations Average off-distance Average climb/descent-distance Average departure/arrival delay Average simulated/real time
10.9 8.5 5.0 1.9 1.4 0.13 0.27 1.0 1.21
Moderate H e a v y L i g h t Moderate Heavy 15.2 11.2 7.5 4.0 0.8 0.10 0.12 1.4 1.39
The faster this user-selected number approached one, the more effective the controller's learning process was. As noted before, the program randomly generated the number of participating aircraft, their flight plans and certain other characteristics. For the benefit of objective comparisons, the task complexity was the same for the corresponding experiments--30 with and 30 without the extrapolation facility. Each session lasted 20 min of simulated time. The time increment was 60 s in the Current World and 30 s in the Extrapolated World. Each group of 30 sessions was further divided into three sets with light, moderate and heavy air traffic d e n s i t y - - a b o u t 10, 15 and 20 airplanes, respectively. The results are summarized in Table 1 and the effect of the extrapolation is shown in Table 2. The statistical analysis of the significance of these figures is summarized in the Appendix. CONCLUSIONS The experimental results clearly indicate that in the task environment chosen, the P M M E is an effective teaching tool, and the subjects' performance with the extrapolation facility was significantly better than without it. (There is also an extraneous and small contributory factor to note. Since the two display units are connected to the same hard disk memory, the Current World environment runs somewhat faster in real-time when the extrapolator is disabled, which makes the controller's task a little more difficult. However, the Table 2. The changes with extrapolation Air traffic density
Light
Average no. of planes Average no. of commands Average no. of extrapolations Average no. of conflicts Average no. of violations Average off-distance Average climb/descent-distance Average departure/arrival delay Average simulation/real time
-+ 12% -- 41% - 36% - 32% - 31% - 17% + 32%
Moderate Heavy -+ 8% -- 57% - 60% - 62% - 71% - 26% + 30%
Without extrapolation
-+ 13% -- 28% - 33% - 29% - 40% - 9% + 42%
20.5 16.1 10.2 9.1 2.6 0.22 0.31 2.1 2.42
10.3 7.6 0.0 3.2 2.2 0.19 0.39 1.2 0.82
15.4 10.4 0.0 9.4 2.0 0.26 0.41 1.9
0.97
20.5 14.3 0.0 12.7 3.9 0.31 0.52 2.3 1.41
difference in speed was small enough to state unequivocally that an accurate evaluation of the usefulness of the P M M E has been obtained.) The results were especially impressive in reducing the number of conflicts when the air traffic density is moderate (57% less conflicts and 60% less violations with the extrapolator enabled). In general, the number of conflicts, violations, average off-distance, average climb/descend distance, and average arrival/departure delay were all reduced significantly. However, the number of commands used increased to some extent. This was expected because some of the commands were made to avoid the potential conflicts warned about by the extrapolator. The simulated and real-time ratios also increased when the extrapolator was used, particularly under heavy traffic conditions. (This, to a large extent, was due to the time-consuming w a y - - v i a the k e y b o a r d - - i n which the subjects communicated with the simulated pilots). Although the use of the extrapolation facility improved the subjects' performance, it was found that the facility can also be abused and lead to undesirable outcomes as follows: • After having gained some experience in the use of the extrapolator, it is easy to become complacent about the control task when no conflicts occur within the time interval between the current and the extrapolated time. • If extrapolations are continually requested for short time intervals, the major part of one's attention is drawn away from the Current World to the Extrapolated W o r l d - - a g a i n resulting in a possible oversight of conflicts in the immediate future. • In "crisis" situations in which many decisions must be made in a short period of time (e.g. several conflicts are imminent), extrapolation can often be more of a hindrance than help. When facing such a confusing situation, a seemingly safe thing to do is to run the extrapolation into the near future. However, this action may result in wasting time during which effective directives should have been issued to resolve the conflicts.
450
NICHOLAS V. FINDLER et al.: A PREDICTIVE MAN-MACHINE ENVIRONMENT
In spite of the above caveats, PMME is a useful, general-purpose environment for real-time decision making and planning. It should, however, be relied upon only as an auxiliary source of information--the planner and decision maker should always focus his attention on the unfolding events in the Current World. The PMME can be improved in several of its aspects. First, low-level mouse programming utilities would strengthen the graphics user interface. The student controller should be able to select different viewing areas and retrieve information about an aircraft via mouse positioning instead of keyboard typing. Second, SATCE should be able to generate more-realistic scenarios, such as those based on Poisson rather than uniform distributions and incorporating also unexpected events (emergency landing, hijacking attempts, sudden changes in the weather, etc.). Summing up, the PMME can in general be used to improve trainees' analytical skills, long-range planning ability and diagnostic capabilities, so that they can balance objectives, associate methods with results, convert objectives--specified as the desired state of the environment--to dynamic process models, utilize resources to minimize risks, and contrast costs with benefits. Several other applications seem also possible, both for automated training and evaluation as well as for routine control operations. These range from investment portfolio management to command and control tasks in different conflict situations. Acknowledgements--The authors are grateful for many useful ideas obtained from T. W. Bickmore, R. F. Cromp and N. Mazur, and for the subjects' efforts and enthusiasm. The DEC Workstation 3100s used for this project were obtained through the Digital Equipment Corporation External Research Grant No. 158 and 774.
REFERENCES 1. Findler N. V. Constributions to a Computer-Based Theory of Strategies. Springer, New York (1990). 2. Andriole S. J. Handbook of Decision Support Systems. Tab Books, Blue Ridge Summit, PA (1989). 3. Hawgood J. and Humphreys P. Effective Decision Support Systems. Gower Technical Press, Aldershot (1987). 4. Ross S. C., Penlesky R. J. and Doney L. D. Developing and Using Decision Support Applications. West Publishing Company, St Paul, MN (1988). 5. Thierauf R. J. User-Oriented Decision Support Systems Accent on Problem Finding. Prentice Hall, Englewood Clifs, NJ (1988).
6. Brenlove M. S. The Air Traffic System. Iowa State University Press, Cedar Falls, IA (1987).
7. Federal Aviation Administration Air Traffic Control Specialist. 8. 9. 10.
11. 12. 13.
14. 15. 16.
17. 18. 19. 20.
Federal Aviation Administration Announcement No. FAA/ATC-008, GS-2152-7 (1988). Federal Aviation Regulations (FAR). Garrison K. Flying in Congested Airspace: A Private Pilot's Guide. Tab Books, Blue Ridge Summit, PA (1989). Findler N. V. Some artificial intelligence contributions to air traffic control. Proc. Fourth Jerusalem Conf. on Information Technology, pp. 470-475 (1984). Findler N. V. The role of strategies in air traffic control. Proc. the Eighteenth Hawaii Int. Conf. on System Sciences, Vol. I, pp. 522529, Honolulu, HI (1985). Findler N. V. Air traffic control: A challenge for artificial intelligence. A1 Expert Mag. 2, 59-66 (1987). Findler N. V. and Lo R. An examination of distributed planning in the world of air traffic control. J. Distributed parallel Processing 3, 411-431 (1986). Also reprinted in Bond A. and Gasser L. (Eds) Readings in Distributed Artificial Intelligence. Morgan Kaufmann, Los Altos, CA (1988). Findler N. V. and Lo R. A distributed artificial intelligence approach to air traffic control, lEE Proc. Part D, Control Theory Applic. 138, 515-524 (1991). Findler N. V., Bickmore T. W. and Cromp R. F. A generalpurpose man-machine environment with special reference to air traffic control. Int. J. Man-Machine Stud. 23, 587-603 (1985). Findler N. V. A prototype of a man-machine environment for the study of air traffic control. In Encyclopedia of Computer Science and Technology (Edited by Kent A. and Williams J. G.), Vol. 19. Marcel Dekker, New York (1988). Korn G. A. Interactive Dynamic System Simulation. McGraw-Hill, New York (1989). Siegel A. I. and Wolf J. J. Man-Machine Simulation Models. Wiley, New York (1969). Hearn D. and Baker M. P. Computer Graphics. Prentice-Hall, Englewood Cliffs, NJ (1986). Rogers D. F. Procedural Elements for Computer Graphics. McGraw-Hill, New York (1985).
APPENDIX The Statistical Analysis of the Results First, assume that the mean of the number of (a) conflicts or (b) violations per session with and without extrapolation follows a normal distribution with a finite mean and variance. Let the means with and without extrapolation b e / h and/~2, respectively. One can state the null and the alternative hypothesis as follows:
Ho:/AI =f12, and H~:/z~