Real-time decision support for air traffic management, utilizing machine learning

ControlEng. Practice,Vol. 4, No. 8, pp. 1129-1141, 1996 Copyright © 1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0967-0661/96 $15.00 + 0.00

Pergamon PII:S0967-0661 (96)00113-X

REAL-TIME DECISION SUPPORT FOR AIR TRAFFIC MANAGEMENT, UTILIZING MACHINE LEARNING J. Nogami, S. Nakasuka and T. Tanabe Department of Aeronautics and Astronautics, Faculty of Engineering, The Universityof Tokyo, Hongo 7, Bunkyo-ku, Tokyo 113, Japan ([email protected])

(Received November 1995; in final form June 1996)

Intelligent air traffic management (ATM) systems will be necessary for future air traffic control (ATC), which can manage air traffic flows and flight schedules efficiently in a real-time fashion. To meet this objective, in this paper, an automated decision support system is described. This system consists of several distributed decision-makers, and uses the concept learning scheme using neural networks. The system has the capability to find a suboptimal solution without interrupting the actual operations, in order to deal with various constraints. Simulation studies show that the proposed scheduling strategy works rather more efficiently than the current ATC procedures based on fixed heuristic rules. Abstract:

K e y w o r d s : Air traffic control, scheduling algorithms, decision support systems, machine learning, optimization problems, neural networks, real-time systems

1. INTRODUCTION

In this paper, the usefulness of the application of a machine learning scheme to future ATM functions is first explained. Then, in order to verify its usefulness, an automated real-time decision support system for ATM in a restricted region such as a single-unit enroute sector or a terminal area is proposed. Dynamic selection of scheduling rules during actual operations, referred to as "dynamic scheduling" in this paper, is utilized in this system. For this strategy to work effectively, sufficient knowledge is required that predicts which rule is the best, reflecting the traffic situations at the decision time. In this system, an inductive learning algorithm using neural networks is proposed for acquiring such knowledge. Simulation studies show that the suggested scheduling scheme works rather more efficiently than the current ATC procedures based on simple fixed heuristic rules such as the first-in first-out (FIFO) discipline.

Current air traffic control (ATC) systems have mainly been conceived to ensure, with tactical interventions, the safety of flights and an orderly traffic flow, although some functions have been partly automated. Today, such systems are no longer efficient because of increasing travel demands and congestion phenomena in major terminal areas. Therefore, it seems necessary and practicable to introduce, in future air traffic management (ATM) systems, not only more-automated procedures to maintain adequate safety levels, but also planning and real-time decision support functions. These would have the capability to minimize ATC controllers' interventions that interrupt the nominal flight plans of the airlines, to deal flexibly with anomalous events, to predict the timevarying traffic evolution as accurately as possible, and to manage air traffic flows and flight schedules effidently and strategically in a real-time fashion in vast air-route and flight networks, in order to reduce unnecessary airborne delays. To meet this objective, a fast, intelligent prediction and scheduling algorithm has to be designed.

Next, a real-time prediction and decision-making support system for ATM in a global airspace consisting of many airports and crossing air routes is further suggested. This system has an ability to 1129

J. Nogami et al.

1130

make probablistic forecasts of congestion delays, etc., and determine the current best control actions in a real-time fashion (e.g., flow regulation, ground holding or rerouting strategies, etc.) at any decision time. A graphic network consisting of various traffic attributes is an evolving knowledge tool for fast prediction and decision making. Simulation studies on this system are currently under way.

2. ATM AND MACHINE LEARNING 2.1 A control model of A T M

Figure 1 shows a control model of the ATM task, which can be decomposed into three levels, namely, air traffic planning, real-time ATM and real-time ATC, according to the time scale of the planning horizon. The main task of air tralfic planning is the regulation of flight schedules. The main task of the real-time ATM is the on-line forecasting of congestion overloads, and on-line decision making regarding flow control actions to prevent costly airborne delays. The main task of the real-time ATC is to ensure the safety of air traffic, and to detect and avoid possible collisions. In this paper, the real-time ATM activity is mainly considered, which can be further decomposed into two types of traffic flow management (TFM) actions, according to the scale of the flow control regions. One type of action, which is addressed in Section 3, is a microscopic or tactical one, exercised when an aircraft is already airborne. The other type of action, which is addressed in Section 4, is a macroscopic or strategic one, with greater potential for regulating aircraft flows.

I

Air Traffic [ Planning System

Real.Time [~ ATC System

I~I

t lCwllkmlIPllktIPlnn

¢eaflkt re~lutlea F

t I *'~'+'

(cement ] ~ Itramc& | system I r tsitna; °ns J

l Centrallzetl I

I ArM Center I •= ' = -

I Disturbance I | Elements | lI "emerlmc! ""+~'+~..,er iI

tt

Im--r l Real-Time ATM System

Figure 1. A control model of the ATM task

1. Immediate decision making at any discrete decision point is needed, so that real operation is not delayed. 2. The environments surrounding this decisionmaking process change dynamically with time, and involve various uncertain elements, some of which are governed by stochastic processes and others of which are difficult to formulate mathematically. 3. Optimization criteria cannot be obtained until a large number of unspecified successive decision timings occur, requiring the long-term anticipation of the discrete events which will be expected to occur throughout the planning horizon. 4. Various strategies to avoid the so-called combinatorial explosion are needed to deal with a great number of control variables and constraints. 5. Negotiation and regulation among multiple agents, which have individual requirements conflicting with each other, are needed. It is impossible to find an optimal solution to this problem, consisting of the best control actions and their timings without interrupting the actual operations, by means of exhaustive search-based methods or model-based mathematical analyses. The introduction of knowledge-engineering methods is thought to be one of the most effective strategies to deal with this problem. Several known expert systems for flow control in one terminal area such as CTAS (Erzberger, et al., 1994; Tobias and Scoggins, 1986; Visser, 1991) or COMPAS (Volckers and Schenk, 1989) have functions to make fast, intelligent decisions. But as far as real-time traffic flow management in a wide region with many airports is concerned, studies on intelligent, efficient scheduling methods, e.g. ground holding or rerouting strategies, are at the stage of trial and error at present. Further, if the free-flight concept is realized, the number of control variables will increase greatly, because the degree of freedom of traffic control will extend from a single dimension (only time) to four dimensions (time and space), so it will be even more difficult to find a suboptimal solution using only the known heuristics that have been empirically obtained by ATC controllers, TFM managers etc. Therefore, intelligent decision support systems are essential for solving this real-time ATM problem, in which new knowledge can be created to assist in making on-line decisions, and a suboptimal schedule can be made in real time using the created knowledge at any decision time.

2.2 Real-time A T M and machine learning

The real-time ATM problem, which is a typical large-scale scheduling problem, has the following characteristics.

Machine learning is recognised as an effective method by which some novel and evolutionary knowledge is acquired and generalized systematically. Various machine-learning techniques have

Real-Time Decision Support for Air Traffic Management been adopted for other scheduling problems with similar characteristics (Grefenstette, et al., 1990; Nakasuka and Yoshida, 1992; Nakasuka, et al., 1994). In this paper, an inductive learning technique is adopted in order to search for a suboptimal solution within a practical computational time, and using the technique, the relationships between the time history of the traffic situations and the best control actions at an arbitrary decision time are explicitly acquired as mapping functions. Such mapping relations can be estimated inductively using a large number of data obtained through off-line simulations and mathematical analyses. In this paper, neural networks and attribute graphs are used to represent such mapping relations, and some inductive learning algorithms such as the backpropagation algorithm are utilized for acquiring the knowledge about such relations.

1131

The control region is described through a network, whose nodes represent the intersection of the plural airways, etc., and a set of discrete altitude levels is assigned for each node. This typical scheduling problem can be formulated mathematically using nonlinear mixed integer programming (MIP) as the following constrained optimization model. The control variables s~, u iI¢~ks, Vk, ij t~ and r~ should be chosen to minimize the multiple objective functions: "

minJ1 = rain E Oti{ E %t~} i i iEi kEK)

(1)

minJ2 = m i n E f l i { E skit i ki - t}.o.~l } iEl kEKi1

(2)

minJa = min E 7i[ E E iEI klEK i ksESikl 3. DECISION S U P P O R T SYSTEM F O R SINGLE-UNIT S E C T O R REAL-TIME ATM

3.1 Problem formulation In this section, the real-time ATM problem within a single-unit sector is considered. The inputs, decision items (amendments to the flight plans) and constraints in this problem are summarized in Table 1. The amendments to the flight plans should optimize certain requirements, while satisfying the given constraints: not only the maintenance of the nominal flight plans as far as possible, but also reducing the total operating costs of all the airplanes in a prefixed region, are required as optimization criteria.

•

+F~(kl, k2, t ks i

i t ikl

--

i - - T Iil l ) } ]

(3)

where, oq, fli and ")'i are weight coefficients depending on the plane class, subject to the flight path and altitude constraints:

s ikl

Ukli k2

E

y~ I

k2fiS~i 1

i 8iks Uklks

(4)

k2ff:S~1 , f o r Vkl E K i, Vi E I

the ATC separation constraints: i j

i

"

sks~{t k

-

t~

-

ji Dk

+ ( Z - D~~ + ~ k )v~ t > 0 i j j ji sk%{t ~ -t' k + Z- D k r~ji~ ij~ - ( Z + D~j - z 2 k )v k ) >_ 0 , f o r Vk E K i n K j, v( i, j ) E I, i < j

(5)

"

Table 1 Inputs, decision items and constraints Inputs 1. the nominal flight plans of the airlines 2. the information relative to the state of the system radar data on the evolution of flights - short and medium meteorological forecasts - the status of the ATC infrastructure 3. the restrictions imposed by flow control Decision Items 1. rerouting to by-pass the congested areas 2. variations of altitude levels in comparison to the nominal flight plans 3. imposing route delays through speed control 4. imposing holding patterns 5. sequencing landing orders, etc. Constraints 1. sector capacity 2. horizontal and vertical separation criteria 3. flight performances dependent on aircraft types and weights 4. forecasting accuracy on wind data

(6)

the initial and terminal conditions: >_

-

r]:: = 0

, yor V i e Z

, .for Vi ~ I

(7) (8) (9)

the passage and holding time constraints:

E uikx k21ftik2 _ t ika -- Tkl i ksES~1 i

E

- t m i n ( k l , k2)} _> 0 u iklkslttikl + Tkl i t iks -

(lO)

-

kzES~I

i

"t-tmax(kl,k2)} > 0 , for Vkl E Ix't, Vi E I

(11)

J. Nogami et al.

1132 and the overtaking prohibition constraints:

(12)

, f o r Vkl e K ' , V ( i , j ) E I, i < j, where

[to,

to + TR] =

IDi

=

KDk

=

t,"i ~ k'I

K' D k~

-_

=

=

t~nom

=

tlmin(kl,k2) -=

D~J

=

F~(k,r)

=

F~(kl,k2,t)

=

S i k

st

=

Ui klk2

ij

Vk

Z

=

strategic planning region (sector region) strategic planning interval set of indexes of flights in R in the interval [to,to + T~] set of indexes of three-dimensional nodes in R set of departure nodes by plane i set of all the possible trajectory nodes traversed in R by plane i (k~ = ko~ ' ..~ k}) actual passing time of plane i on node k actual holding time of plane i on node k nominal departure time from R of plane i minimum time intervals from node kl to node k:~ of plane i maximum time intervals from node kl to node k2 of plane i minimum time separation at node k between preceding plane i and trailing plane j fuel consumed when plane i holds for time r at node k fuel consumed when plane i flies from node kl to node k2 during the flight time t set of indexes of nodes following, when plane i starts from node k binary 0-1 variable indicating whether plane i occupies the node k, or not binary 0-1 variable indicating whether plane i passes over the flight arc consisting of nodes kl and k2, or not binary 0-1 variable; if plane i passes over node k before plane j does v~$ = 0, otherwise v~~ = 1 sufficiently large number•

Several objective functions [Eqs.(1-3)] can be considered relative to the individual requirements of the various agents (e.g., ATC controllers, or airline operators). In this section, only J2 is utilized as the optimization criterion, which represents the sum of the differences between the actual sector departure times and the nominal ones for all the aircraft which are predicted to exist in the sector B. within the planning interval [to, to + Tn]. T o in Eq.(7) indicates the minimum possible passage time of the node k~ for the plane i. If the plane i is not in the sector R at the time to, Tio depends not only on its speed performance but also on the m a x i m u m allowable number of aircraft at

any time within the planning interval in the sector R , denoted as Qn(to, Tn), which can be determined by both a m a x i m u m controller workload and flow restrictions between the sector l~ and its neighboring sectors. In this section, QR(to, TR) is assumed as infinite, and the restrictions imposed by flow control are not considered. This problem is NP-complete, so a fast, intelligent scheduling algorithm is required to solve this problem because the scheduler must deal with various anomalies such as aircraft accidents, airport closures or changes in weather, etc. Exhaustive search-based strategies cannot be utilized directly, because an optimal solution could not be found within a practical computational time if such strategies were used.

3.2 Current procedures for the control of flights Bianco and Bielli (1993) indicated t h a t this problem could be formulated as an optimization problem, with the hypothesis that the nodes to be traversed by single airplanes were fixed on the basis of their nominal flight plans, and altitude levels and passing times on the node trajectories could be determined by means of the branch-and-bound technique in a real-time fashion, on the assumption that flight scheduling was carried out according to the F I F O discipline. T h a t is, this method decomposes the problem into two subproblems that can be denoted, respectively, as "altitude control" and "speed control". This is a compromise method to find an acceptable solution within a short computational time through a large reduction of the solution space by the use of fixed heuristic rules, which becomes consistent with the current ATC procedures. Such rule-oriented reactive scheduling has been widely utilized as a practical and robust scheduling method. The weak point of this strategy is, however, that these rules, empirically extracted from human controllers, only refer to the local information for decision making, and the generated schedule cannot always be good in the global sense.

3.3 Basic idea of dynamic scheduling To compensate for such shortcomings of the reactive scheduling method, a "dynamic scheduling (DS)" method has been adopted in this paper. In this method, many scheduling rules are prepared, from which one is selected dynamically during actual operations, considering the instantaneous traffic situations at each decision time. Therefore, the scheduling rule to be used is not fixed, but switches from time to time. Mapping knowledge as to the relationships between the "traffic situa-

Real-Time Decision Support for Air Traffic Management tion vs. best heuristic rule" is required, because rule selection must be completed in such a short time that real operation is not delayed. A powerful inductive knowledge-acquisition mechanism is needed, so that the effect of the DS method can be fully exploited. For this objective, in this paper, a finite set of basic attributes, approximately representing the instantaneous traffic situations, is extracted and the nonlinear mapping relationships between the "basic attribute set vs. best heuristic rule" are obtained inductively using neural networks. This type of machine learning is named "concept learning", and is a kind of inductive learning technique. A D S method, based on the concept learning scheme and using binary decision trees or neural networks, has already been applied to other scheduling problems (Nakasuka and Yoshida, 1992; Nakasuka, et al., 1994), and good results have been reported.

1133

tion vs. the best heuristic rule in off-line simulation. The following three types of a priori human knowledge are utilized. 1 . P o s s i b l e S i t u a t i o n s This module has the capability to generate various different traffic situations, including possible anomalies such as runway closures, aircraft accidents, etc. 2 . B a s i c A t t r i b u t e s Attributes reflecting the characteristics of overall traffic situations at the decision time, and the status of the ATC infrastructure, are prepared for every manager. More local traffic attributes, e.g. congestion level in the local area around aircraft with a similar priority, are further prepared for Managers 2 to 4. 8 . S c h e d u l i n g R u l e s Several rules with quite different characteristics are prepared for each manager.

3.4 Proposed Scheduler DS1/DS2 To solve the real-time ATM problem within a single-unit sector, an intelligent scheduling system named DS1 has been developed, which is based on hierarchical and iterative cooperation by four distributed managers, as shown in Figure 2. Each manager has, a priori, a set of several heuristic rules, from which one is selected during actual operations by means of the DS method, reflecting the instantaneous traffic situations at each decision time. Manager No. 1 determines the priorities for single airplanes, and extracts the cooperating aircraft with similar priorities. Manager No. 1 has a preferential rule, in which an emergency aeroplane can have the highest priority. Once the priorities of single airplanes have been determined by Manager No. 1, Managers 2, 3 and 4 are activated for airplanes with similar priorities. Manager No. 2 assigns for each airplane, either a runway in a terminal area, or an altitude level at which it leaves an enroute sector. Manager No. 3 determines a flight strategy criterion for each airplane. Manager No. 4 assigns an air-route consisting of nodes to be traversed and an altitude level for each node for each airplane. Once the trajectories and flight strategies have been assigned by these managers, modified time schedules for a group of airplanes with similar priorities are made~ assuming the premodified flight plans of airplanes with higher priorities as constraints, by means of the combined techniques of non-linear programming and the branch-and-bound technique. This process is iterated until rescheduling is completed for the group of airplanes with the lowest priority. Figure 3 depicts the detailed configuration of DS1. The build time module (BTM) in DS1 acquires the mapping knowledge between the traffic situs-

Figure 2. Overall configuration of DS1 system

Posetble S H u

I

Problem ~ GemerRUq AT(: hRnllml~ure fI~qmi remells

Declare ~I~b!

"

..... ~man.~ it.ks I ~e.rcl, . J~J, ~

rAudt,.kSet C I

F =

SwOa'h~1 sh

~ ud-Bev.d

Bait

]~xtn etlm

TImLU. e

j Kw.de4M

- l ~ d f l e Albtbm Extr~Hm

Nmrd

i . -~, i~-~ ~ , Figure 3. Detailed configuration of DS1 system Every manager in DS1 has its own individual neural network consisting of three layers, whose input layer is a set of basic attributes and whose output layer dictates the most suitable decision rule among the candidate rules prepared for that manager. In the build time module, the best heuristic rule of each manager is exhaustively sought, through simulation, for an arbitrarily generated sample problem which is randomly set according to a priori knowledge about the possible situations of the traffic environment. The criterion determining the best solution is the sum of delays for all the aircraft which are predicted to be in the sector within the planning interval [Eq.(2)]. The

J. Nogami et al.

1134

delay of an aeroplane is defined as the difference between its nominal departure time from the sector, and its actual one. Then the pair of d a t a on the basic attribute set vs. the best scheduling rule is extracted for each manager. As for Managers 2 to 4, a generated problem in the build time module produces simultaneously a large number of subproblems, corresponding to individual groups of aircraft, classified according to their priorities by Manager No. 1, and one learning d a t a sample is extracted from each subproblem. A large number of such d a t a are input into the inductive learning part, and the mapping knowledge as to the relationships between the basic attribute set vs. the best heuristic rule for each manager is acquired by means of the backpropagation algorithm. The learned weight matrix of the neural network is stored in the manager. Figure 4 shows the neural networks for managers 1 to 4, each of which consists of three layers which respectively have nl, n2 and na nodes. Table 2 summarizes a set of basic attributes and sets of a priori heuristic rules of these four managers. T h e run time module (RTM) in DS1 selects one scheduling rule for each manager at each decision point during the real operations, using the mapping knowledge stored in the neural network of the manager. DS2, shown in Figure 5, is an extended system of DS1. It has the capability to modify schedules reflecting the local or detailed traffic situations, on the basis of schedules obtained by DS1. DS2 also uses the DS method and the knowledge acquisition mechanism using neural networks. Sequencing, flight-time changes, rerouting, etc., are executed by additional distributed Managers 5 to 8 for an extracted group of neighboring aircraft. A fixed rule is used for classifying the aircraft into groups, and the number of aircraft belonging to the same group is limited to less than nine. The distribution pattern of current 3D positions, nominal landing sequence pattern (e.g., L H L L H . . . ) , variation in assigned routes, etc., of aircraft belonging to a chosen group, the congestion level of a particular node (e.g., runway No. 2 or waypoint No. 10), and so on, are given as basic attributes as to the local traffic situation. Heuristic and optimal constrained position-shift methods (Neuman and Erzberger, 1990) axe partly utilized.

Attributes of Situation (nl)

Hidden Layer (n2)

Best Heuristic Rules (n3)

Figure 4. Neural networks stored in managers

Table 2 Basic attributes and heuristic rules No. Attribute Content 1 No. of aircraft currently within the sector 2 Variance of density distribution 12 Conflict probability, estimated on the assumption that every aircraft flies according to its nominal flight plan ( Attributes 13 to 18 are not used for Manager No. 1. ) 13 No. of the rule chosen by Manager No. 1 14 Congestion level in the local area around aircraft belonging to the same priority group 18 Traffic flow ratio among routes and altitudes of the airplanes with higher priorities Manager No. 1 (n1=8, n2=12, n3=6) No. Heuristic Rule Description 1 FIFO (first-in first-out) 2 The earlier the nominal sector departure time, the higher the priority The larger the estimated conflict probability, the higher the priority Manager No. 2 (n1=18, n2=20, n3=4) No. Heuristic Rule Description 1 Choose the same point as the nominal flight plan Choose a point with the shortest distance from the current position Manager No. 3 (n1=18, n2=24, n3=6) No. Heuristic Rule Description 1 Minimum flight time 2 Minimum difference of time from the nominal flight plan 5 Common flight time (e.g., 2200see in a terminal) 6 Common flight time (e.g., 2400see in a terminal) Manager No. 4 (nl=18, n2=30, n3=8) No. Heuristic Rule Description 1 Choose the same path as the nominal flight plan 2 Choose the shortest path 3 Choose a path with the lowest number of passage waypoints 8 Choose a path with the lowest degree of congestion (Degree of congestion of every possible path is recalculated, based on the premodified flight plans of all the airplanes with higher priorities.)

Of course, use of the DS method does not guarantee t h a t the optimal selection can be made in an actual unknown traffic situation, that is different from the situations generated in the off-line build time phase. NeurM networks are often said

Real-Time Decision Support for Air Traffic Management to be bad at extrapolation. However, a satisfactory sub-optimal schedule can be achieved, which is well verified by computer simulation studies, examples of which are shown in Sections 3.5 and 3.6.

Figure 5. Overall configuration of DS2 system

3.5 Simulation results 1 : terminal area TraffiC model. A model of one extended terminal area, including the neighboring cruise sectors, limited to the arrival flow, is assumed, as shown in Figure 6. There are four entry points, located at 200nm from the airport, and two close parallel runways. The total number of intersection nodes is 16, and three flight levels are considered for the cruise section, namely FL310, 330 and 350. Several types of descent trajectories are also considered, and the vertical minimum separation is 1000ft below FL290.

1135

erated in the run time module by the Montecarlo method according to the Poisson distribution, whose average number of aircraft entering the terminal area per hour is given as N (ACPH).

Learning effect. Table 4 shows the growth of mapping knowledge stored in the neural network of Managers 1 to 4 during the learning phase of DS1. "Prediction accuracy" indicates the percentage of times that the neural network generated in each manager by the build time module predicts the best rule correctly in the run time module. For example, 76 trials can predict the best rule of manager No. i correctly, out of 100 problems tried online, based on the neural network stored in Manager No. 1, which is acquired by 1000 samplegenerated problems of the DS1 build time module. Overfitting often occurs for a large number of learning data. So, the learning phase is terminated when 2000 problems have been generated in the DS1 Build Time Module. As for Managers 2 to 4, the learning phase is terminated when 25437 subproblems have been generated, and the prediction accuracies of Managers 2, 3 and 4 reach about 77, 84 and 72 percent respectively at that time. Based on the acquired knowledge of DS1, the learning phase is repeated in DS2. 1000 problems, which produce approximately 50000 subproblems, are generated as learning data in the DS2 build time module, and the average prediction accuracy for Managers 5 to 8 finally reaches about 73 percent. In order to maximize the prediction accuracy of each manager, similar types of generated traffic samples are automatically removed in the build time module, and the parameters for the back propagation algorithm are carefully selected.

Table 4 Growth of mapping knowledge of Managers No.1 to No.4

Figure 6. Terminal area model In-trail longitudinal minimum separation is 5nm. The minimum spacing distances for final approach depend on the weight classes of the aircraft in the landing sequence, due to wake vortices, as given in Table 3. Table 3 Matrix of separation distance minima for final approach trailing aircraft Heavy

Large

preceding

Heavy

4 nm

5 nm

aircraft

Large

3 nm

3 nm

Two aircraft types belonging to different weight classes axe considered, i.e. B747-200 (Heavy) and B737-300 (Large), each of which has a different speed performance. The actual traffic flow is gen-

Generated Tried Prediction Accuracy Problems (and Problems (percent) Subproblems) Manager in BTM in RTM No.1 No.2 No.3 No.4

100 (1871) 500 (7068) 1000(13954) 1500(20112) 2000 (25437)

10 50 100 150 200

20.0 58.0 76.0 88.0 89.5

5.1 19.0 53.5 73.2 77.4

7.5 23.1 59.8 81.7 84.4

5.2 15.8 52.5 69.3 72.3

Final performance. In order to evaluate the effect of DS1 and DS2, the following two schedulers have been prepared. FIX1 uses fixed rules at each Manager, i.e. "FIFO" as Manager No. 1, "nominal runway" as Manager No. 2, "minimum flight time" as Manager No. 3 and "nominal trajectory" as Manager No. 4. FIX2 uses different fixed rules, i.e. "FIFO" as Manager No. 1, "nominal runway" as Manager No. 2, "common flight time of 2200sec" as Manager No. 3 and "trajectory

J. Nogami et al.

1136

with the lowest congestion degree" as Manager No. 4. OPT, described in Table 5, means a suboptimal solution obtained off-line by the combined techniques of nonlinear programming, genetic algorithms and the branch-and-bound method. Tables 5 and 6 and Figure 7 summarize the typical performance for these schedulers. 1000 traffic samples are made by Montecarlo simulation for each traffic situation case and each scheduling algorithm. The time-length of each traffic sample is 4hrs. The average time delay per aircraft Dk for a random traffic sample k (k = 1 , 2 , - - , 1000), which is defined as the sum of the individual aircraft arrival-time delays divided by the number of aircraft in the sample, is calculated. Each performance data in Tables 5 and 6 represents the average delay D, which is defined as the sum of Dk for a random traffic sample k divided by the number of trial samples. 1000

D = Z

Dk

(13)

k=l

Table 5 shows the effect of the traffic flow imbalance and the mix of aircraft types. The following three cases of the traffic flow mix, i.e. East/West (E/W), and the aircraft type mix, i.e. Large/Heavy (L/H), are considered. C a s e 1 L/H: 70/30, E / W : 50/50 (percent)

Case 2 L/H: 50/50, E / W : 50/50 (percent) C a s e 3 L/H: 50/50, E / W : 70/30 for lhr alternating with 30/70 for lhr (percent)

Table 5 Average delay D (min.) performance N(ACPH) FIXl Scheduling FIX2 Algorithm DS1 DS2 OPT

Case 1 60 90 2.48 10.24 2.44 9.93 1..89 8.97 1.78 8.28 1.73 7.95

Case 2 60 90 3.33 12.95 3.28 12.58 2.75 11.54 2.58 10.40 2.44 9.93

Case 3 60 90 4.45 17.06 4.36 16.58 3.70 14.95 3.32 14.05 3.04 13.52

For example, for Case 3, when N is 90 (ACPH), DS1 and DS2 can mitigate congestion delays by 2.1 (min.) and 3.0 (min.) respectively on average, and they can increase arrival throughputs by 7.8 and 10.6 percent respectively on average, as compared with FIXl. Table 5 indicates that DS1 works more effectively when the traffic flows are imbalanced route by route, and DS2 works more effectively when aircraft with different weight categories and speed classes exist together. This means that DS1 can skillfully modify trajectories and flight strategies, and DS2 can skillfully modify landing orders and control air speeds slightly for every well-extracted group of aircraft.

Figure 7 shows the cumulative probability distributions for the average delay per aircraft D~ in a given traffic sample k, with the parameter N, for Case 3. The benefits of the proposed schedulers can clearly be seen for greater traffic densities. This is valid, because longer groups occur in heavy traffic, and long groups can be optimized more efficiently than short ones. For example, when N equals 90 (ACPH), for FIX2, an average delay of 12 (min.) or less is realized for 41 percent of all the traffic samples generated. For that case, on the other hand, the same average delay per aircraft is realized for 57 percent for DS1, and this delay, or less, is realized for 72 percent of all the samples for DS2.

r ~ /'/,.2"

' °'t /I/] ° " t /"! //7

°12 ./'A °oo 0

$

1'0

1'5

Averagedelay peraircraft(miu.)

I

20

Figure 7. Cumulative probability distributions in Case 2 Moreover, an analysis of the correlation between the input and the output of the trained neural networks using partial differentiation has been undertaken to see what kinds of basic attributes have a large impact on a switch of rules in each manager. As an example, in Manager 3, a "nominal flight time" rule is used when the traffic is not heavy, a "minimum flight time" rule is used when it is slightly heavy and a "common flight time" rule is used when it is very heavy. Such rule switching, reflecting the degree of traffic congestion, seems to be understandable. When the traffic flow is overloaded, it is preferable to speed up every aircraft, even if this causes additional fuel consumption, in order to prevent the propagation of delays into the traffic that follows, and it is preferable to assign the same flight time to every aircraft, in order to minimize its conflict possibility. Table 6 Average delay D (min.) performance Emergency No Emergency Missed Event Anomaly Landing Approach FIX1 12.95 13.60 14.83 Scheduling FIX2 12.58 13.19 14.24 Algorithm DS1 11.54 11.94 12.81 DS2 10.40 10.78 11.57

Table 6 shows the robustness of DS1 and DS2 against anomalies. The average entry rate N is

Real-Time Decision Support for Air Traffic Management assumed as 90 (ACPH). The ratio of L / H is 50/50 percent and the ratio of E / W is also 50/50 percent. Two cases, i.e. emergency landing and missed approach, are considered, both of which are assumed to occur once per hour, and compared with the no-anomaly case. It indicates that the DS method, especially DS2, can suppress the divergence of delay even if anomalies do occur.

Htype 1

Htype 2

Htype3

11

11

11

10

10

10

10

10

10

11

11

11

5

10

20

20

10

5

The following three vertical flow types are also compared, each of which is given as a vector of the traffic flow ratio (percent) among three altitude levels, i.e. FL310, 330 and 350.

3.6 Simulation results 2 : single-unit enroute sector [33

Tra]fic model. A model of a single-unit enroute sector is assumed, shown in Figure 8, and consisting of 20 nodes and three flight levels. The diameter of this sector is 400nm. In-trail longitudinal minimum separation is assumed as 15nm. Other assumptions are the same as those in Section 3.5.

1137

Vtype 1 33 33]

[35

Vtype 2 30 35]

[30

Vtype3 40 30]

Table 7 indicates that DS1 and DS2 can work more effectively when the traffic flows are imbalanced either horizontally or vertically; namely they can acquire some knowledge to change routes or altitudes skillfully into a non-congested area for every well-extracted group of aircraft.

Vertletl Flow

i$ i m -~m-------~ FL$15

Figure 8. Enroute sector model

Final performance. In the learning phase, 3000 and 2000 problems tried off-line are generated for, respectively, Managers 1 to 4 and 5 to 8. The average prediction accuracies finally reach 77 and 72 percent, respectively. Table 7 Average delay D (rain.) performance Htype Vtype

1 1 1 2 3 1 2 3 1 1 FIX1 7.05 8.98 9.66 9.47 12.52 Scheduler DS1 5.82 7.35 7.50 7.58 10.25 DS2 5.48 6.96 7.08 6.62 9.29 Table 7 shows the effect of the imbalance of either horizontal or vertical flow distribution, and compares the performance of three schedulers, namely F l X l , DS1 and DS2. F l X l uses the same rules as FIX1 adopted in Section 3.5. In all the data, scheduling is performed 500 times and the performance is averaged. Each item of performance data represents the average delay per aircraft of its departure time from the sector. The average entry rate N into the sector according to the Poisson distribution is assumed as 120 (ACPH), and the ratio of B747-to-B737 is 50 to 50 percent. The following three horizontal flow types are compared, each of which is given as a matrix of the traffic flow ratio (percent) between an entry node, i.e. A1, A2 or A3, and a departure node, i.e. B1, B2, or B3.

4. DECISION S U P P O R T SYSTEM F O R REAL-TIME ATM IN A VAST T R A F F I C N E T W O R K

4.1 Problem formulation Table 8 Decision items and requirements Decision Items 1. aircraft currently in a terminal area - flexible rescheduhng for missed-approach etc. - landing order, landing time, runway allocation 2. aircraft currently in an enroute sector - destination change for airport closure etc. - flow regulation among sectors (auto-tuning of sector capacities) - rerouting and changes in flight levels - speed control 3. aircraft currently awaiting take-off - flight cancellation - permission for taking-off - ground-hold time allocation - rescheduling flight-plan (rerouting, etc.) Requirements 1. Nominal prescheduled flight plans should be maintained as far as possible 2. Total direct operating costs (DOCs) dependent on the additional fuel usages and delays must be made as low as possible In this section, the real-time ATM problem in a global airspace with many airports and airroutes, the so-called "traffic flow management" (TFM) problem, is considered. A fast, intelligent prediction and scheduling algorithm is required to predict the future congestion overload at each element of the vast air traffic network using the information available from radar, satellites, etc., and to determine the best amendments of flight schedules so as to minimize the total delay cost using

1138

J. Nogami et al.

the predicted performance data. Further, the a l gorithm has to have the capability to mitigate disruptions resulting from severe weather or various emergencies, flexibly. The decision items and requirements in this scheduling problem are summarized in Table 8. The control actions for an aircraft which is already airborne must be determined, not only by the scheduler described in Section 4.4, but also based on the parallel distributed cooperation among the individual schedulers corresponding to single-unit sectors, which have already been suggested in Section 3.

listic distribution function, and the equations for calculating the mean air holding time at the terminal entry, mean delay time in the terminal, air holding probability and delay probability of the departure aeroplane were derived. In addition, several ground-holding policies based on these online predictions were proposed, to alleviate costly airborne congestion. 1000 ~-. ~=, ~.

.

.g~

.

.

.

.

i

~0--No Regulation

/

1--E w = 30Sec.

]

2-'Ew = 60Sec. - - - - - 3--Ew = 90Sec.

/ J

' " ....

~.~ Previous research 1

Bianco and Bielli (1993) designed a constrained linear optimization model for flow control among multiple sectors, and several strategies to perform optimal flow-control actions were suggested. Vranas (1992) indicated that several integer programming formulations could be given for the multi-airport ground-holding problem and the influence of airlines' flight networks on the various ground-holding policies was analyzed by means of operations research (OR). Nogami (1995) and Nogami, et al., (1995) suggested an extended integer programming formulation. Several suboptimized strategies to mitigate congestion delays, both in terminal areas and in vast air traffic networks, have been numerically studied, utilizing the combined techniques of the branch-and-bound method together with genetic algorithms. Several factors which have a great influence on the suboptimized flow-control, ground-holding or rerouting strategies have been identified by these OR-like numerical analyses. Detailed observation of suboptimal solutions searched for by means of various OR-like methods would be very useful to acquire refined heuristic strategies for real-time decision support for the TFM activity. These ORlike methods, however, cannot be utilized directly in the real-time ATM problem, because they cannot find a good solution within a practical computational time, and their analyses neglect the uncertainty in predicting weather events.

4.3 Previous research 2

Selecting the correct amounts of control activity such as ground holding time is made difficult because of the uncertainty in predicting weather events that produce congestion delays. Qiao, et al., (1994) indicated that the air traffic flow could be modelled on the assumption that the flight time error of an aeroplane occurred only in the enroute region, whose capacity was assumed to be infinite. Such an error could be given as a specific probab-

° _

%

'

2'0

2's

,o

~,(Aircraft/Hour)

Figure 9. Amount of the average airborne delay (Mean Wait Time) based on the ground holding policy using the value E w (Qiao, et al., 1994) Figure 9, which is taken from the reference (Qiao, et al., 1994), indicates that, as compared with the case where no ground regulation is imposed, the proposed ground-holding policies, where a departing aeroplane is permitted to take off only when the expected value E w about the airborne delay time of the aeroplane is, respectively, below 30, 60, 90 and 120(sec.), can greatly reduce its airborne delay, without increasing its total travel time, which is the sum of its air travel time and its ground-holding time. The value E w is derived by the proposed equations. Detailed assumptions and parameters are given in the reference (Qiao, et al., 1994). This method is excellent in that it has the capability to make fast decisions considering the uncertainty in predicting future traffic situations. However, in the method, ground holding is imposed on a departing aeroplane, on the assumption that every aeroplane which wiU depart later will fly according to its nominally scheduled flight plan. Such an assumption hinders both the accurate prediction of future traffic situations, and the making of optimal decisions throughout the long-term planning horizon. Furthermore, the method is based on an oversimplified traffic model, which neglects airlines' flight connection networks, airroute networks in enroute regions, changes in sector capacities with time, sequencing in terminal areas, timing regulations among take-off and arrival flights, etc., which play important roles in alleviating congestion delays (Vranas, 1992; Nogami, 1995).

Real-Time Decision Support tbr Air TrafficManagement 4.4 Proposed decision-maker

It is necessary to supplement the above shortcomings of both of the methods in Sections 4.2 and 4.3, in order to construct intelligent decision-makers for this TFM problem. Figure 10 shows a proposed system, which consists of a prediction mode and a control mode.

"-

Control Mode

...

Figure 10. Proposed system architecture The prediction mode can make an approximate estimate of the future traffic performance, such as the distribution pattern of future arrival demands at an airport, or the delay probability of a flight, assuming as its inputs the current traffic situation, the information about weather events and the specific scheduling rules selected by the control mode. Fast, lookahead prediction is needed, so the knowledge representing the relationships between such inputs into this mode, and a large number of data on the future traffic performance, are acquired inductively from various learning data obtained through off-line simulations and modelbased mathematical analyses. A hierarchical attribute graph is used as the knowledge representation scheme, whose nodes consist of • a set of discrete events which will occur throughout the planning horizon, • the scheduling rules selected by the control mode, • the useful attributes representing the structure of the &irroute and flight networks, current traffic situations, forecast data on weather events, etc., and so on, and whose arcs consist of the mapping relationships described by nonlinear functions among attribute nodes. Several algorithms for the inductive acquisition of the refined graph structure are currently under investigation (Riolo, 1991; Sato, et at., 1994). The control mode is based on cooperation between distributed decision makers such as the

1139

flight schedule planner, the ground holding manager, the rerouting manager, the flow control manager, the negotiation manager, etc. For example, the negotiation manager has the capability to arbitrate between the conflicting requirements of multiple agents, such as airline planners, ATC controllers, etc. In each manager, many a priori decision candidates are prepared, from which one is selected at each decision point, on the basis of the instantaneous flight and airroute network situations and the performance data on congestion delays, etc., forecast by the prediction mode. For example, either permission to take-off, or a request for ground holding, is determined for an aeroplane waiting for its take-off. If the latter is selected, the allocation of appropriate gate-holding times must be decided further. Such decision making is so difficult that a human cannot implement sufficient knowledge to pin-point a single best candidate. Therefore, the scheduling should be made according to a time line, and at each future decision point, one decision candidate must be selected. Overall

Scheduling

Process

]

Rea|Time

|

Line

~ !

I

De¢IsloD Po|nt No.I

No,I-I DOC ( )

No.1-2 DO(: ( )

No.2

No I-3 DOC ( )

I

.............

.......

I~""'i I1~'", ~ I1~'"~(~ I Schedullps ~c:hedll In i Into some ff0tere

/

I~to some fots~

.......

L

Figure 11. Recursive scheduling strategy Figure 11 shows this process in more detail. The prediction of the accurate performance data on the assumption that a specific candidate is to be selected will be required, in order to select the correct candidate. But to obtain this performance data, a "virtual scheduling" from the decision point into some point in the future must be made, corresponding to the first move of each of these candidates, which gives birth to another level of scheduling problem. If this new scheduling is badly made, the performance data obtained may be unreliable, which degrades the selection optimality at the upper level. Therefore, the lower level must also make a sufficiently good schedule, which requires another lower-level scheduling for prediction at each decision point, and so on. So, in order to make optimal decisions, the recursion

1140

J. Nogami et al.

will continue almost infinitely, which results in the so-called "combinatorial explosion". To avoid this problem, the proposed system has the capability to control this recursion, and cut the recursion at the specified level by approximating the optimal decision making process by using heuristic rules. That is, the selection of decision candidates at every decision point which follows the decision point at which the recursive process is cut off, is made by a priori heuristic rules. The dynamic scheduling is applicable, instead of using fixed rules. The prediction mode is activated to make fast-time predictions about the future performance data, using the decision point where the recursion is cut off as the starting point. This strategy, of course, degrades the reliability of the performance data obtained. But, the more skillful the cut-off control of the recursive process, the more accurate are the estimates of the performance data in the prediction mode, and better heuristic rules used in the control mode would be able to aid the proposed method in making better decisions. An analogous recursive scheduling is used in the reference (Nakasuka, et al., 1994) and good results have been reported. Simulation studies on this system are currently being performed.

5. ENHANCEMENT OF DYNAMIC SCHEDULING Enhancement of dynamic scheduling is essential for further improvement of the capability of the proposed schedulers described in Sections 3 and 4. When it has been sufficiently improved, the whole schedule could be drawn up by the dynamic scheduling method alone, shown in Figure 12, without any hierarchy or iterative process.

Raw Data Set J on Detailed J Traffic Situation[

:~:~;:~."

l

• Refined • [Mapping [

IX.o,,~p

I

[ Human | a priori i Heuflstk

I Sch~d.li~

. -

II S~hed.,,~ u.,., II I s,.,..~. I Problem Space

Solution Space

Figure 12. Concept of the enhanced dynamic scheduling method To do this, the following problems must be solved, which have been identified as fundamental problems in the field of artificial intelligence. 1. The most important attributes must be utilized for rule selection, which is a difficult

problem, far beyond what a human can do. The automatic generation of useful attributes from raw data on the traffic situations is required for solving this problem. 2. The automatic creation of new, better, scheduling rules is required. Minimization of the number of rules is also needed. The direct observation of sub-optimal solutions searched for exhaustively, utilization of structured classifiers (Iba and Higuchi, 1992), application of genetic algorithms, learning from negative instances of traffic situations where the best scheduling rule cannot be estimated correctly during real operation, and so forth, may be useful for the evolution and generalization of heuristic strategies. 3. Neural networks are not good at extrapolation and representing the mapping relationships explicitly. Decision trees are not good at nonlinear classification. A refined mapping scheme to compensate for both shortcomings is urgently required. Various other machine-learning techniques, such as case based reasoning (Levin and Fearnsides, 1994) or reinforcement learning, should also be considered. 4. An interactive decision support between human managers and machine systems is required, so that human managers can intervene in the generation of useful traffic attributes and novel, improved scheduling strategies, and the regulation of multiple agents' conflicting requirements. A breakthrough in solving these problems is currently being pursued.

6. CONCLUSIONS An automated decision support system for realtime, efficient ATM is proposed, which consists of several distributed decision-makers. Large-scale scheduling problems of this kind are so complicated that man alone cannot implement enough intelligence into each decision-maker. Therefore, the empirical knowledge extracted from ATC controllers, TFM managers, etc., is not sufficient to make optimal decisions, and to deal with sudden anomalies. In the proposed system, this insufficiency is compensated for with the internal simulations for prediction, as well as the dynamic selection of scheduling rules using neural networks. The simulation results verify the versatility and flexibility of the proposed scheduler for the realtime ATM problem within a single-unit sector.

Real-Time Decision Supportfor Air Traffic Management

1141

An intelligent decision support system for realtime ATM in a global airspace is also being addressed in future simulation studies. Aspects which will be addressed include:

Nakasuka, S. and Yoshida, T. (1992). Dynamic Scheduling System Utilizing Machine Learning as a Knowledge Acquisition Tool. Int. J. Production Research, Vol.30, pp.411-431.

• Application of the proposed dynamic scheduling technique to existing methods of handling heavy traffic arrival flows at major terminals (Neuman and Erzberger, 1990; Synnestvedt, et al., 1995).

Nakasuka, S., Kato, H. and Ninomiya, T. (1994). Intelligent Scheduler with Distributed Decision Makers for Managing OTV Network Operations. Trans. Japan Soc. Aero. Space Sci., Vol.37, No.l16, pp.113-124.

• A multiobjective optimization technique, to regulate the conflicting requirements of multiple agents (minimized probability of conflict, minimized fuel usage, maximized throughput, etc.).

Neuman, F. and Erzberger, H. (1990). Analysis of Sequencing and Scheduling Methods for Arrival Traffic. NASA T M 102795.

• The influence of the interaction between the flow-control actions in individual sectors, in relation to the global and individual sector capacities. • The inductive acquisition of a refined spatiotemporal attribute graph, to predict the time-varying traffic evolution as accurately as possible. • Enhancement of the proposed architecture, to accommodate a "free-flight" mode that could generate conflict-free trajectories in real time.

REFERENCES Bianco, L. and Bielli, M. (1993). System Aspects and Optimization Models in ATC Planning. In: Large Scale Computation and Information Processing in A T C (Springer-Verlag), pp.47-99. Erzberger, H., Davis, T.J. and Green, S. (1994). Design of Center-TRACON Automation System. In: Proc. of the A G A R D Guidance and Control Panel, 5fith Symposium on Machine Intelligence in A&. Grefenstette, J.J., Ramsey, C.L. and Schultz, A.C. (1990). Learning Sequential Decision Rules using Simulation Models and Competition. Machine Learning. Iba, H. and Higuchi, T. (1992). Evolutionary Learning of Predatory Behaviors Based on Structured Classifiers. ETR-TR92-34, SAB92, MIT Press. Levin, K.M. and Fearnsides, J.J. (1994). Advances in Development Capabilities for Intelligent Air Traffic Management Systems. In: Proc. of the A G A R D Guidance and Control Panel, 56th Symposium on Machine Intelligence in Air.

Nogami, J. (1995). Concept of the Future ATM and its Real-Time Decision Support utilizing Machine Learning. M.S. Thesis, Univ. of Tokyo, in Japanese. Nogami, J., Nakasuka, S. and Tanabe, T. (1995). Concept of the Future ATM and Sensitivity Analysis on the Propagation of Congestion Delay through Airlines' Flight Network. Proc. of the 33rd Aircraft Symposium, pp.677-680, in Japanese. Qiao, M., Morikawa, H. and Mizumachi, M. (1994). Real-Time Air Traffic Flow Management: Prediction of Terminal Conditions and Evaluation of Ground Holding Policies. IEICE Technical Report, Vol.94, No.187, Sane94-23, pp.9-16, in Japanese. Riolo, R.L. (1991). Lookahead Planning and Latent Learning in Classifier System. From animals to animats, pp.316-326, MIT Press. Sato, Y., Hatano, S., Hatano, H. and Furuya, T. (1994). Lookahead Planning and CoEvolution in Recurrent Neural Networks. Proc. of ICEC'94-Orlando, pp.764-769. Synnestvedt, R.G., Swenson, H. and Erzberger, H. (1995). Scheduling Logic for Miles-InTrail Traffic Management. NASA T M 4700. Tobias, L. and Scoggins, J.L. (1986). TimeBased Air Traffic Management using Expert Systems. NASA TM 88234. Visser, H.G. (1991). Terminal Air Traffic Management. Aerospace Sci., Vol.28, pp.323368. Volckers, U. and Schenk, H.D. (1989). Operation Experience with the Computer Oriented Metering Planning and Advisory System (COMPAS) at Frankfurt, Germany. AIAA Paper 89-3627. Vranas, P.B. (1992). The Multi-Airport Ground Holding Problem in Air Traffic Control. Ph.D. Thesis, MIT.