- Email: [email protected]
An Agent-based Approach to Automated Merge 4D Arrival Trajectories in Busy Terminal Maneuvering Area
Available online at www.sciencedirect.com
ScienceDirect Procedia Engineering 99 (2015) 233 – 243
“APISAT2014”, 2014 Asia-Pacific International Symposium on Aerospace Technology, APISAT2014
An agent-based approach to automated merge 4D arrival trajectories in busy terminal maneuvering area LIANG Mana,b,* a
College of Civil Aviation,Nanjing University of Aeronautics and Astronautics, Nanjing 210016, R.P China b College of Air Traffic Management, Civil Aviation University of China, Tianjin 300300, R.P China
Abstract This paper presents a new approach for automated merging multiple aircraft flows in a busy Terminal Maneuvering Area (TMA). This work is motivated by the current overloaded airspace in the Beijing Capital international airport, the highlighted delay in the China air transportation and the need of more efficient methods to help air traffic controllers. Present research consists of a new approach to optimize a set of aircraft planned to land at a given airport based on multiagents technique, which is automated generating comprehensive sequencing plan and conflicts-free trajectories. The architecture of the system is based on a Point Merge (PM) route structure to enable expediting or delaying aircraft while staying on lateral navigation mode. The whole system is designed by two main models to manage the process of arriving flows: Sequencing leg and Link. And then, 4 kinds of agents are designed to support the implementation of this automated system: aircraft agent, flow manager agent, conflict detection and resolution agent and 4D trajectory planning agent. We consider the 4D trajectory-based operation situation in TMA and apply different wake turbulence constraint in these two models. Finally, a conclusion is made and future work is outlined. © Published by Elsevier Ltd. This © 2015 2014The TheAuthors. Authors. Published by Elsevier Ltd.is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA). Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA)
Keywords: Air traffic management; 4D trajectory planning; terminal control area; agent modeling
* Corresponding author. Tel.: +86-022-24092445; fax: +86-022-24092431. E-mail address: [email protected]
1877-7058 © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA)
doi:10.1016/j.proeng.2014.12.531
234
Liang Man / Procedia Engineering 99 (2015) 233 – 243
1. Introduction Flight delay is now a very serious and widespread problem in China. With rapid economic growth, the demands for air services in China has significantly increased in the past few decades, the number of aircraft movements grew at an average rate of 11.1% per annum between 2008 and 2012 [3]. Beijing Capital international airport is the first busiest airport in Asia and the second busiest airport in the world, with about 557 thousand aircraft movements in 2012 and 4% traffic increase according to airport statistics and data published by Airports Council International. Therefore, air traffic booming is placing an enormous pressure on the Beijing airport system. According to Flight-Stats, in 2013 Beijing airport ranks dead last among the world’s top 35, with fully 82% of flights failing to leave on time. Second worst was Shanghai, at 71%. It’s obvious that the growth of air traffic has already exceeded Beijing airport system capacity, as a result a serious flight delay in Beijing airport occurred. To alleviate this delay problem and increase the airport capacity, in 2010 Chinese civil aviation Air Traffic Management Bureau (ATMB) launched a special project of reducing flight delay in order to enhance the air passenger satisfaction. And then, the high level airspace in the north China was centralized and concentrated into one single organization named North-ATMB area control center (NATM-ACC), which provided the en-route control service for the flights inbound or outbound Beijing capital airport, so as to reduce the unnecessary coordination between air sectors in different regions, to manage the flow from a strategic level. Parallel runway dependency approach mode was implemented in Beijing Capital airport to increase the runway capacity. However, in 2013 the delay problem is still becoming increasingly acute. In fact, the bottleneck of Beijing congestion problem is the terminal control area or Terminal Maneuvering Area (TMA), which is a designated area of controlled airspace surrounding a major airport where there is a high volume of traffic. Due to insufficient number of air traffic controllers (ATCO) and relatively high traffic flows, the North-ATMB terminal control (NATM-TMA) system is close to collapse at the peak time. In order to solve these problems and avoid a worsening situation, a higher level of automation in separation assurance is considered to change the current ATCO-assured separation system in NATMTMA. Automated control of arrival flows, namely automated arrival scheduling, sequencing and conflict resolution, has been a topic of interest for a few decades now. [8] describes a design of an automated air traffic control system based on a hierarchy of advisory tools for controllers, referred to as the Center-TRACON Automation System (CTAS). Later, CTAS was installed and evaluated at the Denver area and Dallas/Ft.Worth area air traffic control facilities [6, 10]. Advances in navigation, communication and surveillance provide new emerging concepts to solve this arrival operational problem. The introduction of area navigation (RNAV, P-RNAV) makes it possible to define a new route structure to revisit the merging of arrival flows. The data-link communication provides us a convenient way to share the trajectory information not only between air-to-ground and also air to air. The Automatic Dependent SurveillanceBroadcast (ADS-B) provides us a new way of surveillance for tracking aircraft via Global Navigation Satellite Systems (GNSS). Thus [9, 11, 7] present an Advanced Airspace Concept (AAC) for the Next Generation Air Traffic Control System, it can handle large increases in capacity and throughput in the way of automating the monitoring and control of separation by using a down-link to share trajectory information between the airborne. On the other hand, several recent analysis studies have investigated in amelioration the arrival management by using airborne Required Time-of-Arrival (RTA) functionality in modern commercial jets. Current RTA control functionality adjusts and regulates the aircraft’s speed along the trajectory in order to arrive at a specified way-point at a specified time, thus improving the predictability of the arriving aircraft [14, 15, 16]. It is clear that the use of airborne RTA functionality is a key enabler to Trajectory Based Operations in TMA, and a time-based spacing approach is more suitable for realization of an automated arrival management. From an intelligent control point of view, new methods from Artificial Intelligence such as multi-agent system (MAS) have been introduced as well to automate air traffic flow management. [24] takes the aircraft as agents that are competing for the limited resource of airspace, applies the Monotonic Concession Protocol (MCP) negotiation techniques to resolve air traffic conflicts, and the result shows that the solutions generated by these methods are more efficient than those created by traditional methods or alternative non-cooperative solutions. While, in [23] research, an agent is associated with a fix (a specific location in 2D space) and its action consists of setting the
Liang Man / Procedia Engineering 99 (2015) 233 – 243
235
separation required among the aircraft going though that fix. Agents use reinforcement learning to set this separation and their actions speed up or slow down traffic to manage congestion. The FACET (an air traffic flow simulator developed at NASA and used extensively by the FAA and industry) based results show that agents receiving personalized rewards reduce congestion by up to 45% over agents receiving a global reward and by up to 67% over a current industry approach. In TMA, the aircraft arrival management is considered as decentralized, distributed and dynamic, collaboration is necessary among different controllers, the environment factors (like capacity, traffic flow, weather) change high dynamically. Therefore, MAS can be suitable for aircraft arrival management. In this paper, an agent-based approach to automatically merge 4D trajectories in TMA is being studied to assist air traffic controllers. This paper is structured as follows: Section 2 we present the merge point route structure. Section 3 introduces an applicable Agent-based framework, with details in the agent-based modeling approach, agent selection, agent actions, as well as some agent implementation. Section 4 outlines the future works. 2. Point merge route structure Point Merge (PM) is a systematized method for sequencing arrival flows developed by the EUROCONTROL Experimental Center in 2006. Instead of using heading vector, PM only uses speed control and "direct to" instruction to guide the aircraft to merging point, and it has two clear steps to execute them one by one. Its special route structure and procedures provides a potential approach to realize an automated separation assurance to the arrival flow under high traffic load situation [13]. As illustrated in the Figure 1, the route structure supporting PM is denoted Point Merge System (PMS). A PMS may be defined as an RNAV STAR (Regional Navigation Standard Arrival Route), transition or initial approach procedure, or a portion thereof, and is characterized by the following features: x A single point, denoted Merge point, is used for traffic integration; x Pre-defined legs, denoted Sequencing legs, iso-distance and equi-distance from the Merge point, are dedicated to path stretching/shortening for each inbound flow. These legs shall be separated by design vertically, laterally or both. Sequencing leg Entry point Entry point
Path envelop
Merge point
Airport
Fig. 1. Structure of the Point Merge System.
An independent analysis conducted by NATS established that, by using Point Merge, airlines landing at Dublin Airport in 2013 reduced their fuel requirement by 19.1%.per flight. It also found that aircraft reduced the length of the flight by 11.3 miles, a 17% saving. Point Merge also provided savings of 23,500 tons of CO2, representing a 19% reduction [12]. Besides the economy benefits, on its implementation in Paris-CDG, some other operational and safety advantages was found, such as less instructions issued by ATCs to pilots which means lower workload, sequence order easily changed to improve runway throughput which is hard to realize by traditional radar control methods, and improved predictability which can provide a precise information about the arrival time. Referring to the statement mentioned above, we can recognize that PM route structure is easier to change the aircraft approach
236
Liang Man / Procedia Engineering 99 (2015) 233 – 243
sequence and very suitable for realizing an automated arrival management, because the aircraft actions can be clearly divided into several individually steps which is easily realized by computer program, so in this research we use PM route structure as a base for building a multi-agent system. 3. Agent modeling for PM-based aircraft arrival management An agent is something that acts in an environment. Agents are considered as social, because they cooperate with humans or other agents in order to achieve their tasks. In fact, agent methods are a natural tool to help automatic existing air traffic systems, and moreover recent researches in Air Traffic Flow Management (ATFM) have successfully combined Reinforcement Learning (RL) and Multi-agent techniques to the ATFM. [23, 22] developed a distributed agent-based ATFM for NAS in United State. [4, 5] used learning agents to solve the air traffic flow management problem and the airspace sectors congestion problem in Brazil. In this paper, the discussion of the agent-based modeling for PM based aircraft arrival management is organized into four subsections. The first one makes a problem analysis, the second one discusses the overall modeling framework, and the third one presents the conflict detection and resolution agent, followed by the forth one trajectory planning agent and flow manager agent. 3.1. Problem analysis The arriving aircraft from different directions are firstly separated into two groups, and then each group follows one sequencing leg. On the same sequencing leg, all the aircraft maintain the same flight level and keep spacing automatically in horizontal distance by speed adjustment. After that, in consider of the fairness and the operation efficiency in the whole terminal area, all the arriving aircraft are sequenced rationally, and then when the first approaching aircraft leaves the sequencing leg and directs to the merge point, the second approaching aircraft keeps flying on the sequencing leg until there is enough spacing between two of them, at that moment, it can leave the sequencing leg and direct to the merge point, the rest of the aircraft will follow the similar procedure. During the process of direct to the merge point, all the aircraft refer to their flight performance to fly in a continuous descend profile. The time to descent can be issued by ATCOs suggested by decision support software, and the process of descent is managed by the airborne flight management system. Conflict Detection and Resolution (CD&R) is considered in a 4D trajectory based dynamic operation situation. When arrival flow of runway reaches to a defined level named maximum available capacity notated by W , then delay is happened, which will propagate back to the PM system. In consider that once aircraft leave the Sequencing leg they can only direct to merge point without any possibility of holding or stopping or changing path, therefore we have to dynamically modify the flow rate at the Merge point or even Entry point in order to maintain a smooth traffic flow. Therefore, when the delay is heavy we can decrease the rate of merge, when there is no delay we can increase the rate of merge. The characteristics of the problem are as follows: x All the approaching aircraft can only enter the PM system via 2 entry points. x Each aircraft j must be well spaced on the Sequencing leg according to distance-based wake turbulence separation standard issued by ICAO. x Each aircraft j on the Sequencing leg maintains a flight level, and the magnitude of speed change is limited to r20% . x Each aircraft j on the Sequencing leg has an associated turning time based on its preceding aircraft in the whole sequence. x Each aircraft j must be well spaced on the process of direct to Merge point according to time-based wake turbulence separation standard issued by ICAO. x On the process of merging, each aircraft j can control its descent profile according to the aircraft performance and cannot change path during the process of merge. x CD&R is considered in a 4D trajectory based dynamic operation situation. x The whole system arrival flow can dynamically adapt to the delay. The notations used in this research are as follows:
Liang Man / Procedia Engineering 99 (2015) 233 – 243
Nomenclature
* j
tej j tRTA ttnj tdtj 'tm 'te hj
a set of aircraft aircraft j , j * , entry time of aircraft j required time of arrival of aircraft j at Merge point time of turning direct to merge point, leaving the Sequencing leg time of commencing descent during the process of direct to Merge point required time interval between two successive aircraft at the Merge point required time interval between two successive aircraft at the Entry point flight level or altitude of aircraft j
3.2. Overall framework A new A/C coming to the PM system
NO
Is there any delay for the new A/C ? YES Sequencing leg model
1.Choose one of the entry point to the PM system 2.Calculate the sequencing landing place for the new A/C, and the Required time of arrival(RTA) at merge point 3.Conflict detect based on 4D trajectory position in consider of distance based wake turbulence separation 4.Conflict resolution: speed control
Update all the A/C position, sequencing and RTA information in time interval
NO
Is the previous A/C in the sequencing leaving the Sequencing leg? YES Link model
1.Calculate the Possible turning time for the new A/C 2.Calculate the best descent-commencing time in order to execute a continuous descent approach in consider of aircraft performance 3.Conflict detect based on 4D trajectory position in consideration of timebased wake turbulence separation on merge point 4.Conflict resolution: speed control
Fly a continuous descent approach profile Arrive at Merge point
Fig. 2. Flow chart for multi-agent modelling of aircraft arriving in PM system.
The system includes two main models to manage the process of arrival flows, showed in Figure 2: Sequencing leg model and Link model. Sequencing leg part is to keep the arriving aircraft in each arc in an orderly and conflictfree flow by a speed change, at the same time it can dynamically prolong or shorten the fly distance of aircraft on the arc according to the flow. Link part is to decide the sequence that aircraft can leave leg and direct to merge point. Due to the high combinatory induced by such a problem, an optimization algorithm can be developed in order to calculate the landing sequence, and then propose to each aircraft the moment to leave the sequencing leg. The optimization criteria are based on the minimum make-span of sequence. Besides, a descent planning is to
237
238
Liang Man / Procedia Engineering 99 (2015) 233 – 243
automatically control the altitude change, which takes into consideration the execution of Continuous Descent Approaches (CDA). Agents organization framework is the fundamental mechanisms to support an effective behavior in real-work, dynamic environment. In this research, we design 4 types of agents to support the two models, as showed in figure 3, they are Trajectory conflict detection and resolution agent, Flow manager agent, 4D trajectory planning agent and Arriving aircraft agent. The role of aircraft agent is to manage the arriving aircraft. For each aircraft, its trajectory contains the information below: x Time slot x Aircraft number (odd number for one sequencing leg, even number for another sequencing leg) x Current position ( x , y ) x Current subsystem (Sequencing leg, Link) x Current flight level x Current airspeed x Wake turbulence category x Entry time to PM system x Fly routes x Landing sequence position x Required time of arrival at Merge point Aircraft agent exchanges the information dynamically with other three agents. In our case, the path change and sequence information are exchanged with 4D trajectory planning agent, Entry time and RTA are exchanged with Flow manager agent, safely spacing is supported by Trajectory conflict detection and resolution agent. 4D trajectory planning agent optimizes the landing sequence and path planning; its optimized results will include the turning time from sequencing leg to Merge point, the descent commencing time during the process to Merge point and the landing position for each arriving aircraft. Flow manager agent is communicating directly with the environment, it can react with delay and congestion situation in the adjacent sectors and the airport, and through the ways of controlling the passing time at entry points and Merge point, the flow manager agent can speed up and slow down the arriving aircraft by a specific policy, so that the global performance can reach a better level. Trajectory conflict detection and resolution agent dynamically detects the conflicts and solves the conflicts, and it is the base of the whole PM system. Sequencing leg model
Link model
Concept-level Agent-level
CD: Multi-map-based dynamic detection CR: speed control(20% variation)
Arriving A/C
Trajectory Conflict Detection and Resolution(CD&R) agent
ng aci
sp Arriving A/C A/C agents Arriving A/C Call-sign Position (X,Y) Flight level Airspeed (IAS) Wake turbulence category Entry time to PM system Fly route RTA at Merge point
Flow manager agent
Entry time RTA
P Seq ath c uen hang ce e pla ce
4D Trajectory planning agent
Optimized Sequencing planning (the landing sequence) Path changing planning (Turning time from Sequencing Leg to Merge Point; Descent commencing time in Link model)
Fig. 3. Overall agent modeling architecture.
Delay
Environment
Congestion
Control the A/C entry time Control the A/C RTA at Merge point
239
Liang Man / Procedia Engineering 99 (2015) 233 – 243
3.3. Conflict detection and resolution agent Conflict detection and resolution agent is the most import element in this agent-based system. We consider the 4D trajectory-based operation situation in TMA, all the aircraft will merge to the same point at the same altitude, therefore in this PM system, we can simplify the conflicts into two categories: catch-up type and merging type. A catch-up conflict occurs when there are two aircraft following the same Sequencing leg and the trailing aircraft is speedy enough to catch the leading aircraft which will induce a safety distance infringement. Obviously, a merging conflict occurs when aircraft pass through the Merge point. According to different fly phrases on the PM system, we can classify the conflict detection in three situations, see Figure 4. Sequencing leg Keep fly until L2-L1=SD a
Sequencing leg Entry point Entry point
b
Entry point Entry point
a
b
67
a
Sequencing leg Entry point
Entry point
6'
La
b
Lb Merge point
Merge point
Merge point Situation A
Situation B
Situation C
Fig. 4. Conflict detection (a) situation A; (b) situation B; (c) situation C.
Firstly, in situation A, the aircraft fly on the Sequencing leg, all of them maintain the same flight level (or altitude) on each Sequencing leg. Here distance-based wake turbulence separation is considered in order to keep the aircraft safe. For any two successive aircraft passing the same Entry point, we need only to compare the longitudinal distance between two successive aircraft to detect the conflicts. Supposing that aircraft a is following aircraft b , the required distance between them which can be notated as SD , should not be less than the ICAO minimum wake vortex separation (DWVS) standard defined by the Air Traffic Management - ICAO DOC 4444, see table 1. tea is the entry time of aircraft a , teb is the entry time of aircraft b , in consider of the delay on airport and congestion on the sector, we have to control the entry time between a and b , then:
SD t DWVS
(1)
tea teb t 'te
a, b *
(2)
Table 1. ICAO minimum wake turbulence separation standard. Trailing aircraft Leading aircraft Heavy Medium Light
Separation in distance (NM)
Separation in time (s)
Heavy
Medium
Light
Heavy
Medium
Light
4
5
6
105
131
158
3
4
79
105
3
79
Secondly, in situation B, when one aircraft is turning to the Merge point, then the next aircraft to turn is determined by landing sequence which is decided by the Trajectory planning agent. During merge process, all the arriving aircraft must direct to the Merge point one by one, and any two aircraft can not turn to the Merge point simultaneously. Here we suppose that aircraft a is still following aircraft b in the landing sequence but they are on
240
Liang Man / Procedia Engineering 99 (2015) 233 – 243
the different Sequencing legs, so aircraft a and aircraft b have vertical separation, then when aircraft b is turning to the merge point, aircraft a must keep flying on its Sequencing leg until ttna in which moment a distance SM is not less than DWVS, this constraint ensures that there is always horizontal separation between any two successive aircraft descending. If we suppose that the distance from aircraft a to the Merge point is La , the distance from aircraft b to the Merge point is Lb , then the SM is the differential distance between La and Lb , therefore:
SM = La Lb
a, b *
(3)
SM t DWVS (4) If aircraft a and aircraft b are on the same sequencing leg, then when b is turning, there may be a minimum separation infringement between a and b , so in this case, we have to calculate the distance between this two aircraft D(a, b) , then: D(a, b) t DWVS
a, b *
(5) Thirdly, in situation C, when the aircraft approach the Merge point, at tdtj they commence descent, just as mentioned before, finally all of them must reach the same altitude at Merge point. In order to remove conflicts at merging point and consider of increasing the capacity of runway in strong headwind condition, we use time-based separation to keep aircraft safe, and during merging process, we use speed control to the trailing aircraft in each pair of two successive aircraft to make sure that there will be continuous time-based separation between them. So merging conflict detection is applied during this process to make sure that with the time passes, the RTA of the aircraft is well separated at the Merge point. Referring to [17], the time separations is equivalent to the time that is required to fly the actual ICAO distance separations under no wind conditions. The time separation are evaluated for a common reference ground speed (GSr) for the worst wake turbulence effecting condition on landing, A 137 kt is selected after examination of engine performance database that have been found to be representative of the actual distribution of the aircraft according to the different weight classes that are defined to characterize the ICAO separation standards. So the corresponding time-based separation (TWVS) are enumerated in table 1. In the case where a 2.5 NM radar separation is applied, the corresponding time interval is 66 seconds. Again, we suppose that a b is tRTA , then: aircraft a follows aircraft b , and denote that ST is the time interval between tRTA a b ST = tRTA tRTA
ST t TWVS ST t 'tm
a, b *
(6)
(7)
(8) All statement mentioned above is considered in a static situation, in our research we focus on the dynamic conflict detection for 4D trajectory, so in the phrase of implementation, a way of converting the continuous trajectory problem into discrete approximation problem should be considered, here a "snapshot" method can be adopted to capture the trajectory spatial information. In [20, 18, 19], the conflict detection is based on a Spatial Data Structure (SDS), avoiding non-efficient pairwise trajectory comparisons, and using a simplified wake vortex modeling through 4D tubes to detect time-based separation infringements between aircraft. Here, in our research, a snapshot mode is applied to dynamically capture the arrival flows over time, a Multi-Map Structure (MMS), also
Liang Man / Procedia Engineering 99 (2015) 233 – 243
241
named multi-hash, which is a generalization of a map or associative array abstract data type in which more than one value may be associated with and returned for a given key, is used to store the 4D trajectories information with multiple events for the conflict detection problem†, see Figure 5.
Fig. 5. Multi-map structure with events.
We choose “time slot” as key, events with different aircraft numbers having the same time slot are stored with the same key. When there are more than 1 event in the same time slot (key), we can choose combination of 2 events from that slot and check whether each of them has conflict or not. If selected combination of aircraft is on the Sequencing leg subsystem, then a catch-up conflict detection is applied, we need only calculate the actual distance between two successive aircraft on the same Sequencing leg, if selected combination of aircraft is on the Link subsystem, then a merge conflict detection is applied, we calculate the RTA at the merge point for each aircraft, and compare the actual time interval with the ICAO time based separation minimum. For the part of conflict resolution, in this research, speed control is the only tactical conflict resolution method in the PM system, and only 20% of speed change is allowed for each aircraft, thus in the whole PM system, the sequencing leg is very important, because it enables expanding or shortening the aircraft fly distance. 3.4. Trajectory planning agent and flow manager agent Trajectory planning agent mainly solves two kinds of problems: to decide the optimized landing sequence and to make path changing planning. Path changing planning mainly refers to the turning time to Merge point and descent commencing time during direct to Merge point. In order to realize this function, trajectory planning agent must communicate with CD&R agent and aircraft agent. As stated previously, all the aircraft on the Sequencing leg will turn to the Merge point one by one, so at each time the trajectory planning agent just focus on one aircraft which is ready to turn. For example, aircraft i is changing its “current system” information from “sequencing leg” to “link” which means it is turning to the Merge point, the trajectory planning agent will take aircraft i 1 as the active aircraft ready to turn, by communicating with the CD&R agent, when there is a sufficient distance SW , then the trajectory planning agent will save the time ttni 1 immediately and transfer ttni 1 to aircraft agent to execute path changing. A descent planning is following the same philosophy, the trajectory planning agent will focus only on the descent commencing time tdti 1 , it will calculate the required distance or time for a continuous descent approach (CDA) in consideration of aircraft performance, and then transfer the parameters to aircraft agent to execute the descent profile. The landing sequence problem has been studied in a lot of papers, [2] solved the static aircraft landing problem by a mixed-integer zero-one formulation together with a population heuristic algorithm. [21, 1] developed Dynamic programming scheduling algorithm with Constrained Position Shifting (CPS) or Maximum Position Shift (MPS)
†
Thanks for the ideas given by Zuniga C.A. and Hidenori CHIDA
242
Liang Man / Procedia Engineering 99 (2015) 233 – 243
constraints. In our research, we focus on more actual and dynamic traffic situation. Because with PM system it is much easier to shift the aircraft sequence, and in the Chinese busy TMA normally more than 90% commercial aircraft belong to medium wake turbulence category, and the rate of arriving aircraft in the peak hour is normally less than 60 aircraft per hour, so we have more flexibility in the aircraft position shifting. However, from the airlines point of view, the fairness must also be considered, so for those aircraft who belong to the same wake turbulence category First come first service policy will be applied, then if one aircraft can change its position, the maximum position shift should remain limited. The algorithm chosen in our research will aim to find the minimum makespan for a set of aircraft 1, 2,3,..., n , so the objective function will be: n
j T = Min ¦(tRTA tej )
j*
j =1
(9)
T is the makespan of arriving aircraft, with different CPS network, there will be different results. In our case, we can apply the dynamic programming method introduced by [1] or evolutionary method as Genetic algorithm to solve this problem. Flow manager agent is designed as a learning agent in our PM system, placed on the Merge point and it is used for control the whole system performance. In our research, we focus the system environment on the delay in runway and the congestion in the two neighbor sectors from which aircraft are transferred to the entry points of the PM system. When delay or congestion happened, the bottleneck may be on the runway, or maybe in the TMA sectors. If the runway is reaching it capacity firstly, then we need reduce the merging rate on the Merge point, if one of the TMA sectors is congested, then we can increase the rate of arriving at the corresponding entry point in PM system. Besides, we can also set different arriving rate ratio between these two entry points to balance the congestion severity in both sectors. According to [23], this agent action is to learn the "Miles in Trail (MIT)", in our cases, the flow manager agent action is to learn the "Times in Trail (TIT)", when flow manager agent sets the TIT value to 'tm for Merge point fix, aircraft going towards Merge point are instructed to line up and keep 'tm times of interval (we suppose aircraft will always keep a safe distance from each other regardless the value of 'tm , higher value of 'tm means slow down the flow, lower value of 'tm means speed up the flow. The similar policy will apply to those two entry points. In [23, 4], Reinforcement learning (RL) was successfully applied to control the traffic flow, see figure 6 , design an efficient reward function and system global performance evaluation algorithm are both important for flow manager agent. Because of complexity of this problem, in this paper, we will not discuss it. Flow manager agent
state
actions
rewards
(Time in trails at the Merge point)
(Delay Congestions)
Environment
Fig. 6. Reinforcement learning in air traffic flow management.
4. Conclusion and future works An agent-based approach to automated merge 4D arrival trajectories in busy Terminal Maneuvering Area has been proposed. Based on the introduction of PM route structure, we separated the overall arrival operation into two models: Sequencing leg and Link. In order to support an effective behavior in real-work and dynamic environment the agents organization framework is designed with 4 types of agents. Furthermore, we designed the algorithms and the functions inside these agents. A time-based wake turbulence separation was considered in the CD&R agent, a Multi-map structure based on event table was used to detect 4D trajectory conflicts at each time interval. The overall system architecture is built up, in the next step, we will step into design the details learning reward function and make simulations based on a real data to test the performance of this multi-agent system.
Liang Man / Procedia Engineering 99 (2015) 233 – 243
243
Acknowledgments This work was supported by the National Natural Science Foundation grant U1333116 and 61039001, Fundamental research funds for the Central Universities Civil Aviation University of China special Grant 3122013D013, and the Opening Science Foundation of Tianjin Key Laboratory of Operation Programming and Safety Technology of Air Traffic Management. This work was done during my scholar visiting in MAIAA lab of ENAC in Toulouse financed by China Scholarship Council. Finally thanks for the suggestions given by professor Daniel DELAHAYE. References [1] Hamsa Balakrishnan and Bala G Chandran. Algorithms for scheduling runway operations under constrained position shifting. Operations Research, 58(6):1650–1665, 2010. [2] John E Beasley, Mohan Krishnamoorthy, Yazid M Sharaiha, and D Abramson. Scheduling aircraft landings-the static case. Transportation Science, 34(2):180–197, 2000. [3] CAAC. 2012 statistial data on civil aviation of china, 2013.URL: http://www.caac.gov.cn/I1/K3/201305/P020130520548774552650.pdf. [4] Leonardo L.B.V. Cruciol, Antonio C. de Arruda Jr., Li Weigang, Leihong Li, and Antonio M.F. Crespo. Reward functions for learning to control in air traffic flow management. Transportation Research Part C: Emerging Technologies, 35(0):141 – 155, 2013. [5] Leonardo L.B.V. Cruciol and Li Weigang. Air holding problem solving by reinforcement learning to reduce the congestion in airspace sectors. In The 2012 International Conference on Artificial Intelligence, Proceedings of ICAI, pages 272–278, 2012. [6] Heinz Erzberger, Thomas J Davis, and Steven Green. Design of center-tracon automation system. 1993. [7] Heinz Erzberger, T Lauderdale, and Y Chu. Automated conflict resolution, arrival management and weather avoidance for atm. In 27th International Congress of the Aeronautical Sciences, Nice, France, 2010. [8] Heinz Erzberger and Bill Nedell. Design of automated system for management of arrival traffic. National Aeronautics and Space Administration, Ames Research Center, 1989. [9] Heinz Erzberger and Russel A Paielli. Concept for next generation air traffic control system. Air Traffic Control Quarterly, 10(4):355–378, 2002. [10] Heinz Erzberger. Design principles and algorithms for automated air traffic management. Knowledge-Based Functions in Aerospace Systems, 7:2, 1995. [11] Heinz Erzberger. Transforming the nas: The next generation air traffic control system. In 24th International Congress of the Aeronautical Sciences, Yokohama, Japan, 2004. [12] Eurocontrol. Point merge: improving and harmonising arrival operations with existing technology, 2013. [13] B Favnnec, Eric Hoffman, Aymeric Trzmiel, Francois Vergne, and Karim Zeghal. The point merge arrival flow integration technique: Towards more complex environments and advanced continuous descent. In AIAA Aviation Technology, Integration, and Operations Conference, 2009. [14] MJ Jackson and Brian E O’ Laughlin. Airborne required time of arrival control and integration with ATM. AIAA Aviation Technologies, Integration and Operations, Belfast, Northern Ireland, 2007. [15] Joel K Klooster, Ana Del Amo, and Patrick Manzi. Controlled time-of-arrival flight trials. In Proc. Eighth USA/Europe Air Traffic Management Research and Development Seminar, 2009. [16] Joel K Klooster and David de Smedt. Controlled time of arrival spacing analysis. In Proceedings of the Ninth USA/Europe Air Traffic Management Research and Development Seminar (ATM2011), Berlin, Germany, 2011. [17] Jean-Pierre Nicolaon, Antoine Vidal, Peter Crick, and E FREVILLE. Potential benefits of a time-based separation procedure to maintain the arrival capacity of an airport in strong head-wind conditions. In Proc. of the Air Traffic Management R&D Seminar, 2003. [18] Sergio Ruiz, Miquel A Piera, and Isabel Del Pozo. A medium term conflict detection and resolution system for terminal maneuvering area based on spatial data structures and 4d trajectories. Transportation Research Part C: Emerging Technologies, 26:396–417, 2013. [19] Sergio Ruiz, Miquel A Piera, Jenaro Nosedal, and Andrea Ranieri. Strategic de-confliction in the presence of a large number of 4d trajectories using a causal modeling approach. Transportation Research Part C: Emerging Technologies, 39:129–147, 2014. [20] Sergio Ruiz and Miquel A Piera. Spatial data structure based algorithm for improving conflict detection/conflict resolution algorithms. In Proceedings of Unmaned Aerial Vehicles Conferences (UAV Conferences 2009), 2009. [21] Dionyssios A Trivizas. Optimal scheduling with maximum position shift (mps) constraints: A runway scheduling application. Journal of Navigation, 51(02):250–266, 1998. [22] Kagan Tumer and Adrian K Agogino. Improving air traffic management with a learning multiagent system. IEEE Intelligent Systems, 24(1):18–21, 2009. [23] Kagan Tumer and Adrian Agogino. Distributed agent-based air traffic flow management. In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, page 255. ACM, 2007. [24] Steven Wollkind, John Valasek, and Thomas R Ioerger. Automated conflict resolution for air traffic management using cooperative multiagent negotiation. In AIAA guidance, navigation, and control conference, pages 16–19, 2004.
ScienceDirect Procedia Engineering 99 (2015) 233 – 243
“APISAT2014”, 2014 Asia-Pacific International Symposium on Aerospace Technology, APISAT2014
An agent-based approach to automated merge 4D arrival trajectories in busy terminal maneuvering area LIANG Mana,b,* a
College of Civil Aviation,Nanjing University of Aeronautics and Astronautics, Nanjing 210016, R.P China b College of Air Traffic Management, Civil Aviation University of China, Tianjin 300300, R.P China
Abstract This paper presents a new approach for automated merging multiple aircraft flows in a busy Terminal Maneuvering Area (TMA). This work is motivated by the current overloaded airspace in the Beijing Capital international airport, the highlighted delay in the China air transportation and the need of more efficient methods to help air traffic controllers. Present research consists of a new approach to optimize a set of aircraft planned to land at a given airport based on multiagents technique, which is automated generating comprehensive sequencing plan and conflicts-free trajectories. The architecture of the system is based on a Point Merge (PM) route structure to enable expediting or delaying aircraft while staying on lateral navigation mode. The whole system is designed by two main models to manage the process of arriving flows: Sequencing leg and Link. And then, 4 kinds of agents are designed to support the implementation of this automated system: aircraft agent, flow manager agent, conflict detection and resolution agent and 4D trajectory planning agent. We consider the 4D trajectory-based operation situation in TMA and apply different wake turbulence constraint in these two models. Finally, a conclusion is made and future work is outlined. © Published by Elsevier Ltd. This © 2015 2014The TheAuthors. Authors. Published by Elsevier Ltd.is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA). Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA)
Keywords: Air traffic management; 4D trajectory planning; terminal control area; agent modeling
* Corresponding author. Tel.: +86-022-24092445; fax: +86-022-24092431. E-mail address: [email protected]
1877-7058 © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA)
doi:10.1016/j.proeng.2014.12.531
234
Liang Man / Procedia Engineering 99 (2015) 233 – 243
1. Introduction Flight delay is now a very serious and widespread problem in China. With rapid economic growth, the demands for air services in China has significantly increased in the past few decades, the number of aircraft movements grew at an average rate of 11.1% per annum between 2008 and 2012 [3]. Beijing Capital international airport is the first busiest airport in Asia and the second busiest airport in the world, with about 557 thousand aircraft movements in 2012 and 4% traffic increase according to airport statistics and data published by Airports Council International. Therefore, air traffic booming is placing an enormous pressure on the Beijing airport system. According to Flight-Stats, in 2013 Beijing airport ranks dead last among the world’s top 35, with fully 82% of flights failing to leave on time. Second worst was Shanghai, at 71%. It’s obvious that the growth of air traffic has already exceeded Beijing airport system capacity, as a result a serious flight delay in Beijing airport occurred. To alleviate this delay problem and increase the airport capacity, in 2010 Chinese civil aviation Air Traffic Management Bureau (ATMB) launched a special project of reducing flight delay in order to enhance the air passenger satisfaction. And then, the high level airspace in the north China was centralized and concentrated into one single organization named North-ATMB area control center (NATM-ACC), which provided the en-route control service for the flights inbound or outbound Beijing capital airport, so as to reduce the unnecessary coordination between air sectors in different regions, to manage the flow from a strategic level. Parallel runway dependency approach mode was implemented in Beijing Capital airport to increase the runway capacity. However, in 2013 the delay problem is still becoming increasingly acute. In fact, the bottleneck of Beijing congestion problem is the terminal control area or Terminal Maneuvering Area (TMA), which is a designated area of controlled airspace surrounding a major airport where there is a high volume of traffic. Due to insufficient number of air traffic controllers (ATCO) and relatively high traffic flows, the North-ATMB terminal control (NATM-TMA) system is close to collapse at the peak time. In order to solve these problems and avoid a worsening situation, a higher level of automation in separation assurance is considered to change the current ATCO-assured separation system in NATMTMA. Automated control of arrival flows, namely automated arrival scheduling, sequencing and conflict resolution, has been a topic of interest for a few decades now. [8] describes a design of an automated air traffic control system based on a hierarchy of advisory tools for controllers, referred to as the Center-TRACON Automation System (CTAS). Later, CTAS was installed and evaluated at the Denver area and Dallas/Ft.Worth area air traffic control facilities [6, 10]. Advances in navigation, communication and surveillance provide new emerging concepts to solve this arrival operational problem. The introduction of area navigation (RNAV, P-RNAV) makes it possible to define a new route structure to revisit the merging of arrival flows. The data-link communication provides us a convenient way to share the trajectory information not only between air-to-ground and also air to air. The Automatic Dependent SurveillanceBroadcast (ADS-B) provides us a new way of surveillance for tracking aircraft via Global Navigation Satellite Systems (GNSS). Thus [9, 11, 7] present an Advanced Airspace Concept (AAC) for the Next Generation Air Traffic Control System, it can handle large increases in capacity and throughput in the way of automating the monitoring and control of separation by using a down-link to share trajectory information between the airborne. On the other hand, several recent analysis studies have investigated in amelioration the arrival management by using airborne Required Time-of-Arrival (RTA) functionality in modern commercial jets. Current RTA control functionality adjusts and regulates the aircraft’s speed along the trajectory in order to arrive at a specified way-point at a specified time, thus improving the predictability of the arriving aircraft [14, 15, 16]. It is clear that the use of airborne RTA functionality is a key enabler to Trajectory Based Operations in TMA, and a time-based spacing approach is more suitable for realization of an automated arrival management. From an intelligent control point of view, new methods from Artificial Intelligence such as multi-agent system (MAS) have been introduced as well to automate air traffic flow management. [24] takes the aircraft as agents that are competing for the limited resource of airspace, applies the Monotonic Concession Protocol (MCP) negotiation techniques to resolve air traffic conflicts, and the result shows that the solutions generated by these methods are more efficient than those created by traditional methods or alternative non-cooperative solutions. While, in [23] research, an agent is associated with a fix (a specific location in 2D space) and its action consists of setting the
Liang Man / Procedia Engineering 99 (2015) 233 – 243
235
separation required among the aircraft going though that fix. Agents use reinforcement learning to set this separation and their actions speed up or slow down traffic to manage congestion. The FACET (an air traffic flow simulator developed at NASA and used extensively by the FAA and industry) based results show that agents receiving personalized rewards reduce congestion by up to 45% over agents receiving a global reward and by up to 67% over a current industry approach. In TMA, the aircraft arrival management is considered as decentralized, distributed and dynamic, collaboration is necessary among different controllers, the environment factors (like capacity, traffic flow, weather) change high dynamically. Therefore, MAS can be suitable for aircraft arrival management. In this paper, an agent-based approach to automatically merge 4D trajectories in TMA is being studied to assist air traffic controllers. This paper is structured as follows: Section 2 we present the merge point route structure. Section 3 introduces an applicable Agent-based framework, with details in the agent-based modeling approach, agent selection, agent actions, as well as some agent implementation. Section 4 outlines the future works. 2. Point merge route structure Point Merge (PM) is a systematized method for sequencing arrival flows developed by the EUROCONTROL Experimental Center in 2006. Instead of using heading vector, PM only uses speed control and "direct to" instruction to guide the aircraft to merging point, and it has two clear steps to execute them one by one. Its special route structure and procedures provides a potential approach to realize an automated separation assurance to the arrival flow under high traffic load situation [13]. As illustrated in the Figure 1, the route structure supporting PM is denoted Point Merge System (PMS). A PMS may be defined as an RNAV STAR (Regional Navigation Standard Arrival Route), transition or initial approach procedure, or a portion thereof, and is characterized by the following features: x A single point, denoted Merge point, is used for traffic integration; x Pre-defined legs, denoted Sequencing legs, iso-distance and equi-distance from the Merge point, are dedicated to path stretching/shortening for each inbound flow. These legs shall be separated by design vertically, laterally or both. Sequencing leg Entry point Entry point
Path envelop
Merge point
Airport
Fig. 1. Structure of the Point Merge System.
An independent analysis conducted by NATS established that, by using Point Merge, airlines landing at Dublin Airport in 2013 reduced their fuel requirement by 19.1%.per flight. It also found that aircraft reduced the length of the flight by 11.3 miles, a 17% saving. Point Merge also provided savings of 23,500 tons of CO2, representing a 19% reduction [12]. Besides the economy benefits, on its implementation in Paris-CDG, some other operational and safety advantages was found, such as less instructions issued by ATCs to pilots which means lower workload, sequence order easily changed to improve runway throughput which is hard to realize by traditional radar control methods, and improved predictability which can provide a precise information about the arrival time. Referring to the statement mentioned above, we can recognize that PM route structure is easier to change the aircraft approach
236
Liang Man / Procedia Engineering 99 (2015) 233 – 243
sequence and very suitable for realizing an automated arrival management, because the aircraft actions can be clearly divided into several individually steps which is easily realized by computer program, so in this research we use PM route structure as a base for building a multi-agent system. 3. Agent modeling for PM-based aircraft arrival management An agent is something that acts in an environment. Agents are considered as social, because they cooperate with humans or other agents in order to achieve their tasks. In fact, agent methods are a natural tool to help automatic existing air traffic systems, and moreover recent researches in Air Traffic Flow Management (ATFM) have successfully combined Reinforcement Learning (RL) and Multi-agent techniques to the ATFM. [23, 22] developed a distributed agent-based ATFM for NAS in United State. [4, 5] used learning agents to solve the air traffic flow management problem and the airspace sectors congestion problem in Brazil. In this paper, the discussion of the agent-based modeling for PM based aircraft arrival management is organized into four subsections. The first one makes a problem analysis, the second one discusses the overall modeling framework, and the third one presents the conflict detection and resolution agent, followed by the forth one trajectory planning agent and flow manager agent. 3.1. Problem analysis The arriving aircraft from different directions are firstly separated into two groups, and then each group follows one sequencing leg. On the same sequencing leg, all the aircraft maintain the same flight level and keep spacing automatically in horizontal distance by speed adjustment. After that, in consider of the fairness and the operation efficiency in the whole terminal area, all the arriving aircraft are sequenced rationally, and then when the first approaching aircraft leaves the sequencing leg and directs to the merge point, the second approaching aircraft keeps flying on the sequencing leg until there is enough spacing between two of them, at that moment, it can leave the sequencing leg and direct to the merge point, the rest of the aircraft will follow the similar procedure. During the process of direct to the merge point, all the aircraft refer to their flight performance to fly in a continuous descend profile. The time to descent can be issued by ATCOs suggested by decision support software, and the process of descent is managed by the airborne flight management system. Conflict Detection and Resolution (CD&R) is considered in a 4D trajectory based dynamic operation situation. When arrival flow of runway reaches to a defined level named maximum available capacity notated by W , then delay is happened, which will propagate back to the PM system. In consider that once aircraft leave the Sequencing leg they can only direct to merge point without any possibility of holding or stopping or changing path, therefore we have to dynamically modify the flow rate at the Merge point or even Entry point in order to maintain a smooth traffic flow. Therefore, when the delay is heavy we can decrease the rate of merge, when there is no delay we can increase the rate of merge. The characteristics of the problem are as follows: x All the approaching aircraft can only enter the PM system via 2 entry points. x Each aircraft j must be well spaced on the Sequencing leg according to distance-based wake turbulence separation standard issued by ICAO. x Each aircraft j on the Sequencing leg maintains a flight level, and the magnitude of speed change is limited to r20% . x Each aircraft j on the Sequencing leg has an associated turning time based on its preceding aircraft in the whole sequence. x Each aircraft j must be well spaced on the process of direct to Merge point according to time-based wake turbulence separation standard issued by ICAO. x On the process of merging, each aircraft j can control its descent profile according to the aircraft performance and cannot change path during the process of merge. x CD&R is considered in a 4D trajectory based dynamic operation situation. x The whole system arrival flow can dynamically adapt to the delay. The notations used in this research are as follows:
Liang Man / Procedia Engineering 99 (2015) 233 – 243
Nomenclature
* j
tej j tRTA ttnj tdtj 'tm 'te hj
a set of aircraft aircraft j , j * , entry time of aircraft j required time of arrival of aircraft j at Merge point time of turning direct to merge point, leaving the Sequencing leg time of commencing descent during the process of direct to Merge point required time interval between two successive aircraft at the Merge point required time interval between two successive aircraft at the Entry point flight level or altitude of aircraft j
3.2. Overall framework A new A/C coming to the PM system
NO
Is there any delay for the new A/C ? YES Sequencing leg model
1.Choose one of the entry point to the PM system 2.Calculate the sequencing landing place for the new A/C, and the Required time of arrival(RTA) at merge point 3.Conflict detect based on 4D trajectory position in consider of distance based wake turbulence separation 4.Conflict resolution: speed control
Update all the A/C position, sequencing and RTA information in time interval
NO
Is the previous A/C in the sequencing leaving the Sequencing leg? YES Link model
1.Calculate the Possible turning time for the new A/C 2.Calculate the best descent-commencing time in order to execute a continuous descent approach in consider of aircraft performance 3.Conflict detect based on 4D trajectory position in consideration of timebased wake turbulence separation on merge point 4.Conflict resolution: speed control
Fly a continuous descent approach profile Arrive at Merge point
Fig. 2. Flow chart for multi-agent modelling of aircraft arriving in PM system.
The system includes two main models to manage the process of arrival flows, showed in Figure 2: Sequencing leg model and Link model. Sequencing leg part is to keep the arriving aircraft in each arc in an orderly and conflictfree flow by a speed change, at the same time it can dynamically prolong or shorten the fly distance of aircraft on the arc according to the flow. Link part is to decide the sequence that aircraft can leave leg and direct to merge point. Due to the high combinatory induced by such a problem, an optimization algorithm can be developed in order to calculate the landing sequence, and then propose to each aircraft the moment to leave the sequencing leg. The optimization criteria are based on the minimum make-span of sequence. Besides, a descent planning is to
237
238
Liang Man / Procedia Engineering 99 (2015) 233 – 243
automatically control the altitude change, which takes into consideration the execution of Continuous Descent Approaches (CDA). Agents organization framework is the fundamental mechanisms to support an effective behavior in real-work, dynamic environment. In this research, we design 4 types of agents to support the two models, as showed in figure 3, they are Trajectory conflict detection and resolution agent, Flow manager agent, 4D trajectory planning agent and Arriving aircraft agent. The role of aircraft agent is to manage the arriving aircraft. For each aircraft, its trajectory contains the information below: x Time slot x Aircraft number (odd number for one sequencing leg, even number for another sequencing leg) x Current position ( x , y ) x Current subsystem (Sequencing leg, Link) x Current flight level x Current airspeed x Wake turbulence category x Entry time to PM system x Fly routes x Landing sequence position x Required time of arrival at Merge point Aircraft agent exchanges the information dynamically with other three agents. In our case, the path change and sequence information are exchanged with 4D trajectory planning agent, Entry time and RTA are exchanged with Flow manager agent, safely spacing is supported by Trajectory conflict detection and resolution agent. 4D trajectory planning agent optimizes the landing sequence and path planning; its optimized results will include the turning time from sequencing leg to Merge point, the descent commencing time during the process to Merge point and the landing position for each arriving aircraft. Flow manager agent is communicating directly with the environment, it can react with delay and congestion situation in the adjacent sectors and the airport, and through the ways of controlling the passing time at entry points and Merge point, the flow manager agent can speed up and slow down the arriving aircraft by a specific policy, so that the global performance can reach a better level. Trajectory conflict detection and resolution agent dynamically detects the conflicts and solves the conflicts, and it is the base of the whole PM system. Sequencing leg model
Link model
Concept-level Agent-level
CD: Multi-map-based dynamic detection CR: speed control(20% variation)
Arriving A/C
Trajectory Conflict Detection and Resolution(CD&R) agent
ng aci
sp Arriving A/C A/C agents Arriving A/C Call-sign Position (X,Y) Flight level Airspeed (IAS) Wake turbulence category Entry time to PM system Fly route RTA at Merge point
Flow manager agent
Entry time RTA
P Seq ath c uen hang ce e pla ce
4D Trajectory planning agent
Optimized Sequencing planning (the landing sequence) Path changing planning (Turning time from Sequencing Leg to Merge Point; Descent commencing time in Link model)
Fig. 3. Overall agent modeling architecture.
Delay
Environment
Congestion
Control the A/C entry time Control the A/C RTA at Merge point
239
Liang Man / Procedia Engineering 99 (2015) 233 – 243
3.3. Conflict detection and resolution agent Conflict detection and resolution agent is the most import element in this agent-based system. We consider the 4D trajectory-based operation situation in TMA, all the aircraft will merge to the same point at the same altitude, therefore in this PM system, we can simplify the conflicts into two categories: catch-up type and merging type. A catch-up conflict occurs when there are two aircraft following the same Sequencing leg and the trailing aircraft is speedy enough to catch the leading aircraft which will induce a safety distance infringement. Obviously, a merging conflict occurs when aircraft pass through the Merge point. According to different fly phrases on the PM system, we can classify the conflict detection in three situations, see Figure 4. Sequencing leg Keep fly until L2-L1=SD a
Sequencing leg Entry point Entry point
b
Entry point Entry point
a
b
67
a
Sequencing leg Entry point
Entry point
6'
La
b
Lb Merge point
Merge point
Merge point Situation A
Situation B
Situation C
Fig. 4. Conflict detection (a) situation A; (b) situation B; (c) situation C.
Firstly, in situation A, the aircraft fly on the Sequencing leg, all of them maintain the same flight level (or altitude) on each Sequencing leg. Here distance-based wake turbulence separation is considered in order to keep the aircraft safe. For any two successive aircraft passing the same Entry point, we need only to compare the longitudinal distance between two successive aircraft to detect the conflicts. Supposing that aircraft a is following aircraft b , the required distance between them which can be notated as SD , should not be less than the ICAO minimum wake vortex separation (DWVS) standard defined by the Air Traffic Management - ICAO DOC 4444, see table 1. tea is the entry time of aircraft a , teb is the entry time of aircraft b , in consider of the delay on airport and congestion on the sector, we have to control the entry time between a and b , then:
SD t DWVS
(1)
tea teb t 'te
a, b *
(2)
Table 1. ICAO minimum wake turbulence separation standard. Trailing aircraft Leading aircraft Heavy Medium Light
Separation in distance (NM)
Separation in time (s)
Heavy
Medium
Light
Heavy
Medium
Light
4
5
6
105
131
158
3
4
79
105
3
79
Secondly, in situation B, when one aircraft is turning to the Merge point, then the next aircraft to turn is determined by landing sequence which is decided by the Trajectory planning agent. During merge process, all the arriving aircraft must direct to the Merge point one by one, and any two aircraft can not turn to the Merge point simultaneously. Here we suppose that aircraft a is still following aircraft b in the landing sequence but they are on
240
Liang Man / Procedia Engineering 99 (2015) 233 – 243
the different Sequencing legs, so aircraft a and aircraft b have vertical separation, then when aircraft b is turning to the merge point, aircraft a must keep flying on its Sequencing leg until ttna in which moment a distance SM is not less than DWVS, this constraint ensures that there is always horizontal separation between any two successive aircraft descending. If we suppose that the distance from aircraft a to the Merge point is La , the distance from aircraft b to the Merge point is Lb , then the SM is the differential distance between La and Lb , therefore:
SM = La Lb
a, b *
(3)
SM t DWVS (4) If aircraft a and aircraft b are on the same sequencing leg, then when b is turning, there may be a minimum separation infringement between a and b , so in this case, we have to calculate the distance between this two aircraft D(a, b) , then: D(a, b) t DWVS
a, b *
(5) Thirdly, in situation C, when the aircraft approach the Merge point, at tdtj they commence descent, just as mentioned before, finally all of them must reach the same altitude at Merge point. In order to remove conflicts at merging point and consider of increasing the capacity of runway in strong headwind condition, we use time-based separation to keep aircraft safe, and during merging process, we use speed control to the trailing aircraft in each pair of two successive aircraft to make sure that there will be continuous time-based separation between them. So merging conflict detection is applied during this process to make sure that with the time passes, the RTA of the aircraft is well separated at the Merge point. Referring to [17], the time separations is equivalent to the time that is required to fly the actual ICAO distance separations under no wind conditions. The time separation are evaluated for a common reference ground speed (GSr) for the worst wake turbulence effecting condition on landing, A 137 kt is selected after examination of engine performance database that have been found to be representative of the actual distribution of the aircraft according to the different weight classes that are defined to characterize the ICAO separation standards. So the corresponding time-based separation (TWVS) are enumerated in table 1. In the case where a 2.5 NM radar separation is applied, the corresponding time interval is 66 seconds. Again, we suppose that a b is tRTA , then: aircraft a follows aircraft b , and denote that ST is the time interval between tRTA a b ST = tRTA tRTA
ST t TWVS ST t 'tm
a, b *
(6)
(7)
(8) All statement mentioned above is considered in a static situation, in our research we focus on the dynamic conflict detection for 4D trajectory, so in the phrase of implementation, a way of converting the continuous trajectory problem into discrete approximation problem should be considered, here a "snapshot" method can be adopted to capture the trajectory spatial information. In [20, 18, 19], the conflict detection is based on a Spatial Data Structure (SDS), avoiding non-efficient pairwise trajectory comparisons, and using a simplified wake vortex modeling through 4D tubes to detect time-based separation infringements between aircraft. Here, in our research, a snapshot mode is applied to dynamically capture the arrival flows over time, a Multi-Map Structure (MMS), also
Liang Man / Procedia Engineering 99 (2015) 233 – 243
241
named multi-hash, which is a generalization of a map or associative array abstract data type in which more than one value may be associated with and returned for a given key, is used to store the 4D trajectories information with multiple events for the conflict detection problem†, see Figure 5.
Fig. 5. Multi-map structure with events.
We choose “time slot” as key, events with different aircraft numbers having the same time slot are stored with the same key. When there are more than 1 event in the same time slot (key), we can choose combination of 2 events from that slot and check whether each of them has conflict or not. If selected combination of aircraft is on the Sequencing leg subsystem, then a catch-up conflict detection is applied, we need only calculate the actual distance between two successive aircraft on the same Sequencing leg, if selected combination of aircraft is on the Link subsystem, then a merge conflict detection is applied, we calculate the RTA at the merge point for each aircraft, and compare the actual time interval with the ICAO time based separation minimum. For the part of conflict resolution, in this research, speed control is the only tactical conflict resolution method in the PM system, and only 20% of speed change is allowed for each aircraft, thus in the whole PM system, the sequencing leg is very important, because it enables expanding or shortening the aircraft fly distance. 3.4. Trajectory planning agent and flow manager agent Trajectory planning agent mainly solves two kinds of problems: to decide the optimized landing sequence and to make path changing planning. Path changing planning mainly refers to the turning time to Merge point and descent commencing time during direct to Merge point. In order to realize this function, trajectory planning agent must communicate with CD&R agent and aircraft agent. As stated previously, all the aircraft on the Sequencing leg will turn to the Merge point one by one, so at each time the trajectory planning agent just focus on one aircraft which is ready to turn. For example, aircraft i is changing its “current system” information from “sequencing leg” to “link” which means it is turning to the Merge point, the trajectory planning agent will take aircraft i 1 as the active aircraft ready to turn, by communicating with the CD&R agent, when there is a sufficient distance SW , then the trajectory planning agent will save the time ttni 1 immediately and transfer ttni 1 to aircraft agent to execute path changing. A descent planning is following the same philosophy, the trajectory planning agent will focus only on the descent commencing time tdti 1 , it will calculate the required distance or time for a continuous descent approach (CDA) in consideration of aircraft performance, and then transfer the parameters to aircraft agent to execute the descent profile. The landing sequence problem has been studied in a lot of papers, [2] solved the static aircraft landing problem by a mixed-integer zero-one formulation together with a population heuristic algorithm. [21, 1] developed Dynamic programming scheduling algorithm with Constrained Position Shifting (CPS) or Maximum Position Shift (MPS)
†
Thanks for the ideas given by Zuniga C.A. and Hidenori CHIDA
242
Liang Man / Procedia Engineering 99 (2015) 233 – 243
constraints. In our research, we focus on more actual and dynamic traffic situation. Because with PM system it is much easier to shift the aircraft sequence, and in the Chinese busy TMA normally more than 90% commercial aircraft belong to medium wake turbulence category, and the rate of arriving aircraft in the peak hour is normally less than 60 aircraft per hour, so we have more flexibility in the aircraft position shifting. However, from the airlines point of view, the fairness must also be considered, so for those aircraft who belong to the same wake turbulence category First come first service policy will be applied, then if one aircraft can change its position, the maximum position shift should remain limited. The algorithm chosen in our research will aim to find the minimum makespan for a set of aircraft 1, 2,3,..., n , so the objective function will be: n
j T = Min ¦(tRTA tej )
j*
j =1
(9)
T is the makespan of arriving aircraft, with different CPS network, there will be different results. In our case, we can apply the dynamic programming method introduced by [1] or evolutionary method as Genetic algorithm to solve this problem. Flow manager agent is designed as a learning agent in our PM system, placed on the Merge point and it is used for control the whole system performance. In our research, we focus the system environment on the delay in runway and the congestion in the two neighbor sectors from which aircraft are transferred to the entry points of the PM system. When delay or congestion happened, the bottleneck may be on the runway, or maybe in the TMA sectors. If the runway is reaching it capacity firstly, then we need reduce the merging rate on the Merge point, if one of the TMA sectors is congested, then we can increase the rate of arriving at the corresponding entry point in PM system. Besides, we can also set different arriving rate ratio between these two entry points to balance the congestion severity in both sectors. According to [23], this agent action is to learn the "Miles in Trail (MIT)", in our cases, the flow manager agent action is to learn the "Times in Trail (TIT)", when flow manager agent sets the TIT value to 'tm for Merge point fix, aircraft going towards Merge point are instructed to line up and keep 'tm times of interval (we suppose aircraft will always keep a safe distance from each other regardless the value of 'tm , higher value of 'tm means slow down the flow, lower value of 'tm means speed up the flow. The similar policy will apply to those two entry points. In [23, 4], Reinforcement learning (RL) was successfully applied to control the traffic flow, see figure 6 , design an efficient reward function and system global performance evaluation algorithm are both important for flow manager agent. Because of complexity of this problem, in this paper, we will not discuss it. Flow manager agent
state
actions
rewards
(Time in trails at the Merge point)
(Delay Congestions)
Environment
Fig. 6. Reinforcement learning in air traffic flow management.
4. Conclusion and future works An agent-based approach to automated merge 4D arrival trajectories in busy Terminal Maneuvering Area has been proposed. Based on the introduction of PM route structure, we separated the overall arrival operation into two models: Sequencing leg and Link. In order to support an effective behavior in real-work and dynamic environment the agents organization framework is designed with 4 types of agents. Furthermore, we designed the algorithms and the functions inside these agents. A time-based wake turbulence separation was considered in the CD&R agent, a Multi-map structure based on event table was used to detect 4D trajectory conflicts at each time interval. The overall system architecture is built up, in the next step, we will step into design the details learning reward function and make simulations based on a real data to test the performance of this multi-agent system.
Liang Man / Procedia Engineering 99 (2015) 233 – 243
243
Acknowledgments This work was supported by the National Natural Science Foundation grant U1333116 and 61039001, Fundamental research funds for the Central Universities Civil Aviation University of China special Grant 3122013D013, and the Opening Science Foundation of Tianjin Key Laboratory of Operation Programming and Safety Technology of Air Traffic Management. This work was done during my scholar visiting in MAIAA lab of ENAC in Toulouse financed by China Scholarship Council. Finally thanks for the suggestions given by professor Daniel DELAHAYE. References [1] Hamsa Balakrishnan and Bala G Chandran. Algorithms for scheduling runway operations under constrained position shifting. Operations Research, 58(6):1650–1665, 2010. [2] John E Beasley, Mohan Krishnamoorthy, Yazid M Sharaiha, and D Abramson. Scheduling aircraft landings-the static case. Transportation Science, 34(2):180–197, 2000. [3] CAAC. 2012 statistial data on civil aviation of china, 2013.URL: http://www.caac.gov.cn/I1/K3/201305/P020130520548774552650.pdf. [4] Leonardo L.B.V. Cruciol, Antonio C. de Arruda Jr., Li Weigang, Leihong Li, and Antonio M.F. Crespo. Reward functions for learning to control in air traffic flow management. Transportation Research Part C: Emerging Technologies, 35(0):141 – 155, 2013. [5] Leonardo L.B.V. Cruciol and Li Weigang. Air holding problem solving by reinforcement learning to reduce the congestion in airspace sectors. In The 2012 International Conference on Artificial Intelligence, Proceedings of ICAI, pages 272–278, 2012. [6] Heinz Erzberger, Thomas J Davis, and Steven Green. Design of center-tracon automation system. 1993. [7] Heinz Erzberger, T Lauderdale, and Y Chu. Automated conflict resolution, arrival management and weather avoidance for atm. In 27th International Congress of the Aeronautical Sciences, Nice, France, 2010. [8] Heinz Erzberger and Bill Nedell. Design of automated system for management of arrival traffic. National Aeronautics and Space Administration, Ames Research Center, 1989. [9] Heinz Erzberger and Russel A Paielli. Concept for next generation air traffic control system. Air Traffic Control Quarterly, 10(4):355–378, 2002. [10] Heinz Erzberger. Design principles and algorithms for automated air traffic management. Knowledge-Based Functions in Aerospace Systems, 7:2, 1995. [11] Heinz Erzberger. Transforming the nas: The next generation air traffic control system. In 24th International Congress of the Aeronautical Sciences, Yokohama, Japan, 2004. [12] Eurocontrol. Point merge: improving and harmonising arrival operations with existing technology, 2013. [13] B Favnnec, Eric Hoffman, Aymeric Trzmiel, Francois Vergne, and Karim Zeghal. The point merge arrival flow integration technique: Towards more complex environments and advanced continuous descent. In AIAA Aviation Technology, Integration, and Operations Conference, 2009. [14] MJ Jackson and Brian E O’ Laughlin. Airborne required time of arrival control and integration with ATM. AIAA Aviation Technologies, Integration and Operations, Belfast, Northern Ireland, 2007. [15] Joel K Klooster, Ana Del Amo, and Patrick Manzi. Controlled time-of-arrival flight trials. In Proc. Eighth USA/Europe Air Traffic Management Research and Development Seminar, 2009. [16] Joel K Klooster and David de Smedt. Controlled time of arrival spacing analysis. In Proceedings of the Ninth USA/Europe Air Traffic Management Research and Development Seminar (ATM2011), Berlin, Germany, 2011. [17] Jean-Pierre Nicolaon, Antoine Vidal, Peter Crick, and E FREVILLE. Potential benefits of a time-based separation procedure to maintain the arrival capacity of an airport in strong head-wind conditions. In Proc. of the Air Traffic Management R&D Seminar, 2003. [18] Sergio Ruiz, Miquel A Piera, and Isabel Del Pozo. A medium term conflict detection and resolution system for terminal maneuvering area based on spatial data structures and 4d trajectories. Transportation Research Part C: Emerging Technologies, 26:396–417, 2013. [19] Sergio Ruiz, Miquel A Piera, Jenaro Nosedal, and Andrea Ranieri. Strategic de-confliction in the presence of a large number of 4d trajectories using a causal modeling approach. Transportation Research Part C: Emerging Technologies, 39:129–147, 2014. [20] Sergio Ruiz and Miquel A Piera. Spatial data structure based algorithm for improving conflict detection/conflict resolution algorithms. In Proceedings of Unmaned Aerial Vehicles Conferences (UAV Conferences 2009), 2009. [21] Dionyssios A Trivizas. Optimal scheduling with maximum position shift (mps) constraints: A runway scheduling application. Journal of Navigation, 51(02):250–266, 1998. [22] Kagan Tumer and Adrian K Agogino. Improving air traffic management with a learning multiagent system. IEEE Intelligent Systems, 24(1):18–21, 2009. [23] Kagan Tumer and Adrian Agogino. Distributed agent-based air traffic flow management. In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, page 255. ACM, 2007. [24] Steven Wollkind, John Valasek, and Thomas R Ioerger. Automated conflict resolution for air traffic management using cooperative multiagent negotiation. In AIAA guidance, navigation, and control conference, pages 16–19, 2004.