A large-scale multiobjective satellite data transmission scheduling algorithm based on SVM+NSGA-II

AIAA AVIATION Forum 17-21 June 2019, Dallas, Texas AIAA Aviation 2019 Forum

10.2514/6.2019-3415

Aviation Safety Assessment using Historical Flight Trajectory Data

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

Xiaoge Zhang∗ and Sankaran Mahadevan† Department of Civil and Environmental Engineering, School of Engineering, Vanderbilt University, Nashville, TN, 37235 A multi-fidelity deep learning-based model is developed in this paper to make predictions on the trajectory of an ongoing flight by learning from the patterns embodied in its historical trajectory data streamed from System Wide Information Management (SWIM) Flight Data Publication Service (SFDPS). The proposed method is illustrated with a four-step procedure. In the first step, a fast and scalable big data engine – Apache Spark – is leveraged to parse the massive raw flight tracking messages in XML format, filter flight position data, correlate flight tracking messages with the corresponding flight ID in an effective manner. In the second step, we build two individual deep learning models to predict the future state of flight trajectory from different perspectives. Specifically, a deep feedforward neural network (DNN) is trained to make one-step-ahead predictions on the latitude and longitude deviation between the actual flight trajectory and target flight trajectory. In parallel, a deep Long Short-Term Memory (LSTM) neural network is trained to make longer-term predictions on the flight trajectory over multiple subsequent time instants. The prediction uncertainties in both deep learning models are characterized following a Bayesian approach. In the third step, LSTM prediction is corrected using the more accurate DNN prediction, thus achieving both accuracy and computational efficiency. Finally, the multi-fidelity approach is extended to multiple flights, then we use separation distance as a quantitative metric to measure the en-route safety between any two flights. Numerical examples are used to demonstrate the effectiveness of the proposed methodology.

I. Nomenclature X = {x1, x2, · · · , x N } Y = {y1, y2, · · · , y N } ω p (ω) p ( y| ω, x) p ( ω| X, Y ) q (ω) x∗ y∗ p ( y∗ | x∗ ) δ ( f ω (x ∗ )) ∗ b y

= = = = = = = = = = = =

input data output data weight of neural network prior distribution on the weight of neural network likelihood function posterior distribution on the weight ω a parametric distribution over ω a new data point model prediction corresponding to the new data point x ∗ probability distribution of model predictions on new data point x ∗ the variance of model prediction on x ∗ mean value of model prediction on x ∗

∗ Postdoctoral † Professor,

Research Fellow, Department of Civil and Environmental Engineering, Vanderbilt University. Department of Civil and Environmental Engineering, AIAA Fellow.

1

Copyright © 2019 by Sankaran Mahadevan. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.

II. Introduction light trajectory prediction is an important component of aviation safety assessment, and accurate prediction of flight trajectory is vital to avoid accidents and reduce errors while ensuring safety and efficiency of air traffic management. Therefore, flight trajectory prediction has received tremendous attention over the past decades, and has been extensively studied from several different perspectives [1, 2]. The existing methods developed for flight trajectory prediction can be broadly categorized into two groups: model-based and data-driven. Most model-based predictive methods characterize each aircraft with a Point Mass Model (PMM), and describe the status of each commercial flight with kinematics and aerodynamic models pertaining to aircraft’s states, such as speed, altitude, position, and course [3, 4]. One representative work conducted by Daigle et al. [5] built a state space model to depict the change of the states of each individual aircraft over time and fused the prediction made by a simplified state space model with measured data using unscented Kalman filter (UKL), thereby enabling to predict the variation of NAS safety margins in the future. In recent years, with the emergence of machine learning and accumulation of massive historical flight trajectory data, research efforts have shifted to develop data-driven predictive models with machine learning algorithms [6, 7]. For example, de Leege et al. [1] trained three respective models with historic flight trajectory and meteorological data to predict flight arrival time, and examined its performance in trajectory prediction with surveillance data and meteorological data for a 45 NM long closed-path continuous descent operation procedure. Shi et al. [8] developed a Long Short-Term Memory (LSTM) recurrent neural network with two hidden layers for flight trajectory prediction, and demonstrated its advantages over two Markov Models. However, in the domain of aviation industry, if a prediction model triggers false alarm (e.g., predict two flights violating separation distance while they are not), the consequence will be costly. From this perspective, model prediction uncertainty needs to be assessed appropriately, thereby enabling us to determine how much we can trust the model prediction before taking corresponding actions. Hence, it is not only necessary to have the prediction of deep learning model on flight trajectory, but more importantly to include the estimation of the model prediction uncertainty on flight trajectory. In this paper, we are motivated to fill this research gap by developing a multi-fidelity deep neural network by learning from the historical flight trajectories streamed from System Wide Information Management (SWIM), in which model prediction uncertainty is characterized following a Bayesian approach. Over the past few years, Bayesian neural network has received a lot of attentions from researchers in the machine learning community [9–11]. In comparison with classical deep neural network, Bayesian neural network is able to capture model uncertainty, which is indispensable for the deep learning practitioner. In addition, Bayesian neural network can also prevent neural network from over-fitting. In this paper, we train a multi-fidelity deep neural network to forecast flight trajectory in the future. Afterwards, the trained deep learning model is extended to make predictions on multiple flight trajectories, thereby supporting in-flight safety assessment. The remainder of this paper is organized as follows. In Section 2, we briefly introduce the concept of Bayesian neural network. In Section 3, we describe the data that is used to build the machine learning model. In Section 4, a multi-fidelity neural network is developed to predict flight trajectory from historical data. In Section 5, we illustrate some preliminary results on model prediction and performance. In Section 6, we end this paper with concluding remarks.

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

F

III. Bayesian Neural Network Bayesian neural networks have received increasing attention because they help to quantify the uncertainty in the model prediction [9, 12]. Unlike a classical neural network, a Bayesian neural network models the weight of the neuron as an epistemically uncertain quantity, and aims to infer the posterior distribution of neuron weight by learning from data. Mathematically, with respect to a specific machine learning problem, we are given multiple pairs of inputs X = {x1, x2, · · · , x N } and their corresponding outputs Y = {y1, y2, · · · , y N }. In the context of Bayesian inference, we would like to infer model parameters ω such that the model y = f ω (x) is likely to generate the outputs we have observed. In the Bayesian inference framework, we first assign a prior distribution p (ω) over the space of model parameter to represent our prior belief on each model parameter to generate the observed output. Next, a likelihood function p ( y| ω, x) is formulated to characterize the probability of observing each output from each specified model parameter ω. By combining prior information with likelihood function, Bayesian inference aims at finding the posterior distribution p ( ω| X, Y ) over model parameters ω. Once the posterior distribution of ω is obtained, then the prediction

2

on a new data point x ∗ can be made by marginalizing over the posterior distribution: ∫ p ( y∗ | x∗ ) = p ( y ∗ | f ω (x ∗ )) p ( ω| X, Y ) dω

(1)

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

w

However, exact Bayesian inference on the weights of a deep neural network is intractable due to the large number of parameters and the non-conjugacy caused by nonlinearities. Instead of inferring the posterior distribution exactly, Gal and Ghahramani [11, 12] developed a novel easy-to-implement alternative to MCMC, which is referred to as Monte Carlo dropout (MC dropout). As shown in Ref. [11], a deep neural network (NN) with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to the deep Gaussian process model [13]. To be specific, in a feedforward deep neural network, with dropout, we generate random samples using a Bernoulli distribution for each unit in the input layer and the hidden layers. Each sample takes the value of 1 with a probability of pi . If the value of the binary variable is zero, then the corresponding unit is dropped accordingly. The implementation of dropout to a neural network amounts to sampling a thinner network from it with some units temporarily being removed. In the backpropagation, the value of each binary variable is used in updating the value of neural network weights. The dropout neural networks are trained in a way similar to standard neural nets with stochastic gradient descent. By doing this, we obtain a number of trained thinner neural networks. The key idea in the Monte Carlo dropout method is to approximate Bayesian inference with variational inference by defining a variational parametric distribution q (ω). Next, we minimize the Kullback–Leibler divergence between the approximating distribution and full posterior with Eq. (2), which is a measure of similarity between the two distributions. ∫ K L ( q (ω)k p ( ω| X, Y )) ∝ − q (ω) logp ( Y | X, ω) dω + K L ( q (ω)k p (ω)) N ∫ (2) Í =− q (ω) logp (Yi | f ω (Xi )) dω + K L ( q (ω)k p (ω)) i=1

By using variational inference to approximate a Gaussian process [14], Gal and Ghahramani [11] showed that a neural network with dropout applied before each weight layer is mathematically equivalent to the approximated Gaussian process. Therefore, dropout can be used as a generic means to characterize the uncertainty in prediction (modeled as a Gaussian process) made by trained deep learning models. In this paper, we follow this idea to approximate model uncertainty with Monte Carlo dropout. Each weight layer is associated with a parameter p ∈ [0, 1] representing the probability of the weight is dropped out. For a given input x ∗ , random  dropout on each layer unit with probability p can ∗ ,b ∗ ,··· ,b ∗ be repeated M times, from which we obtain multiple estimations b y(1) y(2) y(M) on the prediction on the input ∗ vector x . Then the variance of model prediction can be estimated as: δ ( f ω (x ∗ )) =

M ∗2 1 Õ ∗ b y(m) − b y M m=1

∗

(3) ∗

y denotes the mean value of the M model predictions and can be computed as b y = where b

1 M

M Í m=1

∗ . b y(m)

IV. SWIM SFDPS Data System Wide Information Management (SWIM) is an information technology infrastructure program that operates in the background of National Airspace System (NAS) to broadcast different types of data to authorized users. As a NAS-wide information system, it acts as the fundamental digital backbone to support data exchange needs in the next generation air transportation system (NextGen). The objective of SWIM is to facilitate more efficient, seamless sharing of aeronautical, weather and flight information amongst relevant stakeholders, thereby enabling increased and timely situation awareness on system safety [15]. In SWIM, data is shared through NAS Enterprise Messaging Service (EMS), and users can access SWIM database via publication and subscription to particular channel (i.e., SWIM Terminal Data Distribution System – STDDS, Time Based Flow Management – TBFM, Traffic Flow Management System – TFMS, SWIM Flight Data Publication Service – SFDPS, Wx, and others). In this paper, since we emphasize on the en route flight trajectory prediction, we subscribe to 3

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

the SWIM Flight Data Publication Service (SFDPS) that provides real-time en route flight data to NAS consumers. The data streamed from SFDPS can then be used for analytics, business processes, research, and other activities. SFDPS publishes flight trajectory data via batched track messages in FIXM and custom XML formats. All the en route flight tracking information is expressed in XML format, and there are four major components in each flight tracking message: origin and destination airport, en route flight information, flight ID, and airline. The detailed information for each attribute is listed as below. 1) Orign & Destination: SFDPS includes the departure and arrival airport information of each flight as represented by the ICAO codes, e.g., the ICAO code KBWI denotes the Baltimore/Washington International Thurgood Marshall Airport. 2) En route flight information: Flight position including latitude, longitude and altitude, are reported in the en route flight information. The tracked aircraft velocity along latitude and longitude are also included in the en route information. In addition, the flight plan, such as target altitude and target position, is contained in the tracking message to define the position that each flight is expected to be for every given time instant. 3) Flight ID: A unique identification is assigned to each flight to denote the aircraft identity. Similarly, a unique identification assigned by ERAM is also included to denote the flight plan pertaining to each flight. 4) Airline: This attribute represents the airline that operates the flight reported above. The above tracking messages are normally sent every 12 seconds for an active flight while the flight is in an Air Route Traffic Control Center’s (ARTCC) airspace. Since flight trajectory is our research focus, its relevant attributes need to be derived from the massive XML files in an efficient manner. In parallel, flight messages need to be correlated with the corresponding flight ID properly. One computational challenge we encounter here is that the size of tracking messages streamed from SFDPS per day is approximately 100 GB, and each message contains a batched flight tracking message associated with multiple flight IDs. As data sizes have outpaced the processing capabilities of single machines, efficient and scalable techniques need to leveraged to parse the large number of XML files encompassing various flight trajectory information.

V. Proposed Method In this section, we develop a four-step methodology to predict the trajectory of an ongoing flight by mining the complex patterns embodied in its historical trajectory data. The overall framework of the proposed methodology is illustrated in Fig. 1. In the first step, we develop a scalable program with a unified distributed high-performance computing framework – Apache Spark – to process a large volume of raw flight tracking messages in FIXM format. After the data pertaining to flight trajectory is extracted from the raw data, two individual Bayesian deep learning models are trained to make predictions on the future state of ongoing flights from different perspectives. Next, the predictions from the two deep learning models are fused together to retain both the model prediction accuracy as well as long-term prediction capability. Finally, the fused models are then utilized to make trajectory predictions on multiple flights and evaluate the safety of two given flights, where separation distance is used as a quantitative safety metric. /670511PRGHO

7LPHLQVWDQW

+LVWRULFDO IOLJKW WUDMHFWRU\

)OLJKW,'

)XWXUHVWDWHRI IOLJKWWUDMHFWRU\

9HORFLW\ 6:,0)OLJKW'DWD 3XEOLFDWLRQ6HUYLFH 6)'36

$FWXDOIOLJKWWUDMHFWRU\ 7DUJHWIOLJKWWUDMHFWRU\

Fig. 1

'11PRGHO )OLJKWVWDWH &XUUHQW GHYLDWLRQ

7UDMHFWRU\ GHYLDWLRQDWQH[W WLPHLQVWDQW

Flowchart of proposed methodology for flight trajectory prediction

4

)XWXUHIOLJKW WUDMHFWRU\

6DIHW\ PHDVXUH

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

A. Big Data Processing As mentioned earlier, flight trajectory information needs to be parsed from large volume of XML files. Suppose we plan to build a predictive model by learning from its trajectory in the past 10 days, there will be approximately 1 TB XML files to be processed. Any single desktop does not have enough computational power to process such huge amount of data in a timely manner. Thus, we need to seek scalable and efficient big data computing platform to handle the massive XML files appropriately. In this paper, we choose Apache Spark as a big data engine due to its fast speed (in memory computing), efficient fault tolerant mechanism (resilient distributed dataset), as well as rich APIs in Java, Scala, and Python [16, 17]. We implement Spark program on the high performance computing platform – Advanced Computing Center for Research and Education (ACCRE) at Vanderbilt University. In this regard, there are two major tasks in deriving flight trajectory information: parsing XML files and performing rational queries on trajectory data by flight ID. With respect to the first task, the raw XML files are parsed with the package com.databricks:spark-xml in Spark [18], from which the attributes (i.e., arrival airport, departure airport, time stamp, flight ID, position, altitude, velocity, target position, and others) related to flight trajectory can be extracted. Once the flight trajectory data is obtained, they are grouped by flight ID and date. By doing this, the position data pertaining to each flight is aggregated ranked by time in an ascending order. The trajectory data is saved as csv files named after flight ID and flight date simultaneously. From our analysis, there are more than 20,000 flights within USA per day in the peak moment. Trajectory prediction ෝ𝒕+𝟏 ⋯ 𝒙 ෝ𝒕+𝟓 𝒙

Dense layer

Trajectory deviation at time t

Feed-forward neural network

X velocity

LSTM layer

Y velocity Speed

Trajectory deviation at time t+1

Altitude

LSTM layer Input layer 𝒙𝒕−𝟏𝟗 ⋯ 𝒙𝒕

(a) Deep neural network for trajectory deviation prediction

Fig. 2

(b) Long Short-Term Memory (LSTM) neural network

Structure of two developed deep learning models

B. Two Deep Learning Models Fig. 2 shows two deep learning models constructed in this paper. In the first model, since flight plan is already given, one straightforward approach is to forecast the deviation between the actual flight trajectory and the filed target trajectory over time. The critical question to be addressed here is that given the deviation from the filed flight plan at present and the current airplane state, we aim to predict the trajectory deviation in the next time instant. The structure of the deep neural network (DNN) is shown in Fig. 2a. As can be observed, the deviation at time instant t and the current flight state (e.g., velocity along X and Y, flight altitude, and speed) are model inputs, while we aim to predict the trajectory deviation in the next time instant t + 1. The deep neural network consists of three hidden layers with each hidden layer having 32, 64, and 32 hidden units, respectively, and we use rectified linear units (ReLU) as the activation functions across all the layers, to predict trajectory deviation along latitude and longitude.

5

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

In the second model, we construct a LSTM neural network to forecast the full state of flight trajectory over multiple time steps, including latitude, longitude, altitude, velocity along X and Y. Fig. 2b shows the detailed architecture of the developed LSTM model. As can be seen, the model has two hidden layers, each time we feed the flight trajectory in the past 20 steps (t − 19 · · · t) with the goal to predict the flight state from t + 1 to t + 5. The LSTM model predictions xˆt+1 · · · xˆt+5 can be further used as model inputs, thereby enabling LSTM model to make longer-term predictions on the flight trajectory. To obtain the Bayesian posterior distribution for model prediction, we perform Monte Carlo dropout for both inputs, outputs, and recurrent layers in the LSTM neural network (or hidden layers in the deep neural network) with a dropout probability of 0.05 following the guidance in Gal and Ghahramani [12]. The MC dropout enables us to have an approximated Bayesian posterior distribution on the future state of flight trajectory of interest, in which the uncertainty on the flight trajectory is estimated properly. C. Model Integration Suppose we have observations on the actual flight trajectory up to the time instant t, DNN predicts flight trajectory deviation at the next time instant t + 1 based on the degree of trajectory deviation at the previous time instant t. Given the predicted deviation at time instant t + 1, the predicted flight position at time t + 1 by DNN is readily available by combining the predicted deviation and the filed flight plan. Suppose the LSTM model utilizes the past 20 observed trajectory data to predict the state of flight along the subsequent five time instants t + 1, · · · , t + 5. Let DNN’s trajectory deviation prediction at time instant t + 1 be denoted by τt+1 ; then we have: E ( yˆ t+1 ) = qt+1 + E (τt+1 )

(4)

where qt+1 denotes the planed flight position including latitude, longitude, and altitude at the time instant t + 1, E (τt+1 ) is the mean value of DNN’s trajectory deviation prediction at time instant t + 1, and E ( yˆ t+1 ) represents the mean value of DNN’s prediction on the flight trajectory at time instant t + 1. Considering the two response variables (latitude and longitude) shared by the two models, one straightforward way to integrate the two models is to correct the low-accuracy model with the results from high-accuracy model through a discrepancy term as calculated following Eq. (5). c
∆t+1 = E ( yˆ t+1 ) − E xˆ t+1 (5)   c where E xˆ t+1 denotes the mean value of LSTM model’s prediction on the two common quantities (latitude and longitude) at the time instant t + 1, and ∆t+1 represents the discrepancy between DNN’s mean predictions and LSTM model’s mean predictions. Once the discrepancy between the two model predictions in latitude and longitude prediction at time t + 1 is estimated, then the LSTM model’s prediction on the two quantities for the subsequent time instants t + 1, · · · , t + 5 can be updated accordingly by adding the discrepancy term to its predictions as shown in Eq. (6). x ic = xˆ ic + ∆t+1, ∀i ∈ [t + 1, · · · , t + 5]

(6)

where x ic denotes the updated LSTM model prediction on latitude, longitude, and altitude for the subsequent time instants t + 1, · · · , t + 5. Thus, by combining DNN prediction yˆ t+1 with the LSTM model prediction xˆ t+1 , both high accuracy and longer-term prediction capability are achieved in the integrated model. D. Safety Assessment In the en-route airspace, the minimum horizontal separation distance is 5 nautical miles, while no aircraft should come vertically closer than 300 metres at an altitude of 29,000 feet [5]. With the probabilistic predictions on the trajectory of any two given flights, the safety metric used in this paper is mathematically formulated in the following equation. h   i


I p dih x ic (A) , x ic (B) < δh > λ and p div x ic (A) , x ic (B) < δ v > λ , ∀i ∈ [t + 1, · · · , t + 5] (7) 6

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

where x ic (A) and x
ic (B) denote the integrated model’s predictions on the trajectories of flight A and B, respectively;
dih x ic (A) , x ic (B) and div x ic (A) , x ic (B) represent the horizontal and vertical separation distance between the two flights A and B as forecast by the integrated model. Since the
trajectory prediction of the integrated


model is a probabilistic quantity, a probabilistic metric p dih x ic (A) , x ic (B) < δh and p div x ic (A) , x ic (B) < δ v are defined to measure the degree to which the horizontal and vertical separation distance is violated against the recommended threshold values δh and δv , respectively. Herein, I [·] is an indicator function; if both the horizontal are vertical separation distance are violated, then the indicator function takes a value of one, indicating that the safety of the two flights in terms of separation distance is compromised. Otherwise, its takes a value of zero to indicate the two flights are safe in terms of separation distance.

Fig. 3

The historical trajectories of flight AA598 from 19th December 2018 to 8th February 2019

VI. Computational Results In this section, we show some computational results of the trained Bayesian neural network in trajectory prediction. We extract flight trajectories in the past 48 days by processing 4.2 TB of SFDPS messages with Apache Spark. Fig. 3 shows the historical trajectories of American Airline flight 598 from 19th December to 8th February 2019. The top right and bottom left corners indicate the departure airport (LaGuardia Airport) and destination airport (Charlotte Douglas International Airport), respectively. Regarding the two deep learning models, the DNN model is trained for 15,000 iterations to predict the trajectory deviation, while the LSTM model is trained for 1,500 iterations. To verify the performance of the trained deep learning models, we validate the performance through a double loop program. In the inner loop, we randomly pick a test flight trajectory and train the models with the remaining trajectories for ten times. In the outer loop, we run the program within the inner loop for five times. The root mean squared error (RMSE) between the model predictions and actual values is used as a metric to measure model performance. Since training deep learning models, especially LSTM models, is time-consuming, we accelerate the running speed of the program by executing it on a NVIDIA Pascal GPU. Table 1 compares the performance of the deterministic and probabilistic DNN models on the trajectory deviation prediction. Here, we use the mean value of probabilistic model prediction to calculate the RMSE. As indicated in the comparison results, the RMSE of the probabilistic DNN model has reduced by 2% and 74% for the latitude and longitude deviation prediction, respectively. Similarly, we compare the performance of the LSTM model and the integrated model on flight state prediction. As shown in Table 2, the integrated model demonstrates a much better performance on the

7

common quantities of the two models: latitude and longitude prediction. The value of RMSE is reduced by 47% and 53% compared to the LSTM model, respectively. These cross-validation results show that the integrated model achieves much higher accuracy in flight state prediction. Table 1

Performance comparisons on trajectory deviation prediction by DNN model Deterministic

Table 2

Latitude 3.36 × 10−2

Longitude 9.73 × 10−2

Latitude (µ) 3.29 × 10−2

Latitude 2.48 × 10−2

Ground truth Mean estimate Confidence

0

20

40

60 Time Fig. 4

80

100

Integrated Prediction

Longitude 2.96 × 10−2

Vertical separation distance (Unit: feet)

Root Mean Squared Error

250 225 200 175 150 125 100 75 50

Longitude (µ) 2.51 × 10−2

Performance comparisons on flight state prediction LSTM Prediction

Horizontal separation distance (Unit: miles)

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

Root Mean Squared Error

Probabilistic

120

Latitude 1.32 × 10−2

Longitude 1.38 × 10−2

Ground truth Mean estimate Confidence

8000 6000 4000 2000 0 2000

0

20

40

60 Time

80

100

120

Performance comparison of separation distance

To measure the flight safety, we pick another flight UAL1767 that starts from Denver International Airport and ends at Ronald Reagan Washington National Airport on 23rd January, 2019. Once the deep learning models are trained, we calculate the horizontal and vertical separation distances between the two flights. Fig. 4 compares the model prediction and the ground truth of the separation distance between the two flights. In this figure, the red dashed lines indicate the actual horizontal and vertical separation distances between the two flights, the black solid lines denote the mean value of separation distance as estimated by the probabilistic models, and the shaded areas in dark grey and light grey demonstrate the confidence interval for one standard deviation and two standard deviations away from the mean value, respectively. As can be observed, the deep learning model captures both separation distances very well. The probabilistic metric formulated in Eq. (7) indicates that two flights did not violate the separation distance constraint.

VII. Conclusion In this paper, we develop a multi-fidelity deep learning model to predict the flight trajectory, in which model prediction uncertainty is characterized with a Bayesian approach. To be specific, a deep neural network is trained to predict flight trajectory deviation, while a Long Short-Term Memory (LSTM) neural network is constructed to make long-term forecast on flight trajectory. The two models are combined to achieve both high prediction accuracy and long-term prediction capability. Monte Carlo dropout (MC dropout) is implemented to approximate model prediction 8

uncertainty following a Bayesian manner. The multi-fidelity model is expanded to make predictions on multiple flights, thereby supporting the separation distance-based in-flight safety assessment. The computational results have demonstrated the effectiveness of the developed approach in en-route flight safety assessment. In the future, the impact of weather on the flight trajectory needs to be accounted for in an explicit manner. Currently such information is implicitly reflected through the historical data. However, for prediction regarding an ongoing flight, explicit use of available weather forecast information would be valuable. The challenging question is how to incorporate different types of weather and the probabilistic forecasting of the weather condition into the deep learning models.

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

Acknowledgment The research reported in this paper was supported by funds from NASA University Leadership Initiative program (Grant No. NNX17AJ86A, Project Technical Monitor: Dr. Kai Goebel) through subcontract to Arizona State University (Principal Investigator: Dr. Yongming Liu). The support of FAA and Harris Corporation in accessing the SWIM data is appreciated. This work was conducted in part using the resources of the Advanced Computing Center for Research and Education (ACCRE) at Vanderbilt University, Nashville, TN. The support is gratefully acknowledged.

References [1] de Leege, A., van Paassen, M., and Mulder, M., “A machine learning approach to trajectory prediction,” AIAA Guidance, Navigation, and Control (GNC) Conference, 2013, p. 4782. [2] Ayhan, S., and Samet, H., “Aircraft trajectory prediction made easy with predictive analytics,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2016, pp. 21–30. [3] Roskam, J., Airplane flight dynamics and automatic flight controls, DARcorporation, 1998. [4] Lymperopoulos, I., and Lygeros, J., “Sequential Monte Carlo methods for multi-aircraft trajectory prediction in air traffic management,” International Journal of Adaptive Control and Signal Processing, Vol. 24, No. 10, 2010, pp. 830–849. [5] Daigle, M., Roychoudhury, I., Spirkovska, L., Goebel, K., Sankararaman, S., Ossenfort, J., and Kulkarni, C. S., “Real-Time Prediction of Safety Margins in the National Airspace,” 17th AIAA Aviation Technology, Integration, and Operations Conference, 2017, p. 4388. [6] Di Ciccio, C., Van der Aa, H., Cabanillas, C., Mendling, J., and Prescher, J., “Detecting flight trajectory anomalies and predicting diversions in freight transportation,” Decision Support Systems, Vol. 88, 2016, pp. 1–17. [7] Moradi Kordmahalleh, M., Gorji Sefidmazgi, M., and Homaifar, A., “A Sparse Recurrent Neural Network for Trajectory Prediction of Atlantic Hurricanes,” Proceedings of the Genetic and Evolutionary Computation Conference 2016, ACM, 2016, pp. 957–964. [8] Shi, Z., Xu, M., Pan, Q., Yan, B., and Zhang, H., “LSTM-based Flight Trajectory Prediction,” International Joint Conference on Neural Networks, 2018. [9] Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D., “Weight uncertainty in neural networks,” arXiv preprint arXiv:1505.05424, 2015. [10] Zhu, L., and Laptev, N., “Deep and confident prediction for time series at uber,” 2017 IEEE International Conference on Data Mining Workshops (ICDMW), IEEE, 2017, pp. 103–110. [11] Gal, Y., and Ghahramani, Z., “Dropout as a Bayesian approximation: Representing model uncertainty in deep learning,” International Conference on Machine Learning, 2016, pp. 1050–1059. [12] Gal, Y., and Ghahramani, Z., “A theoretically grounded application of dropout in recurrent neural networks,” Advances in Neural Information Processing Systems, 2016, pp. 1019–1027. [13] Damianou, A., and Lawrence, N., “Deep Gaussian processes,” Artificial Intelligence and Statistics, 2013, pp. 207–215. [14] Gal, Y., and Turner, R., “Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs,” International Conference on Machine Learning, 2015, pp. 655–664.

9

[15] Usmani, A., “System wide information management,” 2010 Integrated Communications Navigation and Surveillance Conference (ICNS), IEEE, 2010, pp. 1–17. [16] Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M. J., et al., “Apache Spark: a unified engine for big data processing,” Communications of the ACM, Vol. 59, No. 11, 2016, pp. 56–65. [17] Shanahan, J. G., and Dai, L., “Large scale distributed data science using apache spark,” Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2015, pp. 2323–2324.

Downloaded by UNIVERSITY OF TEXAS AT AUSTIN on September 14, 2019 | http://arc.aiaa.org | DOI: 10.2514/6.2019-3415

[18] “Spark XML,” https://mvnrepository.com/artifact/com.databricks/spark-xml_2.11/0.4.1, 2018. Accessed: 2018-10-31.

10