- Email: [email protected]
Optimal Disturbance Generation for Flight Control Law Testing ⁎
11th IFAC Symposium on Nonlinear Control Systems 11th IFAC Symposium on Nonlinear Control Systems Vienna, Austria, Sept. 4-6, 2019 11th IFAC Symposium on Nonlinear Control Systems Available online at www.sciencedirect.com Vienna, Austria, Sept. 4-6, 2019 11th IFAC Symposium on Nonlinear Control Systems Vienna, Austria, Sept. 4-6, 2019 Vienna, Austria, Sept. 4-6, 2019
ScienceDirect
IFAC PapersOnLine 52-16 (2019) 730–734
Optimal Optimal Disturbance Disturbance Generation Generation for for Optimal Disturbance Generation for Flight Control Law Testing Optimal Disturbance Generation for Flight Flight Control Control Law Law Testing Testing Flight Control Law Testing Johannes Diepolder ∗ Nikolai Botkin ∗∗ Florian Holzapfel ∗∗∗
∗∗∗ Johannes Diepolder ∗∗∗ Nikolai Botkin ∗∗ ∗∗ Florian Holzapfel ∗∗∗ ∗∗∗ Johannes Diepolder ∗ Nikolai Botkin ∗∗ Florian Holzapfel ∗∗ ∗∗∗ Florian Holzapfel ∗ Johannes Diepolder Nikolai Botkin ∗ Institute of Flight System Dynamics, Technical University of Munich, Institute of Flight System Dynamics, Technical University of Munich, ∗ ∗ Institute of Flight System Technical University of Munich, Boltzmannstraße 15,Dynamics, 85748 Garching, Garching, Germany (e-mail: ∗ Boltzmannstraße 15, 85748 Germany (e-mail: Institute of Flight [email protected]). System Dynamics, Technical University of Munich, Boltzmannstraße 15, 85748 Garching, Germany (e-mail: [email protected]). ∗∗ Boltzmannstraße 15, 85748 Garching, Germany (e-mail: Modeling, [email protected]). ∗∗ Department of Mathematical Modeling, Boltzmannstraße Boltzmannstraße 3, 3, 85748 85748 ∗∗ Department of Mathematical [email protected]). ∗∗ Garching, Germany (e-mail: [email protected]). Department of Mathematical Modeling, Boltzmannstraße 3, 85748 ∗∗ Garching, Germany (e-mail: [email protected]). ∗∗∗ Department of Mathematical Modeling, Boltzmannstraße of 3, 85748 Germany (e-mail: [email protected]). of Dynamics, Technical ∗∗∗ InstituteGarching, of Flight Flight System System Dynamics, Technical University University of Munich, Munich, ∗∗∗ Garching, Germany (e-mail: [email protected]). ∗∗∗ Institute Boltzmannstraße 15, 85748 Garching, Germany (e-mail: Institute of Flight System Dynamics, Technical University of Munich, ∗∗∗ Boltzmannstraße 15, 85748 Garching, Germany (e-mail: Institute of Flight System Dynamics, Technical University of Munich, [email protected]). Boltzmannstraße 15, 85748 Garching, Germany (e-mail: [email protected]). Boltzmannstraße 15, 85748 Garching, Germany (e-mail: [email protected]). [email protected]). Abstract: Abstract: In In this this paper paper we we present present an an approach approach for for testing testing flight flight control control laws laws with with optimal optimal disturbances disturbances Abstract: In this paper we present an approach for testing flight control lawsdifferential with optimal disturbances using differential game theory. The approach is based on a state constrained game formulausing differential game we theory. Thean approach is for based on aflight state constrained differential game formulaAbstract: In this paper present approach testing control laws with optimal disturbances tion in which the criterion under investigation introduced in the cost function. In this differential game using differential game theory. The approach is based on a state constrained differential game formulation in which the criterion under investigation is introduced in the cost function. In this differential game using differential game theory. The approach is based on a state constrained differential game formulathe first player representing disturbances acting on the closed loop system is trying to maximize the value tion in which the criterion under investigation is introduced in the cost function. In this differential game the first player representing disturbances acting on the closed loop system is trying to maximize the value tion incriterion which the criterion under investigation ison introduced inloop the this cost function. Innot this differential game of the at a fixed terminal time point. For our application player may only model typical the first player representing disturbances acting the closed system is trying to maximize the value of the criterion at a fixed terminal time point. For our application this player may not only model typical the firstcriterion playerfor representing disturbances acting onwind the application closed loop system is commands trying to maximize the value of the atflight a fixed terminal time point. For our this player may not only model typical disturbances control systems, such as gusts, but also pilot or other external disturbances foratflight control systems, such For as wind gusts, but also pilot commands or other of the criterion acan fixed terminal time loop point. ourtoapplication this player may not minimizing, only modelexternal typical influences which drive the closed system an unsafe state. The second, player disturbances for flight control systems, such as wind gusts, but also pilot commands or other external influences which can drive thesystems, closed loop system to gusts, an unsafe state.pilot The second, minimizing, player disturbances for flight control such as wind but also or other external represents aa control parametrization and is subject state constraints such as limits for actuator influences which canlaw drive the closed loop system to anto unsafe state. Thecommands second, minimizing, player represents control law parametrization and is subject to state constraints such as limits for actuator influences which canThe drive theofclosed loop system toa an unsafe state. The second, minimizing, player positions and rates. value this game provides lower bound of the criterion under investigation represents a control law parametrization and is subject to state constraints such as limits for actuator positions and rates. The value of this game provides a lower bound of the criterion under investigation represents a control law parametrization and is subject to state constraints such as limits for actuator positions and rates. The value of this game provides a lower bound of the criterion under investigation for the control structure and the bounded disturbances as as bounded parametrization for the given given controlThe structure the bounded disturbances as well well as the the bounded parametrization positions and rates. value ofand this game provides a lower bound ofobtained the criterion under investigation of law. The numerical solution for this problem type is by employing aa highly for the the control given control structure and the bounded disturbances as well as the bounded parametrization of the control law. The numerical solution for this problem type is obtained by employing highly for the given control structure and the bounded disturbances as well as the bounded parametrization parallelized solver implemented on a grid computer. The current implementation of this solver of the control law. The numerical solution for this problem type is obtained by employing a allows highly parallelized solver implemented on a grid computer. The current implementation of this solver allows of the control law. The numerical solution for this problem type is to obtained by of employing aexample highly parallelized solver implemented on a grid computer. Thegames current implementation this solver allows for the solution of nonlinear state constrained differential in up seven dimensions. An for the solution of nonlinear stateon constrained differential inimplementation up to seven dimensions. An example parallelized solver implemented atesting grid computer. Thegames current of this solver allows for the solution of nonlinear state constrained differential games in up toofseven dimensions. An example problem formulation is presented for the longitudinal controller a generic aircraft model using problem formulation is presented for testing the longitudinal controller a generic aircraft model using the solution of command nonlinear state constrained differential games in up toof seven dimensions. An example aafor worst case pilot as disturbance input. problem formulation is presented for testing the longitudinal controller of a generic aircraft model using worst case pilot command as disturbance input. problem formulation is presented for testing the longitudinal controller of a generic aircraft model using a worst case pilot command as disturbance input. 2019,case IFACpilot (International Federation of Automatic a©worst command as disturbance input. Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Keywords: Differential Differential Games, Games, Optimization, Optimization, Aircraft Aircraft Control, Control, Dynamics, Dynamics, Parallel Parallel Computation Computation Keywords: Differential Games, Optimization, Aircraft Control, Dynamics, Parallel Computation Keywords: Differential Games, Optimization, Aircraft Control, Dynamics, Parallel Computation nations 1. nations for for the the angle angle of of attack attack limit limit using using aa step step pull pull pilot pilot 1. INTRODUCTION INTRODUCTION 1. INTRODUCTION nations for In theRef. angle ofOliveira attack limit using (2011) a step pull pilot command. De and Puyou aa control command. In Ref. DeofOliveira and Puyou (2011) control 1. INTRODUCTION nations for the angle attack limit using a step pull pilot input parametrization is used to determine worst case pilot command. In Ref. De Oliveira and Puyou (2011) a control In parametrization isOliveira used toand determine worst case pilot In this this paper paper an an approach approach for for determining determining optimal optimal worst worst case case input command. In Ref. De Puyou (2011) a control commands. An to parameter optimization In this paper for an approach for determining optimal worst case input parametrization is used determinebased worst case pilot disturbances flight law is The An extension extension to the theto parameter based optimization disturbances for flight control control law testing testingoptimal is presented. presented. The commands. parametrization is used determine worst case pilot In this paperconsidered an approach for determining worstinflucase commands. An worst extension todisturbance thetoparameter based optimization approaches for case analysis can be for flight control law testing isbepresented. The input disturbances for this approach may external approaches for worst casetodisturbance analysis can be seen seen in in disturbances considered for this approach mayisbepresented. external influcommands. An extension the parameter based optimization disturbances for flight control law testing The approaches for worst case disturbance analysis can be seen in the application of optimal theory as proposed in Ref. Herrmann disturbances considered for this approachfrom may the be external influences such as wind gusts or commands pilot. The inthe application of optimal theory as proposed in Ref. Herrmann ences such as wind gusts or commands from the pilot. The inapproaches for worst case disturbance analysis can be seen in disturbances considered approach maylaws be external influand Ben-Asher (2016). In this the ences such as wind gustsfor orthis commands from the pilot. The in- the application optimal as proposed in Ref.investigate Herrmann vestigation of robustness of with respect and Ben-Asher of (2016). In theory this reference reference the authors authors investigate vestigation of the the robustness of flight flight control control laws with The respect the application of optimal theory as proposed in Ref. Herrmann ences such as wind gusts or commands from the pilot. inan optimal control formulation in to optimal worst vestigation of the robustness of flightdue control laws with respect and Ben-Asher (2016). In this reference authors investigate to disturbances is high to that optimal control formulation in order order the to find find optimal worst to disturbances is of of high importance importance due to the the fact factwith that regardregardand Ben-Asher (2016). In this reference the authors investigate vestigation of the robustness of on flight control respect an an optimal control formulation in order to This find optimal worst case inputs for flight control law testing. approach can to disturbances is of highacting importance due to thelaws fact thatallowed regardless of the disturbances the system it is not case inputs for flight control law testing. This approach can less of the disturbances acting on the system it is not allowed an optimal control formulation in order to find optimal worst to disturbances is of high importance due to the fact that regardcase inputs for flight control law testing. This approach can be seen as an extension of the finite dimensional parameter less of the disturbances acting on the system it is not allowed to enter regions outside the save envelope. In the last years opbe seen as an extension of the finite dimensional parameter to enter regions outside the save envelope. In the last years opcase inputs for flight control law testing. This approach can lessenter of the disturbances acting on the system it islast notlaws allowed optimization problems considered in the other references to to regions outside the save envelope. In control the years op- be seen as an extension of the finite dimensional parameter timization based approaches for testing flight have optimization problems considered in thedimensional other references to an an timization basedoutside approaches for testing flight control laws have be seen as an extension of the finite parameter to enter regions the save envelope. In the last years opinfinite dimensional optimization in function space. timization based approaches for testing flight control lawset have optimization problems consideredproblem in the other references to an been investigated by various researchers. In Ref. Varga al. infinite dimensional optimization problem in function space. been investigated by various researchers. In Ref. Varga et al. optimization problems consideredproblem in the other references to an timization based approaches testing flight lawset have dimensional optimization in function space. been investigated by variousfor researchers. In control Ref. Varga al. infinite (2012) the the of (2012) the authors authorsbyinvestigate investigate the application application of optimization optimization The current approach differs from theses approaches in the infinite dimensional optimization problem in function space. been investigated various researchers. In Ref. Varga et al. The current approach differs from theses approaches in the (2012) the authors investigate the application of optimization methods for flight control law clearance. Besides local gradient methods for flight control law clearance. Besides local gradient following aspects. We formulate the worst case analysis as The current approach differs from theses approaches in the (2012)methods, the investigate the application oflocal optimization following aspects. We formulate the worst case analysis as methods forauthors flight control law clearance. Besides gradient based several global methods are considered to find The current approach from in the aspects. We differs formulate thetheses worstapproaches case analysis as based methods, several global methods are considered to find following aa zero-sum differential game (cf. Krasovskii (1988)) between methods for flight control law clearance. Besides local gradient zero-sumaspects. differential game (cf. Krasovskii (1988)) between based case methods, severalfor global methods are criteria considered toasfind worst pilot inputs testing clearance such the following We formulate the worst case analysis as worst case pilot inputs for testing clearance criteria such as the two players in which the cost function represents the criterion a zero-sum differential game (cf. Krasovskii (1988)) between based case methods, several global methods are criteria considered toasfind players differential in which thegame cost (cf. function represents the criterion worst pilot inputs for testing clearance suchMenon the two maximum angle of attack exceeding criterion. In Ref. a zero-sum Krasovskii (1988)) between maximum of attack exceeding criterion. In Ref. Menon under consideration. application differential for players in which The the cost functionof thegames criterion worst case angle pilot inputs for analysis testing clearance criteria such as the two under consideration. The application ofrepresents differential for maximum angle of attack exceeding criterion. In Ref. Menon et al. (2006a) aa worst case for testing re-entry vehicles two players in which the cost function represents thegames criterion et al. (2006a) worst case analysis for testing re-entry vehicles the construction of disturbances for linear systems has been under consideration. The application of differential games for maximum angle of the attack exceeding In Ref.vehicles Menon the construction of disturbances for linear systems has been et al. (2006a) a worst case analysis forcriterion. testing re-entry is presented using Dividing Rectangles (DIRECT) optiunder consideration. The application of differential games for construction of disturbances for linear systems hasbybeen is presented using the Dividing Rectangles (DIRECT) opti- the investigated for example in Ref. Botkin et al. (2018) the et al. (2006a) a worst case analysis for testing re-entry vehicles investigated for example in Ref. for Botkin et systems al. (2018) the is presented using the Dividing Rectangles (DIRECT) opti- the mization method to test the robustness of aa nonlinear dynamic construction of disturbances linear hasby been mization method to test the robustness of nonlinear dynamic construction of repulsive disturbances using dynamic programinvestigated for example in Ref. Botkin et al. (2018) by the is presented using Dividing Rectangles (DIRECT) opti- construction of repulsive disturbances using dynamic programmization method to the test the robustness of a nonlinear dynamic inversion controller with to disturbances. investigated for example in Ref. Botkin et numerical al. (2018) by the inversion controller withtherespect respect to parametric parametric disturbances. ming. It is important to mention that the method construction of repulsive disturbances using dynamic programmization method to test robustness of a nonlinear dynamic ming. It is important to mention that the numerical method inversion controller with respect to parametric disturbances. Moreover, in Ref. Menon et al. (2006b) the authors propose construction of repulsive disturbances using dynamic programMoreover, in Ref. Menon et al. (2006b) the authors propose ming. employed the at mention hand capable solving difIt isin important the of numerical method inversion controller with respect tocombines parametric disturbances. employed the paper paperto hand is is that capable solving the the difin Ref. Menon et which al. (2006b) the authors propose aaMoreover, hybrid scheme local global It game isin toat the of numerical method employed inimportant theforpaper at mention hand is that capable of solving the difhybrid optimization optimization scheme which combines local and andpropose global ming. ferential general nonlinear systems. Here, the first Moreover, in Ref. Menon et al. (2006b) the authors ferential game forpaper general nonlinear systems. Here, the first a hybrid optimization scheme which combines local andcombiglobal employed optimization algorithms to find worst case parameter in the at hand is capable of solving the difoptimization algorithms to find worst case parameter combi(maximizing) player represents disturbances (e.g. wind or pilot ferential game for general nonlinear systems. Here, the first a hybrid optimization scheme which combines local andcombiglobal (maximizing) player represents disturbances (e.g.Here, windthe or pilot optimization algorithms to find worst case parameter ferential game for general nonlinear systems. first commands) and the second models gen(maximizing) player disturbancesplayer (e.g. wind or pilot This work is supported by the and TU427/2-2 as optimization algorithms toDFG findgrants worstHO4190/8-2 case parameter combicommands) and the represents second (minimizing) (minimizing) models genThis work is supported by the DFG grants HO4190/8-2 and TU427/2-2 as (maximizing) player represents disturbancesplayer (e.g. wind or pilot well as the Leibniz Supercomputing Centre (grant pr74lu). and TU427/2-2 as commands) and the second (minimizing) player models genThis work is supported by the DFG grants HO4190/8-2 well as the Leibniz Supercomputing Centre (grant pr74lu). commands) and the second (minimizing) player models genThis work is supported by the DFG grants HO4190/8-2 well as the Leibniz Supercomputing Centre (grant pr74lu). and TU427/2-2 as
well as the Leibniz Supercomputing Centre (grant pr74lu).of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2019, IFAC (International Federation Copyright © 2019 IFAC 1284 Copyright 2019 IFAC 1284Control. Peer review© under responsibility of International Federation of Automatic Copyright © 2019 IFAC 1284 10.1016/j.ifacol.2019.12.049 Copyright © 2019 IFAC 1284
2019 IFAC NOLCOS Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
eral functions which parametrize the controller (e.g. scheduled gains or other parameterizations of the aircraft control system). Furthermore, the second player needs to ensure that certain state dependent limits of the system, such as rate or position bounds of the actuator, are not exceeded. Note that the first player (disturbance) is not subject to these limits. The value of this game essentially yields a lower bound of the criterion which can be achieved for a particular control structure under the worst case disturbances generated by the first player. Thus the analysis not only considers one parametrization of the control structure but allows for an analysis regarding all possible control law parameterizations within the admissible domain. So far we are able so obtain numerical solutions to the differential game in up to seven dimensions using a highly parallelized solver implementation on a grid computer. The paper is organized as follows. First, the main approach including the differential game formulation with state constraints is presented. Following, the numerical scheme for computing a solution for this differential game formulation is outlined. Finally, an example problem for testing a closed loop flight control system in the longitudinal plane illustrates the approach by maximizing the angle of attack for a given time interval. 2. MAIN APPROACH Consider the following closed loop dynamic model x˙ = f (x, u, v) , (1) in which x ∈ Rnx is the state vector and u ∈ Q ⊂ Rnu (2) as well as v ∈ P ⊂ Rn v (3) represent inputs to the system. We formulate a differential game on the time interval t0 ≤ t ≤ tf , (4) between the first player represented by u and the second player v. In our problem formulation, v represents parameterizations of the controller in the closed loop system that are allowed to vary over time. An example for the control parameters may be gain-scheduled control parameters or adaptive parameter values which may change over time. On the other hand, u contains all external disturbances and “dangerous” pilot commands. The aim of the second player, having control inputs v at his disposal, is to remain as close as possible to the desired point in the state space, whereas the first player, governing the control inputs u, is intended to maximize the deflection from that point. Furthermore, physical limitations such as actuators limits regarding maximal deflections or rates are taken into account, which can be accomplished by imposing state dependent constraints. These limits only need to be respected by the second (minimizing) player v. Moreover, it is assumed that a performance index is constructed in such a way that the second player, v, achieves his aim through minimization of this index, whereas the first player, u, maximizes the deflection through maximization of the performance index. To formulate this problem as a differential game, let us introduce the following functional from Ref. Botkin et al. (2011) γ (x(·)) = max min σ0 (x), max σ1 (x) , (5) t∈[t0 ,tf ]
t∈[t0 ,tf ]
in which x(·) is a state trajectory evolving from the initial point x(t0 ) = x0 . (6)
731
The functions σ0 (x) and σ1 (x) in Eq. (5) have to satisfy the condition σ1 (x) ≤ σ0 (x), (7) for all x ∈ D where the game domain D ⊂ Rnx is defined as the domain containing all possible trajectories. In Fig. 1 the shapes of a relaxed barrier function σ1 (x) and the function σ0 (x) are illustrated. xlb
xub
σ0 (x)
σ1 (x)
σ1 (x)
Fig. 1. Illustration of the relaxed barrier function σ1 (x) modeling the state constraints xlb ≤ x ≤ xub and the function σ0 (x) (to be minimized by v). Furthermore, let the saddle point condition (Isaacs condition, see Ref. Isaacs (1965)) min maxp, f (x, u, v) = max minp, f (x, u, v), (8) u∈P v∈Q
v∈Q u∈P
be fulfilled. This condition holds in particular if the dynamic model in Eq. (1) is of the form f (x, u, v) = f 1 (x, u) + f 2 (x, v) (9) and is obviously fulfilled for input affine systems. The functional in Eq. (5) can be reduced to (10) γ(x(·)) = max σ0 (x(tf )), max σ1 (x) , t∈[t0 ,tf ]
and the goal of the first player, u, is the maximization of γ(x(·)) whereas the second player, v, minimizes it. Moreover, let the value function V(t0 , x0 ) of this game at the initial point be defined as: V(t0 , x0 ) = min max γ(x(·)) = max min γ(x(·)), V ∈P U ∈Q
U ∈Q V ∈P
(11)
where U (t, x) and V (t, x) are feedback strategies of the first and second player, respectively. Note, that due to Eq. (8) the minimization and maximization operators over the feedback strategies of both players can be interchanged. It is important to observe that after the computation of the value function of the differential game the maximum deflection from the desired point and the worst case disturbances associated with the corresponding optimal control law parametrization are readily available. However, note further that the primary interest of the approach presented in this paper is not in designing the control law parametrization but focuses on testing the control structure based on the resulting value of the game. This value represents a bound for the worst case performance of the given control structure (see schematic illustration in Fig. 2) within the admissible domains for u ∈ Q and v ∈ P which additionally has to satisfy the state constraints (e.g. associated with the actuator states).
1285
2019 IFAC NOLCOS 732 Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
Parametrization (Controller) Disturbances (Commands)
Disturbances (e.g. Wind)
Controller
Actuators
Plant
Fig. 2. Illustration of the closed loop system with the disturbance inputs (first, maximizing player) and the control law parametrization inputs (second, minimizing player satisfying the state constraints). 3. NUMERICAL DIFFERENTIAL GAME SOLUTION The solution of the differential game (cf. Crandall and Lions (1983)) is achieved by a discretization of the state space and the temporal dimension, see Ref. Botkin et al. (2011). For this purpose let us introduce the time discretization step length δ and denote the discrete time by (12) ti = i · δ, i = N − 1, . . . , 0, Similarly, for the discretized states define the mesh lengths T
h = [h0 , ..., hnx −1 ] and introduce the discretized states x i k = ik h k . Using these discretized states, the grid functions σ0h (xi0 , ..., xinx −1 ),
(13) (14) (15)
σ1h (xi0 , ..., xinx −1 )
(16) and grid approximations of the value function at times ti , (17) V i (xi0 , ..., xinx −1 ), can be defined. Furthermore, consider the operator F (V) (x) = min max V x + δ · f (x, u, v) , v∈P u∈Q
(18)
and an interpolation operator I [·] which is used to construct values of the grid functions in between grid points. The following grid scheme for i = N − 1, ..., 0 (19) V i−1 = max F I V i , σ1h ,
V N = σ0h , (20) converges pointwise to the value function of the state constrained differential game. Note that due to the discretization of large domains of the full state space the numerical solution is challenging for higher dimensional spaces (curse of dimensionality). In order to alleviate this effect sparse grid representations can be employed. Furthermore, the structure of the grid scheme enables a very high degree of parallelization which allows for an efficient implementation on grid computers. So far we were able to compute solutions to state constraint differential games in aerospace applications of up to seven dimensions. Recent experiments with sparse grid representations indicate that the solution of problems in eight dimensions may be feasible. This would extend the application of the proposed method to a larger class of practical clearance problems. Moreover, we would like to mention that the scheme does not impose
restrictions on the nonlinearity of the dynamic model which is a major advantage considering the fact that aircraft dynamics are often highly complex and may exhibit strong nonlinearities. 4. EXAMPLE PROBLEM FOR A CLOSED LOOP AIRCRAFT MODEL IN THE LONGITUDINAL PLANE 4.1 Problem Formulation In the following, an example problem is presented for testing a generic aircraft closed loop model in the longitudinal plane. The state vector xlon ∈ R7 describing the aircraft closed loop system T xlon = [V, α, q, θ, η, η, ˙ e] , (21) contains the velocity V , the angle of attack α, pitch rate q, pitch angle θ, as well as the two actuator states η and η˙ for the elevator deflection and rate, respectively. Furthermore, the variable e represents an integrator state of the controller. Note that the closed loop system is modeled using seven states which is within the capability of the numerical solver developed by the authors. The plant model of the aircraft for the longitudinal motion is of the form (XT )K¯ V˙ = (22) m (ZT )K¯ +q (23) α˙ = mV θ˙ = q (24) (MT )B (25) q˙ = Iy with the total (subscript T ) forces (XT )K¯ , and (ZT )K¯ in the ¯ as well as x and z direction of the rotated kinematic frame K the total moments (MT )B around the y axis of the body fixed frame B. These total forces and moments originate from the gravitational, aerodynamic, and propulsion forces and moments acting on the airframe. Furthermore, the mass of the vehicle and the moment of inertia for the y-axis in the body-fixed frame are denoted by m and Iy . The aircraft is assumed to be initially in a steady state flight condition (horizontal, straight and level flight) at a reference altitude and speed with an initial angle of attack α0 = 3.68 deg. The thrust is held constant at the reference value for the entire time interval of the game. The dynamic equations including the pilot command input u, representing the first (maximizing)
1286
2019 IFAC NOLCOS Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
player and the inputs for the control parametrization v, modeling the second (minimizing) player, are described by a set of ordinary differential equations of the form x˙ lon = f (xlon , u, v) . (26) The actuator model for the elevator control surface is modeled as a linear second order system ω2 G(s) = 2 , (27) s + 2ζωs + ω 2 with natural frequency ω and damping coefficient ζ. Furthermore, the actuator states for the elevator deflection η and rate η˙ are subject to lower and upper bounds which need to be maintained by the player v representing the controller parametrization. It is interesting to note that the system (26) satisfies the saddle point condition (8), which has been verified numerically, although Eq. (26) is not of the form (9) because it contains products between the states x and the inputs u and v. The cost function σ0 , see Fig. 1, is defined as a linear function of the angle of attack α(tf ). Namely, (28) σ0 (α(tf )) = 15/π α(tf ) − 1, at the final time of the game tf = 5s. The function σ1 , responsible for the state constraints, is chosen as σ1 (xlon ) = ρK (xlon ) − 1, (29) where ρK (xlon ) is the Minkowski function of the state constraint set K. The single input of the first (maximizing) player represents the pilot command for the normal load factor nz,c which is subject to box bounds of the form nz,c,lb ≤ nz,c ≤ nz,c,ub . (30) The opposing (minimizing) player v has four parameters of the controller k1...4 at his disposal which are assumed to be bounded from above and below k1...4,lb ≤ k1...4 ≤ k1...4,ub . (31) 4.2 Numerical Results In this section numerical results for the example problem are presented. The numerical solution for the differential game is obtained using the grid scheme presented in Sec. 3. The value function in the subspace of the state variables α and η˙ is depicted in Fig. 3, and the optimal value of the game is α∗ (tf ) ≈ 16.5 deg according to Eq. (28).
There are several important observations regarding the numerical solution of the differential game problem. Firstly, it can be seen in Fig. 3 that the value function exhibits slightly sloping parts close to the actuator rate limits. This is contributed to the fact that the actuator rate dynamics are fast compared to the dynamics governing the other state variables, and, therefore, it is difficult to keep η˙ inside its constraints. It was found that this fact poses a major challenge for the numerical solution using the grid scheme. Moreover, the value of the time discretization step δ is dominated primarily by the dynamics of the actuator rate and needs to be chosen very small in order to capture the dynamic behaviour of the actuator and ensure a stable numerical solution. The second challenge in solving the differential game is related to the state discretization. For the seven dimensional nonlinear model considered in this example a relatively coarse discretization 20 × 10 × 10 × 10 × 10 × 16 × 16 was chosen in order
733
V alue
η˙ [rad/s]
α [rad] Fig. 3. Value function of the game in the subspace of the state variables α and η. ˙ to enable a solution of the differential game with the current implementation of the solver. The authors would like to emphasize that solving the differential game for this nonlinear system and the given state and control dimensions was found to be computationally very demanding and that a finer discretization regarding the time step δ and the states may further improve the results. 4.3 Discussion of the Modeling Philosophy and Interpretation of the Numerical Results From a control design perspective it is reasonable to assume that the control law with its parametrization v is responsible for not exceeding the bounds of the actuator rate and position limits. This implies that the control law structure needs to be designed and parametrized in such a way that the actuator bounds can be maintained. It is important to mention that the inclusion of actuator limits in the testing procedures of flight control laws is essential for the investigating of worst case disturbances. This is contributed to the fact that these effects can result in potentially hazardous flight conditions due the deteriorated controllability of the system. Note further that in the differential game formulation presented in this paper the limits regarding the actuator bounds are not imposed on the player corresponding to the pilot command input u. This implies that the minimizing player v which has the parametrization of the control law at his disposal tries to maintain the actuator state bounds and at the same time the maximizing player u, acting as the disturbance in our formulation, may try to violate these constraints, i.e. drive the actuators into saturation. Essentially, the result from the differential game can be interpreted as follows. From the value function of the game two optimal state feedbacks, the optimal control law parametrization v∗ (x) on the one side and the corresponding worst case disturbance input u∗ (x) on the other side, can be constructed. These two optimal feedback strategies lead to the optimal value of the criterion under investigation, i.e. the maximum angle of
1287
2019 IFAC NOLCOS 734 Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
attack α∗ (tf ) for the example problem presented in this section. Now, consider the case where a suboptimal parametrization ˜ (x) is chosen based on some control deof the controller v sign methodology (e.g. by using gain scheduling over state dependent quantities such as static and dynamic pressure). If u∗ (x) is chosen as the worst case disturbance input and the ˜ (x) = v∗ (x) is used for the control law suboptimal feedback v parametrization it is not possible to obtain a value α ˜ (tf ) lower than the optimal cost value α∗ (tf ) which implies α ˜ (tf ) ≥ α∗ (tf ) (32) Thus, the value of the game represents a lower bound for the criterion which can be achieved with a given control structure and the bounded control law parametrization v in combination with the bounded disturbance inputs u.
Menon, P.P., Fernandez, V., and Bennani, a.S. (2006a). Worstcase analysis of flight control laws for re-entry vehicles. Menon, P.P., Kim, J., Bates, D.G., and Postlethwaite, I. (2006b). Clearance of nonlinear flight control laws using hybrid evolutionary optimization. IEEE Transactions on Evolutionary Computation, 10(6), 689–699. Varga, A., Hansson, A., and Puyou, G. (2012). Optimization Based Clearance of Flight Control Laws: A Civil Aircraft Application, volume 416 of Lecture Notes in Control and Information Sciences. Springer Berlin Heidelberg, Berlin, Heidelberg. doi:10.1007/978-3-642-22627-4.
5. CONCLUSIONS An approach was presented for the computation of optimal disturbances for testing flight control laws. Major challenges for the numerical solution of the differential game are currently the rapidly increasing computational cost regarding the state dimensions and the fine temporal discretization necessary for capturing fast dynamics such as for the actuator rates. The solution for an example problem formulation for a flight control system in the longitudinal plane in seven dimensions including state constraints on the actuator deflection and rate illustrates the main characteristics of the scheme. It is important to mention that the solver implementation for the differential game solution is an ongoing task and the consideration of higher state dimensions and finer state and time discretization steps is likely to improve the numerical solution. Furthermore, the authors are currently investigating the application of direct optimal control methods (cf. Betts (2010); Gerdts (2012)) to verify the optimality of the disturbances obtained from the differential game solution. REFERENCES Betts, J.T. (2010). Practical Methods for Optimal Control and Estimation Using Nonlinear Programming. Society for Industrial and Applied Mathematics. Botkin, N., Martynov, K., Turova, V., and Diepolder, J. (2018). Generation of dangerous disturbances for flight systems. Dynamic Games and Applications. doi:10.1007/s13235-0180259-5. Botkin, N., Hoffmann, K.H., Mayer, N., and Turova, V. (2011). Approximation schemes for solving disturbed control problems with non-terminal time and state constraints. Analysis, 31(4), 355–379. Crandall, M.G. and Lions, P.L. (1983). Viscosity solutions of hamilton-jacobi equations. T. Am. Math. Soc., 277, 1–47. De Oliveira, R.F. and Puyou, G. (2011). On the use of optimization for flight control laws clearance: a practical approach. IFAC Proceedings Volumes, 44(1), 9881–9886. Gerdts, M. (2012). Optimal Control of ODEs and DAEs. De Gruyter textbook. Walter de Gruyter GmbH Co.KG, s.l., 1. aufl. edition. doi:10.1515/9783110249996. Herrmann, A.A. and Ben-Asher, J.Z. (2016). Flight control law clearance using optimal control theory. Journal of Aircraft, 53(2), 515–529. Isaacs, R. (1965). Differential Games. John Wiley and Sons, New York. Krasovskii, N.N. (1988). Game-theoretical control problems. Springer, New York. 1288
ScienceDirect
IFAC PapersOnLine 52-16 (2019) 730–734
Optimal Optimal Disturbance Disturbance Generation Generation for for Optimal Disturbance Generation for Flight Control Law Testing Optimal Disturbance Generation for Flight Flight Control Control Law Law Testing Testing Flight Control Law Testing Johannes Diepolder ∗ Nikolai Botkin ∗∗ Florian Holzapfel ∗∗∗
∗∗∗ Johannes Diepolder ∗∗∗ Nikolai Botkin ∗∗ ∗∗ Florian Holzapfel ∗∗∗ ∗∗∗ Johannes Diepolder ∗ Nikolai Botkin ∗∗ Florian Holzapfel ∗∗ ∗∗∗ Florian Holzapfel ∗ Johannes Diepolder Nikolai Botkin ∗ Institute of Flight System Dynamics, Technical University of Munich, Institute of Flight System Dynamics, Technical University of Munich, ∗ ∗ Institute of Flight System Technical University of Munich, Boltzmannstraße 15,Dynamics, 85748 Garching, Garching, Germany (e-mail: ∗ Boltzmannstraße 15, 85748 Germany (e-mail: Institute of Flight [email protected]). System Dynamics, Technical University of Munich, Boltzmannstraße 15, 85748 Garching, Germany (e-mail: [email protected]). ∗∗ Boltzmannstraße 15, 85748 Garching, Germany (e-mail: Modeling, [email protected]). ∗∗ Department of Mathematical Modeling, Boltzmannstraße Boltzmannstraße 3, 3, 85748 85748 ∗∗ Department of Mathematical [email protected]). ∗∗ Garching, Germany (e-mail: [email protected]). Department of Mathematical Modeling, Boltzmannstraße 3, 85748 ∗∗ Garching, Germany (e-mail: [email protected]). ∗∗∗ Department of Mathematical Modeling, Boltzmannstraße of 3, 85748 Germany (e-mail: [email protected]). of Dynamics, Technical ∗∗∗ InstituteGarching, of Flight Flight System System Dynamics, Technical University University of Munich, Munich, ∗∗∗ Garching, Germany (e-mail: [email protected]). ∗∗∗ Institute Boltzmannstraße 15, 85748 Garching, Germany (e-mail: Institute of Flight System Dynamics, Technical University of Munich, ∗∗∗ Boltzmannstraße 15, 85748 Garching, Germany (e-mail: Institute of Flight System Dynamics, Technical University of Munich, [email protected]). Boltzmannstraße 15, 85748 Garching, Germany (e-mail: [email protected]). Boltzmannstraße 15, 85748 Garching, Germany (e-mail: [email protected]). [email protected]). Abstract: Abstract: In In this this paper paper we we present present an an approach approach for for testing testing flight flight control control laws laws with with optimal optimal disturbances disturbances Abstract: In this paper we present an approach for testing flight control lawsdifferential with optimal disturbances using differential game theory. The approach is based on a state constrained game formulausing differential game we theory. Thean approach is for based on aflight state constrained differential game formulaAbstract: In this paper present approach testing control laws with optimal disturbances tion in which the criterion under investigation introduced in the cost function. In this differential game using differential game theory. The approach is based on a state constrained differential game formulation in which the criterion under investigation is introduced in the cost function. In this differential game using differential game theory. The approach is based on a state constrained differential game formulathe first player representing disturbances acting on the closed loop system is trying to maximize the value tion in which the criterion under investigation is introduced in the cost function. In this differential game the first player representing disturbances acting on the closed loop system is trying to maximize the value tion incriterion which the criterion under investigation ison introduced inloop the this cost function. Innot this differential game of the at a fixed terminal time point. For our application player may only model typical the first player representing disturbances acting the closed system is trying to maximize the value of the criterion at a fixed terminal time point. For our application this player may not only model typical the firstcriterion playerfor representing disturbances acting onwind the application closed loop system is commands trying to maximize the value of the atflight a fixed terminal time point. For our this player may not only model typical disturbances control systems, such as gusts, but also pilot or other external disturbances foratflight control systems, such For as wind gusts, but also pilot commands or other of the criterion acan fixed terminal time loop point. ourtoapplication this player may not minimizing, only modelexternal typical influences which drive the closed system an unsafe state. The second, player disturbances for flight control systems, such as wind gusts, but also pilot commands or other external influences which can drive thesystems, closed loop system to gusts, an unsafe state.pilot The second, minimizing, player disturbances for flight control such as wind but also or other external represents aa control parametrization and is subject state constraints such as limits for actuator influences which canlaw drive the closed loop system to anto unsafe state. Thecommands second, minimizing, player represents control law parametrization and is subject to state constraints such as limits for actuator influences which canThe drive theofclosed loop system toa an unsafe state. The second, minimizing, player positions and rates. value this game provides lower bound of the criterion under investigation represents a control law parametrization and is subject to state constraints such as limits for actuator positions and rates. The value of this game provides a lower bound of the criterion under investigation represents a control law parametrization and is subject to state constraints such as limits for actuator positions and rates. The value of this game provides a lower bound of the criterion under investigation for the control structure and the bounded disturbances as as bounded parametrization for the given given controlThe structure the bounded disturbances as well well as the the bounded parametrization positions and rates. value ofand this game provides a lower bound ofobtained the criterion under investigation of law. The numerical solution for this problem type is by employing aa highly for the the control given control structure and the bounded disturbances as well as the bounded parametrization of the control law. The numerical solution for this problem type is obtained by employing highly for the given control structure and the bounded disturbances as well as the bounded parametrization parallelized solver implemented on a grid computer. The current implementation of this solver of the control law. The numerical solution for this problem type is obtained by employing a allows highly parallelized solver implemented on a grid computer. The current implementation of this solver allows of the control law. The numerical solution for this problem type is to obtained by of employing aexample highly parallelized solver implemented on a grid computer. Thegames current implementation this solver allows for the solution of nonlinear state constrained differential in up seven dimensions. An for the solution of nonlinear stateon constrained differential inimplementation up to seven dimensions. An example parallelized solver implemented atesting grid computer. Thegames current of this solver allows for the solution of nonlinear state constrained differential games in up toofseven dimensions. An example problem formulation is presented for the longitudinal controller a generic aircraft model using problem formulation is presented for testing the longitudinal controller a generic aircraft model using the solution of command nonlinear state constrained differential games in up toof seven dimensions. An example aafor worst case pilot as disturbance input. problem formulation is presented for testing the longitudinal controller of a generic aircraft model using worst case pilot command as disturbance input. problem formulation is presented for testing the longitudinal controller of a generic aircraft model using a worst case pilot command as disturbance input. 2019,case IFACpilot (International Federation of Automatic a©worst command as disturbance input. Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Keywords: Differential Differential Games, Games, Optimization, Optimization, Aircraft Aircraft Control, Control, Dynamics, Dynamics, Parallel Parallel Computation Computation Keywords: Differential Games, Optimization, Aircraft Control, Dynamics, Parallel Computation Keywords: Differential Games, Optimization, Aircraft Control, Dynamics, Parallel Computation nations 1. nations for for the the angle angle of of attack attack limit limit using using aa step step pull pull pilot pilot 1. INTRODUCTION INTRODUCTION 1. INTRODUCTION nations for In theRef. angle ofOliveira attack limit using (2011) a step pull pilot command. De and Puyou aa control command. In Ref. DeofOliveira and Puyou (2011) control 1. INTRODUCTION nations for the angle attack limit using a step pull pilot input parametrization is used to determine worst case pilot command. In Ref. De Oliveira and Puyou (2011) a control In parametrization isOliveira used toand determine worst case pilot In this this paper paper an an approach approach for for determining determining optimal optimal worst worst case case input command. In Ref. De Puyou (2011) a control commands. An to parameter optimization In this paper for an approach for determining optimal worst case input parametrization is used determinebased worst case pilot disturbances flight law is The An extension extension to the theto parameter based optimization disturbances for flight control control law testing testingoptimal is presented. presented. The commands. parametrization is used determine worst case pilot In this paperconsidered an approach for determining worstinflucase commands. An worst extension todisturbance thetoparameter based optimization approaches for case analysis can be for flight control law testing isbepresented. The input disturbances for this approach may external approaches for worst casetodisturbance analysis can be seen seen in in disturbances considered for this approach mayisbepresented. external influcommands. An extension the parameter based optimization disturbances for flight control law testing The approaches for worst case disturbance analysis can be seen in the application of optimal theory as proposed in Ref. Herrmann disturbances considered for this approachfrom may the be external influences such as wind gusts or commands pilot. The inthe application of optimal theory as proposed in Ref. Herrmann ences such as wind gusts or commands from the pilot. The inapproaches for worst case disturbance analysis can be seen in disturbances considered approach maylaws be external influand Ben-Asher (2016). In this the ences such as wind gustsfor orthis commands from the pilot. The in- the application optimal as proposed in Ref.investigate Herrmann vestigation of robustness of with respect and Ben-Asher of (2016). In theory this reference reference the authors authors investigate vestigation of the the robustness of flight flight control control laws with The respect the application of optimal theory as proposed in Ref. Herrmann ences such as wind gusts or commands from the pilot. inan optimal control formulation in to optimal worst vestigation of the robustness of flightdue control laws with respect and Ben-Asher (2016). In this reference authors investigate to disturbances is high to that optimal control formulation in order order the to find find optimal worst to disturbances is of of high importance importance due to the the fact factwith that regardregardand Ben-Asher (2016). In this reference the authors investigate vestigation of the robustness of on flight control respect an an optimal control formulation in order to This find optimal worst case inputs for flight control law testing. approach can to disturbances is of highacting importance due to thelaws fact thatallowed regardless of the disturbances the system it is not case inputs for flight control law testing. This approach can less of the disturbances acting on the system it is not allowed an optimal control formulation in order to find optimal worst to disturbances is of high importance due to the fact that regardcase inputs for flight control law testing. This approach can be seen as an extension of the finite dimensional parameter less of the disturbances acting on the system it is not allowed to enter regions outside the save envelope. In the last years opbe seen as an extension of the finite dimensional parameter to enter regions outside the save envelope. In the last years opcase inputs for flight control law testing. This approach can lessenter of the disturbances acting on the system it islast notlaws allowed optimization problems considered in the other references to to regions outside the save envelope. In control the years op- be seen as an extension of the finite dimensional parameter timization based approaches for testing flight have optimization problems considered in thedimensional other references to an an timization basedoutside approaches for testing flight control laws have be seen as an extension of the finite parameter to enter regions the save envelope. In the last years opinfinite dimensional optimization in function space. timization based approaches for testing flight control lawset have optimization problems consideredproblem in the other references to an been investigated by various researchers. In Ref. Varga al. infinite dimensional optimization problem in function space. been investigated by various researchers. In Ref. Varga et al. optimization problems consideredproblem in the other references to an timization based approaches testing flight lawset have dimensional optimization in function space. been investigated by variousfor researchers. In control Ref. Varga al. infinite (2012) the the of (2012) the authors authorsbyinvestigate investigate the application application of optimization optimization The current approach differs from theses approaches in the infinite dimensional optimization problem in function space. been investigated various researchers. In Ref. Varga et al. The current approach differs from theses approaches in the (2012) the authors investigate the application of optimization methods for flight control law clearance. Besides local gradient methods for flight control law clearance. Besides local gradient following aspects. We formulate the worst case analysis as The current approach differs from theses approaches in the (2012)methods, the investigate the application oflocal optimization following aspects. We formulate the worst case analysis as methods forauthors flight control law clearance. Besides gradient based several global methods are considered to find The current approach from in the aspects. We differs formulate thetheses worstapproaches case analysis as based methods, several global methods are considered to find following aa zero-sum differential game (cf. Krasovskii (1988)) between methods for flight control law clearance. Besides local gradient zero-sumaspects. differential game (cf. Krasovskii (1988)) between based case methods, severalfor global methods are criteria considered toasfind worst pilot inputs testing clearance such the following We formulate the worst case analysis as worst case pilot inputs for testing clearance criteria such as the two players in which the cost function represents the criterion a zero-sum differential game (cf. Krasovskii (1988)) between based case methods, several global methods are criteria considered toasfind players differential in which thegame cost (cf. function represents the criterion worst pilot inputs for testing clearance suchMenon the two maximum angle of attack exceeding criterion. In Ref. a zero-sum Krasovskii (1988)) between maximum of attack exceeding criterion. In Ref. Menon under consideration. application differential for players in which The the cost functionof thegames criterion worst case angle pilot inputs for analysis testing clearance criteria such as the two under consideration. The application ofrepresents differential for maximum angle of attack exceeding criterion. In Ref. Menon et al. (2006a) aa worst case for testing re-entry vehicles two players in which the cost function represents thegames criterion et al. (2006a) worst case analysis for testing re-entry vehicles the construction of disturbances for linear systems has been under consideration. The application of differential games for maximum angle of the attack exceeding In Ref.vehicles Menon the construction of disturbances for linear systems has been et al. (2006a) a worst case analysis forcriterion. testing re-entry is presented using Dividing Rectangles (DIRECT) optiunder consideration. The application of differential games for construction of disturbances for linear systems hasbybeen is presented using the Dividing Rectangles (DIRECT) opti- the investigated for example in Ref. Botkin et al. (2018) the et al. (2006a) a worst case analysis for testing re-entry vehicles investigated for example in Ref. for Botkin et systems al. (2018) the is presented using the Dividing Rectangles (DIRECT) opti- the mization method to test the robustness of aa nonlinear dynamic construction of disturbances linear hasby been mization method to test the robustness of nonlinear dynamic construction of repulsive disturbances using dynamic programinvestigated for example in Ref. Botkin et al. (2018) by the is presented using Dividing Rectangles (DIRECT) opti- construction of repulsive disturbances using dynamic programmization method to the test the robustness of a nonlinear dynamic inversion controller with to disturbances. investigated for example in Ref. Botkin et numerical al. (2018) by the inversion controller withtherespect respect to parametric parametric disturbances. ming. It is important to mention that the method construction of repulsive disturbances using dynamic programmization method to test robustness of a nonlinear dynamic ming. It is important to mention that the numerical method inversion controller with respect to parametric disturbances. Moreover, in Ref. Menon et al. (2006b) the authors propose construction of repulsive disturbances using dynamic programMoreover, in Ref. Menon et al. (2006b) the authors propose ming. employed the at mention hand capable solving difIt isin important the of numerical method inversion controller with respect tocombines parametric disturbances. employed the paper paperto hand is is that capable solving the the difin Ref. Menon et which al. (2006b) the authors propose aaMoreover, hybrid scheme local global It game isin toat the of numerical method employed inimportant theforpaper at mention hand is that capable of solving the difhybrid optimization optimization scheme which combines local and andpropose global ming. ferential general nonlinear systems. Here, the first Moreover, in Ref. Menon et al. (2006b) the authors ferential game forpaper general nonlinear systems. Here, the first a hybrid optimization scheme which combines local andcombiglobal employed optimization algorithms to find worst case parameter in the at hand is capable of solving the difoptimization algorithms to find worst case parameter combi(maximizing) player represents disturbances (e.g. wind or pilot ferential game for general nonlinear systems. Here, the first a hybrid optimization scheme which combines local andcombiglobal (maximizing) player represents disturbances (e.g.Here, windthe or pilot optimization algorithms to find worst case parameter ferential game for general nonlinear systems. first commands) and the second models gen(maximizing) player disturbancesplayer (e.g. wind or pilot This work is supported by the and TU427/2-2 as optimization algorithms toDFG findgrants worstHO4190/8-2 case parameter combicommands) and the represents second (minimizing) (minimizing) models genThis work is supported by the DFG grants HO4190/8-2 and TU427/2-2 as (maximizing) player represents disturbancesplayer (e.g. wind or pilot well as the Leibniz Supercomputing Centre (grant pr74lu). and TU427/2-2 as commands) and the second (minimizing) player models genThis work is supported by the DFG grants HO4190/8-2 well as the Leibniz Supercomputing Centre (grant pr74lu). commands) and the second (minimizing) player models genThis work is supported by the DFG grants HO4190/8-2 well as the Leibniz Supercomputing Centre (grant pr74lu). and TU427/2-2 as
well as the Leibniz Supercomputing Centre (grant pr74lu).of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2019, IFAC (International Federation Copyright © 2019 IFAC 1284 Copyright 2019 IFAC 1284Control. Peer review© under responsibility of International Federation of Automatic Copyright © 2019 IFAC 1284 10.1016/j.ifacol.2019.12.049 Copyright © 2019 IFAC 1284
2019 IFAC NOLCOS Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
eral functions which parametrize the controller (e.g. scheduled gains or other parameterizations of the aircraft control system). Furthermore, the second player needs to ensure that certain state dependent limits of the system, such as rate or position bounds of the actuator, are not exceeded. Note that the first player (disturbance) is not subject to these limits. The value of this game essentially yields a lower bound of the criterion which can be achieved for a particular control structure under the worst case disturbances generated by the first player. Thus the analysis not only considers one parametrization of the control structure but allows for an analysis regarding all possible control law parameterizations within the admissible domain. So far we are able so obtain numerical solutions to the differential game in up to seven dimensions using a highly parallelized solver implementation on a grid computer. The paper is organized as follows. First, the main approach including the differential game formulation with state constraints is presented. Following, the numerical scheme for computing a solution for this differential game formulation is outlined. Finally, an example problem for testing a closed loop flight control system in the longitudinal plane illustrates the approach by maximizing the angle of attack for a given time interval. 2. MAIN APPROACH Consider the following closed loop dynamic model x˙ = f (x, u, v) , (1) in which x ∈ Rnx is the state vector and u ∈ Q ⊂ Rnu (2) as well as v ∈ P ⊂ Rn v (3) represent inputs to the system. We formulate a differential game on the time interval t0 ≤ t ≤ tf , (4) between the first player represented by u and the second player v. In our problem formulation, v represents parameterizations of the controller in the closed loop system that are allowed to vary over time. An example for the control parameters may be gain-scheduled control parameters or adaptive parameter values which may change over time. On the other hand, u contains all external disturbances and “dangerous” pilot commands. The aim of the second player, having control inputs v at his disposal, is to remain as close as possible to the desired point in the state space, whereas the first player, governing the control inputs u, is intended to maximize the deflection from that point. Furthermore, physical limitations such as actuators limits regarding maximal deflections or rates are taken into account, which can be accomplished by imposing state dependent constraints. These limits only need to be respected by the second (minimizing) player v. Moreover, it is assumed that a performance index is constructed in such a way that the second player, v, achieves his aim through minimization of this index, whereas the first player, u, maximizes the deflection through maximization of the performance index. To formulate this problem as a differential game, let us introduce the following functional from Ref. Botkin et al. (2011) γ (x(·)) = max min σ0 (x), max σ1 (x) , (5) t∈[t0 ,tf ]
t∈[t0 ,tf ]
in which x(·) is a state trajectory evolving from the initial point x(t0 ) = x0 . (6)
731
The functions σ0 (x) and σ1 (x) in Eq. (5) have to satisfy the condition σ1 (x) ≤ σ0 (x), (7) for all x ∈ D where the game domain D ⊂ Rnx is defined as the domain containing all possible trajectories. In Fig. 1 the shapes of a relaxed barrier function σ1 (x) and the function σ0 (x) are illustrated. xlb
xub
σ0 (x)
σ1 (x)
σ1 (x)
Fig. 1. Illustration of the relaxed barrier function σ1 (x) modeling the state constraints xlb ≤ x ≤ xub and the function σ0 (x) (to be minimized by v). Furthermore, let the saddle point condition (Isaacs condition, see Ref. Isaacs (1965)) min maxp, f (x, u, v) = max minp, f (x, u, v), (8) u∈P v∈Q
v∈Q u∈P
be fulfilled. This condition holds in particular if the dynamic model in Eq. (1) is of the form f (x, u, v) = f 1 (x, u) + f 2 (x, v) (9) and is obviously fulfilled for input affine systems. The functional in Eq. (5) can be reduced to (10) γ(x(·)) = max σ0 (x(tf )), max σ1 (x) , t∈[t0 ,tf ]
and the goal of the first player, u, is the maximization of γ(x(·)) whereas the second player, v, minimizes it. Moreover, let the value function V(t0 , x0 ) of this game at the initial point be defined as: V(t0 , x0 ) = min max γ(x(·)) = max min γ(x(·)), V ∈P U ∈Q
U ∈Q V ∈P
(11)
where U (t, x) and V (t, x) are feedback strategies of the first and second player, respectively. Note, that due to Eq. (8) the minimization and maximization operators over the feedback strategies of both players can be interchanged. It is important to observe that after the computation of the value function of the differential game the maximum deflection from the desired point and the worst case disturbances associated with the corresponding optimal control law parametrization are readily available. However, note further that the primary interest of the approach presented in this paper is not in designing the control law parametrization but focuses on testing the control structure based on the resulting value of the game. This value represents a bound for the worst case performance of the given control structure (see schematic illustration in Fig. 2) within the admissible domains for u ∈ Q and v ∈ P which additionally has to satisfy the state constraints (e.g. associated with the actuator states).
1285
2019 IFAC NOLCOS 732 Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
Parametrization (Controller) Disturbances (Commands)
Disturbances (e.g. Wind)
Controller
Actuators
Plant
Fig. 2. Illustration of the closed loop system with the disturbance inputs (first, maximizing player) and the control law parametrization inputs (second, minimizing player satisfying the state constraints). 3. NUMERICAL DIFFERENTIAL GAME SOLUTION The solution of the differential game (cf. Crandall and Lions (1983)) is achieved by a discretization of the state space and the temporal dimension, see Ref. Botkin et al. (2011). For this purpose let us introduce the time discretization step length δ and denote the discrete time by (12) ti = i · δ, i = N − 1, . . . , 0, Similarly, for the discretized states define the mesh lengths T
h = [h0 , ..., hnx −1 ] and introduce the discretized states x i k = ik h k . Using these discretized states, the grid functions σ0h (xi0 , ..., xinx −1 ),
(13) (14) (15)
σ1h (xi0 , ..., xinx −1 )
(16) and grid approximations of the value function at times ti , (17) V i (xi0 , ..., xinx −1 ), can be defined. Furthermore, consider the operator F (V) (x) = min max V x + δ · f (x, u, v) , v∈P u∈Q
(18)
and an interpolation operator I [·] which is used to construct values of the grid functions in between grid points. The following grid scheme for i = N − 1, ..., 0 (19) V i−1 = max F I V i , σ1h ,
V N = σ0h , (20) converges pointwise to the value function of the state constrained differential game. Note that due to the discretization of large domains of the full state space the numerical solution is challenging for higher dimensional spaces (curse of dimensionality). In order to alleviate this effect sparse grid representations can be employed. Furthermore, the structure of the grid scheme enables a very high degree of parallelization which allows for an efficient implementation on grid computers. So far we were able to compute solutions to state constraint differential games in aerospace applications of up to seven dimensions. Recent experiments with sparse grid representations indicate that the solution of problems in eight dimensions may be feasible. This would extend the application of the proposed method to a larger class of practical clearance problems. Moreover, we would like to mention that the scheme does not impose
restrictions on the nonlinearity of the dynamic model which is a major advantage considering the fact that aircraft dynamics are often highly complex and may exhibit strong nonlinearities. 4. EXAMPLE PROBLEM FOR A CLOSED LOOP AIRCRAFT MODEL IN THE LONGITUDINAL PLANE 4.1 Problem Formulation In the following, an example problem is presented for testing a generic aircraft closed loop model in the longitudinal plane. The state vector xlon ∈ R7 describing the aircraft closed loop system T xlon = [V, α, q, θ, η, η, ˙ e] , (21) contains the velocity V , the angle of attack α, pitch rate q, pitch angle θ, as well as the two actuator states η and η˙ for the elevator deflection and rate, respectively. Furthermore, the variable e represents an integrator state of the controller. Note that the closed loop system is modeled using seven states which is within the capability of the numerical solver developed by the authors. The plant model of the aircraft for the longitudinal motion is of the form (XT )K¯ V˙ = (22) m (ZT )K¯ +q (23) α˙ = mV θ˙ = q (24) (MT )B (25) q˙ = Iy with the total (subscript T ) forces (XT )K¯ , and (ZT )K¯ in the ¯ as well as x and z direction of the rotated kinematic frame K the total moments (MT )B around the y axis of the body fixed frame B. These total forces and moments originate from the gravitational, aerodynamic, and propulsion forces and moments acting on the airframe. Furthermore, the mass of the vehicle and the moment of inertia for the y-axis in the body-fixed frame are denoted by m and Iy . The aircraft is assumed to be initially in a steady state flight condition (horizontal, straight and level flight) at a reference altitude and speed with an initial angle of attack α0 = 3.68 deg. The thrust is held constant at the reference value for the entire time interval of the game. The dynamic equations including the pilot command input u, representing the first (maximizing)
1286
2019 IFAC NOLCOS Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
player and the inputs for the control parametrization v, modeling the second (minimizing) player, are described by a set of ordinary differential equations of the form x˙ lon = f (xlon , u, v) . (26) The actuator model for the elevator control surface is modeled as a linear second order system ω2 G(s) = 2 , (27) s + 2ζωs + ω 2 with natural frequency ω and damping coefficient ζ. Furthermore, the actuator states for the elevator deflection η and rate η˙ are subject to lower and upper bounds which need to be maintained by the player v representing the controller parametrization. It is interesting to note that the system (26) satisfies the saddle point condition (8), which has been verified numerically, although Eq. (26) is not of the form (9) because it contains products between the states x and the inputs u and v. The cost function σ0 , see Fig. 1, is defined as a linear function of the angle of attack α(tf ). Namely, (28) σ0 (α(tf )) = 15/π α(tf ) − 1, at the final time of the game tf = 5s. The function σ1 , responsible for the state constraints, is chosen as σ1 (xlon ) = ρK (xlon ) − 1, (29) where ρK (xlon ) is the Minkowski function of the state constraint set K. The single input of the first (maximizing) player represents the pilot command for the normal load factor nz,c which is subject to box bounds of the form nz,c,lb ≤ nz,c ≤ nz,c,ub . (30) The opposing (minimizing) player v has four parameters of the controller k1...4 at his disposal which are assumed to be bounded from above and below k1...4,lb ≤ k1...4 ≤ k1...4,ub . (31) 4.2 Numerical Results In this section numerical results for the example problem are presented. The numerical solution for the differential game is obtained using the grid scheme presented in Sec. 3. The value function in the subspace of the state variables α and η˙ is depicted in Fig. 3, and the optimal value of the game is α∗ (tf ) ≈ 16.5 deg according to Eq. (28).
There are several important observations regarding the numerical solution of the differential game problem. Firstly, it can be seen in Fig. 3 that the value function exhibits slightly sloping parts close to the actuator rate limits. This is contributed to the fact that the actuator rate dynamics are fast compared to the dynamics governing the other state variables, and, therefore, it is difficult to keep η˙ inside its constraints. It was found that this fact poses a major challenge for the numerical solution using the grid scheme. Moreover, the value of the time discretization step δ is dominated primarily by the dynamics of the actuator rate and needs to be chosen very small in order to capture the dynamic behaviour of the actuator and ensure a stable numerical solution. The second challenge in solving the differential game is related to the state discretization. For the seven dimensional nonlinear model considered in this example a relatively coarse discretization 20 × 10 × 10 × 10 × 10 × 16 × 16 was chosen in order
733
V alue
η˙ [rad/s]
α [rad] Fig. 3. Value function of the game in the subspace of the state variables α and η. ˙ to enable a solution of the differential game with the current implementation of the solver. The authors would like to emphasize that solving the differential game for this nonlinear system and the given state and control dimensions was found to be computationally very demanding and that a finer discretization regarding the time step δ and the states may further improve the results. 4.3 Discussion of the Modeling Philosophy and Interpretation of the Numerical Results From a control design perspective it is reasonable to assume that the control law with its parametrization v is responsible for not exceeding the bounds of the actuator rate and position limits. This implies that the control law structure needs to be designed and parametrized in such a way that the actuator bounds can be maintained. It is important to mention that the inclusion of actuator limits in the testing procedures of flight control laws is essential for the investigating of worst case disturbances. This is contributed to the fact that these effects can result in potentially hazardous flight conditions due the deteriorated controllability of the system. Note further that in the differential game formulation presented in this paper the limits regarding the actuator bounds are not imposed on the player corresponding to the pilot command input u. This implies that the minimizing player v which has the parametrization of the control law at his disposal tries to maintain the actuator state bounds and at the same time the maximizing player u, acting as the disturbance in our formulation, may try to violate these constraints, i.e. drive the actuators into saturation. Essentially, the result from the differential game can be interpreted as follows. From the value function of the game two optimal state feedbacks, the optimal control law parametrization v∗ (x) on the one side and the corresponding worst case disturbance input u∗ (x) on the other side, can be constructed. These two optimal feedback strategies lead to the optimal value of the criterion under investigation, i.e. the maximum angle of
1287
2019 IFAC NOLCOS 734 Vienna, Austria, Sept. 4-6, 2019
Johannes Diepolder et al. / IFAC PapersOnLine 52-16 (2019) 730–734
attack α∗ (tf ) for the example problem presented in this section. Now, consider the case where a suboptimal parametrization ˜ (x) is chosen based on some control deof the controller v sign methodology (e.g. by using gain scheduling over state dependent quantities such as static and dynamic pressure). If u∗ (x) is chosen as the worst case disturbance input and the ˜ (x) = v∗ (x) is used for the control law suboptimal feedback v parametrization it is not possible to obtain a value α ˜ (tf ) lower than the optimal cost value α∗ (tf ) which implies α ˜ (tf ) ≥ α∗ (tf ) (32) Thus, the value of the game represents a lower bound for the criterion which can be achieved with a given control structure and the bounded control law parametrization v in combination with the bounded disturbance inputs u.
Menon, P.P., Fernandez, V., and Bennani, a.S. (2006a). Worstcase analysis of flight control laws for re-entry vehicles. Menon, P.P., Kim, J., Bates, D.G., and Postlethwaite, I. (2006b). Clearance of nonlinear flight control laws using hybrid evolutionary optimization. IEEE Transactions on Evolutionary Computation, 10(6), 689–699. Varga, A., Hansson, A., and Puyou, G. (2012). Optimization Based Clearance of Flight Control Laws: A Civil Aircraft Application, volume 416 of Lecture Notes in Control and Information Sciences. Springer Berlin Heidelberg, Berlin, Heidelberg. doi:10.1007/978-3-642-22627-4.
5. CONCLUSIONS An approach was presented for the computation of optimal disturbances for testing flight control laws. Major challenges for the numerical solution of the differential game are currently the rapidly increasing computational cost regarding the state dimensions and the fine temporal discretization necessary for capturing fast dynamics such as for the actuator rates. The solution for an example problem formulation for a flight control system in the longitudinal plane in seven dimensions including state constraints on the actuator deflection and rate illustrates the main characteristics of the scheme. It is important to mention that the solver implementation for the differential game solution is an ongoing task and the consideration of higher state dimensions and finer state and time discretization steps is likely to improve the numerical solution. Furthermore, the authors are currently investigating the application of direct optimal control methods (cf. Betts (2010); Gerdts (2012)) to verify the optimality of the disturbances obtained from the differential game solution. REFERENCES Betts, J.T. (2010). Practical Methods for Optimal Control and Estimation Using Nonlinear Programming. Society for Industrial and Applied Mathematics. Botkin, N., Martynov, K., Turova, V., and Diepolder, J. (2018). Generation of dangerous disturbances for flight systems. Dynamic Games and Applications. doi:10.1007/s13235-0180259-5. Botkin, N., Hoffmann, K.H., Mayer, N., and Turova, V. (2011). Approximation schemes for solving disturbed control problems with non-terminal time and state constraints. Analysis, 31(4), 355–379. Crandall, M.G. and Lions, P.L. (1983). Viscosity solutions of hamilton-jacobi equations. T. Am. Math. Soc., 277, 1–47. De Oliveira, R.F. and Puyou, G. (2011). On the use of optimization for flight control laws clearance: a practical approach. IFAC Proceedings Volumes, 44(1), 9881–9886. Gerdts, M. (2012). Optimal Control of ODEs and DAEs. De Gruyter textbook. Walter de Gruyter GmbH Co.KG, s.l., 1. aufl. edition. doi:10.1515/9783110249996. Herrmann, A.A. and Ben-Asher, J.Z. (2016). Flight control law clearance using optimal control theory. Journal of Aircraft, 53(2), 515–529. Isaacs, R. (1965). Differential Games. John Wiley and Sons, New York. Krasovskii, N.N. (1988). Game-theoretical control problems. Springer, New York. 1288