Improved Implementation of Dynamic Programming on the Example of Hybrid Electric Vehicle Control

9th IFAC International Symposium on Advances in Automotive 9th IFAC International Symposium on Advances in Automotive Control 9th IFAC International Symposium onAvailable Advances in Automotive Control at www.sciencedirect.com Orléans, June Symposium 23-27, 2019 on Advancesonline 9th IFAC France, International in Automotive Control Orléans, France, June 23-27, 2019 Control Orléans, France, June 23-27, 2019 Orléans, France, June 23-27, 2019

ScienceDirect

IFAC PapersOnLine 52-5 (2019) 147–152

Improved Implementation of Dynamic Improved Implementation of Dynamic Improved Implementation of Dynamic Programming on the Example of Hybrid Improved Implementation of Dynamic Programming on the Example of Hybrid Programming on the Example of Hybrid Electric Vehicle Control Programming onVehicle the Example of Hybrid Electric Control Electric Vehicle Control Electric∗∗∗ Niels Vehicle Control ∗ ∗∗ Wilco van Harselaar Schreuders ∗ ∗ Theo Hofman ∗∗ ∗∗ Wilco Wilco Wilco

van van van

Harselaar ∗ Niels Schreuders∗∗∗ ∗ Theo Hofman ∗∗ Stephan Rinderknecht Harselaar Niels Schreuders∗∗∗ Theo Hofman ∗∗ ∗∗∗ Stephan Rinderknecht ∗ ∗ Harselaar Niels Schreuders∗∗∗ Theo Hofman Stephan Rinderknecht ∗ Stephan Rinderknecht ∗∗∗ Daimler AG, Stuttgart, Germany ∗ ∗ Stuttgart, Germany ∗∗ ∗ Daimler AG, Engineering Department, Eindhoven Daimler AG, Stuttgart, Germany ∗∗ ∗∗ Mechanical Mechanical Engineering Department, Eindhoven University University of of ∗ ∗∗ DaimlerEindhoven, AG, Stuttgart, Germany Technology, The Netherlands Mechanical Engineering Department, Eindhoven University of Technology, Eindhoven, The Netherlands ∗∗∗∗∗ Mechanical Engineering Department, Eindhoven of for Systems, Universität Technology, Eindhoven, The Technische NetherlandsUniversity ∗∗∗ ∗∗∗ Institute for Mechatronic Mechatronic Systems, Technische Universität ∗∗∗ Institute Technology, Eindhoven, The Netherlands Darmstadt, Darmstadt, Germany Institute for Mechatronic Systems, Technische Universität ∗∗∗ Darmstadt, Darmstadt, Institute for Mechatronic Systems,Germany Technische Universität Darmstadt, Darmstadt, Germany Darmstadt, Darmstadt, Germany Abstract: Hybrid Hybrid electric electric drivetrains drivetrains are are an an example example of of systems systems that that can can enable enable discrete discrete Abstract: switching between states with very distinct continuous behavior. For the design of these Abstract: Hybrid electric drivetrains are an example of systems that can enable discrete switching between states with very distinct continuous behavior. For the design of these Abstract: Hybrid electric drivetrains are for an continuous example ofbehavior. systems that can enable of discrete drivetrains,between the required computation time optimizing the control a design driving cycle is switching states with very distinct Forover the these drivetrains, the required time forcontinuous optimizing the to control a design driving ofcycle is switching statescomputation with time, very we distinct behavior. Forover the these critical. Tobetween reduce computation propose two methods improve the drivetrains, the required computation time for optimizing the control over a implementation driving cycle is critical. To programming reduce computation time,the we propose to improve drivetrains, the required computation time for optimizing thethat control overthe a implementation driving cycle is of dynamic dynamic by reducing number oftwo gridmethods points computationally demanding critical. To programming reduce computation time,the we propose two methods to improve the implementation of by reducing number of grid points that computationally demanding critical. To programming reduce computation time, wenumber propose two methods to improve the models implementation sub-models are evaluated for. The proposed methods do not require surrogate and can of dynamic by reducing the of grid points that computationally demanding sub-models are evaluatedby for. The proposed methods do not require surrogate models and can of programming reducing the number of grid computationally demanding bedynamic applied to arbitrary drivetrain A case study on require athat parallel, a series-parallel, and sub-models are evaluated for. Thetopologies. proposed methods dopoints not surrogate models and cana be applied to arbitrary drivetrain topologies. A case study oninrequire a parallel, a series-parallel, sub-models are evaluated for. isThe proposedand methods do not surrogate models canaa power-split hybrid drivetrain performed, a reduction time of upand toand 66% be applied to arbitrary drivetrain topologies. A case study on acomputation parallel, a series-parallel, and power-split hybrid drivetrain is performed, and a reduction in computation time of up to 66% be applied to arbitrary drivetrain topologies.and A case study oninacomputation parallel, a series-parallel, is shown. shown. power-split hybrid drivetrain is performed, a reduction time of up toand 66%a is power-split is shown. hybrid drivetrain is performed, and a reduction in computation time of up to 66% © shown. 2019, IFAC (International Federation optimal of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. is Keywords: Dynamic programming, control, complex systems, computation time, Keywords: Dynamic programming, optimal control, complex systems, computation time, automotiveDynamic control, hybrid vehicles,optimal multi-mode Keywords: programming, control, complex systems, computation time, automotive control, hybrid vehicles,optimal multi-mode Keywords: programming, control, complex systems, computation time, automotiveDynamic control, hybrid vehicles, multi-mode automotive control, hybrid vehicles, multi-mode 1. INTRODUCTION Considering the the system system level level design design (SLD) (SLD) problem problem of of Considering 1. INTRODUCTION HEV drivetrains, it is crucial to reduce the computation 1. INTRODUCTION Considering the system level design (SLD) problem of HEV drivetrains, it is crucial to reduce the computation 1. INTRODUCTION the system level to design (SLD) problem(Silof time of of the control control optimization as much as possible possible The dynamic programming (DP) algorithm is often Considering HEV drivetrains, itoptimization is crucial reduce the computation time the as much as (SilThe dynamic programming (DP) algorithm is often HEV drivetrains, it is crucial to reduce the computation vas et al. (2016)). Methods to reduce computation time applied for the control of systems that include nonthe(2016)). control Methods optimization as much as possible time (SilThe dynamic programming (DP) algorithm is often time vas etof al. to reduce computation applied for the control of systems thatsub-models. include nontime the control optimization as much as possible (SilThe dynamic (DP) algorithm is often without using surrogate models include parallel computconvex, non-linear, and non-continuous non-continuous In vas etofal. (2016)). Methods to reduce computation time applied for theprogramming control of systems thatsub-models. include nonwithout using surrogate models include parallel computconvex, non-linear, and In etproposed al.using (2016)). Methods to (2017) reduce computation time applied for control ofcansystems thatsub-models. include noning, by Goos et al. and applied to practice non-linear, thesethe sub-models be implemented as multiwithout surrogate models include parallel computconvex, and non-continuous In vas ing, proposed by Goos models et al. (2017) and applied to aa practice these sub-models can be implemented as multiusing surrogate include parallel computconvex, non-linear, and non-continuous sub-models. In parallel hybrid with one EM, and iterative refinement ofa dimensional in look-up tables, leading to without ing, proposed by Goos et al. (2017) and applied to practice theseinterpolations sub-models can be implemented as multiparallel hybrid by with one et EM, and iterative refinement of dimensional interpolations in look-up tables, leading to ing, proposed Goos al. (2017) and applied to a practice these sub-models can be implemented as multithe battery SOC grid, proposed by Haag et al. (2016) high computational effort. Furthermore, such complex sysparallel hybrid with one EM, and iterative refinement of dimensional interpolations in look-up tables, leading to the battery SOC grid, proposed by Haag et al. (2016) high computational effort. Furthermore, suchvery complex sysparallel hybrid with oneproposed EM, and iterative refinement of dimensional interpolations in look-up tables, leading to the and applied to an undefined hybrid topology with one tems can have multiple discrete states with different battery SOC grid, by Haag et al. (2016) high computational effort. Furthermore, such complex sysand battery applied SOC to an undefined hybrid topology with one tems computational can have multiple discrete states with very different proposed bytwo Haag et al. high effort. Furthermore, such complex sys- the EM. applied The implementation implementation of these these methods is(2016) indecontinuous behavior, requiring different numbers of conand to angrid, undefined hybrid topology with one tems can have multiple discrete states with very different EM. The of two methods is indecontinuous behavior, requiring different numbers of conapplied to drivetrain an undefined hybrid topology one tems cancontrol have multiple discrete states of with very different pendent ofimplementation the topology. Additionally, various tinuous variables. An example such systems for and EM. Theof of these two methodswith isvarious indecontinuous behavior, requiring different numbers of conpendent the drivetrain topology. Additionally, tinuous control variables. An example ofare such systems for EM. The implementation of these two methods is indecontinuous behavior, requiring different numbers of conmethods that are dependent on the drivetrain topology which DP is often used in literature drivetrains of pendent of the drivetrain topology. Additionally, various tinuous control variables. An example of such systems for methods that are dependent on the drivetrain topology which DP is often used(HEVs) in literature are drivetrains of are proposed of thein drivetrain topology. Additionally, various tinuous control variables. An example ofare such systems for literature. Inonthe the work of Goos Goos et al. al. hybrid DP electric vehicles (Sciarretta and Guzzella methods that are dependent thework drivetrain topology which is often used(HEVs) in literature drivetrains of pendent are proposed in literature. In of et hybrid electric vehicles (Sciarretta and Guzzella thatmore areliterature. dependent onreduce thework drivetrain topology which DP is often used in literature are drivetrains of (2017) three methods to computation time (2007); electric Hofmanvehicles et al. (HEVs) (2012); (Sciarretta Lempert etand al. Guzzella (2018)). methods are proposed in In the of Goos et al. hybrid (2017) three more methods In to reduce computation time (2007); Hofmanvehicles et al. (HEVs) (2012); Lempert et al. Guzzella (2018)). proposed in applied literature. the work ofwith Goos al. hybrid electric (Sciarretta proposed and to aa to parallel hybrid oneettime EM: Using clutches within the transmisson, HEV are (2017) three and more methods reduce computation (2007); Hofmanand et brakes al. (2012); Lempert etand al. (2018)). are proposed applied to parallel hybrid with one EM: Using clutches and brakes within the transmisson, HEV (2017) three more methods to reduce computation (2007); Hofman et al. (2012); Lempert et al. (2018)). efficient stacking of vector dimensions, extracting the time drivetrains with up to two electric machines (EMs) can are proposed and applied to a parallel hybrid with one EM: Using clutches transmisson, stacking of vectortodimensions, extracting the time drivetrains withand up brakes to two within electricthe machines (EMs)HEV can efficient proposed and applied a selecting parallel hybrid with indepenone time EM: Using the transmisson, independent equations, and preferred enable clutches transmission modes with different kinematic and efficient stacking of vector dimensions, extracting the drivetrains withand up brakes to two within electric machines (EMs)HEV can are independent equations, and selecting preferred indepenenable transmission modes with different kinematic and stacking of vector dimensions, extracting the time drivetrains with up to twofrom electric machines (EMs) can efficient dent variables. Much research has also been performed tokinetic transmission relations, requiring none up tokinematic three continindependent equations, and selecting preferred indepenenable modes with different and dent variables.equations, Much research has also been performed tokinetic relations, requiring from none up to three continanddemanding selecting preferred indepenenable transmission modes with different and independent wards less computationally control algorithms, uous control variables (Vanfrom Harselaar et al. (2018)). The dent variables. Much research has also been performed tokinetic relations, requiring none up tokinematic three continwardsvariables. less computationally demanding control algorithms, uous control variables (Van Harselaar et al. (2018)). The Muchmodels. research has alsooptimization been performed tokinetic relations, requiring from up toathree requiring surrogate Convex of the the battery state variables of charge charge (SOC) isnone typically statecontinof The the dent wards less surrogate computationally control algorithms, uous control (Van Harselaar et al. (2018)). requiring models.demanding Convex optimization of battery state of (SOC) is typically a state of the wards less computationally demanding control algorithms, uous control variables (Van Harselaar et al.astates (2018)). The torque division, nested within a DP optimization HEV control problem, whereas additional can be requiring surrogate models. Convex optimization of the battery state of charge (SOC) is typically state of the nested within a DP optimization optimization of of the HEV control problem, whereas additional states can be torque division, surrogate models. Convex battery statecombustion of charge whereas (SOC) (ICE) isadditional typically statebattery of the mode selection selection for parallel HEVs with one EM EM is is proposed proposed the internal engine on/off astates state, torque division, nested within a with DP optimization of the the HEV control problem, can be requiring mode for parallel HEVs one the internal combustion engine (ICE) on/off state, battery division, nested within a with DP optimization of the HEV control problem, additional states canand be torque by Ngo et al. (2012), Nüesch et al. (2014), and Larsson behavior related states, whereas and thermal states mode selection for parallel HEVs one EM is proposed the internal combustion engine (ICE) on/off (Guzzella state, battery by Ngo et al. (2012), Nüesch et al. (2014), and Larsson and mode behavior related states, and thermal states (Guzzella selection fora parallel HEVs one EMand is transmisproposed the internal combustion (ICE) on/off (Guzzella state, battery et al. al. (2015). For series HEV with one discrete Sciarretta (2005)). For aaengine discrete transmission mode, the by Ngo et al.For (2012), Nüesch et with al.one (2014), Larsson behavior related states, and thermal states and et (2015). a series HEV with discrete transmisSciarretta (2005)). For discrete transmission mode, the by Ngo et al. (2012), Nüesch et al. (2014), and Larsson behavior thermal (Guzzella sion mode very efficient control algorithms with the the use of of rotationalrelated speedsstates, and of transmission thestates power sources are et al.mode (2015). Forefficient a seriescontrol HEV with one discrete transmisSciarretta (2005)). For torques aand discrete mode, and the very algorithms with use rotational speeds and torques of the power sources are sion al.mode (2015). For a series HEV with onePourabdollah discrete transmisSciarretta (2005)). For aspeed discrete mode, the convex models have been proposed by et al. linear functions the torques and torque at the wheels, sion very efficient control algorithms with the use of rotational speedsofand of transmission the power sources are et convex models have beencontrol proposed by Pourabdollah et al. linear functions ofand the speed and torque at the wheels, mode very efficient algorithms the use of rotational speedsof of the power sources are sion (2015) and Delprat and Hofman (2014). Forwith a single single mode and thefunctions continuous control variables. The HEV drivetrain convex models have beenHofman proposed by Pourabdollah et al. linear the torques speed and torque at the wheels, (2015) and Delprat and (2014). For a mode and the continuous control variables. The HEV drivetrain convex models have beenHofman proposed by Pourabdollah ettwo al. linear functions of control the speed and torque at the wheels, electric variable transmission (EVT) topology with model does however also include non-linear models depen(2015) and Delprat and (2014). For a single mode and the continuous variables. The HEV drivetrain electricand variable transmission (EVT) topology withmode two model does howevercontrol also include non-linear models depen- (2015) Delprat and Hofman (2014).(PMP) For a single and continuous variables. Themass HEV drivetrain EMs, Pontryagin’s Pontryagin’s minimum principle is with proposed dentthe ondoes torques andalso speeds, as the fuel flow of the electric variable transmission (EVT) topology two model however include non-linear models depenEMs, minimum principle (PMP) is proposed dent on torques and speeds, as the fuel mass flow of the variable transmission (EVT) topology model include non-linear models by Kim Kim et al. al. (2011). (2011). The usage usage of convex convex models is two not ICE and thehowever electric power ofasthe EMs. EMs, Pontryagin’s minimum principle (PMP) is with proposed dent ondoes torques andalso speeds, the fuel Furthermore, mass flow depenof the electric by et The of models is not ICE and the electric power ofasthe EMs. Furthermore, the EMs, Pontryagin’s minimum principle (PMP) is proposed dent on torques and speeds, the fuel mass flow of the always desirable, depending on the goal and required level system model can include transmission losses, which are by Kim et al. (2011). The usage of convex models is not ICE and the electric power of the EMs. Furthermore, the always desirable, depending on the goal and required level system model candependent include transmission losses, which are by Kimdesirable, et The usage ofgoal convex islevel not ICE andmodel the electric power transmission of EMs.and Furthermore, of detail detail of al. the(2011). simulations. Forthe HEV drivetrains, convex also non-linearly non-linearly onthe torques speeds. depending on and models required system candependent include losses, which the are always of of the simulations. For HEV drivetrains, convex also on torques and speeds. depending on goaldrivetrains, and required level system model can include transmission losses, which are always of detaildesirable, of the simulations. Forthe HEV convex also non-linearly dependent on torques and speeds. of detail of the simulations. For HEV drivetrains, convex also non-linearly dependent on torques andof speeds. 2405-8963 © 2019 2019, IFAC IFAC (International Federation Automatic Control) Copyright © 147 Hosting by Elsevier Ltd. All rights reserved. Copyright © under 2019 IFAC 147 Control. Peer review responsibility of International Federation of Automatic Copyright © 2019 IFAC 147 10.1016/j.ifacol.2019.09.024 Copyright © 2019 IFAC 147

2019 IFAC AAC 148 Orléans, France, June 23-27, 2019

Wilco van Harselaar et al. / IFAC PapersOnLine 52-5 (2019) 147–152

models for EM losses can lead to significantly different losses in some operating points, as visualized by Egardt et al. (2014). Furthermore, transmission losses as a function of rotational speeds and torques can not be modeled convex (Shen et al. (2017)).

for time steps

2. PROPOSED METHOD The starting point of this research is the original DP implementation using the Matlab function of Sundström and Guzzella (2009). A schematic overview of the DP implementation is displayed in Fig. 1. DP is a discrete algorithm, based on discrete time steps (stages) and discrete or discretized states and controls. To find the optimal control signal map over all states and over the complete time horizon, the DP algorithm evaluates the cost and state change of all control policies for all states backwards over time. Subsequently, the found signal map is used in a forward evaluation of the system model to find the optimal control signal and continuous state trajectory over the discrete time horizon. Therefore, for the backward evaluation, a full grid of all state and control variables is created based on the input parameters. The provided system model is evaluated over the full grid for all time steps. Depending on the system, the system model can contain a number of submodels. In Fig. 1 two sub-models are shown as an example, both containing multi-dimensional interpolations. The DP algorithm needs three outputs from the system model: 1) the system feasibility, 2) the cost, and 3) the changes of the states. These system outputs can be direct outputs of sub-models, yet for clarity Feasibility and Cost & state change blocks are displayed in the schematic overview. 2.1 Redundancy Reduction: Redundant Control The first method to reduce the computation time focuses on redundant control points in the DP grid. The DP algorithm creates a full grid with all combinations of all control and state variables. In case of a system with discrete states that require distinctive numbers of continuous controls, a part of the grid contains redundant control combinations. To enable the removal of grid points, all model inputs are first vectorized. Subsequently, the redundant grid points are identified in the system model, all vectorized inputs are reduced by these points, and the sub-models including their multi-dimensional interpolations are only evaluated 148

System model Sub-model 1 Sub-model 2 nD-Interp

nD-Interp

Feasibility Cost & state change

Result

Fig. 1. Schematic overview of DP. 1 0.75

x c,2

In the remainder of this paper the proposed methods are explained in Section 2, a case study with three distinct topologies is presented in Section 3, and this paper ends with a conclusion in Section 4.

Create full grid

DP-algorithm

In this paper, two methods are presented to reduce the computation time by improving the implementation of DP without the need for surrogate models, and while being independent of the drivetrain topology. The first method reduces the grid for which the system model is evaluated by redundant grid points, and the second method reduces the amount of multi-dimensional interpolations by redundant grid points and grid points where the system is infeasible. Both methods lead to results identical to the results of the original DP implementation and are explained in a general way, as they can also be applied to other systems and optimization problems where DP is used.

Initialize parameters

Vectorize 0.5

Reduce

0.25 0 1

2

x c,1

3

Grid point Redundant grid point

Fig. 2. Visualization of grid vectorization and reduction. for the reduced grid. The vectorization and reduction of the inputs are visualized in Fig. 2 for an exemplary system with three states selected by the first control variable xc,1 , of which the first state requires a second control variable xc,2 . The values of an input are represented by dots and shown over only two grid dimensions, whereas redundant grid points are represented by unfilled dots. As the DP algorithm needs the model outputs over the complete grid in the same form as the inputs, the outputs are restored to the full grid size, redundant grid points are indicated as infeasible, and the model outputs are reshaped to grid matrices. The implementation of the redundancy reduction is schematically shown in Fig. 3a. 2.2 Full Reduction: Redundant Control, System Feasibility Besides the cost and state changes, the model also outputs the feasibility over the grid. For time-variant problems the feasibility of all grid points must be evaluated at every time iteration. In the case that feasibility can be evaluated without multi-dimensional interpolations, but cost and state change related quantities are determined by multidimensional interpolations, the computational effort is to be reduced further as follows. In addition to the previous presented method, the grid is reduced for both redundant control and system feasibility before multidimensional interpolations are performed. As the model often contains sub-models that both determine feasibility and include multi-dimensional interpolations related to cost or state changes, this method might require changing the structure of the model. This is displayed in Fig. 3b,

2019 IFAC AAC Orléans, France, June 23-27, 2019

System model

Wilco van Harselaar et al. / IFAC PapersOnLine 52-5 (2019) 147–152

Table 1. Overview of the transmission mode types with the required number of continuous control variables Nxc,c and the numbers of enabled modes by the three example topologies.

System model

Input vectorization Redundancy reduction

Input vectorization Sub-model 1

Ψ: Mode Type

Sub-model 1

Nxc,c

Sub-model 2

nD-Interp

Sub-model 2 nD-Interp

Feasibility Full reduction

Feasibility

nD-Interpolations

Cost & state change

Cost & state change

Vector restoration

Vector restoration

Reshape to grid matrices

Reshape to grid matrices

(a)

(b)

149

Fig. 3. Schematic overview of the proposed methods. (a) Reduction for redundant control. (b) Reduction for redundant control and system feasibility. where the multi-dimensional interpolations are taken out of the sub-models. In this way, first the system feasibility is determined after which the vectorized grid is reduced for both control redundancy and system feasibility. 2.3 Stepped reduction It can be noticed that in the method presented in Fig. 3b the sub-models, without the multi-dimensional interpolations, are still evaluated for redundant control combinations. To avoid this, a stepped reduction method can be implemented. The first step would be to reduce only by redundant control, after which sub-models are evaluated to determine the system feasibility. In the second step the grid is then further reduced by the infeasible grid points, after which the multi-dimensional interpolations are evaluated. Subsequently, the grid is restored in two steps to output the cost, state change, and feasibility over the full grid. As the systems considered in this research only include linear feasibility conditions, stepped reduction did not show a reduction of computation time and is not elaborated on in this paper. It must however be noted that for systems with more complex feasibility conditions, especially when containing multi-dimensional interpolations, a stepped reduction method could provide a potential to further reduce computational effort. 3. CASE STUDY: HEV CONTROL In the previous section methods have been presented to reduce the computational effort for solving complex timedepend problems with DP. In this section the presented methods are applied to the control optimization of three different hybrid drivetrain topologies over a driving cycle. The three topologies differ by the number of transmission modes that they enable, as well as the mode types of the enabled transmission modes. For drivetrain topologies with a maximum of 1 ICE and 2 EMs a total of 14 149

ICE only 1: FG 0 EM only 2: FG 1 EM 0 3: FG 2 EMs 1 4: EVT 2 EMs 1 Hybrid 5: FG parallel 1 EM 1 6: FG parallel 2 EMs 2 7: Series 2 8: EVT 1 EM 1 9: EVT 2 EMs 2 CDVs 2 10: EVT 2 EMs 3 CDVs 3 Charge (No propulsion possible) 11: FG 1 EM 1 12: FG 2 EMs 1 13: EVT 2 EMs 1 Neutral 14: Neutral 0

Number of enabled modes P2 DE-REX THS -

-

-

5 -

2 4 -

-

5 -

2 4 2 -

1 -

1 -

-

-

1

1

-

different mode types, denoted by symbol Ψ, are possible. Mode types are defined by the dynamic behavior of the drivetrain and subsequently define the required number of continuous control variables Nxc,c (Van Harselaar et al. (2018)). All mode types including Nxc,c for each mode type are listed in Table 1, using abbreviations fixed gear (FG) and control decision variable (CDV). As this work is only focused on computation time, the same exemplary power sources are used in all simulations to ensure comparability of computation times. Because the power sources are not optimized for the respective topologies, the resulting efficiencies cannot be compared and will not be presented. 3.1 Investigated drivetrain topologies The first example topology is a parallel hybrid topology based on a conventional 5 speed gearbox with an electric machine connected to the input shaft of the gearbox, from hereon referred to as the P2 topology. The P2 topology is schematically displayed in Fig. 4a. This type of hybrid drivetrains is mass-produced and can be found on the road today, see e.g. Keller et al. (2015). As for this case study a 5 speed gearbox is used, 5 pure electric transmission modes and 5 parallel hybrid transmission modes are enabled. Furthermore, one charging mode and one neutral mode are enabled, making it a total of 12 modes as listed in Table 1. To divide the required torque over the ICE and EM in the parallel hybrid modes, one continuous control variable is needed for the control optimization of this type of drivetrain topologies. This continuous control variable causes redundant control combinations in the full electric modes, as in these modes all requested torque has to be provided by the EM. The second topology is an innovative series-parallel hybrid drivetrain, designed as a multi-gear electric drivetrain with two EMs and an ICE as range extender (Viehmann et al. (2018)). In German, this concept is called ’Doppel-EAntrieb mit Range-Extender’, and abbreviated as DE-

2019 IFAC AAC 150 Orléans, France, June 23-27, 2019

Wilco van Harselaar et al. / IFAC PapersOnLine 52-5 (2019) 147–152

ICE

EM1

GB

ICE

EM1

5 speed gearbox

W

continuous control variables. In the work of Van Harselaar et al. (2019) the method of implementing mode type Ψ = 7 with one continuous control is explained, programming this method can however be done in different ways which influences the computation time. As this research focuses on computation time, it is chosen to implement mode type Ψ = 7 with two continuous control variables, changing the generic transmission model in the following way: ωice = xc,2 · ω ¯ ice , if Ψ(xc,1 ) = 7 (2)

W

(a)

EM2

EM1

ICE

ICE

EM1

EM2

W

W

(b)

(c)

Fig. 4. Stick diagrams. (a) P2. (b) DE-REX. (c) THS. REX. Fig. 4b shows a stick diagram of the DE-REX topology. Both EMs independently have two gears, with an optimized transmission design. One of the ways in which the transmission design is optimized, is by using form-closed clutches without friction elements such as synchromesh units. The EMs are used to synchronize rotational speeds to enable clutch engagement. According to the here used definitions, this topology enables a total of 15 transmission modes of 6 different mode types, as listed in Table 1. This table also lists the required number of continuous control variables for all mode types, showing that two continuous control variables are required to optimize the control for the DE-REX topology. As in 9 of the 15 enabled transmission modes only one or no continuous control variables are needed, the full DP grid contains a relatively large amount of redundant control combinations. The third example topology is the Toyota Hybrid System (THS) as applied in the model year 2016 Toyota Prius (Fushiki (2016)), of which the stick diagram is displayed in Fig. 4c. It is chosen to use this drivetrain topology as an example because it only enables one discrete transmission mode of type Ψ = 9, as listed in Table 1. As only one transmission mode is enabled there are no redundant control combinations, and this topology allows to evaluate the isolated effect of reduction for system feasibility. 3.2 Original DP implementation For this case study, the generic transmission model is used as described by Van Harselaar et al. (2019). This generic model integrates the dynamic behavior of all conventional-, electric-, and hybrid-transmission mode types. The outputs of this model are the rotational speed of the ICE, ωice , the torque of the ICE, τice , the rotational speed of both EMs, ωem1 and ωem2 , and the torques of both EMs, τem1 and τem2 . The inputs are the torque and rotational speed of the wheels of the vehicle, τw and ωw respectively, and the control variables (1) xc = [xc,1 xc,2 xc,3 xc,4 ] , where xc,1 is a discrete control variable that selects the transmission mode, and the other three are continuous control variables. The continuous control variables are scaled variables between zero and one, and are discretized by the DP algorithm. For this research, one change is made to the generic transmission model: the series hybrid mode (Ψ = 7) is implemented with two instead of one 150

τice = xc,3 · τ¯ice (ωice ), if Ψ(xc,1 ) = 7  if Ψ(xc,1 ) = 7 ∧ iem1 = 0 ωice · ich1 , ωem1 = ωw (Λ) · iem1 , if Ψ(xc,1 ) = 7 ∧ iem1 = 0 τ ice if Ψ(xc,1 ) = 7 ∧ iem1 = 0 i 1, τem1 = τch w (Λ) iem , if Ψ(xc,1 ) = 7 ∧ iem1 = 0

(3)

(4) (5)

1



ωice · ich2 , ωw (Λ) · iem2 , τ ice if i 2, τem2 = τch w (Λ) iem , if

ωem2 =

2

if Ψ(xc,1 ) = 7 ∧ iem2 = 0 if Ψ(xc,1 ) = 7 ∧ iem2 = 0

(6)

Ψ(xc,1 ) = 7 ∧ iem2 = 0

(7)

Ψ(xc,1 ) = 7 ∧ iem2 = 0

with maximum ICE speed ω ¯ ice , maximum ICE torque τ¯ice , drive cycle Λ, charging ratios ich1 = ωem1 /ωice and ich2 = ωem2 /ωice , and EM transmission ratios iem1 = ωem1 /ωw (Λ) and iem2 = ωem2 /ωw (Λ). The generic transmission model forms the base of the ICE and EM sub-models. These sub-models are completed by lookup-tables for fuel mass flow (m ˙ f ) and the electric power demands of the two EMs (Pel1 and Pel2 ), and feasibility functions that check the maximum and minimum torques and speeds of the components. The separate feasibilities are taken together to form the system feasibility output I. In this case study, cost C is set equal to fuel mass mf . The cost function is still displayed as a block in the system model to explicitly show where output C is created. The only state of the problem is the battery SOC, denoted as ξsoc . This output is determined by the battery model, which becomes the electric power demands of the EMs as input. The system model for the original DP implementation is schematically shown in Fig. 5a. For all simulations performed in this case study, the SOC grid is iteratively refined with for each iteration a more dense discretization of the continuous controls (Haag et al. (2016)). The applied discretizations are selected to conform the used power sources and battery models. 3.3 Implementation of Redundancy Reduction For the implementation of the grid reduction for redundant control combinations, the original DP model as described in the previous subsection is used as a starting point. The grid is directly vectorized and reduced, as shown in Fig. 5b. All variables with values over the reduced grid are indicated with an ’r’ in the subscript. The vector restoration is done before the cost and battery models are evaluated. Restoring the vectors here did not show an increase in calculation time compared to restoring right before the outputs are reshaped to grid matrices, and results in code that the authors experienced more practical

2019 IFAC AAC Orléans, France, June 23-27, 2019

Wilco van Harselaar et al. / IFAC PapersOnLine 52-5 (2019) 147–152

System model

Input vectorization

Input vectorization [xc ; Λ]

xc

Redundancy reduction

Pel2 2D-Interp

Ir

Battery

Cost C

ξsoc (t + 1)

I

Pel1

Pel2

Battery

Cost C

m _f 2D-Interp m _ f;r

Pel1 2D-Interp Pel1;r

Pel2 2D-Interp Pel2;r

Vector restoration

Vector restoration m _f

Ir

EM2

[τice;r ; !ice;r ]

Full reduction

Iem2;r

Pel2

Pel2 2D-Interp Iem2

I

Pel2;r

Pel1;r

[τem1;r ; !em1;r ]

Pel1 2D-Interp

EM2

I

EM2

EM1 Iem1;r

[τem1 ; !em1 ]

Pel1

Iem1

Pel1 2D-Interp

[τice;r ; !ice;r ]

m _ f;r

Iice;r

m _f 2D-Interp

EM1

EM1 Iem1

ICE

[τice ; !ice ]

m _f

Iice

m _f 2D-Interp

[τem2;r ; !em2;r ]

ICE

ICE Iice

[xc;r ; Λ]

ξsoc (t)

[τem2 ; !em2 ]

ξsoc;r (t)

Iem2

[xc ; Λ]

[τice ; !ice ]

ξsoc (t)

[τem1 ; !em1 ]

System model [xc ; Λ]

[τem1;r ; !em1;r ]

System model

151

ξsoc (t + 1)

m _f

I

Pel1

Pel2

Battery

Cost C

ξsoc (t + 1)

Reshape to grid matrices

Reshape to grid matrices

(b)

(c)

(a)

Fig. 5. Schematic overview of the DP methods implemented in the case study. (a) Original DP. (b) Redundancy reduction. (c) Full reduction for redundant control and system feasibility. to work with. In the vector restoration, the feasibility of all redundant grid points that were excluded is set to infeasible, and all other vectors are restored using zeros. The reduction itself is based on the number of continuous control variables Nxc,c that is needed for each mode type, as listed in Table 1. The number of points the grid is reduced by, ∆grid , can be determined as a function of the grid discretization. For a problem with 3 control variables, as e.g. the control problems of the DE-REX and THS, the grid reduction is   ∆grid = |Xξ | · N2 · (|X2 | − 1) + N3 · (|X3 | − 1) · |X2 | (8)

with Xξ the domain of ξsoc , N2 and N3 the numbers of transmission modes independent of respectively xc,2 and xc,3 , and X2 and X3 the domains of xc,2 and xc,3 , respectively. 3.4 Implementation of Full Reduction

For the implementation of the full reduction for redundant control and system feasibility, the original DP model is used as a starting point. The two-dimensional interpolations of the lookup tables for m ˙ f , Pel1 , and Pel2 are removed from the ICE and EM sub-models, and the full reduction is performed after the evaluation of these submodels. Subsequently, the two-dimensional interpolations 151

are performed, and the vectors are restored. This implementation is schematically shown in Fig. 5c. As in the redundancy reduction, the feasibility of all restored grid points is set to infeasible. In contrast to the redundancy reduction, the amount of reduction differs per time step as the feasibility is dependent on the vehicle speed and torque demand. 3.5 Results To measure the effect of the presented methods on the computation time 1 , the control is optimized over the worldwide harmonized light vehicles test cycle (WLTC) for the three topologies using the original DP, the redundancy reduction, and the full reduction. Table 2 shows the absolute computation times and numbers of 2D interpolations for the original DP implementation, and the relative values for the redundancy reduction and full reduction implementations. The presence of xc,3 and the number of enabled transmission modes results in a large difference in absolute computation times between the topologies, from 2.7 min for the P2 to 121.2 min for the DE-REX. The computation times with redundancy reduction applied for the P2 and All computations have been performed on an Intel Core i7-6700K CPU at 4.00 GHz with 64 GB RAM. To determine the computation times, the average time of 20 simulations is taken. For all variants the coefficient of variation of the 20 values is below 0.5%.

1

2019 IFAC AAC 152 Orléans, France, June 23-27, 2019

Wilco van Harselaar et al. / IFAC PapersOnLine 52-5 (2019) 147–152

Table 2. Absolute computation times and numbers of 2D interpolations for original DP, and relative values for redundancy reduction (RR) and full reduction (FR). Topology P2 DE-REX THS

Computation time Original RR FR 2.7 min 121.2 min 13.9 min

62% 50% 101%

# 2D interpolations Original RR FR

34% 34% 41%

58 · 106 48 · 109 34 · 107

56% 44% 100%

43% 36% 67%

DE-REX are reduced to 62% and 50%, respectively. For the control optimization for the THS, the redundancy reduction increases the computation time to 101% of the original DP, as extra code is evaluated without the benefit of grid reduction. With the application of full reduction the computation time is significantly reduced for all three topologies, to 34%-41% of the computation time of the original DP. The numbers of 2D interpolations show the same trends as the computation times. The resulting efficiencies of the three topologies are identical for the original DP, redundancy reduction, and full reduction, as only redundant and infeasible grid points are being omitted by the presented methods. 4. CONCLUSION Two methods for the improved implementation of DP are presented. An existing Matlab function for the DP algorithm is used, and the improvements are integrated in the system model. The first method is to reduce the grid over which the system model is evaluated by redundant control points. The second method is to restructure the system model to enable the reduction of the grid for both redundant control and system feasibility before multidimensional interpolations are evaluated. The presented methods do not require surrogate models and lead to results identical to the original DP implementation. In a case study with three distinctive hybrid drivetrain topologies a reduction in computation time of up to 66% is shown. REFERENCES Delprat, S. and Hofman, T. (2014). Hybrid vehicle optimal control: Linear interpolation and singular control. In Proc. IEEE Veh. Power Propulsion Conf., 1–6. Egardt, B., Murgovski, N., Pourabdollah, M., and Mardh, L.J. (2014). Electromobility studies based on convex optimization: Design and control issues regarding vehicle electrification. IEEE Control Systems, 34(2), 32–49. Fushiki, S. (2016). The new generation front wheel drive hybrid system. SAE Int. J. of Alternative Powertrains, 5(1), 109–114. Goos, J., Criens, C., and Witters, M. (2017). Automatic evaluation and optimization of generic hybrid vehicle topologies using dynamic programming. IFACPapersOnLine, 50(1), 10065–10071. Guzzella, L. and Sciarretta, A. (2005). Vehicle Propulsion Systems: Introduction to Modeling and Optimization. Springer. Haag, A., Bargende, M., Antony, P., and Panik, F. (2016). Iterative refinement of the discretization of the dynamic programming state grid. In 16. Int. Stuttgarter Symposium, 145–154. Springer. 152

Hofman, T., Ebbesen, S., and Guzzella, L. (2012). Topology optimization for hybrid electric vehicles with automated transmissions. IEEE Transactions on Vehicular Technology, 61(6), 2442–2451. Keller, U., Schmiedler, S., Strenkert, J., Ruzicka, N., and Nietfeld, F. (2015). Plug-in hybrid from Mercedes-Benz– The next generation plug-in hybrid with 4-cylinder gasoline engine. In 15. Internationales Stuttgarter Symposium, 829–850. Springer. Kim, N., Cha, S., and Peng, H. (2011). Optimal control of hybrid electric vehicles based on Pontryagin’s minimum principle. IEEE Trans. on Control Syst. Technol., 19(5), 1279–1287. Larsson, V., Johannesson, L., and Egardt, B. (2015). Analytic solutions to the dynamic programming subproblem in hybrid vehicle energy management. IEEE Trans. on Veh. Technol., 64(4), 1458–1467. Lempert, J., Vadala, B., Arshad-Aliy, K., Roeleveld, J., and Emadi, A. (2018). Practical considerations for the implementation of dynamic programming for HEV powertrains. In 2018 IEEE Transportation Electrification Conference and Expo (ITEC), 755–760. IEEE. Ngo, V., Hofman, T., Steinbuch, M., and Serrarens, A. (2012). Optimal control of the gearshift command for hybrid electric vehicles. IEEE Trans. on Veh. Technol., 61(8), 3531–3543. Nüesch, T., Elbert, P., Flankl, M., Onder, C., and Guzzella, L. (2014). Convex optimization for the energy management of hybrid electric vehicles considering engine start and gearshift costs. Energies, 7(2), 834–856. Pourabdollah, M., Silvas, E., Murgovski, N., Steinbuch, M., and Egardt, B. (2015). Optimal sizing of a series PHEV: Comparison between convex optimization and particle swarm optimization. IFAC-PapersOnLine, 48(15), 16–22. Sciarretta, A. and Guzzella, L. (2007). Control of hybrid electric vehicles. IEEE Control systems, 27(2), 60–70. Shen, Y., Rinderknecht, S., and Hoppert, M. (2017). General modelling method of power losses in transmission with parameter identification. Forschung im Ingenieurwesen, 81(2-3), 117–123. Silvas, E., Hofman, T., Murgovski, N., Etman, P., and Steinbuch, M. (2016). Review of optimization strategies for system-level design in hybrid electric vehicles. IEEE Trans. on Veh. Technol., 66(1), 57–70. Sundström, O. and Guzzella, L. (2009). A generic dynamic programming Matlab function. In Proc. IEEE Control Appl., 1625–1630. Van Harselaar, W., Brouwer, M., and Hofman, T. (2019). A generic transmission model for hybrid electric drives. Forschung im Ingenieurwesen, 83(1), 1–9. Van Harselaar, W., Hofman, T., and Brouwer, M. (2018). Automated dynamic modeling of arbitrary hybrid and electric drivetrain topologies. IEEE Trans. on Veh. Technol., 67(8), 6921–6934. Viehmann, A., König, R., and Rinderknecht, S. (2018). Investigation of gear shifts in a parallel-series hybrid powertrain with dog clutches in a demonstrator vehicle (two-drive-transmission with range-extender, DEREX). In Int. VDI Congr. Dritev - Drivetrain for Veh., 323–346.