An Efficient Model Based Control Algorithm for the Determination of an Optimal Control Policy for a Constrained Stochastic Linear System

Proceedings, 10th of IFAC International Symposium on Advanced Control Chemical Processes Advanced Control Chemical Processes Proceedings, 10th of IFAC International Symposium on Shenyang, Liaoning, China, July 25-27, 2018 Available Shenyang, Liaoning, July 25-27, 2018 online at www.sciencedirect.com Advanced Control of China, Chemical Processes Proceedings, 10th IFAC International Symposium on Shenyang, Liaoning, China, July 25-27, 2018 Advanced Control of Chemical Processes Shenyang, Liaoning, China, July 25-27, 2018

ScienceDirect

PapersOnLine 51-18 (2018) 590–595 An Efficient Model IFAC Based Control Algorithm for the Determination of an An Efficient Model Based Control Algorithm for the Determination of an PolicyControl for a Constrained Linear System AnOptimal EfficientControl Model Based AlgorithmStochastic for the Determination of an Control Policy for a Constrained Stochastic Linear System Optimal AnOptimal EfficientControl Model Based Control Algorithm for the Determination of an Policy for a Constrained Stochastic Linear System Optimal Control Policy for a Constrained Stochastic Linear System

J. Prakash* David Zamar** Bhushan J. Prakash* DavidEzra Zamar** Bhushan Gopaluni** Kwok** J. Prakash* DavidEzra Zamar** Bhushan Gopaluni** Kwok** Gopaluni** Ezra Kwok** J. Prakash* David Zamar** Bhushan *Department of Instrumentation Engineering, Madras Institute of Technology Gopaluni** Ezra Kwok** *Department of Instrumentation Engineering, Madras Institute of Technology Campus, Anna University, Chennai, India ( e-mail: [email protected]) *Department of Instrumentation Engineering, [email protected]) Institute of Technology University, Chennai, India ( e-mail: Campus, Anna Anna University, Chennai, India ( e-mail: [email protected]) Campus, Instrumentation Engineering, MadrasofInstitute Technology **Department*Department of Chemical of and Biological Engineering, University British of Columbia, Canada (e-mail: **Department of Chemical andUniversity, [email protected], Engineering, University of British Columbia, Canada (e-mail: Anna Chennai, India ( e-mail: [email protected]) Campus, [email protected], [email protected] ) **Department of Chemical and [email protected], Engineering, University of British Columbia, Canada (e-mail: [email protected] ) [email protected], [email protected] ) [email protected], [email protected], **Department of Chemical and Biological Engineering, University of British Columbia, Canada (e-mail: Abstract: In [email protected], this paper, the authors have proposed an ensemble Kalman filter based [email protected] ) stochastic [email protected], Abstract: In this control paper, the authorstohave proposed an ensemble based stochastic algorithm determine an optimal controlKalman policy atfilter every sampling time model predictive Abstract: In this control paper, the authorstohave proposed an ensemble Kalman filter based stochastic predictive algorithm determine an optimal control policy at every sampling model the instant for a constrained stochastic linear system. To determine an optimal control policy fortime algorithm determine anTo optimal control policy atfilter every sampling model predictive constrained stochastic linear system. determine anKalman optimal control policy fortime the instant forIna linear Abstract: this control paper, the authorsto have proposed an ensemble based stochastic random disturbances and measurements corrupted by constrained system affected by constrained stochastic linear system.an To determine an optimal control policy fortime the instant for a linear random disturbances and measurements corrupted by constrained system affected by predictive control algorithm to determine optimal control policy at every sampling model random noise, the authors have minimized the uncertain objective function, subject to uncertain disturbances and measurements corrupted by constrained linear system affected by random uncertain objective function, subject to uncertain random noise, the authors have minimized the constrained stochastic linear system. To determine an optimal control policy for the instant for a state & output constraints and deterministic input constraints using the quantile based scenario uncertain objective function, subject to uncertain random the authors have minimized theinput constraints using the quantile based scenario state & noise, output constraints and ensemble deterministic disturbances measurements corrupted by constrained linear system affected by random employed, to generate a recursive analysis approach. In this work, Kalman filter is beingand constraints using the quantile based scenario state & noise, output the constraints and ensemble deterministic input being employed, to generate a recursive analysis approach. In this work, Kalman filter is uncertain objective function, subject to uncertain random authors have minimized the estimate of states of the constrained stochastic linear system. The number of scenarios is considered employed, to generate a recursive analysis approach. In this work, ensemble Kalman filter is beingnumber of quantile scenarios is filter. considered estimate of statesconstraints of stochastic linear system. constraints using the based scenario state output deterministic input Kalman Each to be & equivalent to the thatconstrained of and number of sample points usedThe in the ensemble number of scenarios is filter. considered estimate of states of the constrained stochastic linear system. The Kalman Each to be equivalent to that of number of sample points used in the ensemble being employed, to generate a recursive analysis approach. In this work, ensemble Kalman filter is scenario is viewed as one realization of the process noise, measurement noise over the prediction points usedmeasurement in the ensemble Kalman filter. Each to be equivalent toas that numberstochastic of sample scenario is viewed one realization of the process noise, noise over the prediction thof number of scenarios is considered estimate of states of the constrained linear system. The horizon as well as the i sample point of the state estimate at the beginning of the prediction th realization of the process noise, measurement noise over the scenario is viewed as one sample point of the state estimate at the beginning of the prediction horizon as well as the i Kalman filter. Each to be equivalent to that of number of sample points used in the ensemble horizon generated by the th ensemble Kalman filter. Simulation studies have been carried out to assess sample Kalman point of filter. the state estimate at thehave beginning of the prediction as well asby the iensemble horizon generated the Simulation studies been carried out to assess of the process noise, measurement noise over the prediction scenario is viewed as one realization the efficacy of the proposed control scheme on the simulated model of the constrained singlehorizon generated theith ensemble Kalman Simulation studies tosingleassess efficacy of the control scheme the estimate simulated ofbeen the carried constrained the sample point of filter. theonstate atmodel thehave beginning of theoutprediction horizon well asbyproposed the linear stochastic system. input andassingle-output efficacy of thebyproposed control scheme on Simulation the simulated model the carried constrained the single-output stochastic system. input and horizon generated thelinear ensemble Kalman filter. studies haveofbeen out tosingleassess Keywords: Ensemble Kalman filter, of Stochastic Model Predictive Scenario Optimization © 2018, IFAC (International Federation Automatic Control) Hosting model byControl, Elsevier Ltd. All rights reserved. single-output linear stochastic system. input and efficacy of the proposed control scheme on the simulated of the constrained singlethe Keywords: Kalman filter, Stochastic Model Predictive Control, Scenario Optimization and quantileEnsemble scenario analysis. linear stochastic system. Model Predictive Control, Scenario Optimization inputquantile and single-output Keywords: Ensemble Kalman filter, Stochastic and scenario analysis. and quantileEnsemble scenario analysis. Keywords: Kalman filter, Stochastic Model Predictive Control, Scenario Optimization and quantile scenario analysis. 1. INTRODUCTION Depending upon the type of state space model used for INTRODUCTION Depending the type state space model usedinto for Model Predictive 1. Control (MPC) is the most preferred prediction, upon the MPC canofbe broadly classified INTRODUCTION 1. Depending upon the type of state space model used for Model Predictive Control (MPC) is the most preferred prediction, the MPC can be broadly classified into multivariable control scheme at the supervisory level deterministic MPC and stochastic MPC. The second Model Predictive 1. Control (MPC) is the most preferred prediction, upon theMPC MPC canstochastic broadly classified into multivariable scheme at the supervisory deterministic MPC. second INTRODUCTION Depending theisand type ofbe state model used for industries because of its abilitylevel to in the processcontrol type classification based on thespace type of The uncertainty multivariable control scheme at the supervisory level deterministic MPC and stochastic MPC. The second process industries because of its ability to in the typethe classification ispropagation based on broadly the type classified of uncertainty Model Predictive Control (MPC) is theinteractions most preferred prediction, the MPC can be into the multivariable and handle systematically Carlo and uncertainty methods (Monte processcontrol industries because its ability to in the systematically typethe classification based on the type uncertainty multivariable interactions and handle Carlo and uncertainty methods (Monte multivariable scheme at outputs, the of supervisory level deterministic MPCispropagation and stochastic MPC.of The second such asthe states, and inputs constraints simulation, Moment method and Polynomial Chaos) systematically the multivariable interactions and handle Carlo and the uncertainty propagation methods (Monte such as states, outputs, and inputs constraints method Chaos) simulation, because its ability to in the process type classification is based onand the Polynomial type uncertainty time of instant, the MPC respectively. At industries each sampling being usedMoment in the stochastic MPCof formulation such as states, outputs, and inputs constraints simulation, Moment method and Polynomial Chaos) sampling time instant, the MPC respectively. At each being in based the stochastic MPC (Monte formulation multivariable interactions and handle systematically and the used uncertainty propagation methods Carlo future controller outputs, computes the currenttheand tube SMPC; Scenario/Sample based (Stochastic time instant, the inputs MPC respectively. Atcurrent each sampling being usedtube in based the method stochastic MPC formulation future controller computes thesuch and SMPC; Scenario/Sample based (Stochastic states, outputs, constraints simulation, Moment and Polynomial Chaos) deviation of and the outputs, process by minimizing the as predicted SMPC and Generalized Polynomial Chaos based controller outputs, computes the current and future tube based Scenario/Sample based (Stochastic of thethe process by minimizing predicted Generalized Polynomial Chaos SMPC and sampling time instant, MPC respectively. each being in the SMPC; stochastic MPCclassified formulation horizon output from At thethe setpoint overdeviation the prediction schemes are also SMPC).used Further SMPC deviation of the process by minimizing the predicted Generalized Polynomial Chaos based SMPC and prediction horizon output from the setpoint over the schemes are classified(either also SMPC). SMPCSMPC; controller computes current and tubecontrol based Scenario/Sample based (Stochastic well as the minimizing the future expenditure of theoutputs, control as parameterization based onFurther the input horizon output fromminimizing thethe setpoint the prediction schemes are classified also SMPC).onand Further SMPC input the setpoint control as well theover expenditure ofthe parameterization (either based the control deviation the process by minimizing predicted Generalized Polynomial Chaos based SMPC effort inasdriving the process output toof open loop control or pre-stabilizing feedback control) well as minimizing the expenditure of the control as parameterization (either based on the control input effort in driving the process output to the setpoint control) open looptype control or pre-stabilizing output setpoint over the prediction schemes arefeedback classified also SMPC). Further subject from to the the deterministic constraints in states,horizon inputs, function, constraints and and the of SMPC objective effort intoasdriving the process output in to states, the setpoint feedback control) open loop control or pre-stabilizing subject the deterministic inputs, constraints and the type of objective function, control as well minimizing the constraints expenditure parameterization (either based on the control input outputs (Qin and Badgwell, 2003). of It the should be and to constrained optimization algorithm being employedand subject to the(Qin deterministic constraints in states, inputs, function, constraints and and the of objective 2003). Ittheshould be and effort in the Badgwell, process output setpoint employed to constrained optimization being feedback control) open looptype control oftothe controller notedoutputs thatdriving only and the current value numerically solveor pre-stabilizing the algorithm constrained optimization outputs and Badgwell, 2003). It should be and employed to constrained optimization algorithm being only value controller noted subject the(Qin deterministic constraints in the states, inputs, numerically solve the constrained optimization constraints and and the type of objective function, whole procedure output that isto applied tothethecurrent plant and the of problem to obtain the current and future control value of the controller noted that only the current numerically solve the the algorithm constrained optimization whole procedure output is applied to the plant and2003). thetime outputs (Qin andnext Badgwell, Itinstant. should be and problem obtain current control employed to constrained optimization being The is repeated at the sampling be also noted thatand the future robust model outputs. Ittomay whole procedure output is applied tothenext thecurrent plant and thetime problem tomay obtain the current and future control The is repeated at the sampling instant. value of the controller noted that only be also noted that the robust model outputs. It numerically solve the constrained optimization aforementioned deterministic MPC formulation hasn’t predictive control scheme (the min-max approach The is repeated at the sampling time instant. aforementioned deterministic MPC formulation hasn’t may be scheme also noted the future robust model outputs. whole procedure output isinto applied to next the plant the (thethat min-max approach predictive problem Itto obtain the and control taken account the and model uncertainty, Mayne, 1998, the tube-based described incontrol Scokaert andcurrent aforementioned deterministic formulation hasn’t taken into atuncertainty account the MPC model uncertainty, scheme (thethat min-max approach predictive control is repeated the next sampling time instant. Mayne, 1998, the tube-based described in Scokaert and may be also noted the robust outputs. It and probabilistic state The and measurement only MPC in Mayne et al. 2014) will handle model taken into uncertainty account the MPC model uncertainty, and probabilistic state and measurement 1998, tube-based described Scokaert and aforementioned deterministic formulation hasn’t handle only MPC in in Mayne et al. Mayne, 2014) will the scheme (the min-max approach predictive control 2016). output constraints (Mesbah, deterministic description of plant uncertainties and probabilistic state and measurement output (Mesbah, handle only MPC in in Mayne et and al. Mayne, 2014) will the taken constraints into uncertainty account the2016). model uncertainty, deterministic description of plant uncertainties 1998, tube-based described Scokaert whereas stochastic MPC can systematically handle output constraints (Mesbah, deterministic description of systematically plant uncertainties uncertainty and2016). probabilistic state and measurement whereas handle only MPC in stochastic Mayne etMPC al. can 2014) will handle whereas stochastic MPC can handle output constraints (Mesbah, 2016). deterministic description of systematically plant uncertainties Copyright © 2018 IFAC 584 whereas stochastic MPCreserved. can systematically handle 2405-8963 © IFAC (International Federation of Automatic Control) by Elsevier Ltd. All rights Copyright © 2018, 2018 IFAC 584Hosting Peer review©under of International Federation of Automatic Copyright 2018 responsibility IFAC 584Control. 10.1016/j.ifacol.2018.09.360 Copyright © 2018 IFAC 584

2018 IFAC ADCHEM Shenyang, Liaoning, China, July 25-27, Jagadeesan Prakash et al. / IFAC PapersOnLine 51-18 (2018) 590–595 2018

probabilistic description of uncertainties (multi-stage or a scenario based NMPC approach in Lucia et al., 2013, Bavdekar and Mesbah, 2016). Tube-based MPC for linear systems and nonlinear systems has been proposed as an alternative to min–max approaches based robust MPC. Even though tube based MPC can guarantee stability and satisfy constraints, it does not address the issue of optimal performance in the presence of uncertainties (Mayne et al. 2014).

591

The organization of the paper is as follows: Section 2 presents the ensemble Kalman filter algorithm. Section 4 reports the algorithm for the determination of an optimal control policy for the stochastic linear system. Simulation studies have been reported in section 4 followed by concluding remarks in section 5. 2. ENSEMBLE KALMAN FILTER ALGORITHM Let us assume that the stochastic linear system is represented using the state and measurement equations given below: 𝐱𝐱(k) = 𝚽𝚽𝚽𝚽(k − 1) + 𝚪𝚪u 𝐮𝐮(k − 1) + 𝚪𝚪d 𝐝𝐝(k − 1) + 𝐰𝐰(k) }(1) 𝐲𝐲(k) = 𝐂𝐂𝐂𝐂(k) + 𝐯𝐯(k) n u where 𝐱𝐱(k)ϵR are the state variables, 𝐮𝐮(k)ϵR are the manipulated inputs, 𝐝𝐝(k)ϵRd are the disturbance variables and 𝐲𝐲(k)ϵRy are the measured output variables. It is assumed that the state and measurement equations are affected by additive process noise and measurement noise, respectively as shown in equation 1. The system matrices (i.e. 𝚽𝚽, 𝚪𝚪u , 𝚪𝚪d and 𝐂𝐂 ) and the distribution of process noise {𝐰𝐰(k)} and measurement noise {𝐯𝐯(k)} are assumed to be known in this work. The determination of the optimal state estimates using an ensemble Kalman filter algorithm is as follows.

Several review articles have appeared which comprehensively survey the theoretical developments and industrial practices in the area of model predictive control (Qin and Badgwell 2003; Forbes et al. 2015). In recent years, the concept of stochastic MPC (Mesbah et al. 2014; Mesbah, 2016) has attracted the attention of several researchers and has been applied in many different areas, such as building climate control, power generation and distribution, chemical processes etc. Mesbah et al. 2016, while discussing future research directions in his review article on stochastic MPC, stressed that most SMPC approaches are developed under the assumption of full state feedback. Unfortunately, all states are not available for measurement in many situations. Hence, SMPC algorithms that include state estimation remain an open problem for stochastic systems. It should be noted that the state estimator based MPC formulations, which use extended Kalman filter (Ricker 1990; Subramanian et al. 2015), derivative free Kalman filter (Prakash et al. 2010), and particle filter (Sehr & Bitmead, 2017) have been reported in the process control literature.

The ensemble Kalman filter (EnKF) is initialized by drawing N samples {𝐱𝐱̂ (i) (0|0): i = 1, … N} from a suitable initial state distribution(𝐩𝐩[𝐱𝐱(0)]). At each time step, N samples {𝐰𝐰 (i) (k), 𝐯𝐯 (i) (k): i = 1, … N} for {𝐰𝐰(k)} and {𝐯𝐯(k)} are drawn randomly using the distribution of process noise and measurement noise respectively. The computation of the optimal state estimates using an EnKF is as follows: 𝐱𝐱̂ (i) (k|k − 1) = [𝚽𝚽𝐱𝐱̂ (i) (k − 1|k − 1) + 𝚪𝚪u 𝐮𝐮(k − 1) + 𝐰𝐰 (i) (k)] (2) These transformed particles are then used to estimate the sample mean and sample covariance as follows:

An important contribution of this paper is the development of a state estimator based stochastic model predictive control algorithm to determine an optimal control policy for a constrained stochastic linear system. In order to determine an optimal control policy for the system affected by random disturbances and measurements corrupted by random noise, the authors have minimized the uncertain objective function, subject to uncertain constraints such as states and outputs as well as deterministic input constraints using the quantile-based scenario analysis (QSA) approach proposed by Zamar et al. 2017. The advantages of the QSA approach for stochastic optimization over the mean-based based stochastic optimization approach are outlined in Zamar et al. 2017.

N

1 𝐱𝐱̅(k|k − 1) = ∑ 𝐱𝐱̂ (i) (k|k − 1) N 𝐲𝐲̅(k|k − 1) = 𝐏𝐏ε,e (k|k − 1)

i=1 N

1 ∑[𝐂𝐂𝐱𝐱̂ (i) (k|k − 1) + 𝐯𝐯 (i) (k)] N i=1

(3)

(4)

N

1 T = ∑[𝛆𝛆(i) (k|k − 1)][𝛆𝛆(i) (k|k − 1)] (5) N−1

𝐏𝐏e,e (k|k − 1) =

It should be noted that the QSA approach minimizes a weighted average of the quantiles of the objective and constraint distributions. Hence, it will be much more robust than simply minimizing the corresponding expected values of the objective and constraint distributions, as it is typically done in all Stochastic Model Predictive Control formulations.

i=1 N

1 T ∑[𝐞𝐞(i) (k|k − 1)][𝐞𝐞(i) (k|k − 1)] (6) N−1 i=1

where, 𝛆𝛆(i) (k|k − 1) = 𝐱𝐱̂ (i) (k|k − 1) − 𝐱𝐱̅(k|k − 1) (7) 𝐞𝐞(i) (k|k − 1) = [𝐂𝐂𝐱𝐱̂ (i) (k|k − 1) + 𝐯𝐯 (i) (k)] − 𝐲𝐲̅(k|k − 1) (8) The Kalman gain and updated sample points are then computed as follows: 585

2018 IFAC ADCHEM 592 Prakash et al. / IFAC PapersOnLine 51-18 (2018) 590–595 Shenyang, Liaoning, China, July 25-27, Jagadeesan 2018 −1

𝐋𝐋(k|k − 1) = 𝐏𝐏ε,e (k|k − 1)[𝐏𝐏e,e (k|k − 1)] (9) (i) (i) (i) 𝚼𝚼 (k|k − 1) = 𝐲𝐲(k) + 𝐯𝐯 (k) − 𝐂𝐂𝐱𝐱̂ (k|k − 1) (10) 𝐱𝐱̂ (i) (k|k) = 𝐱𝐱̂ (i) (k|k − 1) + 𝐋𝐋(k|k − 1)𝚼𝚼 (i) (k|k − 1) (11) (i) 𝚼𝚼 (k|k) = 𝐲𝐲(k) − 𝐂𝐂𝐱𝐱̂ (i) (k|k) (12) The accuracy of the estimates depends on the number of sample point (N). Evensen (2003) has indicated that sample points between 50 and 100 suffices even for large dimensional systems.

is minimized P

2 min (i) 𝐉𝐉 = (i) [∑‖𝐞𝐞f (k + j|k)‖𝐖𝐖 𝐔𝐔f E j=1

M−1

+ ∑‖Δ𝐮𝐮(i) (k + j|k)‖𝐖𝐖 ] (13) 2

j=0

∆u

subject to the following constraints: 𝐱𝐱̂ (i) (k + j + 1|k) = [𝚽𝚽𝐱𝐱̂ (i) (k + j|k) + 𝚪𝚪u 𝐮𝐮(i) (k + j|k) + 𝐰𝐰 (i) (k + j + 1|k) (i) +𝐋𝐋(k)𝛈𝛈e (k + j + 1|k)] ∀ j = 0,1 … P − 1 (14) (i) 𝐲𝐲̂ (k + j + 1|k) = [𝐂𝐂𝐱𝐱̂ (i) (k + j + 1|k) + 𝐯𝐯 (i) (k + j + 1|k) (i) +𝛈𝛈d (k + j + 1|k)]∀j = 0,1 … P − 1 (15) (i) 𝐮𝐮 (k + j|k) = 𝐮𝐮(i) (k + M − 1) ∀j = M, M + 1 … P − 1 (16) 𝐱𝐱 L ≤ 𝐱𝐱̂ (i) (k + j|k) ≤ 𝐱𝐱 H ∀ j = 1…P (17) (i) (k + j|k) ≤ 𝐲𝐲H 𝐲𝐲L ≤ 𝐲𝐲̂ ∀ j = 1…P (18) 𝐮𝐮L ≤ 𝐮𝐮(i) (k + j|k) ≤ 𝐮𝐮H ∀ j = 0…M − 1 (19) Δ𝐮𝐮L ≤ Δ𝐮𝐮(i) (k + j|k) ≤ Δ𝐮𝐮H ∀ j = 0 … M − 1 (20) where (i) 𝐞𝐞f (k + j|k) = 𝐲𝐲sp (k + j|k) − 𝐲𝐲̂ (i) (k + j|k) ∀ j = 1 … P (21) (i) (i) (i) (k (k (k + j|k) = 𝐮𝐮 + j|k) − 𝐮𝐮 + j − 1|k) Δ𝐮𝐮 ∀ j = 0 … M − 1(22) The filtered innovation signals are computed as follows: (i) (i) 𝛈𝛈e (k + j + 1|k) = 𝛈𝛈e (k + j|k) } ∀ j = 0,1 … P − 1 (23) (i) (i) 𝛈𝛈e (k|k) = 𝛄𝛄f (k|k − 1) (i) (i) 𝛈𝛈d (k + j + 1|k) = 𝛈𝛈d (k + j|k) } ∀ j = 0,1 … P − 1 (24) (i) (i) 𝛈𝛈d (k|k) = 𝛄𝛄f (k|k) (i) (i) 𝛄𝛄f (k|k − 1) = [𝚽𝚽x 𝚼𝚼f (k − 1|k − 2) + [𝐈𝐈 − 𝚽𝚽x ] 𝚼𝚼 (i) (k|k − 1)] (25) (i) (i) 𝛄𝛄f (k|k) = [𝚽𝚽𝑦𝑦 𝚼𝚼f (k − 1|k − 1) +[𝐈𝐈 − 𝚽𝚽𝑦𝑦 ]𝚼𝚼 (i) (k|k)](26)

3. EnKF BASED STOCHATIC MODEL PREDICTIVE CONTROLLER The determination of an optimal control policy for the stochastic linear system using the proposed ensemble Kalman filter based stochastic linear model predictive control scheme (EnKF - SMPC) is as follows: Given the future set point trajectory 𝐲𝐲sp (k + j|k), (j = 1 … P), the proposed EnKF - SMPC will determine the current and future controller outputs 𝐔𝐔f = {𝐮𝐮(k|k) … 𝐮𝐮(k + 1|k) … 𝐮𝐮(k + M − 1|k)} in two stages using QSA approach:  Stage-1: Compute the current and future (i) controller outputs 𝐔𝐔f for each scenario by minimizing the predicted deviation of the process output from the setpoint over the prediction horizon as well as minimizing the expenditure of control effort in driving the process output to setpoint subject to constraints such as states, outputs as well as inputs.  Stage-2: Determine a single feasible control policy 𝐔𝐔f , by minimizing the mean of the objective function distribution subject to satisfying the mean of the constraint distribution. The detailed computation procedure in each stage is as follows: Stage-1: There are three sources of uncertainties that arise while performing predictions, they are (a) uncertainty in initial state at the beginning of the prediction {𝐱𝐱̂ (i) (k|k)∀i = 1, … N} and (b) unmeasured random disturbances {𝐰𝐰 (i) (k + j|k)∀j = 1, … P} that may occur in future and that affects the state equation and (c) the unmeasured random output disturbances {𝐯𝐯 (i) (k + j|k)∀j = 1, … P} that may occur in future and that affects the measurement model. In this work, the number of scenarios is considered to be equivalent to that of number of sample points (N) used in the ensemble Kalman filter. Each scenario is viewed as one realization of the process noise 𝐖𝐖 (i) = {𝐰𝐰 (i) (k + 1|k) 𝐰𝐰 (i) (k + 2|k) … 𝐰𝐰 (i) (k + P|k)}, measurement noise 𝐕𝐕 (i) = {𝐯𝐯 (i) (k + 1|k) 𝐯𝐯 (i) (k + 2|k) … 𝐯𝐯 (i) (k + P|k)} over the prediction horizon (P) as well as the i th sample point of the state estimate at the beginning of the prediction horizon generated by the ensemble Kalman filter {𝐱𝐱̂ (i) (k|k)}. For each scenario, the following performance measure

It may be noted that 𝛄𝛄f (k|k − 1) and 𝛄𝛄f (k|k) are filtered values of innovation signals 𝚼𝚼 (i) (k|k − 1) and 𝚼𝚼 (i) (k|k), respectively, which are defined by equation (10) and equation (12). The 𝚽𝚽x and 𝚽𝚽𝑦𝑦 matrices are parameterized as follows, 𝚽𝚽x = diag{α1 α2 … αy } and 𝚽𝚽y = diag{β1 β2 … βy } where 0 ≤ αi ≤ 1 and 0 ≤ βi ≤ 1 ∀ i = 1,2, … y can be chosen to shape the response of the stochastic model predictive controller in the presence of unmeasured disturbance. Equation (16) states that no future control moves are planned beyond the control horizon of M steps. Stage-2: The QSA method developed by Zamar et al. (2017), is used to find a single, feasible, and robust control policy. That is, each scenario solution (i) 𝐔𝐔f is evaluated across all sampled scenarios as shown below. (i)

586

(i)

2018 IFAC ADCHEM Shenyang, Liaoning, China, July 25-27, Jagadeesan Prakash et al. / IFAC PapersOnLine 51-18 (2018) 590–595 2018

for i:1:N { for j:1:N { (i) 𝐡𝐡(i,j) = 𝐉𝐉(𝐔𝐔f , {𝐖𝐖 (j) , 𝐕𝐕 (j) , 𝐱𝐱̂ (j) (k|k)}) for r:1:m { (i) 𝐂𝐂 (i,j,r) = 𝚿𝚿r (𝐔𝐔f , {𝐖𝐖 (j) , 𝐕𝐕 (j) , 𝐱𝐱̂ (j) (k|k)}) } } } Here, 𝚿𝚿r (. ) represents the probabilistic constraints. Next, the cumulative distribution functions (CDF) of the objective function (𝐅𝐅𝐔𝐔 (i) (z)) and constraints

4. SIMULATION STUDY The efficacy of the proposed control scheme has been validated on the constrained single-input and singleoutput linear system given by 𝐱𝐱(k + 1) = 0.5𝐱𝐱(k) + 𝐮𝐮(k) + d(k) + 𝐰𝐰(k) 𝐲𝐲(k) = 𝐱𝐱(k) + 𝐯𝐯(k); −2 ≤ 𝐮𝐮 ≤ 2; −1 ≤ 𝐱𝐱 ≤ 1 TABLE 1. PARAMETER ASSOCIATED WITH EnKF and MPC Parameter Value Parameter Value R 0.001 Q 0.01 𝚽𝚽x & No. of 0 200 𝚽𝚽y Scenarios P(0|0) 0.01 x̂(0|0) 0.5 P 10 M 1 1000 1000 𝐰𝐰E 𝐰𝐰∆u 1 1 𝚪𝚪𝑢𝑢 𝚪𝚪d The random disturbances {𝐰𝐰(k)} and measurement noise {𝐯𝐯(k)}are assumed to be zero mean Gaussian white noise sequences with covariance matrices Q & R respectively. The disturbance term 𝐝𝐝(k) is assumed to deterministic in this work. It is assumed that the system is controllable and also observable. The servoregulatory performance of the system with the proposed EnKF-SMPC and EnKF-min-max MPC in the presence of model-plant mismatch (MPM), which is 50 % increase in the system matrix (𝚽𝚽) are reported in Fig. 1. The controller computations, however, are based on the nominal model parameters. The parameters associated with the EnKF and MPC are reported in Table 1. The evolution of true and estimated state variables of the system with EnKF-SMPC is reported in Fig. 2. It can be inferred from Fig.2 that the EnKF is able to generate fairly accurate filtered estimate of the state variable. The evolution of controller outputs is reported in Fig.3. The inferences drawn from the simulation studies are as follows:  It may be noted that both control schemes approached the desired setpoint, as shown in Fig. 1, during the discrete time interval between 1 and 49. This part of the simulation demonstrates the ability to transfer the system from the initial state 𝐱𝐱(0) = 0.5 to the desired setpoint (i.e. the origin).  A step change disturbance (d) of magnitude 0.5 is introduced at discrete time instant 50 and both schemes are able to reject the disturbance. As a result, the output reaches the setpoint, as shown in Fig. 1, during the discrete time interval between 50 and 99.  With the disturbance being persistent, a step change in the setpoint of magnitude 0.8 (See Fig. 1) is introduced at the 100th sampling instant. Both schemes are able to maintain the output at the desired setpoint, as evident from Fig.1, during the discrete time interval between 100 and 300.

f

(𝐆𝐆𝐔𝐔 (i) ,r (ζ)) of each solution are obtained as follows, f

for i:1:N {

N

for r:1:m {

1 𝐅𝐅𝐔𝐔 (i) (z) = ∑ 𝐈𝐈(𝐡𝐡(i,j) ≤ z) N f j=1

N

1 𝐆𝐆𝐔𝐔 (i) ,r (ζ) = ∑ 𝐈𝐈(𝐂𝐂 (i,j,r) ≤ ζ) N f j=1

}

} Subsequently, the optimal values of the current and future controller outputs are computed by solving the following coordination model. argmin

1

𝐔𝐔f = i ϵ 𝐈𝐈 ∫ 𝐅𝐅 −1 (i) (t)𝛀𝛀0 (t)dt subject to 1

0

𝐔𝐔f

∫ 𝐆𝐆 −1(i) (t)𝛀𝛀r (t)dt ≤ 𝛄𝛄r rϵR 0

𝐔𝐔f ,r

𝐈𝐈 = [1,2, … N]

593

(27) (28)

The above formulation of the problem attempts to minimize the weighted average of the quantiles of the objective function, subject to satisfying a weighted average of the quantiles of the constraint performance functions, also called a risk spectrum (Zamar et al. (2017)). 𝛀𝛀0 (t) & 𝛀𝛀r (t) are positive weighting functions that integrate to unity over the range 0-1. In the present work all quantiles have been given equal weights. The desired closed loop performance of the proposed SMPC scheme can be achieved by appropriately selecting the prediction horizon P, control horizon M, the error weighting matrix (𝐖𝐖E ) input weighting matrix(𝐖𝐖Δu ) and other parameters. Further, the SMPC scheme is implemented in a receding horizon framework. That is, only the current controller output 𝐮𝐮(k|k) is implemented on the plant and the constrained optimization problem is reformulated at the next sampling instant based on the updated information from the plant. 587

1

1

0.8

0.8

0.6

0.6

State Variable(x)

Process Output(y)

2018 IFAC ADCHEM Shenyang, Liaoning, China, July 25-27, 2018 594 Jagadeesan Prakash et al. / IFAC PapersOnLine 51-18 (2018) 590–595

EnKF-SMPC Setpoint EnKF-Min-max MPC

0.4

0.2

0

-0.2

True Estimated

0.4

0.2

0

50

100

150 200 Sampling Instants

250

300

-0.2

Fig.1. Servo-Regulatory Response of a Stochastic Linear System with EnKF-SMPC and EnKF based min-max based MPC The performance of the proposed control scheme has been assessed through stochastic simulation studies. A simulation run consisting of N TR= 25 trials with the length of each simulation trail, L, being equal to 300 is conducted. The sum of squared output error (SSOE), defined 2

is

used

as

100

150 200 Sampling Instants

250

300

Fig.2. EnKF-SMPC: Evolution of true and estimated state variable using EnKF 0.5 0.4 EnKF-SMPC EnKF-Min-max MPC

0.3

Controller Output(u)

SSOE = ∑Lk=1 [(𝐲𝐲sp (k) − 𝐲𝐲(k)) ]

50

a

performance index, where 𝐲𝐲sp (k) denotes the setpoint at time step k. Statistics of SSOE computed for each simulation run is used to assess the efficacy of the control scheme. The mean and standard deviation of SSOE values based on the NTR = 25 trials for EnKFSMPC and EnKF-min-max MPC are reported in Table 2. As expected, the state estimates generated by EnKF are found to be biased after the introduction of step change in the unmeasured disturbance at discrete time instant 50 (Fig.2). It should be noted that even if the states are biased the proposed EnKF-SMPC scheme and EnKF-min-max MPC scheme are able to achieve offset free servo-regulatory performance. From Table 2, it can be inferred that the average SSOE is found to be less for the proposed control scheme. The results of a ttest comparing the mean difference in SSOE between the two control schemes revealed that EnKF-SMPC obtained a statistically significant improvement in the SSOE compared to that of the EnKF-min-max MPC. The improvement is estimated to be 0.1518 with a standard error of 0.0477.

0.2 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 50

100

150 200 Sampling Instants

250

300

Fig.3. Evolution of Controller Outputs TABLE 2. AVERAGE SSOE VALUES FOR 25 TRIALS SSOE in the Presence of Control Scheme Model Plant Mismatch EnKF – MPC 3.1713 (0.0668) EnKF-Min – Max MPC 3.3220 (0.2291) 5. CONCLUSIONS The quantile based scenarios analysis (QSA) approach was used to determine an optimal control policy for a constrained stochastic linear system in an elegant manner. Monte Carlo simulation analyses found that the proposed ensemble Kalman filter based stochastic model predictive control scheme, EnKF, can reject step like disturbances by bringing the process variable back to the setpoint and exhibits offset free performance. The average SSOE of the proposed control scheme was found to be less compared to the min-max MPC scheme in the presence of model-plant mismatch. It should be noted that, both EnKF schemes were able to generate accurate state estimates. Since, the EnKF-SMPC obtained a statistically significant improvement in the SSOE, compared to that of the EnKF-min-max MPC in the presence of mode-plant mismatch, its performance can be considered to be efficient. It should be noted that with the help of parallel computing, it is possible to

The efficacy of the proposed control scheme has been also validated on the constrained single-input and single-output linear system with non-negative constraints on process noise as shown below: 𝐱𝐱(k + 1) = 0.5𝐱𝐱(k) + 𝐮𝐮(k) + |𝐰𝐰(k)| The servo-regulatory performance of the linear system in the presence of non-negative constraints on the process noise is reported in Fig. 4. It can be inferred from Fig.4, that the EnKF-SMPC is able to reject the non-negative random disturbances (Fig.5b) and achieve offset-free servo-regulatory performance. The evolution of controller output is shown in Fig.5a. 588

2018 IFAC ADCHEM Shenyang, Liaoning, China, July 25-27, Jagadeesan Prakash et al. / IFAC PapersOnLine 51-18 (2018) 590–595 2018

reduce the computational time of the control scheme proposed in this work. Further work is in progress to extend the proposed control scheme for a stochastic non-linear system in the presence of probabilistic state constraints.

Industrial & Engineering Chemistry Research, 33(6), 1530–1541. Lucia, S., Finkler, T., and Engell, S. (2013). Multistage nonlinear model predictive control applied to a semi- batch polymerization reactor under uncertainty. Journal of Process Control, 23(9), 1306–1319.

0.8

Martin A. Sehr &. Bitmead, R.R. (2017). Particle Model Predictive Control: Tractable Stochastic Nonlinear Output-Feedback MPC, IFAC–Papers Online, 50(1), 15361-15366.

Process Output(y)

0.6 EnKF-SMPC Setpoint

0.4

595

0.2

Mesbah, A.(2016). Stochastic Model Predictive Control: An Overview and Perspectives for Future Research, IEEE Control Systems, 30-34.

0

-0.2

10

20

30

40 50 60 Sampling Instants

70

80

90

100

Mayne, D.Q. (2014). Model predictive control: Recent developments and future promise. Automatica, 50, 2967–2986.

Controller Output(u)

Fig.4. Servo-Regulatory Response of a Stochastic Linear System with EnKF-SMPC in the presence of nonnegative constraints on the process noise

Mesbah, A., Streif, S., Findeisen, R., and Braatz, R. D.(2014). Stochastic nonlinear model predictive control with probabilistic constraints in Proc. American Control Conf., Portland, OR, 2413– 2419.

0.4 0.2 0 -0.2 10

20

30

40 50 60 Sampling Instants

70

80

90

100

10

20

30

40 50 60 Sampling Instants

70

80

90

100

J. Prakash, Patwardhan, S.C., Shah, S.L. (2010), “State estimation and nonlinear predictive control of autonomous hybrid system using Derivative free estimators”, Journal of Process Control, 20, 787799.

0.3

abs(w)

0.2 0.1 0 -0.1

Scokaert, P. and Mayne, D.(1998). Min-max feedback model predictive control for constrained linear systems. IEEE Transactions on Automatic Control, 43(8), 1136–1142

Fig.5. Evolution of Controller Output (EnKF-SMPC) & non-negative Random Disturbance

Subramanian, S., Lucia, S., and Engell, S. (2015) Adaptive Multi-stage Output Feedback NMPC using the Extended Kalman Filter for time varying uncertainties applied to a CSTR, IFACPapersOnLine, 48(23), 242–247

REFERENCES Bavdekar, V. and Mesbah, A.(2016). Stochastic nonlinear model predictive control with joint chance constraints,” in Proc.10th IFAC Symp. Nonlinear Control Systems, Monterey, CA, 270– 275.

Qin, S.J. and Badgwell, T.A. (2003). A Survey of Industrial Model Predictive Control Strategy, Control Engineering Practice, 11(7), 733-764.

Evenson, G. (2003). The Ensemble Kalman Filter: Theoretical Formulation and Practical Implementation, Ocean Dynamics, 53, 343-367.

Zamar, S.D., Gopaluni, R.B., Sokhansanj, S., Newlands, N.K. (2017) A quantile-based scenario analysis approach to biomass supply chain optimization under uncertainty, Computers and Chemical Engineering, 97, 114–123

Forbes, M. G., Patwardhan, R. S., Hamadah, H., & Gopaluni,R. B. (2015). Model predictive control in industry: Challenges and opportunities. IFACPapersOnLine, 48(8), 531-538. Lee, J.H. and Ricker, N.L. (1994). Extended Kalman filter based nonlinear model predictive control. 589