- Email: [email protected]
New internal optimal neurocontrol for a series FACTS device in a power transmission line
Neural Networks 16 (2003) 881–890 www.elsevier.com/locate/neunet
2003 Special issue
New internal optimal neurocontrol for a series FACTS device in a power transmission line Jung-Wook Parka,*, Ronald G. Harleya, Ganesh K. Venayagamoorthyb b
a School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0250, USA Department of Electrical and Computer Engineering, University of Missouri-Rolla, Missouri-Rolla, MO 65409-0249, USA
Abstract In this paper, the proportional-integral (PI) based conventional internal controller (CONVC) of a power electronic based series compensator in an electric power system, is replaced by a nonlinear optimal controller based on the dual heuristic programming (DHP) optimization algorithm. The performance of the CONVC is compared with that of the DHP controller with respect to damping low frequency oscillations. Simulation results using the PSCAD/EMTDC software package are presented. q 2003 Elsevier Science Ltd. All rights reserved. Keywords: Neurocontroller; Dual heuristic programming; Power system
1. Introduction In the last decade, flexible AC transmission system (FACTS) devices (Hingorani & Gyugyi, 2002; Makombe & ¨ ngquist, Ghandhari, & AndersJenkins, 1999; Noroozian, A son, 1997; Schauder et al., 1998) have been progressively developed for controlling power flow in a power transmission system, improving the transient stability, damping power system oscillations, and providing voltage stability by using high power semiconductor technology based inverters connected to the electric power grid. Recently, the static synchronous series compensator (SSSC) (Gyugyi, Schauder, & Sen, 1997; Rigby & Harley, 1998; Sen, 1998) among the FACTS device family has attracted considerable attention for damping of low frequency power oscillations in the lines by using controllable series voltage compensation. The internal control strategy for the inverter of a SSSC as well as other FACTS devices has traditionally been based on linear proportional-integral (PI) regulators. These conventional PI linear controllers (CONVC) operate well at one particular operating point where they have been designed. In other words, their transient and dynamic performances are degraded at any other operating point, or * Corresponding author. E-mail addresses: [email protected] (J.-W. Park), rharley@ ece.gatech.edu (R.G. Harley), [email protected] (G.K. Venayagamoorthy).
their gains have to be re-tuned for the new operating points. Artificial neural networks (ANNs) can offer an efficient alternative to overcome the above limitation of the CONVC for the internal control. However, only a few researchers (Dash, Mishra, & Panda, 2000; Mohaddes, Gole, & McLaren, 1999) have reported on FACTS device control using ANNs in the literature due to the difficulty of implementing an effective and fast internal control of the inverter with the ANNs. The recently developed adaptive critic designs (ACDs) (Park, Harley, & Venayagamoorthy, 2002; Prokhorov & Wunsch, 1997; Si & Wang, 2001; Venayagamoorthy, Harley, & Wunsch, 2002; Werbos, 1992) based techniques avoid the possibility of instability of the neurocontroller, and also find an optimal control (Bertsekas, 2001; White & Jordan, 1992). This paper proposes a novel intelligent internal control approach using the multilayer perceptron neural network (MLPNN) as an optimal nonlinear neurocontroller. The background of the dual heuristic programming (DHP) algorithm, which has the best robust control capability among the ACDs family, is presented. Based on the DHP algorithm, a novel nonlinear optimal neurocontroller (called DHPNC) is designed as an alternative for internal control of the SSSC, thereby replacing the PI based regulators for the currents but not for the DC link voltage.
0893-6080/03/$ - see front matter q 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0893-6080(03)00121-7
882
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
The performances of the DHPNC and CONVC are compared with respect to the damping of power oscillations by time-domain simulations in the PSCAD/EMTDC software environment.
2. Static synchronous series compensator The SSSC converter can control the reactive and/or active power on an AC system by changing both phasor angle and magnitude of the converter’s output voltage with a fast control action. Especially, the exchange of active power, which is the particular characteristic of the SSSC, is accomplished by controlling the DC voltage inside the SSSC (Hingorani & Gyugyi, 2002). 2.1. Modeling of SSSC The single machine infinite bus (SMIB) system shown in Fig. 1 is used to compare the damping control capabilities of the proposed DHPNC and CONVC for the SSSC. The plant consists of the synchronous generator (160 MV A, 15 kV (L –L)), turbine – governor system, automatic voltage regulator (AVR) – exciter system, transmission line connected to an infinite bus, and the SSSC connected in series with transmission line. The parameters of the synchronous generators and transmission line are given in Anderson and Fouad (1994). The EXACIA (IEEE alternator supplied rectifier excitation systems) and H_TURI/GOV1 (IEEE type hydro turbine – governor) models in PSCAD/EMTDC software package are used as the AVR/exciter system and turbine/governor, respectively. For the mathematical model of the SSSC, the associated equation can be represented with the lumped series transmission line reactance x0e (transmission line xe plus leakage reactance of series-connected transformer) and transmission series resistance re (the inverter is
regarded simply to have no conduction losses) in per unit as follows 3 2 rv e s 2 0 0 2 3 6 72 3 x0e 7 isa 6 isa 7 6 re vs 76 7 7 6 d6 76 isb 7 6 isb 7 ¼ 6 0 2 0 0 74 5 6 4 5 xe dt 7 6 7 6 isc 4 re vs 5 isc 0 0 2 0 xe 3 2 vs ðvsa þ vca 2 vra Þ 7 6 x0e 7 6 7 6 vs 7 6 ð1Þ þ 6 0 ðvsb þ vcb 2 vrb Þ 7 7 6 xe 7 6 5 4 vs ðv þ v 2 v Þ sc cc rc x0e where vs is the synchronous speed of the power system, vs is the sending-end voltage (terminal voltage in practice), is is the current in transmission line, vr is the receiving-end voltage in the infinite bus, and vc is the injected series compensation voltage. Using the synchronously rotating reference frame based transformation (Makombe & Jenkins, 1999), in which the daxis is always coincident with the instantaneous voltage vector v and the q-axis leads the d-axis by 908, the threephase circuit equations in Eq. (1) can be transformed to the following d – q axis vector resprentation d dt
"
id iq
#
2
re v s x0e
3
" # 7 id 7 rv 5 i q 2v 2 e0 s xe 2 vs 3 ðlvs l þ vcd 2 vrd Þ 0 6 xe 7 7 þ6 4 5 vs ðv 2 v Þ cq rq 0 xe
6 ¼6 4
2
Fig. 1. Plant: 160 MV A, 15 kV (L–L) SMIB test system.
v
ð2Þ
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Neglecting the series inverter harmonics, the AC side injected voltage vc in Fig. 1 can be expressed with relation to the capacitor voltage Vdc on the DC link as follows vcd ¼ mVdc cosðaÞ;
vcq ¼ mVdc sinðaÞ
ð3Þ
where a is the phase voltage difference between the voltages vc and vs (the vc leads the vs ), and m is the modulation index of the series inverter. The dynamics of the DC capacitor voltage is given by dVdc 1 Pdc 1 Vdc idc 1 ðvcd id þ vcq iq Þ ¼ ¼ ¼ C Vdc C Vdc C dt Vdc ¼
1 ½m{cosðaÞid þ sinðaÞiq } C
ð4Þ
2.2. Conventional control strategy The main goal of the SSSC is to inject the series voltage in quadrature with the line current and to maintain the DC voltage Vdc : For this purpose, the P – Q (real and reactive power) automatic power flow control mode (Hingorani & Gyugyi, 2002) in Fig. 2 is used. In Fig. 2, an instantaneous three-phase set of line voltages, va ; vb ; and vc is used to calculate the transformation angle, u provided by the vector phase-locked loop for synchronous operation of the series voltage source inverter shown in Fig. 1. As shown in Eq. (2), the three-phase set of measured line currents at the AC terminal of the SSSC is decomposed into its real/direct component, id ; and reactive/quadrature.component, iq : These actual signals (id and iq ) and the reference d – q current signals (ipd and ipq ) are compared, respectively.
883
The error signal Diq for the reactive power exchange is passed through the PI regulator PI-iq : The signal Dip for the real power exchange and maintenance of a constant Vdc ; is passed through the PI-ip : The Dip consists of the Did and error signal DV 0dc (which passes through the PI-Vdc ). p p The Vdc is the desired value for Vdc : The value of Vdc is determined by selecting a suitable value of the modulation index within its range of 0 to 1. Finally, the estimated signals (~vcq and v~ cd ) in Fig. 2 are used to compute the angle a and modulation index m to drive the gate turn-off (GTO) thyristor of the inverter.
3. DHP based neurocontroller design ACDs proposed by Werbos (1992) are new optimization techniques to handle the nonlinear optimal control problem using ANNs. The adaptive critic method finds the optimal control in the infinite horizon problem to minimize/maximize the userdefined heuristic cost-to-go function J for a system, by successively adapting two ANNs, namely the action network (which dispenses the control signals) and the critic network (which learns to approximate the cost-to-go function J). This process is called the approximate dynamic programming (ADP) for the value iteration J: The adaptation process starts with a nonoptimal, arbitrarily chosen, control by the action network; the critic network then guides the action network towards the optimal solution at each successive adaptation. During the adaptations, neither of the networks needs any ‘information’ of an optimal trajectory, only the desired cost needs to be known (Prokhorov & Wunsch, 1997).
Fig. 2. P – Q automatic power flow control diagram for the internal control of the SSSC.
884
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 3. Critic network adaptation in DHP: This diagram shows the implementation of Eq. (8). The same critic network is shown for two consecutive times, t and t þ 1: The discount factor g is chosen to be 0.5. Backpropagation paths are shown by dotted and dash-dot lines. The output of the critic network l~ðt þ 1Þ is backpropagated through the model network from its outputs to its inputs, yielding the first term of Eq. (7) and l~Jðt þ 1Þ=›AðtÞ: The latter is backpropagated through the action network from its outputs to its inputs forming the second term of Eq. (7). Backpropagation of the vector ›UðtÞ=›AðtÞ through the action network results in a vector with components computed as the last term of Eq. (8). The summation of all these signals produces the error vector eC ðtÞ used for training the critic network.
The DHP technique (among the ACDs family) has the strong control capability in that the critic network of the DHP approximates the derivatives of the function J with respect to the states of the plant to be controlled. The DHP algorithm described in this paper uses three different multilayer (three layer) perceptron neural networks (MLPNNs), namely one for each of the critic, model, and action networks. The weight vector V of the MLPNN is adjusted/ trained using the gradient descent based backpropagation algorithm. By trial and error, 14 neurons are used in the hidden layer of the MLPNN for the model network, and 10 neurons for each of the critic and action networks.
The critic network estimates the derivatives of function J with respect to the vector of observables of the plant ~ ¼ (identified by the model network), which is the DYðtÞ ½D~ıq ðtÞ; D^ıp ðtÞ (input vector of the critic network), and it learns to minimize the following error measure over time kEc k ¼
X
eTC ðtÞeC ðtÞ
ð5Þ
t
3.1. Critic network As mentioned before, the DHPNC is designed to replace the PI regulators PI-iq and PI-ip in Fig. 2. The input reference vector Jref into the SSSC and output vector DJ from the SSSC are † Jref ðtÞ; input reference vector to the SSSC ¼ p ½ipp ðtÞ; ipd ðtÞ; Vdc ðtÞ: † DJðtÞ; output vector from the SSSC ¼ ½Diq ðtÞ; Dip ðtÞ: The configuration for the critic network adaptation in the DHP is shown in Fig. 3. The inputs and outputs of the action and model networks used in the critic network adaptation are shown in Figs. 4 and 5.
Fig. 4. Input–output mapping of the model network at t used in the critic network adaptation in Fig. 3.
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 5. Input–output mapping of the action network at t used in the critic network adaptation in Fig. 3.
eC ðtÞ ¼
^ ^ Yðt ^ þ 1Þ ›J½DYðtÞ ›J½D ›U½DYðtÞ 2g 2 ^ ›DYðtÞ ›DYðtÞ ›DYðtÞ
ð6Þ
After exploiting all relevant pathways of backpropagation as shown in Fig. 3, where the paths of derivatives and adaptation of the critic network are depicted by dotted and dash-dot lines, the error signal eC ðtÞ is used for training to update the weights of the critic network. The jth component of the second term in Eq. (6) can be expressed by the output ^ Yðt ^ þ 1Þ= of critic network at time t þ 1; l^i ðt þ 1Þ ¼ ›J½D ^ ›DYi ðt þ 1Þ as follows n m X n ^ Yðt ^ þ 1Þ X ^ ðt þ 1Þ X ›J½D ›Y ¼ l^i ðt þ 1Þ i þ l^i ðt þ 1Þ ›DYj ðtÞ › DY ðtÞ j i¼1 k¼1 i¼1
^ j ðt þ 1Þ ›A ðtÞ ›DY k ›Ak ðtÞ ›DYj ðtÞ
ð7Þ
^ ^ Yðt ^ þ 1Þ ›J½DYðtÞ ›J½D ›U½DYðtÞ 2g 2 ^ ›DYj ðtÞ ›DYj ðtÞ ›DYj ðtÞ 2
m X ›U½DYðtÞ ›Ak ðtÞ ›Ak ðtÞ ›DYj ðtÞ k¼1
›eC ðtÞ ›VC ðtÞ
3.2. Action network The adaptation of the action network in Fig. 3 is shown in Fig. 6, which propagates l^ðt þ 1Þ back through the model network to the action network. The goal of this adaptation is expressed in Eq. (10), and the weights of the action network are updated by Eq. (11) ^ Yðt ^ þ 1Þ ›U½DYðtÞ ›J½D þg ¼ 0; ;t: ð10Þ ›AðtÞ ›AðtÞ " # ^ Yðt ^ þ 1Þ T ›AðtÞ ›U½DYðtÞ ›J½D DVA ðtÞ ¼ 2hA þg ›AðtÞ ›AðtÞ ›VA ðtÞ where hA is a positive learning rate and VA contains the weights of the DHP action network. The discount factor g of 0.5 and the user-defined utility function UðtÞ in Eq. (12) are used in Eqs. (8) and (10) during adaptation of the critic and action networks UðtÞ ¼ ½Diq ðtÞ þ Diq ðt 2 1Þ þ Diq ðt 2 2Þ2 þ ½Dip ðtÞ þ Dip ðt 2 1Þ þ Dip ðt 2 2Þ2
ð8Þ
Using Eq. (8), the expression for the weights’ update for the critic network is as follows DVC ðtÞ ¼ 2hC eTC ðtÞ
Fig. 6. Action network adaptation in DHP: The discount factor g is chosen to be 0.5. Backpropagation paths are shown by dotted lines. The output of the critic network l^ðt þ 1Þ at time ðt þ 1Þ is backpropagated through the model network from its outputs to its inputs (output of the action network), and the resulting vector multiplied by the discount factor ðg ¼ 0:5Þ and added to ›UðtÞ=›AðtÞ: Then, an incremental adaptation of the action network is carried out by Eqs. (10) and (11).
ð11Þ
where n and m are the numbers of outputs of the model and the action networks, respectively. By using Eq. (7), each component of the vector eC ðtÞ from Eq. (6) is determined by eCj ðtÞ ¼
885
ð9Þ
where hC is a positive learning rate and VC contains the weights of the DHP critic network.
ð12Þ
3.3. Model network Fig. 7 shows how the model network (identifier) is trained to identify the dynamics of the plant in Fig. 1. The nonlinear autoregressive moving average with exogenous inputs (NARMAX) model is used as the structure for the online identification. The components of vectors Yref ðtÞ; D ^ YðtÞ; AðtÞ; and DYðtÞ are already noted in subsection A above (Figs. 4 and 5).
886
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
et al. (2002), Prokhorov & Wunsch (1997) and Venayagamoorthy, Harley, & Wunsch (2003). Note that the model network is trained before the training of the action and critic networks, and the DHPNC with the fixed (converged) weights for the critic and action networks is used to control the plant for real-time operation. In other words, they have been successfully trained to their optimization purposes, which are the value iteration (to minimize the value of derivatives of the function J) for the critic network and policy iteration (to solve the optimal control u in optimal policy) for the action network (Bertsekas, 2001).
Fig. 7. NARMAX model for on-line training of the model network.
The residual vector, eM ðtÞ given in Eq. (13) is used for updating the model network’s weights VM in Eq. (14) during training by the backpropagation algorithm ^ eM ðtÞ ¼ DYðtÞ 2 DYðtÞ ¼ ½Diq ðtÞ 2 D^ıq ðtÞ; Dip ðtÞ 2 D^ıp ðtÞ; DVM ðtÞ ¼ 2hM eTM ðtÞ
›eM ðtÞ ›VM ðtÞ
ð13Þ ð14Þ
where hM is a positive learning rate and VM is the weights of the DHP model network. The training of the critic, action, and model networks in the DHP algorithm is carried out by different kinds of small pseudo-random binary signals (PRBSs) signals (Park et al., 2002), and the training procedures are explained in Park
4. Case studies 4.1. Dynamic performance for fault at receiving end To evaluate the damping performance of the proposed neurocontroller for the control of the SSSC, 100 and 120 ms three phase short circuits are applied to the infinite bus (receiving end) at t ¼ 1 s: The generator operates with a rotor angle of 53.68 ðPt ¼ 1:0 pu; Qt ¼ 0:59 puÞ in a steadystate operating point. The results are shown in Figs. 8– 12, where ‘Uncompensated’ and ‘CONVC’ denote the response of generator controlled without SSSC and with the PI controlled SSSC, respectively. From Fig. 8, it can be observed that the transmission line current is leads the compensating injection voltage vc by almost 908 (considering the re ) in steady-state such that the SSSC controlled by both the CONVC and DHPNC can establish the same effect as the series capacitive
Fig. 8. Line current ia (kA) and injected voltage vca (kV).
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 9. A 100 ms three phase short circuit test d (8).
Fig. 10. A 100 ms three phase short circuit test: Vt (pu).
Fig. 11. A 120 ms three phase short circuit test: d (8).
887
888
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 12. A 120 ms three phase short circuit test: Vt (pu).
compensation resulting in increasing the line current ðis Þ and transmitted power ðPÞ: From Figs. 9– 10, the DHPNC damping control is more effective compared to the CONVC. Also, it is clear from Figs. 11 and 12 that the generator controlled without the SSSC goes unstable and loses synchronism when the fault duration is 120 ms. In contrast, the DHPNC and CONVC restore the generator to a stable mode, and the DHPNC damping control is more effective compared to the CONVC, which means that the DHPNC allows the generator to be operated closer to its stability limit during steady state and thus would make it possible in a practical plant for the generation of more mega watts of power per dollar of invested capital.
4.2. Dynamic performances for single line-to-ground fault at machine terminal and multiple short circuits at both receiving and sending ends To further evaluate the damping performance of the DHPNC, a 150 ms single (a-phase) line-to-ground short circuit is applied at the generator terminals (the sending end in the SMIB system in Fig. 1) at t ¼ 1 s: The generator operates at the same steady-state operating condition as the previous test. The result in Fig. 13 shows that the DHPNC once gain has the best damping performance compared to the CONVC as well as the uncompensated. The same controllers are again evaluated for multiple different types of short circuits at different points. A 100 ms
Fig. 13. A 150 ms single (a-phase) line-to-ground short circuit fault occurs at t ¼ 1 s: d (8).
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
889
Fig. 14. Multiple short circuits test at both receiving and sending ends: d (8).
three phase short circuit is applied at the infinite bus at t ¼ 1 s: Meanwhile, a 150 ms single (a-phase) line-toground fault is applied at the generator terminals at t ¼ 1:05 s: The result in Fig. 14 shows that the CONVC and DHPNC restore the system to a stable mode, but that the uncompensated system goes unstable.
low frequency power oscillations and minimizing the terminal voltage variations. The use of fixed parameters in the DHPNC for real-time control not only has an important significance in terms of reducing the number of computations in dealing with the infinite optimal control problem by using the ANNs, but also proves robustness of the ACDs based controllers.
4.3. Selection of sampling rates during simulation Acknowledgements The numerical integration time step of PSCAD/EMTDC is 50 ms in order to obtain a switching frequency of 1980 Hz (33 times the fundamental) for the gate turn-off thyristors (GTOs). A sampling rate of 50 ms is used for the DHPNC because the outputs (^vcq and v^ cd in Fig. 2) of the DHPNC should be generated fast enough to compute the angle a and modulation index m to drive the GTOs of the inverter. Moreover, the simulation results with 10 times slower sampling rates (500 ms) do not show any severe degradation in performance of the DHPNC.
5. Conclusions This paper has proposed a novel DHP based design of a nonlinear optimal neurocontroller (DHPNC) for the internal control of a SSSC used in a transmission line of an electric power grid. The multiplayer perceptron neural network (MLPNN) is used as function approximator for the critic, action, and model networks to implement the DHP algorithm. The PSCAD/EMTDC simulation results show that the DHPNC has a better performance than the PI based conventional controller (CONVC) with respect to damping
Financial support by the National Science Foundation (NSF), USA under Grant No. ECS-0080764 for this research is gratefully acknowledged.
References Anderson, P. M., & Fouad, A. A. (1994). Power system control and stability. New York: IEEE Press. Bertsekas, D. P. (2001). Dynamic programming and optimal control. Belmont, MA: Athena Scientific. Dash, P. K., Mishra, S., & Panda, G. (2000). A radial basis function neural network controller for UPFC. IEEE Transaction on Power Systems, 15(4), 1293– 1299. Gyugyi, L., Schauder, C. D., & Sen, K. K. (1997). Static synchronous series compensator: a solid-state approach to the series compensation of transmission lines. IEEE Transaction on Power Delivery, 12(1), 406– 417. Hingorani, N. G., & Gyugyi, L. (2002). Understanding FACTS: concepts and technology of flexible AC transmission systems. New York: IEEE Press. Makombe, T., & Jenkins, N. (1999). Investigation of a unified power flow controller. IEEE Proceedings on Generation Transmission Distribution, 146(4), 400–408. Mohaddes, M., Gole, A. M., & McLaren, P. G. (1999). A neural network controlled optimal pulse-width modulated STATCOM. IEEE Transaction on Power Delivery, 14(2), 481 –488.
890
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
¨ ngquist, L., Ghandhari, M., & Andersson, G. (1997). Noroozian, M., A Improving power system dynamics by series-connected FACTS devices. IEEE Transaction on Power Delivery, 12(4), 1635–1641. Park, J.-W., Harley, R. G., & Venayagamoorthy, G. K. (2002). Adaptive critic based optimal neurocontrol for synchronous generator in power system using MLP/RBF neural networks. 37th IEEE Annual Industry Application Society (IAS) Meeting, 2, 1447–1454. Prokhorov, D. V., & Wunsch, D. C. (1997). Adaptive critic designs. IEEE Transaction on Neural Networks, 8(5), 997 –1007. Rigby, B. S., & Harley, R. G. (1998). An improved control scheme for a series-capacitive reactance compensator based on a voltage-source inverter. IEEE Transaction on Industry Applications, 34(2), 355–363. Schauder, C. D., et al. (1998). Operation of the unified power flow controller (UPFC) under practical constraints. IEEE Transaction on Power Delivery, 13(2), 630–639. Sen, K. K. (1998). SSSC: static synchronous series compensator: theory, modeling, and application. IEEE Transaction on Power Delivery, 13(1), 241– 246.
Si, J., & Wang, Y.-T. (2001). On-line learning control by association and reinforcement. IEEE Transaction on Neural Networks, 12(2), 264 –276. Venayagamoorthy, G. K., Harley, R. G., & Wunsch, D. C. (2002). Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator. IEEE Transaction on Neural Networks, 13(3), 764– 773. Venayagamoorthy, G. K., Harley, R. G., & Wunsch, D. C. (2003). Dual heuristic programming excitation neurocontrol for generators in a multimachine power system. IEEE Transaction on Industry Applications, 39(2). Werbos, P. J. (1992). Approximate dynamic programming for real-time control and neural modeling. In D. White, & D. Sofge (Eds.), Handbook of intelligent control (pp. 493 – 526). New York: Van Nostrand Reinhold. White, D. A., & Jordan, M. I. (1992). Optimal control: A foundation for intelligent control. In D. White, & D. Sofge (Eds.), Handbook of intelligent control (pp. 185 – 214). New York: Van Nostrand Reinhold.
2003 Special issue
New internal optimal neurocontrol for a series FACTS device in a power transmission line Jung-Wook Parka,*, Ronald G. Harleya, Ganesh K. Venayagamoorthyb b
a School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0250, USA Department of Electrical and Computer Engineering, University of Missouri-Rolla, Missouri-Rolla, MO 65409-0249, USA
Abstract In this paper, the proportional-integral (PI) based conventional internal controller (CONVC) of a power electronic based series compensator in an electric power system, is replaced by a nonlinear optimal controller based on the dual heuristic programming (DHP) optimization algorithm. The performance of the CONVC is compared with that of the DHP controller with respect to damping low frequency oscillations. Simulation results using the PSCAD/EMTDC software package are presented. q 2003 Elsevier Science Ltd. All rights reserved. Keywords: Neurocontroller; Dual heuristic programming; Power system
1. Introduction In the last decade, flexible AC transmission system (FACTS) devices (Hingorani & Gyugyi, 2002; Makombe & ¨ ngquist, Ghandhari, & AndersJenkins, 1999; Noroozian, A son, 1997; Schauder et al., 1998) have been progressively developed for controlling power flow in a power transmission system, improving the transient stability, damping power system oscillations, and providing voltage stability by using high power semiconductor technology based inverters connected to the electric power grid. Recently, the static synchronous series compensator (SSSC) (Gyugyi, Schauder, & Sen, 1997; Rigby & Harley, 1998; Sen, 1998) among the FACTS device family has attracted considerable attention for damping of low frequency power oscillations in the lines by using controllable series voltage compensation. The internal control strategy for the inverter of a SSSC as well as other FACTS devices has traditionally been based on linear proportional-integral (PI) regulators. These conventional PI linear controllers (CONVC) operate well at one particular operating point where they have been designed. In other words, their transient and dynamic performances are degraded at any other operating point, or * Corresponding author. E-mail addresses: [email protected] (J.-W. Park), rharley@ ece.gatech.edu (R.G. Harley), [email protected] (G.K. Venayagamoorthy).
their gains have to be re-tuned for the new operating points. Artificial neural networks (ANNs) can offer an efficient alternative to overcome the above limitation of the CONVC for the internal control. However, only a few researchers (Dash, Mishra, & Panda, 2000; Mohaddes, Gole, & McLaren, 1999) have reported on FACTS device control using ANNs in the literature due to the difficulty of implementing an effective and fast internal control of the inverter with the ANNs. The recently developed adaptive critic designs (ACDs) (Park, Harley, & Venayagamoorthy, 2002; Prokhorov & Wunsch, 1997; Si & Wang, 2001; Venayagamoorthy, Harley, & Wunsch, 2002; Werbos, 1992) based techniques avoid the possibility of instability of the neurocontroller, and also find an optimal control (Bertsekas, 2001; White & Jordan, 1992). This paper proposes a novel intelligent internal control approach using the multilayer perceptron neural network (MLPNN) as an optimal nonlinear neurocontroller. The background of the dual heuristic programming (DHP) algorithm, which has the best robust control capability among the ACDs family, is presented. Based on the DHP algorithm, a novel nonlinear optimal neurocontroller (called DHPNC) is designed as an alternative for internal control of the SSSC, thereby replacing the PI based regulators for the currents but not for the DC link voltage.
0893-6080/03/$ - see front matter q 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0893-6080(03)00121-7
882
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
The performances of the DHPNC and CONVC are compared with respect to the damping of power oscillations by time-domain simulations in the PSCAD/EMTDC software environment.
2. Static synchronous series compensator The SSSC converter can control the reactive and/or active power on an AC system by changing both phasor angle and magnitude of the converter’s output voltage with a fast control action. Especially, the exchange of active power, which is the particular characteristic of the SSSC, is accomplished by controlling the DC voltage inside the SSSC (Hingorani & Gyugyi, 2002). 2.1. Modeling of SSSC The single machine infinite bus (SMIB) system shown in Fig. 1 is used to compare the damping control capabilities of the proposed DHPNC and CONVC for the SSSC. The plant consists of the synchronous generator (160 MV A, 15 kV (L –L)), turbine – governor system, automatic voltage regulator (AVR) – exciter system, transmission line connected to an infinite bus, and the SSSC connected in series with transmission line. The parameters of the synchronous generators and transmission line are given in Anderson and Fouad (1994). The EXACIA (IEEE alternator supplied rectifier excitation systems) and H_TURI/GOV1 (IEEE type hydro turbine – governor) models in PSCAD/EMTDC software package are used as the AVR/exciter system and turbine/governor, respectively. For the mathematical model of the SSSC, the associated equation can be represented with the lumped series transmission line reactance x0e (transmission line xe plus leakage reactance of series-connected transformer) and transmission series resistance re (the inverter is
regarded simply to have no conduction losses) in per unit as follows 3 2 rv e s 2 0 0 2 3 6 72 3 x0e 7 isa 6 isa 7 6 re vs 76 7 7 6 d6 76 isb 7 6 isb 7 ¼ 6 0 2 0 0 74 5 6 4 5 xe dt 7 6 7 6 isc 4 re vs 5 isc 0 0 2 0 xe 3 2 vs ðvsa þ vca 2 vra Þ 7 6 x0e 7 6 7 6 vs 7 6 ð1Þ þ 6 0 ðvsb þ vcb 2 vrb Þ 7 7 6 xe 7 6 5 4 vs ðv þ v 2 v Þ sc cc rc x0e where vs is the synchronous speed of the power system, vs is the sending-end voltage (terminal voltage in practice), is is the current in transmission line, vr is the receiving-end voltage in the infinite bus, and vc is the injected series compensation voltage. Using the synchronously rotating reference frame based transformation (Makombe & Jenkins, 1999), in which the daxis is always coincident with the instantaneous voltage vector v and the q-axis leads the d-axis by 908, the threephase circuit equations in Eq. (1) can be transformed to the following d – q axis vector resprentation d dt
"
id iq
#
2
re v s x0e
3
" # 7 id 7 rv 5 i q 2v 2 e0 s xe 2 vs 3 ðlvs l þ vcd 2 vrd Þ 0 6 xe 7 7 þ6 4 5 vs ðv 2 v Þ cq rq 0 xe
6 ¼6 4
2
Fig. 1. Plant: 160 MV A, 15 kV (L–L) SMIB test system.
v
ð2Þ
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Neglecting the series inverter harmonics, the AC side injected voltage vc in Fig. 1 can be expressed with relation to the capacitor voltage Vdc on the DC link as follows vcd ¼ mVdc cosðaÞ;
vcq ¼ mVdc sinðaÞ
ð3Þ
where a is the phase voltage difference between the voltages vc and vs (the vc leads the vs ), and m is the modulation index of the series inverter. The dynamics of the DC capacitor voltage is given by dVdc 1 Pdc 1 Vdc idc 1 ðvcd id þ vcq iq Þ ¼ ¼ ¼ C Vdc C Vdc C dt Vdc ¼
1 ½m{cosðaÞid þ sinðaÞiq } C
ð4Þ
2.2. Conventional control strategy The main goal of the SSSC is to inject the series voltage in quadrature with the line current and to maintain the DC voltage Vdc : For this purpose, the P – Q (real and reactive power) automatic power flow control mode (Hingorani & Gyugyi, 2002) in Fig. 2 is used. In Fig. 2, an instantaneous three-phase set of line voltages, va ; vb ; and vc is used to calculate the transformation angle, u provided by the vector phase-locked loop for synchronous operation of the series voltage source inverter shown in Fig. 1. As shown in Eq. (2), the three-phase set of measured line currents at the AC terminal of the SSSC is decomposed into its real/direct component, id ; and reactive/quadrature.component, iq : These actual signals (id and iq ) and the reference d – q current signals (ipd and ipq ) are compared, respectively.
883
The error signal Diq for the reactive power exchange is passed through the PI regulator PI-iq : The signal Dip for the real power exchange and maintenance of a constant Vdc ; is passed through the PI-ip : The Dip consists of the Did and error signal DV 0dc (which passes through the PI-Vdc ). p p The Vdc is the desired value for Vdc : The value of Vdc is determined by selecting a suitable value of the modulation index within its range of 0 to 1. Finally, the estimated signals (~vcq and v~ cd ) in Fig. 2 are used to compute the angle a and modulation index m to drive the gate turn-off (GTO) thyristor of the inverter.
3. DHP based neurocontroller design ACDs proposed by Werbos (1992) are new optimization techniques to handle the nonlinear optimal control problem using ANNs. The adaptive critic method finds the optimal control in the infinite horizon problem to minimize/maximize the userdefined heuristic cost-to-go function J for a system, by successively adapting two ANNs, namely the action network (which dispenses the control signals) and the critic network (which learns to approximate the cost-to-go function J). This process is called the approximate dynamic programming (ADP) for the value iteration J: The adaptation process starts with a nonoptimal, arbitrarily chosen, control by the action network; the critic network then guides the action network towards the optimal solution at each successive adaptation. During the adaptations, neither of the networks needs any ‘information’ of an optimal trajectory, only the desired cost needs to be known (Prokhorov & Wunsch, 1997).
Fig. 2. P – Q automatic power flow control diagram for the internal control of the SSSC.
884
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 3. Critic network adaptation in DHP: This diagram shows the implementation of Eq. (8). The same critic network is shown for two consecutive times, t and t þ 1: The discount factor g is chosen to be 0.5. Backpropagation paths are shown by dotted and dash-dot lines. The output of the critic network l~ðt þ 1Þ is backpropagated through the model network from its outputs to its inputs, yielding the first term of Eq. (7) and l~Jðt þ 1Þ=›AðtÞ: The latter is backpropagated through the action network from its outputs to its inputs forming the second term of Eq. (7). Backpropagation of the vector ›UðtÞ=›AðtÞ through the action network results in a vector with components computed as the last term of Eq. (8). The summation of all these signals produces the error vector eC ðtÞ used for training the critic network.
The DHP technique (among the ACDs family) has the strong control capability in that the critic network of the DHP approximates the derivatives of the function J with respect to the states of the plant to be controlled. The DHP algorithm described in this paper uses three different multilayer (three layer) perceptron neural networks (MLPNNs), namely one for each of the critic, model, and action networks. The weight vector V of the MLPNN is adjusted/ trained using the gradient descent based backpropagation algorithm. By trial and error, 14 neurons are used in the hidden layer of the MLPNN for the model network, and 10 neurons for each of the critic and action networks.
The critic network estimates the derivatives of function J with respect to the vector of observables of the plant ~ ¼ (identified by the model network), which is the DYðtÞ ½D~ıq ðtÞ; D^ıp ðtÞ (input vector of the critic network), and it learns to minimize the following error measure over time kEc k ¼
X
eTC ðtÞeC ðtÞ
ð5Þ
t
3.1. Critic network As mentioned before, the DHPNC is designed to replace the PI regulators PI-iq and PI-ip in Fig. 2. The input reference vector Jref into the SSSC and output vector DJ from the SSSC are † Jref ðtÞ; input reference vector to the SSSC ¼ p ½ipp ðtÞ; ipd ðtÞ; Vdc ðtÞ: † DJðtÞ; output vector from the SSSC ¼ ½Diq ðtÞ; Dip ðtÞ: The configuration for the critic network adaptation in the DHP is shown in Fig. 3. The inputs and outputs of the action and model networks used in the critic network adaptation are shown in Figs. 4 and 5.
Fig. 4. Input–output mapping of the model network at t used in the critic network adaptation in Fig. 3.
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 5. Input–output mapping of the action network at t used in the critic network adaptation in Fig. 3.
eC ðtÞ ¼
^ ^ Yðt ^ þ 1Þ ›J½DYðtÞ ›J½D ›U½DYðtÞ 2g 2 ^ ›DYðtÞ ›DYðtÞ ›DYðtÞ
ð6Þ
After exploiting all relevant pathways of backpropagation as shown in Fig. 3, where the paths of derivatives and adaptation of the critic network are depicted by dotted and dash-dot lines, the error signal eC ðtÞ is used for training to update the weights of the critic network. The jth component of the second term in Eq. (6) can be expressed by the output ^ Yðt ^ þ 1Þ= of critic network at time t þ 1; l^i ðt þ 1Þ ¼ ›J½D ^ ›DYi ðt þ 1Þ as follows n m X n ^ Yðt ^ þ 1Þ X ^ ðt þ 1Þ X ›J½D ›Y ¼ l^i ðt þ 1Þ i þ l^i ðt þ 1Þ ›DYj ðtÞ › DY ðtÞ j i¼1 k¼1 i¼1
^ j ðt þ 1Þ ›A ðtÞ ›DY k ›Ak ðtÞ ›DYj ðtÞ
ð7Þ
^ ^ Yðt ^ þ 1Þ ›J½DYðtÞ ›J½D ›U½DYðtÞ 2g 2 ^ ›DYj ðtÞ ›DYj ðtÞ ›DYj ðtÞ 2
m X ›U½DYðtÞ ›Ak ðtÞ ›Ak ðtÞ ›DYj ðtÞ k¼1
›eC ðtÞ ›VC ðtÞ
3.2. Action network The adaptation of the action network in Fig. 3 is shown in Fig. 6, which propagates l^ðt þ 1Þ back through the model network to the action network. The goal of this adaptation is expressed in Eq. (10), and the weights of the action network are updated by Eq. (11) ^ Yðt ^ þ 1Þ ›U½DYðtÞ ›J½D þg ¼ 0; ;t: ð10Þ ›AðtÞ ›AðtÞ " # ^ Yðt ^ þ 1Þ T ›AðtÞ ›U½DYðtÞ ›J½D DVA ðtÞ ¼ 2hA þg ›AðtÞ ›AðtÞ ›VA ðtÞ where hA is a positive learning rate and VA contains the weights of the DHP action network. The discount factor g of 0.5 and the user-defined utility function UðtÞ in Eq. (12) are used in Eqs. (8) and (10) during adaptation of the critic and action networks UðtÞ ¼ ½Diq ðtÞ þ Diq ðt 2 1Þ þ Diq ðt 2 2Þ2 þ ½Dip ðtÞ þ Dip ðt 2 1Þ þ Dip ðt 2 2Þ2
ð8Þ
Using Eq. (8), the expression for the weights’ update for the critic network is as follows DVC ðtÞ ¼ 2hC eTC ðtÞ
Fig. 6. Action network adaptation in DHP: The discount factor g is chosen to be 0.5. Backpropagation paths are shown by dotted lines. The output of the critic network l^ðt þ 1Þ at time ðt þ 1Þ is backpropagated through the model network from its outputs to its inputs (output of the action network), and the resulting vector multiplied by the discount factor ðg ¼ 0:5Þ and added to ›UðtÞ=›AðtÞ: Then, an incremental adaptation of the action network is carried out by Eqs. (10) and (11).
ð11Þ
where n and m are the numbers of outputs of the model and the action networks, respectively. By using Eq. (7), each component of the vector eC ðtÞ from Eq. (6) is determined by eCj ðtÞ ¼
885
ð9Þ
where hC is a positive learning rate and VC contains the weights of the DHP critic network.
ð12Þ
3.3. Model network Fig. 7 shows how the model network (identifier) is trained to identify the dynamics of the plant in Fig. 1. The nonlinear autoregressive moving average with exogenous inputs (NARMAX) model is used as the structure for the online identification. The components of vectors Yref ðtÞ; D ^ YðtÞ; AðtÞ; and DYðtÞ are already noted in subsection A above (Figs. 4 and 5).
886
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
et al. (2002), Prokhorov & Wunsch (1997) and Venayagamoorthy, Harley, & Wunsch (2003). Note that the model network is trained before the training of the action and critic networks, and the DHPNC with the fixed (converged) weights for the critic and action networks is used to control the plant for real-time operation. In other words, they have been successfully trained to their optimization purposes, which are the value iteration (to minimize the value of derivatives of the function J) for the critic network and policy iteration (to solve the optimal control u in optimal policy) for the action network (Bertsekas, 2001).
Fig. 7. NARMAX model for on-line training of the model network.
The residual vector, eM ðtÞ given in Eq. (13) is used for updating the model network’s weights VM in Eq. (14) during training by the backpropagation algorithm ^ eM ðtÞ ¼ DYðtÞ 2 DYðtÞ ¼ ½Diq ðtÞ 2 D^ıq ðtÞ; Dip ðtÞ 2 D^ıp ðtÞ; DVM ðtÞ ¼ 2hM eTM ðtÞ
›eM ðtÞ ›VM ðtÞ
ð13Þ ð14Þ
where hM is a positive learning rate and VM is the weights of the DHP model network. The training of the critic, action, and model networks in the DHP algorithm is carried out by different kinds of small pseudo-random binary signals (PRBSs) signals (Park et al., 2002), and the training procedures are explained in Park
4. Case studies 4.1. Dynamic performance for fault at receiving end To evaluate the damping performance of the proposed neurocontroller for the control of the SSSC, 100 and 120 ms three phase short circuits are applied to the infinite bus (receiving end) at t ¼ 1 s: The generator operates with a rotor angle of 53.68 ðPt ¼ 1:0 pu; Qt ¼ 0:59 puÞ in a steadystate operating point. The results are shown in Figs. 8– 12, where ‘Uncompensated’ and ‘CONVC’ denote the response of generator controlled without SSSC and with the PI controlled SSSC, respectively. From Fig. 8, it can be observed that the transmission line current is leads the compensating injection voltage vc by almost 908 (considering the re ) in steady-state such that the SSSC controlled by both the CONVC and DHPNC can establish the same effect as the series capacitive
Fig. 8. Line current ia (kA) and injected voltage vca (kV).
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 9. A 100 ms three phase short circuit test d (8).
Fig. 10. A 100 ms three phase short circuit test: Vt (pu).
Fig. 11. A 120 ms three phase short circuit test: d (8).
887
888
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
Fig. 12. A 120 ms three phase short circuit test: Vt (pu).
compensation resulting in increasing the line current ðis Þ and transmitted power ðPÞ: From Figs. 9– 10, the DHPNC damping control is more effective compared to the CONVC. Also, it is clear from Figs. 11 and 12 that the generator controlled without the SSSC goes unstable and loses synchronism when the fault duration is 120 ms. In contrast, the DHPNC and CONVC restore the generator to a stable mode, and the DHPNC damping control is more effective compared to the CONVC, which means that the DHPNC allows the generator to be operated closer to its stability limit during steady state and thus would make it possible in a practical plant for the generation of more mega watts of power per dollar of invested capital.
4.2. Dynamic performances for single line-to-ground fault at machine terminal and multiple short circuits at both receiving and sending ends To further evaluate the damping performance of the DHPNC, a 150 ms single (a-phase) line-to-ground short circuit is applied at the generator terminals (the sending end in the SMIB system in Fig. 1) at t ¼ 1 s: The generator operates at the same steady-state operating condition as the previous test. The result in Fig. 13 shows that the DHPNC once gain has the best damping performance compared to the CONVC as well as the uncompensated. The same controllers are again evaluated for multiple different types of short circuits at different points. A 100 ms
Fig. 13. A 150 ms single (a-phase) line-to-ground short circuit fault occurs at t ¼ 1 s: d (8).
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
889
Fig. 14. Multiple short circuits test at both receiving and sending ends: d (8).
three phase short circuit is applied at the infinite bus at t ¼ 1 s: Meanwhile, a 150 ms single (a-phase) line-toground fault is applied at the generator terminals at t ¼ 1:05 s: The result in Fig. 14 shows that the CONVC and DHPNC restore the system to a stable mode, but that the uncompensated system goes unstable.
low frequency power oscillations and minimizing the terminal voltage variations. The use of fixed parameters in the DHPNC for real-time control not only has an important significance in terms of reducing the number of computations in dealing with the infinite optimal control problem by using the ANNs, but also proves robustness of the ACDs based controllers.
4.3. Selection of sampling rates during simulation Acknowledgements The numerical integration time step of PSCAD/EMTDC is 50 ms in order to obtain a switching frequency of 1980 Hz (33 times the fundamental) for the gate turn-off thyristors (GTOs). A sampling rate of 50 ms is used for the DHPNC because the outputs (^vcq and v^ cd in Fig. 2) of the DHPNC should be generated fast enough to compute the angle a and modulation index m to drive the GTOs of the inverter. Moreover, the simulation results with 10 times slower sampling rates (500 ms) do not show any severe degradation in performance of the DHPNC.
5. Conclusions This paper has proposed a novel DHP based design of a nonlinear optimal neurocontroller (DHPNC) for the internal control of a SSSC used in a transmission line of an electric power grid. The multiplayer perceptron neural network (MLPNN) is used as function approximator for the critic, action, and model networks to implement the DHP algorithm. The PSCAD/EMTDC simulation results show that the DHPNC has a better performance than the PI based conventional controller (CONVC) with respect to damping
Financial support by the National Science Foundation (NSF), USA under Grant No. ECS-0080764 for this research is gratefully acknowledged.
References Anderson, P. M., & Fouad, A. A. (1994). Power system control and stability. New York: IEEE Press. Bertsekas, D. P. (2001). Dynamic programming and optimal control. Belmont, MA: Athena Scientific. Dash, P. K., Mishra, S., & Panda, G. (2000). A radial basis function neural network controller for UPFC. IEEE Transaction on Power Systems, 15(4), 1293– 1299. Gyugyi, L., Schauder, C. D., & Sen, K. K. (1997). Static synchronous series compensator: a solid-state approach to the series compensation of transmission lines. IEEE Transaction on Power Delivery, 12(1), 406– 417. Hingorani, N. G., & Gyugyi, L. (2002). Understanding FACTS: concepts and technology of flexible AC transmission systems. New York: IEEE Press. Makombe, T., & Jenkins, N. (1999). Investigation of a unified power flow controller. IEEE Proceedings on Generation Transmission Distribution, 146(4), 400–408. Mohaddes, M., Gole, A. M., & McLaren, P. G. (1999). A neural network controlled optimal pulse-width modulated STATCOM. IEEE Transaction on Power Delivery, 14(2), 481 –488.
890
J.-W. Park et al. / Neural Networks 16 (2003) 881–890
¨ ngquist, L., Ghandhari, M., & Andersson, G. (1997). Noroozian, M., A Improving power system dynamics by series-connected FACTS devices. IEEE Transaction on Power Delivery, 12(4), 1635–1641. Park, J.-W., Harley, R. G., & Venayagamoorthy, G. K. (2002). Adaptive critic based optimal neurocontrol for synchronous generator in power system using MLP/RBF neural networks. 37th IEEE Annual Industry Application Society (IAS) Meeting, 2, 1447–1454. Prokhorov, D. V., & Wunsch, D. C. (1997). Adaptive critic designs. IEEE Transaction on Neural Networks, 8(5), 997 –1007. Rigby, B. S., & Harley, R. G. (1998). An improved control scheme for a series-capacitive reactance compensator based on a voltage-source inverter. IEEE Transaction on Industry Applications, 34(2), 355–363. Schauder, C. D., et al. (1998). Operation of the unified power flow controller (UPFC) under practical constraints. IEEE Transaction on Power Delivery, 13(2), 630–639. Sen, K. K. (1998). SSSC: static synchronous series compensator: theory, modeling, and application. IEEE Transaction on Power Delivery, 13(1), 241– 246.
Si, J., & Wang, Y.-T. (2001). On-line learning control by association and reinforcement. IEEE Transaction on Neural Networks, 12(2), 264 –276. Venayagamoorthy, G. K., Harley, R. G., & Wunsch, D. C. (2002). Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator. IEEE Transaction on Neural Networks, 13(3), 764– 773. Venayagamoorthy, G. K., Harley, R. G., & Wunsch, D. C. (2003). Dual heuristic programming excitation neurocontrol for generators in a multimachine power system. IEEE Transaction on Industry Applications, 39(2). Werbos, P. J. (1992). Approximate dynamic programming for real-time control and neural modeling. In D. White, & D. Sofge (Eds.), Handbook of intelligent control (pp. 493 – 526). New York: Van Nostrand Reinhold. White, D. A., & Jordan, M. I. (1992). Optimal control: A foundation for intelligent control. In D. White, & D. Sofge (Eds.), Handbook of intelligent control (pp. 185 – 214). New York: Van Nostrand Reinhold.