- Email: [email protected]
Sensor scheduling in continuous time
Automatica 37 (2001) 2017}2023
Brief Paper
Sensor scheduling in continuous time夽 H.W.J. Lee *, K.L. Teo, Andrew E.B. Lim Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Department of Applied Mathematics and Centre for Multimedia Signal Processing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Department of Industrial Engineering and Operations Research, Columbia University, New York, USA Received 21 December 1999; revised 27 September 2000; received in "nal form 6 June 2001
Abstract Let N be the number of available sensor sources. Noisy observations of an underlying state process are available for these N sources. We consider the continuous time sensor scheduling problem in which N of these N sources are to be chosen to collect data at each time point. This sensor scheduling problem (with switching costs and switching constraints) is formulated as a constrained optimal control problem. In this framework, the controls represent the sensors that are chosen at a particular time. Thus, the control variables are constrained to take values in a discrete set, and switchings between sensors can occur in continuous time. By incorporating recent results on discrete valued optimal control, we show that this problem can be transformed into an equivalent continuous optimal control problem. In this way, we obtain the sensor scheduling policy as well as the associated switching times. 2001 Elsevier Science Ltd. All rights reserved. Keywords: Optimal control; Parametrization; Scheduling algorithms; Sensors; Stochastic systems; Switching times
1. Introduction Sensor scheduling is motivated by the need for a systematic approach to the problem of data acquisition. In many applications, data are available from a number of sources. Once obtained, it can be incorporated in a signal processing scheme (in this case, the Kalman "lter) to provide estimates of an underlying state process that is of interest. On the other hand, due to factors such as limits on computing power, the cost of obtaining data, or constraints associated with the data sources, many applications allow only a small number of data sources to be engaged in operation at any time point. Thus, an appropriate scheduling scheme is needed for choosing these few data sources among all available data sources at each time in an optimal way. This task is referred to as the sensor scheduling problem in continuous time.
夽 This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Patrizio Colaneri under the direction of Editor Roberto Tempo. * Corresponding author. E-mail addresses: [email protected] (H.W.J. Lee), mateokl@polyu. edu.hk (K.L. Teo), [email protected] (A.E.B. Lim).
One of the major di$culties in the sensor scheduling problem is the combinatorial nature of the problem. These problems are compounded further if sensor switching is allowed to take place in continuous time. In Meier, Peschon, and Dressler (1967), the problem of sensor selection in discrete time is solved by complete enumeration of all scheduling policies. However, since the number of policies grows exponentially with the number of allowable switching points, this approach is only applicable in the discrete time setting (with a small number of time points) and is not easily extendable to continuous time. Even then, the complexity of the problem grows quickly with the number of sensors, and for this reason, it can only be used for relatively small problems. Furthermore, it is di$cult to incorporate additional constraints. In this paper, we consider the sensor scheduling problem in continuous time as an optimal discrete-valued control problem. In this formulation, the control variables (which represent the chosen sensor combination at each time) is constrained to take values in a discrete set. We consider a cost function that contains terms associated with estimation error as well as switching costs. In addition, we allow for the possibility of there being constraints on sensor switching. Our solution of the sensor
0005-1098/01/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 5 - 1 0 9 8 ( 0 1 ) 0 0 1 5 9 - 5
2018
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
scheduling problem makes use of some recently obtained results on discrete-valued optimal control problems. By introducing a certain transformation, it is shown in Lee, Teo, Rehbock, and Jennings (1999) and Teo, Jennings, Lee, and Rehbock (1999) that discrete valued control problems with variable switching times can be transformed into equivalent continuous valued optimal control problems. These continuous optimal control problems can then be solved using readily available optimal control techniques (c.f. Teo et al., 1999; Teo, Goh, & Wong, 1991 and relevant reference cited therein), and consequently the optimal control software package mentioned in Jennings, Fisher, Teo, and Goh (1997). The optimal control (and the associated switching times) for the original problem can be obtained by a reverse transformation. Thus, we are able to determine the scheduling policy as well as the associated switching times for the sensor scheduling problem. A continuous time formulation of the sensor scheduling problem, which includes switching cost considerations, is also considered in Baras and Bensoussan (1989). They show that the optimal scheduling policy can be obtained by solving a quasi-variational inequality. One important di!erence between the formulation in Baras and Bensoussan (1989) and the one presented in this paper is the class of admissible scheduling policies. In Baras and Bensoussan (1989), scheduling policies are processes adapted to the observation -algebra. In our approach, we restrict ourselves (as in Meier et al., 1967) to open-loop policies with switchings being allowed to take place in continuous time. For this special situation, a complete solution can be obtained. On the other hand, the general situation considered in Baras and Bensoussan (1989) is much too complex for computing optimal solution.
2. Problem formulation: optimal sensor scheduling problem Let (, F, P) be a given probability space. Consider a system governed by the following stochastic di!erential equation: dx(t)"A(t)x(t) dt#B(t) d*(t), t*0,
(1.1)
x(0)"x ,
(1.2)
where for each t*0, A(t)31L"L and B(t)31L"N. Elements of A(t) and B(t) are uniformly bounded measurable functions. The initial state x is a 1L-valued Gaussian random vector on (, F, P) with mean E(x )"x and covariance E(x0 !x )(x0 !x )?"P . The process *(t) is an 1N-valued Brownian motion on (, F, P) with covariance E(*(t)!*(s))(*(t)!*(s))?"Q(t!s). We as-
sume that Q is a symmetric, positive semi-de"nite p;p matrix. The unique strong solution x(t) is an 1L-valued square integrable process which we refer to as the state process. Suppose that there are N active sensors. Each sensor makes noisy observations of the state process x(t). At every t, observations from N 31,2, N of these sen sors can be used to construct an estimate x( (t) of the state x(t). For the sake of simplicity, we assume that N "1. There is no essential di$culty in extending these results to the case of N '1. For details, see Remark 2. A sensor schedule can be represented by a function u : [0, ¹]P"1,2, N. In particular, u(t)"i means that the sensor i is used at time t. Hence, the set of admissible sensor schedules is given by the set U"u : [0, ¹]P u( ) ) is measurable. We refer to any u( ) )3U as an admissible sensor schedule. For any u( ) )3U, we have the following output equation: , dy(t)" (t)[C (t)x(t) dt#D (t) dw (t)], SRG G G G G t*0,
(2.1)
y(0)"0,
(2.2)
where
1, (t)" SRG 0,
u(t)"i, u(t)Oi.
For each t*0, C (t)31K"L and D (t)31K"I. Each eleG G ment of the matrix-valued functions C (t) and D (t) are G G uniformly bounded measurable functions. The processes w (t) are 1I-valued Brownian motions on (, F, P) with G covariance E(w (t)!w (s))(w (t)!w (s))?"R (t!s). We G G G G G assume that R is a k;k symmetric and positive de"nite G matrix. We assume throughout that D (t)R D?(t)'0 G G G (and hence, invertible). Furthermore, we assume that x , *(t), and w (t),w (t),2,w (t) are mutually independent. , Under these assumptions, there is a unique strong solution y(t) of (2), which we refer to as the observation process associated with the scheduling policy u( ) )3U. Once u( ) ) has been chosen, the optimal mean-square estimate of the state is given by the well-known Kalman "lter. In particular, let u( ) )3U be given and let FW"y(s), R 0)s)t be the -algebra generated by the observation process y(t) associated with u( ) ). The optimal meansquare estimate of x(t) given FW is x( (t)"Ex(t) FW and R R the associated error covariance is P(t)"E(x(t)! ? x( (t))(x(t)!x( (t)) . For any u( ) )3U, x( (t) and P(t) are given as follows (see Ahmed, 1998; Anderson, & Moore, 1990):
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
Theorem 1. For a given sensor schedule u( ) )3U, the optimal mean-square estimate x( (t) is the unique strong solution of the stochastic diwerential equation
, dx( (t)" A(t)!P(t) C?(t)RI i\(t)C (t) x( (t) dt SRG G G G , # P(t) C?(t)RI i\(t) dy(t), (3.1) SRG G G x( (0)"x , (3.2) ? where RI i\(t)"[D (t)R D (t)]\, and the error covariance G G G matrix P(t) is the unique solution of the matrix Riccati diwerential equation
PQ (t)"A(t)P(t)#P(t)A?(t)#B(t)QB?(t) , ! P(t)C?(t)RI i\(t)C (t)P(t), G SRG G G P(0)"P .
By adopting the recently developed control parametrization enhancing transform (CPET) Technique (Lee et al., 1999; Teo et al., 1999), we develop a computational procedure for solving Problem (P). We introduce the new time scale s which varies from 0 to M. Let V denote the class of non-negative piecewise constant scalar functions de"ned on [0, M) with "xed interior knot points located at 1, 2, 3,2, M!1. Note that we let M" N, where is the assumed maximum number of times any sensor i3 is being selected. The transformation (CPET) from t3[0, ¹] to s3[0, M] is de"ned by the di!erential equation
(4.1) (4.2)
where the scalar function v(s)3V is called the enhancing control, and it satis"es
Problem. Subject to (4), "nd an admissible sensor schedule uH( ) )3U so that (5)
2 J(u( ) ))" traceP(t) dt#traceP(¹). (6) The optimal sensor scheduling problem (5) is an optimal discrete-valued control problem, where the system dynamics and the objective function are, respectively, given by (4) and (6). The main di$culty associated with (5) is that the control u( ) ) is constrained to take values in a discrete set 1,2,2, N. In addition, switchings may occur at any time in the time horizon [0, ¹]. This problem is referred to as Problem (P). Remark 2. The proposed method can be extended to the general case of N '1 without much di$culty. The only change we need to make is the de"nition of . More speci"cally, for N '1, there are altogether N ways of turning-on N sensors from a total of N sensors, where N N! " . N " N !(N!N )! N Now, de"ne "1, 2,2, N . The meaning of each i3 is to be understood as one of these sensors combinations, instead of just one particular sensor in the case of N "1.
3. A computational procedure
dt "v(s) (with initial condition t"0 when s"0), (7) ds
Clearly, P(t) depends on the sensor schedule u( ) )3U that is chosen and the estimation error can be reduced by choosing u( ) ) so as to make P(t) &small'. This motivates the sensor scheduling problem, which we state as follows:
J(uH( ) ))" inf J(u( ) )), S ZU 2
where
2019
+
v(s) ds"¹.
Moreover, we introduce a "xed function (s): (s),(i mod N), s3[i, i#1), i"0, 1, 2,2, M!1. (8) Note that the idea of this CPET transformation is to let any u(t)3U be naturally represented by a v(s)3V whenever this "xed (s) is de"ned. For example, if we have N"4, ¹"1, and
1, t3[0,0.3),
u(t)" 3, t3[0.3,0.7), 2, t3[0.7,1),
then, we can naturally "nd a representation v(s)3V given by
v(s)"
0.3, 0, 0.4, 0, 0, 0.3, 0, 0, 0,
s3[0,1) (corresponds s3[1, 2) (corresponds s3[2,3) (corresponds s3[3,4) (corresponds s3[4,5) (corresponds s3[5,6) (corresponds s3[6,7) (corresponds s3[7,8) (corresponds s3M!1, M).
to to to to to to to to
"1), "2), "3), "4), "1), "2), "3), "4),
The transformed problem can thus be stated as follows: Subject to the system equations
d P(s)"v(s) A(s)P(s)#P(s)A?(s)#B(s)QB?(s) ds
, ! P(s)C?(s)RI i\(t)C (s)P(s) , G IQG G G P(0)"P ,
(9.1) (9.2)
2020
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
"nd a v3V such that the cost
J(v( ) ))"
+
Consider the function (10)
is minimized over V subject to the equality constraint
+
v(0,
1,
traceP(s)v(s) ds#traceP(M)
v(s) ds"¹.
(11)
Let this problem be referred to as Problem (TP). It is clear from Lee et al. (1999) and Teo et al. (1999) that Problem (TP) is an optimal control problem in canonical form. Furthermore, all the possible discontinuities for any v( ) )3V are pre-assigned at 1, 2,2, M. From the results obtained in Lee et al. (1999) and Teo et al. (1999), the following theorem is obtained: Theorem 3. Suppose the maximum number of times any sensor i3 is being selected, or `turned-on , equals . If M* N, Problem (TP) and Problem (P) are equivalent. Proof. See the appendix. 䊐 Remark 4. By examining the proof of Theorem 3 given in the appendix, we note that if M is large enough to allow switchings to be more than or equal to what the optimal solution requires, then V contains the optimal solution.
4. Switching penalization It is of practical importance to associate a cost with the switching of the sensors. This cost may be just the wear and tear on the switching mechanism, or it may be the risk cost of revealing the existence of the observer (e.g. active sensors in submarine detection). In this section, we propose a way to penalize the switchings. Observe that when s3[i, i#1), v(s) represents the time length of which sensor (i mod N) is selected, or `turnedona. If the value of v(s) is `negligibly smalla, or only `slightly greater than zeroa for some s, then we consider that the sensor is not `turned-ona, and hence, does not contribute to the total number of switchings. To quantify this convention, let '0 denote a small prespeci"ed constant, which is referred to as the vanishing tolerance level. Then we adopt the convention of rounding o! the value of v(s) to zero if v(s)) . Thus, for a given v3V such that (11) is satis"ed, we de"ne (v( ) )) as the C total number of pieces v(s)* , s3[i, i#1), over i"0, 1, 2,2, M!1.
h , 0)v( , uB(v)" h , )v(2 , 0, v*2 , where 0( (0.5 and
(v! ) 2(v! ) h "10 !0.05 ! ! #1 #0.1 , h "10 !0.05
(v! ) 2(v! ) ! #1 #0.1 .
This function has the following interesting properties: (i) The derivative of uB( ) ) with respect to its argument exists everywhere. (ii) The discontinuities of uB(v( ) )) are all located at 1, 2,2, M!1. (iii) For a given v3V such that (11) is satis"ed and '0 a prespeci"ed vanishing tolerance level, there exists a '0 such that (v( ) ))) +uB(v(s)) ds)M. C The "rst two properties are clear, while the proof of the third property is given in the appendix. Clearly, +uB(v(s)) ds approaches (v( ) )) from above. C Thus, +uB(v(s)) ds can be used to penalize any unnecess ary number of switchings. Our optimal sensor scheduling problem with a penalty on the number of switchings can thus be stated as: Subject to the system dynamics (9) and the equality constraint (11), "nd a v3V such that the cost
J(v( ) ))"
+
[traceP(s)v(s)#uB(v(s))] ds # traceP(M)
(12)
is minimized over V, where is a weighting. This is a standard optimal control problem in canonical form (Fig. 1).
Fig. 1. The plot uB(v).
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
5. Lower bounds on the turned-on times
(v(s)!)uB(v(s)) (s)*0. IQG
!1.1725
In addition to the above formulation, one practically meaningful constraint we would like to consider is `if sensor i3 is selected, then it must be used for at least units of timea. We assume throughout that ' where is a prespeci"ed vanishing tolerance level. Then, by the de"nition of the function uB(v(s)) given in Section 4, we consider, for a given i, the all-time constraint: (13)
The possible discontinuities of the all-time constraint (13) are located at 1, 2,2, M!1. Proposition 5. Let v3V be such that (11) is satisxed. Suppose (13) is satisxed for some given i31, 2,2, N. Then, for any '0, where is a prespecixed vanishing tolerance level, there exists a '0 such that either v(s)* or v(s)( whenever (s)"i. Proof. See the appendix. 䊐 Since all switchings are located at the integer knots, (13) is already in a form readily solvable by the techniques developed in Teo et al. (1991), and thus by MISER3.2 (Jennings et al., 1997). The central idea behind the software MISER3 is the concept of control parametrization. Essentially, this procedure approximates the control functions by a linear sum of basis functions, chosen for e$cient computation and reality of control approximation. These basis functions are the piecewise constant functions on "nite support or the piecewise linear continuous functions on "nite support (the witches' hat functions). Thus, by appropriate reformulation, the optimal control problem is transformed into a nonlinearly constrained mathematical programming problem, and hence is solvable by existing e$cient software packages for mathematical programming problems. See Teo et al. (1991) for details on the control parametrization technique. The control parametrization enhancing transform (CPET) is introduced in Teo et al. (1999) to enhance the classical control parametrization technique. This transform involves the introduction of an additional piecewise constant function, mapping the switching times on to a set of integer knots in a new time scale. Hence, the transformed optimal control problem can be handled by the usual control parametrization technique.
6. An illustrative example Consider a system with n"5, m"1, p"5, N"10 (10 sensors), ¹"1,
2021
9.3567 !2.0278
A" !1.6009 !6.2658
3.9829 !9.7397 !4.9574 !1.7843 6.7470
1.5886 !7.7981
9.8620 !1.2604 !7.0801
1.3012 ,
3.6903 !3.1387 !1.6332 !1.2068
!3.0191 !8.3465
1 1 1 1 1 1 1 1 1 1
1.1865
!1.1452 !2.0959
10
0
0
0
0
0 10
0
0
0
0 10
0
0 ,
Q" 1 1 1 1 1 , P " 0 1 1 1 1 1 0 1 1 1 1 1
0
0
0 10
0
0
0
0 10
1 0 0 0 0 0 1 0 0 0
B" 0 0 1 0 0 , 0 0 0 1 0 0 0 0 0 1
C "[1 1 0 0 0], D "1, R "1.0, C "[1 0 1 0 0], D "1, R "2.25, C "[1 0 0 1 0], D "1, R "2.56, C "[1 0 0 0 1], D "1, R "3.4225, C "[0 1 1 0 0], D "1, R "4.0, C "[0 1 0 1 0], D "1, R "4.0, C "[0 1 0 0 1], D "1, R "3.4225, C "[0 0 1 1 0], D "1, R "2.56, C "[0 0 1 0 1], D "1, R "2.25, C "[0 0 0 1 1], D "1, R "1.0. There is a lower bound of the `turned-ona time constraint imposed on sensor C1 where "0.5. In this example, "1, and we choose "0.005, M"80. The optimal control software package MISER3.2 is then used to solve the corresponding version of the transformed problem, i.e. Problem (TP). It was run on a PC booted with LINUX, having a CPU speed of 300 MHz and equipped with 128 MB of RAM. It takes a total of 298 iterations to converge, where each iteration takes roughly 2 s. Relevant results obtained are shown in Figs. 2 and 3. Note that although there are 10 sensors, only three of them have been turned `ona in the converged optimal solution, namely sensor C1, C8 and C9. In particular, sensor C1 is `ona for 0.5 unit of time, hitting its lower
2022
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
Appendix. Proof of Theorem 3.1 Let v 3V be any enhancing control for Problem (TP) , 0)s(1, , 1)s(2, v (s)" , 2)s(3,
Fig. 2. The plot of all 5 diagonal elements of P(t).
, M!1)s(M, + where *0 for j"1, 2,2, M. The corresponding H u (t)3U for Problem (P) in the original time scale t is uniquely given by
1 mod N,
0)t( , 2 mod N, )t( # , u (t)" 3 mod N, # )t( # # , +\ + M mod N, )t( "¹. G G G G Clearly, J(u ) of (10) equals J(v ) of (6). Now, we let u 3U be any sensor schedule for Problem (P):
Fig. 3. The plot of u(t).
bound of the `turned-ona time. Seven other sensors are not used in this optimal solution. The converged cost is 23.797.
7. Conclusions We proposed a computational scheme to solve the optimal sensor scheduling problem. We have overcome the di$culties of computing the exact switching times, and the combinatorial nature of the scheduling problem. Moreover, a novel technique to penalize the number of switchings is developed, and a lower bound of the `turned-ona time constraint for any particular sensor combination is also considered. An illustrative example is provided to demonstrate our proposed method.
Acknowledgements The Research Committee of The Hong Kong Polytechnic University is acknowledged. Moreover, the authors are grateful to the Associate Editor and the 2 anonymous referees for their valuable comments and suggestions which have helped to improve the paper.
i , i , u (t)" i ,
0)t(t , t )t(t , t )t(t ,
i , t )t(t "¹, ( (\ ( where i , i ,2, i 31, 2,2, N, is the total number of ( `piecesa, and it is bounded above by N. Construct
t , t !t , t !t , v (s)" ¹!t , ( 0,
s3[0;N#i !1,0;N#i ), s3[1;N#i !1,1;N#i ), s3[2;N#i !1, 2;N#i ), s3[(!1);N#i !1, ( (!1);N#i ), ( otherwise.
Clearly, v (s)3V for Problem (TP) is corresponding to u (t) for Problem (P) in the transformed time scale s. (Note that this correspondence is not unique in general). The largest possible value for i is N. A su$cient condi( tion for the existence of such a v (s) is when M*(!1)N#N"N" N. Under this su$cient condition, J(v ) of (6) equals J(u ) of (10). 䊐 Proof of Property (iii) of uB in Section 4. Take ) /2, we have (v( ) ))) +uB(v(s)) ds. Clearly, +uB(v(s)) ds"M if C and only if v(s)*2 for all s3[0, M]. On the other hand,
+uB(v(s)) ds(M. 䊐
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
Proof of Proposition 5.1. For those s such that (s)"i, (13) is reduced to (v(s)!)uB(v(s))*0. For any '0, where is a prespeci"ed vanishing tolerance level, there exist a '0 such that uB( )" . There are two possibilities, either v(s)( , or v(s)* . If v(s)( , then uB(v(s))( (they are rounded o! to zero by the convention mentioned in Section 4). On the other hand, if v(s)* , then uB(v(s))* . Thus (v(s)!)uB(v(s))*0 implies (v(s)!)*0. Hence, v(s)*. 䊐 References Ahmed, N. U. (1998). Linear and nonlinear xltering for scientists and engineers. Singapore: World Scienti"c. Anderson, B. D. O., & Moore, J. B. (1990). Optimal control*linear quadratic methods. Information and system science series. Englewood Cli!s, NJ: Prentice-Hall. Baras, J. S., & Bensoussan, A. (1989). Optimal sensor scheduling in nonlinear "ltering of di!usion process. SIAM Journal of Control and Optimization, 27(4), 786}813. Jennings, L. S., Fisher, M. E., Teo, K. L., & Goh, C. J. (1997). MISER3 optimal control software (Version 2): Theory and user manual. Centre for Applied Dynamics Optimization. The University of Western Australia, Perth, Australia. Lee, H. W. J., Teo, K. L., Rehbock, V., & Jennings, L. S. (1999). Control parametrization enhancing technique for optimal discrete-valued control problems. Automatica, 35, 8. Teo, K. L., Goh, C. J., & Wong, K. H. (1991). A unixed computational approach to optimal control problems. Marlow, UK: Longman Scienti"c and Technical. Teo, K. L., Jennings, L. S., Lee, H. W. J., & Rehbock, V. (1999). The control parametrization enhancing transform for constrained optimal control problems. Journal of the Australian Mathematical Society, Series B, 40(Part 3), 314}335. Meier, L., Peschon, J., & Dressler, R. M. (1967). Optimal control of measurement subsystems. IEEE Transactions on Automatic Control, AC-12(5), 528}536. Andrew Lim was born in Penang, Malaysia, in 1973. He obtained his undergraduate degree in Mathematics from the University of Western Australia in 1995, and his Ph.D. in Systems Engineering from the Australian National University in 1998. He has held research positions at the Chinese University of Hong Kong, Columbia University (New York), and the University of Maryland (College Park). Currently, he is a visiting Assistant Professor in the Department of Industrial Engineering and Operations Research at Columbia University. His research interests are in the areas of optimization, stochastic control, backward
2023
stochastic di!erential equations, and Markov decision processes, with applications to problems in operations research, engineering and "nance.
Heung Wing Joseph Lee was born in Hong Kong in December 1969. He received his B.Sc.(1st Hons) in 1994 and Ph.D. (with distinction) in 1997, both at the Department of Mathematics, The University of Western Australia. In 1997}1998, he was working as a Postdoctoral Research Fellow at the Department of System Engineering and Engineering Management, The Chinese University of Hong Kong. He worked as a Visiting Assistant Professor at the Department of Mathematics, The Hong Kong University of Science and Technology in 1998}1999. He is currently an Assistant Professor in the Department of Applied Mathematics at The Hong Kong Polytechnic University. His research interests include Computational Optimal Control, Mixed Discrete Programming, Scheduling, and Control of Chaotic Systems.
Kok Lay Teo received the Ph.D. degree in Electrical Engineering from the University of Ottawa, Canada in 1974. He was with the Department of Applied Mathematics, the University of New South Wales, Australia, from 1974 to 1985, the Department of Industrial and Systems Engineering, National University of Singapore, Singapore, from 1985 to 1987, and the Department of Mathematics, the University of Western Australia, Australia, from 1988 to 1996. He was Professor of Applied Mathematics at Curtin University of Technology, Australia, from 1996 to 1998. He is currently the Chair Professor of Applied Mathematics and Head of Department of Applied Mathematics at The Hong Kong Polytechnic University, Hong Kong. He has also been an Adjunct Professor of the University of Waterloo, Canada, from May 2001, and Chongqing Normal University, China, from September 1999. He has published 5 books, edited several books and special issues for international journals. He has also published numerous journal and conference papers in areas, which include control, optimal control, optimization, signal processing, and biomachine. The software package, MISER3.2, for solving general constrained optimal control problems was developed by the research team under his leadership. His research interests includes both the theoretical and practical aspects of optimal control and optimization, and their applications, in particular, signal processing in telecommunications. Dr. Teo is a Senior Member of Institute of Electrical and Electronic Engineers (IEEE) (USA), and is in the editorial board of a number of international journals in control, dynamical systems and optimization, including Automatica and Journal of Global Optimization.
Brief Paper
Sensor scheduling in continuous time夽 H.W.J. Lee *, K.L. Teo, Andrew E.B. Lim Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Department of Applied Mathematics and Centre for Multimedia Signal Processing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Department of Industrial Engineering and Operations Research, Columbia University, New York, USA Received 21 December 1999; revised 27 September 2000; received in "nal form 6 June 2001
Abstract Let N be the number of available sensor sources. Noisy observations of an underlying state process are available for these N sources. We consider the continuous time sensor scheduling problem in which N of these N sources are to be chosen to collect data at each time point. This sensor scheduling problem (with switching costs and switching constraints) is formulated as a constrained optimal control problem. In this framework, the controls represent the sensors that are chosen at a particular time. Thus, the control variables are constrained to take values in a discrete set, and switchings between sensors can occur in continuous time. By incorporating recent results on discrete valued optimal control, we show that this problem can be transformed into an equivalent continuous optimal control problem. In this way, we obtain the sensor scheduling policy as well as the associated switching times. 2001 Elsevier Science Ltd. All rights reserved. Keywords: Optimal control; Parametrization; Scheduling algorithms; Sensors; Stochastic systems; Switching times
1. Introduction Sensor scheduling is motivated by the need for a systematic approach to the problem of data acquisition. In many applications, data are available from a number of sources. Once obtained, it can be incorporated in a signal processing scheme (in this case, the Kalman "lter) to provide estimates of an underlying state process that is of interest. On the other hand, due to factors such as limits on computing power, the cost of obtaining data, or constraints associated with the data sources, many applications allow only a small number of data sources to be engaged in operation at any time point. Thus, an appropriate scheduling scheme is needed for choosing these few data sources among all available data sources at each time in an optimal way. This task is referred to as the sensor scheduling problem in continuous time.
夽 This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Patrizio Colaneri under the direction of Editor Roberto Tempo. * Corresponding author. E-mail addresses: [email protected] (H.W.J. Lee), mateokl@polyu. edu.hk (K.L. Teo), [email protected] (A.E.B. Lim).
One of the major di$culties in the sensor scheduling problem is the combinatorial nature of the problem. These problems are compounded further if sensor switching is allowed to take place in continuous time. In Meier, Peschon, and Dressler (1967), the problem of sensor selection in discrete time is solved by complete enumeration of all scheduling policies. However, since the number of policies grows exponentially with the number of allowable switching points, this approach is only applicable in the discrete time setting (with a small number of time points) and is not easily extendable to continuous time. Even then, the complexity of the problem grows quickly with the number of sensors, and for this reason, it can only be used for relatively small problems. Furthermore, it is di$cult to incorporate additional constraints. In this paper, we consider the sensor scheduling problem in continuous time as an optimal discrete-valued control problem. In this formulation, the control variables (which represent the chosen sensor combination at each time) is constrained to take values in a discrete set. We consider a cost function that contains terms associated with estimation error as well as switching costs. In addition, we allow for the possibility of there being constraints on sensor switching. Our solution of the sensor
0005-1098/01/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 5 - 1 0 9 8 ( 0 1 ) 0 0 1 5 9 - 5
2018
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
scheduling problem makes use of some recently obtained results on discrete-valued optimal control problems. By introducing a certain transformation, it is shown in Lee, Teo, Rehbock, and Jennings (1999) and Teo, Jennings, Lee, and Rehbock (1999) that discrete valued control problems with variable switching times can be transformed into equivalent continuous valued optimal control problems. These continuous optimal control problems can then be solved using readily available optimal control techniques (c.f. Teo et al., 1999; Teo, Goh, & Wong, 1991 and relevant reference cited therein), and consequently the optimal control software package mentioned in Jennings, Fisher, Teo, and Goh (1997). The optimal control (and the associated switching times) for the original problem can be obtained by a reverse transformation. Thus, we are able to determine the scheduling policy as well as the associated switching times for the sensor scheduling problem. A continuous time formulation of the sensor scheduling problem, which includes switching cost considerations, is also considered in Baras and Bensoussan (1989). They show that the optimal scheduling policy can be obtained by solving a quasi-variational inequality. One important di!erence between the formulation in Baras and Bensoussan (1989) and the one presented in this paper is the class of admissible scheduling policies. In Baras and Bensoussan (1989), scheduling policies are processes adapted to the observation -algebra. In our approach, we restrict ourselves (as in Meier et al., 1967) to open-loop policies with switchings being allowed to take place in continuous time. For this special situation, a complete solution can be obtained. On the other hand, the general situation considered in Baras and Bensoussan (1989) is much too complex for computing optimal solution.
2. Problem formulation: optimal sensor scheduling problem Let (, F, P) be a given probability space. Consider a system governed by the following stochastic di!erential equation: dx(t)"A(t)x(t) dt#B(t) d*(t), t*0,
(1.1)
x(0)"x ,
(1.2)
where for each t*0, A(t)31L"L and B(t)31L"N. Elements of A(t) and B(t) are uniformly bounded measurable functions. The initial state x is a 1L-valued Gaussian random vector on (, F, P) with mean E(x )"x and covariance E(x0 !x )(x0 !x )?"P . The process *(t) is an 1N-valued Brownian motion on (, F, P) with covariance E(*(t)!*(s))(*(t)!*(s))?"Q(t!s). We as-
sume that Q is a symmetric, positive semi-de"nite p;p matrix. The unique strong solution x(t) is an 1L-valued square integrable process which we refer to as the state process. Suppose that there are N active sensors. Each sensor makes noisy observations of the state process x(t). At every t, observations from N 31,2, N of these sen sors can be used to construct an estimate x( (t) of the state x(t). For the sake of simplicity, we assume that N "1. There is no essential di$culty in extending these results to the case of N '1. For details, see Remark 2. A sensor schedule can be represented by a function u : [0, ¹]P"1,2, N. In particular, u(t)"i means that the sensor i is used at time t. Hence, the set of admissible sensor schedules is given by the set U"u : [0, ¹]P u( ) ) is measurable. We refer to any u( ) )3U as an admissible sensor schedule. For any u( ) )3U, we have the following output equation: , dy(t)" (t)[C (t)x(t) dt#D (t) dw (t)], SRG G G G G t*0,
(2.1)
y(0)"0,
(2.2)
where
1, (t)" SRG 0,
u(t)"i, u(t)Oi.
For each t*0, C (t)31K"L and D (t)31K"I. Each eleG G ment of the matrix-valued functions C (t) and D (t) are G G uniformly bounded measurable functions. The processes w (t) are 1I-valued Brownian motions on (, F, P) with G covariance E(w (t)!w (s))(w (t)!w (s))?"R (t!s). We G G G G G assume that R is a k;k symmetric and positive de"nite G matrix. We assume throughout that D (t)R D?(t)'0 G G G (and hence, invertible). Furthermore, we assume that x , *(t), and w (t),w (t),2,w (t) are mutually independent. , Under these assumptions, there is a unique strong solution y(t) of (2), which we refer to as the observation process associated with the scheduling policy u( ) )3U. Once u( ) ) has been chosen, the optimal mean-square estimate of the state is given by the well-known Kalman "lter. In particular, let u( ) )3U be given and let FW"y(s), R 0)s)t be the -algebra generated by the observation process y(t) associated with u( ) ). The optimal meansquare estimate of x(t) given FW is x( (t)"Ex(t) FW and R R the associated error covariance is P(t)"E(x(t)! ? x( (t))(x(t)!x( (t)) . For any u( ) )3U, x( (t) and P(t) are given as follows (see Ahmed, 1998; Anderson, & Moore, 1990):
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
Theorem 1. For a given sensor schedule u( ) )3U, the optimal mean-square estimate x( (t) is the unique strong solution of the stochastic diwerential equation
, dx( (t)" A(t)!P(t) C?(t)RI i\(t)C (t) x( (t) dt SRG G G G , # P(t) C?(t)RI i\(t) dy(t), (3.1) SRG G G x( (0)"x , (3.2) ? where RI i\(t)"[D (t)R D (t)]\, and the error covariance G G G matrix P(t) is the unique solution of the matrix Riccati diwerential equation
PQ (t)"A(t)P(t)#P(t)A?(t)#B(t)QB?(t) , ! P(t)C?(t)RI i\(t)C (t)P(t), G SRG G G P(0)"P .
By adopting the recently developed control parametrization enhancing transform (CPET) Technique (Lee et al., 1999; Teo et al., 1999), we develop a computational procedure for solving Problem (P). We introduce the new time scale s which varies from 0 to M. Let V denote the class of non-negative piecewise constant scalar functions de"ned on [0, M) with "xed interior knot points located at 1, 2, 3,2, M!1. Note that we let M" N, where is the assumed maximum number of times any sensor i3 is being selected. The transformation (CPET) from t3[0, ¹] to s3[0, M] is de"ned by the di!erential equation
(4.1) (4.2)
where the scalar function v(s)3V is called the enhancing control, and it satis"es
Problem. Subject to (4), "nd an admissible sensor schedule uH( ) )3U so that (5)
2 J(u( ) ))" traceP(t) dt#traceP(¹). (6) The optimal sensor scheduling problem (5) is an optimal discrete-valued control problem, where the system dynamics and the objective function are, respectively, given by (4) and (6). The main di$culty associated with (5) is that the control u( ) ) is constrained to take values in a discrete set 1,2,2, N. In addition, switchings may occur at any time in the time horizon [0, ¹]. This problem is referred to as Problem (P). Remark 2. The proposed method can be extended to the general case of N '1 without much di$culty. The only change we need to make is the de"nition of . More speci"cally, for N '1, there are altogether N ways of turning-on N sensors from a total of N sensors, where N N! " . N " N !(N!N )! N Now, de"ne "1, 2,2, N . The meaning of each i3 is to be understood as one of these sensors combinations, instead of just one particular sensor in the case of N "1.
3. A computational procedure
dt "v(s) (with initial condition t"0 when s"0), (7) ds
Clearly, P(t) depends on the sensor schedule u( ) )3U that is chosen and the estimation error can be reduced by choosing u( ) ) so as to make P(t) &small'. This motivates the sensor scheduling problem, which we state as follows:
J(uH( ) ))" inf J(u( ) )), S ZU 2
where
2019
+
v(s) ds"¹.
Moreover, we introduce a "xed function (s): (s),(i mod N), s3[i, i#1), i"0, 1, 2,2, M!1. (8) Note that the idea of this CPET transformation is to let any u(t)3U be naturally represented by a v(s)3V whenever this "xed (s) is de"ned. For example, if we have N"4, ¹"1, and
1, t3[0,0.3),
u(t)" 3, t3[0.3,0.7), 2, t3[0.7,1),
then, we can naturally "nd a representation v(s)3V given by
v(s)"
0.3, 0, 0.4, 0, 0, 0.3, 0, 0, 0,
s3[0,1) (corresponds s3[1, 2) (corresponds s3[2,3) (corresponds s3[3,4) (corresponds s3[4,5) (corresponds s3[5,6) (corresponds s3[6,7) (corresponds s3[7,8) (corresponds s3M!1, M).
to to to to to to to to
"1), "2), "3), "4), "1), "2), "3), "4),
The transformed problem can thus be stated as follows: Subject to the system equations
d P(s)"v(s) A(s)P(s)#P(s)A?(s)#B(s)QB?(s) ds
, ! P(s)C?(s)RI i\(t)C (s)P(s) , G IQG G G P(0)"P ,
(9.1) (9.2)
2020
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
"nd a v3V such that the cost
J(v( ) ))"
+
Consider the function (10)
is minimized over V subject to the equality constraint
+
v(0,
1,
traceP(s)v(s) ds#traceP(M)
v(s) ds"¹.
(11)
Let this problem be referred to as Problem (TP). It is clear from Lee et al. (1999) and Teo et al. (1999) that Problem (TP) is an optimal control problem in canonical form. Furthermore, all the possible discontinuities for any v( ) )3V are pre-assigned at 1, 2,2, M. From the results obtained in Lee et al. (1999) and Teo et al. (1999), the following theorem is obtained: Theorem 3. Suppose the maximum number of times any sensor i3 is being selected, or `turned-on , equals . If M* N, Problem (TP) and Problem (P) are equivalent. Proof. See the appendix. 䊐 Remark 4. By examining the proof of Theorem 3 given in the appendix, we note that if M is large enough to allow switchings to be more than or equal to what the optimal solution requires, then V contains the optimal solution.
4. Switching penalization It is of practical importance to associate a cost with the switching of the sensors. This cost may be just the wear and tear on the switching mechanism, or it may be the risk cost of revealing the existence of the observer (e.g. active sensors in submarine detection). In this section, we propose a way to penalize the switchings. Observe that when s3[i, i#1), v(s) represents the time length of which sensor (i mod N) is selected, or `turnedona. If the value of v(s) is `negligibly smalla, or only `slightly greater than zeroa for some s, then we consider that the sensor is not `turned-ona, and hence, does not contribute to the total number of switchings. To quantify this convention, let '0 denote a small prespeci"ed constant, which is referred to as the vanishing tolerance level. Then we adopt the convention of rounding o! the value of v(s) to zero if v(s)) . Thus, for a given v3V such that (11) is satis"ed, we de"ne (v( ) )) as the C total number of pieces v(s)* , s3[i, i#1), over i"0, 1, 2,2, M!1.
h , 0)v( , uB(v)" h , )v(2 , 0, v*2 , where 0( (0.5 and
(v! ) 2(v! ) h "10 !0.05 ! ! #1 #0.1 , h "10 !0.05
(v! ) 2(v! ) ! #1 #0.1 .
This function has the following interesting properties: (i) The derivative of uB( ) ) with respect to its argument exists everywhere. (ii) The discontinuities of uB(v( ) )) are all located at 1, 2,2, M!1. (iii) For a given v3V such that (11) is satis"ed and '0 a prespeci"ed vanishing tolerance level, there exists a '0 such that (v( ) ))) +uB(v(s)) ds)M. C The "rst two properties are clear, while the proof of the third property is given in the appendix. Clearly, +uB(v(s)) ds approaches (v( ) )) from above. C Thus, +uB(v(s)) ds can be used to penalize any unnecess ary number of switchings. Our optimal sensor scheduling problem with a penalty on the number of switchings can thus be stated as: Subject to the system dynamics (9) and the equality constraint (11), "nd a v3V such that the cost
J(v( ) ))"
+
[traceP(s)v(s)#uB(v(s))] ds # traceP(M)
(12)
is minimized over V, where is a weighting. This is a standard optimal control problem in canonical form (Fig. 1).
Fig. 1. The plot uB(v).
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
5. Lower bounds on the turned-on times
(v(s)!)uB(v(s)) (s)*0. IQG
!1.1725
In addition to the above formulation, one practically meaningful constraint we would like to consider is `if sensor i3 is selected, then it must be used for at least units of timea. We assume throughout that ' where is a prespeci"ed vanishing tolerance level. Then, by the de"nition of the function uB(v(s)) given in Section 4, we consider, for a given i, the all-time constraint: (13)
The possible discontinuities of the all-time constraint (13) are located at 1, 2,2, M!1. Proposition 5. Let v3V be such that (11) is satisxed. Suppose (13) is satisxed for some given i31, 2,2, N. Then, for any '0, where is a prespecixed vanishing tolerance level, there exists a '0 such that either v(s)* or v(s)( whenever (s)"i. Proof. See the appendix. 䊐 Since all switchings are located at the integer knots, (13) is already in a form readily solvable by the techniques developed in Teo et al. (1991), and thus by MISER3.2 (Jennings et al., 1997). The central idea behind the software MISER3 is the concept of control parametrization. Essentially, this procedure approximates the control functions by a linear sum of basis functions, chosen for e$cient computation and reality of control approximation. These basis functions are the piecewise constant functions on "nite support or the piecewise linear continuous functions on "nite support (the witches' hat functions). Thus, by appropriate reformulation, the optimal control problem is transformed into a nonlinearly constrained mathematical programming problem, and hence is solvable by existing e$cient software packages for mathematical programming problems. See Teo et al. (1991) for details on the control parametrization technique. The control parametrization enhancing transform (CPET) is introduced in Teo et al. (1999) to enhance the classical control parametrization technique. This transform involves the introduction of an additional piecewise constant function, mapping the switching times on to a set of integer knots in a new time scale. Hence, the transformed optimal control problem can be handled by the usual control parametrization technique.
6. An illustrative example Consider a system with n"5, m"1, p"5, N"10 (10 sensors), ¹"1,
2021
9.3567 !2.0278
A" !1.6009 !6.2658
3.9829 !9.7397 !4.9574 !1.7843 6.7470
1.5886 !7.7981
9.8620 !1.2604 !7.0801
1.3012 ,
3.6903 !3.1387 !1.6332 !1.2068
!3.0191 !8.3465
1 1 1 1 1 1 1 1 1 1
1.1865
!1.1452 !2.0959
10
0
0
0
0
0 10
0
0
0
0 10
0
0 ,
Q" 1 1 1 1 1 , P " 0 1 1 1 1 1 0 1 1 1 1 1
0
0
0 10
0
0
0
0 10
1 0 0 0 0 0 1 0 0 0
B" 0 0 1 0 0 , 0 0 0 1 0 0 0 0 0 1
C "[1 1 0 0 0], D "1, R "1.0, C "[1 0 1 0 0], D "1, R "2.25, C "[1 0 0 1 0], D "1, R "2.56, C "[1 0 0 0 1], D "1, R "3.4225, C "[0 1 1 0 0], D "1, R "4.0, C "[0 1 0 1 0], D "1, R "4.0, C "[0 1 0 0 1], D "1, R "3.4225, C "[0 0 1 1 0], D "1, R "2.56, C "[0 0 1 0 1], D "1, R "2.25, C "[0 0 0 1 1], D "1, R "1.0. There is a lower bound of the `turned-ona time constraint imposed on sensor C1 where "0.5. In this example, "1, and we choose "0.005, M"80. The optimal control software package MISER3.2 is then used to solve the corresponding version of the transformed problem, i.e. Problem (TP). It was run on a PC booted with LINUX, having a CPU speed of 300 MHz and equipped with 128 MB of RAM. It takes a total of 298 iterations to converge, where each iteration takes roughly 2 s. Relevant results obtained are shown in Figs. 2 and 3. Note that although there are 10 sensors, only three of them have been turned `ona in the converged optimal solution, namely sensor C1, C8 and C9. In particular, sensor C1 is `ona for 0.5 unit of time, hitting its lower
2022
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
Appendix. Proof of Theorem 3.1 Let v 3V be any enhancing control for Problem (TP) , 0)s(1, , 1)s(2, v (s)" , 2)s(3,
Fig. 2. The plot of all 5 diagonal elements of P(t).
, M!1)s(M, + where *0 for j"1, 2,2, M. The corresponding H u (t)3U for Problem (P) in the original time scale t is uniquely given by
1 mod N,
0)t( , 2 mod N, )t( # , u (t)" 3 mod N, # )t( # # , +\ + M mod N, )t( "¹. G G G G Clearly, J(u ) of (10) equals J(v ) of (6). Now, we let u 3U be any sensor schedule for Problem (P):
Fig. 3. The plot of u(t).
bound of the `turned-ona time. Seven other sensors are not used in this optimal solution. The converged cost is 23.797.
7. Conclusions We proposed a computational scheme to solve the optimal sensor scheduling problem. We have overcome the di$culties of computing the exact switching times, and the combinatorial nature of the scheduling problem. Moreover, a novel technique to penalize the number of switchings is developed, and a lower bound of the `turned-ona time constraint for any particular sensor combination is also considered. An illustrative example is provided to demonstrate our proposed method.
Acknowledgements The Research Committee of The Hong Kong Polytechnic University is acknowledged. Moreover, the authors are grateful to the Associate Editor and the 2 anonymous referees for their valuable comments and suggestions which have helped to improve the paper.
i , i , u (t)" i ,
0)t(t , t )t(t , t )t(t ,
i , t )t(t "¹, ( (\ ( where i , i ,2, i 31, 2,2, N, is the total number of ( `piecesa, and it is bounded above by N. Construct
t , t !t , t !t , v (s)" ¹!t , ( 0,
s3[0;N#i !1,0;N#i ), s3[1;N#i !1,1;N#i ), s3[2;N#i !1, 2;N#i ), s3[(!1);N#i !1, ( (!1);N#i ), ( otherwise.
Clearly, v (s)3V for Problem (TP) is corresponding to u (t) for Problem (P) in the transformed time scale s. (Note that this correspondence is not unique in general). The largest possible value for i is N. A su$cient condi( tion for the existence of such a v (s) is when M*(!1)N#N"N" N. Under this su$cient condition, J(v ) of (6) equals J(u ) of (10). 䊐 Proof of Property (iii) of uB in Section 4. Take ) /2, we have (v( ) ))) +uB(v(s)) ds. Clearly, +uB(v(s)) ds"M if C and only if v(s)*2 for all s3[0, M]. On the other hand,
+uB(v(s)) ds(M. 䊐
H.W.J. Lee et al. / Automatica 37 (2001) 2017}2023
Proof of Proposition 5.1. For those s such that (s)"i, (13) is reduced to (v(s)!)uB(v(s))*0. For any '0, where is a prespeci"ed vanishing tolerance level, there exist a '0 such that uB( )" . There are two possibilities, either v(s)( , or v(s)* . If v(s)( , then uB(v(s))( (they are rounded o! to zero by the convention mentioned in Section 4). On the other hand, if v(s)* , then uB(v(s))* . Thus (v(s)!)uB(v(s))*0 implies (v(s)!)*0. Hence, v(s)*. 䊐 References Ahmed, N. U. (1998). Linear and nonlinear xltering for scientists and engineers. Singapore: World Scienti"c. Anderson, B. D. O., & Moore, J. B. (1990). Optimal control*linear quadratic methods. Information and system science series. Englewood Cli!s, NJ: Prentice-Hall. Baras, J. S., & Bensoussan, A. (1989). Optimal sensor scheduling in nonlinear "ltering of di!usion process. SIAM Journal of Control and Optimization, 27(4), 786}813. Jennings, L. S., Fisher, M. E., Teo, K. L., & Goh, C. J. (1997). MISER3 optimal control software (Version 2): Theory and user manual. Centre for Applied Dynamics Optimization. The University of Western Australia, Perth, Australia. Lee, H. W. J., Teo, K. L., Rehbock, V., & Jennings, L. S. (1999). Control parametrization enhancing technique for optimal discrete-valued control problems. Automatica, 35, 8. Teo, K. L., Goh, C. J., & Wong, K. H. (1991). A unixed computational approach to optimal control problems. Marlow, UK: Longman Scienti"c and Technical. Teo, K. L., Jennings, L. S., Lee, H. W. J., & Rehbock, V. (1999). The control parametrization enhancing transform for constrained optimal control problems. Journal of the Australian Mathematical Society, Series B, 40(Part 3), 314}335. Meier, L., Peschon, J., & Dressler, R. M. (1967). Optimal control of measurement subsystems. IEEE Transactions on Automatic Control, AC-12(5), 528}536. Andrew Lim was born in Penang, Malaysia, in 1973. He obtained his undergraduate degree in Mathematics from the University of Western Australia in 1995, and his Ph.D. in Systems Engineering from the Australian National University in 1998. He has held research positions at the Chinese University of Hong Kong, Columbia University (New York), and the University of Maryland (College Park). Currently, he is a visiting Assistant Professor in the Department of Industrial Engineering and Operations Research at Columbia University. His research interests are in the areas of optimization, stochastic control, backward
2023
stochastic di!erential equations, and Markov decision processes, with applications to problems in operations research, engineering and "nance.
Heung Wing Joseph Lee was born in Hong Kong in December 1969. He received his B.Sc.(1st Hons) in 1994 and Ph.D. (with distinction) in 1997, both at the Department of Mathematics, The University of Western Australia. In 1997}1998, he was working as a Postdoctoral Research Fellow at the Department of System Engineering and Engineering Management, The Chinese University of Hong Kong. He worked as a Visiting Assistant Professor at the Department of Mathematics, The Hong Kong University of Science and Technology in 1998}1999. He is currently an Assistant Professor in the Department of Applied Mathematics at The Hong Kong Polytechnic University. His research interests include Computational Optimal Control, Mixed Discrete Programming, Scheduling, and Control of Chaotic Systems.
Kok Lay Teo received the Ph.D. degree in Electrical Engineering from the University of Ottawa, Canada in 1974. He was with the Department of Applied Mathematics, the University of New South Wales, Australia, from 1974 to 1985, the Department of Industrial and Systems Engineering, National University of Singapore, Singapore, from 1985 to 1987, and the Department of Mathematics, the University of Western Australia, Australia, from 1988 to 1996. He was Professor of Applied Mathematics at Curtin University of Technology, Australia, from 1996 to 1998. He is currently the Chair Professor of Applied Mathematics and Head of Department of Applied Mathematics at The Hong Kong Polytechnic University, Hong Kong. He has also been an Adjunct Professor of the University of Waterloo, Canada, from May 2001, and Chongqing Normal University, China, from September 1999. He has published 5 books, edited several books and special issues for international journals. He has also published numerous journal and conference papers in areas, which include control, optimal control, optimization, signal processing, and biomachine. The software package, MISER3.2, for solving general constrained optimal control problems was developed by the research team under his leadership. His research interests includes both the theoretical and practical aspects of optimal control and optimization, and their applications, in particular, signal processing in telecommunications. Dr. Teo is a Senior Member of Institute of Electrical and Electronic Engineers (IEEE) (USA), and is in the editorial board of a number of international journals in control, dynamical systems and optimization, including Automatica and Journal of Global Optimization.