- Email: [email protected]
Random Search in Adaptation Problems of Stochastic Systems
Copn-ight © IL\C Stochastic Control Yilniu s. Lithuanian SSR. l·SSR. 19H6
ADAPTln: ~IETHODS OF I'AR.-\'.IETER EST1\I.-\TIO:\ .-\:\D IDE:\Tl FIC.-\TIO:\
RANDOM SEARCH IN ADAPTATION PROBLEMS OF STOCHASTIC SYSTEMS L. A. Rastrigin RiK{[ Po/y/nlilli((I/ IIISli/II/I' , RiK(I, L'SSR
Abstract. Algorithm of binar~ random search allowing to solve parameter adaptation problems and the problems of structure of stochastic objects is presented. The algorithm uses techniques of sequential accumulation for each of the binar~ variables that enable to reduce the dimensions of the problem in the process of determination of each variable value with given probability. Alternatives of the method for variables measured in metric and order (discrete) scales are presented. Ke;ywords. Adaptive control; adaptive system; Boolean function; sequential accumulation; stochastic systems; stochastic control; random processes; random search. For digital values CI'J' the sequence relation ~ changes into the sign "less"
'l'he problem of adaptation of stochastic sytems is reduced to deter[!lination X .. , i.e. solving the optimization problem
/)/X) -
m~r, ~X*
>.
3. Domain;) maJ posses combinatorial
(1)
'5'( t XG$ in the A>Jesence of disturbances when the index ~(X)is always determined erraneouslJ, and the character of these errors is unknown and changes in tj,me. In general cases both the index rv(Xj and the domain of the determination X change in time. That is the reason wh~ adaptation unlike optimization usuallJ does not terminate.
character. Moreover, the values of ai} in (2) are the names of X" and there is no order relation between these names. To be more precise: there exist on1J two relations between the names - those of equality an inequality. A special case of such a domain are the vertices of a hypercube, when = 2, and the variables are either binary variables or Boolean variables: a l , = 0 ; Q i2 = 1 •
P
ri
4. Domain ~ can be generated bJ the product
To solve problems of such a kind it is necessarJ to organize a search process of X* by means of adaptation. Adaptation algorithms can be subdivided into two classes: deterrninated and stochastic (random, rando~ized). The latter differ in that some element of randomness is purposefully introduced into the process of adaptation (e.g., in the form of a random step).
S:: SI X.,s2 X,s3 ,
,sf
where £' R~, - the domain of metric var ia bles .:x, , •.. , 0::* ;;;2 EO]) "2 - the dO:liain of order variables.::t' .:c".L LJIr Ic,+""') ",+,<~ and =~ 3 - combinatorial domain with varia es :X:k~k +1 , ... ).::tL L L that are ,z. ",f''
~
The problems of adaptation differ in the structure of the domain ~ that determines the specific character of the factor X that is to be adaptized. Let us distinguish between the following characteristics of the domain
5. The set of graphs ~
produces the adapting factor in the form of graph
,s .
X = (4.1 I3)C>
1. ,$£ Rn. , Le. the dO O1ain $. is continuous. I~ this case the 0ytimization factor X ~s a vector X = (;X, ).::t:,a),., .A:11.) in n-dimensional Cartesian space. )
It is quite evident that this case can be reduced to a case with mixed variables where weights C of the arcs f3 are measured in ~etric, order and name scales, but arcs are measured according to the binary scale (of names).
(2)
bound by the order relation Qil>- Q i 2.
>'" ~ 0"9,'
(5)
where A = (a" ••• , an.) - the set of vertices, B = (8/I} 8,;z , ... , B"n., ) - the set of arcs (~;;EtO;1}l,C = ( C, , . . . ,c*)_ che set of weights of these arcs. Adaptation with such a set is of a structural character.
2. Domain ;3 can be formed by nodes of an n-dimensional array. In this case each parameter of the vector takes discrete values
X ~ ~ {ai' ) a. i2 ).") Cl,?,') ~':~ ..../l)
(4)
((':/.; .. ,)n.) .0) 195
L. A . Rastri g in
196
To solve all these problems the following random search algorithm can be proposed. REDUCING THE PROBLEMS TO A BINARY PROBUM All t he described adapt ation pr oble ms can be reduced by way of one-to-one transformation
Y :: F(X')
(6)
y:-
to one - binar;y problem of the form
~(y)= r;(c-'(Y)) --,m(~ ~ where
52.:
~
YEJ'2
F-I(Y)
(8)
,." y~ S) {3 - s et of vertices of m-dimensional hypercube: Y~(!ln " ') Ym};)li 6 {Ojt} ; ~ _ limitations cause d b;y the transformation ~ • Transformation ~ itself is carried o~t b;y fairly simple and evident means; )' - s olution of the problem ( 7 ):
y* = ((/' ~ . .. ) ?J;') , corresponds to X*: X *: F -I (y *) ,
t ha t
(9 )
Le t us accumula te the sum for each parameter ,//i "
t;{L ('(; )-L ((;})(yU -Yi.;.J,
(11)
J "-/
where >;, - the j-th r~liza tion of random vector ( y. ~J2 ); 'Y.;' - a dd itional v ector to y. ; c/. - t he j -th random r eali.' .;TU za tion of Yc' ; Y'i - ne gation ofYcj' . It i s evident tha t when
Pt=- PZO~(ilt= // S/> oj ')
( 12 )
i.e., t he probabilit;y of error. If this error is sufficiently small, i.e. - less than the given ~* , then the sol ution N
when S, > 0 when iS~< 0 can be considered to be correct with probability 1 - f* .
,
( 13 )
N
De ter mination of Pi ( 12 ) does not cause an;y diffic ult;y as it is known that
st
:=
~, N
(14)
where ~ - the sign of mat hematical expectation and the sum SiN is distributed nor mall;y. The es-hma-\:.es of the value d,. and dispersion .s;.'" can be accomplish-
e{a/j..,; ~i~}'
XI '
t
L
(15)
Then each variable .z,' is transformed into a binar;y v a~iable in the q-th stage:
o
when
1
,J:: ' =:Q .q.,
(16)
"ll
when ::Cc' =: 8,"" The number of such binar;y variables will be equal to t he number of metric and order variables. When solving the problem of binar;y optimization in the q-th stage there is a change of levels (15) already at the first determination according to t he following formula. For metric variables:
(o.9-11!9-1-'l_ t'
/
0 t here is
«
/'1
Let us divide the process of adaptation into stages. For each metric or order variable in the q-th search stage let us take onl;y t wo a d jacent va lues a ,9 and t. ~
L )
"
*Si»
a great probabilit;y of U. =: 0; and if N S, ' 0 we have J, u .* ~L =: 1. 'r hus the pr oblem is reduc ed to evaluation of probability of t he event
~ { O ,x.:: 1
It is ev i dent tha t the proposed method allows us to solve the problems of adaptation if the stocha s tic factor is given. These problems can be multi-extremal as the method is not orie nted on unimodality of the function 9 (X). Nevertheless in the last case it can be modified in order to decrease the dimension m. This can be accomplished in the foll owing wa;y.
lI ,9-= { Vit
RANDOM SEARCH WITH ACCUMULATION
L
MEl'HOD ANALYSIS AND MODIFICATION
(1 0 )
Let us assume tha t the condition YE Sl is observed without any difficulties and therefore random vectors Y1 , • • • , ~ E.Q. can easil;y be generated.
S ,II:
p/I~ pfl. we assume the solution (13) and the i-th variable is being fixed, thw decreasing the d i ~eusions of the problem b;y one unit. Let us consider the solution of d ifferent proble ms by the given method.
If
(7)
E$
YE-8fT)
ed b;y a standard method (the method of the least squares), tha t enables to evaluate probabilities p,.N (12) in each step by means of Laplace integral.
-
{ ~,~'lt.L1.J when y/II-=: "
{ Z', Ja'!-A , j Q,9} l
I
1 ( 17) when u,'f4=: 0
,
./l)
where ,j c' - .::c,' change s tep_ The c hange of the order va riables i s analagous. Thus in the space of parame ters {%} there t akes place a move ment of h;ypercube ( 15 ) i n the extre mum direction. The value of ;0+ can be increased as the final decision is not made. Thus the speed of the method is accele rated considerably. When working in extremum domain it is natural to expand the hypercube (15):
.x. I
6./Q.9-((.)
1>, .9--) c (.CJ )
(1 8 )
in order to search for the optimum in the midpoint t.'" , ,
:x/ :
The efficiency of the method fox solving problems of all five t;y pes has been proved experimentally. REFERENC&) Rastrig in, L.A. (1981). Adap tation of Complex Systems. Zinatne, Riga (in Russian).
ADAPTln: ~IETHODS OF I'AR.-\'.IETER EST1\I.-\TIO:\ .-\:\D IDE:\Tl FIC.-\TIO:\
RANDOM SEARCH IN ADAPTATION PROBLEMS OF STOCHASTIC SYSTEMS L. A. Rastrigin RiK{[ Po/y/nlilli((I/ IIISli/II/I' , RiK(I, L'SSR
Abstract. Algorithm of binar~ random search allowing to solve parameter adaptation problems and the problems of structure of stochastic objects is presented. The algorithm uses techniques of sequential accumulation for each of the binar~ variables that enable to reduce the dimensions of the problem in the process of determination of each variable value with given probability. Alternatives of the method for variables measured in metric and order (discrete) scales are presented. Ke;ywords. Adaptive control; adaptive system; Boolean function; sequential accumulation; stochastic systems; stochastic control; random processes; random search. For digital values CI'J' the sequence relation ~ changes into the sign "less"
'l'he problem of adaptation of stochastic sytems is reduced to deter[!lination X .. , i.e. solving the optimization problem
/)/X) -
m~r, ~X*
>.
3. Domain;) maJ posses combinatorial
(1)
'5'( t XG$ in the A>Jesence of disturbances when the index ~(X)is always determined erraneouslJ, and the character of these errors is unknown and changes in tj,me. In general cases both the index rv(Xj and the domain of the determination X change in time. That is the reason wh~ adaptation unlike optimization usuallJ does not terminate.
character. Moreover, the values of ai} in (2) are the names of X" and there is no order relation between these names. To be more precise: there exist on1J two relations between the names - those of equality an inequality. A special case of such a domain are the vertices of a hypercube, when = 2, and the variables are either binary variables or Boolean variables: a l , = 0 ; Q i2 = 1 •
P
ri
4. Domain ~ can be generated bJ the product
To solve problems of such a kind it is necessarJ to organize a search process of X* by means of adaptation. Adaptation algorithms can be subdivided into two classes: deterrninated and stochastic (random, rando~ized). The latter differ in that some element of randomness is purposefully introduced into the process of adaptation (e.g., in the form of a random step).
S:: SI X.,s2 X,s3 ,
,sf
where £' R~, - the domain of metric var ia bles .:x, , •.. , 0::* ;;;2 EO]) "2 - the dO:liain of order variables.::t' .:c".L LJIr Ic,+""') ",+,<~ and =~ 3 - combinatorial domain with varia es :X:k~k +1 , ... ).::tL L L that are ,z. ",f''
~
The problems of adaptation differ in the structure of the domain ~ that determines the specific character of the factor X that is to be adaptized. Let us distinguish between the following characteristics of the domain
5. The set of graphs ~
produces the adapting factor in the form of graph
,s .
X = (4.1 I3)C>
1. ,$£ Rn. , Le. the dO O1ain $. is continuous. I~ this case the 0ytimization factor X ~s a vector X = (;X, ).::t:,a),., .A:11.) in n-dimensional Cartesian space. )
It is quite evident that this case can be reduced to a case with mixed variables where weights C of the arcs f3 are measured in ~etric, order and name scales, but arcs are measured according to the binary scale (of names).
(2)
bound by the order relation Qil>- Q i 2.
>'" ~ 0"9,'
(5)
where A = (a" ••• , an.) - the set of vertices, B = (8/I} 8,;z , ... , B"n., ) - the set of arcs (~;;EtO;1}l,C = ( C, , . . . ,c*)_ che set of weights of these arcs. Adaptation with such a set is of a structural character.
2. Domain ;3 can be formed by nodes of an n-dimensional array. In this case each parameter of the vector takes discrete values
X ~ ~ {ai' ) a. i2 ).") Cl,?,') ~':~ ..../l)
(4)
((':/.; .. ,)n.) .0) 195
L. A . Rastri g in
196
To solve all these problems the following random search algorithm can be proposed. REDUCING THE PROBLEMS TO A BINARY PROBUM All t he described adapt ation pr oble ms can be reduced by way of one-to-one transformation
Y :: F(X')
(6)
y:-
to one - binar;y problem of the form
~(y)= r;(c-'(Y)) --,m(~ ~ where
52.:
~
YEJ'2
F-I(Y)
(8)
,." y~ S) {3 - s et of vertices of m-dimensional hypercube: Y~(!ln " ') Ym};)li 6 {Ojt} ; ~ _ limitations cause d b;y the transformation ~ • Transformation ~ itself is carried o~t b;y fairly simple and evident means; )' - s olution of the problem ( 7 ):
y* = ((/' ~ . .. ) ?J;') , corresponds to X*: X *: F -I (y *) ,
t ha t
(9 )
Le t us accumula te the sum for each parameter ,//i "
t;{L ('(; )-L ((;})(yU -Yi.;.J,
(11)
J "-/
where >;, - the j-th r~liza tion of random vector ( y. ~J2 ); 'Y.;' - a dd itional v ector to y. ; c/. - t he j -th random r eali.' .;TU za tion of Yc' ; Y'i - ne gation ofYcj' . It i s evident tha t when
Pt=- PZO~(ilt= // S/> oj ')
( 12 )
i.e., t he probabilit;y of error. If this error is sufficiently small, i.e. - less than the given ~* , then the sol ution N
when S, > 0 when iS~< 0 can be considered to be correct with probability 1 - f* .
,
( 13 )
N
De ter mination of Pi ( 12 ) does not cause an;y diffic ult;y as it is known that
st
:=
~, N
(14)
where ~ - the sign of mat hematical expectation and the sum SiN is distributed nor mall;y. The es-hma-\:.es of the value d,. and dispersion .s;.'" can be accomplish-
e{a/j..,; ~i~}'
XI '
t
L
(15)
Then each variable .z,' is transformed into a binar;y v a~iable in the q-th stage:
o
when
1
,J:: ' =:Q .q.,
(16)
"ll
when ::Cc' =: 8,"" The number of such binar;y variables will be equal to t he number of metric and order variables. When solving the problem of binar;y optimization in the q-th stage there is a change of levels (15) already at the first determination according to t he following formula. For metric variables:
(o.9-11!9-1-'l_ t'
/
0 t here is
«
/'1
Let us divide the process of adaptation into stages. For each metric or order variable in the q-th search stage let us take onl;y t wo a d jacent va lues a ,9 and t. ~
L )
"
*Si»
a great probabilit;y of U. =: 0; and if N S, ' 0 we have J, u .* ~L =: 1. 'r hus the pr oblem is reduc ed to evaluation of probability of t he event
~ { O ,x.:: 1
It is ev i dent tha t the proposed method allows us to solve the problems of adaptation if the stocha s tic factor is given. These problems can be multi-extremal as the method is not orie nted on unimodality of the function 9 (X). Nevertheless in the last case it can be modified in order to decrease the dimension m. This can be accomplished in the foll owing wa;y.
lI ,9-= { Vit
RANDOM SEARCH WITH ACCUMULATION
L
MEl'HOD ANALYSIS AND MODIFICATION
(1 0 )
Let us assume tha t the condition YE Sl is observed without any difficulties and therefore random vectors Y1 , • • • , ~ E.Q. can easil;y be generated.
S ,II:
p/I~ pfl. we assume the solution (13) and the i-th variable is being fixed, thw decreasing the d i ~eusions of the problem b;y one unit. Let us consider the solution of d ifferent proble ms by the given method.
If
(7)
E$
YE-8fT)
ed b;y a standard method (the method of the least squares), tha t enables to evaluate probabilities p,.N (12) in each step by means of Laplace integral.
-
{ ~,~'lt.L1.J when y/II-=: "
{ Z', Ja'!-A , j Q,9} l
I
1 ( 17) when u,'f4=: 0
,
./l)
where ,j c' - .::c,' change s tep_ The c hange of the order va riables i s analagous. Thus in the space of parame ters {%} there t akes place a move ment of h;ypercube ( 15 ) i n the extre mum direction. The value of ;0+ can be increased as the final decision is not made. Thus the speed of the method is accele rated considerably. When working in extremum domain it is natural to expand the hypercube (15):
.x. I
6./Q.9-((.)
1>, .9--) c (.CJ )
(1 8 )
in order to search for the optimum in the midpoint t.'" , ,
:x/ :
The efficiency of the method fox solving problems of all five t;y pes has been proved experimentally. REFERENC&) Rastrig in, L.A. (1981). Adap tation of Complex Systems. Zinatne, Riga (in Russian).