- Email: [email protected]
Non-parametric Procedures under Estimating Stochastic Dependencies in Nonlinear Systems
IFAC Conference on Manufacturing Modelling, IFAC Conference on Manufacturing Modelling, Management and on Control IFAC Manufacturing Available online at www.sciencedirect.com Management and on Control IFAC Conference Conference Manufacturing Modelling, Modelling, June 28-30, 2016. Troyes, France Management and Control Control June 28-30, 2016. Troyes, France Management and June June 28-30, 28-30, 2016. 2016. Troyes, Troyes, France France
ScienceDirect
IFAC-PapersOnLine 49-12 (2016) 1779–1784
Non-parametric Non-parametric Procedures Procedures under under Estimating Estimating Stochastic Stochastic Dependencies Dependencies Non-parametric Procedures under Estimating Stochastic Dependencies Non-parametric Procedures under Estimating Stochastic Dependencies in Nonlinear Systems in Nonlinear Systems in Nonlinear Systems in Nonlinear Systems K.R. K.R. Chernyshov Chernyshov K.R. K.R. Chernyshov Chernyshov V.A. Trapeznikov Institute of Control Sciences, 65 Profsoyuznaya, Profsoyuznaya, Moscow Moscow 117997, 117997, Russia Russia V.A. Trapeznikov Institute of Control Sciences, 65 (e-mail: [email protected]) V.A. Trapeznikov Institute of Control Sciences, 65 Profsoyuznaya, Moscow 117997, Russia V.A. Trapeznikov Institute of Control Sciences, 65 Profsoyuznaya, Moscow 117997, Russia (e-mail: [email protected]) (e-mail: (e-mail: [email protected]) [email protected])
Abstract. In the paper, a measure of dependence coupling k pairs of random processes is introduced. Such a measure, Abstract. In the paper, a measure of dependence coupling k pairs of random processes is introduced. Such a measure, based on using conditional mathematical expectations of the processes, may processes be considered as a further generalization Abstract. In paper, of coupling kk pairs of is Such aa measure, Abstract. In the theconditional paper, aa measure measure of dependence dependence coupling pairs of random random is introduced. introduced. Such measure, based on using mathematical expectations of the processes, may processes be considered as a further generalization of the on dispersion (variance)mathematical functions. Convergence with probability 1 of nonparametric estimates of such a measure based using conditional expectations of the processes, may be considered as a further generalization based using conditional expectationswith of the processes, may be considered as a further generalization of the on dispersion (variance)mathematical functions. Convergence probability 1 of nonparametric estimates of such a measure is derived using (variance) sampled data. TheseConvergence estimates are applied to deriving sampled analogues of some nonlinear of dispersion functions. with probability 11 of estimates such measure of the the dispersion functions. withapplied probability of nonparametric nonparametric estimatesofof of some such aanonlinear measure is derived using (variance) sampled data. TheseConvergence estimates are to deriving sampled analogues measures ofusing stochastic dependence of random processes, in particular, to asampled consistent, in the Kolmogorov sense, is derived sampled data. These estimates are applied to deriving analogues of some is derivedofusing sampled data. These estimates are applied to deriving analogues of some nonlinear nonlinear measures stochastic dependence of random processes, in particular, to asampled consistent, in the Kolmogorov sense, measure ofofdependence. measures stochastic dependence of random processes, in particular, to a consistent, in the Kolmogorov sense, measures of stochastic dependence of random processes, in particular, to a consistent, in the Kolmogorov sense, measure of dependence. measure of dependence. measure of dependence. © 2016, IFAC (International Automatic Strong Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Stochastic systems, Federation Measures ofofdependence, consistency, Nonlinearity, Non-parametric estimates Keywords: Stochastic systems, Measures of dependence, Strong consistency, Nonlinearity, Non-parametric estimates Keywords: Stochastic systems, Measures of dependence, Strong consistency, Nonlinearity, Non-parametric estimates Keywords: Stochastic systems, Measures of dependence, Strong consistency, Nonlinearity, Non-parametric estimates 1. INTRODUCTION 1. INTRODUCTION 1. 1. INTRODUCTION INTRODUCTION Non-parametric approaches as a tool to model various manufacturNon-parametric approaches as a tool to model various manufacturing process are approaches at present asincreasingly applied. Meanwhile, the Non-parametric aa tool various manufacturNon-parametric tool to to model model various manufacturing process are approaches at present asincreasingly applied. Meanwhile, the range of the are applicability of increasingly the approaches is veryMeanwhile, broad and the ining process at present applied. ing process are at present increasingly applied. Meanwhile, range of the applicability of the approaches is very broad and the involves,offrom one hand side, theapproaches economy efficiency/inefficiency range the applicability of the is very broad and range offrom the applicability of the is very broad and ininvolves, one hand side, theapproaches economy efficiency/inefficiency analysis from to, from another hand the side,economy particularefficiency/inefficiency kinds of manufacturvolves, one hand volves, from oneanother hand side, side, analysis to, from hand the side,economy particularefficiency/inefficiency kinds of manufacturing. Within the former, Data envelopment analysis (DEA) being a analysis to, another hand side, kinds of analysis to, from from anotherData handenvelopment side, particular particular kinds (DEA) of manufacturmanufacturing. Within the former, analysis being a “data-oriented” approach for evaluating the analysis performance of abeing set ofa ing. Within the former, Data envelopment (DEA) ing. Within the approach former, Data envelopment (DEA) “data-oriented” for evaluating the analysis performance of abeing set ofa entities called Decision Making Units whose performanceof is acatego“data-oriented” approach for the set “data-oriented” approachMaking for evaluating evaluating the performance performance set of of entities called Decision Units whose performanceof is acategorized bycalled multiple metrics (Zhu,Units 2015)whose is emphasized as isa categowidely entities Decision Making performance entitiesbycalled Decision Making performance rized multiple metrics (Zhu,Units 2015)whose is emphasized as isa categowidely known and used technique. Kuosmanen et al. (2015) provided an rized multiple metrics 2015) as rized by byand multiple metrics (Zhu, (Zhu, 2015) is isetemphasized emphasized as aa widely widely known used technique. Kuosmanen al. (2015) provided an updatedand andused elaborated presentation of the Convex Nonparametric known technique. Kuosmanen et al. (2015) provided known and technique. Kuosmanen et Convex al. (2015) provided an an updated andused elaborated presentation of the Nonparametric Least Squares method and StochasticofNonparametric Envelopment updated and presentation the updated and elaborated elaborated presentation the Convex Convex Nonparametric Nonparametric Least Squares method and StochasticofNonparametric Envelopment of Data methods, with and the latter beingNonparametric the full integration of DEA Least Squares method Stochastic Envelopment Least Squares method Stochastic Envelopment of Data methods, with and the latter beingNonparametric the full integration of DEA andData Stochastic Frontier Analysis, within the efficiency analysis. of methods, with the latter being the full integration of of Data methods,Frontier with theAnalysis, latter being the the full efficiency integration analysis. of DEA DEA and Stochastic within Using DEA, RayFrontier (2015) proposed a within nonparametric method of measand Stochastic Analysis, the efficiency analysis. and Stochastic Analysis,a within the efficiency analysis. Using DEA, RayFrontier (2015) proposed nonparametric method of measuring output level of a firm. The paper of Huynh et method al. (2015) invesUsing DEA, Ray proposed aa nonparametric of measUsingoutput DEA, level Ray (2015) (2015) proposed nonparametric of invesmeasuring of a firm. The paper of Huynh et method al. (2015) tigatesoutput the evolution offirm. firmThe distributions for entrant manufacturing uring level of a paper of Huynh et al. (2015) invesuring output level of aoffirm. paper of Huynh et al.manufacturing (2015) investigates the evolution firmThe distributions for entrant firms in Canada using functional principalforcomponents to describe tigates evolution of firm entrant tigatesinthe the evolution of functional firm distributions distributions entrant manufacturing manufacturing firms Canada using principalforcomponents to describe these in distributions overfunctional time. Bădin et al. components (2012) extend previous firms Canada using principal to describe firms in Canada using principal to previous describe these distributions overfunctional time. Bădin et al. components (2012) extend studies on conditional efficiency measures by(2012) showing that aprevious careful these distributions over time. Bădin et al. extend these distributions over time. Bădin et al. (2012) extend previous studies on conditional efficiency measures by showing that a careful analysisonofconditional them allows the disentangling of the impactthat of aenvironstudies efficiency measures by showing careful studies onofconditional efficiency measures of by the showing careful analysis them allows the disentangling impactthat of aenvironmental factors onallows the production processofinthe itsimpact two components: analysis of them the disentangling of environanalysisfactors of themonallows the disentangling of environmental the production processofinthe itsimpact two components: impact on the attainable set and/or process impact on the distribution of the mental factors on in its two mental on factors on the the production production in the its distribution two components: components: impact the attainable set and/or process impact on of the efficiency scores. As well, the paper examines the impact ofofenviimpact on the attainable set and/or impact on the distribution the impact on the attainable set the and/or impact on thethe distribution the efficiency scores. As well, paper examines impact ofofenvironmental factors on the production process in a the newimpact two-stage type efficiency scores. As well, the examines of efficiency factors scores. on Asthe well, the paper paperprocess examines of envienvironmental production in a the newimpact two-stage type approach but using conditional measures to avoid the flaws oftype the ronmental factors on the process aa new two-stage ronmentalbut factors onconditional the production production processtoin inavoid newthe two-stage approach using measures flaws oftype the traditional two-stage analysis. The purposetoofavoid the paper of Tran and approach but using conditional measures the flaws of the approach but using conditional measures the flaws of and the traditional two-stage analysis. The purposetoofavoid the paper of Tran Tsionas (2009) is to propose a simple stochastic frontier model with traditional two-stage analysis. purpose of paper Tran and traditional two-stage analysis.a The The purpose of the thefrontier paper of of Tranwith and Tsionas (2009) is to propose simple stochastic model a non-parametric specification for covariates affecting the mean of Tsionas (2009) to stochastic frontier model with (2009) is is specification to propose propose aa simple simple stochastic frontierthe model with aTsionas non-parametric for covariates affecting mean of technical inefficiency. atechnical specification a non-parametric non-parametric specification for for covariates covariates affecting affecting the the mean mean of of inefficiency. In (Tsolas, 2011), a DEA model combined with bootstrapping to technical inefficiency. technical inefficiency. In (Tsolas, 2011), a DEA model combined with bootstrapping to assess performance in mining operations was with presented. For an anIn (Tsolas, 2011), model combined bootstrapping to In (Tsolas, 2011), aainDEA DEA model combined bootstrapping to assess performance mining operations was with presented. For an anthropogenic-hazardous plant, the failure and restoration ofFor which in assess performance in mining operations was presented. an assess performance in mining operations wasrestoration presented.ofFor an ananthropogenic-hazardous plant, the failure and which in the process of operationplant, is of the an anthropogenic hazard, formulas for thropogenic-hazardous failure of in thropogenic-hazardous failure and and restoration restoration of which whichfor in the process of operationplant, is of the an anthropogenic hazard, formulas calculating probabilities of of hazardous and safe hazard, states through charthe of an formulas for the process process probabilities of operation operation is is an anthropogenic anthropogenic formulas for calculating of of hazardous and safe hazard, states through characteristics ofprobabilities reliability and restoration of the plant were obtained by calculating of hazardous safe states through charcalculatingofprobabilities of restoration hazardous and and safe states through characteristics reliability and of the plant were obtained by Sadykhov and Babaev and (2015). Musalam et al. (2013) presented a acteristics of reliability restoration of the plant were obtained acteristics of reliability and restoration of the plant were obtained bya Sadykhov and Babaev (2015). Musalam et al. (2013) presentedby method forand nonparametric trendMusalam modelinget from multidimensional Sadykhov Babaev (2015). al. (2013) presented Sadykhov Babaev (2015). al. (2013) presented aa method forand nonparametric trendMusalam modelinget from multidimensional sensory data so as to use such trends in machinery health prognosmethod for nonparametric trend modeling from multidimensional method data for nonparametric trend modeling from multidimensional sensory so as to use such trends in machinery health prognostics by use of nonparametric time series approaches. The prognosnonparasensory data as trends in health sensory dataofso sononparametric as to to use use such suchtime trends in machinery machinery health tics by use series approaches. The prognosnonparametric model adaptive compact-format linearization control method tics by use of nonparametric time series approaches. The tics by model use ofadaptive nonparametric time series approaches.control The nonparanonparametric compact-format linearization method based on the adaptive data-driven was applied to a linear servo system by metric model compact-format linearization control method metric on model compact-format linearization control method based the adaptive data-driven was applied to a linear servo system by based on the data-driven was applied to a linear servo system based on the data-driven was applied to a linear servo system by by
Cao et al. (2014). In the paper of Schirru et al. (2013), a novel learnCao et al. (2014). In the paper of Schirru et al. (2013), a novel learning methodology is the presented with reference to the application of Cao et paper Schirru et aa novel Cao methodology et al. al. (2014). (2014). In In paper of ofwith Schirru et al. al. (2013), (2013), novel learnlearning is the presented reference to the application of virtual sensors in the semiconductor manufacturing A ing methodology methodology is presented presented with reference reference to the theenvironment. application of of ing is with to application virtual sensors in the semiconductor manufacturing environment. A machine-learning based technique was proposed within real-world virtual sensors in the semiconductor manufacturing environment. A virtual sensors in the semiconductor manufacturing environment. A machine-learning based technique was proposed within real-world manufacturing databased processing by Liwas andproposed Yeh (2009). In the paper of machine-learning technique within real-world machine-learning technique within real-world manufacturing databased processing by Liwas andproposed Yeh (2009). In the paper of Allen et al. (2008) a method for identifying a piecewise-linear apmanufacturing data by and In of manufacturing data processing processing by Li Li and Yeh Yeh (2009). (2009). In the the paper paperapof Allen et al. (2008) a method for identifying a piecewise-linear proximation to the nonlinear forces acting on a piecewise-linear system was presentAllen et al. (2008) a method for identifying apAllen et al. (2008) a methodforces for identifying approximation to the nonlinear acting on a piecewise-linear system was presented and demonstrated using response data on from a micro-cantilever proximation to forces aa system was proximation to the the nonlinear nonlinear forces acting acting system was presentpresented and demonstrated using response data on from a micro-cantilever beam. ed and demonstrated using response data from ed and demonstrated using response data from aa micro-cantilever micro-cantilever beam. The present paper deals with a problem of estimating some beam. beam. The present paper deals with a problem of estimating some measures of stochastic dependence of random processes. GenericalThe paper with of some The present present paper deals deals with aa ofproblem problem of estimating estimating some measures of stochastic dependence random processes. Generically, ordinary product correlation function is the processes. most common measmeasures of stochastic dependence of random Genericalmeasures of product stochastic dependence of random Generically, ordinary correlation function is the processes. most common measure of stochastic dependence but, as known, it does not always ly, product correlation function is common measly, ordinary ordinary product correlation but, function is the the most most common measure of stochastic dependence as known, it does not always properly reflect thedependence actual dependence betweenitrandom processes. ure of stochastic but, as known, does always ure of stochastic but, as known, does not not always properly reflect thedependence actual dependence betweenitrandom processes. Such a circumstance is frequently a significant disadvantage within properly reflect dependence between random properly reflect the the actual actual dependence betweendisadvantage random processes. processes. Such a circumstance is frequently a significant within many aapplications. Iniscontrast to the ordinary disadvantage product correlation, Such circumstance frequently a significant within Such aapplications. circumstanceIniscontrast frequently a significant within many to the ordinary disadvantage product correlation, using smoothed values that are based, for instance, on conditional many applications. In contrast to the ordinary product correlation, many smoothed applications. In contrast the ordinary product using values that are tobased, for instance, on correlation, conditional mathematical expectations leads to morefor complete characteristics of using values are instance, on using smoothed smoothed values that thatleads are based, based, instance, on conditional conditional mathematical expectations to morefor complete characteristics of dependence. Inexpectations particular, leads covariances ofcomplete conditional mathematical mathematical to more characteristics of mathematicalInexpectations to moreofcomplete characteristics of dependence. particular, leads covariances conditional mathematical expectations are known as the dispersionof(referred alsomathematical in the literadependence. In covariances conditional dependence. are In particular, particular, covariances conditional expectations known as the dispersionof(referred alsomathematical in the literature as variance) functions (Rajbman, 1981). In thealso present paper, a expectations are as dispersion (referred in literaexpectations are known known as the the dispersion (referred in the the litera-a ture as variance) functions (Rajbman, 1981). In thealso present paper, measure couplingfunctions k pairs of(Rajbman, random processes isthe introduced. Such a ture as variance) 1981). In present paper, ture as variance) 1981). Inisthe present paper, measure couplingfunctions k pairs of(Rajbman, random processes introduced. Such a may be considered as a further generalization of the dispermeasure coupling kk pairs random processes is Such measure may coupling pairs of of as random processes is introduced. introduced. Such aa be considered a further generalization of the dispersion functions, and the purpose of the paper is to derive the disperstrong measure may be considered as a further generalization of the measure may beand considered as a of further generalization of the sion functions, the purpose the paper is to derive the disperstrong consistency, i.e.and converging withofprobability 1, of their estimates by sion the the is to derive the sion functions, functions, the purpose purpose the paper paper 1, is of to their derive the strong strong consistency, i.e.and converging withofprobability estimates by use of sampled data. These with estimates are applied to deriving samconsistency, i.e. converging probability 1, of their estimates by consistency, i.e. data. converging probability 1, of their estimates by use of sampled These with estimates are applied to deriving sampled analogues of some nonlinear measures of stochastic dependuse of sampled data. These estimates are applied to deriving samuse of sampled data. These estimates are applied to deriving sampled analogues of some nonlinear measures of stochastic dependence of random processes. pled analogues of pled of analogues of some some nonlinear nonlinear measures measures of of stochastic stochastic dependdependence random processes. 2.random PRELIMINARIES AND PROBLEM STATEMENT ence of processes. ence of2.random processes. AND PROBLEM STATEMENT PRELIMINARIES Within2.the dispersion measures of dependence mentioned in the PRELIMINARIES AND PROBLEM STATEMENT PRELIMINARIES AND of PROBLEM STATEMENT Within2.the dispersion measures dependence mentioned in the preceding Section, the following dispersion functions are commonly Within the dispersion measures of dependence mentioned in Within theSection, dispersion measures dispersion of dependence mentioned in the the preceding the following functions are commonly used: preceding Section, the following dispersion functions are commonly preceding Section, the following dispersion functions are commonly used: used:the proper cross-dispersion function of the random processes used: the proper cross-dispersion function of the random processes y(t) and u(s)cross-dispersion function of the random processes the proper the and proper y(t) u(s)cross-dispersion function of the random processes 2 y(t) y(t) and andu(s) u(s) E y (t ) 22 , t s ; E yy ((tt )) yu (( )) E (1) E E y (t ) u ( s) E y (t ) 2 , t s ; (1) u s ( ) y t ( ) y t ( ) yu E E E ( ) , ; (1) t s yu y t ( ) E E E ( ) , ; (1) t s u s ( ) u ( sfunction ) yu the auto-dispersion of the random process u(t) the auto-dispersion function of the random process u(t) the auto-dispersion auto-dispersion function function of of the22random random process process u(t) u(t) the u (t ) Eu (t )the uu (( )) E E , ; (2) t s u t ( ) 2 E E u (t ) u ( s ) Eu (t ) 2 , t s ; (2) u s ( ) u t ( ) u t ( ) uu E E E ( ) , ; (2) t s uu u t ( ) E E E ( ) , ts; (2) u s ( ) uu udispersion ( s ) of the random processes the generalized function the generalized dispersion function of the random processes y(t), generalized z(v), and u(s)dispersion function of the random processes the the y(t), generalized z(v), and u(s)dispersion function of the random processes y(t), z(v), z(v), and and u(s) u(s) y(t),
2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2016, 2016 IFAC 1779Hosting by Elsevier Ltd. All rights reserved. Copyright 2016 responsibility IFAC 1779Control. Peer review©under of International Federation of Automatic Copyright © 1779 10.1016/j.ifacol.2016.07.840 Copyright © 2016 2016 IFAC IFAC 1779
IFAC MIM 2016 1780 June 28-30, 2016. Troyes, France
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
yzu ( , ) E E y (t ) u ( s) E y (t ) E z (v) u ( s ) Ez (v) , t s , v s ,
standing for the conditional mathematical expecta tion, as well as E , for the mathematical expectation. Functions
with E
(3)
(1) to (3) are more complete measures of dependence between random processes in comparison to the conventional product correla-
tion functions, and, in particular, are able to account the stochastic dependence between random processes more representatively. Besides the above ones, the dispersion R-function is the most general kind of the dispersion functions, and for stationary and joint stationary in the strict sense ergodic random processes y (t s v) ,
z (t ) , u (t s ) , w(t) it is of the form
Ez () E E y (t s v) E y () . (4) R yzuw (v, , s ) E E z (t ) w ( t ) u ( t s ) In along to characteristics (1)-(4) presented, the following measure with i ( ni ) being some increasing natural-valued functions in ni , z1,u1 zk ,uk (1,, k ) , coupling k pairs of stationary and ni 0 as ni , i 1,, k . Within this, it is assumed joint stationary in the strict sense ergodic (zero mean, for simplicity i (ni ) , …, of consideration) random processes z1 (t1 ), u1 ( s ) (n ) that the new sets U i keep all the previous elements of the prez k (t k ), u k ( s ) , i t i s , i 1,, k may be introduced as: ceding sets as ni , i 1,, k increase, and k z (t ) (n ) (n ) (n ) E i i z1,u1 zk ,uk ( 1 ,, k ) E (5) u j i u j i1 0, u0 i , ui ( s ) . j i 1 , i 1,, k . (6) j 1,..., i ( ni ) In terms of distribution densities, such a measure is of the form
z1,u1 zk ,uk (1,, k )
( ni ) i ( ni )
u
p( zi , ui , i ) dzi p(u1,, uk )du1 duk , zi p (ui ) i 1 k
where p ( z i , u i , i ) is the joint distribution density of the random processes zi (ti ), ui ( s ) , i t i s , i 1,, k ; p (u i ) is the marginal distribution density of
ui (s ),
i 1,, k ; p (u1 , , u k ) is the joint distribution density of the processes u1 ( s ) , …, u k (s ) .
Thus, the aim of the present paper is deriving an estimate of function z ,u z ,u ( 1 , , k ) (5) by use of sampled observa1 1 k k
tions of the random processes z1 (t1 ), u1 ( s ) , …, z k (t k ), u k ( s ) , i ti s , i 1,, k . Within the problem, the corresponding methodology is a generalization of that of proposed in the fullness of time by Varlaki and Seidl (1983) for estimating the above mentioned dispersion function yu ( ) (1). 3. ESTIMATION TECHNIQUE
n
( ) Let L i , ni 1,2, be sequences of partitions of the real axis R by sequences of real numbers
( ni )
U ( ni ) u0
( ni ) i ( ni )
,, u
( ni )
u0
( ni ) i ( ni )
u
i 1,, k onto i ( ni ) 2 , i 1, , k intervals of the form ( ni )
l0
(n ) , u0 i , l1( ni ) u0( ni ) , u1( ni ) , …,
(n ) (n ) l i( n ) u i( n ) , , i i i i
, ni
In particular, the following partitions given by the sequences of intervals
max
a n (n ) l j 1 , j 0, 1, ,2ni i bi i :
j 1 n ai b ni j n ai b ni (n ) a (n ) i i i i l0 i , ni i , l j i , , ni n bi bi i a (n ) a n j 1,,2ni i bi i , l ia n ni i , , i 1,, k 2 ni i bi i 1 meet the above conditions, with ai , bi , i 1,, k being some
natural-valued constants and bi 1 . k
The direct production k
dimensional space tangular regions
i 1
L i 1
( ni )
R onto
forms a partition of the k-
k
( i (ni ) 2) k-dimensional rec i 1
(n ) (n ) ( n ,, n ) j 1,, j k (u1,, uk ) u j i1 ui u j i , i 1,, k , i 1 i k
and, due to (6),
max j
,
i ji 1,..., i ( ni )
( n ,, n )
V j 1,, j k 0, ni , i 1,, k . 1 k
Here V () stands for the volume of a set, and , for the closing of the set.
( n )
Let u j i ( s ) be the event a value of the random process i (n )
belongs to the interval l j i : i ( ni ) i 1
uj
1780
( ni ) , i
ui u j
i 1,, k ,
ui (s )
IFAC MIM 2016 June 28-30, 2016. Troyes, France
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
( n ) (n ) u j11 ( s ),, u jkk ( s ) be the event a value of the k-dimensional random function u1 ( s ), , u k ( s ) in the variable s belongs to ( n ,, n )
the domain j 1,, j k . Let, again, P{} be the probability of an 1 k event,
I{},
the
indicator
function
of
an
event,
i.e.
1781
In turn, due to the stationarity, joint stationarity, and ergodicity of the processes z i (t i ) and u i (s ) , i 1,, k , there are valid for all
ti , i 1,, k and s with probability 1 T
1 T T
( n )
zi (s i ) I u ji i (s)ds
lim
0
def 1, {} I {} . 0, {}
( n ) E zi (ti ) I u j i ( s ) , i ti s , i 1,, k , i
(8а)
Then, in accordance to Varlaki and Seidl (1983)
T ( n ) 1 ( ni ) I u j ( s )ds P I u j i ( s ) , i 1,, k , T T i i 0
(8b)
lim
( n ) E z (ti ) I u j i ( s ) i
( n ) P u j i ( s) i z (t ) almost surely, i 1,, k , E i (7) ui ( s) with the ratio 0 0 in the left hand part of (7) being considered as ni
lim
T
1 lim T T
( n )
(n )
I u j11 (s),, u jkk 0
( s)ds
( n ) (n ) P I u j 1 ( s ),, u j k ( s ) , i 1,, k . k 1
(8c)
Let now
zero by definition.
( n1,,nk ) ( ,, k ) z1,u1 zk ,uk 1
=
k ( nk ) 1 k
1 ( n1 ) 1
j1 0
jk 0
i 1
( n ) E zi (ti ) I u j i ( s ) i ( ni ) P u j ( s ) i
(n ) P I u (jn1 ) ( s ),, u j k ( s ) , i ti s , i 1,, k , (9a) k 1
( n1 ,, nk ,T ) (1,, k ) z1,u1 zk ,uk 1 ( n1 ) 1 k ( nk ) 1 k 1 T j1 0 jk 0 i 1
T
( n ) zi ( s i ) I x j i ( s )ds T i I u ( n1 ) ( s ),, u ( nk ) ( s)ds , t s , i 1,, k . (9b) 0 j1 i i jk T ( n ) 0 I x j i ( s )ds i 0
By virtue of (7), (8)
Relationships (9) imply, as partial cases, strongly consistent estimates of all kinds of the dispersion functions that are, in turn, particular cases of the dispersion R-function. As to the dispersion Rfunction, R yzuw (v, , s ) y ,u z , w (v, , s ) ,
( n ,, nk ) lim 1 (1,, k ) n1 z1,u1 z k ,u k nk
z1 ,u1 z k ,u k (1,, k ) ,
( n ,, nk ,T ) lim 1 (1,, k ) T z1 ,u1 z k ,u k
T
1 ( n1 ) 1 2 ( n2 ) 1
j1 0
j2 0
1 T
0
yn1,u,n2z,,Tw (v, , s )
(n ) 1
T
y ( s v) I x j 1 ( s ) d T
I x j11 ( s)d 0
n , n and lim y1,u 2z , w (v, , s ) y ,u z , w (v, , s ) , n1 n2
it directly follows, assuming the processes y(t) and z(t) to be zero mean ones,
( n1 ,, nk ) ( ,, k ) . z1,u1 z k ,uk 1
(n )
z( )I w j22 ( )d 0
(n )
T
I w j22 ( )d (n )
T
I u j11 ( s), w j22 ( )d , (n )
0
0
n ,n ,T n ,n lim y1,u 2z , w (v, , s ) y1,u 2z , w (v, , s ) . T
1781
(n )
IFAC MIM 2016 1782 June 28-30, 2016. Troyes, France
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
4. APPLICATION TO ESTIMATING CONSISTENT MEASURES OF DEPENDENCE Within numerous of applications, e.g. system identification, consistent measures of dependence of random processes play an important role. As known, following to the Kolmogorov terminology, a measure of dependence of two random processes is said to be consistent if it vanishes if and only if the random processes are stochastically independent. For a case of system identification, using consistent measures of stochastic dependence of random processes is a natural way to establish an approximate empirical input/output relationship describing the system model. Hence, a significant feature of such an approach to system identification is just the choice of an appropriate measure of stochastic dependence between the input and output processes. Among various measures of dependence, the product correlation, as it has already been noted, is well known and commonly used. However, the ordinary product correlation of random values Y and X may vanish even if the random values are completely dependent, i.e. if there exists a deterministic function f () such that Y f (X ) with probability 1 (Rajbman, 1981, Rényi, 1959). Rényi (1959) has presented seven axioms that were recognized to be the most natural to describe a measure of dependence X , Y between two random values X and Y. A) X , Y is defined for any pair of random values X and Y, neither of them being constant with probability 1. B) X , Y = Y , X . C) D) E)
0 X ,Y 1 . X , Y 0 if and only if X and Y are independent. X , Y 1 if there exists a strict dependence between X and
Y, i.e. either Y (X ) or X (Y ) where and are Borel-measurable functions. F) If a Borel-measurable functions and map the real axis in a one-to-one way onto itself, ( X ), (Y ) X , Y . G) If the joint distribution of X and Y is normal, then X , Y r X , Y , where r X , Y is the ordinary correlation coefficient of X and Y. In along with the ordinary correlation coefficient, commonly used measures of dependence are the correlation ratio
var E Y X ( X ,Y ) var (Y ) with nonzero variance var(Y), and the maximal correlation coefficient
S X , Y sup
covB (Y ), C ( X )
B , C var B (Y ) var C ( X )
with
the
variances
,
var B (Y ) 0, var C ( X ) 0 ,
functions (Rajbman, 1981) may be considered as a modification and extension of the correlation ratio. In turn, the maximal correlation coefficient is transformed to the following function
S yx ( ) sup
, t s B , C var B y (t ) var C x ( s ) The functional S yx ( ) is referred as the maximal correlation function of the random processes y (t ) and x( s ) . The notion of the maximal correlation as a measure of dependence has been originally introduced and investigated for random values by Rényi (1959), Sarmanov (1963a) and extended to random processes by Sarmanov (1963b). A significant disadvantage of S X , Y is concerned with considerable computational difficulties under its calculation by use of observed sampled data x k , y k , k 1,2, of the random val-
ues X , Y . These difficulties are of course resides in the maximal
correlation function S yx ( ) . To avoid the disadvantage and simultaneously to use a consistent measure of dependence within a problem statement, some restrictive assumptions with respect to such a measure of dependence are needed to be imposed. In particular, for sign-constant random processes (Ovsepyan and Lepsky, 1987), i.e. the processes that do not alter their signs with probability 1, the results of (Ovsepyan and Lepsky, 1987) may be applied. Namely, the quantity yx defined as
yx
2 var E y (t ) x( s) 4
2C yx , vary 2 (t ) 2C yx
(10)
where 2 y (t ) var , C yx cov E y (t ) . x( s ) x( s)
meets the above axioms A to G except B for any sign-constant random processes y (t ) and x (s ) with non-zero variances and finite fourth moments. One should be noted that axiom B (symmetry) does not play a significant role within majority of applications. In contrast to the maximal correlation, yx can be effectively calculated using sampled data in accordance to the methodology derived in the preceding Section. In terms of probability distribution densities, the corresponding values in (10) are (generically) of the form
E y (t ) yp y | x dy , x( s )
and
supremum being taken over all Borel-measurable functions B and
C , and
covB y (t ) , C x( s )
B B , C C .
However, the only S X , Y has been shown by Rényi (1959) to
satisfy all the above axioms, while r X , Y and X , Y do not, in particularly axioms D, E, F are not met for the correlation coefficient, and axioms D, F are not met for the correlation ratio. Under investigation of random processes, the above coefficients are transformed to the corresponding functions. As pointed out above, the ordinary product correlation functions are not the exhaustive tool to be used within identification problems. The above dispersion 1782
var y (t ) x( s ) 2 y (t ) x( s ) var y (t ) E x( s )
2
y yp y | x dy p y | x dy ,
(11)
(12)
IFAC MIM 2016 June 28-30, 2016. Troyes, France
var E y (t ) x( s )
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
2
y (t ) EE E y (t ) x ( s )
U ( n) u 0( n) , , u( n()n) 0 u 0( n) u( n()n)
2
yp y | x dy E y (t ) p( x)dx , (13) p y, x, and, conventionally, p y | x is the conditional disp( x)
p y, x, , t s stand for the joint and marginal distribution densities of the random processes x (s ) and y (t ) correspondingly.
Remark. Of course, under consideration of the sign-constant random processes, say, positive ones, the lower integration limits in the above integrals expressions (11) to (13) are automatically substituted by zeros. In the above yx in (10), the quantities
and C yx in accordance to expres
sions (11) to (13) may be represented as follows
2 var E y (t ) x( s)
2 E E y (t ) x( s )
2 E y (t )
2
2
2
(14)
2 2 y (t ) E C yx E E y (t ) x( s) x( s) 2 4 y (t ) y (t ) 2 E y (t ) E E E E x( s) x( s)
As above, (n) is some increasing natural-valued function in n ,
n
( n)
0 as n , as well as it is assumed that the new sets
U (n) keep all the previous elements of the preceding sets as n
increases, and
u j max j
( n)
u( n()n) , n In particular, the following partition given by the sequences of in(n)
tervals l j , j 0, 1,,2n a b n ,
l0( n) 0 , exp n a , l (jn ) ( n ) ( j 1), ( n ) ( j ) ,
a n j n ab n , if j n b 0 exp def n bn (n) ( j ) b , j n ab n j n ab n 1, if 0 bn bn
Due to (16),
(n)
l j max j
0, n .
Here stands for the length of an interval, and , for the closing of a set. Thus, in accordance to the above general case, the fol(15)
sampled data is to estimate the summands lyx , l 1,,4 in (14) and (15) containing the corresponding conditional means. In turn, to estimate the functions lyx , l 1,,4 thus introduced,
lowing estimate
(zn),u z 1
one
should
1
k ,u
( ) of z1,u zk ,u ( ) may be
( )
Pu
( n) 1 k E z (t ) I u ( n ) ( s ) i i j
j 0 i 1
P u (jn) ( s)
(n) j (s)
Ezi (ti ) I u (jn) (s) k
( n ) 1
consider
L , n 1,2, as a sequence of a partition of the real positive semi axis R+ (rather than R) by sequences of real numbers
k ,u
The only difference in comparison to the general case of estimating z1,u1 z ,u ( 1 ,, k ) is that, for z1,u zk ,u ( ) coupling processes,
(zn),u z
applied
the approach presented above may be applied with respect to the function z1,u zk ,u ( ) .
(n)
(16)
meets the above conditions, with a , b being some natural-valued constants, and b 1 .
Thus, the entity of the problem of estimating yx by use of
random
.
j 1,..., ( n )
2 yx 3 yx Ey 2 (t ) 4 yx 3 yx 2 .
sign-constant
u (jn)1 0, u 0( n) 0,
j 1,..., ( n )
2
the
j 1, ,2n a b n ,
2 y (t ) E Ε x( s )
k k
l0( n) 0, u0( n) , l1( n ) u0( n ) ,u1( n) , , l( n( )n) u( n()n ) , .
( n) j 1, ,2n a b n , l a n n a , , where 2 n b 1
1 yx Ey 2 (t ) ,
onto (n) 2 intervals of the form
tribution density of y (t ) with respect to x (s ) ; and p(x) , p( y) ,
2 var E y (t ) x( s)
1783
j 0
and
1783
i 1
Pu
k 1 (n) j (s)
,
(17)
IFAC MIM 2016 1784 June 28-30, 2016. Troyes, France
(zn,,Tu)z 1
k ,u
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
( )
T ( n) z ( s ) I x j ( s)ds i ( n) 1 1 i 1 0 . k 1 T T j 0 ( n) I x j ( s )ds 0 k
(18)
Thus, by virtue of (7), (8), (17), (18),
lim lim (zn,,Tu) z ,u ( ) z1,u zk ,u ( ) almost surely. 1 k
n T
Such a reasoning directly implies the corresponding formulae to estimate the functions lyx , l 1,,4 . Namely: 1) by imposing z1 (t ) z 2 (t ) y 2 (t ) , u ( s ) x( s ) ,
z1,u z2 ,u ( )
( ) 2 2 y , x y , x 1 yx ( ) lim lim n,2T ( ) almost surely; n T y , x y 2 , x 2) by imposing z1 (t ) z 2 (t ) y (t ), z3 y 2 (t ) , u ( s )
z1,u z2 ,u z3 ,u ( )
y, x y, x y 2 , x
x( s ) ,
( )
2 yx ( ) lim lim n, T ( ) almost surely; n T y , x y , x y 2 , x
3) by imposing z1 (t ) z 2 (t ) y (t ) , u ( s ) x( s ) ,
z1,u z2 ,u ( ) u , x u , x ( )
3 yx lim lim (un,,xT)u , x ( ) almost surely; n T
4) by
imposing
u ( s ) x( s ) ,
z1 (t ) z 2 (t ) z 3 (t ) z 4 (t ) y (t ) ,
z1,u z2 ,u z3 ,u z4 ,u ( ) y , x y, x y , x y, x ( )
4 yx lim lim (yn,,xT) y , x y , x y , x ( ) almost surely. n T
Thus obtained, the formulae are applied to constructing the sampled estimates of measure of dependence (10). 5. CONCLUSIONS A problem of estimating some measures of stochastic dependence of random processes has been considered. In particular, a measure coupling k pairs of stationary and joint stationary in the strict sense ergodic random processes has been introduced, and strongly consistent, i.e. converging with probability 1, nonparametric estimates of such a measure have been obtained. These estimates has been applied to deriving sampled analogues of a nonlinear measure of stochastic dependence of sign-constant random processes, for which the measure of dependence is the consistent one in the Kolmogorov sense, i.e. vanishing if and only if the random processes are stochastically independent. REFERENCES Allen, M.S., Sumali, H., and D.S. Epp (2008). “Piecewise-linear restoring force surfaces for semi-nonparametric identification of nonlinear systems”, Nonlinear Dynamics, vol. 54, no. 1, pp. 123-135. Bădina, L., Daraioc, C., and L. Simar (2012). “How to measure the impact of environmental factors in a nonparametric production
model”, European Journal of Operational Research, vol. 223, no. 3, pp. 818-833. Cao Rong-Min, Hou Zhong-Sheng, and Hui-Xing Zhou (2014). “Data-driven Nonparametric Model Adaptive Precision Control for Linear Servo Systems”, International Journal of Automation and Computing, vol. 11, no. 5, pp. 517-526. Huynh, K.P., Jacho-Chávez, D.T., Petrunia, R.J., and M.C. Voia (2015). “A nonparametric analysis of firm size, leverage and labour productivity distribution dynamics”, Empirical Economics, vol. 48, no. 1, pp. 337-360. Kuosmanen, T., Johnson, A., and A. Saastamoinen (2015). “Stochastic Nonparametric Approach to Efficiency Analysis: A Unified Framework”, Data Envelopment Analysis. A Handbook of Models and Methods. Springer, International Series in Operations Research & Management Science, vol. 221, pp. 191-244. Li, Der-Chang and Chun-Wu Yeh (2008). “A non-parametric learning algorithm for small manufacturing data sets”, Expert Systems with Applications, vol. 34, no. 1, pp. 391-398. Mosallam, A., Medjaher, K., and N. Zerhouni (2013). “Nonparametric time series modelling for industrial prognostics and health management”, The International Journal of Advanced Manufacturing Technology, vol. 69, no. 5, pp. 1685-1699. Ovsepyan, F.A. and D.M. Lepsky (1987). “On identification of nonlinear nonparametric control systems”, 10th World Cong. Aut. Cont. Prepr., vol. 10. Rajbman, N.S. (1981). “Extensions to nonlinear and minimax approaches'', In: Trends and Progress in System Identification, Ed. by P. Eykhoff, Pergamon Press, Oxford, pp. 185-237. Ray, S.C. (2015). “Nonparametric measures of scale economies and capacity utilization: An application to U.S. manufacturing”, European Journal of Operational Research, vol. 245, no. 2, pp. 602-611. Rényi, A. (1959). “On measures of dependence”, Acta Math. Hung., vol. 10, no. 3-4, pp. 441-451. Sadykhov, G.S. and I.A. Babaev (2015). “Nonparametric assessments and limiting probability values of the hazardous and safe states of a technogenic-hazardous object”, Journal of Machinery Manufacture and Reliability, vol. 44, no. 3, pp. 298-304. Sarmanov, O.V. (1963a). “The maximal correlation coefficient (nonsymmetric case)”, Sel. Trans. Math. Statist. Probability, vol. 4, pp. 207-210. Sarmanov, O.V. (1963b). “Investigation of stationary Markov processes by the method of eigenfunction expansion”, Sel. Trans. Math. Statist. Probability, vol. 4, pp. 245-269. Schirru, A., Pampuri, S. De Luca, C., and G. De Nicolao (2013). “Virtual Sensors for Semiconductor Manufacturing: A Nonparametric Approach - Exploiting Information Theoretic Learning and Kernel Machines”, Informatics in Control, Automation and Robotics. Springer, Lecture Notes in Electrical Engineering, vol. 174, pp. 175-193. Tran, K.C. and E.G. Tsionas (2009). “Estimation of nonparametric inefficiency effects stochastic frontier models with an application to British manufacturing”, Economic Modelling, vol. 26, no. 5, pp. 904-909. Tsolas, I.E. (2011). “Performance assessment of mining operations using nonparametric production analysis: A bootstrapping approach in DEA, Resources Policy, vol. 36, no. 2, pp. 159-167. Varlaki, P. and L. Seidl (1983). “Estimating the variance functions of ergodic stochastic processes”, Automation and Remote Control, vol. 44, no. 6, part 1, pp. 737-739. Zhu, J. (Ed.) (2015). Data Envelopment Analysis. A Handbook of Models and Methods. Springer, International Series in Operations Research & Management Science, vol. 221, 465 p.
1784
ScienceDirect
IFAC-PapersOnLine 49-12 (2016) 1779–1784
Non-parametric Non-parametric Procedures Procedures under under Estimating Estimating Stochastic Stochastic Dependencies Dependencies Non-parametric Procedures under Estimating Stochastic Dependencies Non-parametric Procedures under Estimating Stochastic Dependencies in Nonlinear Systems in Nonlinear Systems in Nonlinear Systems in Nonlinear Systems K.R. K.R. Chernyshov Chernyshov K.R. K.R. Chernyshov Chernyshov V.A. Trapeznikov Institute of Control Sciences, 65 Profsoyuznaya, Profsoyuznaya, Moscow Moscow 117997, 117997, Russia Russia V.A. Trapeznikov Institute of Control Sciences, 65 (e-mail: [email protected]) V.A. Trapeznikov Institute of Control Sciences, 65 Profsoyuznaya, Moscow 117997, Russia V.A. Trapeznikov Institute of Control Sciences, 65 Profsoyuznaya, Moscow 117997, Russia (e-mail: [email protected]) (e-mail: (e-mail: [email protected]) [email protected])
Abstract. In the paper, a measure of dependence coupling k pairs of random processes is introduced. Such a measure, Abstract. In the paper, a measure of dependence coupling k pairs of random processes is introduced. Such a measure, based on using conditional mathematical expectations of the processes, may processes be considered as a further generalization Abstract. In paper, of coupling kk pairs of is Such aa measure, Abstract. In the theconditional paper, aa measure measure of dependence dependence coupling pairs of random random is introduced. introduced. Such measure, based on using mathematical expectations of the processes, may processes be considered as a further generalization of the on dispersion (variance)mathematical functions. Convergence with probability 1 of nonparametric estimates of such a measure based using conditional expectations of the processes, may be considered as a further generalization based using conditional expectationswith of the processes, may be considered as a further generalization of the on dispersion (variance)mathematical functions. Convergence probability 1 of nonparametric estimates of such a measure is derived using (variance) sampled data. TheseConvergence estimates are applied to deriving sampled analogues of some nonlinear of dispersion functions. with probability 11 of estimates such measure of the the dispersion functions. withapplied probability of nonparametric nonparametric estimatesofof of some such aanonlinear measure is derived using (variance) sampled data. TheseConvergence estimates are to deriving sampled analogues measures ofusing stochastic dependence of random processes, in particular, to asampled consistent, in the Kolmogorov sense, is derived sampled data. These estimates are applied to deriving analogues of some is derivedofusing sampled data. These estimates are applied to deriving analogues of some nonlinear nonlinear measures stochastic dependence of random processes, in particular, to asampled consistent, in the Kolmogorov sense, measure ofofdependence. measures stochastic dependence of random processes, in particular, to a consistent, in the Kolmogorov sense, measures of stochastic dependence of random processes, in particular, to a consistent, in the Kolmogorov sense, measure of dependence. measure of dependence. measure of dependence. © 2016, IFAC (International Automatic Strong Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Stochastic systems, Federation Measures ofofdependence, consistency, Nonlinearity, Non-parametric estimates Keywords: Stochastic systems, Measures of dependence, Strong consistency, Nonlinearity, Non-parametric estimates Keywords: Stochastic systems, Measures of dependence, Strong consistency, Nonlinearity, Non-parametric estimates Keywords: Stochastic systems, Measures of dependence, Strong consistency, Nonlinearity, Non-parametric estimates 1. INTRODUCTION 1. INTRODUCTION 1. 1. INTRODUCTION INTRODUCTION Non-parametric approaches as a tool to model various manufacturNon-parametric approaches as a tool to model various manufacturing process are approaches at present asincreasingly applied. Meanwhile, the Non-parametric aa tool various manufacturNon-parametric tool to to model model various manufacturing process are approaches at present asincreasingly applied. Meanwhile, the range of the are applicability of increasingly the approaches is veryMeanwhile, broad and the ining process at present applied. ing process are at present increasingly applied. Meanwhile, range of the applicability of the approaches is very broad and the involves,offrom one hand side, theapproaches economy efficiency/inefficiency range the applicability of the is very broad and range offrom the applicability of the is very broad and ininvolves, one hand side, theapproaches economy efficiency/inefficiency analysis from to, from another hand the side,economy particularefficiency/inefficiency kinds of manufacturvolves, one hand volves, from oneanother hand side, side, analysis to, from hand the side,economy particularefficiency/inefficiency kinds of manufacturing. Within the former, Data envelopment analysis (DEA) being a analysis to, another hand side, kinds of analysis to, from from anotherData handenvelopment side, particular particular kinds (DEA) of manufacturmanufacturing. Within the former, analysis being a “data-oriented” approach for evaluating the analysis performance of abeing set ofa ing. Within the former, Data envelopment (DEA) ing. Within the approach former, Data envelopment (DEA) “data-oriented” for evaluating the analysis performance of abeing set ofa entities called Decision Making Units whose performanceof is acatego“data-oriented” approach for the set “data-oriented” approachMaking for evaluating evaluating the performance performance set of of entities called Decision Units whose performanceof is acategorized bycalled multiple metrics (Zhu,Units 2015)whose is emphasized as isa categowidely entities Decision Making performance entitiesbycalled Decision Making performance rized multiple metrics (Zhu,Units 2015)whose is emphasized as isa categowidely known and used technique. Kuosmanen et al. (2015) provided an rized multiple metrics 2015) as rized by byand multiple metrics (Zhu, (Zhu, 2015) is isetemphasized emphasized as aa widely widely known used technique. Kuosmanen al. (2015) provided an updatedand andused elaborated presentation of the Convex Nonparametric known technique. Kuosmanen et al. (2015) provided known and technique. Kuosmanen et Convex al. (2015) provided an an updated andused elaborated presentation of the Nonparametric Least Squares method and StochasticofNonparametric Envelopment updated and presentation the updated and elaborated elaborated presentation the Convex Convex Nonparametric Nonparametric Least Squares method and StochasticofNonparametric Envelopment of Data methods, with and the latter beingNonparametric the full integration of DEA Least Squares method Stochastic Envelopment Least Squares method Stochastic Envelopment of Data methods, with and the latter beingNonparametric the full integration of DEA andData Stochastic Frontier Analysis, within the efficiency analysis. of methods, with the latter being the full integration of of Data methods,Frontier with theAnalysis, latter being the the full efficiency integration analysis. of DEA DEA and Stochastic within Using DEA, RayFrontier (2015) proposed a within nonparametric method of measand Stochastic Analysis, the efficiency analysis. and Stochastic Analysis,a within the efficiency analysis. Using DEA, RayFrontier (2015) proposed nonparametric method of measuring output level of a firm. The paper of Huynh et method al. (2015) invesUsing DEA, Ray proposed aa nonparametric of measUsingoutput DEA, level Ray (2015) (2015) proposed nonparametric of invesmeasuring of a firm. The paper of Huynh et method al. (2015) tigatesoutput the evolution offirm. firmThe distributions for entrant manufacturing uring level of a paper of Huynh et al. (2015) invesuring output level of aoffirm. paper of Huynh et al.manufacturing (2015) investigates the evolution firmThe distributions for entrant firms in Canada using functional principalforcomponents to describe tigates evolution of firm entrant tigatesinthe the evolution of functional firm distributions distributions entrant manufacturing manufacturing firms Canada using principalforcomponents to describe these in distributions overfunctional time. Bădin et al. components (2012) extend previous firms Canada using principal to describe firms in Canada using principal to previous describe these distributions overfunctional time. Bădin et al. components (2012) extend studies on conditional efficiency measures by(2012) showing that aprevious careful these distributions over time. Bădin et al. extend these distributions over time. Bădin et al. (2012) extend previous studies on conditional efficiency measures by showing that a careful analysisonofconditional them allows the disentangling of the impactthat of aenvironstudies efficiency measures by showing careful studies onofconditional efficiency measures of by the showing careful analysis them allows the disentangling impactthat of aenvironmental factors onallows the production processofinthe itsimpact two components: analysis of them the disentangling of environanalysisfactors of themonallows the disentangling of environmental the production processofinthe itsimpact two components: impact on the attainable set and/or process impact on the distribution of the mental factors on in its two mental on factors on the the production production in the its distribution two components: components: impact the attainable set and/or process impact on of the efficiency scores. As well, the paper examines the impact ofofenviimpact on the attainable set and/or impact on the distribution the impact on the attainable set the and/or impact on thethe distribution the efficiency scores. As well, paper examines impact ofofenvironmental factors on the production process in a the newimpact two-stage type efficiency scores. As well, the examines of efficiency factors scores. on Asthe well, the paper paperprocess examines of envienvironmental production in a the newimpact two-stage type approach but using conditional measures to avoid the flaws oftype the ronmental factors on the process aa new two-stage ronmentalbut factors onconditional the production production processtoin inavoid newthe two-stage approach using measures flaws oftype the traditional two-stage analysis. The purposetoofavoid the paper of Tran and approach but using conditional measures the flaws of the approach but using conditional measures the flaws of and the traditional two-stage analysis. The purposetoofavoid the paper of Tran Tsionas (2009) is to propose a simple stochastic frontier model with traditional two-stage analysis. purpose of paper Tran and traditional two-stage analysis.a The The purpose of the thefrontier paper of of Tranwith and Tsionas (2009) is to propose simple stochastic model a non-parametric specification for covariates affecting the mean of Tsionas (2009) to stochastic frontier model with (2009) is is specification to propose propose aa simple simple stochastic frontierthe model with aTsionas non-parametric for covariates affecting mean of technical inefficiency. atechnical specification a non-parametric non-parametric specification for for covariates covariates affecting affecting the the mean mean of of inefficiency. In (Tsolas, 2011), a DEA model combined with bootstrapping to technical inefficiency. technical inefficiency. In (Tsolas, 2011), a DEA model combined with bootstrapping to assess performance in mining operations was with presented. For an anIn (Tsolas, 2011), model combined bootstrapping to In (Tsolas, 2011), aainDEA DEA model combined bootstrapping to assess performance mining operations was with presented. For an anthropogenic-hazardous plant, the failure and restoration ofFor which in assess performance in mining operations was presented. an assess performance in mining operations wasrestoration presented.ofFor an ananthropogenic-hazardous plant, the failure and which in the process of operationplant, is of the an anthropogenic hazard, formulas for thropogenic-hazardous failure of in thropogenic-hazardous failure and and restoration restoration of which whichfor in the process of operationplant, is of the an anthropogenic hazard, formulas calculating probabilities of of hazardous and safe hazard, states through charthe of an formulas for the process process probabilities of operation operation is is an anthropogenic anthropogenic formulas for calculating of of hazardous and safe hazard, states through characteristics ofprobabilities reliability and restoration of the plant were obtained by calculating of hazardous safe states through charcalculatingofprobabilities of restoration hazardous and and safe states through characteristics reliability and of the plant were obtained by Sadykhov and Babaev and (2015). Musalam et al. (2013) presented a acteristics of reliability restoration of the plant were obtained acteristics of reliability and restoration of the plant were obtained bya Sadykhov and Babaev (2015). Musalam et al. (2013) presentedby method forand nonparametric trendMusalam modelinget from multidimensional Sadykhov Babaev (2015). al. (2013) presented Sadykhov Babaev (2015). al. (2013) presented aa method forand nonparametric trendMusalam modelinget from multidimensional sensory data so as to use such trends in machinery health prognosmethod for nonparametric trend modeling from multidimensional method data for nonparametric trend modeling from multidimensional sensory so as to use such trends in machinery health prognostics by use of nonparametric time series approaches. The prognosnonparasensory data as trends in health sensory dataofso sononparametric as to to use use such suchtime trends in machinery machinery health tics by use series approaches. The prognosnonparametric model adaptive compact-format linearization control method tics by use of nonparametric time series approaches. The tics by model use ofadaptive nonparametric time series approaches.control The nonparanonparametric compact-format linearization method based on the adaptive data-driven was applied to a linear servo system by metric model compact-format linearization control method metric on model compact-format linearization control method based the adaptive data-driven was applied to a linear servo system by based on the data-driven was applied to a linear servo system based on the data-driven was applied to a linear servo system by by
Cao et al. (2014). In the paper of Schirru et al. (2013), a novel learnCao et al. (2014). In the paper of Schirru et al. (2013), a novel learning methodology is the presented with reference to the application of Cao et paper Schirru et aa novel Cao methodology et al. al. (2014). (2014). In In paper of ofwith Schirru et al. al. (2013), (2013), novel learnlearning is the presented reference to the application of virtual sensors in the semiconductor manufacturing A ing methodology methodology is presented presented with reference reference to the theenvironment. application of of ing is with to application virtual sensors in the semiconductor manufacturing environment. A machine-learning based technique was proposed within real-world virtual sensors in the semiconductor manufacturing environment. A virtual sensors in the semiconductor manufacturing environment. A machine-learning based technique was proposed within real-world manufacturing databased processing by Liwas andproposed Yeh (2009). In the paper of machine-learning technique within real-world machine-learning technique within real-world manufacturing databased processing by Liwas andproposed Yeh (2009). In the paper of Allen et al. (2008) a method for identifying a piecewise-linear apmanufacturing data by and In of manufacturing data processing processing by Li Li and Yeh Yeh (2009). (2009). In the the paper paperapof Allen et al. (2008) a method for identifying a piecewise-linear proximation to the nonlinear forces acting on a piecewise-linear system was presentAllen et al. (2008) a method for identifying apAllen et al. (2008) a methodforces for identifying approximation to the nonlinear acting on a piecewise-linear system was presented and demonstrated using response data on from a micro-cantilever proximation to forces aa system was proximation to the the nonlinear nonlinear forces acting acting system was presentpresented and demonstrated using response data on from a micro-cantilever beam. ed and demonstrated using response data from ed and demonstrated using response data from aa micro-cantilever micro-cantilever beam. The present paper deals with a problem of estimating some beam. beam. The present paper deals with a problem of estimating some measures of stochastic dependence of random processes. GenericalThe paper with of some The present present paper deals deals with aa ofproblem problem of estimating estimating some measures of stochastic dependence random processes. Generically, ordinary product correlation function is the processes. most common measmeasures of stochastic dependence of random Genericalmeasures of product stochastic dependence of random Generically, ordinary correlation function is the processes. most common measure of stochastic dependence but, as known, it does not always ly, product correlation function is common measly, ordinary ordinary product correlation but, function is the the most most common measure of stochastic dependence as known, it does not always properly reflect thedependence actual dependence betweenitrandom processes. ure of stochastic but, as known, does always ure of stochastic but, as known, does not not always properly reflect thedependence actual dependence betweenitrandom processes. Such a circumstance is frequently a significant disadvantage within properly reflect dependence between random properly reflect the the actual actual dependence betweendisadvantage random processes. processes. Such a circumstance is frequently a significant within many aapplications. Iniscontrast to the ordinary disadvantage product correlation, Such circumstance frequently a significant within Such aapplications. circumstanceIniscontrast frequently a significant within many to the ordinary disadvantage product correlation, using smoothed values that are based, for instance, on conditional many applications. In contrast to the ordinary product correlation, many smoothed applications. In contrast the ordinary product using values that are tobased, for instance, on correlation, conditional mathematical expectations leads to morefor complete characteristics of using values are instance, on using smoothed smoothed values that thatleads are based, based, instance, on conditional conditional mathematical expectations to morefor complete characteristics of dependence. Inexpectations particular, leads covariances ofcomplete conditional mathematical mathematical to more characteristics of mathematicalInexpectations to moreofcomplete characteristics of dependence. particular, leads covariances conditional mathematical expectations are known as the dispersionof(referred alsomathematical in the literadependence. In covariances conditional dependence. are In particular, particular, covariances conditional expectations known as the dispersionof(referred alsomathematical in the literature as variance) functions (Rajbman, 1981). In thealso present paper, a expectations are as dispersion (referred in literaexpectations are known known as the the dispersion (referred in the the litera-a ture as variance) functions (Rajbman, 1981). In thealso present paper, measure couplingfunctions k pairs of(Rajbman, random processes isthe introduced. Such a ture as variance) 1981). In present paper, ture as variance) 1981). Inisthe present paper, measure couplingfunctions k pairs of(Rajbman, random processes introduced. Such a may be considered as a further generalization of the dispermeasure coupling kk pairs random processes is Such measure may coupling pairs of of as random processes is introduced. introduced. Such aa be considered a further generalization of the dispersion functions, and the purpose of the paper is to derive the disperstrong measure may be considered as a further generalization of the measure may beand considered as a of further generalization of the sion functions, the purpose the paper is to derive the disperstrong consistency, i.e.and converging withofprobability 1, of their estimates by sion the the is to derive the sion functions, functions, the purpose purpose the paper paper 1, is of to their derive the strong strong consistency, i.e.and converging withofprobability estimates by use of sampled data. These with estimates are applied to deriving samconsistency, i.e. converging probability 1, of their estimates by consistency, i.e. data. converging probability 1, of their estimates by use of sampled These with estimates are applied to deriving sampled analogues of some nonlinear measures of stochastic dependuse of sampled data. These estimates are applied to deriving samuse of sampled data. These estimates are applied to deriving sampled analogues of some nonlinear measures of stochastic dependence of random processes. pled analogues of pled of analogues of some some nonlinear nonlinear measures measures of of stochastic stochastic dependdependence random processes. 2.random PRELIMINARIES AND PROBLEM STATEMENT ence of processes. ence of2.random processes. AND PROBLEM STATEMENT PRELIMINARIES Within2.the dispersion measures of dependence mentioned in the PRELIMINARIES AND PROBLEM STATEMENT PRELIMINARIES AND of PROBLEM STATEMENT Within2.the dispersion measures dependence mentioned in the preceding Section, the following dispersion functions are commonly Within the dispersion measures of dependence mentioned in Within theSection, dispersion measures dispersion of dependence mentioned in the the preceding the following functions are commonly used: preceding Section, the following dispersion functions are commonly preceding Section, the following dispersion functions are commonly used: used:the proper cross-dispersion function of the random processes used: the proper cross-dispersion function of the random processes y(t) and u(s)cross-dispersion function of the random processes the proper the and proper y(t) u(s)cross-dispersion function of the random processes 2 y(t) y(t) and andu(s) u(s) E y (t ) 22 , t s ; E yy ((tt )) yu (( )) E (1) E E y (t ) u ( s) E y (t ) 2 , t s ; (1) u s ( ) y t ( ) y t ( ) yu E E E ( ) , ; (1) t s yu y t ( ) E E E ( ) , ; (1) t s u s ( ) u ( sfunction ) yu the auto-dispersion of the random process u(t) the auto-dispersion function of the random process u(t) the auto-dispersion auto-dispersion function function of of the22random random process process u(t) u(t) the u (t ) Eu (t )the uu (( )) E E , ; (2) t s u t ( ) 2 E E u (t ) u ( s ) Eu (t ) 2 , t s ; (2) u s ( ) u t ( ) u t ( ) uu E E E ( ) , ; (2) t s uu u t ( ) E E E ( ) , ts; (2) u s ( ) uu udispersion ( s ) of the random processes the generalized function the generalized dispersion function of the random processes y(t), generalized z(v), and u(s)dispersion function of the random processes the the y(t), generalized z(v), and u(s)dispersion function of the random processes y(t), z(v), z(v), and and u(s) u(s) y(t),
2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2016, 2016 IFAC 1779Hosting by Elsevier Ltd. All rights reserved. Copyright 2016 responsibility IFAC 1779Control. Peer review©under of International Federation of Automatic Copyright © 1779 10.1016/j.ifacol.2016.07.840 Copyright © 2016 2016 IFAC IFAC 1779
IFAC MIM 2016 1780 June 28-30, 2016. Troyes, France
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
yzu ( , ) E E y (t ) u ( s) E y (t ) E z (v) u ( s ) Ez (v) , t s , v s ,
standing for the conditional mathematical expecta tion, as well as E , for the mathematical expectation. Functions
with E
(3)
(1) to (3) are more complete measures of dependence between random processes in comparison to the conventional product correla-
tion functions, and, in particular, are able to account the stochastic dependence between random processes more representatively. Besides the above ones, the dispersion R-function is the most general kind of the dispersion functions, and for stationary and joint stationary in the strict sense ergodic random processes y (t s v) ,
z (t ) , u (t s ) , w(t) it is of the form
Ez () E E y (t s v) E y () . (4) R yzuw (v, , s ) E E z (t ) w ( t ) u ( t s ) In along to characteristics (1)-(4) presented, the following measure with i ( ni ) being some increasing natural-valued functions in ni , z1,u1 zk ,uk (1,, k ) , coupling k pairs of stationary and ni 0 as ni , i 1,, k . Within this, it is assumed joint stationary in the strict sense ergodic (zero mean, for simplicity i (ni ) , …, of consideration) random processes z1 (t1 ), u1 ( s ) (n ) that the new sets U i keep all the previous elements of the prez k (t k ), u k ( s ) , i t i s , i 1,, k may be introduced as: ceding sets as ni , i 1,, k increase, and k z (t ) (n ) (n ) (n ) E i i z1,u1 zk ,uk ( 1 ,, k ) E (5) u j i u j i1 0, u0 i , ui ( s ) . j i 1 , i 1,, k . (6) j 1,..., i ( ni ) In terms of distribution densities, such a measure is of the form
z1,u1 zk ,uk (1,, k )
( ni ) i ( ni )
u
p( zi , ui , i ) dzi p(u1,, uk )du1 duk , zi p (ui ) i 1 k
where p ( z i , u i , i ) is the joint distribution density of the random processes zi (ti ), ui ( s ) , i t i s , i 1,, k ; p (u i ) is the marginal distribution density of
ui (s ),
i 1,, k ; p (u1 , , u k ) is the joint distribution density of the processes u1 ( s ) , …, u k (s ) .
Thus, the aim of the present paper is deriving an estimate of function z ,u z ,u ( 1 , , k ) (5) by use of sampled observa1 1 k k
tions of the random processes z1 (t1 ), u1 ( s ) , …, z k (t k ), u k ( s ) , i ti s , i 1,, k . Within the problem, the corresponding methodology is a generalization of that of proposed in the fullness of time by Varlaki and Seidl (1983) for estimating the above mentioned dispersion function yu ( ) (1). 3. ESTIMATION TECHNIQUE
n
( ) Let L i , ni 1,2, be sequences of partitions of the real axis R by sequences of real numbers
( ni )
U ( ni ) u0
( ni ) i ( ni )
,, u
( ni )
u0
( ni ) i ( ni )
u
i 1,, k onto i ( ni ) 2 , i 1, , k intervals of the form ( ni )
l0
(n ) , u0 i , l1( ni ) u0( ni ) , u1( ni ) , …,
(n ) (n ) l i( n ) u i( n ) , , i i i i
, ni
In particular, the following partitions given by the sequences of intervals
max
a n (n ) l j 1 , j 0, 1, ,2ni i bi i :
j 1 n ai b ni j n ai b ni (n ) a (n ) i i i i l0 i , ni i , l j i , , ni n bi bi i a (n ) a n j 1,,2ni i bi i , l ia n ni i , , i 1,, k 2 ni i bi i 1 meet the above conditions, with ai , bi , i 1,, k being some
natural-valued constants and bi 1 . k
The direct production k
dimensional space tangular regions
i 1
L i 1
( ni )
R onto
forms a partition of the k-
k
( i (ni ) 2) k-dimensional rec i 1
(n ) (n ) ( n ,, n ) j 1,, j k (u1,, uk ) u j i1 ui u j i , i 1,, k , i 1 i k
and, due to (6),
max j
,
i ji 1,..., i ( ni )
( n ,, n )
V j 1,, j k 0, ni , i 1,, k . 1 k
Here V () stands for the volume of a set, and , for the closing of the set.
( n )
Let u j i ( s ) be the event a value of the random process i (n )
belongs to the interval l j i : i ( ni ) i 1
uj
1780
( ni ) , i
ui u j
i 1,, k ,
ui (s )
IFAC MIM 2016 June 28-30, 2016. Troyes, France
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
( n ) (n ) u j11 ( s ),, u jkk ( s ) be the event a value of the k-dimensional random function u1 ( s ), , u k ( s ) in the variable s belongs to ( n ,, n )
the domain j 1,, j k . Let, again, P{} be the probability of an 1 k event,
I{},
the
indicator
function
of
an
event,
i.e.
1781
In turn, due to the stationarity, joint stationarity, and ergodicity of the processes z i (t i ) and u i (s ) , i 1,, k , there are valid for all
ti , i 1,, k and s with probability 1 T
1 T T
( n )
zi (s i ) I u ji i (s)ds
lim
0
def 1, {} I {} . 0, {}
( n ) E zi (ti ) I u j i ( s ) , i ti s , i 1,, k , i
(8а)
Then, in accordance to Varlaki and Seidl (1983)
T ( n ) 1 ( ni ) I u j ( s )ds P I u j i ( s ) , i 1,, k , T T i i 0
(8b)
lim
( n ) E z (ti ) I u j i ( s ) i
( n ) P u j i ( s) i z (t ) almost surely, i 1,, k , E i (7) ui ( s) with the ratio 0 0 in the left hand part of (7) being considered as ni
lim
T
1 lim T T
( n )
(n )
I u j11 (s),, u jkk 0
( s)ds
( n ) (n ) P I u j 1 ( s ),, u j k ( s ) , i 1,, k . k 1
(8c)
Let now
zero by definition.
( n1,,nk ) ( ,, k ) z1,u1 zk ,uk 1
=
k ( nk ) 1 k
1 ( n1 ) 1
j1 0
jk 0
i 1
( n ) E zi (ti ) I u j i ( s ) i ( ni ) P u j ( s ) i
(n ) P I u (jn1 ) ( s ),, u j k ( s ) , i ti s , i 1,, k , (9a) k 1
( n1 ,, nk ,T ) (1,, k ) z1,u1 zk ,uk 1 ( n1 ) 1 k ( nk ) 1 k 1 T j1 0 jk 0 i 1
T
( n ) zi ( s i ) I x j i ( s )ds T i I u ( n1 ) ( s ),, u ( nk ) ( s)ds , t s , i 1,, k . (9b) 0 j1 i i jk T ( n ) 0 I x j i ( s )ds i 0
By virtue of (7), (8)
Relationships (9) imply, as partial cases, strongly consistent estimates of all kinds of the dispersion functions that are, in turn, particular cases of the dispersion R-function. As to the dispersion Rfunction, R yzuw (v, , s ) y ,u z , w (v, , s ) ,
( n ,, nk ) lim 1 (1,, k ) n1 z1,u1 z k ,u k nk
z1 ,u1 z k ,u k (1,, k ) ,
( n ,, nk ,T ) lim 1 (1,, k ) T z1 ,u1 z k ,u k
T
1 ( n1 ) 1 2 ( n2 ) 1
j1 0
j2 0
1 T
0
yn1,u,n2z,,Tw (v, , s )
(n ) 1
T
y ( s v) I x j 1 ( s ) d T
I x j11 ( s)d 0
n , n and lim y1,u 2z , w (v, , s ) y ,u z , w (v, , s ) , n1 n2
it directly follows, assuming the processes y(t) and z(t) to be zero mean ones,
( n1 ,, nk ) ( ,, k ) . z1,u1 z k ,uk 1
(n )
z( )I w j22 ( )d 0
(n )
T
I w j22 ( )d (n )
T
I u j11 ( s), w j22 ( )d , (n )
0
0
n ,n ,T n ,n lim y1,u 2z , w (v, , s ) y1,u 2z , w (v, , s ) . T
1781
(n )
IFAC MIM 2016 1782 June 28-30, 2016. Troyes, France
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
4. APPLICATION TO ESTIMATING CONSISTENT MEASURES OF DEPENDENCE Within numerous of applications, e.g. system identification, consistent measures of dependence of random processes play an important role. As known, following to the Kolmogorov terminology, a measure of dependence of two random processes is said to be consistent if it vanishes if and only if the random processes are stochastically independent. For a case of system identification, using consistent measures of stochastic dependence of random processes is a natural way to establish an approximate empirical input/output relationship describing the system model. Hence, a significant feature of such an approach to system identification is just the choice of an appropriate measure of stochastic dependence between the input and output processes. Among various measures of dependence, the product correlation, as it has already been noted, is well known and commonly used. However, the ordinary product correlation of random values Y and X may vanish even if the random values are completely dependent, i.e. if there exists a deterministic function f () such that Y f (X ) with probability 1 (Rajbman, 1981, Rényi, 1959). Rényi (1959) has presented seven axioms that were recognized to be the most natural to describe a measure of dependence X , Y between two random values X and Y. A) X , Y is defined for any pair of random values X and Y, neither of them being constant with probability 1. B) X , Y = Y , X . C) D) E)
0 X ,Y 1 . X , Y 0 if and only if X and Y are independent. X , Y 1 if there exists a strict dependence between X and
Y, i.e. either Y (X ) or X (Y ) where and are Borel-measurable functions. F) If a Borel-measurable functions and map the real axis in a one-to-one way onto itself, ( X ), (Y ) X , Y . G) If the joint distribution of X and Y is normal, then X , Y r X , Y , where r X , Y is the ordinary correlation coefficient of X and Y. In along with the ordinary correlation coefficient, commonly used measures of dependence are the correlation ratio
var E Y X ( X ,Y ) var (Y ) with nonzero variance var(Y), and the maximal correlation coefficient
S X , Y sup
covB (Y ), C ( X )
B , C var B (Y ) var C ( X )
with
the
variances
,
var B (Y ) 0, var C ( X ) 0 ,
functions (Rajbman, 1981) may be considered as a modification and extension of the correlation ratio. In turn, the maximal correlation coefficient is transformed to the following function
S yx ( ) sup
, t s B , C var B y (t ) var C x ( s ) The functional S yx ( ) is referred as the maximal correlation function of the random processes y (t ) and x( s ) . The notion of the maximal correlation as a measure of dependence has been originally introduced and investigated for random values by Rényi (1959), Sarmanov (1963a) and extended to random processes by Sarmanov (1963b). A significant disadvantage of S X , Y is concerned with considerable computational difficulties under its calculation by use of observed sampled data x k , y k , k 1,2, of the random val-
ues X , Y . These difficulties are of course resides in the maximal
correlation function S yx ( ) . To avoid the disadvantage and simultaneously to use a consistent measure of dependence within a problem statement, some restrictive assumptions with respect to such a measure of dependence are needed to be imposed. In particular, for sign-constant random processes (Ovsepyan and Lepsky, 1987), i.e. the processes that do not alter their signs with probability 1, the results of (Ovsepyan and Lepsky, 1987) may be applied. Namely, the quantity yx defined as
yx
2 var E y (t ) x( s) 4
2C yx , vary 2 (t ) 2C yx
(10)
where 2 y (t ) var , C yx cov E y (t ) . x( s ) x( s)
meets the above axioms A to G except B for any sign-constant random processes y (t ) and x (s ) with non-zero variances and finite fourth moments. One should be noted that axiom B (symmetry) does not play a significant role within majority of applications. In contrast to the maximal correlation, yx can be effectively calculated using sampled data in accordance to the methodology derived in the preceding Section. In terms of probability distribution densities, the corresponding values in (10) are (generically) of the form
E y (t ) yp y | x dy , x( s )
and
supremum being taken over all Borel-measurable functions B and
C , and
covB y (t ) , C x( s )
B B , C C .
However, the only S X , Y has been shown by Rényi (1959) to
satisfy all the above axioms, while r X , Y and X , Y do not, in particularly axioms D, E, F are not met for the correlation coefficient, and axioms D, F are not met for the correlation ratio. Under investigation of random processes, the above coefficients are transformed to the corresponding functions. As pointed out above, the ordinary product correlation functions are not the exhaustive tool to be used within identification problems. The above dispersion 1782
var y (t ) x( s ) 2 y (t ) x( s ) var y (t ) E x( s )
2
y yp y | x dy p y | x dy ,
(11)
(12)
IFAC MIM 2016 June 28-30, 2016. Troyes, France
var E y (t ) x( s )
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
2
y (t ) EE E y (t ) x ( s )
U ( n) u 0( n) , , u( n()n) 0 u 0( n) u( n()n)
2
yp y | x dy E y (t ) p( x)dx , (13) p y, x, and, conventionally, p y | x is the conditional disp( x)
p y, x, , t s stand for the joint and marginal distribution densities of the random processes x (s ) and y (t ) correspondingly.
Remark. Of course, under consideration of the sign-constant random processes, say, positive ones, the lower integration limits in the above integrals expressions (11) to (13) are automatically substituted by zeros. In the above yx in (10), the quantities
and C yx in accordance to expres
sions (11) to (13) may be represented as follows
2 var E y (t ) x( s)
2 E E y (t ) x( s )
2 E y (t )
2
2
2
(14)
2 2 y (t ) E C yx E E y (t ) x( s) x( s) 2 4 y (t ) y (t ) 2 E y (t ) E E E E x( s) x( s)
As above, (n) is some increasing natural-valued function in n ,
n
( n)
0 as n , as well as it is assumed that the new sets
U (n) keep all the previous elements of the preceding sets as n
increases, and
u j max j
( n)
u( n()n) , n In particular, the following partition given by the sequences of in(n)
tervals l j , j 0, 1,,2n a b n ,
l0( n) 0 , exp n a , l (jn ) ( n ) ( j 1), ( n ) ( j ) ,
a n j n ab n , if j n b 0 exp def n bn (n) ( j ) b , j n ab n j n ab n 1, if 0 bn bn
Due to (16),
(n)
l j max j
0, n .
Here stands for the length of an interval, and , for the closing of a set. Thus, in accordance to the above general case, the fol(15)
sampled data is to estimate the summands lyx , l 1,,4 in (14) and (15) containing the corresponding conditional means. In turn, to estimate the functions lyx , l 1,,4 thus introduced,
lowing estimate
(zn),u z 1
one
should
1
k ,u
( ) of z1,u zk ,u ( ) may be
( )
Pu
( n) 1 k E z (t ) I u ( n ) ( s ) i i j
j 0 i 1
P u (jn) ( s)
(n) j (s)
Ezi (ti ) I u (jn) (s) k
( n ) 1
consider
L , n 1,2, as a sequence of a partition of the real positive semi axis R+ (rather than R) by sequences of real numbers
k ,u
The only difference in comparison to the general case of estimating z1,u1 z ,u ( 1 ,, k ) is that, for z1,u zk ,u ( ) coupling processes,
(zn),u z
applied
the approach presented above may be applied with respect to the function z1,u zk ,u ( ) .
(n)
(16)
meets the above conditions, with a , b being some natural-valued constants, and b 1 .
Thus, the entity of the problem of estimating yx by use of
random
.
j 1,..., ( n )
2 yx 3 yx Ey 2 (t ) 4 yx 3 yx 2 .
sign-constant
u (jn)1 0, u 0( n) 0,
j 1,..., ( n )
2
the
j 1, ,2n a b n ,
2 y (t ) E Ε x( s )
k k
l0( n) 0, u0( n) , l1( n ) u0( n ) ,u1( n) , , l( n( )n) u( n()n ) , .
( n) j 1, ,2n a b n , l a n n a , , where 2 n b 1
1 yx Ey 2 (t ) ,
onto (n) 2 intervals of the form
tribution density of y (t ) with respect to x (s ) ; and p(x) , p( y) ,
2 var E y (t ) x( s)
1783
j 0
and
1783
i 1
Pu
k 1 (n) j (s)
,
(17)
IFAC MIM 2016 1784 June 28-30, 2016. Troyes, France
(zn,,Tu)z 1
k ,u
K.R. Chernyshov / IFAC-PapersOnLine 49-12 (2016) 1779–1784
( )
T ( n) z ( s ) I x j ( s)ds i ( n) 1 1 i 1 0 . k 1 T T j 0 ( n) I x j ( s )ds 0 k
(18)
Thus, by virtue of (7), (8), (17), (18),
lim lim (zn,,Tu) z ,u ( ) z1,u zk ,u ( ) almost surely. 1 k
n T
Such a reasoning directly implies the corresponding formulae to estimate the functions lyx , l 1,,4 . Namely: 1) by imposing z1 (t ) z 2 (t ) y 2 (t ) , u ( s ) x( s ) ,
z1,u z2 ,u ( )
( ) 2 2 y , x y , x 1 yx ( ) lim lim n,2T ( ) almost surely; n T y , x y 2 , x 2) by imposing z1 (t ) z 2 (t ) y (t ), z3 y 2 (t ) , u ( s )
z1,u z2 ,u z3 ,u ( )
y, x y, x y 2 , x
x( s ) ,
( )
2 yx ( ) lim lim n, T ( ) almost surely; n T y , x y , x y 2 , x
3) by imposing z1 (t ) z 2 (t ) y (t ) , u ( s ) x( s ) ,
z1,u z2 ,u ( ) u , x u , x ( )
3 yx lim lim (un,,xT)u , x ( ) almost surely; n T
4) by
imposing
u ( s ) x( s ) ,
z1 (t ) z 2 (t ) z 3 (t ) z 4 (t ) y (t ) ,
z1,u z2 ,u z3 ,u z4 ,u ( ) y , x y, x y , x y, x ( )
4 yx lim lim (yn,,xT) y , x y , x y , x ( ) almost surely. n T
Thus obtained, the formulae are applied to constructing the sampled estimates of measure of dependence (10). 5. CONCLUSIONS A problem of estimating some measures of stochastic dependence of random processes has been considered. In particular, a measure coupling k pairs of stationary and joint stationary in the strict sense ergodic random processes has been introduced, and strongly consistent, i.e. converging with probability 1, nonparametric estimates of such a measure have been obtained. These estimates has been applied to deriving sampled analogues of a nonlinear measure of stochastic dependence of sign-constant random processes, for which the measure of dependence is the consistent one in the Kolmogorov sense, i.e. vanishing if and only if the random processes are stochastically independent. REFERENCES Allen, M.S., Sumali, H., and D.S. Epp (2008). “Piecewise-linear restoring force surfaces for semi-nonparametric identification of nonlinear systems”, Nonlinear Dynamics, vol. 54, no. 1, pp. 123-135. Bădina, L., Daraioc, C., and L. Simar (2012). “How to measure the impact of environmental factors in a nonparametric production
model”, European Journal of Operational Research, vol. 223, no. 3, pp. 818-833. Cao Rong-Min, Hou Zhong-Sheng, and Hui-Xing Zhou (2014). “Data-driven Nonparametric Model Adaptive Precision Control for Linear Servo Systems”, International Journal of Automation and Computing, vol. 11, no. 5, pp. 517-526. Huynh, K.P., Jacho-Chávez, D.T., Petrunia, R.J., and M.C. Voia (2015). “A nonparametric analysis of firm size, leverage and labour productivity distribution dynamics”, Empirical Economics, vol. 48, no. 1, pp. 337-360. Kuosmanen, T., Johnson, A., and A. Saastamoinen (2015). “Stochastic Nonparametric Approach to Efficiency Analysis: A Unified Framework”, Data Envelopment Analysis. A Handbook of Models and Methods. Springer, International Series in Operations Research & Management Science, vol. 221, pp. 191-244. Li, Der-Chang and Chun-Wu Yeh (2008). “A non-parametric learning algorithm for small manufacturing data sets”, Expert Systems with Applications, vol. 34, no. 1, pp. 391-398. Mosallam, A., Medjaher, K., and N. Zerhouni (2013). “Nonparametric time series modelling for industrial prognostics and health management”, The International Journal of Advanced Manufacturing Technology, vol. 69, no. 5, pp. 1685-1699. Ovsepyan, F.A. and D.M. Lepsky (1987). “On identification of nonlinear nonparametric control systems”, 10th World Cong. Aut. Cont. Prepr., vol. 10. Rajbman, N.S. (1981). “Extensions to nonlinear and minimax approaches'', In: Trends and Progress in System Identification, Ed. by P. Eykhoff, Pergamon Press, Oxford, pp. 185-237. Ray, S.C. (2015). “Nonparametric measures of scale economies and capacity utilization: An application to U.S. manufacturing”, European Journal of Operational Research, vol. 245, no. 2, pp. 602-611. Rényi, A. (1959). “On measures of dependence”, Acta Math. Hung., vol. 10, no. 3-4, pp. 441-451. Sadykhov, G.S. and I.A. Babaev (2015). “Nonparametric assessments and limiting probability values of the hazardous and safe states of a technogenic-hazardous object”, Journal of Machinery Manufacture and Reliability, vol. 44, no. 3, pp. 298-304. Sarmanov, O.V. (1963a). “The maximal correlation coefficient (nonsymmetric case)”, Sel. Trans. Math. Statist. Probability, vol. 4, pp. 207-210. Sarmanov, O.V. (1963b). “Investigation of stationary Markov processes by the method of eigenfunction expansion”, Sel. Trans. Math. Statist. Probability, vol. 4, pp. 245-269. Schirru, A., Pampuri, S. De Luca, C., and G. De Nicolao (2013). “Virtual Sensors for Semiconductor Manufacturing: A Nonparametric Approach - Exploiting Information Theoretic Learning and Kernel Machines”, Informatics in Control, Automation and Robotics. Springer, Lecture Notes in Electrical Engineering, vol. 174, pp. 175-193. Tran, K.C. and E.G. Tsionas (2009). “Estimation of nonparametric inefficiency effects stochastic frontier models with an application to British manufacturing”, Economic Modelling, vol. 26, no. 5, pp. 904-909. Tsolas, I.E. (2011). “Performance assessment of mining operations using nonparametric production analysis: A bootstrapping approach in DEA, Resources Policy, vol. 36, no. 2, pp. 159-167. Varlaki, P. and L. Seidl (1983). “Estimating the variance functions of ergodic stochastic processes”, Automation and Remote Control, vol. 44, no. 6, part 1, pp. 737-739. Zhu, J. (Ed.) (2015). Data Envelopment Analysis. A Handbook of Models and Methods. Springer, International Series in Operations Research & Management Science, vol. 221, 465 p.
1784