- Email: [email protected]
Stable estimation of autoregressive model parameters with exogenous variables on the basis of the generalized least absolute deviation method
Proceedings,16th IFAC Symposium on Proceedings,16th IFAC Symposium on Proceedings,16th IFAC Symposium on Information Control Problems in Manufacturing Proceedings,16th IFAC Symposium on Information Control Problems in Manufacturing Information Control Problems in Manufacturing Available online at www.sciencedirect.com Bergamo, Italy, June 11-13, 2018 Information Control Problems in Manufacturing Proceedings,16th IFAC Symposium on Bergamo, Italy, Italy, June June 11-13, 11-13, 2018 2018 Bergamo, Bergamo, Italy, JuneProblems 11-13, 2018 Information Control in Manufacturing Bergamo, Italy, June 11-13, 2018
ScienceDirect
IFAC PapersOnLine 51-11 (2018) 1666–1669
Stable estimation Stable Stable estimation estimation of autoregressive model parameters Stable of autoregressive model of autoregressiveestimation model parameters parameters with exogenous variables on the basis of autoregressive model with exogenous variables on with exogenous variables parameters on the the basis basis of the generalized least absolute deviation with exogenous variables on the basis of the generalized least absolute deviation of the generalized least absolute deviation method of the generalizedmethod least absolute deviation method method ∗ ∗∗ ∗ Anatoly V. Panyukov ∗ Yasir Ali Mezaal ∗∗ ∗∗
Anatoly V. Panyukov Yasir Ali Mezaal ∗∗ Anatoly Anatoly V. V. Panyukov Panyukov ∗ Yasir Yasir Ali Ali Mezaal Mezaal ∗ ∗∗ ∗ Anatoly V. Panyukov Yasir Ali ∗ South Ural State University, Chelyabinsk, Mezaal Russia (e-mail: (e-mail: ∗ Ural State University, Chelyabinsk, Russia South Ural State University, Chelyabinsk, Russia (e-mail: ∗ South South Ural State [email protected]). University, Chelyabinsk, Russia (e-mail: [email protected]). [email protected]). ∗ ∗∗ South Ural State University, Chelyabinsk, Russia (e-mail: [email protected]). ∗∗ South Ural State Chelyabinsk, Russia (e-mail: ∗∗ South Ural State University, University, Chelyabinsk, Russia (e-mail: University, Chelyabinsk, Russia (e-mail: ∗∗ South Ural State [email protected]). South Ural State University, Chelyabinsk, Russia (e-mail: yaser ali [email protected]) yaser ali ali [email protected]) [email protected]) yaser ∗∗ South Ural State University, Chelyabinsk, Russia (e-mail: yaser ali [email protected]) yaser ali [email protected]) Abstract: The generalized least absolute deviation method is to the least Abstract: The The generalized generalized least least absolute absolute deviation deviation method method is is an an alternative alternative to to the the least least squares squares Abstract: Abstract: The generalized least absolute deviation method is an an alternative alternative to the the stability least squares squares method. Together with an appropriate choice of the loss function, it ensures and method. Together with an appropriate choice of the loss function, it ensures the stability and method. Together with an appropriate choice of the loss function, it ensures the stability and Abstract: The generalized least absolute deviation method is an alternative to the least squares method. Together with an appropriate choice of the loss function, it ensures the stability and efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the method. Together with an appropriate choice of the loss function, it ensures the stability and efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the previously considered methods for finding the parameters of linear regression and autoregressive previously considered considered methods methods for for finding finding the parameters of linear regression and autoregressive previously the parameters of linear regression and autoregressive efficiency ofconsidered estimating the variables coefficients of the autoregressive models. This paperand is devoted to the previously methods for finding parameters of linear regression autoregressive models without exogenous extend to the problem of estimating the parameters of models without without exogenous variables extend to the problem problem of estimating estimating the parameters of models exogenous variables extend to the of the parameters of previously considered methods for finding the parameters of linear regression and autoregressive models without exogenous variables extend to the problem of estimating the parameters of autoregressive models with exogenous variables. autoregressive models with exogenous variables. autoregressive models with exogenous variables. models withoutmodels exogenous variables extend to the problem of estimating the parameters of autoregressive with exogenous variables. © 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. autoregressive models with exogenous variables. Keywords: Demand Forecasting; Optimization and Control; Keywords: Demand Demand Forecasting; Forecasting; Optimization Optimization and and Control; Control; Computational Computational Science Science Keywords: Keywords: Demand Forecasting; Optimization and Control; Computational Computational Science Science Keywords: Demand Forecasting; Optimization and Control; Computational Science 1. INTRODUCTION The interrelation of methods established in Panyukov and 1. INTRODUCTION INTRODUCTION The interrelation of methods established in Panyukov and 1. The interrelation of methods established in Panyukov and 1. INTRODUCTION The interrelation of methods established in Panyukov Tyrsin (2008) made it possible to reduce the problem of Tyrsin (2008) (2008) made made it it possible possible to to reduce reduce the the problem problemand of Tyrsin of 1. INTRODUCTION The interrelation of methods established inthe Panyukov and Tyrsin (2008) made it possible toto reduce problem of determining the GLAD estimates an iterative procedure determining the GLAD estimates to an iterative procedure Let us consider the evaluation of the coefficients of the determining the GLAD estimates to an iterative procedure Let us us consider consider the the evaluation evaluation of of the the coefficients coefficients of of the the Tyrsin (2008) made it possible to reduce the problem of determining the GLAD estimates to an iterative procedure with WLAD estimates. The latter is calculated by solvLet WLAD estimates. The latter is calculated by solvLet usautoregressive consider the evaluation of the coefficients of the with linear equation with exogenous variables with WLAD estimates. The latter is calculated by solvlinear autoregressive equation with with exogenous variables determining the GLAD estimates to an iterative procedure with WLAD estimates. The latter is calculated by solving the corresponding linear programming problem. The linear autoregressive equation exogenous variables ing the the corresponding corresponding linear linear programming programming problem. problem. The The Let usautoregressive consider the evaluation of the coefficients of the ing linear equation with exogenous variables m n with WLAD estimates. The latter is calculated by solving the corresponding linear programming problem. The sufficient condition Panyukov and Tyrsin Tyrsin (2008) imposed m n m n sufficient condition Panyukov and (2008) imposed linear autoregressive equation with exogenous variables sufficient condition Panyukov and Tyrsin (2008) imposed m n ing the corresponding linear programming problem. The yyt = a y + b x + , t = 1, 2, . . . , T, (1) sufficient condition Panyukov and Tyrsin (2008) imposed on the loss function is found to ensure the stability of j t−j j tj t = m a + bjj x + , tt = 1, 2, . . . , T, (1) on the loss function is found to ensure the stability of ajj yyt−j xtj the loss function is found to ensure the stability of t−j + n b tj + tt , yyttt = = t= = 1, 1, 2, 2, .. .. .. ,, T, T, (1) (1) on sufficient condition Panyukov and Tyrsin (2008) imposed on the loss function is found to ensure the stability of j=1 aj yt−j + j=1 bj xtj + t , the GLAD estimators of the autoregressive models coefj=1 j=1 the GLAD estimators of the autoregressive models coefj=1 j=1 the GLAD estimators of the autoregressive models coefyt = j=1 aj yt−j + j=1 bj xtj + t , t = 1, 2, . . . , T, (1) on the loss function is found to ensure the stability of the GLAD estimators of the autoregressive models coefficients under emission conditions. It ensures the stability ficients under under emission emission conditions. conditions. It It ensures ensures the the stability stability ficients ,, yy2 ,, .. .. .. ,, yyn are the values of the state varihere yy1j=1 j=1 the GLAD estimators of the autoregressive models coefare the values of the state varihere ficients under emission conditions. It ensures the stability of GLAD-estimates GLAD-estimates of of autoregressive autoregressive models models in in terms terms of of the values of the varihere yy11 ,, yy22 ,, .. .. .. ,, yynn are of the values of the state state varihere of GLAD-estimates of models in of able, x are the values controls at time 1 22t ,, .. .. .. ,,nx 1t ,, x ntare ficients under emission conditions. It ensures stability able, x x x are the values of controls at time of GLAD-estimates of autoregressive autoregressive models the in terms terms of outliers. 1t 2t nt able, x , x , . . . , x are the values of controls at time 1t 2t nt outliers. , y , . . . , y are the values of the state varihere y able, x , x , . . . , x are the values controls at time outliers. points t = 1, 2, . . . , T , , , . . . , are random errors, 1 2 n 1t 2t nt 1 2 t of GLAD-estimates of autoregressive models in terms of points tt = 1, 2, .. .. .. ,, T ,, 11 ,, 22 ,, .. .. .. ,, tt are random errors, outliers. points = 1, 2, T are random errors, able, x.. ..2t,,1, ,a.m . . .,and of time outliers. points t3 ,..= 2, .x.nt , Tbbare , ,b1 ,the . . ., ,b t are arecontrols randomaterrors, Special features features of of the the GLAD GLAD method method application application for for the the a a2 ,,xa a1t and , b2 ,. values are unknown coeffi1 ,, a Special a unknown coeffi1 2 3 m m Special features of GLAD application for a .. .. ,,1,a a and bb,111 ,,,bbb222,,,, bbb333,... ... ... .,,, ,bbbm are unknown coeffi1 ,, a 2 ,, a 3 ..= m m points t 2, . . . , T are random errors, Special features of the the GLAD method method application for the the a a a a and are unknown coefficonstruction of the regression equation are considered in cients. 1 2 t 1 2 3 m 1 2 3 m construction of the regression equation are considered in cients. construction of the regression equation are considered in cients. Special features of the GLAD method application for the a construction of the regression equation are considered in cients. Tyrsin (2006). Special features of the use of GLAD for 1 , a2 , a3 . . . , am and b1 , b2 , b3 . . . , bm are unknown coeffiTyrsin (2006). Special features of the use of GLAD for Tyrsin (2006). Special features of the use of GLAD for Ordinary Least Least Squares Squares (OLS) (OLS) is is the the parametric parametric method method construction of the regression equation are considered in Ordinary cients. Tyrsin (2006). Special features of the use of GLAD for constructing the autoregressive equation without exogeOrdinary Least Squares (OLS) is the parametric method the autoregressive equation without exogeOrdinary Least Squares (OLS) of is the parametric method constructing constructing the autoregressive equation without exogein common used for estimation regression equation Tyrsin (2006). Special features of the use of GLAD for in common used for estimation of the regression equation constructing the autoregressive equation without exogenous variables variables are are considered considered in in Panyukov Panyukov and and Tyrsin Tyrsin in common used for of the parametric regression equation nous Ordinary Squares (OLS) isassumptions the method in commonLeast used for estimation estimation of regression equation nous variables are considered in Panyukov and Tyrsin coefficients. We need some strict to use OLS. constructing the autoregressive equation without exogecoefficients. We need some strict assumptions to use OLS. nous variables are considered in Panyukov and Tyrsin (2008). This paper is devoted to extension of the precoefficients. We need some strict assumptions to use OLS. (2008). This paper is devoted to extension of the prein common used for estimation of the regression equation coefficients. We need some strict assumptions to use OLS. (2008). This paper is devoted to extension of the preThey include independence and normal distribution of ernous variables are considered in Panyukov and Tyrsin They include independence and normal distribution of er(2008). This paper is devoted to extension of the previously discussed methods to the problem of estimating They include independence and normal distribution of erviously discussed methods to the problem of estimating coefficients. We need some strict assumptions to use OLS. They include independence and normal distribution of erviously discussed methods to the problem of estimating rors and and determinacy determinacy of of explanatory explanatory variables variables (see (see Huber Huber (2008). This paper is devoted to extension of the prerors viously discussed methods to the problem of estimating the parameters of autoregressive models with exogenous rors and determinacy of explanatory variables (see Huber parameters of autoregressive models with exogenous They include independence andminor normal distribution of er- the rors and determinacy of explanatory variables (see Huber the parameters of autoregressive models with exogenous and Ronchetti (2009)). Even violations of stated viously discussed methods to the problem of estimating and Ronchetti (2009)). Even minor violations of stated the parameters of autoregressive models with exogenous variables. and Ronchetti (2009)). Even minor violations of Huber stated rors and determinacy of explanatory and Ronchetti (2009)). Even minor violations(see stated variables. variables. assumptions dramatically lower the efficiency of the parameters of autoregressive models with exogenous assumptions dramatically lower the variables efficiency ofofestimaestimavariables. assumptions dramatically lower the efficiency of estimaand Ronchetti (2009)). Even minor violations of stated assumptions dramatically lower the efficiency of estimators. Let us note the instability of OLS estimation process variables. tors. Let us note the instability of OLS estimation process tors. Let us note the instability of OLS estimation process 2. THE WLADAND assumptions dramatically lower efficiency of estimators. Letof uspresence note theof instability ofthe OLS estimation process in case large measurements errors. In this 2. THE RELATIONSHIP RELATIONSHIP BETWEEN BETWEEN WLADAND in case of presence of large measurements errors. In this 2. BETWEEN WLADAND in case of presence of large measurements errors. In this 2. THE THE RELATIONSHIP RELATIONSHIP BETWEEN WLADAND GLAD ESTIMATES tors. Let us note the instability of OLS estimation process in case of presence of large measurements errors. In this case, estimated coefficients become inconsistent. Finding GLAD ESTIMATES case, estimated coefficients become inconsistent. Finding GLAD ESTIMATES case, estimated coefficients become inconsistent. Finding 2. THE RELATIONSHIP BETWEEN WLADAND GLAD ESTIMATES in case of of of largeequation measurements errors. In this case, estimated coefficients become inconsistent. Finding estimates autoregressive becomes substantially estimates ofpresence autoregressive equation becomes substantially estimates of autoregressive equation becomes substantially GLAD ESTIMATES case, estimated coefficients become inconsistent. Finding estimates of autoregressive equation becomes substantially One can can get get the the WLAD WLAD estimations estimations of of coefficients coefficients by by complicated complicated due due to to the the poor poor conditionality conditionality of of the the equations equations One can get the complicated due poor conditionality of equations One canthe get the WLAD WLAD estimations estimations of of coefficients coefficients by by estimates of autoregressive becomes substantially complicated due to to the the poorequation conditionality of the the equations One solving problem system representing necessary conditions for minimization solving the problem system representing necessary conditions for minimization solving problem system representing necessary conditions for minimization One can∗the the WLAD estimations of coefficients by solving theget problem complicated due to the poor conditionality of the equations system representing necessary conditions for minimization of squared deviations sum. ∗ ∗ ∗ ∗ ∗ of squared deviations sum. ∗ ∗ (a ,, .. .. problem .. ,, a of deviations sum. ∗ ,, a ∗ ,, b ∗ ,, b 1 1 (a a∗2∗2∗2the a∗m b∗2∗ ,, .. .. .. ,, bb∗n∗ )) = = system representing conditions for minimization solving of squared squared deviationsnecessary sum. ∗ (a m, b (a1∗11 ,, a a2 ,, .. .. .. ,, a am , bb1∗11 ,, bbT2∗22 ,, .. .. .. ,, bbn∗nn ))m= = Least Absolute Deviations (LAD) method of Powell (1984) m n Least Absolute Deviations (LAD) method of Powell (1984) of squared deviations sum. ∗ ∗ ∗ ∗ ∗ ∗ T m n Least Absolute Deviations (LAD) method of Powell (1984) T m n (a1 , a2 , . . . , am , b1 , bT2 , . . . , bn )m= Least Absolute Deviations (LAD) method of Powell (1984) is a method alternative to OLS. It allows to obtain n is aa method alternative to OLS. It allows to obtain arg min yytt − a bbjj x jy t−j − tj (2) is method alternative to OLS. It allows to obtain arg min − a y − x m j t−j tj y (2) arg min − a y − b x Least Absolute Deviations (LAD) method of Powell (1984) (2) is a method alternative to OLS. It allows to obtain (a ,a ,...,a )∈R robust errors in case of violation of OLS assumptions (see t j t−j j tj m, T 1 2 m m n m arg (a minm )∈R , yt − robust errors in case of violation of OLS assumptions (see j=1 aj yt−j − j=1 bj xtj (2) (a(b111,a ,a,b222,...,a ,...,a )∈Rn robust errors in case of violation of OLS assumptions (see m m , t=1 ,...,b )∈R t=1 j=1 j=1 n n (a ,a ,...,a )∈R , is a method alternative to OLS. It allows to obtain robust errors in case of violation of OLS assumptions (see t=1 j=1 j=1 11 ,b 22 ,...,bm n Dielman. (2003)). We present two types of LAD: Weighted (b )∈R n y arg min − a y − b x (b ,b ,...,b )∈R t=1 j=1 j=1 Dielman. (2003)). (2003)). We We present present two two types types of of LAD: LAD: Weighted Weighted 1 2 n n j t−j j tj (2) t Dielman. m 1 2 ,...,bm n )∈R (a(b1p ,a,b )∈R , 1, 2, robust errors in case ofpresent violation OLS assumptions (see where Dielman. (2003)). We twooftypes of LAD: Weighted LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, tt = .. .. .. ,, T are predetermined coef t2 ,...,a t=1 j=1 j=1 n where p LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, = 1, 2, T are predetermined coeftt 2 ,...,b where LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, tt = 1, 2, .. .. .. ,, T are predetermined coef(bp 1 ,b n )∈R Dielman. (2003)). We present two types of LAD: Weighted where p LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, = 1, 2, T are predetermined coefmethod ( see Tyrsin (2006)). ficients. This problem represents the problem of convex t method ( see Tyrsin (2006)). ficients. This problem represents the problem of convex method ( see Tyrsin (2006)). ficients. This problem represents the problem of convex where p LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, t = 1, 2, . . . , T are predetermined coefmethod ( see Tyrsin (2006)). ficients. tThis problem represents the problem of convex method ( see Tyrsin (2006)). ficients. This problem represents the problem of convex 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2018, 2018 IFAC 1726Hosting by Elsevier Ltd. All rights reserved. Copyright 2018 1726 Copyright ©under 2018 IFAC IFAC 1726Control. Peer review© of International Federation of Automatic Copyright © 2018 responsibility IFAC 1726 10.1016/j.ifacol.2018.08.217 Copyright © 2018 IFAC 1726
IFAC INCOM 2018 Bergamo, Italy, June 11-13, 2018
Anatoly V. Panyukov et al. / IFAC PapersOnLine 51-11 (2018) 1666–1669
piecewise linear optimization, and the introduction of additional variables reduces it to the problem of linear programming T min pt u t : m (a1 ,a2 ,...,am )∈R , (b1 ,b2 ,...,bn )∈Rn (u1 ,u2 ,...,uT )∈RT
−ut ≤ yt −
m j=1
aj yt−j −
n j=1
bj xtj
(1) for each collection of weights {pt }nt=1 arg
The GLAD-estimates can be obtained from the solution of the problem . ., bn ∗ ) = (a1 ∗ , . . . , am ∗ , b1 ∗ , . T m n arg min m ρ yt − aj yt−j − bj xtj , (a1 ,...,am )∈R t=1 j=1 j=1 n (b1 ,...,bn )∈R
(4) where ρ(∗) is a convex upward monotonically increasing twice continuously differentiable function such that ρ(0) = 0. Theorem 1. All local minima of the GLAD-estimation problem for the coefficients of the autoregressive equation (1) belong to the set a1 (k) , a2 (k) , . . . , am (k) , (k) (k) (k) : b1 , b2 , . . . , bn m n . U = yt = aj yt−j + bj xtj , j=1 j=1 t ∈ k = {k1 , k2 , . . . , km+n : 1 ≤ k1 < k2 < . . . < km+n ≤ T } Proof. The set U contains the solutions of all possible joint systems of m + n linearly independent equations m n yt = aj yt−j + bj xtj , t ∈ k j=1
with m + n unknowns a1 , a2 , a3 . . . , am , b1 , b2 , b3 . . . , bn .
If the solution (a1 , a2 , a3 . . . , am , b1 , b2 , b3 . . . , bn ) ∈ U then there exists an -neighbourhood for which the loss function is continuous and convex upwards. Consequently, such a solution can not be a local minimum. The theorem now follows Obviously, the number of systems is equal Cn+m . Thus, T the solution of problem (3) can be reduced to choosing the best of Cn+m solutions of linear algebraic equations T systems. This approach is applicable for m ≤ 3 . To compute GLAD-estimates for higher order dimension problems the interrelation between WLAD and GLAD-estimates have to be used from
min
(a1 ,a2 ,...,am )∈Rm (b1 ,b2 ,...,bn )∈Rn
n m pt × yt − bj xtj ∈ U ; aj yt−j − t=1 j=1 j=1
n
(3)
ut ≥ 0, t = 1, 2, . . . , T This problem has a canonical form with n + m + T + 1 variables and 3n inequality constraints including the conditions for the non-negativity of the variables uj , j = 1, 2, . . . , T . The main problem with the use of WLAD method is the absence of general formal rules for choosing weight coefficients. Consequently, this approach requires additional research.
j=1
Theorem 2. Let U be the set of local extreme of problem (4) then:
t=1
≤ ut , .
1667
(2) for all a1 (k) , a2 (k) , . . . , am (k) ,
(5)
b1 (k) , b2 (k) , . . . , bn (k) ∈ U
there is such a collection of weights {pt }nt=1 that a1 (k) , a2 (k) , . . . , am (k) , b1 (k) , b2 (k) , . . . , bn (k) = n m n arg min pt yt − aj yt−j − bj xtj . m (a1 ,a2 ,...,am )∈R t=1 j=1 j=1 n (b1 ,b2 ,...,bn )∈R
(6)
Proof. The proof of the first part of the theorem essentially repeats the proof of Theorem 1. The validity of the second part of the theorem follows from the fact that the weights of the active part of the constraints can be considered as non-zero, and the weights of the inactive part are equal to zero. In this case the minimal value of the loss function is equal to zero and it is achieved by solving the chosen system of equations. The theorem now follows Theorems 1 and 2 give a way to determine the weight coefficients for the linear programming problem (3) and thus allow the problem (4) to be reduced to solving a sequence of linear programming problems (3). 3. ALGORITHM FOR COMPUTING GLAD-ESTIMATES Primal solution of problem (4) is based on the usage of theorem 1 and involves finding all node points and choosing one of them as a solution that ensures the minimum of the objective function. The brute force algorithm requires the solution of Cn+m T linear equations systems of order m+n. For large values of n and m this leads to a significant computational complexity. An alternative approach is based on the reduction of this problem to the sequence linear programming problems (3). Consider possible algorithms based on this approach. Algorithm GLAD-estimator Input: number of measures T ; values {yt }Tt=1 of the endogenous variable; values {{xtj }Tt=1 }nj=1 of exogenous variables; function ρ(∗). Output: estimation of coefficients of autoregressive equation (a1 ∗ , a2 ∗ , a3 ∗ . . . , am ∗ , b1 ∗ , b2 ∗ , . . . , bn ∗ ) . Step 1. For all t = 1, 2, . . . , T do pt = 1; k := 0;
1727
IFAC INCOM 2018 1668 Bergamo, Italy, June 11-13, 2018
Anatoly V. Panyukov et al. / IFAC PapersOnLine 51-11 (2018) 1666–1669
m n (k) (k) ρ yt − aj yt−j − bj xtj = t=1 j=1 j=1 m n (k) ρ yt − a y − bj (k) xtj − j t−j j=1 j=1 n m n (k) (k) pt · yt − aj yt−j − bj xtj + ≥ t=1 j=1 j=1 m n (k) (k) pt · y t − aj yt−j − bj xtj j=1 j=1 m n (k) (k) ρ yt − aj yt−j − bj xtj − n j=1 j=1 + n m t=1 (k) (k) pt · yt − aj yt−j − bj xtj j=1 j=1 n min pt × m
a1 (k) , a2 (k) , a3 (k) . . . , am (k) b1 (k) , b2 (k) , b3 (k) . . . , bn (k) := (k) (k) (k) (k) u1 , u2 , u3 , . . . , ut T arg min pt u t : m (a1 ,a2 ,...,am )∈R t=1 (b1 ,b2 ,,...,bn )∈Rn T (u1 ,u2 ,,...,ut )∈R m n
−ut ≤ yt −
j=1
aj yt−j −
bj xtj
j=1
ut ≥ 0, t = 1, 2, . . . , T Step 2. For all t = 1, 2, . . . , T do k := k + 1; a1 (k) , a2 (k) , a3 (k) . . . , am (k) b1 (k) , b2 (k) , b3 (k) . . . , bn (k) u1 (k) , u2 (k) , u3 (k) , . . . , ut (k) arg
min
(a1 ,a2 ,...,am )∈Rm (b1 ,b2 ,,...,bn )∈Rn (u1 ,u2 ,,...,ut )∈RT m
−ut ≤ yt −
j=1
n
≤ ut , .
:=
T
(a1 ,a2 ,...,am )∈R
n
pt u t :
t=1
aj yt−j −
n
bj xtj
j=1
ut ≥ 0, t = 1, 2, . . . , T
≤ ut , .
Step 3. If (k) (k) (k) a1 , a2 , a3 . . . , am (k) = b1 (k) , b2 (k) , b3 (k) . . . , bn (k) (k−1) (k−1) (k−1) , a2 , a3 . . . , am (k−1) a1 b1 (k−1) , b2 (k−1) , b3 (k−1) . . . , bn (k−1) then go to Step 2. Step 4. Stop. Target values are (k) (k) (k) a1 , a2 , a3 . . . , am (k) . b1 (k) , b2 (k) , b3 (k) . . . , bn (k) Performance justification of this algorithm leads us to the following theorem. Theorem 3. If the loss function ρ(∗) is convex upward, monotone increasing, continuously differentiable on the positive semi-axis, and satisfies the condition ρ(0) = M < ∞ then the sequence (k) (k) (k) a1 , a2 , a3 . . . , am (k) b1 (k) , b2 (k) , b3 (k) . . . , bn (k) constructed by the GLAD-estimation algorithm converges to the global extremum of problem (4).
The first equality and the inequality following it are obvious. The second equation is a consequence of changing the notation of the variables in step 1. The third equation is the result of the choice of the weight coefficients in step 2 and equality (7). The last inequality is a consequence of relations (8). Therefore
(k)
(8)
m n (k) (k) ρ yt − aj yt−j − bj xtj = t=1 j=1 j=1 n n m (k) (k+1) u ) (k+1) ( aj yt−j − bj xtj ≥ ν yt − t=1 j=1 j=1 n m n ρ yt − aj (k+1) yt−j − bj (k+1) xtj , t=1 j=1 j=1 n
Proof. It follows from the requirements imposed on the function ρ(∗) that at any point uk an approximation (k) ν (u ) (u) = ρ(u(k) ) − ρ (u(k) ) · u(k) + ρ (u(k) ) · u (7) is a majorant, i.e. (∀u = uk )(ρ(u) < ν (u ) (u)), ρ(uk ) = ν(uk ). Therefore, in accordance with the algorithm
t=1
(b1 ,b2 ,...,bn )∈R m n = yt − a y − b x j t−j j tj j=1 j=1 m n (k) (k) ρ yt − a y − b x j t−j j tj − n j=1 j=1 + m n t=1 (k) (k) pt · yt − a y − b x j t−j j tj j=1 j=1 n m n (k+1) (k+1) = pt · yt − x a y − b tj j t−j j t=1 j=1 j=1 n m n (k) ν (u ) yt − aj (k+1) yt−j − bj (k+1) xtj ≥ t=1 j=1 j=1 n m n ρ yt − aj (k) yt−j − bj (k) xtj . t=1 j=1 j=1
moreover, equality is attained only if for all t = 1, 2, . . . n and for all k = 1, 2, . . . , m. That is why the sequence
1728
IFAC INCOM 2018 Bergamo, Italy, June 11-13, 2018
Anatoly V. Panyukov et al. / IFAC PapersOnLine 51-11 (2018) 1666–1669
n n m (k) (k) bj xtj aj yt−j − ρ yt − t=1 j=1 j=1
follows from continuity and monotonicity of functions ρ(∗) . The limit point
a 1 ∗ , a2 ∗ , a3 ∗ , . . . , a m ∗ b1 ∗ , b2 ∗ , b3 ∗ , . . . , bn ∗
m n ∗ ∗ ρ y t − bj xtj . aj yt−j − j=1 j=1
k=0,1,...
is monotonically decreasing and bounded below by zero, hence it has a unique limit point. An existence of limit point of the sequence (k) (k) (k) a1 , a2 , a3 . . . , am (k) , r = 1, 2, . . . b1 (k) , b2 (k) , b3 (k) . . . , bn (k)
built by the algorithm is the global minimum because for any a 1 , a2 , a3 , . . . , a m b1 , b2 , b3 , . . . , bn
we have the following sequence of statements T m n ρ yt − aj ∗ yt−j − bj ∗ xtj = t=1 j=1 j=1 T m n ∗ (u∗ ) ∗ ≤ y ν − a y − b x j t−j j tj t t=1 j=1 j=1 ⇔ T m n (u∗ ) yt − ν a y − b x j t−j j tj t=1 j=1 j=1 m n ∗ ∗ ρ yt − a y − b x j t−j j tj × T j=1 j=1 ≤ m n t=1 ∗ ∗ yt − x a y − b tj j t−j j j=1 j=1 m n ∗ ∗ ρ yt − × x a y − b tj j t−j j T j=1 j=1 m n t=1 yt − a y − b x j t−j j tj j=1 j=1 m n T ∗ ∗ ≤ yt − a y − b x j t−j j tj t=1 j=1 j=1 ⇒ ⇔ T m n yt − a y − b x j t−j j tj t=1 j=1 j=1 T m n ∗ ∗ ≤ yt − ρ a y − b x j t−j j tj t=1 j=1 j=1 . T m n ρ yt − aj yt−j − bj xtj t=1 j=1 j=1
1669
The last implication is a consequence of the monotonicity of ρ. Theorem 3 now follows CONCLUSION The advantage of the proposed algorithm in front of the bushing is a sufficiently high rate of convergence with efficient use of linear programming methods. Indeed, the linear programming problem in step 2 for iteration k differs from the corresponding problem at step k − 1 only by the coefficients of the objective function which allows us to use the optimal basic solution of the previous iteration as the initial basic solution at the current iteration. A feature of finding the high-order autoregressive equation is the high sensitivity of the algorithm to rounding errors (see Panyukov and Golodov (2014)). One may eliminate this problem by using the unerring execution of basic arithmetic operations over the field of rational numbers (see Panyukov (2015)) and the application of parallelization (see Panyukov and Gorbik (2012)). REFERENCES
The first equality and the following inequality are a consequence of (8). The first equivalence relation is consequence of (7). The second equivalence relation is consequence of nonnegativity
Dielman., E.T. (2003). Least absolute value regression: recent contributions. Journal of Statistical Computation and Simulation, 75(4), 263–286. doi:dx.doi.org/10.1080/ 0094965042000223680. Huber, P. and Ronchetti, E.M. (2009). Robust Statistics, 2nd Edition. Wiley. Pan, J., Wang, H., and Qiwei, Y. (2007). Weighted least absolute deviations estimation for arma models with infinite variance. Econometric Theory, 23, 852–879. doi: 10.1017/S0266466607070363. Panyukov, A. (2015). Scalability of algorithms for arithmetic operations in radix notation. Reliable Computing, 19, 417–434. URL http://interval. louisiana.edu/reliable-computing-journal/ volume-19/reliable-computing-19-pp-417-434. pdf. Panyukov, A. and Golodov, V. (2014). Computing best possible pseudo-solutions to interval linear systems of equations. Reliable Computing, 19(2), 215–228. URL http://interval.louisiana. edu/reliable-computing-journal/volume-19/ reliable-computing-19-pp-215-228.pdf. Panyukov, A. and Gorbik, V. (2012). Using massively parallel computations for absolutely precise solution of the linear programming problems. Automation and Remote Control, 73(2), 276–290. doi:doi.org/10.1134/ S0005117912020063. Panyukov, A. and Tyrsin, A. (2008). Stable parametric identification of vibratory diagnostics objects. Journal of Vibroengineering, 10(2), 142–146. Powell, J.L. (1984). Least absolute deviations estimation for the censored regression model. Journal of Econometrics, 25, 303–325. Tyrsin, A. (2006). Robust construction of regression models based on the generalized least absolute deviations method. Journal of Mathematical Sciences, 139(3), 6634–6642. doi:doi.org/10.1007/s10958-006-0380-7.
1729
ScienceDirect
IFAC PapersOnLine 51-11 (2018) 1666–1669
Stable estimation Stable Stable estimation estimation of autoregressive model parameters Stable of autoregressive model of autoregressiveestimation model parameters parameters with exogenous variables on the basis of autoregressive model with exogenous variables on with exogenous variables parameters on the the basis basis of the generalized least absolute deviation with exogenous variables on the basis of the generalized least absolute deviation of the generalized least absolute deviation method of the generalizedmethod least absolute deviation method method ∗ ∗∗ ∗ Anatoly V. Panyukov ∗ Yasir Ali Mezaal ∗∗ ∗∗
Anatoly V. Panyukov Yasir Ali Mezaal ∗∗ Anatoly Anatoly V. V. Panyukov Panyukov ∗ Yasir Yasir Ali Ali Mezaal Mezaal ∗ ∗∗ ∗ Anatoly V. Panyukov Yasir Ali ∗ South Ural State University, Chelyabinsk, Mezaal Russia (e-mail: (e-mail: ∗ Ural State University, Chelyabinsk, Russia South Ural State University, Chelyabinsk, Russia (e-mail: ∗ South South Ural State [email protected]). University, Chelyabinsk, Russia (e-mail: [email protected]). [email protected]). ∗ ∗∗ South Ural State University, Chelyabinsk, Russia (e-mail: [email protected]). ∗∗ South Ural State Chelyabinsk, Russia (e-mail: ∗∗ South Ural State University, University, Chelyabinsk, Russia (e-mail: University, Chelyabinsk, Russia (e-mail: ∗∗ South Ural State [email protected]). South Ural State University, Chelyabinsk, Russia (e-mail: yaser ali [email protected]) yaser ali ali [email protected]) [email protected]) yaser ∗∗ South Ural State University, Chelyabinsk, Russia (e-mail: yaser ali [email protected]) yaser ali [email protected]) Abstract: The generalized least absolute deviation method is to the least Abstract: The The generalized generalized least least absolute absolute deviation deviation method method is is an an alternative alternative to to the the least least squares squares Abstract: Abstract: The generalized least absolute deviation method is an an alternative alternative to the the stability least squares squares method. Together with an appropriate choice of the loss function, it ensures and method. Together with an appropriate choice of the loss function, it ensures the stability and method. Together with an appropriate choice of the loss function, it ensures the stability and Abstract: The generalized least absolute deviation method is an alternative to the least squares method. Together with an appropriate choice of the loss function, it ensures the stability and efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the method. Together with an appropriate choice of the loss function, it ensures the stability and efficiency of estimating the coefficients of autoregressive models. This paper is devoted to the previously considered methods for finding the parameters of linear regression and autoregressive previously considered considered methods methods for for finding finding the parameters of linear regression and autoregressive previously the parameters of linear regression and autoregressive efficiency ofconsidered estimating the variables coefficients of the autoregressive models. This paperand is devoted to the previously methods for finding parameters of linear regression autoregressive models without exogenous extend to the problem of estimating the parameters of models without without exogenous variables extend to the problem problem of estimating estimating the parameters of models exogenous variables extend to the of the parameters of previously considered methods for finding the parameters of linear regression and autoregressive models without exogenous variables extend to the problem of estimating the parameters of autoregressive models with exogenous variables. autoregressive models with exogenous variables. autoregressive models with exogenous variables. models withoutmodels exogenous variables extend to the problem of estimating the parameters of autoregressive with exogenous variables. © 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. autoregressive models with exogenous variables. Keywords: Demand Forecasting; Optimization and Control; Keywords: Demand Demand Forecasting; Forecasting; Optimization Optimization and and Control; Control; Computational Computational Science Science Keywords: Keywords: Demand Forecasting; Optimization and Control; Computational Computational Science Science Keywords: Demand Forecasting; Optimization and Control; Computational Science 1. INTRODUCTION The interrelation of methods established in Panyukov and 1. INTRODUCTION INTRODUCTION The interrelation of methods established in Panyukov and 1. The interrelation of methods established in Panyukov and 1. INTRODUCTION The interrelation of methods established in Panyukov Tyrsin (2008) made it possible to reduce the problem of Tyrsin (2008) (2008) made made it it possible possible to to reduce reduce the the problem problemand of Tyrsin of 1. INTRODUCTION The interrelation of methods established inthe Panyukov and Tyrsin (2008) made it possible toto reduce problem of determining the GLAD estimates an iterative procedure determining the GLAD estimates to an iterative procedure Let us consider the evaluation of the coefficients of the determining the GLAD estimates to an iterative procedure Let us us consider consider the the evaluation evaluation of of the the coefficients coefficients of of the the Tyrsin (2008) made it possible to reduce the problem of determining the GLAD estimates to an iterative procedure with WLAD estimates. The latter is calculated by solvLet WLAD estimates. The latter is calculated by solvLet usautoregressive consider the evaluation of the coefficients of the with linear equation with exogenous variables with WLAD estimates. The latter is calculated by solvlinear autoregressive equation with with exogenous variables determining the GLAD estimates to an iterative procedure with WLAD estimates. The latter is calculated by solving the corresponding linear programming problem. The linear autoregressive equation exogenous variables ing the the corresponding corresponding linear linear programming programming problem. problem. The The Let usautoregressive consider the evaluation of the coefficients of the ing linear equation with exogenous variables m n with WLAD estimates. The latter is calculated by solving the corresponding linear programming problem. The sufficient condition Panyukov and Tyrsin Tyrsin (2008) imposed m n m n sufficient condition Panyukov and (2008) imposed linear autoregressive equation with exogenous variables sufficient condition Panyukov and Tyrsin (2008) imposed m n ing the corresponding linear programming problem. The yyt = a y + b x + , t = 1, 2, . . . , T, (1) sufficient condition Panyukov and Tyrsin (2008) imposed on the loss function is found to ensure the stability of j t−j j tj t = m a + bjj x + , tt = 1, 2, . . . , T, (1) on the loss function is found to ensure the stability of ajj yyt−j xtj the loss function is found to ensure the stability of t−j + n b tj + tt , yyttt = = t= = 1, 1, 2, 2, .. .. .. ,, T, T, (1) (1) on sufficient condition Panyukov and Tyrsin (2008) imposed on the loss function is found to ensure the stability of j=1 aj yt−j + j=1 bj xtj + t , the GLAD estimators of the autoregressive models coefj=1 j=1 the GLAD estimators of the autoregressive models coefj=1 j=1 the GLAD estimators of the autoregressive models coefyt = j=1 aj yt−j + j=1 bj xtj + t , t = 1, 2, . . . , T, (1) on the loss function is found to ensure the stability of the GLAD estimators of the autoregressive models coefficients under emission conditions. It ensures the stability ficients under under emission emission conditions. conditions. It It ensures ensures the the stability stability ficients ,, yy2 ,, .. .. .. ,, yyn are the values of the state varihere yy1j=1 j=1 the GLAD estimators of the autoregressive models coefare the values of the state varihere ficients under emission conditions. It ensures the stability of GLAD-estimates GLAD-estimates of of autoregressive autoregressive models models in in terms terms of of the values of the varihere yy11 ,, yy22 ,, .. .. .. ,, yynn are of the values of the state state varihere of GLAD-estimates of models in of able, x are the values controls at time 1 22t ,, .. .. .. ,,nx 1t ,, x ntare ficients under emission conditions. It ensures stability able, x x x are the values of controls at time of GLAD-estimates of autoregressive autoregressive models the in terms terms of outliers. 1t 2t nt able, x , x , . . . , x are the values of controls at time 1t 2t nt outliers. , y , . . . , y are the values of the state varihere y able, x , x , . . . , x are the values controls at time outliers. points t = 1, 2, . . . , T , , , . . . , are random errors, 1 2 n 1t 2t nt 1 2 t of GLAD-estimates of autoregressive models in terms of points tt = 1, 2, .. .. .. ,, T ,, 11 ,, 22 ,, .. .. .. ,, tt are random errors, outliers. points = 1, 2, T are random errors, able, x.. ..2t,,1, ,a.m . . .,and of time outliers. points t3 ,..= 2, .x.nt , Tbbare , ,b1 ,the . . ., ,b t are arecontrols randomaterrors, Special features features of of the the GLAD GLAD method method application application for for the the a a2 ,,xa a1t and , b2 ,. values are unknown coeffi1 ,, a Special a unknown coeffi1 2 3 m m Special features of GLAD application for a .. .. ,,1,a a and bb,111 ,,,bbb222,,,, bbb333,... ... ... .,,, ,bbbm are unknown coeffi1 ,, a 2 ,, a 3 ..= m m points t 2, . . . , T are random errors, Special features of the the GLAD method method application for the the a a a a and are unknown coefficonstruction of the regression equation are considered in cients. 1 2 t 1 2 3 m 1 2 3 m construction of the regression equation are considered in cients. construction of the regression equation are considered in cients. Special features of the GLAD method application for the a construction of the regression equation are considered in cients. Tyrsin (2006). Special features of the use of GLAD for 1 , a2 , a3 . . . , am and b1 , b2 , b3 . . . , bm are unknown coeffiTyrsin (2006). Special features of the use of GLAD for Tyrsin (2006). Special features of the use of GLAD for Ordinary Least Least Squares Squares (OLS) (OLS) is is the the parametric parametric method method construction of the regression equation are considered in Ordinary cients. Tyrsin (2006). Special features of the use of GLAD for constructing the autoregressive equation without exogeOrdinary Least Squares (OLS) is the parametric method the autoregressive equation without exogeOrdinary Least Squares (OLS) of is the parametric method constructing constructing the autoregressive equation without exogein common used for estimation regression equation Tyrsin (2006). Special features of the use of GLAD for in common used for estimation of the regression equation constructing the autoregressive equation without exogenous variables variables are are considered considered in in Panyukov Panyukov and and Tyrsin Tyrsin in common used for of the parametric regression equation nous Ordinary Squares (OLS) isassumptions the method in commonLeast used for estimation estimation of regression equation nous variables are considered in Panyukov and Tyrsin coefficients. We need some strict to use OLS. constructing the autoregressive equation without exogecoefficients. We need some strict assumptions to use OLS. nous variables are considered in Panyukov and Tyrsin (2008). This paper is devoted to extension of the precoefficients. We need some strict assumptions to use OLS. (2008). This paper is devoted to extension of the prein common used for estimation of the regression equation coefficients. We need some strict assumptions to use OLS. (2008). This paper is devoted to extension of the preThey include independence and normal distribution of ernous variables are considered in Panyukov and Tyrsin They include independence and normal distribution of er(2008). This paper is devoted to extension of the previously discussed methods to the problem of estimating They include independence and normal distribution of erviously discussed methods to the problem of estimating coefficients. We need some strict assumptions to use OLS. They include independence and normal distribution of erviously discussed methods to the problem of estimating rors and and determinacy determinacy of of explanatory explanatory variables variables (see (see Huber Huber (2008). This paper is devoted to extension of the prerors viously discussed methods to the problem of estimating the parameters of autoregressive models with exogenous rors and determinacy of explanatory variables (see Huber parameters of autoregressive models with exogenous They include independence andminor normal distribution of er- the rors and determinacy of explanatory variables (see Huber the parameters of autoregressive models with exogenous and Ronchetti (2009)). Even violations of stated viously discussed methods to the problem of estimating and Ronchetti (2009)). Even minor violations of stated the parameters of autoregressive models with exogenous variables. and Ronchetti (2009)). Even minor violations of Huber stated rors and determinacy of explanatory and Ronchetti (2009)). Even minor violations(see stated variables. variables. assumptions dramatically lower the efficiency of the parameters of autoregressive models with exogenous assumptions dramatically lower the variables efficiency ofofestimaestimavariables. assumptions dramatically lower the efficiency of estimaand Ronchetti (2009)). Even minor violations of stated assumptions dramatically lower the efficiency of estimators. Let us note the instability of OLS estimation process variables. tors. Let us note the instability of OLS estimation process tors. Let us note the instability of OLS estimation process 2. THE WLADAND assumptions dramatically lower efficiency of estimators. Letof uspresence note theof instability ofthe OLS estimation process in case large measurements errors. In this 2. THE RELATIONSHIP RELATIONSHIP BETWEEN BETWEEN WLADAND in case of presence of large measurements errors. In this 2. BETWEEN WLADAND in case of presence of large measurements errors. In this 2. THE THE RELATIONSHIP RELATIONSHIP BETWEEN WLADAND GLAD ESTIMATES tors. Let us note the instability of OLS estimation process in case of presence of large measurements errors. In this case, estimated coefficients become inconsistent. Finding GLAD ESTIMATES case, estimated coefficients become inconsistent. Finding GLAD ESTIMATES case, estimated coefficients become inconsistent. Finding 2. THE RELATIONSHIP BETWEEN WLADAND GLAD ESTIMATES in case of of of largeequation measurements errors. In this case, estimated coefficients become inconsistent. Finding estimates autoregressive becomes substantially estimates ofpresence autoregressive equation becomes substantially estimates of autoregressive equation becomes substantially GLAD ESTIMATES case, estimated coefficients become inconsistent. Finding estimates of autoregressive equation becomes substantially One can can get get the the WLAD WLAD estimations estimations of of coefficients coefficients by by complicated complicated due due to to the the poor poor conditionality conditionality of of the the equations equations One can get the complicated due poor conditionality of equations One canthe get the WLAD WLAD estimations estimations of of coefficients coefficients by by estimates of autoregressive becomes substantially complicated due to to the the poorequation conditionality of the the equations One solving problem system representing necessary conditions for minimization solving the problem system representing necessary conditions for minimization solving problem system representing necessary conditions for minimization One can∗the the WLAD estimations of coefficients by solving theget problem complicated due to the poor conditionality of the equations system representing necessary conditions for minimization of squared deviations sum. ∗ ∗ ∗ ∗ ∗ of squared deviations sum. ∗ ∗ (a ,, .. .. problem .. ,, a of deviations sum. ∗ ,, a ∗ ,, b ∗ ,, b 1 1 (a a∗2∗2∗2the a∗m b∗2∗ ,, .. .. .. ,, bb∗n∗ )) = = system representing conditions for minimization solving of squared squared deviationsnecessary sum. ∗ (a m, b (a1∗11 ,, a a2 ,, .. .. .. ,, a am , bb1∗11 ,, bbT2∗22 ,, .. .. .. ,, bbn∗nn ))m= = Least Absolute Deviations (LAD) method of Powell (1984) m n Least Absolute Deviations (LAD) method of Powell (1984) of squared deviations sum. ∗ ∗ ∗ ∗ ∗ ∗ T m n Least Absolute Deviations (LAD) method of Powell (1984) T m n (a1 , a2 , . . . , am , b1 , bT2 , . . . , bn )m= Least Absolute Deviations (LAD) method of Powell (1984) is a method alternative to OLS. It allows to obtain n is aa method alternative to OLS. It allows to obtain arg min yytt − a bbjj x jy t−j − tj (2) is method alternative to OLS. It allows to obtain arg min − a y − x m j t−j tj y (2) arg min − a y − b x Least Absolute Deviations (LAD) method of Powell (1984) (2) is a method alternative to OLS. It allows to obtain (a ,a ,...,a )∈R robust errors in case of violation of OLS assumptions (see t j t−j j tj m, T 1 2 m m n m arg (a minm )∈R , yt − robust errors in case of violation of OLS assumptions (see j=1 aj yt−j − j=1 bj xtj (2) (a(b111,a ,a,b222,...,a ,...,a )∈Rn robust errors in case of violation of OLS assumptions (see m m , t=1 ,...,b )∈R t=1 j=1 j=1 n n (a ,a ,...,a )∈R , is a method alternative to OLS. It allows to obtain robust errors in case of violation of OLS assumptions (see t=1 j=1 j=1 11 ,b 22 ,...,bm n Dielman. (2003)). We present two types of LAD: Weighted (b )∈R n y arg min − a y − b x (b ,b ,...,b )∈R t=1 j=1 j=1 Dielman. (2003)). (2003)). We We present present two two types types of of LAD: LAD: Weighted Weighted 1 2 n n j t−j j tj (2) t Dielman. m 1 2 ,...,bm n )∈R (a(b1p ,a,b )∈R , 1, 2, robust errors in case ofpresent violation OLS assumptions (see where Dielman. (2003)). We twooftypes of LAD: Weighted LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, tt = .. .. .. ,, T are predetermined coef t2 ,...,a t=1 j=1 j=1 n where p LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, = 1, 2, T are predetermined coeftt 2 ,...,b where LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, tt = 1, 2, .. .. .. ,, T are predetermined coef(bp 1 ,b n )∈R Dielman. (2003)). We present two types of LAD: Weighted where p LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, = 1, 2, T are predetermined coefmethod ( see Tyrsin (2006)). ficients. This problem represents the problem of convex t method ( see Tyrsin (2006)). ficients. This problem represents the problem of convex method ( see Tyrsin (2006)). ficients. This problem represents the problem of convex where p LAD method (see Pan et al. (2007)) and Generalized LAD ≥ 0, t = 1, 2, . . . , T are predetermined coefmethod ( see Tyrsin (2006)). ficients. tThis problem represents the problem of convex method ( see Tyrsin (2006)). ficients. This problem represents the problem of convex 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2018, 2018 IFAC 1726Hosting by Elsevier Ltd. All rights reserved. Copyright 2018 1726 Copyright ©under 2018 IFAC IFAC 1726Control. Peer review© of International Federation of Automatic Copyright © 2018 responsibility IFAC 1726 10.1016/j.ifacol.2018.08.217 Copyright © 2018 IFAC 1726
IFAC INCOM 2018 Bergamo, Italy, June 11-13, 2018
Anatoly V. Panyukov et al. / IFAC PapersOnLine 51-11 (2018) 1666–1669
piecewise linear optimization, and the introduction of additional variables reduces it to the problem of linear programming T min pt u t : m (a1 ,a2 ,...,am )∈R , (b1 ,b2 ,...,bn )∈Rn (u1 ,u2 ,...,uT )∈RT
−ut ≤ yt −
m j=1
aj yt−j −
n j=1
bj xtj
(1) for each collection of weights {pt }nt=1 arg
The GLAD-estimates can be obtained from the solution of the problem . ., bn ∗ ) = (a1 ∗ , . . . , am ∗ , b1 ∗ , . T m n arg min m ρ yt − aj yt−j − bj xtj , (a1 ,...,am )∈R t=1 j=1 j=1 n (b1 ,...,bn )∈R
(4) where ρ(∗) is a convex upward monotonically increasing twice continuously differentiable function such that ρ(0) = 0. Theorem 1. All local minima of the GLAD-estimation problem for the coefficients of the autoregressive equation (1) belong to the set a1 (k) , a2 (k) , . . . , am (k) , (k) (k) (k) : b1 , b2 , . . . , bn m n . U = yt = aj yt−j + bj xtj , j=1 j=1 t ∈ k = {k1 , k2 , . . . , km+n : 1 ≤ k1 < k2 < . . . < km+n ≤ T } Proof. The set U contains the solutions of all possible joint systems of m + n linearly independent equations m n yt = aj yt−j + bj xtj , t ∈ k j=1
with m + n unknowns a1 , a2 , a3 . . . , am , b1 , b2 , b3 . . . , bn .
If the solution (a1 , a2 , a3 . . . , am , b1 , b2 , b3 . . . , bn ) ∈ U then there exists an -neighbourhood for which the loss function is continuous and convex upwards. Consequently, such a solution can not be a local minimum. The theorem now follows Obviously, the number of systems is equal Cn+m . Thus, T the solution of problem (3) can be reduced to choosing the best of Cn+m solutions of linear algebraic equations T systems. This approach is applicable for m ≤ 3 . To compute GLAD-estimates for higher order dimension problems the interrelation between WLAD and GLAD-estimates have to be used from
min
(a1 ,a2 ,...,am )∈Rm (b1 ,b2 ,...,bn )∈Rn
n m pt × yt − bj xtj ∈ U ; aj yt−j − t=1 j=1 j=1
n
(3)
ut ≥ 0, t = 1, 2, . . . , T This problem has a canonical form with n + m + T + 1 variables and 3n inequality constraints including the conditions for the non-negativity of the variables uj , j = 1, 2, . . . , T . The main problem with the use of WLAD method is the absence of general formal rules for choosing weight coefficients. Consequently, this approach requires additional research.
j=1
Theorem 2. Let U be the set of local extreme of problem (4) then:
t=1
≤ ut , .
1667
(2) for all a1 (k) , a2 (k) , . . . , am (k) ,
(5)
b1 (k) , b2 (k) , . . . , bn (k) ∈ U
there is such a collection of weights {pt }nt=1 that a1 (k) , a2 (k) , . . . , am (k) , b1 (k) , b2 (k) , . . . , bn (k) = n m n arg min pt yt − aj yt−j − bj xtj . m (a1 ,a2 ,...,am )∈R t=1 j=1 j=1 n (b1 ,b2 ,...,bn )∈R
(6)
Proof. The proof of the first part of the theorem essentially repeats the proof of Theorem 1. The validity of the second part of the theorem follows from the fact that the weights of the active part of the constraints can be considered as non-zero, and the weights of the inactive part are equal to zero. In this case the minimal value of the loss function is equal to zero and it is achieved by solving the chosen system of equations. The theorem now follows Theorems 1 and 2 give a way to determine the weight coefficients for the linear programming problem (3) and thus allow the problem (4) to be reduced to solving a sequence of linear programming problems (3). 3. ALGORITHM FOR COMPUTING GLAD-ESTIMATES Primal solution of problem (4) is based on the usage of theorem 1 and involves finding all node points and choosing one of them as a solution that ensures the minimum of the objective function. The brute force algorithm requires the solution of Cn+m T linear equations systems of order m+n. For large values of n and m this leads to a significant computational complexity. An alternative approach is based on the reduction of this problem to the sequence linear programming problems (3). Consider possible algorithms based on this approach. Algorithm GLAD-estimator Input: number of measures T ; values {yt }Tt=1 of the endogenous variable; values {{xtj }Tt=1 }nj=1 of exogenous variables; function ρ(∗). Output: estimation of coefficients of autoregressive equation (a1 ∗ , a2 ∗ , a3 ∗ . . . , am ∗ , b1 ∗ , b2 ∗ , . . . , bn ∗ ) . Step 1. For all t = 1, 2, . . . , T do pt = 1; k := 0;
1727
IFAC INCOM 2018 1668 Bergamo, Italy, June 11-13, 2018
Anatoly V. Panyukov et al. / IFAC PapersOnLine 51-11 (2018) 1666–1669
m n (k) (k) ρ yt − aj yt−j − bj xtj = t=1 j=1 j=1 m n (k) ρ yt − a y − bj (k) xtj − j t−j j=1 j=1 n m n (k) (k) pt · yt − aj yt−j − bj xtj + ≥ t=1 j=1 j=1 m n (k) (k) pt · y t − aj yt−j − bj xtj j=1 j=1 m n (k) (k) ρ yt − aj yt−j − bj xtj − n j=1 j=1 + n m t=1 (k) (k) pt · yt − aj yt−j − bj xtj j=1 j=1 n min pt × m
a1 (k) , a2 (k) , a3 (k) . . . , am (k) b1 (k) , b2 (k) , b3 (k) . . . , bn (k) := (k) (k) (k) (k) u1 , u2 , u3 , . . . , ut T arg min pt u t : m (a1 ,a2 ,...,am )∈R t=1 (b1 ,b2 ,,...,bn )∈Rn T (u1 ,u2 ,,...,ut )∈R m n
−ut ≤ yt −
j=1
aj yt−j −
bj xtj
j=1
ut ≥ 0, t = 1, 2, . . . , T Step 2. For all t = 1, 2, . . . , T do k := k + 1; a1 (k) , a2 (k) , a3 (k) . . . , am (k) b1 (k) , b2 (k) , b3 (k) . . . , bn (k) u1 (k) , u2 (k) , u3 (k) , . . . , ut (k) arg
min
(a1 ,a2 ,...,am )∈Rm (b1 ,b2 ,,...,bn )∈Rn (u1 ,u2 ,,...,ut )∈RT m
−ut ≤ yt −
j=1
n
≤ ut , .
:=
T
(a1 ,a2 ,...,am )∈R
n
pt u t :
t=1
aj yt−j −
n
bj xtj
j=1
ut ≥ 0, t = 1, 2, . . . , T
≤ ut , .
Step 3. If (k) (k) (k) a1 , a2 , a3 . . . , am (k) = b1 (k) , b2 (k) , b3 (k) . . . , bn (k) (k−1) (k−1) (k−1) , a2 , a3 . . . , am (k−1) a1 b1 (k−1) , b2 (k−1) , b3 (k−1) . . . , bn (k−1) then go to Step 2. Step 4. Stop. Target values are (k) (k) (k) a1 , a2 , a3 . . . , am (k) . b1 (k) , b2 (k) , b3 (k) . . . , bn (k) Performance justification of this algorithm leads us to the following theorem. Theorem 3. If the loss function ρ(∗) is convex upward, monotone increasing, continuously differentiable on the positive semi-axis, and satisfies the condition ρ(0) = M < ∞ then the sequence (k) (k) (k) a1 , a2 , a3 . . . , am (k) b1 (k) , b2 (k) , b3 (k) . . . , bn (k) constructed by the GLAD-estimation algorithm converges to the global extremum of problem (4).
The first equality and the inequality following it are obvious. The second equation is a consequence of changing the notation of the variables in step 1. The third equation is the result of the choice of the weight coefficients in step 2 and equality (7). The last inequality is a consequence of relations (8). Therefore
(k)
(8)
m n (k) (k) ρ yt − aj yt−j − bj xtj = t=1 j=1 j=1 n n m (k) (k+1) u ) (k+1) ( aj yt−j − bj xtj ≥ ν yt − t=1 j=1 j=1 n m n ρ yt − aj (k+1) yt−j − bj (k+1) xtj , t=1 j=1 j=1 n
Proof. It follows from the requirements imposed on the function ρ(∗) that at any point uk an approximation (k) ν (u ) (u) = ρ(u(k) ) − ρ (u(k) ) · u(k) + ρ (u(k) ) · u (7) is a majorant, i.e. (∀u = uk )(ρ(u) < ν (u ) (u)), ρ(uk ) = ν(uk ). Therefore, in accordance with the algorithm
t=1
(b1 ,b2 ,...,bn )∈R m n = yt − a y − b x j t−j j tj j=1 j=1 m n (k) (k) ρ yt − a y − b x j t−j j tj − n j=1 j=1 + m n t=1 (k) (k) pt · yt − a y − b x j t−j j tj j=1 j=1 n m n (k+1) (k+1) = pt · yt − x a y − b tj j t−j j t=1 j=1 j=1 n m n (k) ν (u ) yt − aj (k+1) yt−j − bj (k+1) xtj ≥ t=1 j=1 j=1 n m n ρ yt − aj (k) yt−j − bj (k) xtj . t=1 j=1 j=1
moreover, equality is attained only if for all t = 1, 2, . . . n and for all k = 1, 2, . . . , m. That is why the sequence
1728
IFAC INCOM 2018 Bergamo, Italy, June 11-13, 2018
Anatoly V. Panyukov et al. / IFAC PapersOnLine 51-11 (2018) 1666–1669
n n m (k) (k) bj xtj aj yt−j − ρ yt − t=1 j=1 j=1
follows from continuity and monotonicity of functions ρ(∗) . The limit point
a 1 ∗ , a2 ∗ , a3 ∗ , . . . , a m ∗ b1 ∗ , b2 ∗ , b3 ∗ , . . . , bn ∗
m n ∗ ∗ ρ y t − bj xtj . aj yt−j − j=1 j=1
k=0,1,...
is monotonically decreasing and bounded below by zero, hence it has a unique limit point. An existence of limit point of the sequence (k) (k) (k) a1 , a2 , a3 . . . , am (k) , r = 1, 2, . . . b1 (k) , b2 (k) , b3 (k) . . . , bn (k)
built by the algorithm is the global minimum because for any a 1 , a2 , a3 , . . . , a m b1 , b2 , b3 , . . . , bn
we have the following sequence of statements T m n ρ yt − aj ∗ yt−j − bj ∗ xtj = t=1 j=1 j=1 T m n ∗ (u∗ ) ∗ ≤ y ν − a y − b x j t−j j tj t t=1 j=1 j=1 ⇔ T m n (u∗ ) yt − ν a y − b x j t−j j tj t=1 j=1 j=1 m n ∗ ∗ ρ yt − a y − b x j t−j j tj × T j=1 j=1 ≤ m n t=1 ∗ ∗ yt − x a y − b tj j t−j j j=1 j=1 m n ∗ ∗ ρ yt − × x a y − b tj j t−j j T j=1 j=1 m n t=1 yt − a y − b x j t−j j tj j=1 j=1 m n T ∗ ∗ ≤ yt − a y − b x j t−j j tj t=1 j=1 j=1 ⇒ ⇔ T m n yt − a y − b x j t−j j tj t=1 j=1 j=1 T m n ∗ ∗ ≤ yt − ρ a y − b x j t−j j tj t=1 j=1 j=1 . T m n ρ yt − aj yt−j − bj xtj t=1 j=1 j=1
1669
The last implication is a consequence of the monotonicity of ρ. Theorem 3 now follows CONCLUSION The advantage of the proposed algorithm in front of the bushing is a sufficiently high rate of convergence with efficient use of linear programming methods. Indeed, the linear programming problem in step 2 for iteration k differs from the corresponding problem at step k − 1 only by the coefficients of the objective function which allows us to use the optimal basic solution of the previous iteration as the initial basic solution at the current iteration. A feature of finding the high-order autoregressive equation is the high sensitivity of the algorithm to rounding errors (see Panyukov and Golodov (2014)). One may eliminate this problem by using the unerring execution of basic arithmetic operations over the field of rational numbers (see Panyukov (2015)) and the application of parallelization (see Panyukov and Gorbik (2012)). REFERENCES
The first equality and the following inequality are a consequence of (8). The first equivalence relation is consequence of (7). The second equivalence relation is consequence of nonnegativity
Dielman., E.T. (2003). Least absolute value regression: recent contributions. Journal of Statistical Computation and Simulation, 75(4), 263–286. doi:dx.doi.org/10.1080/ 0094965042000223680. Huber, P. and Ronchetti, E.M. (2009). Robust Statistics, 2nd Edition. Wiley. Pan, J., Wang, H., and Qiwei, Y. (2007). Weighted least absolute deviations estimation for arma models with infinite variance. Econometric Theory, 23, 852–879. doi: 10.1017/S0266466607070363. Panyukov, A. (2015). Scalability of algorithms for arithmetic operations in radix notation. Reliable Computing, 19, 417–434. URL http://interval. louisiana.edu/reliable-computing-journal/ volume-19/reliable-computing-19-pp-417-434. pdf. Panyukov, A. and Golodov, V. (2014). Computing best possible pseudo-solutions to interval linear systems of equations. Reliable Computing, 19(2), 215–228. URL http://interval.louisiana. edu/reliable-computing-journal/volume-19/ reliable-computing-19-pp-215-228.pdf. Panyukov, A. and Gorbik, V. (2012). Using massively parallel computations for absolutely precise solution of the linear programming problems. Automation and Remote Control, 73(2), 276–290. doi:doi.org/10.1134/ S0005117912020063. Panyukov, A. and Tyrsin, A. (2008). Stable parametric identification of vibratory diagnostics objects. Journal of Vibroengineering, 10(2), 142–146. Powell, J.L. (1984). Least absolute deviations estimation for the censored regression model. Journal of Econometrics, 25, 303–325. Tyrsin, A. (2006). Robust construction of regression models based on the generalized least absolute deviations method. Journal of Mathematical Sciences, 139(3), 6634–6642. doi:doi.org/10.1007/s10958-006-0380-7.
1729