- Email: [email protected]
Semi-nonparametric estimation with Bernstein polynomials
Economics Letters 89 (2005) 153 – 156 www.elsevier.com/locate/econbase
Semi-nonparametric estimation with Bernstein polynomials Pok Man Chaka,T, Neal Madrasb, Barry Smithc a
b
Department of Economics, Wilfrid Laurier University, Waterloo, Ontario, Canada N2L 3C5 Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada M3J 1P3 c Department of Economics, York University, Toronto, Ontario, Canada M3J 1P3 Received 9 March 2004; accepted 7 January 2005 Available online 24 August 2005
Abstract This paper proposes a semi-nonparametric model based upon Bernstein polynomials. We prove that the univariate Bernstein semi-nonparametric estimator is consistent for the true function and its first and second derivatives, and thus it is globally flexible and globally regular. D 2005 Elsevier B.V. All rights reserved. Keywords: Flexible functional forms; Bernstein polynomials JEL classification: C13; C14; C20; C51
1. Introduction Since the introduction of Cobb–Douglas function in 1928, economists have searched for flexible functional forms having more desirable properties. The functional forms developed in the 1970s are finite parameter models1 and can provide up to a second-order approximation of an arbitrary function at a specific point, and hence they are locally flexible in the approximation setting. In estimation, they often perform poorly. Gallant’s (1981, 1982, 1984) Fourier model, an infinite parameter model, provided a new direction in the search for flexible functional forms. Here Sobolev flexibility or global flexibility T Corresponding author. Tel.: +1 519 884 0710 x 2654; fax: +1 519 888 1015. E-mail addresses: [email protected] (P.M. Chak)8 [email protected] (N. Madras)8 [email protected] (B. Smith). 1 For example, see Christensen et al. (1971) and Diewert (1971). 0165-1765/$ - see front matter D 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.econlet.2005.01.025
154
P.M. Chak et al. / Economics Letters 89 (2005) 153–156
was also defined. In 1983, Barnett and Jonas (1983) suggested a second infinite parameter model based upon the Mu¨ntz–Szatz series. They called it the Asymptotically Ideal Model (AIM). However, Terrell (1995) pointed out that there is a trade-off between flexibility and regularity in the AIM model. In this paper, we describe the Bernstein functional form and show its usefulness in approximating and estimating functions which are continuous, non-negative, monotone non-decreasing and strictly concave. These shape restrictions are the regularity conditions for a cost function in its intensive form. In approximation, the Bernstein model is shown to be globally flexible. In estimation, we show that the sequence of our univariate Bernstein estimators and the sequences of the first and second derivatives of the estimators converge to the true function and its first and second derivatives in the L 2 metric space. This implies that the univariate Bernstein estimator and the first and second derivatives of the estimator are consistent for the true function and its first and second derivatives. It follows that our estimator is globally flexible. Moreover, linear constraints on the parameters are sufficient to guarantee that our estimator is monotone and concave for all sample sizes. Therefore, our univariate Bernstein estimator is globally flexible and globally regular.
2. Bernstein polynomials in approximation The Bernstein polynomial approximation of order n of a function f(x)is defined as: n v X f Pv;n ð xÞ; n v¼0 n v x ð1 xÞnv and xa½0; 1; naN. Bernstein (1912) used these polynomials to where Pv;n ð xÞ ¼ v provide a simpler proof of the approximation theorem of Weierstrass. The sequence (in terms of n 1 and n 2) of two-variable Bernstein polynomials first appeared in Kingsley (1951); it is defined as: n1 X n2 X v1 v2 f f ; Pv1 ; n1 ðx1 ÞPv2 ; n2 ðx2 Þ Bn1 ;n2 ðx1 ; x2 Þ ¼ n1 n2 v1 ¼0 v2 ¼0
Bnf ð xÞ ¼
where (x 1, x 2) a [0, 1] [0, 1] and n1 ; n2 aN. The most important property of bivariate Bernstein polynomials for our purpose is the convergence of the true function and its derivatives. Butzer (1953) and Kingsley (1951) show that, given a mild restriction on the growth of n 1 and n 2, the sequences of derivatives of the bivariate Bernstein polynomials converge at least pointwise to those of their respective true derivatives as n 1, n 2Yl. This means that two-dimensional Bernstein polynomials are shapepreserving functions for large n 1 and n 2, and hence global flexibility is guaranteed. If the true function has continuous derivatives on [0, 1] [0, 1], then we have uniform convergence.
3. Bernstein polynomials in estimation Consider the following univariate model: Yi ¼ f ðXi Þ þ ei ;
P.M. Chak et al. / Economics Letters 89 (2005) 153–156
155
where (X i , Yi ) are i.i.d. on [0, 1] [0, d¯ ] with joint probability measure P and E P [qi jX i ] = 0, and f is an arbitrary continuous, non-negative, monotone non-decreasing and strictly concave function with continuous third derivatives on (0, 1). We assume that Pr[X = 0] = 0. The regression of Y on X is EP ½Yi jXi ¼ xi ¼ f ðxi Þ; which is the unknown function. The data available for estimation are realizations ( y i of Yi and x i of X i ) yi ¼ f ðxi Þ þ ei : To estimate this model, we use Bernstein polynomials which are defined as follows: Bn ð x Þ ¼
n X
cv Pv;n ð xÞ;
v¼0
where the c v ’s are coefficients that are restricted in such a way that B n is non-negative, monotone nondecreasing, and strictly concave. The restrictions are: for non-negativity, c v z 0; for monotonicity, c v+1 c v z 0; for strict concavity, c v 2c v+1 + c v+2 b 0, v = 0, 1,. . ., n. We call it the Bernstein seminonparametric estimator. To carry out the estimation, we use the sieve method introduced by Grenander in 1981 as described in Grenander (1981), Geman (1981), and Geman and Hwang (1982). In this method, the best fitting restricted function is chosen from a subset of the parameter space, and this subset is allowed to grow with the sample size. This sequence of subsets is called a sieve. Let Y 0 be the set of all continuous, non-negative functions f from [0, 1] into [0, d¯ ] with the property that f is monotone nondecreasing, strictly concave with continuous derivatives on (0, 1). Also let Y 0[A] be the set of functions f in Y 0 that satisfy the third-derivative bounds |d3f/dx 3| V A. We define a stage T sieve to be the set of all Bernstein polynomials B n of degree n in Y 0[A], i.e., n o ½ A ½ A ST ¼ fT uBn jBn aY 0 ; where T is the sample size and n is a function of T, and nYl as TYl. The constant A can be finite or [A] o S T[A], 8T, and thus the sieve grows with the sample size T. For each T, the infinite. Note that ST1 optimal estimate ˆf T is selected from the Bernstein polynomials in S T[A], and is defined as follows: T n o 1 X yi fT ðxi Þ2 : fˆ T ¼ arg min ½ A fT aST T i¼1
Choosing ˆf T is equivalent to choosing a vector of (T + 1) optimal, restricted coefficients {cˆ 0, cˆ 1,. . ., cˆ T }. Because the constraints are all linear in the parameters, this optimization problem can be solved by quadratic programming methods. In the following theorem, we prove the L 2 and pointwise convergences of {fˆT }, {dfˆT /dx} and {d2ˆf T / dx 2} to f, df/dx and d2f/dx 2. Theorem. Let D b A b l. Assume that f a Y 0[D] and ˆf T a S T[A] . Also assume that the support of P is the whole interval [0, 1]. Then with probability one, {fˆT }, {dfˆT /dx} and {d 2ˆf T /dx 2 } converge to f, df/dx and d 2 f/dx 2 on [0, 1] in the L 2 (P) metric space and pointwise on [0, 1]. Proof. See Chak (2001), Theorems 3.2.7, 3.3.4 and 3.3.5.
156
P.M. Chak et al. / Economics Letters 89 (2005) 153–156
The assumption on the support of P can be weakened, leading to appropriate changes in the conclusions. The main concern is that the support of P should not have isolated points, because the derivatives cannot be estimated reliably at such points. Also, the L 2 convergence of {fˆT } to f can be achieved even when f a Y 0[l] and ˆf T a S T[l]. The result in the theorem can be extended to the multivariate case (Chak, 2001). 4. Conclusion We have examined the Bernstein semi-nonparametric model in approximation and in estimation. In approximation, among the many interesting properties of Bernstein polynomials is the global flexibility property which allows Bernstein polynomials to preserve the shape of the true function. In estimation, we obtain the consistency results for the one-dimensional Bernstein estimator and its first and second derivatives, and establish the global flexibility property for this estimator. Future work on the Bernstein estimator includes establishing consistency results for the multivariate case, rates of convergence and asymptotic normality. Acknowledgments The author would like to acknowledge the support by the Natural Sciences and Engineering Research Council of Canada. References Barnett, W.A., Jonas, A.B., 1983. The Mu¨ntz–Szatz demand system: an application of a globally well behaved series expansion. Economics Letters 11, 337 – 342. Bernstein, S., 1912. De´monstration du the´ore`me de Weierstrass, fondee´ sur le calcul des probabilite´s. Communications of Kharkov Mathematical Society 13 (2), 1 – 2. Butzer, Paul L., 1953. On two-dimensional Bernstein polynomials. Canadian Journal of Mathematics 5, 107 – 113. Chak, Pok Man, 2001. Approximation and consistent estimation of shape-restricted functions and their derivatives. Ph.D. thesis, Department of Economics, York University. Christensen, L.R., Jorgenson, D.W., Lau, L.J., 1971. Conjugate duality and the transcendental logarithmic production function. Econometrica 39, 255 – 256. Diewert, W.E., 1971. An application of the Shephard duality theorem: a generalized Leontief production function. Journal of Political Economy 79, 481 – 507. Gallant, A. Roland, 1981. On the bias in flexible functional forms and an essentially unbiased form: the Fourier flexible form. Journal of Econometrics 15, 211 – 245. Gallant, A. Roland, 1982. Unbiased determination of production technologies. Journal of Econometrics 20, 285 – 323. Gallant, A. Roland, 1984. The Fourier flexible form. American Journal of Agricultural Economics 66, 204 – 208. Geman, Stuart, 1981. Sieves for Nonparametric Estimation of Densities and Regressions, Reports in Pattern Analysis, No. 99. Division of Applied Mathematics, Brown University. Geman, S., Hwang, Chii-Ruey, 1982. Nonparametric maximum likelihood estimation by the method of sieves. The Annals of Statistics 10 (2), 401 – 414. Grenander, Ulf, 1981. Abstract Inference. Wiley, New York. Kingsley, E.H., 1951. Bernstein polynomials for functions of two variables of class C(k). Proceedings of the American Mathematical Society, 64 – 71. Terrell, Dek, 1995. Flexibility and regularity properties of the asymptotically ideal production model. Econometric Reviews 14, 1 – 17.
Semi-nonparametric estimation with Bernstein polynomials Pok Man Chaka,T, Neal Madrasb, Barry Smithc a
b
Department of Economics, Wilfrid Laurier University, Waterloo, Ontario, Canada N2L 3C5 Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada M3J 1P3 c Department of Economics, York University, Toronto, Ontario, Canada M3J 1P3 Received 9 March 2004; accepted 7 January 2005 Available online 24 August 2005
Abstract This paper proposes a semi-nonparametric model based upon Bernstein polynomials. We prove that the univariate Bernstein semi-nonparametric estimator is consistent for the true function and its first and second derivatives, and thus it is globally flexible and globally regular. D 2005 Elsevier B.V. All rights reserved. Keywords: Flexible functional forms; Bernstein polynomials JEL classification: C13; C14; C20; C51
1. Introduction Since the introduction of Cobb–Douglas function in 1928, economists have searched for flexible functional forms having more desirable properties. The functional forms developed in the 1970s are finite parameter models1 and can provide up to a second-order approximation of an arbitrary function at a specific point, and hence they are locally flexible in the approximation setting. In estimation, they often perform poorly. Gallant’s (1981, 1982, 1984) Fourier model, an infinite parameter model, provided a new direction in the search for flexible functional forms. Here Sobolev flexibility or global flexibility T Corresponding author. Tel.: +1 519 884 0710 x 2654; fax: +1 519 888 1015. E-mail addresses: [email protected] (P.M. Chak)8 [email protected] (N. Madras)8 [email protected] (B. Smith). 1 For example, see Christensen et al. (1971) and Diewert (1971). 0165-1765/$ - see front matter D 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.econlet.2005.01.025
154
P.M. Chak et al. / Economics Letters 89 (2005) 153–156
was also defined. In 1983, Barnett and Jonas (1983) suggested a second infinite parameter model based upon the Mu¨ntz–Szatz series. They called it the Asymptotically Ideal Model (AIM). However, Terrell (1995) pointed out that there is a trade-off between flexibility and regularity in the AIM model. In this paper, we describe the Bernstein functional form and show its usefulness in approximating and estimating functions which are continuous, non-negative, monotone non-decreasing and strictly concave. These shape restrictions are the regularity conditions for a cost function in its intensive form. In approximation, the Bernstein model is shown to be globally flexible. In estimation, we show that the sequence of our univariate Bernstein estimators and the sequences of the first and second derivatives of the estimators converge to the true function and its first and second derivatives in the L 2 metric space. This implies that the univariate Bernstein estimator and the first and second derivatives of the estimator are consistent for the true function and its first and second derivatives. It follows that our estimator is globally flexible. Moreover, linear constraints on the parameters are sufficient to guarantee that our estimator is monotone and concave for all sample sizes. Therefore, our univariate Bernstein estimator is globally flexible and globally regular.
2. Bernstein polynomials in approximation The Bernstein polynomial approximation of order n of a function f(x)is defined as: n v X f Pv;n ð xÞ; n v¼0 n v x ð1 xÞnv and xa½0; 1; naN. Bernstein (1912) used these polynomials to where Pv;n ð xÞ ¼ v provide a simpler proof of the approximation theorem of Weierstrass. The sequence (in terms of n 1 and n 2) of two-variable Bernstein polynomials first appeared in Kingsley (1951); it is defined as: n1 X n2 X v1 v2 f f ; Pv1 ; n1 ðx1 ÞPv2 ; n2 ðx2 Þ Bn1 ;n2 ðx1 ; x2 Þ ¼ n1 n2 v1 ¼0 v2 ¼0
Bnf ð xÞ ¼
where (x 1, x 2) a [0, 1] [0, 1] and n1 ; n2 aN. The most important property of bivariate Bernstein polynomials for our purpose is the convergence of the true function and its derivatives. Butzer (1953) and Kingsley (1951) show that, given a mild restriction on the growth of n 1 and n 2, the sequences of derivatives of the bivariate Bernstein polynomials converge at least pointwise to those of their respective true derivatives as n 1, n 2Yl. This means that two-dimensional Bernstein polynomials are shapepreserving functions for large n 1 and n 2, and hence global flexibility is guaranteed. If the true function has continuous derivatives on [0, 1] [0, 1], then we have uniform convergence.
3. Bernstein polynomials in estimation Consider the following univariate model: Yi ¼ f ðXi Þ þ ei ;
P.M. Chak et al. / Economics Letters 89 (2005) 153–156
155
where (X i , Yi ) are i.i.d. on [0, 1] [0, d¯ ] with joint probability measure P and E P [qi jX i ] = 0, and f is an arbitrary continuous, non-negative, monotone non-decreasing and strictly concave function with continuous third derivatives on (0, 1). We assume that Pr[X = 0] = 0. The regression of Y on X is EP ½Yi jXi ¼ xi ¼ f ðxi Þ; which is the unknown function. The data available for estimation are realizations ( y i of Yi and x i of X i ) yi ¼ f ðxi Þ þ ei : To estimate this model, we use Bernstein polynomials which are defined as follows: Bn ð x Þ ¼
n X
cv Pv;n ð xÞ;
v¼0
where the c v ’s are coefficients that are restricted in such a way that B n is non-negative, monotone nondecreasing, and strictly concave. The restrictions are: for non-negativity, c v z 0; for monotonicity, c v+1 c v z 0; for strict concavity, c v 2c v+1 + c v+2 b 0, v = 0, 1,. . ., n. We call it the Bernstein seminonparametric estimator. To carry out the estimation, we use the sieve method introduced by Grenander in 1981 as described in Grenander (1981), Geman (1981), and Geman and Hwang (1982). In this method, the best fitting restricted function is chosen from a subset of the parameter space, and this subset is allowed to grow with the sample size. This sequence of subsets is called a sieve. Let Y 0 be the set of all continuous, non-negative functions f from [0, 1] into [0, d¯ ] with the property that f is monotone nondecreasing, strictly concave with continuous derivatives on (0, 1). Also let Y 0[A] be the set of functions f in Y 0 that satisfy the third-derivative bounds |d3f/dx 3| V A. We define a stage T sieve to be the set of all Bernstein polynomials B n of degree n in Y 0[A], i.e., n o ½ A ½ A ST ¼ fT uBn jBn aY 0 ; where T is the sample size and n is a function of T, and nYl as TYl. The constant A can be finite or [A] o S T[A], 8T, and thus the sieve grows with the sample size T. For each T, the infinite. Note that ST1 optimal estimate ˆf T is selected from the Bernstein polynomials in S T[A], and is defined as follows: T n o 1 X yi fT ðxi Þ2 : fˆ T ¼ arg min ½ A fT aST T i¼1
Choosing ˆf T is equivalent to choosing a vector of (T + 1) optimal, restricted coefficients {cˆ 0, cˆ 1,. . ., cˆ T }. Because the constraints are all linear in the parameters, this optimization problem can be solved by quadratic programming methods. In the following theorem, we prove the L 2 and pointwise convergences of {fˆT }, {dfˆT /dx} and {d2ˆf T / dx 2} to f, df/dx and d2f/dx 2. Theorem. Let D b A b l. Assume that f a Y 0[D] and ˆf T a S T[A] . Also assume that the support of P is the whole interval [0, 1]. Then with probability one, {fˆT }, {dfˆT /dx} and {d 2ˆf T /dx 2 } converge to f, df/dx and d 2 f/dx 2 on [0, 1] in the L 2 (P) metric space and pointwise on [0, 1]. Proof. See Chak (2001), Theorems 3.2.7, 3.3.4 and 3.3.5.
156
P.M. Chak et al. / Economics Letters 89 (2005) 153–156
The assumption on the support of P can be weakened, leading to appropriate changes in the conclusions. The main concern is that the support of P should not have isolated points, because the derivatives cannot be estimated reliably at such points. Also, the L 2 convergence of {fˆT } to f can be achieved even when f a Y 0[l] and ˆf T a S T[l]. The result in the theorem can be extended to the multivariate case (Chak, 2001). 4. Conclusion We have examined the Bernstein semi-nonparametric model in approximation and in estimation. In approximation, among the many interesting properties of Bernstein polynomials is the global flexibility property which allows Bernstein polynomials to preserve the shape of the true function. In estimation, we obtain the consistency results for the one-dimensional Bernstein estimator and its first and second derivatives, and establish the global flexibility property for this estimator. Future work on the Bernstein estimator includes establishing consistency results for the multivariate case, rates of convergence and asymptotic normality. Acknowledgments The author would like to acknowledge the support by the Natural Sciences and Engineering Research Council of Canada. References Barnett, W.A., Jonas, A.B., 1983. The Mu¨ntz–Szatz demand system: an application of a globally well behaved series expansion. Economics Letters 11, 337 – 342. Bernstein, S., 1912. De´monstration du the´ore`me de Weierstrass, fondee´ sur le calcul des probabilite´s. Communications of Kharkov Mathematical Society 13 (2), 1 – 2. Butzer, Paul L., 1953. On two-dimensional Bernstein polynomials. Canadian Journal of Mathematics 5, 107 – 113. Chak, Pok Man, 2001. Approximation and consistent estimation of shape-restricted functions and their derivatives. Ph.D. thesis, Department of Economics, York University. Christensen, L.R., Jorgenson, D.W., Lau, L.J., 1971. Conjugate duality and the transcendental logarithmic production function. Econometrica 39, 255 – 256. Diewert, W.E., 1971. An application of the Shephard duality theorem: a generalized Leontief production function. Journal of Political Economy 79, 481 – 507. Gallant, A. Roland, 1981. On the bias in flexible functional forms and an essentially unbiased form: the Fourier flexible form. Journal of Econometrics 15, 211 – 245. Gallant, A. Roland, 1982. Unbiased determination of production technologies. Journal of Econometrics 20, 285 – 323. Gallant, A. Roland, 1984. The Fourier flexible form. American Journal of Agricultural Economics 66, 204 – 208. Geman, Stuart, 1981. Sieves for Nonparametric Estimation of Densities and Regressions, Reports in Pattern Analysis, No. 99. Division of Applied Mathematics, Brown University. Geman, S., Hwang, Chii-Ruey, 1982. Nonparametric maximum likelihood estimation by the method of sieves. The Annals of Statistics 10 (2), 401 – 414. Grenander, Ulf, 1981. Abstract Inference. Wiley, New York. Kingsley, E.H., 1951. Bernstein polynomials for functions of two variables of class C(k). Proceedings of the American Mathematical Society, 64 – 71. Terrell, Dek, 1995. Flexibility and regularity properties of the asymptotically ideal production model. Econometric Reviews 14, 1 – 17.