- Email: [email protected]
On the existence of the best discrete approximation in lp norm by reciprocals of real polynomials
Journal of Approximation Theory 156 (2009) 212–222 www.elsevier.com/locate/jat
On the existence of the best discrete approximation in l p norm by reciprocals of real polynomialsI Dragan Juki´c Department of Mathematics, University of Osijek, Trg Ljudevita Gaja 6, HR-31 000 Osijek, Croatia Received 19 June 2007; received in revised form 29 April 2008; accepted 9 May 2008 Available online 31 October 2008 Communicated by Borislav Bojanov
Abstract For the given data (wi , xi , yi ), i = 1, . . . , M, we consider the problem of existence of the best discrete approximation in l p norm (1 ≤ p < ∞) by reciprocals of real polynomials. For this problem, the existence of best approximations is not always guaranteed. In this paper, we give a condition on data which is necessary and sufficient for the existence of the best approximation in l p norm. This condition is theoretical in nature. We apply it to obtain several other existence theorems very useful in practice. Some illustrative examples are also included. c 2008 Elsevier Inc. All rights reserved.
Keywords: Reciprocals of polynomials; Data fitting; Discrete rational approximation; Existence problem
1. Introduction The subject of approximation by reciprocals of polynomials with real coefficients is a special field in rational approximation, and has attracted great interest. Generally speaking, the research on this topic is hard (see e.g. [4,7,13,16,19]). We denote by Πn the set of all polynomials with real coefficients of degree at most n, and by Πn−1 the set of all reciprocals of the polynomials from Πn , i.e. 1 Πn−1 := : g ∈ Πn , g 6≡ 0 . g I This work was supported by the Ministry of Science, Education and Sports, Republic of Croatia, through research grant 235-2352818-1034. E-mail address: [email protected].
c 2008 Elsevier Inc. All rights reserved. 0021-9045/$ - see front matter doi:10.1016/j.jat.2008.05.001
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
213
Suppose we are given the experimental or empirical set of M data (wi , xi , yi ),
i = 1, . . . , M,
M > n + 1,
(1)
where the xi ’s represent the distinct values of the independent variable, the yi ’s are the values of the measured function, and each wi > 0 is the weight associated to the data (xi , yi ). Let p, 1 ≤ p < ∞, be given. In this paper, we consider the problem of finding a best approximation 1/g ? from Πn−1 to given data (1) in the sense that p p M M X X 1 1 wi wi ? − yi = inf − yi . g (xi ) g(xi ) 1/g∈Πn−1 i=1 i=1 To be more precise, let every 1/g from Πn−1 be written as an explicit formula 1 1 1 ≡ = . g g(x; a) a0 + a1 x + · · · + an x n
(2)
The unknown vector a = (a0 , a1 , . . . , an ) has to be estimated by minimizing the following functional p M X 1 S p (a) = wi − yi . (3) n a0 + a1 x i + · · · + an x i i=1 Thus, in order to find the desired reciprocal polynomial 1/g ? from Πn−1 , we are required to solve the following weighted l p norm minimization problem: Does there exist a point a? ∈ D, such that S p (a? ) = inf S p (a), a∈D
(4)
where D := a ∈ Rn+1 : a0 + a1 xi + · · · + an xin 6= 0, i = 1, . . . , M ? The minimizing value a ? ∈ D is called the best l p estimate, if it exists, and the reciprocal 1/g(x; a? ) ∈ Πn−1 is called the best l p approximation to the data. Classically, on minimizing (3), p is set to 2, which gives the least squares (LS) approximations (see e.g. [2,3,15,17]). During the last few decades an increased interest in alternative p-values has become apparent (see e.g. [1,18]). In this paper, we are interested in the following natural questions about problem (4): (a) Does there exist a solution to the problem or not? (b) What are the conditions satisfied by a solution (necessary, sufficient, necessary and sufficient)? (c) Is a solution unique or not? The l p approximation problem (4) is a nonlinear problem which can only be solved in an iterative way. Very little has been known about this problem so far (e.g. [9,8,14,18]). Before iterative minimization, it is still necessary to ask whether the best l p estimate exists. Even in the case of nonlinear LS problems ( p = 2) it is still extremely difficult to answer this question (see [3,6,5,10–12]). The paper is organized as follows. In Section 2 we first present our main result (Theorem 1) which gives a condition on data which is necessary and sufficient for the existence of the best l p estimate. This theorem is theoretical in nature and it is used for proving the remaining results. For practical purposes, Theorem 2 is extremely important, which gives a very simple sufficient condition for the existence of the best l p estimate. Special attention is given to a discrete least squares approximation by reciprocals of linear functions (Theorem 3). Example 4 illustrates that
214
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
problem (4) may have or may not have a solution. In Example 5 it is shown that the best estimate need not be unique. To avoid nonnecessary technicalities at an early stage, all proofs are given in Section 3. 2. The existence theorems In order to simplify the algebra in further consideration from now on and throughout the whole paper we assume, as may be done without loss of generality, that the given data (1) are rearranged so that w1 |y1 | p ≥ w2 |y2 | p ≥ · · · ≥ w M |y M | p .
(5)
We also assume that M X
|yi |2 > 0.
i=1
Namely, it is easy to show that problem (4) has no solution if PM 2 i=1 |yi | = 0. Then lim S p (1/α, 0, . . . , 0) = lim
α→0
α→0
M X
PM
2 i=1 |yi |
= 0. To see this, assume
wi |α| p = 0,
i=1
and hence infa∈D S p (a) = 0. Furthermore, since S p (a) > 0 for all a ∈ D, problem (4) has no solution. The following notations will be used in the rest of the paper: E ?p :=
M X
wi |yi | p
(6)
i=n+1
I M := {i : 1 ≤ i ≤ M&yi 6= 0} In := {i : 1 ≤ i ≤ n&yi 6= 0} Y Ω (x) := (x − xk ) k∈In
1 p :=
M X i=n+1
wi
|yi | p−1 sign(yi ) Ω (xi )
Y x − xk , x j − xk n l j (x) := k∈I k6= j 1,
if |In | > 1
( j ∈ In ).
if |In | = 1
We begin with our main theorem, which is the key to proving all other results of this paper. Theorem 1 (Necessary and Sufficient Condition). Problem (4) has a solution if and only if there exists a point a0 ∈ D such that S p (a0 ) ≤ E ?p . Theorem 1 gives a necessary and sufficient condition on data which guarantee the existence of the best l p estimate. Unfortunately, it is very difficult to apply it in a specific situation. For practical purposes the following theorem is more important.
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
215
Theorem 2 (Sufficient Condition). (i) If |I M | ≤ n, then problem (4) has no solution. (ii) Suppose |I M | > n. If 1p =
M X i=n+1
wi
|yi | p−1 sign(yi ) 6= 0, Ω (xi )
(7)
then problem (4) has a solution. No mention is made in Theorem 2 of the case |I M | > n and 1 p = 0. As will be shown in the next section, the reason for this omission is that anything can happen; problem (4) may have or may not have a solution (see Theorem 3 and Example 4). In this case, one must look for other criteria to determine whether problem (4) has or does not have a solution. 2.1. Least squares approximation by Π1−1 Here we consider problem (4) when the norm is an l2 norm and where n = 1. It is easy to verify that 12 and E 2? can be rewritten as 12 =
M X
wi
i=2
yi , xi − x1
E 2? =
M X
wi yi2 .
i=2
Before formulating the next theorem, define a function D : R → R by the formula: !2 M M X X D(b) = 4Θ 2 (b) wi yi Θi (b) − w1 y12 wi Θi2 (b) i=1
i=1
where Θ(b) :=
M Y
(b + x j ),
Θi (b) :=
M Y
(b + x j ),
i = 1, . . . , M.
j=1 j6=i
j=1
Theorem 3. (a) If 12 6= 0, then the best l2 approximation from Π1−1 exists. PM PM (b) Suppose 12 = 0. Let y¯ := i=1 wi yi / i=1 wi . Then: (b1) If y¯ 6= 0 and S2 (1/ y¯ , 0) ≤ E 2? , then the best l2 approximation from Π1−1 exists. (b2) If y¯ 6= 0 and S2 (1/ y¯ , 0) > E 2? or y¯ = 0, then the best l2 approximation from Π1−1 exists if and only if there is b ∈ R \ {−x1 , . . . , −x M } such that D(b) ≥ 0. The next example illustrates the use of claim (b2) from the above theorem. Example 4. Let the data be as given in Table 1. By a routine calculation we obtain 12 =
3 X i=2
wi
yi = 0, xi − x1
y¯ =
5 , 3
182 S2 (1/ y¯ , 0) = S2 (3/5, 0) = > E 2? . 3
E 2? =
3 X i=2
wi yi2 = 20,
216
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
Table 1 wi xi yi
1 1 7
1 2 2
1 3 −4
1 1 5
1 2 2
1 3 −4
Table 2 wi xi yi
Fig. 1.
Fig. 2.
Since D(b) = −4(1+b)4 (2+b)2 (3+b)2 (122b2 +622b+801) < 0 for all b ∈ R\{−1, −2, −3} (see also Fig. 1.), according to Theorem 3(b2), the best l2 approximation from Π1−1 does not exist. For the data given in Table 2. we have 12 =
3 X i=2
wi
yi = 0, xi − x1
y¯ = 1,
E 2? =
3 X
wi yi2 = 20,
i=2
S2 (1/ y¯ , 0) = S2 (1, 0) = 42 > E 2? and D(b) = −12(1 + b)4 (2 + b)2 (3 + b)2 (22b2 + 114b + 147). Since D(−2.6) > 0 (see also Fig. 2.), according to Theorem 3(b2), the best l2 approximation from Π1−1 exists. It is well known that the best l p approximations from Πn−1 need not be unique (see e.g. [8]). Here we mention one more example.
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
217
Table 3 wi xi yi
1 −5 −3
1 −4 −2
1 4 2
1 5 3
Example 5. Consider the l2 norm approximation problem from Π1−1 for the data as given in Table 3. Since 12 = −13/9 6= 0, by Theorem 3(a) the best l2 approximation from Π1−1 exists. Let x 7→ 1/(a0? + a1? x) be the best l2 approximation from Π1−1 ; then by symmetry x 7→ 1/(−a0? + a1? x) must also be the best. 3. Proofs Proof of Theorem 1. Suppose first that problem (4) has a solution. Let a? be the best l p estimate. Then S p (a? ) ≤ S p (a),
∀a ∈ D.
Now consider the following class of reciprocals of polynomials: %(x; α) :=
Ω (x) +
α P j∈In
, α y j l j (x)
α 6= 0.
(8)
Note that %(xi , α) = yi ,
∀i ∈ In ,
lim %(xi , α) = 0,
(9)
∀i ∈ {1, . . . , M} \ In .
α→0
(10)
and S p (a? ) ≤
M X
wi |%(xi ; α) − yi | p .
i=1
Taking a limit as α → 0 in the last inequality we obtain S p (a? ) ≤ lim
α→0
M X
wi |%(xi ; α) − yi | p =
i=1
M X
wi |yi | p = E ?p .
i=n+1
Let us show the converse. Suppose that there is a point a0 ∈ D such that S p (a0 ) ≤ E ?p and let us show that problem (4) has a solution. Since functional S p is nonnegative, there exists S ?p := infa∈D S p (a). It should be shown that there exists a point a? ∈ D such that S p (a? ) = S ?p . First note that S ?p = inf S p (a) ≤ S p (a0 ) ≤ E ?p . a∈D
S ?p
If = S p (a0 ), to complete the proof it is enough to set a? = a0 . Hence, we can further assume that S ?p < S p (a0 ) ≤ E ?p .
(11)
218
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
Let ak = (a0,k , a1,k , . . . , an,k ) be a sequence in D, such that S ?p = lim S p (ak ) = lim k→∞
M X
k→∞
i=1
wi
a0,k
p 1 − y i . n + a1,k xi + · · · + an,k xi
(12)
In order to simplify the algebra in further consideration, denote: xi := (1, xi , xi2 , . . . , xin ),
i = 1, . . . , M
and αk := p
1 1 + kak
k2
,
β k := p
ak 1 + kak k2
,
k ∈ N.
With the above notations functional S p can be rewritten as p M α X k S p (ak ) = wi − y . i T β x k i i=1
(13)
(14)
Since k(αk , β k )k = 1, the sequence (αk , β k ) is bounded. We may assume that it is convergent, otherwise by the Bolzano–Weierstrass theorem, we take a convergent subsequence. Let (α ? , β ? ) := lim (αk , β k ). k→∞
Note that α ? ≥ 0, since αk > 0 for all k ∈ N. Now we are going to show that α ? > 0. We prove this by contradiction. Suppose α ? = 0. Let q be a polynomial defined by q(x) := β ? xT ,
x = (1, x, x 2 , . . . , x n ).
Since α ? = limk→∞ αk = 0, for every i ∈ {1, . . . , M} such that q(xi ) = β ? xiT = limk→∞ β k xiT 6= 0 we have p α k lim − y = |yi | p . i T k→∞ β k x i Furthermore, if q(xi ) = 0, then obviously p α k − y lim ≥ 0. i k→∞ β k xT i Taking into account these limits and letting k → ∞ in (14), we obtain p M α X k S ?p = lim S p (ak ) = lim wi − y i T k→∞ k→∞ β k xi i=1 p M M α X X k ≥ lim wi − y = wi |yi | p i β k xiT k→∞ i=1 i=1 q(xi )6=0
≥
M X i=n+1
wi |yi | p = E ?p .
q(xi )6=0
(15)
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
219
The last inequality in the above chain follows from assumption (5) and the fact that polynomial q has at most n distinct zeros. Inequality (15) contradicts assumption (11). This means that in this way (α ? = 0) functional S p cannot attain its infimum. Therefore, we have proved that α ? > 0. Furthermore, we claim that β ? ∈ D, i.e. β ? xiT 6= 0,
i = 1, . . . , M.
Namely, otherwise we would have β ? xiT0 = 0 for some i 0 , so because of (12) and (14), there would be p α k S ?p = lim S p (ak ) ≥ lim wi0 − y = ∞. i 0 β k xiT k→∞ k→∞ 0 Since α ? > 0 and β ? ∈ D, from (13) we obtain βk β? = ? ∈ D. k→∞ αk α
a? := lim ak = lim k→∞
Now due to continuity of functional S p , taking the limit k → ∞ in (14) we obtain p p M M α 1 X X k − y − y S ?p = lim S p (ak ) = lim wi = lim w i i i T 1 T k→∞ k→∞ k→∞ β k xi i=1 i=1 αk β k xi p M 1 X = wi ? T − yi = S p (a? ). a xi i=1 This completes the proof of Theorem 1.
Proof of Theorem 2. (i) Since |I M | ≤ n, it follows from (6) that E ?p = 0. By Theorem 1 it is enough to prove that S p (a) > 0 for all a = (a0 , a1 , . . . , an ) ∈ D. Let i 0 ∈ {1, . . . , M} be such that yi0 = 0. Since |I M | ≤ n and M > n + 1 by assumption (1), such an i 0 exists. By the definition of the set D, g(xi0 ; a) = a0 + a1 xi0 + · · · an xin0 6= 0, where g is defined by (2). Thus p p M X 1 1 S p (a) = − yi ≥ wi0 − yi0 > 0. wi g(xi ; a) g(xi0 ; a) i=1 (ii) Suppose that |I M | > n and 1 p 6= 0. According to Theorem 1, it is enough to find a point in D at which functional S p attains a value smaller than E ?p . For this purpose we define a function M X wi |%(xi ; α) − yi | p , α 6= 0 ϕ(α) = i=1 E ?p , α = 0, where %(x; α) is given by (8). Since |I M | > n by assumption, |In | = n. Now by using (9) it is easy to verify that p M X α P α wi − yi , α 6= 0. ϕ(α) = Ω (x ) + l (x ) i j i yj i=n+1 j∈I n
From this and by means of (10), we find that limα→0 ϕ(α) = E ?p = ϕ(0), which means that ϕ is continuous at α = 0. Thus, ϕ is continuous on the whole set R.
220
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
For every sufficiently small α the derivative of the function ϕ equals p−1 M X α α P α P sign ϕ 0 (α) = p wi − yi Ω (x ) + l (x ) Ω (x ) + i j i i yj i=n+1 j∈In j∈In P α y j l j (x i ) 1 j∈In P α , − × ! 2 Ω (xi ) + l (x ) j i yj P α j∈In Ω (xi ) + y j l j (x i )
α y j l j (x i )
− yi
j∈In
wherefrom by using L’Hospital’s rule, (10) and (7) we obtain M X ϕ(α) − ϕ(0) |yi | p−1 sign(yi ) = lim ϕ 0 (α) = − p wi 6= 0. α→0 α→0 α Ω (xi ) i=n+1
ϕ 0 (0) = lim
This means that ϕ is strictly monotonic on some interval (−δ, δ), δ > 0. Therefore, there is a point α0 ∈ (−δ, δ) \ {0}, such that ϕ(α0 ) < ϕ(0), i.e. p M X α0 P α0 wi − yi < E ?p . Ω (xi ) + y j l j (x i ) i=1 j∈I n
In other words, the last inequality tells us that there exists a point in D at which functional S p attains a value smaller than E ?p . Thus, Theorem 2 is proved. Proof of Theorem 3. Note that (a) and (b1) follow from Theorems 2(ii) and 1, respectively. To prove the assertion in (b2), let us first show that S2 (a0 , 0) > E 2? =
M X
wi yi2 ,
∀(a0 , 0) ∈ D.
(16)
i=2
PM 2 Indeed, since the quadratic function u → 7 i=1 wi (u − yi ) attains its minimum at point y¯ , we obtain 2 X M M X 1 S2 (a0 , 0) = wi − yi ≥ wi ( y¯ − yi )2 . a 0 i=1 i=1 PM In the first case ( y¯ 6= 0 and S2 (1/ y¯ , 0) > E 2? ), by the obvious equality i=1 wi ( y¯ − yi )2 = PM 2 S2 (1/ y¯ , 0) we obtain (16). Next, if y¯ = 0, then obviously i=1 wi ( y¯ − yi ) > E 2? , because by assumption y1 6= 0. Taking into account (16) and Theorem 1, the best l2 approximation from Π1−1 exists if and only if there exists a point (α0 , α1 ) ∈ D, α1 6= 0, such that 2 M X 1 S2 (α0 , α1 ) = wi − yi ≤ E 2? . α0 + α1 x i i=1 To complete the proof, it remains to show that such a point exists if and only if there is some b ∈ R \ {−x1 , . . . , −x M } such that D(b) ≥ 0.
221
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
If S2 (α0 , α1 ) ≤ E 2? for some (α0 , α1 ) ∈ D, α1 6= 0, put: α :=
1 , α1
b :=
α0 . α1
By the definition of the set D, b ∈ R \ {−x1 , . . . , −x M }. Furthermore, since Θ 2 (b) > 0 and 2 X M M X 1 − yi S2 (α0 , α1 ) ≤ E 2? ⇐⇒ wi ≤ wi yi2 α + α x 0 1 i i=1 i=2 M M 2 X X α ⇐⇒ wi − yi ≤ wi yi2 b + x i i=1 i=2 ⇐⇒
M X wi [α − yi (b + xi )]2
(b + xi )2
i=1
⇐⇒
M X
≤
M X
wi yi2
i=2
[α − yi (b + xi )]2 Θi2 (b) ≤ Θ 2 (b)
i=1
⇐⇒ α 2
M X
wi yi2
i=2
M X
wi Θi2 (b) − 2α
i=1
M X
wi (b + xi )Θi2 (b) + w1 y12 Θ 2 (b) ≤ 0,
i=1
(17) S2 (α0 , α1 ) ≤
E 2?
α 7→ α 2
implies that the determinant of the quadratic function
M X
wi Θi2 (b) − 2α
i=1
M X
wi (b + xi )Θi2 (b) + w1 y12 Θ 2 (b) =: s(α; b)
i=1
is nonnegative, i.e. D(b) = 4Θ 2 (b)
M X i=1
!2 wi yi Θi (b)
− w1 y12
M X
wi Θi2 (b) ≥ 0.
i=1
Conversely, suppose that D(b) ≥ 0 for some b ∈ R \ {−x1 , . . . , −x M }. Then there exists α ∈ R such that s(α; b) ≤ 0. Since w1 y12 Θ 2 (b) > 0, we conclude that α 6= 0. It is easy to check that (17) holds for (α0 , α1 ) := (b/α, 1/α) ∈ D. Thus S2 (α0 , α1 ) ≤ E 2? . Acknowledgments The author would like to thank the referees for the helpful suggestions. References [1] A. Atieg, G.A. Watson, Use of l p norms in fitting curves and surfaces to data, ANZIM J. 45 (E) (2004) C187–C200. [2] M. VanBarel, A. Bultheel, A parallel algorithm for discrete least squares rational approximation, Numer. Math. 63 (1992) 99–121. ˚ Bj¨orck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996. [3] A [4] E.W. Cheney, Introduction to Approximation Theory, AMS Chelsea Publishing, New York, 1982. [5] E.Z. Demidenko, Criteria for global minimum of sum of squares in nonlinear regression, Comput. Statist. Data Anal. 51 (2006) 1739–1753. [6] E.Z. Demidenko, Is this the least squares estimate? Biometrika 87 (2000) 437–452.
222
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
[7] R. DeVore, D. Leviatan, X. Yu, L p approximation by reciprocals of trigonometric and algebraic polynomials, Canad. Math. Bull. 33 (1990) 460–469. [8] C.B. Dunham, Nonlinear mean-square approximation on finite sets, SIAM J. Numer. Anal. 12 (1975) 105–110. [9] C.B. Dunham, Best discrete mean rational approximation, Aequationes Math. 11 (1974) 107–110. [10] K.P. Hadeler, D. Juki´c, K. Sabo, Least-squares problems for Michaelis–Menten kinetics, Math. Methods Appl. Sci. 30 (2007) 1231–1241. [11] D. Jukic, R. Scitovski, Least squares fitting Gaussian type curve, Appl. Math. Comput. 167 (2005) 286–298. [12] D. Juki´c, R. Scitovski, H. Sp¨ath, Partial linearization of one class of the nonlinear total least squares problem by using the inverse model function, Computing 62 (1999) 163–178. [13] D. Leviatan, A.L. Levin, E.B. Saff, On approximation in L p norm by reciprocals of polynomials, J. Approx. Theory 57 (1989) 322–331. [14] T. Pomentale, On discrete rational least squares approximation, Numer. Math. 12 (1968) 40–46. [15] G.J.S. Ross, Nonlinear Estimation, Springer-Verlag, New York, 1990. [16] D.S. Yu, S.P. Zhou, On approximation by rational functions with prescribed numerator degree in L p spaces, Acta Math. Hungar. 111 (3) (2006) 221–236. [17] G.A.F. Seber, C.J. Wild, Nonlinear Regression, John Wiley, New York, 1989. [18] G. Watson, Approximation in normed linear spaces, J. Comput. Appl. Math. 121 (2000) 1–36. [19] S.P. Zhou, Approximation by reciprocals of polynomials with positive coefficients, Southeast Asian Bull. Math. 28 (2004) 773–781.
On the existence of the best discrete approximation in l p norm by reciprocals of real polynomialsI Dragan Juki´c Department of Mathematics, University of Osijek, Trg Ljudevita Gaja 6, HR-31 000 Osijek, Croatia Received 19 June 2007; received in revised form 29 April 2008; accepted 9 May 2008 Available online 31 October 2008 Communicated by Borislav Bojanov
Abstract For the given data (wi , xi , yi ), i = 1, . . . , M, we consider the problem of existence of the best discrete approximation in l p norm (1 ≤ p < ∞) by reciprocals of real polynomials. For this problem, the existence of best approximations is not always guaranteed. In this paper, we give a condition on data which is necessary and sufficient for the existence of the best approximation in l p norm. This condition is theoretical in nature. We apply it to obtain several other existence theorems very useful in practice. Some illustrative examples are also included. c 2008 Elsevier Inc. All rights reserved.
Keywords: Reciprocals of polynomials; Data fitting; Discrete rational approximation; Existence problem
1. Introduction The subject of approximation by reciprocals of polynomials with real coefficients is a special field in rational approximation, and has attracted great interest. Generally speaking, the research on this topic is hard (see e.g. [4,7,13,16,19]). We denote by Πn the set of all polynomials with real coefficients of degree at most n, and by Πn−1 the set of all reciprocals of the polynomials from Πn , i.e. 1 Πn−1 := : g ∈ Πn , g 6≡ 0 . g I This work was supported by the Ministry of Science, Education and Sports, Republic of Croatia, through research grant 235-2352818-1034. E-mail address: [email protected].
c 2008 Elsevier Inc. All rights reserved. 0021-9045/$ - see front matter doi:10.1016/j.jat.2008.05.001
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
213
Suppose we are given the experimental or empirical set of M data (wi , xi , yi ),
i = 1, . . . , M,
M > n + 1,
(1)
where the xi ’s represent the distinct values of the independent variable, the yi ’s are the values of the measured function, and each wi > 0 is the weight associated to the data (xi , yi ). Let p, 1 ≤ p < ∞, be given. In this paper, we consider the problem of finding a best approximation 1/g ? from Πn−1 to given data (1) in the sense that p p M M X X 1 1 wi wi ? − yi = inf − yi . g (xi ) g(xi ) 1/g∈Πn−1 i=1 i=1 To be more precise, let every 1/g from Πn−1 be written as an explicit formula 1 1 1 ≡ = . g g(x; a) a0 + a1 x + · · · + an x n
(2)
The unknown vector a = (a0 , a1 , . . . , an ) has to be estimated by minimizing the following functional p M X 1 S p (a) = wi − yi . (3) n a0 + a1 x i + · · · + an x i i=1 Thus, in order to find the desired reciprocal polynomial 1/g ? from Πn−1 , we are required to solve the following weighted l p norm minimization problem: Does there exist a point a? ∈ D, such that S p (a? ) = inf S p (a), a∈D
(4)
where D := a ∈ Rn+1 : a0 + a1 xi + · · · + an xin 6= 0, i = 1, . . . , M ? The minimizing value a ? ∈ D is called the best l p estimate, if it exists, and the reciprocal 1/g(x; a? ) ∈ Πn−1 is called the best l p approximation to the data. Classically, on minimizing (3), p is set to 2, which gives the least squares (LS) approximations (see e.g. [2,3,15,17]). During the last few decades an increased interest in alternative p-values has become apparent (see e.g. [1,18]). In this paper, we are interested in the following natural questions about problem (4): (a) Does there exist a solution to the problem or not? (b) What are the conditions satisfied by a solution (necessary, sufficient, necessary and sufficient)? (c) Is a solution unique or not? The l p approximation problem (4) is a nonlinear problem which can only be solved in an iterative way. Very little has been known about this problem so far (e.g. [9,8,14,18]). Before iterative minimization, it is still necessary to ask whether the best l p estimate exists. Even in the case of nonlinear LS problems ( p = 2) it is still extremely difficult to answer this question (see [3,6,5,10–12]). The paper is organized as follows. In Section 2 we first present our main result (Theorem 1) which gives a condition on data which is necessary and sufficient for the existence of the best l p estimate. This theorem is theoretical in nature and it is used for proving the remaining results. For practical purposes, Theorem 2 is extremely important, which gives a very simple sufficient condition for the existence of the best l p estimate. Special attention is given to a discrete least squares approximation by reciprocals of linear functions (Theorem 3). Example 4 illustrates that
214
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
problem (4) may have or may not have a solution. In Example 5 it is shown that the best estimate need not be unique. To avoid nonnecessary technicalities at an early stage, all proofs are given in Section 3. 2. The existence theorems In order to simplify the algebra in further consideration from now on and throughout the whole paper we assume, as may be done without loss of generality, that the given data (1) are rearranged so that w1 |y1 | p ≥ w2 |y2 | p ≥ · · · ≥ w M |y M | p .
(5)
We also assume that M X
|yi |2 > 0.
i=1
Namely, it is easy to show that problem (4) has no solution if PM 2 i=1 |yi | = 0. Then lim S p (1/α, 0, . . . , 0) = lim
α→0
α→0
M X
PM
2 i=1 |yi |
= 0. To see this, assume
wi |α| p = 0,
i=1
and hence infa∈D S p (a) = 0. Furthermore, since S p (a) > 0 for all a ∈ D, problem (4) has no solution. The following notations will be used in the rest of the paper: E ?p :=
M X
wi |yi | p
(6)
i=n+1
I M := {i : 1 ≤ i ≤ M&yi 6= 0} In := {i : 1 ≤ i ≤ n&yi 6= 0} Y Ω (x) := (x − xk ) k∈In
1 p :=
M X i=n+1
wi
|yi | p−1 sign(yi ) Ω (xi )
Y x − xk , x j − xk n l j (x) := k∈I k6= j 1,
if |In | > 1
( j ∈ In ).
if |In | = 1
We begin with our main theorem, which is the key to proving all other results of this paper. Theorem 1 (Necessary and Sufficient Condition). Problem (4) has a solution if and only if there exists a point a0 ∈ D such that S p (a0 ) ≤ E ?p . Theorem 1 gives a necessary and sufficient condition on data which guarantee the existence of the best l p estimate. Unfortunately, it is very difficult to apply it in a specific situation. For practical purposes the following theorem is more important.
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
215
Theorem 2 (Sufficient Condition). (i) If |I M | ≤ n, then problem (4) has no solution. (ii) Suppose |I M | > n. If 1p =
M X i=n+1
wi
|yi | p−1 sign(yi ) 6= 0, Ω (xi )
(7)
then problem (4) has a solution. No mention is made in Theorem 2 of the case |I M | > n and 1 p = 0. As will be shown in the next section, the reason for this omission is that anything can happen; problem (4) may have or may not have a solution (see Theorem 3 and Example 4). In this case, one must look for other criteria to determine whether problem (4) has or does not have a solution. 2.1. Least squares approximation by Π1−1 Here we consider problem (4) when the norm is an l2 norm and where n = 1. It is easy to verify that 12 and E 2? can be rewritten as 12 =
M X
wi
i=2
yi , xi − x1
E 2? =
M X
wi yi2 .
i=2
Before formulating the next theorem, define a function D : R → R by the formula: !2 M M X X D(b) = 4Θ 2 (b) wi yi Θi (b) − w1 y12 wi Θi2 (b) i=1
i=1
where Θ(b) :=
M Y
(b + x j ),
Θi (b) :=
M Y
(b + x j ),
i = 1, . . . , M.
j=1 j6=i
j=1
Theorem 3. (a) If 12 6= 0, then the best l2 approximation from Π1−1 exists. PM PM (b) Suppose 12 = 0. Let y¯ := i=1 wi yi / i=1 wi . Then: (b1) If y¯ 6= 0 and S2 (1/ y¯ , 0) ≤ E 2? , then the best l2 approximation from Π1−1 exists. (b2) If y¯ 6= 0 and S2 (1/ y¯ , 0) > E 2? or y¯ = 0, then the best l2 approximation from Π1−1 exists if and only if there is b ∈ R \ {−x1 , . . . , −x M } such that D(b) ≥ 0. The next example illustrates the use of claim (b2) from the above theorem. Example 4. Let the data be as given in Table 1. By a routine calculation we obtain 12 =
3 X i=2
wi
yi = 0, xi − x1
y¯ =
5 , 3
182 S2 (1/ y¯ , 0) = S2 (3/5, 0) = > E 2? . 3
E 2? =
3 X i=2
wi yi2 = 20,
216
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
Table 1 wi xi yi
1 1 7
1 2 2
1 3 −4
1 1 5
1 2 2
1 3 −4
Table 2 wi xi yi
Fig. 1.
Fig. 2.
Since D(b) = −4(1+b)4 (2+b)2 (3+b)2 (122b2 +622b+801) < 0 for all b ∈ R\{−1, −2, −3} (see also Fig. 1.), according to Theorem 3(b2), the best l2 approximation from Π1−1 does not exist. For the data given in Table 2. we have 12 =
3 X i=2
wi
yi = 0, xi − x1
y¯ = 1,
E 2? =
3 X
wi yi2 = 20,
i=2
S2 (1/ y¯ , 0) = S2 (1, 0) = 42 > E 2? and D(b) = −12(1 + b)4 (2 + b)2 (3 + b)2 (22b2 + 114b + 147). Since D(−2.6) > 0 (see also Fig. 2.), according to Theorem 3(b2), the best l2 approximation from Π1−1 exists. It is well known that the best l p approximations from Πn−1 need not be unique (see e.g. [8]). Here we mention one more example.
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
217
Table 3 wi xi yi
1 −5 −3
1 −4 −2
1 4 2
1 5 3
Example 5. Consider the l2 norm approximation problem from Π1−1 for the data as given in Table 3. Since 12 = −13/9 6= 0, by Theorem 3(a) the best l2 approximation from Π1−1 exists. Let x 7→ 1/(a0? + a1? x) be the best l2 approximation from Π1−1 ; then by symmetry x 7→ 1/(−a0? + a1? x) must also be the best. 3. Proofs Proof of Theorem 1. Suppose first that problem (4) has a solution. Let a? be the best l p estimate. Then S p (a? ) ≤ S p (a),
∀a ∈ D.
Now consider the following class of reciprocals of polynomials: %(x; α) :=
Ω (x) +
α P j∈In
, α y j l j (x)
α 6= 0.
(8)
Note that %(xi , α) = yi ,
∀i ∈ In ,
lim %(xi , α) = 0,
(9)
∀i ∈ {1, . . . , M} \ In .
α→0
(10)
and S p (a? ) ≤
M X
wi |%(xi ; α) − yi | p .
i=1
Taking a limit as α → 0 in the last inequality we obtain S p (a? ) ≤ lim
α→0
M X
wi |%(xi ; α) − yi | p =
i=1
M X
wi |yi | p = E ?p .
i=n+1
Let us show the converse. Suppose that there is a point a0 ∈ D such that S p (a0 ) ≤ E ?p and let us show that problem (4) has a solution. Since functional S p is nonnegative, there exists S ?p := infa∈D S p (a). It should be shown that there exists a point a? ∈ D such that S p (a? ) = S ?p . First note that S ?p = inf S p (a) ≤ S p (a0 ) ≤ E ?p . a∈D
S ?p
If = S p (a0 ), to complete the proof it is enough to set a? = a0 . Hence, we can further assume that S ?p < S p (a0 ) ≤ E ?p .
(11)
218
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
Let ak = (a0,k , a1,k , . . . , an,k ) be a sequence in D, such that S ?p = lim S p (ak ) = lim k→∞
M X
k→∞
i=1
wi
a0,k
p 1 − y i . n + a1,k xi + · · · + an,k xi
(12)
In order to simplify the algebra in further consideration, denote: xi := (1, xi , xi2 , . . . , xin ),
i = 1, . . . , M
and αk := p
1 1 + kak
k2
,
β k := p
ak 1 + kak k2
,
k ∈ N.
With the above notations functional S p can be rewritten as p M α X k S p (ak ) = wi − y . i T β x k i i=1
(13)
(14)
Since k(αk , β k )k = 1, the sequence (αk , β k ) is bounded. We may assume that it is convergent, otherwise by the Bolzano–Weierstrass theorem, we take a convergent subsequence. Let (α ? , β ? ) := lim (αk , β k ). k→∞
Note that α ? ≥ 0, since αk > 0 for all k ∈ N. Now we are going to show that α ? > 0. We prove this by contradiction. Suppose α ? = 0. Let q be a polynomial defined by q(x) := β ? xT ,
x = (1, x, x 2 , . . . , x n ).
Since α ? = limk→∞ αk = 0, for every i ∈ {1, . . . , M} such that q(xi ) = β ? xiT = limk→∞ β k xiT 6= 0 we have p α k lim − y = |yi | p . i T k→∞ β k x i Furthermore, if q(xi ) = 0, then obviously p α k − y lim ≥ 0. i k→∞ β k xT i Taking into account these limits and letting k → ∞ in (14), we obtain p M α X k S ?p = lim S p (ak ) = lim wi − y i T k→∞ k→∞ β k xi i=1 p M M α X X k ≥ lim wi − y = wi |yi | p i β k xiT k→∞ i=1 i=1 q(xi )6=0
≥
M X i=n+1
wi |yi | p = E ?p .
q(xi )6=0
(15)
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
219
The last inequality in the above chain follows from assumption (5) and the fact that polynomial q has at most n distinct zeros. Inequality (15) contradicts assumption (11). This means that in this way (α ? = 0) functional S p cannot attain its infimum. Therefore, we have proved that α ? > 0. Furthermore, we claim that β ? ∈ D, i.e. β ? xiT 6= 0,
i = 1, . . . , M.
Namely, otherwise we would have β ? xiT0 = 0 for some i 0 , so because of (12) and (14), there would be p α k S ?p = lim S p (ak ) ≥ lim wi0 − y = ∞. i 0 β k xiT k→∞ k→∞ 0 Since α ? > 0 and β ? ∈ D, from (13) we obtain βk β? = ? ∈ D. k→∞ αk α
a? := lim ak = lim k→∞
Now due to continuity of functional S p , taking the limit k → ∞ in (14) we obtain p p M M α 1 X X k − y − y S ?p = lim S p (ak ) = lim wi = lim w i i i T 1 T k→∞ k→∞ k→∞ β k xi i=1 i=1 αk β k xi p M 1 X = wi ? T − yi = S p (a? ). a xi i=1 This completes the proof of Theorem 1.
Proof of Theorem 2. (i) Since |I M | ≤ n, it follows from (6) that E ?p = 0. By Theorem 1 it is enough to prove that S p (a) > 0 for all a = (a0 , a1 , . . . , an ) ∈ D. Let i 0 ∈ {1, . . . , M} be such that yi0 = 0. Since |I M | ≤ n and M > n + 1 by assumption (1), such an i 0 exists. By the definition of the set D, g(xi0 ; a) = a0 + a1 xi0 + · · · an xin0 6= 0, where g is defined by (2). Thus p p M X 1 1 S p (a) = − yi ≥ wi0 − yi0 > 0. wi g(xi ; a) g(xi0 ; a) i=1 (ii) Suppose that |I M | > n and 1 p 6= 0. According to Theorem 1, it is enough to find a point in D at which functional S p attains a value smaller than E ?p . For this purpose we define a function M X wi |%(xi ; α) − yi | p , α 6= 0 ϕ(α) = i=1 E ?p , α = 0, where %(x; α) is given by (8). Since |I M | > n by assumption, |In | = n. Now by using (9) it is easy to verify that p M X α P α wi − yi , α 6= 0. ϕ(α) = Ω (x ) + l (x ) i j i yj i=n+1 j∈I n
From this and by means of (10), we find that limα→0 ϕ(α) = E ?p = ϕ(0), which means that ϕ is continuous at α = 0. Thus, ϕ is continuous on the whole set R.
220
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
For every sufficiently small α the derivative of the function ϕ equals p−1 M X α α P α P sign ϕ 0 (α) = p wi − yi Ω (x ) + l (x ) Ω (x ) + i j i i yj i=n+1 j∈In j∈In P α y j l j (x i ) 1 j∈In P α , − × ! 2 Ω (xi ) + l (x ) j i yj P α j∈In Ω (xi ) + y j l j (x i )
α y j l j (x i )
− yi
j∈In
wherefrom by using L’Hospital’s rule, (10) and (7) we obtain M X ϕ(α) − ϕ(0) |yi | p−1 sign(yi ) = lim ϕ 0 (α) = − p wi 6= 0. α→0 α→0 α Ω (xi ) i=n+1
ϕ 0 (0) = lim
This means that ϕ is strictly monotonic on some interval (−δ, δ), δ > 0. Therefore, there is a point α0 ∈ (−δ, δ) \ {0}, such that ϕ(α0 ) < ϕ(0), i.e. p M X α0 P α0 wi − yi < E ?p . Ω (xi ) + y j l j (x i ) i=1 j∈I n
In other words, the last inequality tells us that there exists a point in D at which functional S p attains a value smaller than E ?p . Thus, Theorem 2 is proved. Proof of Theorem 3. Note that (a) and (b1) follow from Theorems 2(ii) and 1, respectively. To prove the assertion in (b2), let us first show that S2 (a0 , 0) > E 2? =
M X
wi yi2 ,
∀(a0 , 0) ∈ D.
(16)
i=2
PM 2 Indeed, since the quadratic function u → 7 i=1 wi (u − yi ) attains its minimum at point y¯ , we obtain 2 X M M X 1 S2 (a0 , 0) = wi − yi ≥ wi ( y¯ − yi )2 . a 0 i=1 i=1 PM In the first case ( y¯ 6= 0 and S2 (1/ y¯ , 0) > E 2? ), by the obvious equality i=1 wi ( y¯ − yi )2 = PM 2 S2 (1/ y¯ , 0) we obtain (16). Next, if y¯ = 0, then obviously i=1 wi ( y¯ − yi ) > E 2? , because by assumption y1 6= 0. Taking into account (16) and Theorem 1, the best l2 approximation from Π1−1 exists if and only if there exists a point (α0 , α1 ) ∈ D, α1 6= 0, such that 2 M X 1 S2 (α0 , α1 ) = wi − yi ≤ E 2? . α0 + α1 x i i=1 To complete the proof, it remains to show that such a point exists if and only if there is some b ∈ R \ {−x1 , . . . , −x M } such that D(b) ≥ 0.
221
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
If S2 (α0 , α1 ) ≤ E 2? for some (α0 , α1 ) ∈ D, α1 6= 0, put: α :=
1 , α1
b :=
α0 . α1
By the definition of the set D, b ∈ R \ {−x1 , . . . , −x M }. Furthermore, since Θ 2 (b) > 0 and 2 X M M X 1 − yi S2 (α0 , α1 ) ≤ E 2? ⇐⇒ wi ≤ wi yi2 α + α x 0 1 i i=1 i=2 M M 2 X X α ⇐⇒ wi − yi ≤ wi yi2 b + x i i=1 i=2 ⇐⇒
M X wi [α − yi (b + xi )]2
(b + xi )2
i=1
⇐⇒
M X
≤
M X
wi yi2
i=2
[α − yi (b + xi )]2 Θi2 (b) ≤ Θ 2 (b)
i=1
⇐⇒ α 2
M X
wi yi2
i=2
M X
wi Θi2 (b) − 2α
i=1
M X
wi (b + xi )Θi2 (b) + w1 y12 Θ 2 (b) ≤ 0,
i=1
(17) S2 (α0 , α1 ) ≤
E 2?
α 7→ α 2
implies that the determinant of the quadratic function
M X
wi Θi2 (b) − 2α
i=1
M X
wi (b + xi )Θi2 (b) + w1 y12 Θ 2 (b) =: s(α; b)
i=1
is nonnegative, i.e. D(b) = 4Θ 2 (b)
M X i=1
!2 wi yi Θi (b)
− w1 y12
M X
wi Θi2 (b) ≥ 0.
i=1
Conversely, suppose that D(b) ≥ 0 for some b ∈ R \ {−x1 , . . . , −x M }. Then there exists α ∈ R such that s(α; b) ≤ 0. Since w1 y12 Θ 2 (b) > 0, we conclude that α 6= 0. It is easy to check that (17) holds for (α0 , α1 ) := (b/α, 1/α) ∈ D. Thus S2 (α0 , α1 ) ≤ E 2? . Acknowledgments The author would like to thank the referees for the helpful suggestions. References [1] A. Atieg, G.A. Watson, Use of l p norms in fitting curves and surfaces to data, ANZIM J. 45 (E) (2004) C187–C200. [2] M. VanBarel, A. Bultheel, A parallel algorithm for discrete least squares rational approximation, Numer. Math. 63 (1992) 99–121. ˚ Bj¨orck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996. [3] A [4] E.W. Cheney, Introduction to Approximation Theory, AMS Chelsea Publishing, New York, 1982. [5] E.Z. Demidenko, Criteria for global minimum of sum of squares in nonlinear regression, Comput. Statist. Data Anal. 51 (2006) 1739–1753. [6] E.Z. Demidenko, Is this the least squares estimate? Biometrika 87 (2000) 437–452.
222
D. Juki´c / Journal of Approximation Theory 156 (2009) 212–222
[7] R. DeVore, D. Leviatan, X. Yu, L p approximation by reciprocals of trigonometric and algebraic polynomials, Canad. Math. Bull. 33 (1990) 460–469. [8] C.B. Dunham, Nonlinear mean-square approximation on finite sets, SIAM J. Numer. Anal. 12 (1975) 105–110. [9] C.B. Dunham, Best discrete mean rational approximation, Aequationes Math. 11 (1974) 107–110. [10] K.P. Hadeler, D. Juki´c, K. Sabo, Least-squares problems for Michaelis–Menten kinetics, Math. Methods Appl. Sci. 30 (2007) 1231–1241. [11] D. Jukic, R. Scitovski, Least squares fitting Gaussian type curve, Appl. Math. Comput. 167 (2005) 286–298. [12] D. Juki´c, R. Scitovski, H. Sp¨ath, Partial linearization of one class of the nonlinear total least squares problem by using the inverse model function, Computing 62 (1999) 163–178. [13] D. Leviatan, A.L. Levin, E.B. Saff, On approximation in L p norm by reciprocals of polynomials, J. Approx. Theory 57 (1989) 322–331. [14] T. Pomentale, On discrete rational least squares approximation, Numer. Math. 12 (1968) 40–46. [15] G.J.S. Ross, Nonlinear Estimation, Springer-Verlag, New York, 1990. [16] D.S. Yu, S.P. Zhou, On approximation by rational functions with prescribed numerator degree in L p spaces, Acta Math. Hungar. 111 (3) (2006) 221–236. [17] G.A.F. Seber, C.J. Wild, Nonlinear Regression, John Wiley, New York, 1989. [18] G. Watson, Approximation in normed linear spaces, J. Comput. Appl. Math. 121 (2000) 1–36. [19] S.P. Zhou, Approximation by reciprocals of polynomials with positive coefficients, Southeast Asian Bull. Math. 28 (2004) 773–781.