- Email: [email protected]
A note on “A globally convergent BFGS method with nonmonotone line search for non-convex minimization”
Applied Mathematics and Computation 219 (2012) 764–766
Contents lists available at SciVerse ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
A note on ‘‘A globally convergent BFGS method with nonmonotone line search for non-convex minimization’’ q Li Zhang ⇑, Xinlong Chen Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China
a r t i c l e
i n f o
Keywords: BFGS method Nonmonotone line search Global convergence
a b s t r a c t In this note, we point out that Assumptions 3.1, 3.2 in [Y. Xiao, H. Sun, Z. Wang, A globally convergent BFGS method with nonmonotone line search for non-convex minimization, J. Comput. Appl. Math. 230 (2009) 95–106] are not enough to guarantee the global convergence result Theorem 3.1 in [1]. We present a new assumption and show that the proposed method in [1] still converges globally under these three assumptions. Ó 2012 Elsevier Inc. All rights reserved.
1. Introduction In this note we consider the following unconstrained optimization problem:
min f ðxÞ; n
ð1:1Þ
x2R
where f : Rn ! R is continuously differentiable. In [1], Xiao et al. proposed a nonmonotone BFGS method for solving (1.1) and obtained some results as follows. Lemma 1.1 ([1], Lemma 3.2). Let the sequence fxk g be generated by Algorithm 2.1 in [1]. If kg k k P holds for all k with some constant > 0, then there are positive constants bj ; j ¼ 1; 2; 3, such that, for any k, the inequalities
kBi si k 6 b1 ksi k and b2 ksi k2 6 sTi Bi si 6 b3 ksi k2
ð1:2Þ
hold for at least a half of the indices i 2 f1; 2; . . . ; kg, where Bk is the iterative matrix and sk ¼ xkþ1 xk ¼ ak dk . Lemma 1.2 ([1], Lemma 3.4). If kg k k P holds for all k with some constant for all i 2 fij ð1:2Þ holdsg. that ai P a
> 0, then there exists a positive constant a such
Theorem 1.1 ([1], Theorem 3.1). Let Assumptions 3.1,3.2 in [1] hold. Then the sequence fxk g generated by Algorithm 2.1 in [1] converges globally in the sense that
lim inf k!1 kg k k ¼ 0: The authors in [1] obtained Theorem 1.1 under the following assumptions:
q This work was supported by the NSF foundation (10901026) of China and The Open Fund Project of Key Research Institute of Philosophies and Social Sciences in Hunan Universities. ⇑ Corresponding author. E-mail address: [email protected] (L. Zhang).
0096-3003/$ - see front matter Ó 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amc.2012.06.073
L. Zhang, X. Chen / Applied Mathematics and Computation 219 (2012) 764–766
765
Assumption 3.1 [1]. The level set L0 ¼ fxjf ðxÞ 6 f ðx0 Þg is bounded. Assumption 3.2 [1]. Function f is continuously differentiable on L0 , and there exists a constant L > 0 such that
8x; y 2 L0 ;
kgðxÞ gðyÞk 6 kx yk;
where gðxÞ is the gradient of f at x. In this paper, we first point out that Assumptions 3.1,3.2 are not enough to guarantee the global convergence result Theorem 1.1 by an example. Then we present a new assumption and give a simpler proof to show that the proposed method in [1] still converges globally under these three assumptions.
2. A new assumption and global convergence In this paper, we use the same notations as those of [1]. For simplicity, we first introduce some notations as follows:
f ðxhðkÞ Þ ¼ max f ðxkj Þ; 06j6M0
k M0 6 hðkÞ 6 k;
where M 0 is a given positive integer and
J ¼ fij ð1:2Þ holdsg; J ¼ frj
X
ðg Tr sr Þ < 1g;
r
where J is only a subset of f0; 1; 2; g; g r ¼ gðxr Þ and sr ¼ ar dr . In the proof of Theorem 3.1 in [1], the authors used the condition J # J which is the key in their proof. However, this is not proved in [1]. In fact it is not easy to prove this condition in theory. The main reason is that we can not control the behavior of the iterative matrix Bk . And hence we do not know when the matrix Bk generates a so-called good iteration which satisfies (1.2). Hence there is some possibility that J \ J may be an empty set, for example: Assume that
J ¼ f3mg1 m¼1 and
J ¼ f3m 2; 3m 1g1 m¼1 : Then for all k > 0, the number of J is 2=3 times k, which satisfies Lemma 1.1. It is clear that J \ J ¼ /, i.e., an empty set. To guarantee that the main results of [1] still hold, we give a new assumption as follows. Assumption A. G,K \ J is infinite, where K ¼ fhðkÞ 1g. Now we present the following global convergence result under this additional condition. Theorem 2.1. Let Assumptions 3.1,3.2 in [1] and Assumption A hold. Then the sequence fxk g generated by Algorithm 2.1 in [1] converges globally, that is,
lim inf k!1 kg k k ¼ 0: Proof. We prove this theorem by contradiction. Assume that the conclusion is not true, then there exists a positive constant e such that
kg k k P e;
8k P 0:
ð2:1Þ
From Lemma 3.3 in [1], we have that the sequence ff ðxhðkÞ Þg is monotonically nonincreasing and fxk g L0 . By the line search (2.3) in [1], we have
f ðxhðkþ1Þ Þ 6 f ðxhðkÞ Þ ¼ f ðxhðkÞ1 þ ahðkÞ1 dhðkÞ1 Þ 6 max f ðxhðkÞ1j Þ þ dahðkÞ1 g ThðkÞ1 dhðkÞ1 06j6M0
T hðkÞ1 g hðkÞ1 dhðkÞ1 :
¼ f ðxhðhðkÞ1Þ Þ þ da
Let k ! 1 in the above inequality, we have
ahðkÞ1 g ThðkÞ1 dhðkÞ1 ! 0; where we use the fact g ThðkÞ1 dhðkÞ1 < 0. This and Lemma 1.2 imply
lim g Tk dk ¼ 0:
k2G;k!1
ð2:2Þ
766
L. Zhang, X. Chen / Applied Mathematics and Computation 219 (2012) 764–766
Moreover, from Lemma 1.1 and sk ¼ ak dk , we have for all k 2 G, T
kBk dk k 6 b1 kdk k and b2 kdk k2 6 dk Bk dk 6 b3 kdk k2 ; which together with Bk dk þ g k ¼ 0 yields that
kg k k 6 b1 kdk k and b2 kdk k2 6 g Tk dk 6 b3 kdk k2 :
ð2:3Þ
By (2.2) and (2.3), we have that
limk2G;k!1 kg k k ¼ 0; which contradicts with (2.1). This completes the proof. h Reference [1] Y. Xiao, H. Sun, Z. Wang, A globally convergent BFGS method with nonmonotone line search for non-convex minimization, J. Comput. Appl. Math. 230 (2009) 95–106.
Contents lists available at SciVerse ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
A note on ‘‘A globally convergent BFGS method with nonmonotone line search for non-convex minimization’’ q Li Zhang ⇑, Xinlong Chen Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China
a r t i c l e
i n f o
Keywords: BFGS method Nonmonotone line search Global convergence
a b s t r a c t In this note, we point out that Assumptions 3.1, 3.2 in [Y. Xiao, H. Sun, Z. Wang, A globally convergent BFGS method with nonmonotone line search for non-convex minimization, J. Comput. Appl. Math. 230 (2009) 95–106] are not enough to guarantee the global convergence result Theorem 3.1 in [1]. We present a new assumption and show that the proposed method in [1] still converges globally under these three assumptions. Ó 2012 Elsevier Inc. All rights reserved.
1. Introduction In this note we consider the following unconstrained optimization problem:
min f ðxÞ; n
ð1:1Þ
x2R
where f : Rn ! R is continuously differentiable. In [1], Xiao et al. proposed a nonmonotone BFGS method for solving (1.1) and obtained some results as follows. Lemma 1.1 ([1], Lemma 3.2). Let the sequence fxk g be generated by Algorithm 2.1 in [1]. If kg k k P holds for all k with some constant > 0, then there are positive constants bj ; j ¼ 1; 2; 3, such that, for any k, the inequalities
kBi si k 6 b1 ksi k and b2 ksi k2 6 sTi Bi si 6 b3 ksi k2
ð1:2Þ
hold for at least a half of the indices i 2 f1; 2; . . . ; kg, where Bk is the iterative matrix and sk ¼ xkþ1 xk ¼ ak dk . Lemma 1.2 ([1], Lemma 3.4). If kg k k P holds for all k with some constant for all i 2 fij ð1:2Þ holdsg. that ai P a
> 0, then there exists a positive constant a such
Theorem 1.1 ([1], Theorem 3.1). Let Assumptions 3.1,3.2 in [1] hold. Then the sequence fxk g generated by Algorithm 2.1 in [1] converges globally in the sense that
lim inf k!1 kg k k ¼ 0: The authors in [1] obtained Theorem 1.1 under the following assumptions:
q This work was supported by the NSF foundation (10901026) of China and The Open Fund Project of Key Research Institute of Philosophies and Social Sciences in Hunan Universities. ⇑ Corresponding author. E-mail address: [email protected] (L. Zhang).
0096-3003/$ - see front matter Ó 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amc.2012.06.073
L. Zhang, X. Chen / Applied Mathematics and Computation 219 (2012) 764–766
765
Assumption 3.1 [1]. The level set L0 ¼ fxjf ðxÞ 6 f ðx0 Þg is bounded. Assumption 3.2 [1]. Function f is continuously differentiable on L0 , and there exists a constant L > 0 such that
8x; y 2 L0 ;
kgðxÞ gðyÞk 6 kx yk;
where gðxÞ is the gradient of f at x. In this paper, we first point out that Assumptions 3.1,3.2 are not enough to guarantee the global convergence result Theorem 1.1 by an example. Then we present a new assumption and give a simpler proof to show that the proposed method in [1] still converges globally under these three assumptions.
2. A new assumption and global convergence In this paper, we use the same notations as those of [1]. For simplicity, we first introduce some notations as follows:
f ðxhðkÞ Þ ¼ max f ðxkj Þ; 06j6M0
k M0 6 hðkÞ 6 k;
where M 0 is a given positive integer and
J ¼ fij ð1:2Þ holdsg; J ¼ frj
X
ðg Tr sr Þ < 1g;
r
where J is only a subset of f0; 1; 2; g; g r ¼ gðxr Þ and sr ¼ ar dr . In the proof of Theorem 3.1 in [1], the authors used the condition J # J which is the key in their proof. However, this is not proved in [1]. In fact it is not easy to prove this condition in theory. The main reason is that we can not control the behavior of the iterative matrix Bk . And hence we do not know when the matrix Bk generates a so-called good iteration which satisfies (1.2). Hence there is some possibility that J \ J may be an empty set, for example: Assume that
J ¼ f3mg1 m¼1 and
J ¼ f3m 2; 3m 1g1 m¼1 : Then for all k > 0, the number of J is 2=3 times k, which satisfies Lemma 1.1. It is clear that J \ J ¼ /, i.e., an empty set. To guarantee that the main results of [1] still hold, we give a new assumption as follows. Assumption A. G,K \ J is infinite, where K ¼ fhðkÞ 1g. Now we present the following global convergence result under this additional condition. Theorem 2.1. Let Assumptions 3.1,3.2 in [1] and Assumption A hold. Then the sequence fxk g generated by Algorithm 2.1 in [1] converges globally, that is,
lim inf k!1 kg k k ¼ 0: Proof. We prove this theorem by contradiction. Assume that the conclusion is not true, then there exists a positive constant e such that
kg k k P e;
8k P 0:
ð2:1Þ
From Lemma 3.3 in [1], we have that the sequence ff ðxhðkÞ Þg is monotonically nonincreasing and fxk g L0 . By the line search (2.3) in [1], we have
f ðxhðkþ1Þ Þ 6 f ðxhðkÞ Þ ¼ f ðxhðkÞ1 þ ahðkÞ1 dhðkÞ1 Þ 6 max f ðxhðkÞ1j Þ þ dahðkÞ1 g ThðkÞ1 dhðkÞ1 06j6M0
T hðkÞ1 g hðkÞ1 dhðkÞ1 :
¼ f ðxhðhðkÞ1Þ Þ þ da
Let k ! 1 in the above inequality, we have
ahðkÞ1 g ThðkÞ1 dhðkÞ1 ! 0; where we use the fact g ThðkÞ1 dhðkÞ1 < 0. This and Lemma 1.2 imply
lim g Tk dk ¼ 0:
k2G;k!1
ð2:2Þ
766
L. Zhang, X. Chen / Applied Mathematics and Computation 219 (2012) 764–766
Moreover, from Lemma 1.1 and sk ¼ ak dk , we have for all k 2 G, T
kBk dk k 6 b1 kdk k and b2 kdk k2 6 dk Bk dk 6 b3 kdk k2 ; which together with Bk dk þ g k ¼ 0 yields that
kg k k 6 b1 kdk k and b2 kdk k2 6 g Tk dk 6 b3 kdk k2 :
ð2:3Þ
By (2.2) and (2.3), we have that
limk2G;k!1 kg k k ¼ 0; which contradicts with (2.1). This completes the proof. h Reference [1] Y. Xiao, H. Sun, Z. Wang, A globally convergent BFGS method with nonmonotone line search for non-convex minimization, J. Comput. Appl. Math. 230 (2009) 95–106.