- Email: [email protected]
FR type methods for systems of large-scale nonlinear monotone equations
Applied Mathematics and Computation 269 (2015) 816–823
Contents lists available at ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
FR type methods for systems of large-scale nonlinear monotone equations Zoltan Papp, Sanja Rapajic´ ∗ Department of Mathematics and Informatics, University of Novi Sad, Trg Dositeja Obradovi´ca 4, 21000 Novi Sad, Serbia
a r t i c l e
i n f o
Keywords: Nonlinear monotone systems Hyperplane projection method Derivative-free line search Conjugate gradient directions
a b s t r a c t A large class of iterative methods for solving nonlinear monotone systems is developed in recent years. In this paper we propose some new FR type directions in the frame of algorithm which is a combination of conjugate gradient approach and hyperplane projection technique. Derivative-free, function-value-based line search combined with projection procedure is used for globalization strategy. Numerical performances of methods with different search directions are compared. © 2015 Elsevier Inc. All rights reserved.
1. Introduction We consider the system of nonlinear monotone equations of the form
F (x) = 0,
(1)
where function F: Rn → Rn is continuous and monotone, which means
(F (x) − F (y))T (x − y) ≥ 0, ∀x, y ∈ Rn . It is well known that under this assumption, the solution set of (1) is convex unless is empty, see [1]. Nonlinear monotone systems arise in various practical situations and applications in industry, technology and engineering, so a large class of iterative methods for solving these systems is developed in recent years. Conjugate gradient (CG) techniques are efficient for large-scale optimization problems and nonlinear systems, due to low memory. This is the reason why many iterative methods with CG directions are derived in the past. First of all, CG methods and their modifications for unconstrained optimization problems are presented in many papers [2–12]. Motivated by them, during the last decade, some authors adapted this CG approach to solving nonlinear monotone systems [13–19]. On the other hand, projection method is suitable for monotone equations, because it enables simply globalization. Solodov and Svaiter first introduced this idea. The method presented in [20] is truly globally convergent i.e. the whole sequence of iterates globally converges to the solution of system (1). Inspired by this and the fact that CG framework is at low cost, many algorithms for solving systems of monotone equations combines CG directions with projection technique. As opposed to classical globalization strategy where the line search is based on merit function, the derivative-free line search without merit function is proposed in some papers [13–15,17–22], with the aim of overcoming disadvantages of the classical approach.
∗
Corresponding author. Tel.: +381214852856; Ffax: +381216350458. ´ E-mail addresses: [email protected] (Z. Papp), [email protected] (S. Rapajic).
http://dx.doi.org/10.1016/j.amc.2015.08.002 0096-3003/© 2015 Elsevier Inc. All rights reserved.
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
817
The derivative-free line search, in a given direction, is used to construct a hyperplane which separates the current iterate from the solution set. Projecting the current iterate onto this hyperplane ensures global convergence and it is computationally negligible, so this approach is appropriate for solving monotone systems. Using derivative-free, function-value-based line search is promising, because it can also be applied to nonsmooth systems. So, the idea is to combine derivative-free line search with conjugate gradient direction and projection procedure, in order to construct a function-value-based method for solving largescale monotone systems. In this paper we propose some new Fletcher–Reeves (FR) type directions in the frame of algorithm given in [13] and compare the behavior of methods with various search directions. The paper is organized as follows. The algorithm and new FR type search directions are presented in Section 2. Convergence results are established in Section 3. Preliminary numerical results, obtained by comparing methods with different search directions, are given in Section 4. Some concluding remarks are made in the last section. 2. The algorithm One way of dealing with system (1) is considering the equivalent problem of minimizing the merit function 12 F (x)2 , so a large number of iterative methods have been developed to solve this unconstrained optimization problem. Unfortunately, these methods have some certain shortcomings. As it is mentioned before, in order to eliminate them, some algorithms without merit function have been presented recently, see [13–15,17–19,21,22]. The line search proposed in them is based only on the value of function F and it doesn’t use merit function. These algorithms are also based on projection technique which fully exploits the monotonicity property of the system (1). Projection procedure generates a sequence {zk } such that zk = xk + αk dk , where the steplength α k > 0 is obtained using some line search along the search direction dk such that
F (zk )T (xk − zk ) > 0.
(2)
On the other hand, the monotonicity of F implies that for every solution
F (zk )T (x∗ − zk ) = (F (zk ) − F (x∗ ))T (x∗ − zk ) ≤ 0.
x∗
of the system (1) we have (3)
It is clear from (2) and (3) that the hyperplane
Hk = {x ∈ Rn |F (zk )T (x − zk ) = 0}
(4) x∗
strictly separates the current iterate xk from the solution of system (1). So, the idea is to project the current iterate xk onto this hyperplane Hk and to compute the next iterate xk+1 in this way
xk+1 = xk −
F (zk )(xk − zk ) F (zk ). F (zk )2
(5)
The most computational complexity of method depends on defining the search direction and steplength, so it is very important to choose a proper line search. The derivative-free line search, proposed in [15] and given in [13], doesn’t use merit function and it is based only on the evaluation of function F and because of that it is suitable for large-scale systems and also for nonsmooth systems. The steplength α k > 0 is determined such that
−F (xk + αk dk )T dk ≥ σ αk F (xk + αk dk )dk 2 ,
(6)
where dk is a search direction satisfying the condition
F (xk )T dk ≤ −τ F (xk )2
(7)
for τ > 0. We consider the algorithm proposed in [13], but with different CG directions, and compare numerical performances of the algorithm with various search directions. This is derivative-free, function-value-based algorithm which combines projection procedure and CG techniques. Projection procedure ensures simply globalization and thanks to low memory requirement and simplicity of CG framework, this algorithm is appropriate for solving large-scale monotone systems. In this section we define three new FR type directions which satisfy condition (7). Now, we can state the main algorithm as follows. ALGORITHM 1. Input: Choose x0 ∈ Rn , constants kmax > 0, σ > 0, s > 0, ε > 0, ρ ∈ (0, 1). Begin Set k := 0, F0 := F(x0 ). While (Fk ≥ ε and k ≤ kmax ) Determine the direction dk which satisfies (7); α = s; Compute F (xk + α dk ); While ( − F (xk + α dk )T dk < σ αF (xk + α dk )dk 2 ) α := ρα ;
818
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
compute F (xk + α dk ); EndWhile αk = α ; zk = xk + αk dk ; If F(zk ) ≤ ε then xk+1 = zk ; STOP;
F (zk )T (xk −zk ) F (zk ); F (zk )2
Else compute xk+1 = xk − EndIf Fk+1 = F (xk+1 ); k := k + 1; EndWhile End
The line search condition in Algorithm 1 doesn’t use a merit function and derivatives, so it is derivative-free, function-valuebased line search. Since the direction dk satisfies (7), it is easy to see that this line search necessarily holds for sufficiently small steplength α k > 0, which can be obtained after a finite number of reductions. This means that the algorithm is well defined. Defining suitable search direction and using inexpensive line search can decrease the computational cost of method. As we mentioned before, CG directions are suitable for solving large-scale unconstrained optimization problems and large-scale systems, because of low storage requirement. A comprehensive overview of nonlinear CG methods for unconstrained minimization is given in [4] and several modifications of CG methods have been developed recently. Modified two-term and three-term CG directions are proposed in [10–12,23]. Motivated by them, many authors apply that idea to solving nonlinear monotone systems, see [13–17]. Inspired by the directions proposed in [11] and [13], which are used in methods for solving unconstrained optimization problems and monotone equations, we define several new modified three-term FR type directions for nonlinear monotone systems, depending on the way of determining θ k . From now, the notation Fk = F (xk ) will be used. The search direction is defined by
dk =
−Fk ,
if k = 0,
−Fk + βkF R wk−1 − θk Fk ,
if k > 0,
(8)
where
βkF R =
Fk 2 , Fk−1 2
(9)
wk = zk − xk = αk dk and a coefficient θ k is determined in three different ways, such that the condition (7) is satisfied. M3TFR1 direction: From (8) and (9) we have
FkT dk = −Fk 2 +
Fk 2 T F w − θk Fk 2 Fk−1 2 k k−1
(10)
for k ∈ N. Choosing
θk =
FkT wk−1
Fk−1 2
,
(11)
there follows FkT dk = −Fk 2 , so (7) holds for τ = 1. The first modified three-term FR type direction is called M3TFR1 direction and it is defined by (8), (9) and (11). Algorithm 1 with M3TFR1 direction will be named M3TFR1 method. M3TFR2 direction: In a similar way as in [13], we have
FkT dk =
−Fk 2 Fk−1 4 + Fk 2 Fk−1 2 FkT wk−1 − θk Fk 2 Fk−1 4
Fk−1 4
for k ∈ N. Setting u = implies
FkT dk
√1 Fk−1 2 Fk , 2
√
v = 2Fk 2 wk−1 and using the inequality uT v ≤ 12 (u2 + v2 ), the previous relation (12)
Fk−1 4 Fk 2 + 2Fk 4 wk−1 2 ≤ − θk Fk 2 Fk−1 4 1 Fk 4 = −Fk 2 + Fk 2 + w 2 − θk Fk 2 4 Fk−1 4 k−1 3 Fk 4 = − Fk 2 + w 2 − θk Fk 2 . 4 Fk−1 4 k−1 −Fk 2 Fk−1 4 +
1 2
(12)
1 2
(13)
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
819
Defining
θk =
Fk 2 wk−1 2 , Fk−1 4
(14)
we get FkT dk ≤ − 34 Fk 2 , so the condition (7) holds for τ = 3/4. The direction dk defined by (8), (9) and (14) is M3TFR2 direction, while Algorithm 1 with this direction is named M3TFR2 method. M3TFR3 direction: The coefficient θ k is determined in such a way that it consists of two parts. The first one is (11), while the second one is properly chosen such that (7) is satisfied. So,
θk =
FkT wk−1
Fk−1
2
+
Fk 2 . Fk−1 4
(15) F 4
Then, it is not difficult to verify that FkT dk ≤ −Fk 2 − F k 4 ≤ −Fk 2 , for k ∈ N, which means that the relation (7) holds for k−1 τ = 1. The direction M3TFR3 is defined by (8), (9) and (15) and Algorithm 1 with this direction is M3TFR3 method. Lemma 1. The directions M3TFR1, M3TFR2 and M3TFR3 satisfy condition (7) for every k ∈ N ∪ {0}. Proof. Since F0T d0 = −F0 2 for every of three given directions, for k = 0 the condition (7) trivially holds with τ = 1. For k ∈ N, we have just shown that both directions M3TFR1 and M3TFR3 satisfy (7) with τ = 1, while the direction M3TFR2 satisfies (7) with τ = 3/4. 3. Convergence analysis The global convergence of Algorithm 1 with new modified three-term directions M3TFR1, M3TFR2 and M3TFR3 will be proved under the following assumptions: A1 the function F(x) is monotone on Rn , A2 the solution set of system (1) is nonempty, A3 the function F(x) is Lipschitz continuous on Rn . The following lemma is originally from Solodov and Svaiter [20] and it also holds for Algorithm 1 with any of three new directions M3TFR1, M3TFR2 or M3TFR3, so the proof is omitted. This lemma guaranties that {xk − x∗ } is a decreasing and convergent sequence and in particular, the sequence {xk } is bounded. Lemma 2 ([20]). Suppose that assumptions A1-A3 are satisfied and the sequence {xk } is generated by Algorithm 1 with M3TFR1, M3TFR2 or M3TFR3 direction. For any solution x∗ of system (1) there holds
xk+1 − x∗ 2 ≤ xk − x∗ 2 − xk+1 − xk 2
(16)
and the sequence {xk } is bounded. Furthermore, either {xk } is finite and the last iteration is a solution of (1), or {xk } is infinite and ∞
xk+1 − xk 2 < ∞,
k=0
which means
lim xk+1 − xk = 0.
(17)
k→∞
The next lemma states that the sequence {Fk } is bounded. Lemma 3. Suppose that the assumptions A1–A3 are satisfied and the sequence {xk } is generated by Algorithm 1 with M3TFR1, M3TFR2 or M3TFR3 direction. Then the sequence {Fk } is bounded, i.e. there exists a constant κ > 0 such that Fk ≤ κ for every k ∈ N ∪ {0}. Proof. Lemma 2 implies that xk − x∗ ≤ x0 − x∗ . From that fact and the assumption A3 there follows
F (xk ) = F (xk ) − F (x∗ ) ≤ Lxk − x∗ ≤ Lx0 − x∗ . The statement of lemma holds taking κ = Lx0 − x∗ .
The following theorem guaranties that the directions M3TFR1, M3TFR2 and M3TFR3 are bounded. Theorem 1. Suppose that the assumptions A1-A3 are satisfied and the sequence of directions {dk } is generated by Algorithm 1 with M3TFR1, M3TFR2 or M3TFR3 direction. Let ε 0 > 0 is a constant such that
Fk ≥ ε0
(18)
holds for every k ∈ N ∪ {0}. Then the directions {dk } are bounded, i.e. there exists a constant M > 0 such that
d k ≤ M holds for every k ∈ N ∪ {0}.
(19)
820
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
Proof. The definition of zk , relation (5) and line search condition (6) imply
|F (zk )T (xk − zk )| |αk F (zk )T dk | = F (zk ) F (zk ) αk2 σ F (zk )dk 2 ≥ = σ αk2 dk 2 ≥ 0. F (zk )
xk+1 − xk =
(20)
From (17) and (20) there follows
lim
k→∞
αk dk = 0.
(21)
On the other hand using Cauchy–Schwartz-inequality we have
xk+1 − xk =
|αk F (zk )T dk | αk F (zk )dk ≤ = αk dk . F (zk ) F (zk )
(22)
Now, we are going to prove that the directions M3TFR1, M3TFR2 and M3TFR3 are bounded. M3TFR1 direction: The definition of M3TFR1 direction given by (8), (9) and (11), Lemma 3 and (18) imply
FkT wk−1 Fk 2 w − F k−1 Fk−1 2 Fk−1 2 k Fk 2 Fk 2 ≤ Fk + αk−1 dk−1 + α d 2 Fk−1 Fk−1 2 k−1 k−1 Fk 2 = Fk + 2 α d Fk−1 2 k−1 k−1 κ2 ≤ κ + 2 2 αk−1 dk−1 ε0
dk = − Fk +
(23)
for all k ∈ N. From (21) follows that for every ε 1 > 0 there exists k0 such that αk−1 dk−1 < ε1 for every k > k0 . Choosing ε1 = ε02 and M = max{d0 , d1 , . . . , dk0 , M1 }, where M1 = κ(1 + 2κ), it holds dk ≤ M for every k ∈ N. M3TFR2 direction: Analogously, the definition of M3TFR2 direction given by (8), (9) and (14), Lemma 3 and (18) imply
Fk 2 F 2 wk−1 2 wk−1 − k Fk 2 Fk−1 Fk−1 4 Fk 2 Fk 3 ≤ Fk + α d + (α d )2 Fk−1 2 k−1 k−1 Fk−1 4 k−1 k−1 κ2 κ3 ≤ κ + 2 αk−1 dk−1 + 4 (αk−1 dk−1 )2 ε0 ε0
dk = − Fk +
(24)
for every k ∈ N. The limit (21) implies that for every ε 1 > 0 there exists an index k0 such that αk−1 dk−1 < ε1 for all k > k0 . If we choose ε1 = ε02 and M = max{d0 , d1 , . . . , dk0 , M1 }, where M1 = κ(1 + κ + κ 2 ), then the direction M3TFR2 is bounded i.e. dk ≤ M for every k ∈ N. M3TFR3 direction: From the definition of M3TFR3 direction given by (8), (9) and (15), Lemma 3 and (18), it follows
dk = − Fk +
T Fk wk−1 Fk 2 Fk 2 w − + F Fk−1 2 k−1 Fk−1 2 Fk−1 4 k
Fk 2 Fk 3 α d + Fk−1 2 k−1 k−1 Fk−1 4 κ2 κ3 ≤ κ + 2 2 αk−1 dk−1 + 4 ε0 ε0 ≤ Fk + 2
(25)
for every k ∈ N. Choosing ε1 = ε02 , M = max{d0 , d1 , . . . , dk0 , M1 } and M1 = κ(1 + 2κ + κ 2 /ε04 ) there follows that dk ≤ M, for all k ∈ N. As it is mentioned before, it is not difficult to prove that the line search procedure, along the direction dk which is bounded and satisfies (7), in Algorithm 1 terminates for sufficiently small step-length α k > 0 after a finite number of reductions. This means that the line search is well defined. Lemma 4. [13] Suppose that all conditions of Theorem 1 are satisfied. Then the line search procedure in Algorithm 1 is well defined. Since we have proved by Theorem 1 and Lemma 1 that M3TFR1, M3TFR2 and M3TFR3 directions are bounded and satisfy (7), there follows by the above lemma that the line search procedure is well defined, which also implies that M3TFR1, M3TFR2 and M3TFR3 methods are well defined.
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
821
1 0.9 0.8 0.7
ρ(t)
0.6 0.5 0.4 0.3 M3TFR1 M3TFR2 M3TFR3 DFPB1 DFPB2
0.2 0.1 0
1
2
3
4
5
t Fig. 1. Performance profiles for the number of iterations.
The main theorem of global convergence of Algorithm 1, with bounded sequence of directions {dk } satisfying (7), is established in [13]. This theorem also holds for M3TFR1, M3TFR2 and M3TFR3 methods, because M3TFR1, M3TFR2 and M3TFR3 directions are bounded and satisfy condition (7), so the proof of the theorem is omitted here. Theorem 2. [13] Suppose that assumptions A1–A3 are satisfied and the sequence {xk } is generated by M3TFR1, M3TFR2 or M3TFR3 method. Then
lim inf Fk = 0.
(26)
k→∞
From the continuity of F, the boundedness of {xk } and the above theorem, there follows that the sequence {xk }, generated by any of three methods: M3TFR1, M3TFR2 or M3TFR3, has the accumulation point x∗ such that F (x∗ ) = 0. On the other hand, the sequence {xk − x∗ } is convergent by Lemma 2, which means that the whole sequence {xk } globally converges to the solution x∗ of the system (1). 4. Numerical results In this section we present some numerical results obtained by applying five methods on the following set of test problems, which can be found in [1,14,17,19]. We tested Problems 1–3 from [19] with dimensions 1000, 20, 000 and 50, 000, Problem 1 from [14] with dimensions 1000, 20, 000 and 50, 000, Problem 2 from [14] with dimensions 1000 and 5000, Problems 2 from [17] with dimension 1000, Problem 3 from [17] with dimensions 1000 and 3000 and Problem 4 from [17] with dimensions 1000, 20, 000 and 50, 000. We also tested the monotone, nonlinear system with dimension 20, 164, obtained by discretization of Dirichlet problem, which is given in [1]. Numerical performances of three new methods proposed in this paper are compared with two methods given in [13]. More precisely, we tested 5 methods: M3TFR1, M3TFR2, M3TFR3, DFPB1 and DFPB2, where DFPB1 and DFPB2 are presented in [13]. All algorithms are implemented in Matlab R2011b environment on a 2.13 GHz Intel dual-core processor computer with 4 GB of RAM. The stopping criteria in all experiments is the same as in [13]
Fk ≤ ε or F (zk ) ≤ ε or k > kmax , where ε = 10−4 and maximum number of iteration is kmax = 500, 000. In a similar way as in [13,15,17], the initial trial steplength sk is determined by the approximation of |
FkT dk
dT Jk dk k
|, where Jk = J(xk ) is the Jacobian matrix of F(xk ). To avoid calculating derivatives,
the finite differences are used to approximate the Jacobian, so
T F ( x ) d k k , sk = T ((F (xk + tdk ) − F (xk )) dk )/t
(27)
where t = 10−8 . We also used the same parameters σ = 0.3 and ρ = 0.7 as in [13]. All methods are initialized with sev(1) (2) (3) (4) (5) (6) (7) eral starting points: x0 = 10 · 1n×1 , x0 = −10 · 1n×1 , x0 = 1n×1 , x0 = −1n×1 , x0 = 0.1 · 1n×1 , x0 = [1, 12 , 13 , . . . , 1n ]T , x0 = [ 1n , 2n , . . . ,
(8) n−1 T n , 1] , x0
= [1 − 1n , 1 − 2n , . . . , 1 −
n−1 T n , 0] ,
where n is a dimension of the system.
822
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
1 0.9 0.8 0.7
ρ(t)
0.6 0.5 0.4 0.3 M3TFR1 M3TFR2 M3TFR3 DFPB1 DFPB2
0.2 0.1 0
1
2
3
4
5
t Fig. 2. Performance profiles for the number of function evaluations.
1 0.9 0.8 0.7
ρ(t)
0.6 0.5 0.4 0.3 M3TFR1 M3TFR2 M3TFR3 DFPB1 DFPB2
0.2 0.1 0
1
2
3
4
5
t Fig. 3. Performance profiles for the CPU time.
In order to make detailed comparison of the efficiency and robustness of all methods, we have used performance profile proposed in [24], which is useful methodology for standardizing the comparison of algorithms. We considered the number of iterations, the number of function evaluations and CPU time as a measure of performance profile. Fig. 1 reveals that M3TFR2 method is the most efficient in the sense of number of iterations, since it solves 40% of problems with the smallest number of iterations, while M3TFR1 method solves 37% of problems. The most robust algorithms are M3TFR3 and DFPB1, because both of their cumulative distribution functions ρ (t) reach 1 for minimal t. These two robust methods are followed by M3TFR2 method. It is clear from Fig. 2 that the most efficient method in the sense of number of function evaluations is M3TFR2 method, with 44% problems solved with the smallest number of function evaluations, while the most robust is DFPB1 method, which is also followed by M3TFR2 method. Considering the CPU time, Fig. 3 shows that the best method is M3TFR2, because it is the most efficient and the most robust. It is followed by M3TFR1 and M3TFR3 methods in the sense of efficiency and by DFPB1 and DFPB2 in the sense of robustness.
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
823
Based on Fig. 1–3, it can be concluded that M3TFR2 method performs better than other methods. Preliminary numerical results pointed out that employing new modified three-term FR type directions is promising, because new methods are competitive with methods presented in [13]. Numerical experiments report that new methods are suitable for large-scale monotone systems, especially M3TFR2 method, which is computationally very efficient. 5. Conclusions This paper presents several new modified three-term FR type directions in the frame of algorithm which combines conjugate gradient techniques and projection method. The algorithm belongs to the class of derivative-free, function-value-based methods, since it does not use merit function and derivatives. The low memory storage and simplicity of the CG approach makes the algorithm appropriate for solving large-scale monotone systems, while the projection procedure is suitable for monotone equations, because it enables simply globalization. Introducing new FR type search directions, we propose three new methods for solving large-scale monotone systems. The sequence {xk } generated by any of these new methods globally converges to the solution of the monotone system (1), without any regularity and differentiability assumption. Numerical experiments indicate that new methods based on modified three-term FR type directions are efficient, robust and comparable with other methods suitable for solving large-scale monotone systems. Moreover, these methods can be successfully applied to nonsmooth monotone systems. Acknowledgments Research supported by Serbian Ministry of Education, Science and Technological Development, grant no. 174030. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
J.M. Ortega, W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970. N. Andrei, Another hybrid conjugate gradient algorithm for unconstrained optimization, Numer. Algor. 47 (2008) 143–156. W. Cheng, Q. Liu, Sufficient descent nonlinear conjugate gradient methods with conjugacy condition, Numer. Algor. 53 (2010) 113–131. W.W. Hager, H. Zhang, A survey of nonlinear conjugate gradient methods, Pac. J. Optim. 2 (2006) 35–58. E. Polak, G. Ribiere, Note sur la convergence de directions conjugees, Rev. Francaise Imformat Recherche Opertionelle 16 (1969) 35–43. B.T. Polyak, The conjugate gradient method in extreme problems, USSR Comp. Math. Phys. 9 (1969) 94–112. M. Raydan, On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numer. Anal. 13 (1993) 321–326. M. Raydan, The Barzilai and Borwein gradient method for the large-scale unconstrained minimization problem, SIAM J. Optim. 7 (1997) 26–33. G. Yu, L. Guan, W. Chen, Spectral conjugate gradient method with sufficient descent property for large-scale unconstrained optimization, Optim. Methods Softw. 23 (2) (2008) 275–293. L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiere–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640. L. Zhang, W.J. Zhou, D.H. Li, Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search, Numer. Math. 104 (2006) 561–572. L. Zhang, W.J. Zou, D.H. Li, Some descent three-term conjugate gradient method and their global convergence, Optim. Methods Softw. 22 (4) (2007) 697–711. M. Ahookhosh, K. Amini, S. Bahrami, Two derivative-free projection approaches for systems of large-scale nonlinear monotone equations, Numer. Algor. 64 (4) (2013) 21–42. W. Cheng, A PRP type method for systems of monotone equations, Math. Comput. Model. 50 (2009) 15–20. Q. Li, D.H. Li, A class of derivative-free methods for large-scale nonlinear monotone equations, IMA J. Numer. Anal. 31 (2011) 1625–1635. D.H. Li, X.L. Wang, A modified Fletcher–Reeves-type derivative-free method for symmetric nonlinear equations, Numer. Algebra Control Optim. 1 (1) (2011) 71–82. Q.R. Yan, X.Z. Peng, D.H. Li, A globally convergent derivative-free method for solving large-scale nonlinear monotone equations, J. Comput. Appl. Math. 234 (2010) 649–657. L. Zhang, W.J. Zhou, Spectral gradient projection method for solving nonlinear monotone equations, J. Comput. Appl. Math. 196 (2006) 478–484. W.J. Zhou, D.H. Li, A globally convergent BFGS method for nonlinear monotone equations without any merit functions, Math. Comput. 77 (2008) 2231–2240. M.V. Solodov, B.F. Svaiter, A globally convergent inexact Newton method for systems of monotone equations., in: M. Fukushima, L. Qi (Eds.), Reformulation: Nonsmooth, Piecewise smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, 1998, pp. 355–369. W.J. Zhou, D.H. Li, Limited memory BFGS method for nonlinear monotone equations, J. Comput. Math. 25 (1) (2007) 89–96. W.J. Zhou, K.C. Toh, Superlinear convergence of a Newton-type algorithm for monotone equations, J. Optim. Theory and Appl. 125 (1) (2005) 205–221. W. Cheng, A two-term PRP-based descent method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230. E.D. Dolan, J.J. Moré, Benchmarking optimization software with performance profiles, Math. program. Ser. A 91 (2002) 201–213.
Contents lists available at ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
FR type methods for systems of large-scale nonlinear monotone equations Zoltan Papp, Sanja Rapajic´ ∗ Department of Mathematics and Informatics, University of Novi Sad, Trg Dositeja Obradovi´ca 4, 21000 Novi Sad, Serbia
a r t i c l e
i n f o
Keywords: Nonlinear monotone systems Hyperplane projection method Derivative-free line search Conjugate gradient directions
a b s t r a c t A large class of iterative methods for solving nonlinear monotone systems is developed in recent years. In this paper we propose some new FR type directions in the frame of algorithm which is a combination of conjugate gradient approach and hyperplane projection technique. Derivative-free, function-value-based line search combined with projection procedure is used for globalization strategy. Numerical performances of methods with different search directions are compared. © 2015 Elsevier Inc. All rights reserved.
1. Introduction We consider the system of nonlinear monotone equations of the form
F (x) = 0,
(1)
where function F: Rn → Rn is continuous and monotone, which means
(F (x) − F (y))T (x − y) ≥ 0, ∀x, y ∈ Rn . It is well known that under this assumption, the solution set of (1) is convex unless is empty, see [1]. Nonlinear monotone systems arise in various practical situations and applications in industry, technology and engineering, so a large class of iterative methods for solving these systems is developed in recent years. Conjugate gradient (CG) techniques are efficient for large-scale optimization problems and nonlinear systems, due to low memory. This is the reason why many iterative methods with CG directions are derived in the past. First of all, CG methods and their modifications for unconstrained optimization problems are presented in many papers [2–12]. Motivated by them, during the last decade, some authors adapted this CG approach to solving nonlinear monotone systems [13–19]. On the other hand, projection method is suitable for monotone equations, because it enables simply globalization. Solodov and Svaiter first introduced this idea. The method presented in [20] is truly globally convergent i.e. the whole sequence of iterates globally converges to the solution of system (1). Inspired by this and the fact that CG framework is at low cost, many algorithms for solving systems of monotone equations combines CG directions with projection technique. As opposed to classical globalization strategy where the line search is based on merit function, the derivative-free line search without merit function is proposed in some papers [13–15,17–22], with the aim of overcoming disadvantages of the classical approach.
∗
Corresponding author. Tel.: +381214852856; Ffax: +381216350458. ´ E-mail addresses: [email protected] (Z. Papp), [email protected] (S. Rapajic).
http://dx.doi.org/10.1016/j.amc.2015.08.002 0096-3003/© 2015 Elsevier Inc. All rights reserved.
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
817
The derivative-free line search, in a given direction, is used to construct a hyperplane which separates the current iterate from the solution set. Projecting the current iterate onto this hyperplane ensures global convergence and it is computationally negligible, so this approach is appropriate for solving monotone systems. Using derivative-free, function-value-based line search is promising, because it can also be applied to nonsmooth systems. So, the idea is to combine derivative-free line search with conjugate gradient direction and projection procedure, in order to construct a function-value-based method for solving largescale monotone systems. In this paper we propose some new Fletcher–Reeves (FR) type directions in the frame of algorithm given in [13] and compare the behavior of methods with various search directions. The paper is organized as follows. The algorithm and new FR type search directions are presented in Section 2. Convergence results are established in Section 3. Preliminary numerical results, obtained by comparing methods with different search directions, are given in Section 4. Some concluding remarks are made in the last section. 2. The algorithm One way of dealing with system (1) is considering the equivalent problem of minimizing the merit function 12 F (x)2 , so a large number of iterative methods have been developed to solve this unconstrained optimization problem. Unfortunately, these methods have some certain shortcomings. As it is mentioned before, in order to eliminate them, some algorithms without merit function have been presented recently, see [13–15,17–19,21,22]. The line search proposed in them is based only on the value of function F and it doesn’t use merit function. These algorithms are also based on projection technique which fully exploits the monotonicity property of the system (1). Projection procedure generates a sequence {zk } such that zk = xk + αk dk , where the steplength α k > 0 is obtained using some line search along the search direction dk such that
F (zk )T (xk − zk ) > 0.
(2)
On the other hand, the monotonicity of F implies that for every solution
F (zk )T (x∗ − zk ) = (F (zk ) − F (x∗ ))T (x∗ − zk ) ≤ 0.
x∗
of the system (1) we have (3)
It is clear from (2) and (3) that the hyperplane
Hk = {x ∈ Rn |F (zk )T (x − zk ) = 0}
(4) x∗
strictly separates the current iterate xk from the solution of system (1). So, the idea is to project the current iterate xk onto this hyperplane Hk and to compute the next iterate xk+1 in this way
xk+1 = xk −
F (zk )(xk − zk ) F (zk ). F (zk )2
(5)
The most computational complexity of method depends on defining the search direction and steplength, so it is very important to choose a proper line search. The derivative-free line search, proposed in [15] and given in [13], doesn’t use merit function and it is based only on the evaluation of function F and because of that it is suitable for large-scale systems and also for nonsmooth systems. The steplength α k > 0 is determined such that
−F (xk + αk dk )T dk ≥ σ αk F (xk + αk dk )dk 2 ,
(6)
where dk is a search direction satisfying the condition
F (xk )T dk ≤ −τ F (xk )2
(7)
for τ > 0. We consider the algorithm proposed in [13], but with different CG directions, and compare numerical performances of the algorithm with various search directions. This is derivative-free, function-value-based algorithm which combines projection procedure and CG techniques. Projection procedure ensures simply globalization and thanks to low memory requirement and simplicity of CG framework, this algorithm is appropriate for solving large-scale monotone systems. In this section we define three new FR type directions which satisfy condition (7). Now, we can state the main algorithm as follows. ALGORITHM 1. Input: Choose x0 ∈ Rn , constants kmax > 0, σ > 0, s > 0, ε > 0, ρ ∈ (0, 1). Begin Set k := 0, F0 := F(x0 ). While (Fk ≥ ε and k ≤ kmax ) Determine the direction dk which satisfies (7); α = s; Compute F (xk + α dk ); While ( − F (xk + α dk )T dk < σ αF (xk + α dk )dk 2 ) α := ρα ;
818
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
compute F (xk + α dk ); EndWhile αk = α ; zk = xk + αk dk ; If F(zk ) ≤ ε then xk+1 = zk ; STOP;
F (zk )T (xk −zk ) F (zk ); F (zk )2
Else compute xk+1 = xk − EndIf Fk+1 = F (xk+1 ); k := k + 1; EndWhile End
The line search condition in Algorithm 1 doesn’t use a merit function and derivatives, so it is derivative-free, function-valuebased line search. Since the direction dk satisfies (7), it is easy to see that this line search necessarily holds for sufficiently small steplength α k > 0, which can be obtained after a finite number of reductions. This means that the algorithm is well defined. Defining suitable search direction and using inexpensive line search can decrease the computational cost of method. As we mentioned before, CG directions are suitable for solving large-scale unconstrained optimization problems and large-scale systems, because of low storage requirement. A comprehensive overview of nonlinear CG methods for unconstrained minimization is given in [4] and several modifications of CG methods have been developed recently. Modified two-term and three-term CG directions are proposed in [10–12,23]. Motivated by them, many authors apply that idea to solving nonlinear monotone systems, see [13–17]. Inspired by the directions proposed in [11] and [13], which are used in methods for solving unconstrained optimization problems and monotone equations, we define several new modified three-term FR type directions for nonlinear monotone systems, depending on the way of determining θ k . From now, the notation Fk = F (xk ) will be used. The search direction is defined by
dk =
−Fk ,
if k = 0,
−Fk + βkF R wk−1 − θk Fk ,
if k > 0,
(8)
where
βkF R =
Fk 2 , Fk−1 2
(9)
wk = zk − xk = αk dk and a coefficient θ k is determined in three different ways, such that the condition (7) is satisfied. M3TFR1 direction: From (8) and (9) we have
FkT dk = −Fk 2 +
Fk 2 T F w − θk Fk 2 Fk−1 2 k k−1
(10)
for k ∈ N. Choosing
θk =
FkT wk−1
Fk−1 2
,
(11)
there follows FkT dk = −Fk 2 , so (7) holds for τ = 1. The first modified three-term FR type direction is called M3TFR1 direction and it is defined by (8), (9) and (11). Algorithm 1 with M3TFR1 direction will be named M3TFR1 method. M3TFR2 direction: In a similar way as in [13], we have
FkT dk =
−Fk 2 Fk−1 4 + Fk 2 Fk−1 2 FkT wk−1 − θk Fk 2 Fk−1 4
Fk−1 4
for k ∈ N. Setting u = implies
FkT dk
√1 Fk−1 2 Fk , 2
√
v = 2Fk 2 wk−1 and using the inequality uT v ≤ 12 (u2 + v2 ), the previous relation (12)
Fk−1 4 Fk 2 + 2Fk 4 wk−1 2 ≤ − θk Fk 2 Fk−1 4 1 Fk 4 = −Fk 2 + Fk 2 + w 2 − θk Fk 2 4 Fk−1 4 k−1 3 Fk 4 = − Fk 2 + w 2 − θk Fk 2 . 4 Fk−1 4 k−1 −Fk 2 Fk−1 4 +
1 2
(12)
1 2
(13)
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
819
Defining
θk =
Fk 2 wk−1 2 , Fk−1 4
(14)
we get FkT dk ≤ − 34 Fk 2 , so the condition (7) holds for τ = 3/4. The direction dk defined by (8), (9) and (14) is M3TFR2 direction, while Algorithm 1 with this direction is named M3TFR2 method. M3TFR3 direction: The coefficient θ k is determined in such a way that it consists of two parts. The first one is (11), while the second one is properly chosen such that (7) is satisfied. So,
θk =
FkT wk−1
Fk−1
2
+
Fk 2 . Fk−1 4
(15) F 4
Then, it is not difficult to verify that FkT dk ≤ −Fk 2 − F k 4 ≤ −Fk 2 , for k ∈ N, which means that the relation (7) holds for k−1 τ = 1. The direction M3TFR3 is defined by (8), (9) and (15) and Algorithm 1 with this direction is M3TFR3 method. Lemma 1. The directions M3TFR1, M3TFR2 and M3TFR3 satisfy condition (7) for every k ∈ N ∪ {0}. Proof. Since F0T d0 = −F0 2 for every of three given directions, for k = 0 the condition (7) trivially holds with τ = 1. For k ∈ N, we have just shown that both directions M3TFR1 and M3TFR3 satisfy (7) with τ = 1, while the direction M3TFR2 satisfies (7) with τ = 3/4. 3. Convergence analysis The global convergence of Algorithm 1 with new modified three-term directions M3TFR1, M3TFR2 and M3TFR3 will be proved under the following assumptions: A1 the function F(x) is monotone on Rn , A2 the solution set of system (1) is nonempty, A3 the function F(x) is Lipschitz continuous on Rn . The following lemma is originally from Solodov and Svaiter [20] and it also holds for Algorithm 1 with any of three new directions M3TFR1, M3TFR2 or M3TFR3, so the proof is omitted. This lemma guaranties that {xk − x∗ } is a decreasing and convergent sequence and in particular, the sequence {xk } is bounded. Lemma 2 ([20]). Suppose that assumptions A1-A3 are satisfied and the sequence {xk } is generated by Algorithm 1 with M3TFR1, M3TFR2 or M3TFR3 direction. For any solution x∗ of system (1) there holds
xk+1 − x∗ 2 ≤ xk − x∗ 2 − xk+1 − xk 2
(16)
and the sequence {xk } is bounded. Furthermore, either {xk } is finite and the last iteration is a solution of (1), or {xk } is infinite and ∞
xk+1 − xk 2 < ∞,
k=0
which means
lim xk+1 − xk = 0.
(17)
k→∞
The next lemma states that the sequence {Fk } is bounded. Lemma 3. Suppose that the assumptions A1–A3 are satisfied and the sequence {xk } is generated by Algorithm 1 with M3TFR1, M3TFR2 or M3TFR3 direction. Then the sequence {Fk } is bounded, i.e. there exists a constant κ > 0 such that Fk ≤ κ for every k ∈ N ∪ {0}. Proof. Lemma 2 implies that xk − x∗ ≤ x0 − x∗ . From that fact and the assumption A3 there follows
F (xk ) = F (xk ) − F (x∗ ) ≤ Lxk − x∗ ≤ Lx0 − x∗ . The statement of lemma holds taking κ = Lx0 − x∗ .
The following theorem guaranties that the directions M3TFR1, M3TFR2 and M3TFR3 are bounded. Theorem 1. Suppose that the assumptions A1-A3 are satisfied and the sequence of directions {dk } is generated by Algorithm 1 with M3TFR1, M3TFR2 or M3TFR3 direction. Let ε 0 > 0 is a constant such that
Fk ≥ ε0
(18)
holds for every k ∈ N ∪ {0}. Then the directions {dk } are bounded, i.e. there exists a constant M > 0 such that
d k ≤ M holds for every k ∈ N ∪ {0}.
(19)
820
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
Proof. The definition of zk , relation (5) and line search condition (6) imply
|F (zk )T (xk − zk )| |αk F (zk )T dk | = F (zk ) F (zk ) αk2 σ F (zk )dk 2 ≥ = σ αk2 dk 2 ≥ 0. F (zk )
xk+1 − xk =
(20)
From (17) and (20) there follows
lim
k→∞
αk dk = 0.
(21)
On the other hand using Cauchy–Schwartz-inequality we have
xk+1 − xk =
|αk F (zk )T dk | αk F (zk )dk ≤ = αk dk . F (zk ) F (zk )
(22)
Now, we are going to prove that the directions M3TFR1, M3TFR2 and M3TFR3 are bounded. M3TFR1 direction: The definition of M3TFR1 direction given by (8), (9) and (11), Lemma 3 and (18) imply
FkT wk−1 Fk 2 w − F k−1 Fk−1 2 Fk−1 2 k Fk 2 Fk 2 ≤ Fk + αk−1 dk−1 + α d 2 Fk−1 Fk−1 2 k−1 k−1 Fk 2 = Fk + 2 α d Fk−1 2 k−1 k−1 κ2 ≤ κ + 2 2 αk−1 dk−1 ε0
dk = − Fk +
(23)
for all k ∈ N. From (21) follows that for every ε 1 > 0 there exists k0 such that αk−1 dk−1 < ε1 for every k > k0 . Choosing ε1 = ε02 and M = max{d0 , d1 , . . . , dk0 , M1 }, where M1 = κ(1 + 2κ), it holds dk ≤ M for every k ∈ N. M3TFR2 direction: Analogously, the definition of M3TFR2 direction given by (8), (9) and (14), Lemma 3 and (18) imply
Fk 2 F 2 wk−1 2 wk−1 − k Fk 2 Fk−1 Fk−1 4 Fk 2 Fk 3 ≤ Fk + α d + (α d )2 Fk−1 2 k−1 k−1 Fk−1 4 k−1 k−1 κ2 κ3 ≤ κ + 2 αk−1 dk−1 + 4 (αk−1 dk−1 )2 ε0 ε0
dk = − Fk +
(24)
for every k ∈ N. The limit (21) implies that for every ε 1 > 0 there exists an index k0 such that αk−1 dk−1 < ε1 for all k > k0 . If we choose ε1 = ε02 and M = max{d0 , d1 , . . . , dk0 , M1 }, where M1 = κ(1 + κ + κ 2 ), then the direction M3TFR2 is bounded i.e. dk ≤ M for every k ∈ N. M3TFR3 direction: From the definition of M3TFR3 direction given by (8), (9) and (15), Lemma 3 and (18), it follows
dk = − Fk +
T Fk wk−1 Fk 2 Fk 2 w − + F Fk−1 2 k−1 Fk−1 2 Fk−1 4 k
Fk 2 Fk 3 α d + Fk−1 2 k−1 k−1 Fk−1 4 κ2 κ3 ≤ κ + 2 2 αk−1 dk−1 + 4 ε0 ε0 ≤ Fk + 2
(25)
for every k ∈ N. Choosing ε1 = ε02 , M = max{d0 , d1 , . . . , dk0 , M1 } and M1 = κ(1 + 2κ + κ 2 /ε04 ) there follows that dk ≤ M, for all k ∈ N. As it is mentioned before, it is not difficult to prove that the line search procedure, along the direction dk which is bounded and satisfies (7), in Algorithm 1 terminates for sufficiently small step-length α k > 0 after a finite number of reductions. This means that the line search is well defined. Lemma 4. [13] Suppose that all conditions of Theorem 1 are satisfied. Then the line search procedure in Algorithm 1 is well defined. Since we have proved by Theorem 1 and Lemma 1 that M3TFR1, M3TFR2 and M3TFR3 directions are bounded and satisfy (7), there follows by the above lemma that the line search procedure is well defined, which also implies that M3TFR1, M3TFR2 and M3TFR3 methods are well defined.
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
821
1 0.9 0.8 0.7
ρ(t)
0.6 0.5 0.4 0.3 M3TFR1 M3TFR2 M3TFR3 DFPB1 DFPB2
0.2 0.1 0
1
2
3
4
5
t Fig. 1. Performance profiles for the number of iterations.
The main theorem of global convergence of Algorithm 1, with bounded sequence of directions {dk } satisfying (7), is established in [13]. This theorem also holds for M3TFR1, M3TFR2 and M3TFR3 methods, because M3TFR1, M3TFR2 and M3TFR3 directions are bounded and satisfy condition (7), so the proof of the theorem is omitted here. Theorem 2. [13] Suppose that assumptions A1–A3 are satisfied and the sequence {xk } is generated by M3TFR1, M3TFR2 or M3TFR3 method. Then
lim inf Fk = 0.
(26)
k→∞
From the continuity of F, the boundedness of {xk } and the above theorem, there follows that the sequence {xk }, generated by any of three methods: M3TFR1, M3TFR2 or M3TFR3, has the accumulation point x∗ such that F (x∗ ) = 0. On the other hand, the sequence {xk − x∗ } is convergent by Lemma 2, which means that the whole sequence {xk } globally converges to the solution x∗ of the system (1). 4. Numerical results In this section we present some numerical results obtained by applying five methods on the following set of test problems, which can be found in [1,14,17,19]. We tested Problems 1–3 from [19] with dimensions 1000, 20, 000 and 50, 000, Problem 1 from [14] with dimensions 1000, 20, 000 and 50, 000, Problem 2 from [14] with dimensions 1000 and 5000, Problems 2 from [17] with dimension 1000, Problem 3 from [17] with dimensions 1000 and 3000 and Problem 4 from [17] with dimensions 1000, 20, 000 and 50, 000. We also tested the monotone, nonlinear system with dimension 20, 164, obtained by discretization of Dirichlet problem, which is given in [1]. Numerical performances of three new methods proposed in this paper are compared with two methods given in [13]. More precisely, we tested 5 methods: M3TFR1, M3TFR2, M3TFR3, DFPB1 and DFPB2, where DFPB1 and DFPB2 are presented in [13]. All algorithms are implemented in Matlab R2011b environment on a 2.13 GHz Intel dual-core processor computer with 4 GB of RAM. The stopping criteria in all experiments is the same as in [13]
Fk ≤ ε or F (zk ) ≤ ε or k > kmax , where ε = 10−4 and maximum number of iteration is kmax = 500, 000. In a similar way as in [13,15,17], the initial trial steplength sk is determined by the approximation of |
FkT dk
dT Jk dk k
|, where Jk = J(xk ) is the Jacobian matrix of F(xk ). To avoid calculating derivatives,
the finite differences are used to approximate the Jacobian, so
T F ( x ) d k k , sk = T ((F (xk + tdk ) − F (xk )) dk )/t
(27)
where t = 10−8 . We also used the same parameters σ = 0.3 and ρ = 0.7 as in [13]. All methods are initialized with sev(1) (2) (3) (4) (5) (6) (7) eral starting points: x0 = 10 · 1n×1 , x0 = −10 · 1n×1 , x0 = 1n×1 , x0 = −1n×1 , x0 = 0.1 · 1n×1 , x0 = [1, 12 , 13 , . . . , 1n ]T , x0 = [ 1n , 2n , . . . ,
(8) n−1 T n , 1] , x0
= [1 − 1n , 1 − 2n , . . . , 1 −
n−1 T n , 0] ,
where n is a dimension of the system.
822
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
1 0.9 0.8 0.7
ρ(t)
0.6 0.5 0.4 0.3 M3TFR1 M3TFR2 M3TFR3 DFPB1 DFPB2
0.2 0.1 0
1
2
3
4
5
t Fig. 2. Performance profiles for the number of function evaluations.
1 0.9 0.8 0.7
ρ(t)
0.6 0.5 0.4 0.3 M3TFR1 M3TFR2 M3TFR3 DFPB1 DFPB2
0.2 0.1 0
1
2
3
4
5
t Fig. 3. Performance profiles for the CPU time.
In order to make detailed comparison of the efficiency and robustness of all methods, we have used performance profile proposed in [24], which is useful methodology for standardizing the comparison of algorithms. We considered the number of iterations, the number of function evaluations and CPU time as a measure of performance profile. Fig. 1 reveals that M3TFR2 method is the most efficient in the sense of number of iterations, since it solves 40% of problems with the smallest number of iterations, while M3TFR1 method solves 37% of problems. The most robust algorithms are M3TFR3 and DFPB1, because both of their cumulative distribution functions ρ (t) reach 1 for minimal t. These two robust methods are followed by M3TFR2 method. It is clear from Fig. 2 that the most efficient method in the sense of number of function evaluations is M3TFR2 method, with 44% problems solved with the smallest number of function evaluations, while the most robust is DFPB1 method, which is also followed by M3TFR2 method. Considering the CPU time, Fig. 3 shows that the best method is M3TFR2, because it is the most efficient and the most robust. It is followed by M3TFR1 and M3TFR3 methods in the sense of efficiency and by DFPB1 and DFPB2 in the sense of robustness.
Z. Papp, S. Rapaji´c / Applied Mathematics and Computation 269 (2015) 816–823
823
Based on Fig. 1–3, it can be concluded that M3TFR2 method performs better than other methods. Preliminary numerical results pointed out that employing new modified three-term FR type directions is promising, because new methods are competitive with methods presented in [13]. Numerical experiments report that new methods are suitable for large-scale monotone systems, especially M3TFR2 method, which is computationally very efficient. 5. Conclusions This paper presents several new modified three-term FR type directions in the frame of algorithm which combines conjugate gradient techniques and projection method. The algorithm belongs to the class of derivative-free, function-value-based methods, since it does not use merit function and derivatives. The low memory storage and simplicity of the CG approach makes the algorithm appropriate for solving large-scale monotone systems, while the projection procedure is suitable for monotone equations, because it enables simply globalization. Introducing new FR type search directions, we propose three new methods for solving large-scale monotone systems. The sequence {xk } generated by any of these new methods globally converges to the solution of the monotone system (1), without any regularity and differentiability assumption. Numerical experiments indicate that new methods based on modified three-term FR type directions are efficient, robust and comparable with other methods suitable for solving large-scale monotone systems. Moreover, these methods can be successfully applied to nonsmooth monotone systems. Acknowledgments Research supported by Serbian Ministry of Education, Science and Technological Development, grant no. 174030. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
J.M. Ortega, W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970. N. Andrei, Another hybrid conjugate gradient algorithm for unconstrained optimization, Numer. Algor. 47 (2008) 143–156. W. Cheng, Q. Liu, Sufficient descent nonlinear conjugate gradient methods with conjugacy condition, Numer. Algor. 53 (2010) 113–131. W.W. Hager, H. Zhang, A survey of nonlinear conjugate gradient methods, Pac. J. Optim. 2 (2006) 35–58. E. Polak, G. Ribiere, Note sur la convergence de directions conjugees, Rev. Francaise Imformat Recherche Opertionelle 16 (1969) 35–43. B.T. Polyak, The conjugate gradient method in extreme problems, USSR Comp. Math. Phys. 9 (1969) 94–112. M. Raydan, On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numer. Anal. 13 (1993) 321–326. M. Raydan, The Barzilai and Borwein gradient method for the large-scale unconstrained minimization problem, SIAM J. Optim. 7 (1997) 26–33. G. Yu, L. Guan, W. Chen, Spectral conjugate gradient method with sufficient descent property for large-scale unconstrained optimization, Optim. Methods Softw. 23 (2) (2008) 275–293. L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiere–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640. L. Zhang, W.J. Zhou, D.H. Li, Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search, Numer. Math. 104 (2006) 561–572. L. Zhang, W.J. Zou, D.H. Li, Some descent three-term conjugate gradient method and their global convergence, Optim. Methods Softw. 22 (4) (2007) 697–711. M. Ahookhosh, K. Amini, S. Bahrami, Two derivative-free projection approaches for systems of large-scale nonlinear monotone equations, Numer. Algor. 64 (4) (2013) 21–42. W. Cheng, A PRP type method for systems of monotone equations, Math. Comput. Model. 50 (2009) 15–20. Q. Li, D.H. Li, A class of derivative-free methods for large-scale nonlinear monotone equations, IMA J. Numer. Anal. 31 (2011) 1625–1635. D.H. Li, X.L. Wang, A modified Fletcher–Reeves-type derivative-free method for symmetric nonlinear equations, Numer. Algebra Control Optim. 1 (1) (2011) 71–82. Q.R. Yan, X.Z. Peng, D.H. Li, A globally convergent derivative-free method for solving large-scale nonlinear monotone equations, J. Comput. Appl. Math. 234 (2010) 649–657. L. Zhang, W.J. Zhou, Spectral gradient projection method for solving nonlinear monotone equations, J. Comput. Appl. Math. 196 (2006) 478–484. W.J. Zhou, D.H. Li, A globally convergent BFGS method for nonlinear monotone equations without any merit functions, Math. Comput. 77 (2008) 2231–2240. M.V. Solodov, B.F. Svaiter, A globally convergent inexact Newton method for systems of monotone equations., in: M. Fukushima, L. Qi (Eds.), Reformulation: Nonsmooth, Piecewise smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, 1998, pp. 355–369. W.J. Zhou, D.H. Li, Limited memory BFGS method for nonlinear monotone equations, J. Comput. Math. 25 (1) (2007) 89–96. W.J. Zhou, K.C. Toh, Superlinear convergence of a Newton-type algorithm for monotone equations, J. Optim. Theory and Appl. 125 (1) (2005) 205–221. W. Cheng, A two-term PRP-based descent method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230. E.D. Dolan, J.J. Moré, Benchmarking optimization software with performance profiles, Math. program. Ser. A 91 (2002) 201–213.