A smoothing iterative method for the finite minimax problem

Journal Pre-proof A smoothing iterative method for the finite minimax problem J.K. Liu, L. Zheng

PII: DOI: Reference:

S0377-0427(20)30032-7 https://doi.org/10.1016/j.cam.2020.112741 CAM 112741

To appear in:

Journal of Computational and Applied Mathematics

Received date : 8 March 2019 Revised date : 25 December 2019 Please cite this article as: J.K. Liu and L. Zheng, A smoothing iterative method for the finite minimax problem, Journal of Computational and Applied Mathematics (2020), doi: https://doi.org/10.1016/j.cam.2020.112741. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2020 Elsevier B.V. All rights reserved.

Journal Pre-proof

1

J. K. Liua,b , L. Zhengc

of

A smoothing iterative method for the finite minimax problem a. School of Mathematics and Statistics, Chongqing Three Gorges University, Chongqing, China

p ro

b. Key Laboratory of Intelligent Information Processing and Control, Chongqing Three Gorges University, Chongqing, China

c. Faculty of Information Technique, Macau University of Science and Technology, Macau, China

Pr e-

Abstract

In this paper, we proposed a smoothing iterative method to solve the finite minimax problems based on the exponential penalty function of Kort and Bertsekas [Proc. IEEE Conf. New Orleans (1972)]. This approach can be viewed as the extension of one conjugate gradient method. Under suitable conditions, the proposed method is globally convergent. Preliminary numerical results and comparisons show that the proposed method is effective and promising. Keywords: minimax problem, smoothing method, conjugate gradient method, global convergence.

al

1. Introduction

where

urn

In this paper we consider the following minimax optimization problem min f (x), x ∈ Rn

(1.1)

f (x) , max fi (x).

(1.2)

i=1,...,m

Jo

Here the function fi : Rn → R is continuous differentiable, ∀i = 1, 2, . . . , m. The problem (1.1) arises from many practical applications, e.g., the personnel scheduling problems in service organizations [1], the engineering design [2], the computer-aided-design [3], the staffing problems in call centers [4], the optimal control [5] and some economic problems [6]-[7]. For the detailed introduction of the problem (1.1), interested readers can see the references [8]-[9]. 1 This research was partially supported by Chongqing Research Program of Basic Research and Frontier Technology (Grant number:cstc2017jcyjAX0318), The fund of Scientific and Technological Research Program of Chongqing Municipal Education Commission (Grant number:KJQN201801208), Major Cultivation Program of Chongqing Three Gorges University(Grant number: 16PY12) and Chongqing Municipal Key Laboratory of Institutions of Higher Education (Grant No.[2017]3). Corresponding author. Email address: [email protected] (J. K. Liu) Preprint submitted to January 27, 2020

Journal Pre-proof

2

Pr e-

p ro

of

To solve the problem (1.1), assuming that the function fi (x) is twice continuous differentiable, many methods have been established for the solution of the problem (1.1) by solving a sequence of either unconstrained subproblems or constrained subproblems of the trust-region type (Refs.[10]-[13]), but there are only few approaches for solving the large-scale problems. Recently, Luksan e.t. [14] proposed a primal interior-point method for large sparse minimax optimization. Li and Huo [15] proposed an inexact Newton-type algorithm for solving large-scale minimax problems based on the maximum entropy function. The proposed algorithm is showed to be globally and super-linearly convergent. But an approximate solution of a linear system is required at each iteration. In this paper we are interested in solving the minimax problems (1.1), where the function fi (x) (i = 1, 2, . . . , m) is only continuous differentiable. This implies that the objective function f (x) is nonsmooth at points where two or more of the functions fi (x) are equal to f (x) even if each fi (x) has a continuous first partial derivative. Hence, the minimax problem (1.1) is a special nonsmooth unconstrained optimization problem, which is more complex than the methods for solving the smooth unconstrained optimization problem. To handle this difficulty, some regularization approaches have been used to obtain smoothing approximations of the minimax problem, see Refs. [13] and [16], which can convert the problem (1.1) into simple smooth unconstrained optimization problem. To establish a smoothing method for solving the problem (1.1), in this paper we need the following the smoothing function ( ) m ∑ fi (x) (1.3) f (x, u) = u ln exp , u > 0, u i=1 to approximate the objective function f (x), where m comes from the problem (1.1). This function is called the exponential penalty function [17]-[18], and has been used to solve nonlinear programming problems, the generalized linear or nonlinear complementarity problems, see Refs.[19]-[21]. An remarkable property of this function is that

al

f (x) ≤ f (x, u) ≤ f (x) + u ln m.

Jo

urn

It is easy to observe that f (x, u) −→ f (x) as u −→ 0+ . Thus, the function f (x, u) is a good approximation to the objective function f (x). In this paper, we propose and analyze an effective iterative method for solving the problem (1.1) by using the exponential penalty function, which can be viewed as an extension of one conjugate gradient method in the new field. The proposed method owns some attractive properties, for example, it doesn’t require the Jacobian matrix of the merit function f (x, u), and doesn’t store any matrix in per-iteration. A line search based on the merit function f (x, u) is used to prove the global convergence of the proposed method. Moreover, numerical results show that the proposed method is competitive with its competitor. The remainder of this paper is organized as follows. In Section 2 we introduce some properties and establish the smoothing method. In Section 3 we prove its global convergence. In Section 4, we provide numerical experiments to show the performance of the proposed method. 2. Some properties and the algorithm In this section we first recall some famous properties on the minimax problem, which come from the references [8] and [21].

Journal Pre-proof

3 Proposition 2.1. A point x∗ ∈ Rn is a stationary point of the minimax problem (1.1) if and only if there exists a vector y∗ = (y∗1 , y∗2 , . . . , y∗m )T such that

i=1

y∗i ∇ fi (x∗ ) = 0, y∗ ≥ 0,

m ∑

y∗i = 1,

i=1

of

m ∑

y∗i = 0 i f fi (x∗ ) < max{ f1 (x∗ ), f2 (x∗ ), . . . , fm (x∗ )}.

(2.1) (2.2)

p ro

Proposition 2.2. If x∗ is a local minimum of the minimax problem (1.1), then x∗ is a stationary point satisfying (2.1)-(2.2). Conversely, assume that f (x) is convex, then if x∗ is a stationary point, x∗ is a global minimum of the minimax problem (1.1). In [21], Peng and Lin showed some nice properties of the exponential penalty function f (x, u) when fi (x)(i = 1, 2, . . . , m) are continuous differentiable.

Pr e-

Proposition 2.3. Suppose that fi (x)(i = 1, 2, . . . , m) are continuous differentiable, then we have (1) f (x, u) is increasing with respect to u, and f (x) ≤ f (x, u) ≤ f (x) + u ln m. (2) For any x ∈ Rn and u > 0, it holds that 0 ≤ fu′ (x, u) ≤ ln m. In the following, we describe the algorithm based on the exponential penalty function f (x, u). Throughout the paper, || · || denotes the ℓ2 norm, and gk = ∇ x f (xk , uk ).

Algorithm 2.1. Step 0: Choose the initial point x0 , set ρ, β ∈ (0, 1), u0 , γ, σ, t > 0. Let d0 = −g0 . Set k := 0. Step 1: Terminate if the stopping criteria is satisfied. Otherwise, compute αk by the Armijo line search, i.e., αk = max{ρi | i = 0, 1, 2, . . .} satisfies f (xk + αk dk , uk ) ≤ f (xk , uk ) + σαk gTk dk .

(2.3)

al

Step 2: Let xk+1 = xk + αk dk , and compute gk+1 . If ||gk+1 || ≤ γuk , then set uk+1 = βuk ; otherwise, let uk+1 = uk Step 3: Obtain the search direction dk+1 by dk+1 = −gk+1 + βk+1 dk + θk+1 yk , ) T 2 gT d where βk+1 = ||d1k ||2 yk − t ||y||dk ||k ||d2 k gk+1 , θk+1 = − ||dk+1k ||2k , yk = gk+1 − gk . Step 4: Set k := k + 1, go to step 1.

urn

(

(2.4)

Lemma 2.1. Suppose that fi (x)(i = 1, 2, . . . , m) are continuous differentiable, then Algorithm 2.1 is well-defined. Proof. From Step 3, for any k ≥ 1 we have = −||gk ||2 + βk gTk dk−1 + θk gTk yk−1

gTk yk−1 · gTk dk−1 ||yk−1 ||2 · (gTk dk−1 )2 gTk yk−1 · gTk dk−1 − t − ||dk−1 ||2 ||dk−1 ||4 ||dk−1 ||2 2 T 2 ||yk−1 || · (gk dk−1 ) = −||gk ||2 − t ||dk−1 ||4 ≤ −||gk ||2 .

Jo

gTk dk

= −||gk ||2 +

Journal Pre-proof

4 Since d0 = −g0 , we have gT0 d0 = −||g0 ||2 . Thus, for any k ≥ 0 we have gTk dk ≤ −||gk ||2 .

(2.5)

of

This implies that dk is a sufficient descent direction of the smoothing function f (x, u) at the k − th iteration whenever dk , 0. Therefore the Armijo line search procedure is finite terminating.

p ro

Remark 2.1. The proposed algorithm may be viewed as an extension of the RMIL conjugate gradient method [22] based on the three-term conjugate gradient method [23] and the famous CG DESCENT method [24]. An remarkable property is that the proposed algorithm satisfies the sufficient descent condition (2.5) for the smoothing function f (x, u) without any line search. 3. Convergence

In this section we analyze and prove the convergence of the proposed algorithm. The following assumptions for the objective function f (x) and the smoothing function f (x, u) are needed.

Pr e-

Assumption 3.1. The function f (x) defined in (1.2) is coercive, i.e., f (x) → +∞ as ||x|| → +∞. Assumption 3.2. The function ∇ x f (x, u) is Lipschitz continuous with respect to x, i.e., there exists a constant L > 0 such that ||∇ x f (x, u) − ∇ x f (y, u)|| ≤ L||x − y||, ∀x, y ∈ Rn .

(3.1)

al

Lemma 3.1. Suppose that Assumptions 3.1-3.2 hold. Let the sequences {xk } and {dk } be generated by Algorithm 2.1, then we have (1) the sequence { f (xk , uk )} is monotonically decreasing. (2) the sequence {xk } is bounded. (3) the sequence { f (xk+1 , uk )} and { f (xk , uk )} are both convergent and have the same limit. (4) the sequence {dk } is bounded. Proof. (1) From (2.3) and (2.5), we have

urn

f (xk+1 , uk ) ≤ f (xk , uk ).

(3.2)

By Step 2 in Algorithm 2.1, the inequality uk+1 ≤ uk holds. Then it together with the part (1) of Proposition 2.3 obtains that f (xk+1 , uk+1 ) ≤ f (xk+1 , uk ). Thus, for any k ≥ 0 we have

f (xk+1 , uk+1 ) ≤ f (xk+1 , uk ) ≤ f (xk , uk ).

Jo

This means that the sequence { f (xk , uk )} is monotonically decreasing. (2) Set A = {x| f (x) ≤ f (x0 , u0 )}. From Assumption 3.1 we have that the set A is bounded. By the part (1) of Proposition 2.3 and the property of the sequence { f (xk , uk )}, we have xk ∈ {x| f (x, u) ≤ f (x0 , u0 )} ⊆ A.

Thus, the sequence {xk } is bounded.

Journal Pre-proof

5 (3) From Taylor expansion for u, we have f (xk+1 , uk+1 ) = f (xk+1 , uk ) + fu′ (xk+1 , u¯ )(uk+1 − uk ), where u¯ = uk+1 + θ(uk − uk+1 ) and θ ∈ [0, 1]. Then, from Step 2 in Algorithm 2.1 we have

of

f (xk+1 , uk+1 ) ≤ f (xk+1 , uk ) + | fu′ (xk+1 , u¯ )|(1 − β)uk .

It follows from the part (2) of Proposition 2.3 that

p ro

f (xk+1 , uk+1 ) ≤ f (xk+1 , uk ) + (1 − β)uk ln m. From (3.2) we have

f (xk+1 , uk+1 ) − (1 − β)uk ln m ≤ f (xk+1 , uk ) ≤ f (xk , uk ).

(3.3)

Pr e-

Since fi (x) is continuous differentiable and A is bounded, then fi (x) is bounded in A, i.e., there exists a constant q > 0 such that | fi (x)| ≤ q, ∀x ∈ A. Then we have fi (x) ≥ −q in A, ∀i = 1, 2, . . . , m. From (1.2) and the part (1) of Proposition 2.3, we have f (x, u) ≥ −q. Thus, the part (1) of Lemma 3.1 implies that the sequence { f (xk , uk )} is convergent. Since uk → 0 as k → ∞, by (3.3) we obtain that the sequence { f (xk+1 , uk )} converges to the same value as the sequence { f (xk , uk )}. (4) From the definition of f (x, u), it is not difficult to obtain that ∇ x f (x, u) = where

m ∑

λi (x, u)∇ fi (x),

i=1

m

Then we have

∑ exp( fi (x)/u) λi (x, u) = ∑m ∈ (0, 1), λi (x, u) = 1. j=1 exp( f j (x)/u) i=1 m ∑

λi (x, u)||∇ fi (x)|| ≤

al

||∇ x f (x, u)|| ≤

i=1

m ∑ i=1

||∇ fi (x)||.

urn

Since fi (x) is continuous differentiable and the sequence {xk } is bounded, the sequence {∇ x f (xk , uk )} is bounded, i.e., there exists a constant M > 0 such that ||gk || = ||∇ x f (xk , uk )|| ≤ M, ∀k ≥ 0.

From (2.4) we have

||gk || · ||yk−1 || · ||dk−1 || ||yk−1 ||2 · ||gk || · ||dk−1 ||2 ||gk || · ||dk−1 || · ||yk−1 || + t + ||dk−1 ||2 ||dk−1 ||4 ||dk−1 ||2 2 2 L ||sk−1 || · ||gk || ||gk || · L||sk−1 || ||gk || · L||sk−1 || +t ≤ ||gk || + + ||dk−1 || ||dk−1 || ||dk−1 ||2 2 2 = ||gk || + 2Lαk−1 ||gk || + tL αk−1 ||gk || ≤ M + 2LM + tL2 M, ≤ ||gk || +

Jo

||dk ||

where the first inequality follows from the Cauchy-Schwarz inequality, the second inequality follows from Assumption 3.2, and the third inequality follows from the definition of αk in (2.3) and the bounded property of the sequence {gk }.

Journal Pre-proof

6 Theorem 3.1. Suppose that Assumptions 3.1-3.2 hold. Let the sequence {xk } be generated by Algorithm 2.1, then the sequence {xk } has a limit point. Moreover, every limit point is a stationary point of the minimax problem (1.1).

p ro

f (xk+1 , uk ) ≤ f (xk , uk ) − σαk ||gk ||2 .

of

Proof. From the part (2) of Lemma 3.1, the sequence {xk } is bounded. Thus, the sequence {xk } has a limit point x¯. In the following, we assume that the sequence {xk } converges to x¯. From the part (4) of ¯ Lemma 3.1, we further assume that the sequence {dk } converges to d. From (2.3) and (2.5), we have

By the part (3) of Lemma 3.1, it is not difficult to obtain that lim αk ||gk || = 0.

k→∞

Pr e-

Assume that limk→∞ inf ||gk || > 0 holds, we have limk→∞ inf αk = 0. Without loss of generality, we assume that lim αk = 0. k→∞

ik

Since αk = ρ and ρ ∈ (0, 1), we have

lim ik = ∞.

k→∞

(3.4)

From the definition of αk and (2.5), it holds that

f (xk + ρik −1 dk , uk ) − f (xk , uk ) > σgTk dk ≥ −σ||gk ||2 . ρik −1

al

From the mean value theorem, it is easy to obtain that ∇ x f (xk + δk ρik −1 dk , uk )T dk > −σ||gk ||2 ,

urn

where δk ∈ (0, 1). By (3.4) and taking the limit in the above inequality, we have g¯ T d¯ > −σ||¯g||2 ,

(3.5)

where g¯ = ∇ x f ( x¯, 0+ ), and ∇ x f ( x¯, 0+ ) := limu→0+ ∇ x f ( x¯, u). By taking the limit in the inequality (2.5), we also have g¯ T d¯ ≤ −||¯g||2 .

(3.6)

Jo

Since σ ∈ (0, 1), it is clear to find that (3.5) contradicts with (3.6). Thus, we have g¯ = 0.

Let C(x) = {i ∈ {1, 2, . . . , m}| fi ( x¯) = f ( x¯)}. Then from (1.2) we have ) ( fi ( x¯) − f ( x¯) = 0, i < C(x). lim+ exp u→0 u

(3.7)

Journal Pre-proof

7 Thus, it holds that limu→0+ exp

λi ( x¯, 0+ ) = ∑ m

( f ( x¯)− f ( x¯) ) i

j=1 limu→0+ exp

∑m

u

( f j ( x¯)− f ( x¯) ) = 0, i < C(x). u

m ∑ i=1

λi ( x¯, 0+ )∇ fi ( x¯) = 0.

p ro

g¯ = ∇ x f ( x¯, 0+ ) =

of

where i=1 λi ( x¯, 0 ) = 1. Since fi (x) is continuous differentiable, we have ∇ fi ( x¯) = 0 for i ∈ C(x). Then from (3.7) it is not difficulty to obtain that +

From Proposition 2.1, it is easy to find that x¯ is a stationary point of the minimax problem (1.1). 4. Preliminary numerical experiments

Pr e-

In this section, we test our algorithm on some minimax problems, and compare its performance with the smoothing Fletcher-Reeves (FR) method [25] which performs better than the SGM method [26]. The parameters in the smoothing FR method come from the reference [25], and the parameters in Algorithm 2.1 are chosen to be σ = 0.25, ρ = 0.5, β = γ = 0.5, t = 1.5 and u0 = 0.5. All test algorithms were coded in Matlab 7.0 and they were run on a HP personal computer with 3.20GHz CPU processor, 4.0 GB memory and Windows 7 operating system. In this paper we stop the iteration if

urn

al

|gTk sk | ≤ 10−6 , ( ) where sk = xk+1 − xk . Note that exp fi u(x) in the exponential penalty function f (x, u) tends to be large when the parameter u approaches 0+ and fi (x) > 0. To present overflow, in our experiment we compute the exponential penalty function f (xk , uk ) defined in (1.3) and the parameter λi (xk , uk ) defined in Lemma 3.1 (4) as follows: ( ) m ∑ fi (xk ) f (xk , uk ) = uk ln exp uk i=1  ( ) m ( )  f (xk ) ∑ fi (xk ) − f (xk )  = uk ln exp exp  uk uk i=1 ( ) m ∑ fi (xk ) − f (xk ) = f (xk ) + uk ln exp . uk i=1 ( ) ) ( f (xk ) exp fi (xk )− exp fi u(xkk ) uk λi (xk , uk ) = ∑ ( f j (xk ) ) = ∑ ( f j (xk )− f (xk ) ) . m m exp exp j=1 j=1 uk uk The test problems are listed as:

Jo

Problem 4.1. This problem is Charalambous-Conn 1 problem[10]. f1 (x) = x12 + x24 , f2 (x) = (2 − x1 )2 + (2 − x2 )2 , f3 (x) = 2e−x1 +x2 .

The exact solution is x¯ = (1.1390, 0.8996)T .

Journal Pre-proof

8 Problem 4.2. This problem is Charalambous-Conn 2 problem [10].

The exact solution is x¯ = (1.0000, 1.0000)T . Problem 4.3. This problem is Cresent problem [10].

The exact solution is x¯ = (0.0000, 0.0000)T .

p ro

f1 (x) = x12 + (x2 − 1)2 + x2 − 1, f2 (x) = −x12 − (x2 − 1)2 + x2 + 1.

of

f1 (x) = x14 + x22 , f2 (x) = (2 − x1 )2 + (2 − x2 )2 , f3 (x) = 2e−x1 +x2 .

Problem 4.4. This problem is Demyanon-Malozemov probelm [10].

Pr e-

f1 (x) = 5x1 + x2 , f2 (x) = −5x1 + x2 , f3 (x) = x12 + x22 + 4x2 .

The exact solution is x¯ = (0.0000, −3.0000)T .

Problem 4.5. This problem is LQ problem [27].

f1 (x) = −x1 − x2 , f2 (x) = −x1 − x2 + (x12 + x22 − 1).

√ √ The exact solution is x¯ = ( 2/2, 2/2)T .

al

In addition, in this paper we also consider the following nonlinear programming problem: min g(x), s.t.gi (x) ≥ 0, i = 2, 3, . . . , m.

urn

In order to solve the above problem, we transform it into the minimax problem (1.1), i.e., f1 (x) = g(x), fi (x) = g(x) − ai gi (x), 2 ≤ i ≤ m, ai > 0.

Bandler and Charalambous [28] has proved that the optimum of the minimax problem (1.1) coincides with that the nonlinear programming problem for sufficiently large ai . Problem 4.6. This problem is Rosen-Suzuki problem [10].

Jo

g(x) = x12 + x22 + 2x32 + x42 − 5x1 − 5x2 − 21x3 + 7x4 , g2 (x) = −x12 − x22 − x33 − x42 − x1 + x2 − x3 + x4 + 8, g3 (x) = −x12 − 2x22 − x32 − 2x42 + x1 + x4 + 10, g4 (x) = −x12 − x22 − x32 − 2x1 + x2 + x4 + 5,

we use a2 = a3 = a4 = 10. The exact solution is x¯ = (0.0000, 1.0000, 2.0000, −1.0000)T .

Journal Pre-proof

9

Pr e-

p ro

of

We test each problem with 10 different initial points, and record the number of the iterations (Niter), the number of the function evaluations (Nf), the number of the gradient evaluations (Ng), the computational time in seconds (Time) and the approximate optimal solution x∗ , respectively. The detailed numerical results are listed in Tables 1-6. For each experiment, we compute the error between the exact solution x¯ and the approximate optimal solution x∗ obtained by the smoothing FR method or Algorithm 2.1, i.e., ε = || x¯ − x∗ ||, and plot two error curves, see Figure 1. Note that the lower the curve, the smaller the error. This implies that the corresponding method is more effective in terms of calculation results. According to the trend of curves, we find that, for Problem 4.1, the errors of Algorithm 2.1 is comparable with those of the smoothing FR method, but for other problems, the errors of Algorithm 2.1 are obviously smaller than those of the smoothing FR method. These imply that Algorithm 2.1 is more effective to get the better approximate solutions of the problems. Since the test methods belong to gradient-type iteration methods, in this section we apply the performance profiles of Dolan and Mor´e [29]-[30] to further analyze the effectiveness of the test methods based on the results (i.e., Niter, NF, Ng, Time, respectively) in Tables 1-6, and by the technique in [29]-[30] we obtain Figures 2-5. Dolan and Mor´e pointed out that the test method s with high value of χ s (τ) is preferable or represents the best method when τ takes certain value. Therefore, these figures indicate that Algorithm 2.1 performs better than the smoothing FR method. Table 1: The numerical results of the problem 4.1 by the smoothing FR method and Algorithm 2.1.

urn

(-1.2,-1) (0.4,0.7) (0.5,2) (1,-1) (1.3,-1.15) (1.3,0.5) (1.4,0.9) (1.4,1) (1.5,-1) (1.5,1)

Smoothing FR method Niter/Nf/Ng/Time x∗ 20/143/32/0.015 (1.1352,0.9017) 21/142/31/0.014 (1.1394,0.8977) 17/122/29/0.015 (1.1349,0.9019) 21/142/31/0.015 (1.1328,0.9027) 22/160/34/0.015 (1.1379,0.8996) 18/139/32/0.015 (1.1379,0.9001) 20/148/32/0.016 (1.1373,0.9000) 11/86/21/0.014 (1.1401,0.8970) 22/145/32/0.016 (1.1356,0.9005) 12/88/22/0.014 (1.1395,0.8975)

al

x0

Algorithm 2.1 Niter/Nf/Ng/Time 18/124/26/0.013 23/176/33/0.014 19/139/29/0.014 11/84/21/0.012 13/89/21/0.013 13/98/23/0.013 15/116/25/0.014 18/139/28/0.015 18/140/30/0.015 23/166/33/0.016

x∗ (1.1399,0.8956) (1.1412,0.8963) (1.1370,0.8995) (1.1412,0.8961) (1.1385,0.8967) (1.1430,0.8948) (1.1402,0.8969) (1.1370,0.8994) (1.1378,0.8996) (1.1390,0.8978)

5. Conclusion

Jo

In this paper, we proposed a smoothing algorithm for solving the minimax problems based on one new conjugate gradient method which has some nice properties. An remarkable feature of the proposed algorithm is that it only needs one-order derivative of the exponential penalty function f (x, u). Under suitable conditions, we proved that the algorithm is globally convergent. Preliminary numerical results and comparisons indicates the promising of the algorithm. References

[1] M.J. Brusco, L.W. Jacobs, Personnel tour scheduling when starting-time restrictions are present, Manage. Sci.,44(1998) 534-547.

Journal Pre-proof

10

0.05 0.045

Smoothing FR Algorithm 2.1

of

0.04 0.035 0.03 0.025

0.01 0.005 0

0

10

20

30

p ro

0.02 0.015

40

50

60

Pr e-

Figure 1: The error curves of the Smooth FR method and Algorithm 2.1

1

1

0.9

0.9

0.8

0.8

0.7

0.7 0.6

χS(τ)

χS(τ)

0.6 0.5 0.4

0.4

0.3

0.3 0.2

0.1

al

0.2

0

Smoothing FR Algorithm 2.1

0

0.2

0.4

0.6

0.8

1 τ

1.2

1.4

1.6

urn

0.9 0.8 0.7

χS(τ)

0.6 0.5 0.4 0.3 0.2

Jo

0.1

0

0.2

0.4

0.6

0.8

τ

1.2

1.4

Smoothing FR Algorithm 2.1 0

0.5

1

1.5

τ

2

Figure 3: Performance profiles on the Nf 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2

Smoothing FR Algorithm 2.1

1

0

χS(τ)

1

0.1

1.8

Figure 2: Performance profiles on the Niter

0

0.5

1.6

Figure 4: Performance profiles on the Ng

0.1 0

Smoothing FR Algorithm 2.1 0

0.2

0.4

0.6

τ

0.8

1

1.2

Figure 5: Performance profiles on CPU time

Journal Pre-proof

11 Table 2: The numerical results of the problem 4.2 by the smoothing FR method and Algorithm 2.1.

Algorithm 2.1 Niter/Nf/Ng/Time 22/168/32/0.015 14/101/22/0.013 30/245/42/0.016 15/115/23/0.014 17/125/25/0.015 28/231/42/0.016 24/188/34/0.015 26/191/36/0.016 25/189/35/0.016 22/173/32/0.015

x∗ (1.0034,0.9917) (1.0057,0.9952) (1.0017,0.9958) (1.0064,0.9939) (1.0063,0.9969) (1.0018,0.9977) (1.0032,0.9950) (1.0035,0.9976) (1.0033,0.9968) (1.0033,0.9969)

of

(-1,-2) (1,-2) (1,-1) (1,0.5) (1.4,-0.7) (1.5,-1) (1.5,-0.5) (2,-2) (3.1,-2.9) (3.1,-1.9)

Smoothing FR method Niter/Nf/Ng/Time x∗ 24/169/34/0.016 (1.0033,0.9918) 18/137/30/0.015 (1.0019,0.9956) 23/158/33/0.016 (1.0036,0.9915) 26/194/34/0.016 (1.0065,0.9840) 50/361/62/0.018 (1.0018,0.9957) 43/316/55/0.018 (1.0017,0.9958) 23/157/33/0.015 (1.0033,0.9919) 22/162/34/0.015 (1.0017,0.9958) 21/140/31/0.015 (1.0034,0.9918) 21/147/31/0.015 (1.0034,0.9958)

p ro

x0

Table 3: The numerical results of the problem 4.3 by the smoothing FR method and Algorithm 2.1.

al

(-1.4,1.4) (0,0.5) (0.1,-0.5) (0.1,0.5) (0.5,-0.5) (1,-2) (1,-0.3) (1.4,3) (2,-2) (2,2)

Smoothing FR method Niter/Nf/Ng/Time x∗ 16/99/28/0.014 (0.0107,-0.0043) 13/77/23/0.014 (0.0000,-0.0085) 12/73/22/0.014 (0.0043,-0.0085) 16/95/28/0.013 (0.0100,-0.0042) 20/122/32/0.014 (0.0099,-0.0042) 17/96/27/0.014 (0.0001,-0.0085) 21/111/31/0.014 (-0.0104,-0.0083) 12/78/20/0.014 (-0.0337,-0.0166) 21/126/33/0.014 (0.0033,-0.0042) 11/56/19/0.013 (-0.0065,-0.0167)

Algorithm 2.1 Niter/Nf/Ng/Time 13/65/21/0.012 16/131/30/0.015 16/122/30/0.015 13/83/23/0.012 16/99/26/0.013 17/97/27/0.014 13/70/21/0.012 20/137/32/0.015 19/130/31/0.014 16/95/26/0.014

Pr e-

x0

x∗ (0.0041,-0.0106) (0.0000,-0.0021) (0.0048,-0.0021) (0.0068,-0.0084) (0.0036,-0.0085) (-0.0070,-0.0084) (-0.0066,-0.0065) (-0.0051,-0.0043) (0.0076,-0.0042) (0.0021,-0.0085)

Jo

urn

[2] E. Polak, On the mathematical foundations of nondifferentiable optimization, SIAM Rev., 29(1987) 21-89. [3] E. Polak, J.E. Higgins, D.Q. Mayne, A barrier function method for minimax problems, Math. Program., 64(1994) 277-294. [4] A. Pot, S. Bhulai, G. Koole, A simple staffing method for multiskill call centers, Manuf. Ser. Oper. Manage., 10(2008) 421-428. [5] R.T. Rockafellar, Computational schemes for large-scale problems in extended linear-quadratic programming, Math. Program., 48(1990) 447-474. [6] Z.F. Dai, H. Zhu, Forecasting stock market returns by combining sum-of-the-parts and ensemble empirical mode decomposition, Applied Economice, https://doi.org/10.1080/00036846.2019.1688244, 2019. [7] Z.F. Dai, H. Zhu, Stock return predictability from a mixed model perspective, Pac-Basin. Finac. J., doi:10.1016/j.pacfin.2020.101267, 2020. [8] V.F. Demyanov, V.N. Molozemov, Introduction to Minimax, Wiley, New York, 1974. [9] D.Z. Du, P.M. Pardalos, Minimax and Applications, Kluwer Academic Publishers: Dordrecht, 1995. [10] C. Charalambous, A.R. Conn, An efficient method to solve the minimax problem directly, SIAM J. Numer. Anal., 15(1978) 162-187. [11] W. Murray, M.L. Overton, A projected Lagrangian algorithm for non-linear minimax optimization, SIAM J. Sci. Comput., 1(1980) 345-370. [12] A. Vardi, New minimax algorithm, J. Optim. Theory Appl., 75(1992) 613-634. [13] S. Xu, Smoothing method for minimax problems, Comput. Optim. Appl., 20(2001) 267-279. [14] L. Luksan, C. Matonoha, J. Vicek, Primal interior-point method for large sparse minimax optimization, Technical Report V-941, Prague, ICS AS CR, 2005.

Journal Pre-proof

12 Table 4: The numerical results of the problem 4.4 by the smoothing FR method and Algorithm 2.1.

Algorithm 2.1 Niter/Nf/Ng/Time 11/95/23/0.014 9/63/15/0.012 15/107/25/0.013 9/55/15/0.012 10/74/18/0.013 12/79/20/0.014 16/128/24/0.015 13/92/23/0.013 16/105/24/0.014 12/87/22/0.013

x∗ (0.0000,-3.0001) (0.0000,-3.0003) (0.0000,-3.0002) (0.0002,-3.0010) (-0.0000,-2.9993) (-0.0000,-3.0001) (-0.0000,-3.0002) (-0.0000,-3.0000) (-0.0001,-3.0001) (0.0000,-2.9999)

of

(0.5,-3) (1,-2) (1,-1) (1,1) (1,2) (1.3,-1) (1.5,-3) (1.5,-1) (2,-1) (3,-3)

Smoothing FR method Niter/Nf/Ng/Time x∗ 8/55/16/0.013 (0.0000,-3.0002) 13/85/23/0.014 (0.0000,-2.9998) 17/108/23/0.014 (-0.0001,-3.0032) 11/72/19/0.014 (0.0000,-3.0000) 21/151/31/0.014 (-0.0000,-3.0000) 47/351/53/0.017 (-0.0000,-3.0000) 14/96/24/0.014 (0.0000,-2.9999) 25/163/33/0.015 (0.0000,-2.9999) 40/290/52/0.016 (0.0000,-2.9999) 17/112/27/0.015 (0.0000,-2.9999)

p ro

x0

Table 5: The numerical results of the problem 4.5 by the smoothing FR method and Algorithm 2.1.

al

(-1.5,1) (-1,-1) (-1,1) (-1,2) (1,0.2) (1,0.7) (1,2) (1,2.3) (1,3) (2,2)

Smoothing FR method Niter/Nf/Ng/Time x∗ 9/43/19/0.013 (0.7118,0.7206) 7/75/15/0.014 (0.7405,0.7405) 23/113/37/0.014 (0.7128,0.7062) 17/99/31/0.014 (0.7096,0.7094) 14/90/30/0.014 (0.7088,0.7078) 18/95/32/0.014 (0.7122,0.7069) 16/82/28/0.013 (0.7143,0.7096) 15/95/31/0.014 (0.7077,0.7089) 15/94/31/0.014 (0.7079,0.7087) 7/42/19/0.013 (0.7118,0.7118)

Algorithm 2.1 Niter/Nf/Ng/Time 13/74/25/0.015 9/59/21/0.015 11/59/21/0.012 17/90/29/0.014 12/74/24/0.013 15/88/27/0.014 15/84/27/0.014 11/75/25/0.012 14/81/26/0.013 10/67/22/0.014

Pr e-

x0

x∗ (0.7107,0.7089) (0.7119,0.7119) (0.7099,0.7068) (0.7106,0.7101) (0.7053,0.7084) (0.7045,0.7092) (0.7028,0.7089) (0.7051,0.7089) (0.7050,0.7107) (0.7109,0.7109)

Jo

urn

[15] J.X. Li, J.Z. Huo, Inexact smoothing method for large scale minimax optimization, Appl. Math. Comput., 218(2011) 2750-2760. [16] E. Polak, J.O. Royset, R.S. Womersley, Algorithms with adaptive smoothing for minimax problems, J. Optim. Theory Appl., 119(2003) 459-484. [17] B.W. Kort, D.P. Bertsekas, A new penalty function method for constrained minimization, Proc. IEEE Conf. on Decision and Control, New Orleans, Dec. 1972. [18] D.P. Bertsekas, On penalty and multiplier methods for constrained minimization, SIAM J. Control Optim., 14(1976) 216-235. [19] D.P. Bertsckas, Approximation procedures based on the method of multipliers, J. Optim. Theory Appl., 23(1977) 487-510. [20] Z.H. Lin, H.D. Qi, A non-interior homotopy system for generalized nonlinear complementarity problem, Preprint of the state key laboratory of scientific and engineering computing, Academic Sinica, Beijing, China, 1998. [21] J.M. Peng, Z. Lin, A non-interior continuation method for generalized linear complementarity problems, Math. Program., 86(1999) 533-563. [22] M. Rivaie, M. Mamat, L.W. June, I. Mohd, A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 218(2012) 11323-11332. [23] J.K. Liu, S. J. Li, New three-term conjugate gradient method with guaranteed global convergence, Int. J. Comput. Math., 91(2014) 1744-1754. [24] W.W. hager, H.Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16(2005) 170-192. [25] D.Y. Pang, S.Q. Du, J.J. Ju, The smoothing Fletcher-Reeves conjugate gradient method for solving finite minimax

Journal Pre-proof

13 Table 6: The numerical results of the problem 4.6 by the smoothing FR method and Algorithm 2.1

x∗ (-0.0468,1.0054,1.9948,-0.9993) (0.0029,0.9965,1.9893,-0.9832) (-0.0016,0.9963,1.9987,-0.9959) (0.0006,0.9953,1.9945,-0.9942) (-0.0009,0.9996,1.9990,-0.9983) (0.0001,1.0035,1.9956,-0.9980) (-0.0013,0.9828,1.9972,-0.9936) (0.0025,0.9948,1.9931,-0.9962) (-0.0015,0.9917,2.0004,-0.9950) (-0.0048,1.0009,2.0015,-0.9947) (-0.0010,1.0078,1.9953,-0.9974) (-0.0017,0.9985,2.0010,-0.9993) (-0.0019,1.0011,1.9983,-0.9979) (0.0002,0.9978,1.9979,-0.9987) (0.0008,0.9997,1.9992,-0.9995) (-0.0009,0.9996,2.0016,-0.9977) (-0.0001,0.9980,2.0012,-0.9964) (-0.0022,1.0009,1.9979,-0.9957) (-0.0012,0.9992,2.0003,-0.9933) (-0.0013,1.0003,2.0010,-0.9959)

of

Niter/Nf/Ng/Time 204/3089/206/0.050 74/959/76/0.026 45/477/51/0.022 32/325/36/0.020 22/276/30/0.019 48/513/54/0.021 58/675/62/0.024 21/205/25/0.018 32/352/38/0.020 32/336/40/0.020 55/586/61/0.020 45/473/51/0.019 32/329/36/0.017 43/439/47/0.022 8/105/12/0.014 47/487/53/0.020 35/345/39/0.018 40/406/44/0.025 57/554/61/0.028 51/509/55/0.026

Pr e-

Algorithm 2.1

x0 (-1,-2,-2,1) (-1,1,1,-1) (-1,2,1,-2) (-1,2,1,2) (0,1,2,-1) (0.2,1.1,2.2,-0.2) (0.28,1.6,1.79,-0.2) (0.8,1.7,1.4,-0.5) (1,1,1,1) (2,1,1,2) (-1,-2,-2,1) (-1,1,1,-1) (-1,2,1,-2) (-1,2,1,2) (0,1,2,-1) (0.2,1.1,2.2,-0.2) (0.28,1.6,1.79,-0.2) (0.8,1.7,1.4,-0.5) (1,1,1,1) (2,1,1,2)

p ro

method Smoothing FR

Jo

urn

al

problems, ScienceAsia, 42(2016) 40-45. [26] X. Chen, Smoothing methods for nonsmooth, nonconvex minimization, Math. Program., 134(2012) 71-99. [27] M.M. M¨akel¨a, Nonsmooth optimization, Ph.D. Thesis, University of Jyv¨askyl¨a(Jyv¨askyl¨a, 1990). [28] J.W. Bandler, C. Charalambous, Nonlinear programming using minimax techniques, J. Optim. Theory Appl., 13(1974) 607-619. [29] E.D. Dolan, J.J. Mor´e, Benchmarking optimization software with performance profiles, Math. Program., 91(2002) 201-213. [30] N. Gould, J.Scott, A note on performance profiles for benchmarking software, ACM T. Math. Sofware 43(2016) 15:1-15:5.