Fixed point iterations coupled with relaxation factors and inertial effects

Nonlinear Analysis 72 (2010) 720–733

Contents lists available at ScienceDirect

Nonlinear Analysis journal homepage: www.elsevier.com/locate/na

Fixed point iterations coupled with relaxation factors and inertial effects Paul-Emile Maingé GRIMAAG, Université des Antilles-Guyane, Département Scientifique Interfacultaire, Campus de Schoelcher, 97233 Cedex, Martinique

article

info

Article history: Received 2 September 2008 Accepted 6 July 2009 MSC: 47N10 47H10 65K05

abstract This paper deals with a general fixed point method which unifies relaxation factors and a two step inertial type extrapolation. These strategies are intended to improve the convergence of many existing algorithms. A convergence theorem, which improves the known ones, is established in this new setting. © 2009 Elsevier Ltd. All rights reserved.

Keywords: Fixed point problem Convex minimization Heavy ball with friction Dynamical system Inertial extrapolation Convex feasibility

1. Introduction Throughout this paper, H stands for a real Hilbert space with inner product and induced norm denoted by h., .i and |.|, respectively. For any mapping T : H → H , we set Fix(T ) := {x ∈ H ; Tx = x} as the fixed point set of T . In recent years, fixed point methods became a topic of great interest because of their successful applications in various areas of engineering sciences such as signal processing and image reconstruction. A main observation that can be done is that various concrete algorithms arising in optimization theory enter iterations of the T general form xn+1 = Tn xn , where (Tn )n≥0 is a sequence of self-mappings on H associated with some solution set S = n≥0 Fix(Tn ). Related examples can be found in [1–3] with regard to convex feasibility problems, subgradient projection techniques, common fixed points problems and monotone inclusions. Note also that much attention has been paid to the development of fixed point methods regarding important aspects such as types of involved operators, robustness, efficiency and other convergence properties. Specifically, an inertial proximal algorithm was obtained in [4] (also see [5–8]), by implicit discretization of the so-called ‘‘heavy ball with friction’’ second-order (in time) dynamical system. This inertial type extrapolation which incorporates second-order information was coupled with relaxation factors to achieve faster convergence (see [9]), and also extended to other fixed point problems (see [10,3]). Afterward, more efficient inertial type extrapolations, based upon the work of [11], were developed in [12,13] again in the context of proximal algorithms. In this paper we investigate an (two step) iterative formalism which extends that of [12] to the more general setting of fixed point problems and also includes all the previously mentioned processes. The proposed algorithm consists of the sequences (xn ) and (yn ) generated as follows: (Initialization) (x0 , y0 ) ∈ H × H ,

E-mail addresses: [email protected], [email protected]. 0362-546X/$ – see front matter © 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.na.2009.07.013

(1a)

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

721

(Step 1) xn+1 = [(1 − wn )I + wn Tn ](xn − θn (α xn + γ yn )),

(1b)

(Step 2) yn+1 = yn − (1/k)[α xn + γ yn + να(xn+1 − xn )],

(1c)

where I : H → H is the identity mapping on H , (θn ) ⊂ [0, +∞) can be viewed as a damping parameter, (wn ) ⊂ (0, +∞) is a relaxation factor, k ∈ (0, +∞), γ ∈ (0, 2k), α and ν belong to (−∞, +∞), and (Tn )n≥0 are self-mappings on H with an associated solution set S=

\

Fix(Tn ) 6= ∅.

(2)

n≥0

First of all, before we enumerate specific examples regarding the above formalism, we fix some notations and we recall some concepts of common use in fixed point theory, through the following two remarks. Remark 1.1. Let us introduce the well-known classes of mappings:

• EFN denotes the set of firmly nonexpansive mappings, EFN = {T : H → H ; ∀x, y ∈ H , |Tx − Ty|2 ≤ |x − y|2 − |(x − y) − (Tx − Ty)|2 },

• EN is the set of nonexpansive mappings, EN = {T : H → H ; ∀x, y ∈ H , |Tx − Ty| ≤ |x − y|},

• EFQ is the set of firmly quasi-nonexpansive mappings, EFQ = {T : H → H ; ∀(x, q) ∈ H × Fix(T ), |Tx − q|2 ≤ |x − q|2 − |x − Tx|2 },

• EQ stands for the set of quasi-nonexpansive mappings, EQ = {T : H → H ; ∀(x, q) ∈ H × Fix(T ), |Tx − q| ≤ |x − q|}. Clearly, we observe that EFN ⊂ EN and EFQ ⊂ EQ , while EQ also contains every operator in EN with nonempty fixed point set. Moreover, as classical results, EFN includes all the resolvents and projection operators, while EFQ contains many subgradient projection operators (see [1,2,14]). Remark 1.2. Let us recall that a multi-valued mapping A : H → 2H is called monotone on H if, for all (x, y) ∈ H × H , it holds that hu − v, x − yi ≥ 0, whenever u ∈ Ax, v ∈ Ay. It is then said to be maximal monotone if graph(A) := {(x, u) ∈ H × H ; u ∈ Ax} is not properly contained in the graph of any other monotone operator. Another classical result is that if A : H → 2H is maximal monotone, x ∈ H and λ > 0, then there exists a unique z ∈ H such that x ∈ (I + λA)z. The singlevalued operator JλA := (I + λA)−1 , so-called the resolvent of A of parameter λ, is firmly nonexpansive and Fix(JλA ) = A−1 (0), where A−1 (0) = {x ∈ H ; 0 ∈ Ax} is the set of zeroes of A (see [15]). As an example, the Fenchel subdifferential ∂ f of a proper convex lower semicontinuous function f : H → R ∪ {+∞} is maximal monotone and (∂ f )−1 (0) is the set of minimizers of f over H . Now, to motivate our study, we point out the fact that Algorithm (1) covers many existing algorithms of relaxed and/or inertial type which are linked to several continuous dynamical systems: (1) For θn = 0, Step 1 in (1) reduces to Krasnoselskii–Mann iterations (see [2,16–19,14]), namely xn+1 = (1 − wn )xn + wn Tn xn .

(3)

Algorithm (3) was considered in [14] in the case when the solution set S coincides with each Fix(Tn ), either with (Tn ) ⊂ EN and (wn ) ⊂ (0, 1), or (Tn ) ⊂ EFN and (wn ) ⊂ (0, 2). In particular, when Tn = JλAn (where (λn ) ⊂ (0, ∞)), the solution set is written as S = A−1 (0), while (3) reduces to the relaxed proximal point algorithm xn+1 = (1 − wn )xn + wn JλAn xn ,

(4)

which was discussed in [18] for (wn ) ⊂ (0, 2). This numerical scheme, specifically in the overrelaxation range (wn ) ⊂ (1, 2), was intended to accelerate the convergence of the standard proximal point algorithm (see [20]) xn+1 = JλAn xn ,

(5)

which is obviously the special case of (4) when wn = 1. Let us precise that (5) can be regarded as an implicit discretization of the first-order steepest descent method x0 (t ) + A(x(t )) 3 0.

(6)

It can also be checked that the relaxed process (4) comes naturally from a discrete version of (6) when replacing A by its Yosida approximation Aλn := (1/λn )(I − JλAn ) which have the same set of zeroes as A (see [15]). (2) The following remark can be made regarding (1) in the special range of parameters k = γ,

α 6= 0 and ν = 1 + 1/α.

(7)

722

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

Remark 1.3. If (7) holds, then the sequence (xn )n≥0 given by (1) satisfies the following relaxed and inertial algorithm (for n ≥ 1) xn+1 = [(1 − wn )I + wn Tn ](xn + θn (xn − xn−1 )).

(8)

Indeed, by (7), we obviously have α(1 − ν) = −1. Hence, from k = γ (given by (7)), we can see that Step 2 in (1) can be equivalently rewritten as yn = −(α/γ )(xn−1 + ν(xn − xn−1 )) (for n ≥ 1), so that α xn + γ yn = −(xn − xn−1 ). Then Step 1 in (1) reduces (8). Iteration (8) was investigated in the range (θn ) ⊂ [0, 1) by several authors, e.g.: (i1) for fixed point problems in [3], either with (Tn ) ⊂ EN and (wn ) ⊂ (0, 1), or (Tn ) ⊂ EFN and (wn ) ⊂ (0, 2); (i2) for resolvents Tn = JλAn with (wn ) ⊂ (0, 2) and an additional error tolerance strategy in [9]; (i3) for resolvents Tn = JλAn and wn = 1 in [5,6]. As mentioned in these works, we recall that the inertial proximal algorithm initiated in [4] was inspired by an implicit discrete version of the second-order heavy ball with friction dissipative dynamical system (which incorporates a viscous damping term)



x00 (t ) + γd x(t ) + ∇ f (x(t )) = 0, x(0) = x0 , x0 (0) = y0 ,

(9)

where ∇ f is the gradient of a convex differentiable function f : H → R, γd is a positive damping parameter and (x0 , y0 ) ∈ H 2 . The underlying idea is that, contrarily to (6) with A = ∇ f , the ‘‘heavy ball with friction’’ dynamical system is no more a descent method, but it is some global energy that is decreasing, which confers to these dynamical systems better convergence properties. Numerical simulations are performed in [6], showing possible improvements in the speed of convergence, comparing with some first-order-in-time methods. (3) Finally, the two step inertial proximal method proposed in [12] corresponds to the special case of (1) when wn = 1, Tn = JλAn and k = 1, namely



xn+1 = JλAn (xn − θn (α xn + γ yn )), yn+1 = (1 − γ )yn − α(ν xn+1 + (1 − ν)xn ),

(10)

where (λn ) ⊂ (0, ∞), (θn ) ⊂ [0, 1), and α , γ , ν are real values. This algorithm was inspired by an implicit discretization of the first-order (overdamped) dissipative dynamical system (discussed in [11]) x0 (t ) + βd ∇ f (x(t )) + ax(t ) + by(t ) = 0, y0 (t ) + ax(t ) + by(t ) = 0, x(0) = x0 , y(0) = y0 ,

(

(11)

where βd , a, b are real values and (x0 , y0 ) ∈ H 2 . Indeed, in [11], system (11) was shown to be equivalent (in some sense) to the second-order overdamped system (which incorporates an additional geometric damping by the Hessian of f ) x00 (t ) + αd x0 (t ) + βd ∇ 2 f (x(t ))x(t ) + ∇ f (x(t )) = 0,

(12)

where αd and βd are real parameters. This latter equation can be regarded as an combination of the ‘‘heavy ball with friction’’ system (9) and the following continuous Newton method (see [21])

∇ 2 f (x(t ))x0 (t ) + ∇ f (x(t )) = 0.

(13)

It turns out that system (11), with no occurrence of the Hessian of f , inherits most of the advantages of (9) and (13) and corrects their drawbacks (see [11,12] for more details and also see [22,23] for other related works). It is also worth noting that Step 2 in (1) has a very low computational coast comparing with Step 1. The present work establishes the asymptotic convergence, for the weak topology, of the formalism (1) and (2) under conditions on the operators (Tn ) and the involved parameters. As a consequence, results related to many existing algorithms are established in a more general setting which allows possible improvements in convergence properties.

2. Main convergence results To establish our convergence results in a general framework, we introduce two conditions related to the operators (Tn ) involved in (1): (c1) First, we assume that (Tn ) ⊂ Fη for some positive value η, where Fη is the set of mappings defined by

Fη := T : H → H | hx − Tx, x − qi ≥ η|x − Tx|2 , ∀(x, q) ∈ H × Fix(T ) .





(14)

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

723

Remark 2.1. As special cases, F1 contains the set of firmly nonexpansive mappings (having fixed points), such as resolvents and projection operators, while F1/2 includes the wide class of quasi-nonexpansive operators. These two facts can be easily deduced from the following classical equality:

∀ u, v ∈ H ,

hu, vi = −(1/2)|u − v|2 + (1/2)|u|2 + (1/2)|v|2 .

(15)

Note also that the set Fη is growing with respect to 1/η, so that F1 ⊂ F1/2 . (c2) Secondly, we suppose that the sequence (Tn ) satisfies the following property: For any subsequence (Tnj ) of (Tn ), for (ξnj ) ⊂ H , for ξ ∈ H ,

(ξnj ) → ξ weakly and ξnj − Tnj ξnj → 0 strongly ⇒ ξ ∈

\

(16)

Fix(Tn ).

n ≥0

Remark 2.2. Note that the condition (16) can be regarded as sort of demiclosedness of the sequence (Tn ) which reduces to the classical demiclosedness property when Tn = T is a constant sequence (see [24]): For any sequence (zk ) ⊂ H and z ∈ H ,

(17)

zk * z weakly, (I − T )(zk ) → 0 strongly ⇒ z ∈ Fix(T ).

This latter condition is well-known to be satisfied by any nonexpansive operator with nonempty fixed point set and by many quasi-nonexpansive operators (see [1,3,25]). It is worth noticing that the more general condition (16) was also used in [10, 3], with applications to several examples of common fixed point problems and monotone inclusions. At once we claim the main convergence result of this section. Theorem 2.1. Assume that (Tn ) ⊂ Fη , for some η > 0, S := ∩n≥0 Fix(Tn ) 6= ∅, (Tn ) satisfies (16), and α , γ , ν , k and (wn ) are real values such that k > 0,

γ ∈ (0, 2k),

(wn ) ⊂ [wa , wb ],

(18)

where 0 < wa ≤ wb < 2η.

(19)

Suppose in addition that (θn ) is a nondecreasing sequence in (0, θ], where θ is any positive value verifying 1

> max{l1 , l2 },

1−σ

1 − ρ(wb )

,

(20)

   k k   + , l = α ν − 1 γ γ    2 [(γ − k)[2αδ(γ ν − k) + 1] − γ ρ(wb )]2 1   l2 = + α(γ ν − k) αδ(γ ν − k) + , γ ρ(wb ) 4γ σ δ(2k − γ ) 2

(21)

θ

with ρ(wb ) =

2η−wb

wb

and

θ

≥

2δ(2k − γ )

and l1 , l2 being defined by

where σ and δ are any constants such that

σ ∈ (0, 1) if ρ(wb ) 6= 1, σ ∈ (0, 1] otherwise, and δ ≥ 1/(2k).

(22)

Then the sequences (xn ) and (yn ) generated by scheme (1) satisfy the following properties: (i1) limn→+∞ |xn+1 − xn | = limn→+∞ |yn+1 − yn | = limn→+∞ |α xn + γ yn | = 0. (i2) There exists x in S such that (xn ) weakly converges to x as n → +∞. Remark 2.3. It is an easy matter to see that Theorem 2.1 remains valid when replacing condition (19) by 0 < wa ≤ wb < 2η,

where wa := lim inf wn n→+∞

and wb := lim sup wn .

(23)

n→+∞

Remark 2.4. Let us emphasize that the proof of Theorem 2.1 will be mainly based upon the existence of some decreasing energy-like sequence related to (1). Specifically, given any element q in the solution set S, it will be established the decreasing property of (Gn ) defined by Gn = kan + ρn bn + 2δ k2 cn ,

(24)



where an := hq − xn , α xn + γ yn i, bn := (1/2)|xn − q|2 , cn := (1/2)|α xn + γ yn |2 and ρn := α k + γ ( θ1

n

 − αν) .

724

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

Before proceeding with the proof of Theorem 2.1, we state some key results regarding the quantities an and cn involved in (24). Remark 2.5. It is easily observed that Step 2 in (1) can be written as

α xn + γ yn = −k(yn+1 − yn ) − αν(xn+1 − xn ),

(25)

or equivalently (for k 6= 0) 1 αν yn+1 − yn = − (α xn + γ yn ) − (xn+1 − xn ). k k

(26)

Some useful estimates are given by the following two preliminary lemmas. Lemma 2.1. Assume that (Tn ) ⊂ Fη for some η > 0 and the parameters k, α , γ , ν , (wn ) and (θn ) in (1) are real values such that k > 0, γ > 0, (θn ) ⊂ [0, +∞), (wn ) ⊂ (0, wb ], Then, for any q ∈ S :=

T

n ≥0

where wb ∈ (0, 2η).

Fix(Tn ) 6= ∅ and for vn := xn − θn (α xn + γ yn ), it holds that

|xn+1 − q|2 ≤ |vn − q|2 − wn (2η − wn )|Tn vn − vn |2 .

(27)

If in addition θn 6= 0, then the following estimate is reached



γ

 − αν hxn+1 − q, xn+1 − xn i k θn   1 ρ(wb ) θn (1 − ρ(wb )) +γ − |xn+1 − xn |2 + γ |α xn + γ yn |2 2θn k 2θn k 2k   γ ρ(wb ) + −1−γ hα xn + γ yn , xn+1 − xn i,

an+1 − an ≤ − α +



1

k

k

where an := hq − xn , α xn + γ yn i and ρ(wb ) =

2η−wb

wb

(28)

.

Proof. Taking q ∈ S and using the definition of (an ), we have an+1 = hq − xn+1 , α xn+1 + γ yn+1 i

= hq − xn+1 , α(xn+1 − xn ) + γ (yn+1 − yn )i + hq − xn+1 , α xn + γ yn i, so that an+1 − an = αhq − xn+1 , xn+1 − xn i + γ hq − xn+1 , yn+1 − yn i + hxn − xn+1 , α xn + γ yn i, that is an+1 − an = −αhxn+1 − q, xn+1 − xn i + γ hq − xn , yn+1 − yn i

− γ hxn+1 − xn , yn+1 − yn i + hxn − xn+1 , α xn + γ yn i.

(29)

Furthermore, by (26) we have 1

να

k

k

hxn+1 − xn , yn+1 − yn i = − hxn+1 − xn , α xn + γ yn i −

|xn+1 − xn |2 .

(30)

Consequently, from (29) and (30), we obtain an+1 − an = −αhxn+1 − q, xn+1 − xn i + γ hq − xn , yn+1 − yn i

+

γ

k

hxn+1 − xn , α xn + γ yn i + γ

να k

|xn+1 − xn |2 + hxn − xn+1 , α xn + γ yn i,

that is an+1 − an = −αhxn+1 − q, xn+1 − xn i + γ hq − xn , yn+1 − yn i

+

γ

k

 να − 1 hxn+1 − xn , α xn + γ yn i + γ |xn+1 − xn |2 . k

(31)

Let us estimate the second term in the right-hand side of the above inequality. To this end, we assume that (Tn ) ⊂ Fη and we recall that vn = xn − θn (α xn + γ yn ). Then Step 1 in (1) can be written as xn+1 = (1 − wn )vn + wn Tn vn ,

(32)

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

725

hence, from a simple computation, we obtain

|xn+1 − q|2 = |vn − q − wn (vn − Tn vn )|2 = |vn − q|2 − 2wn hvn − q, vn − Tn vn i + wn2 |Tn vn − vn |2 .

(33)

Since q ∈ Fix(Tn ) and Tn ∈ Fη , by (14) we also have

hvn − q, vn − Tn vn i ≥ η|Tn vn − vn |2 , which combined with (33) yields (27). Using (32) again, we additionally have Tn vn − vn = (1/wn )(xn+1 − vn ), hence, by (27), we get

|xn+1 − q|2 ≤ |vn − q|2 −

2η − wn

wn 2η−u u

Note that the mapping ρ : u 7→ which by (34) leads to

|xn+1 − vn |2 .

(34)

is decreasing on (0, 2η), hence, by 0 < wn ≤ wb < 2η, we have ρ(wn ) ≥ ρ(wb ),

|xn+1 − q|2 ≤ |vn − q|2 − ρ(wb )|xn+1 − vn |2 .

(35)

Moreover, by an easy computation, we obtain

|vn − q|2 = |(xn − q) − θn (α xn + γ yn )|2 = |xn − q|2 − 2θn hxn − q, α xn + γ yn i + θn2 |α xn + γ yn |2 , which by (35) amounts to

|xn+1 − q|2 ≤ |xn − q|2 + 2θn hq − xn , α xn + γ yn i + θn2 |α xn + γ yn |2 − ρ(wb )|xn+1 − vn |2 .

(36)

As a classical result, by (15) we also have

hxn+1 − q, xn+1 − xn i = −(1/2)|xn − q|2 + (1/2)|xn+1 − q|2 + (1/2)|xn+1 − xn |2 ,

(37)

|xn+1 − q|2 − |xn − q|2 = 2hxn+1 − q, xn+1 − xn i − |xn+1 − xn |2 ,

(38)

that is

hence, (36) can be rewritten as 2hxn+1 − q, xn+1 − xn i ≤ 2θn hq − xn , α xn + γ yn i + |xn+1 − xn |2 + θn2 |α xn + γ yn |2 − ρ(wb )|xn+1 − vn |2 .

(39)

Furthermore, by (25), we have

hq − xn , α xn + γ yn i = hq − xn , −k(yn+1 − yn ) − αν(xn+1 − xn )i = khxn − q, yn+1 − yn i + ανhxn − q, xn+1 − xn i = khxn − q, yn+1 − yn i + ανhxn+1 − q, xn+1 − xn i − αν|xn+1 − xn |2 . It is then immediately seen that (39) is equivalently written as

hxn+1 − q, xn+1 − xn i ≤ θn khxn − q, yn+1 − yn i + ανθn hxn+1 − q, xn+1 − xn i − θn αν|xn+1 − xn |2 + (1/2)|xn+1 − xn |2 + (1/2)θn2 |α xn + γ yn |2 − (1/2)ρ(wb )|xn+1 − vn |2 , that is

θn khq − xn , yn+1 − yn i ≤ (ανθn − 1)hxn+1 − q, xn+1 − xn i + (1/2 − θn αν) |xn+1 − xn |2 + (1/2)θn2 |α xn + γ yn |2 − (1/2)ρ(wb )|xn+1 − vn |2 , or equivalently

hq − xn , yn+1 − yn i ≤

1 k

+



αν −

θn 2k

1

θn



hxn+1 − q, xn+1 − xn i +

|α xn + γ yn |2 −

1/2 − θn αν

θn k

|xn+1 − xn |2

ρ(wb ) |xn+1 − vn |2 . 2θn k

In addition, we clearly have

|xn+1 − vn |2 = |(xn+1 − xn ) + θn (α xn + γ yn )|2 = |xn+1 − xn |2 + 2θn hα xn + γ yn , xn+1 − xn i + θn2 |α xn + γ yn |2 ,

(40)

726

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

which together with (40) yields

    1/2 − θn αν 1 ρ(wb ) hq − xn , yn+1 − yn i ≤ hxn+1 − q, xn+1 − xn i + αν − − |xn+1 − xn |2 k θn θn k 2θn k θn ρ(wb ) + hα xn + γ yn , xn+1 − xn i. (1 − ρ(wb )) |α xn + γ yn |2 − 1

2k

k

Then, in light of (31), we obtain an+1 − an ≤ −αhxn+1 − q, xn+1 − xn i + 1/2 − θn αν

γ



k

αν −

1



θn

hxn+1 − q, xn+1 − xn i

ρ(wb ) |xn+1 − xn |2 2θn k θn ρ(wb ) +γ hα xn + γ yn , xn+1 − xn i (1 − ρ(wb )) |α xn + γ yn |2 − γ k   2k να γ − 1 hxn+1 − xn , α xn + γ yn i + γ |xn+1 − xn |2 , + +γ



θn k



−

k

k

namely

  ρ(wb ) 1 hxn+1 − q, xn+1 − xn i + γ − |xn+1 − xn |2 k θn 2θn k 2θn k   θn γ ρ(wb ) +γ −1−γ hα xn + γ yn , xn+1 − xn i, (1 − ρ(wb )) |α xn + γ yn |2 + 

an+1 − an ≤ − α +

γ



1

− αν



2k

k

that is exactly the desired result.

k



Lemma 2.2. If (wn ) ⊂ (0, +∞) and the other parameters α , γ , k, (θn ) are chosen as in Lemma 2.1, then, for any q ∈ S := T Fix (Tn ) 6= ∅, it holds that n≥0 cn+1 − cn =

α

(γ ν − k) (γ − k)hxn+1 − xn , α xn + γ yn i

k2

+

α2 2k2

(νγ − k)2 |xn+1 − xn |2 + γ

γ 2

 1 − k 2 |α xn + γ yn |2 , k

where cn = (1/2)|α xn + γ yn | . 2

Proof. Clearly, using the definition of (cn ), we have cn+1 − cn =

1 2

1

|α xn+1 + γ yn+1 |2 − |α xn + γ yn |2 2

  yn+1 + yn xn+1 + xn +γ . = α(xn+1 − xn ) + γ (yn+1 − yn ), α 2

2

(41)

Moreover, it is easily checked that

α xn + γ yn = α

xn+1 + xn 2

+γ

yn+1 + yn 2

−

α 2

(xn+1 − xn ) −

γ 2

(yn+1 − yn ),

which in light of (25) yields

α

xn+1 + xn 2

+γ

yn+1 + yn 2

α γ = −k(yn+1 − yn ) − αν(xn+1 − xn ) + (xn+1 − xn ) + (yn+1 − yn ) 2 γ  α 2 = − k (yn+1 − yn ) + − αν (xn+1 − xn ). 2

2

Consequently, by (41), we deduce that

D

cn+1 − cn = α(xn+1 − xn ) + γ (yn+1 − yn ),

γ 2

 α  E − k (yn+1 − yn ) + − αν (xn+1 − xn ) , 2

that is cn+1 − cn =

 γ  α  α −k +γ − αν hxn+1 − xn , yn+1 − yn i 2 2 α  γ  +α − αν |xn+1 − xn |2 + γ − k |yn+1 − yn |2 , 2

2

(42)

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

727

or equivalently cn+1 − cn = α (γ (1 − ν) − k) hxn+1 − xn , yn+1 − yn i + α 2



1 2

 γ  − ν |xn+1 − xn |2 + γ − k |yn+1 − yn |2 . 2

(43)

Hence, by (26), we obtain cn+1 − cn = −(α/k) (γ (1 − ν) − k) hxn+1 − xn , (α xn + γ yn ) + αν(xn+1 − xn )i

+ α2



1 2

 γ  1 − ν |xn+1 − xn |2 + γ − k 2 |(α xn + γ yn ) + (αν)(xn+1 − xn )|2 , 2

k

namely cn+1 − cn = −(α/k) (γ (1 − ν) − k) hxn+1 − xn , α xn + γ yn i

− (α ν/k) (γ (1 − ν) − k) |xn+1 − xn | + α 2

2

2



1 2



− ν |xn+1 − xn |2

 1 γ   να 2 |xn+1 − xn |2 − k 2 |α xn + γ yn |2 + γ −k 2 k 2 k  2να γ −k hxn+1 − xn , α xn + γ yn i. +γ 2 +γ

γ

2

k

This latter equality can be equivalently rewritten as cn+1 − cn = δ1 hxn+1 − xn , α xn + γ yn i + α 2 δ2 |xn+1 − xn |2 + γ

γ 2

 1 − k 2 |α xn + γ yn |2 , k

(44)

where the coefficients δ1 and δ2 are defined by

 2να −k , k 2 k2    ν 2 γ 1 ν −k δ2 = − (γ (1 − ν) − k) + − ν + γ . δ1 = −

α

(γ (1 − ν) − k) + γ

k

γ

2

2

k

Then, by an easy computation, we get

    2ν  γ α ν 2 α γ −k − γ (1 − ν) + k = γ − (ν + 1)γ + k = 2 (γ ν − k) (γ − k), k 2 k k k k   2 ν2 1 ν2 ν2 ν2 ν 1 1 ν ν γ + + 2γ2 − γ = 2γ2 − γ + = γ −1 , δ2 = − + δ1 =

α

k

k

2

2k

which by (44) leads to the desired result.

k

2k

k

2

2

k



To prove our main result, we also need the following well-known Opial lemma. Lemma 2.3 (See [26]). Let H be a Hilbert space and (zn ) a sequence in H such that there exists a nonempty set Ω ⊂ H verifying: (i) For every q ∈ Ω , limn→+∞ |zn − q| exists. (ii) Any weak cluster-point of (zn ) belongs to Ω . Then, there exists x¯ ∈ Ω such that (zn ) weakly converges to x¯ as n → +∞. Now we are in a position to prove our convergence theorem. Proof of Theorem 2.1. Let q be any element in S and consider the discrete energy-like sequence (Gn ) introduced in (24), namely Gn = kan + ρn bn + 2δ k2 cn ,

(45)

  where an := hq − xn , α xn +γ yn i, bn := (1/2)|xn − q|2 , cn := (1/2)|α xn +γ yn |2 and ρn := α k + γ ( θ1 − αν) . It is obviously n observed that an easy computation yields Gn = k(an + bn + cn ) + (ρn − k)bn + k(2δ k − 1)cn , so that a more interesting formulation of Gn is given by Gn =

k 2

1

k

2

2

|(q − xn ) − (α xn + γ yn )|2 + (ρn − k)|xn − q|2 + (2δ k − 1)|α xn + γ yn |2 .

(46)

728

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

In what follows, we first establish some inequality showing the decreasing property of (Gn ). Indeed, using Lemma 2.1, we obviously have



+γ



1

− αν hxn+1 − q, xn+1 − xn i θn  ρ(wb ) 1 − |xn+1 − xn |2 + (1/2)γ θn (1 − ρ(wb ))|α xn + γ yn |2 2θn 2θn

k(an+1 − an ) ≤ − α k + γ





+ (γ − k − γ ρ(wb )) hα xn + γ yn .xn+1 − xn i.

(47)

Note also that by (15) we have

hxn+1 − q, xn+1 − xn i = bn+1 − bn + (1/2)|xn+1 − xn |2 . As a result, we deduce that

     1 1 1 ρ(wb ) − αk + γ − αν +γ − |xn+1 − xn |2 2 θn 2θn 2θn    1 − αk + γ − αν (−bn + bn+1 ) + (1/2)γ (1 − ρ(wb ))θn |α xn + γ yn |2 θn + (γ − k − γ ρ(wb )) hα xn + γ yn , xn+1 − xn i.   Then, recalling that ρn = α k + γ ( θ1 − αν) , we equivalently obtain n k(an+1 − an ) ≤



k(an+1 − an ) + ρn (bn+1 − bn ) ≤

 −

α 2

 ρ(wb ) |xn+1 − xn |2 + (1/2)γ (1 − ρ(wb ))θn |α xn + γ yn |2 2θn

(k − γ ν) − γ

+ (γ − k − γ ρ(wb )) hα xn + γ yn , xn+1 − xn i.

(48)

Furthermore, from cn = (1/2)|α xn + γ yn | and using Lemma 2.2, we have 2

2k2 (cn+1 − cn ) = 2α (γ ν − k) (γ − k)hxn+1 − xn , α xn + γ yn i

+ α 2 (νγ − k)2 |xn+1 − xn |2 + γ (γ − 2k)|α xn + γ yn |2 .

(49)

Consequently, taking any values δ ∈ (0, +∞) and using (48) and (49), we deduce that k(an+1 − an ) + ρn (bn+1 − bn ) + 2δ k2 (cn+1 − cn ) ≤

+

γ 2





δα 2 (νγ − k)2 −

(1 − ρ(wb ))θn − γ δ(2k − γ ) |α xn + γ yn |

α 2

(k − γ ν) − γ

 ρ(wb ) |xn+1 − xn |2 2θn

2

+ [(γ − k)[2αδ(γ ν − k) + 1] − γ ρ(wb )]hα xn + γ yn , xn+1 − xn i, hence, for σ ∈ (0, 1], we equivalently obtain k(an+1 − an ) + ρn (bn+1 − bn ) + 2δ k2 (cn+1 − cn )

≤ d1,n |xn+1 − xn |2 − (e1,n + d2 )|α xn + γ yn |2 + d3 hα xn + γ yn , xn+1 − xn i,

(50)

where the coefficients d1,n , e1,n , d2 and d3 are defined by d1,n = δα 2 (νγ − k)2 − e1,n = γ



α 2

(k − γ ν) − γ

ρ(wb ) , 2θn

 1 − (1 − ρ(wb ))θn + δ(1 − σ )(2k − γ ) , 2

d2 = γ σ δ(2k − γ ), d3 = (γ − k)[2αδ(γ ν − k) + 1] − γ ρ(wb ). Assuming that (θn ) is a nondecreasing sequence in (0, θ] (for some positive θ ), we easily observe that (ρn ) is nonincreasing, so that by (45) and (50) we obtain Gn+1 − Gn + e1,n |α xn + γ yn |2 ≤ d1,n |xn+1 − xn |2 − d2 |α xn + γ yn |2 + d3 hα xn + γ yn , xn+1 − xn i.

(51)

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

729

Moreover, as d2 > 0 (for γ ∈ (0, 2k)), we observe that (51) can be rewritten as Gn+1 − Gn + e1,n |α xn + γ yn | ≤ −d2 2



|α xn + γ yn | − 2

d3 d2

 hα xn + γ yn , xn+1 − xn i + d1,n |xn+1 − xn |2

2  2  d3 d3 (xn+1 − xn ) + + d1,n |xn+1 − xn |2 , = −d2 (α xn + γ yn ) − 2d2

4d2

hence, setting

 d4,n = −

d23 4d2



+ d1,n ,

(52)

we immediately obtain



Gn+1 − Gn + e1,n |α xn + γ yn |2 + d4,n |xn+1 − xn |2 + d2 (α xn + γ yn ) −

d3 2d2

2 (xn+1 − xn ) ≤ 0.

(53)

Let us choose σ such that σ ∈ (0, 1) if ρ(wb ) 6= 1 and σ ∈ (0, 1] otherwise. It is then easily observed that (e1,n ) is nonnegative, provided that 1−σ

θ

≥

1 − ρ(wb ) 2δ(2k − γ )

.

(54)

Now, set e2 = δα 2 (νγ − k)2 − α2 (k − γ ν) and e3 = γ

ρ(wb ) 2

n

Hence, (d4,n ) is nonnegative and bounded away from zero if (1/θ ) > (1/e3 ) 1

θ

> l2 =

2

γ ρ(wb )



d2

e

e

. It is clear that d1,n = e2 − θ3 , so that we get d4,n = − 4d3 −e2 + θ3 .



d23 4d2



2

n

+ e2 , namely if

  [(γ − k)[2αδ(γ ν − k) + 1] − γ ρ(wb )]2 1 + α(γ ν − k) αδ(γ ν − k) + . 4γ σ δ(2k − γ ) 2

(55)

Consequently, by (53), there exist two positive constants τ1 and τ2 such that Gn+1 − Gn + τ1 |(α xn + γ yn ) −

d3 2d2

(xn+1 − xn )|2 + τ2 |xn+1 − xn |2 ≤ 0,

(56)

which shows that the sequence (Gn ) is nonincreasing. Next, from this latter result, we derive convergent properties regarding the sequence (xn ) and (yn ). Let δ ≥ 1/(2k) and assume that θ verifies

  k k > l1 = α ν − + . θ γ γ 1

(57)

It is easily checked that (57) entails that (ρn ) ⊂ (k, +∞) and (ρn ) bounded away from k. Then, we deduce by (46) that (Gn ) is nonnegative, hence (Gn ) is convergent and bounded (as it is nonincreasing). Therefore, in light of (56), we immediately obtain

X

|xn+1 − xn |2 < ∞ and

n ≥0

X

|(α xn + γ yn ) −

n≥0

which (by the Cauchy–Schwarz inequality) also gives us

d3 2d2

P

(xn+1 − xn )|2 < ∞,

n ≥0

|α xn + γ yn |2 < ∞, so that

lim |xn+1 − xn | = lim |α xn + γ yn | = 0.

n→+∞

n→+∞

(58)

Moreover, observing that (θn ) is convergent, because it is nondecreasing and θn ∈ (0, θ], we deduce that (ρn − k) converges to some positive real number. As a result, by (46), we can see that (|xn − q|) is a converging sequence. Furthermore, as (xn ) is bounded, we know that its weak cluster-point set is nonempty. Now, take any weak cluster-point p of (xn ) and let (xnj ) be a subsequence of (xn ) such that (xnj ) converges weakly to p as j → ∞. Also consider some q ∈ S. Then, by vn = xn − θn (α xn + γ yn ), using the fact that (|xn − q|) is convergent, and by (58), we deduce that limn→+∞ |vn − q| = limn→+∞ |xn+1 − q|. Consequently, by (27), and observing that the quantity wn (2λ − wn ) is bounded away from zero, we conclude that limn→+∞ |Tn vn − vn | = 0, while it is obvious that (vnj ) converges weakly to p. Then, by condition (16), we get p ∈ S. Applying the Opial lemma (Lemma 2.3), we obtain the weak convergence of the whole sequence (xn ).  Let us make a remark concerning the case of (1) when θn ≡ 0, which was omitted in Theorem 2.1.

730

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

Remark 2.6. Note that if θn = 0 then Step 1 in (1) reduces to xn+1 = [(1 − wn )I + wn Tn ](xn ).

(59)

It is then a classical matter to that if (Tn ) ⊂ Fη satisfies (16) and (wn ) satisfies (19), then the following properties are T reached: (i1) limn→+∞ |xn+1 − xn | = 0; (i2) (xn ) weakly converges in H to some element x¯ ∈ S := n≥0 Fix(Tn ). Indeed, in this situation, for any p ∈ S, and using inequality P (27), we can see that the quantity (|xn − q|) is nonincreasing (hence it is convergent and (xn ) is bounded) together with n wn (2η − wn )|Tn xn − xn |2 < ∞. It is easily deduced that this latter result and condition (19) yields limn→+∞ |Tn xn − xn | = 0, which by (59) amounts to |xn+1 − xn | = wn |Tn xn − xn | → 0. Property (i2) can be obtained by following the same lines as in the last part of the Proof of Theorem 2.1. 3. Some specific cases In this section, we specialize Theorem 2.1 to some situations related to previous works by Alvarez–Attouch [5], Alvarez [9], Maingé [10] and Maingé–Moudafi [12]. (A) We first pay some attention to inertial type iterations of the form (see [5,9,10]) zn+1 = [(1 − wn )I + wn Tn ][zn + θn (zn − zn−1 )],

for n ≥ 1,

(60)

where (wn )n≥1 ⊂ (0, +∞), (θn )n≥1 ⊂ [0, +∞), (z0 , z1 ) ∈ H , and (Tn ) is a sequence of mappings from H into itself such that S := ∩n≥0 Fix(Tn ) 6= ∅. Let us precise the link between algorithms (60) and (1) . 2

Remark 3.1. It is interesting to see that (zn ) given by (60) is nothing but the sequence (xn ) generated by (1) with involved parameters verifying

γ > 0,

k = γ,

α 6= 0,

ν = 1 + 1/α,

while starting from the initial data x0 := z0 and y0 := (1/γ )

(61)





− α z0 and taking w0 := 0 and θ0 6= 0. Indeed, in light of Remark 1.3, we know that (xn ) satisfies the same recursion formula as in (60). It is then immediate that (zn ) coincides with (xn ) provided that z1 = x1 (as it is assumed that z0 = x0 ). Note that, by (1) with n = 0, we also have x1 = z0 − θ0 (α z0 + γ y0 ). As a result, the considered choice of y0 (given above) leads to z1 = x1 , so that (xn ) = (zn ). Then, in the range of parameters (61), Theorem 2.1 (in light of Remark 2.3) establishes the weak asymptotic convergence of (zn ) (which is the same as for (xn ), no matter what the two limit points are) towards an element of S. z0 −z1

θ0

As a consequence of Theorem 2.1 (also see Remark 2.6) we obtain the following result. Corollary 3.1. Assume that (Tn ) ⊂ Fη , for some η > 0, S := ∩n≥0 Fix(Tn ) 6= ∅, (Tn ) satisfies (16) and (wn ) is such that

(wn ) ⊂ [wa , wb ], Set ρ(wb ) =

2η−wb

wb

where 0 < wa ≤ wb < 2η.

(62)

and suppose in addition that (θn ) is a nondecreasing sequence in [0, θ], where θ is any positive value verifying

1/θ > max{2, l2 (u, σ ), l3 (u, σ )}, with some σ ∈ (0, 1) and u ≥ 1/2, if ρ(wb ) 6= 1, (63) θ < 1/3, otherwise,   [ρ(wb )]2 1−ρ(w ) 2 where l2 (u, σ ) := ρ(w + u + 12 and l3 (u, σ ) := 2u(1−σb) . Then the sequence (zn ) generated by scheme (60) satisfies ) 4 σ u b



the following properties: (i1) limn→+∞ |zn+1 − zn | = 0; (i2) there exists z in S such that (zn ) weakly converges to z as n → +∞. Proof. The case θn ≡ 0 is deduced from Remark 2.6. Now let (θn ) ⊂ (0, +∞). By Remark 3.1 we know that Theorem 2.1, with parameters such as in (61), remains valid when replacing (xn ) (in its conclusion) by (zn ) given by (60). Let us emphasize that (61) yields α(ν − 1) = 1. It is then a simple matter to check that condition (20) in Theorem 2.1 becomes θ1 > max{l1 , l2 }





[ρ(wb )]2 2 b) and 1−σ ≥ 1−ρ(w , with l1 and l2 reduced to l1 = 2 and l2 = ρ(w + (δγ ) + 12 , where δ is any constant verifying θ 2(δγ ) 4σ (δγ ) b) δγ ≥ 1/2, while σ is any constant such that σ ∈ (0, 1) if ρ(wb ) 6= 1, σ ∈ (0, 1] otherwise. Consequently, condition (20), together with (61), can be rewritten as

1/θ > max{2, l2 (u, σ ), l3 (u, σ )}, with σ ∈ (0, 1) and u ≥ 1/2, if ρ(wb ) 6= 1, 1/θ > max{2, l2 (u)}, with σ ∈ (0, 1] and u ≥ 1/2, otherwise,



(64)

where l2 (u, σ ) and l3 (u, σ ) are defined above. It is not also difficult to check that, when ρ(wb ) = 1, condition (64) is optimal for (u, σ ) = (1/2, 1). Indeed, a simple calculation leads to minu≥1/2,σ ∈(0,1] l2 (u, σ ) = l2 (1/2, 1) = 3. Therefore condition (64) holds whenever so does (63), which leads to the desired result. 

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

731

Remark 3.2. Corollary 3.1 provides, for convergence of (60), alternative conditions to that given in [3] which requires that P 2 n θn |zn − zn−1 | < ∞. Remark 3.3. Let us mention that most of the works dealing with algorithms of the form (1) or (60) involve sequence of resolvents, namely Tn := JλAn , with A : H → 2H maximal monotone and (λn ) bounded away from zero. Theorem 2.1 and

Corollary 3.1 can then be applied with η = 1 (see Remark 1.2) and S := ∩n≥0 Fix(JλAn ) = A−1 (0). Moreover, it can be easily checked that condition (16) is satisfied, since the graph of such an operator A is weakly-strongly closed (see [15]). In particular, we recall that the method considered by Alvarez–Attouch [5] corresponds to the special instance of (60) with wn = 1 and Tn := JλAn . In this framework, by applying Corollary 3.1 with wa = wb = 1 (hence ρ(wb ) = 1) and η = 1, we retrieve the Alvarez–Attouch result (see [5], Proposition 2.1): Corollary 3.2. Let A : H → 2H be a maximal monotone operator, with A−1 (0) 6= ∅, (λn ) ⊂ (0, +∞) such that lim infn λn > 0, and suppose that (θn ) is a nondecreasing sequence in [0, θ], where θ is any value in (0, 1/3). Then (zn ) given from any (z0 , z1 ) ∈ H 2 by zn+1 = JλAn [zn + θn (zn − zn−1 )] weakly converges in H to some element z in A−1 (0). It is also worth recalling that when considering the method of Alvarez [9] in the absence of error tolerance in computations, it reduces to the special case of (60) when Tn := JλAn . Applying Corollary 3.1 (with η = 1), we obviously recover Alvarez’s result (with regard to exact computations, see [9]). Corollary 3.3. Let A : H → 2H be a maximal monotone operator, and (zn ) the sequence given from any (z0 , z1 ) ∈ H 2 by zn+1 = [(1 − wn )I + wn JλAn ][zn + θn (zn − zn−1 )], where (θn ) ⊂ [0, ∞), (wn ) and (λn ) in (0, +∞) are such that 0 < infn wn ≤ supn wn < 2 and infn λn > 0. There exists a positive constant θc (depending on (wn )) such that if (θn ) is a nondecreasing sequence in [0, θ], with some θ ∈ (0, θc ), then the sequence (zn ) weakly converges in H to some element z in A−1 (0). Let us recall that the special instance of (60) when wn = 1 is nothing but the method discussed by Maingé [10]. From Corollary 3.1, we recover the Maingé convergence result (see [10]). Corollary 3.4. Let (Tn ) ⊂ Fη (with η > 1/2) be a sequence of demiclosed self-mappings on H with S := and set

  1−η θc = 1/ 3 + 8 2 4η − 1

if 1/2 < η < 1 and θc = 1/3 otherwise (i.e, η ≥ 1).

T

n≥0

Fix(Tn ) 6= ∅

(65)

Suppose in addition that (θn ) is a nondecreasing sequence in [0, θ], where θ is any value in (0, θc ). Then the sequence (zn ) generated from any (z0 , z1 ) ∈ H 2 by zn+1 = Tn [zn + θn (zn − zn−1 )] weakly converges in H to some element z in S. Proof. In this situation where wn = 1, we can apply Corollary 3.1 with wa = wb = 1 (hence η > 1/2 and ρ(wb ) = 2η − 1). Let us observe that, for η = 1 (i.e., ρ(wb ) = 1), Corollary 3.4 holds with θc = 1/3. Let η 6= 1 (hence ρ(wb ) 6= 1). It follows from Corollary 3.1 that the convergence result of Corollary 3.4 holds whenever θ is any positive value satisfying 1/θ > max{2, l2 (u, σ ), l3 (u, σ )}



for some u ≥ 1/2 and σ ∈ (0, 1),



)]2

(66)

2 b l2 (u, σ ) := ρ(w + u + 12 and l3 (u, σ ) := u(1−σ ) . We divide the rest of the proof into two parts: 4σ u b) (a) Assume that η > 1. It is also easily checked that the mappings l2 (., σ ) (with any fixed value σ ∈ (0, 1)) attains its ρ(w ) minimum on [0, +∞) at um = 2√σb . Note also that um ≥ 1/2 (as ρ(wb ) > 1 and σ ∈ (0, 1)), hence we deduce that

[ρ(w

1−η

1 minu≥1/2 l2 (u, σ ) = l2 (um , σ ) = √2σ + ρ(w , and it can be noticed that l2 (um , σ ) ≤ √2σ + 1 (because ρ(wb ) > 1). Moreover, b) it is obvious that l3 (um , σ ) < 0 (as η > 1). As a result, we deduce that condition (66) holds if





2 2 1/θ > √ + 1 = max 2, √ + 1

σ

σ



for some σ ∈ (0, 1),

which is true if 1/θ > 3. Thus Corollary 3.4 holds for θc = 1/3 (when η > 1).

(67)

732

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

(a) Assume that 1/2 < η < 1 and set σ1 = l2 (u1 , σ1 ) := 3 + 8

1−η 4η2 − 1

1+2η 3

(hence σ1 ∈ (0, 1)) and u1 = 1/2. A simple calculation leads to

and l3 (u1 , σ1 ) := 3.

(68)

Hence, noticing that l2 (u1 , σ1 ) ≥ l3 (u1 , σ1 ) ≥ 2, we deduce that condition (66) is satisfied if 1/θ > l2 (u1 , σ1 ), which shows that Corollary 3.4 holds for θc = (l2 (u1 , σ1 ))−1 (when η ∈ (1/2, 1))  Remark 3.4. Note that the value of θc obtained in [10] (Theorem 3.3) is as follows: θc = (2η − 1)/(2η + 1) if 1/2 < η < 1 and θc = 1/3 otherwise (i.e., η ≥ 1). Regarding the value l2 (u1 , σ1 ) defined in (68), by η ≥ 1/2 we have l2 (u1 , σ1 ) ≤ 3 + 4(1 − η)/(2η − 1) = (2η + 1)/(2η − 1). It is then clear that Corollary 3.4 improves Theorem 3.3 in [10] with regard to the parameter θc . (B) As a straightforward consequence of Theorem 2.1 when wn = 1, k = 1 and Tn := JλAn (hence wa = wb = 1, ρ(wb ) = 1 and η = 1), we recover the Maingé–Moudafi convergence result (see [12]). Corollary 3.5. Let A : H → 2H be a maximal monotone operator, with A−1 (0) 6= ∅, (xn ) and (yn ) the sequences given from any (x0 , y0 ) ∈ H 2 by



xn+1 = JλAn (xn − θn (α xn + γ yn )), yn+1 = (1 − γ )yn − α(ν xn+1 + (1 − ν)xn ),

(69)

where α and ν are any real values, γ ∈ (0, 2), (θn ) ⊂ [0, +∞), and (λn ) ⊂ (0, ∞) is such that infn λn > 0. There exists a positive constant θc such that if (θn ) is a nondecreasing sequence in [0, θ], with θ ∈ [0, θc ), then the sequences (xn ) and (yn ) satisfy the following properties: (p1) limn→+∞ |xn+1 − xn | = limn→+∞ |yn+1 − yn | = 0; (p2) (xn ) weakly converges in H to some element z in A−1 (0). Remark 3.5. Let us also mention that all the applications given in [3] can be also formulated in the context of (1) by particular choices of operators (Tn ). A main example is given through the following corollary, regarding the computation of common fixed points of countable families of mappings. Corollary 3.6. Let (Rj )j≥0 ⊂ Fη (for η > 0) be a sequence of demiclosed self-mappings on H such that


−1 P n

T

j ≥0

Fix(Rj ) 6= ∅ and

set (∀n ≥ 0)Tn = j =0 γ j j=0 γj Rj , where (γj )j≥0 ⊂ (0, +∞) and j≥0 γj < ∞. Suppose that α , γ , ν , k and (wn ) are real values satisfying (18) and (19) and (θn ) is a nondecreasing sequence in (0, θ], where θ is any positive value verifying (20). Then (xn ) and (yn ) generated by (1) satisfy: (i1) limn→+∞ |xn+1 − Txn | = limn→+∞ |yn+1 − yn | = limn→+∞ |α xn + γ yn | = 0. (i2) There exists x ∈ j≥0 Fix(Rj ) such that (xn ) weakly converges to x as n → +∞.

Pn

P


−1 Pn wn,j Rj with wn,j = γj , so that (wn,j )nj=0 ⊂ (0, 1] and i=0 γi Tn is essentially due to the fact j=0 wn,j = 1, hence it can be easily checked that Fix(Tn ) = j=0 Fix(Rj ) (this last equality T T that Tn is a convex combination of (Rj )0≤j≤n with each Rj in Fη ). This easily amounts to n≥0 Fix(Tn ) = j≥0 Fix(Rj ). Let us Pn prove that (Tn ) ⊂ Fη . Taking any qn ∈ Fix(Tn ) and x ∈ H , we have hx − Tn x, x − qn i = j=0 wn,j hx − Rj x, x − qn i, hence, by (Rj ) ⊂ Fη and observing that qn belongs to each Fix(Rj ) (for j = 0, . . . , n), we obtain Proof. Clearly, each Tn can be rewritten as Tn =

Pn

Pn

hx − Tn x, x − qn i ≥ η

n X

j =0

wn,j |x − Rj x|2 ,

(70)

j =0

while we obviously have

2 !2 n n X X |x − Tn x| = w (x − Rj x) ≤ wn,j |x − Rj x| , j = 0 n ,j j =0 2

which by Young’s inequality entails 2

|x − Tn x| ≤

n X j =0

! wn,j

n X j =0

! 2

|x − Rj x|

=

n X

|x − Rj x|2 ,

j=0

then, together with (70), we deduce that hx − Tn x, x − qn i ≥ η|x − Tn x|2 , so that (Tn ) ⊂ Fη . Now we prove that (Tn )n≥0 satisfies condition (16). Let (ξni ) ⊂ H be a sequence such T that (ξni ) converges weakly to some element ξ in H (as i → +∞) and limi→+∞ |ξni − Tni ξni | = 0. Taking any q in S := n≥0 Fix(Tn ) (hence q belongs to each Fix(Tn )) and using (70), we have

P.-E. Maingé / Nonlinear Analysis 72 (2010) 720–733

733

Pni

wni ,j |ξni − Rj ξni |2 . Consequently, by the boundedness of (ξni ) (as it is weakly convergent),
−1 P > 0, we deduce that, for by the fact that limi→+∞ |ξni − Tni ξni | = 0, and observing that wni ,j ≥ γj l ≥0 γ l T all j ≥ 0, limi→+∞ |ξni − Rj ξni | = 0, which by demiclosedness of each Rj entails ξ ∈ Fix(Rj ). It is then immediate that ξ ∈ j≥0 Fix(Rj ), hence ξ ∈ S. Applying Theorem 2.1 we therefore reach the desired result.  hξni − Tni ξni , ξni − qi ≥ η

j=0

Concluding remarks: This work provides a unified frame, combining relaxation factors and a two step inertial type extrapolation, for the improvement of many concrete algorithms. The proposed method is intended to be used as a speeding up process for other existing fixed point methods, by a suitable control of the involved parameters. Then it would be interesting in future research to exhibit explicit ranges of these parameters, to ensure the efficiency of the method. References [1] H.H. Bauschke, P.L. Combettes, A weak-to-strong convergence principle for Fejer monotone methods in Hilbert space, Math. Oper. Res. 26 (2001) 248–264. [2] C.L. Byrne, A unified treatment of some iterative algorithms in signal processing and image reconstruction, Inverse Problems 18 (2004) 441–453. [3] P.E. Maingé, Convergence theorems for inertial KM-type algorithms, J. Comput. Appl. Math. 219 (1) (2008) 223–236. [4] F. Alvarez, On the minimizing property of a second order dissipative dynamical system in Hilbert spaces, SIAM J. Control Optim. 39 (2000) 1102–1119. [5] F. Alvarez, H. Attouch, An inertial proximal method for monotone operators via discretization of a nonlinear oscillator with damping, Set-Valued Anal. 9 (2001) 3–11. [6] F. Jules, P.E. Maingé, Numerical approaches to a stationary solution of a second order dissipative dynamical system, Optimization 51 (2) (2002) 235–255. [7] A. Moudafi, E. Elisabeth, An approximate inertial proximal method using enlargement of a maximal monotone operator, Int. J. Pure Appl. Math. 5 (3) (2003) 283–299. [8] A. Moudafi, M. Oliny, Convergence of a splitting inertial proximal method for monotone operators, J. Comput. Appl. Math. 155 (2003) 447–454. [9] F. Alvarez, Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space, SIAM J. Optim. 14 (3) (2004) 773–782. [10] P.E. Maingé, Inertial iterative process for fixed points of certain quasi-nonexpansive mappings, Set-Valued Anal. 15 (1) (2007) 67–79. [11] F. Alvarez, H. Attouch, J. Bolte, P. Redont, A second-order gradient-like dissipative dynamical system with Hessian driven damping. Application to Optimization and Mechanics, J. Math. Pures Appl., IX Sér. 81 (8) (2002) 747–779. [12] P.E. Maingé, A. Moudafi, A proximal method for maximal monotone operators via discretization of a first order dissipative dynamical system, J. Convex Anal. 14 (4) (2007) 869–878. [13] P.E. Maingé, A. Moudafi, Convergence of new inertial proximal methods for DC programming, SIAM J. Optim. 19 (1) (2008) 397–413. [14] Q. Yang, J. Zhao, Generalized KM theorems and their applications, Inverse Problems 22 (2006) 833–844. [15] H. Brezis, Operateurs Maximaux Monotones et semi-groupes de contractions dans les espaces de hilbert, in: Mathematics Studies, North Holland, 1973. [16] P.L. Combettes, Fejér monotonicity in convex optimization, in: C.A. Floudas, P.M. Pardalos (Eds.), Encyclopedia of Optimization, Kluwer, Boston, MA, 2000. [17] Y. Censor, A. Motova, A. Segal, Perturbed projections and subgradients projections for the multiple-sets split feasibility problem, J. Math. Anal. Appl. 327 (2007) 1244–1256. [18] J. Eckstein, D.P. Bertsekas, On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program. 55 (1992) 293–318. [19] S. Reich, A.J. Zaslavski, Convergence of Krasnoselskii–Mann iterations of nonexpansive operators, Math. Comput. Modelling 32 (2000) 1423–1431. [20] R.T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control. Opt. 14 (5) (1976) 877–898. [21] F. Alvarez, J.M. Pérez, A dynamical system associated with Newton’s method for parametric approximations of convex minimization problems, Appl. Math. Optim. 38 (1998) 193–217. [22] H. Attouch, J. Bolte, P. Redont, Optimizing properties of an inertial dynamical system with geometric damping, Control Cybernet. 31 (2002) 643–657. [23] M. Pellicer, J. Solà-Morales, Optimal decay rates and the selfadjoint property in overdamped systems, J. Differential Equations 246 (2009) 2813–2828. [24] K. Goebel, W.A. Kirk, Topics in Metric Fixed Point Theory, in: Cambridge Studies in Advanced Mathematics, vol. 28, Cambridge University Press, Cambridge, 1990. [25] I. Yamada, N. Ogura, Hybrid steepest descent method for variational inequality problem over the fixed point set of certain quasi-nonexpansive mappings, Numer. Funct. Anal. Optim. 25 (7–8) (2004) 619–655. [26] Z. Opial, Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Amer. Math. Soc. 73 (1967) 591–597.