Rockafellar’s celebrated theorem based on A -maximal monotonicity design

Applied Mathematics Letters 21 (2008) 355–360 www.elsevier.com/locate/aml

Rockafellar’s celebrated theorem based on A-maximal monotonicity design Ram U. Verma University of Central Florida, Orlando, FL 32816, USA Received 4 April 2007; accepted 23 May 2007

Abstract A generalization to Rockafellar’s theorem (1976) in the context of approximating a solution to a general inclusion problem involving a set-valued A-maximal monotone mapping using the proximal point algorithm in a Hilbert space setting is presented. Although there exists a vast literature on this theorem, most of the studies are focused on just relaxing the proximal point algorithm and applying to the inclusion problems. The general framework for A-maximal monotonicity (also referred to as the A-monotonicity framework in literature) generalizes the general theory of set-valued maximal monotone mappings, including the H -maximal monotonicity (also referred to as H -monotonicity). c 2007 Elsevier Ltd. All rights reserved.

Keywords: Inclusion problems; Maximal monotone mapping; A-maximal monotone mapping; Generalized resolvent operator

1. Introduction Let X be a real Hilbert space with the norm k · k and the inner product h., .i. We consider the inclusion problem: find a solution to 0 ∈ M(x),

(1) 2X

where M : X → is a set-valued mapping on X. In [1], Rockafellar investigated the general convergence and rate of convergence for an algorithm (referred to as the proximal point algorithm in the literature) in the context of solving (1) by showing, when M is maximal monotone, that the sequence {x k } generated for an initial point x 0 by x k+1 ≈ Pk (x k )

(2)

converges weakly to a solution to (1), provided the approximation is made sufficiently accurate as the iteration proceeds, where Pk = (I + ck M)−1 for a sequence {ck } of positive real numbers that is bounded away from zero. It follows from (2) that x k+1 is an approximate solution to the inclusion problem 0 ∈ M(x) + ck−1 (x − x k ). E-mail address: [email protected]. c 2007 Elsevier Ltd. All rights reserved. 0893-9659/$ - see front matter doi:10.1016/j.aml.2007.05.004

(3)

356

R.U. Verma / Applied Mathematics Letters 21 (2008) 355–360

As a matter of fact, a general class of problems of variational character, including minimization or maximization of functions, variational inequality problems, and minimax problems, can be unified into the form (1). General maximal monotonicity has been a powerful framework to studying convex programming and variational inequalities. It turned out that one of the fundamental algorithms applied for solving these problems was in fact the proximal point algorithm. Furthermore, Rockafellar [2] applied the proximal point algorithm in convex programming. For more details, we refer the reader to [1–10]. In this work, we first examine some auxiliary results involving A-maximal monotone and generalized firmly nonexpansive mappings. Second, we generalize Rockafellar’s theorem [1] to the case of A-maximal monotone mappings. The results obtained, in turn, generalize a general class of results, including the investigations involving H -maximal monotone mappings. 2. A-Maximal monotonicity and firm nonexpansiveness In this section we discuss some results based on basic properties of A-maximal monotonicity, and then we derive results involving A-maximal monotonicity and the generalized firm nonexpansiveness. Let X denote a real Hilbert space with the norm k · k and inner product h., .i. Let M : X → 2 X be a multivalued mapping on X . We shall denote both the map M and its graph by M, that is, the set {(x, y) : y ∈ M(x)}. This is equivalent to stating that a mapping is any subset M of X × X , and M(x) = {y : (x, y) ∈ M}. The domain of a map M is defined (as its projection onto the first argument) by dom(M) = {x ∈ X : ∃ y ∈ X : (x, y) ∈ M} = {x ∈ X : M(x) 6= ∅}. dom(M) = X will denote the full domain of M, and the range of M is defined by range(M) = {y ∈ X : ∃ x ∈ X : (x, y) ∈ M}. The inverse M −1 of M is {(y, x) : (x, y) ∈ M}. For a real number ρ and a mapping M, let ρ M = {(x, ρy) : (x, y) ∈ M}. If L and M are any mappings, we define L + M = {(x, y + z) : (x, y) ∈ L , (x, z) ∈ M}. Definition 2.1. Let M : X → 2 X be a multivalued mapping on X . The map M is said to be: (i) (r )-strongly monotone if there exists a positive constant r such that hu ∗ − v ∗ , u − vi ≥ r ku − vk2

∀ (u, u ∗ ), (v, v ∗ ) ∈ graph(M).

(ii) (m)-relaxed monotone if there exists a positive constant m such that hu ∗ − v ∗ , u − vi ≥ (−m)ku − vk2

∀ (u, u ∗ ), (v, v ∗ ) ∈ graph(M).

Definition 2.2. Let M : X → 2 X be a mapping on X . The map M is said to be: (i) Nonexpansive if ku ∗ − v ∗ k ≤ ku − vk

∀ (u, u ∗ ), (v, v ∗ ) ∈ graph(M).

(ii) Firmly nonexpansive if ku ∗ − v ∗ k2 ≤ hu ∗ − v ∗ , u − vi

∀ (u, u ∗ ), (v, v ∗ ) ∈ graph(M).

(iii) (c)-firmly nonexpansive if there exists a constant c > 0 such that ku ∗ − v ∗ k2 ≤ chu ∗ − v ∗ , u − vi

∀ (u, u ∗ ), (v, v ∗ ) ∈ graph(M).

Definition 2.3 ([3]). Let A : X → X be (r )-strongly monotone. The map M : X → 2 X is said to be A-maximal monotone (also referred to as A-monotone) if (i) M is (m)-relaxed monotone for m > 0. (ii) R(A + ρ M) = X for ρ > 0.

357

R.U. Verma / Applied Mathematics Letters 21 (2008) 355–360

Example 2.1. Let A : X → X be (r )-strongly monotone on a Hilbert space X for r > 0. Consider a locally Lipschitz continuous functional f : X → R such that ∂ f is (m)-relaxed monotone for m > 0. Then A + ∂ f is maximal monotone, that is, ∂ f is A-maximal monotone for m < r. Definition 2.4. Let A : X → X be an (r )-strongly monotone mapping and let M : X → 2 X be an A-monotone M : X → X is defined by mapping. Then the generalized resolvent operator Jρ,A M Jρ,A (u) = (A + ρ M)−1 (u).

Proposition 2.1 ([3]). Let A : X → X be an (r )-strongly monotone mapping and let M : X → 2 X be an A-maximal monotone mapping. Then (A + ρ M) is maximal monotone for ρ > 0. Proposition 2.2. Let A : X → X be an (r )-strongly monotone mapping and let M : X → 2 X be an A-maximal monotone mapping. Then the operator (A + ρ M)−1 is single valued. 3. Generalization to Rockafellar’s theorem This section deals with a generalization to Rockafellar’s theorem [1, Theorem 1] under the framework of A-maximal monotonicity [3]. Furthermore, some results connecting A-maximal monotonicity and the corresponding generalized resolvent operator are established, which generalize the results on the firm nonexpansiveness and H -maximal monotonicity [5]. Lemma 3.1 ([3]). Let X be a real Hilbert space, let A : X → X be (r )-strongly monotone, and let M : X → 2 X be A-maximal monotone. Then the generalized resolvent operator associated with M and defined by M Jρ,A (u) = (A + ρ M)−1 (u)

∀u ∈ X,

1 is ( r −ρm )-Lipschitz continuous.

Lemma 3.2. Let X be a real Hilbert space, let A : X → X be (r )-strongly monotone, and let M : X → 2 X be A-maximal monotone. Then the generalized resolvent operator associated with M and defined by M Jρ,A (u) = (A + ρ M)−1 (u)

∀ u ∈ X,

satisfies 1 M M hu − v, Jρ,A (u) − Jρ,A (v)i. r − ρm Theorem 3.1. Let X be a real Hilbert space, let A : X → X be (r )-strongly monotone, and let M : X → 2 X be A-maximal monotone. Then the following statements are mutually equivalent: M M kJρ,A (u) − Jρ,A (v)k2 ≤

(i) An element u ∈ X is a solution to (1). (ii) For an u ∈ X , we have M u = Jρ,A (A(u)).

where M Jρ,A (u) = (A + ρ M)−1 (u).

Proof. It follows from the definition of the A-resolvent operator corresponding to M.



Lemma 3.3. Let X be a real Hilbert space, let A : X → X be (r )-strongly monotone and (s)-Lipschitz continuous, and let M : X → 2 X be A-maximal monotone. Then s M M kJρ,A (A(u)) − Jρ,A (A(v))k ≤ ku − vk. (4) r − ρm

358

R.U. Verma / Applied Mathematics Letters 21 (2008) 355–360

Consequently, M M h(I − Jρ,A o A)(u) − (I − Jρ,A o A)(v), u − vi ≥

 1+

s r − ρm

−2  1−

s r − ρm



M M × k(I − Jρ,A o A)(u) − (I − Jρ,A o A)(v)k2 ,

where ρ <

r −s m

(5)

and s < r.

1 M and (s)-Lipschitz continuity of A. Proof. The proof follows from the ( r −ρm )-Lipschitz continuity of Jρ,A



Theorem 3.2. Let X be a real Hilbert space, let A : X → X be (r )-strongly monotone and (s)-Lipschitz continuous, and let M : X → 2 X be A-maximal monotone. For an arbitrarily chosen initial point x 0 , suppose that the sequence {x k } is generated by the proximal point algorithm x k+1 ≈ JρMk ,A (A(x k ))

(6)

such that kx k+1 − JρMk ,A (A(x k ))k ≤ k , ∞  < ∞, and {ρ } is where JρMk ,A = (A + ρk M)−1 , and {k }, {ρk } ⊆ [0, ∞) are scalar sequences with e1 = Σk=0 k k bounded away from zero. Then the following conclusions hold:

(i) The sequence {x k } is bounded. −s (ii) limk→∞ Jk∗ (x k ) = 0, for ρk < r m and s < r. k (iii) The sequence {x } converges weakly to a solution of (1). Proof. Suppose that x ∗ is a zero of M. For all k ≥ 0, we set Jk∗ = I − JρMk ,A o A. Then, in the light of Lemma 3.3, Jk∗ is (1 + r −ρs k m )2 (1 − r −ρs k m )−1 -firmly nonexpansive. Also from Theorem 3.1, it follows that any solution to (1) is a fixed point of JρMk ,A o A, and hence a zero of Jk∗ . Next, we find the estimate kx k+1 − x ∗ k = kx k+1 − JρMk ,A (A(x k )) + JρMk ,A (A(x k )) − x ∗ k ≤ kx k+1 − JρMk ,A (A(x k ))k + kJρMk ,A (A(x k )) − x ∗ k ≤ k + kJρMk ,A (A(x k )) − x ∗ k = kJρMk ,A (A(x k )) − JρMk ,A (A(x ∗ ))k + k   s kx k − x ∗ k + k ≤ r − ρk m k X ≤ kx 0 − x ∗ k + j j=0

≤ kx − x k + e1 . 0

∗

This implies that sequence {x k } is bounded. To establish the weak convergence of the sequence {x k }, we need to examine the estimate kx k+1 − x ∗ k2 = kx k+1 − JρMk ,A (A(x k )) − Jk∗ (x k ) + x k − x ∗ k2 = kx k+1 − JρMk ,A (A(x k )) − Jk∗ (x k )k2 + kx k − x ∗ k2 + 2hx k+1 − JρMk ,A (A(x k )) − Jk∗ (x k ), x k − x ∗ i = kx k+1 − JρMk ,A (A(x k )) − Jk∗ (x k )k2 + kx k − x ∗ k2 + 2hx k+1 − JρMk ,A (A(x k )), x k − x ∗ i − 2hJk∗ (x k ), x k − x ∗ i

R.U. Verma / Applied Mathematics Letters 21 (2008) 355–360

359

≤ kx k+1 − JρMk ,A (A(x k )) − Jk∗ (x k )k2 + kx k − x ∗ k2  −2   s s k+1 M k k ∗ 1− kJk∗ (x k )k2 + 2hx − Jρk ,A (A(x )), x − x i − 2 1 + r − ρk m r − ρk m = kx k+1 − JρMk ,A (A(x k ))k2 + kJk∗ (x k )k2 − 2hx k+1 − JρMk ,A (A(x k )), Jk∗ (x k )i + kx k − x ∗ k2 + 2hx k+1 − JρMk ,A (A(x k )), x k − x ∗ i  −2   s s −2 1 + 1− kJk∗ (x k )k2 r − ρk m r − ρk m  −2   s s 2 ≤ k − 2 1 + 1− kJk∗ (x k )k2 r − ρk m r − ρk m + kJk∗ (x k )k2 − 2hx k+1 − JρMk ,A (A(x k )), Jk∗ (x k )i + kx k − x ∗ k2 + 2hx k+1 − JρMk ,A (A(x k )), x k − x ∗ i  −2   s s k ∗ 2 1− kJk∗ (x k )k2 = kx − x k − 2 1 + r − ρk m r − ρk m  4  −2 s s + 1+ 1− kx k − x ∗ k2 + k2 r − ρk m r − ρk m + 2hx k+1 − JρMk ,A (A(x k )), x k − x ∗ − Jk∗ (x k )i " −2 # 4   s s kx k − x ∗ k2 1− = 1+ 1+ r − ρk m r − ρk  −2   s s −2 1 + 1− kJk∗ (x k )k2 r − ρk m r − ρk m + k2 + 2hx k+1 − JρMk ,A (A(x k )), JρMk ,A (A(x k )) − JρMk ,A (A(x ∗ ))i "  4  −2 # s s ≤ 1+ 1+ 1− kx k − x ∗ k2 r − ρk m r − ρk m  −2     s s s ∗ k 2 2 1− kJk (x )k + k + 2k kx k − x ∗ k, (7) −2 1 + r − ρk m r − ρk m r − ρk m where ρk <

r −s m

and s < r.

Using the summability of the sequence {k }, we have that e2 =

∞ X

k2 < ∞.

k=0

As a result, we get "

kx

k+1

4  −2 # s s −x k ≤ 1+ 1+ 1− kx 0 − x ∗ k2 r − ρk m r − ρk m  −2  X k s s −2 1 + 1− kJ ∗ (x j )k2 + e2 r − ρk m r − ρk m j=0 j   s kx 0 − x ∗ k. + 2e1 r − ρk m ∗ 2



(8)

360

R.U. Verma / Applied Mathematics Letters 21 (2008) 355–360

We infer that k X

kJ j∗ (x j )k2 < ∞ ⇒ lim Jk∗ (x k ) = 0.

j=0

k→∞

It follows that there exists a unique element (u k , v k ) ∈ M represented by A(u k ) + ρk v k = A(x k ) for all k. Since = (JρMk ,A o A)(x k ) and limk→∞ Jk∗ (x k ) = 0, this implies that x k − u k → 0 as k → ∞. It further follows, since the sequence {ρk } is bounded away from zero, that

uk

lim

k→∞

Jk∗ (x k ) = lim v k = 0. k→∞ ρk

Thus, since the sequence {x k } is bounded, it has at least one weak cluster point, say x 0 . Let {x k( j) } be a subsequence of {x k } such that x k( j) converges weakly to x 0 . Since x k − u k → 0, this implies that u k( j) also converges weakly to x 0 . Let some (u, v) ∈ M. Then the A-maximal monotonicity (and hence, (m)-relaxed monotonicity) of M implies that hu − u k , v − v k i ≥ (−m)ku − u k k2

for all k ≥ 0.

Therefore, hu − x 0 , v − 0i ≥ (−m)ku − x 0 k2

for all k ≥ 0.

Since M is (m)-relaxed monotone, and (u, v) is arbitrary, it follows that (x 0 , 0) ∈ M, that is, x 0 is a solution to (1). Finally, it turns out that the weak cluster point of the sequence {x k } is unique under the assumptions of the theorem.  References [1] R.T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim. 14 (1976) 877–898. [2] R.T. Rockafellar, Augmented Lagrangians and applications of the proximal point algorithm in convex programming, Math. Oper. Res. 1 (1976) 97–116. [3] R.U. Verma, A-monotonicity and its role in nonlinear variational inclusions, J. Optim. Theory Appl. 129 (3) (2006) 457–467. [4] R.U. Verma, A-monotone nonlinear relaxed cocoercive variational inclusions, Cent. Eur. J. Math. 5 (2) (2007) 386–396. [5] Y.P. Fang, N.J. Huang, H -monotone operators and system of variational inclusions, Comm. Appl. Nonlinear Anal. 11 (1) (2004) 93–101. [6] Y.P. Fang, N.J. Huang, H.B. Thompson, A new system of variational inclusions with (H, η)-monotone operators, Comput. Math. Appl. 49 (2–3) (2005) 365–374. [7] J. Eckstein, D. Bertsekas, On the Douglas–Rachford splitting and the proximal point algorithm for maximal monotone operators, Math. Program. 55 (1992) 293–318. [8] R. Glowinski, P. Le Tellec, Augmented Lagrangians and Operator-Splitting Methods in Continuum Mechanics, SIAM, Philadelphia, 1989. [9] P. Tossings, The perturbed proximal point algorithm and some of its applications, Appl. Math. Optim. 29 (1994) 125–159. [10] P. Tseng, A modified forward–backward splitting method for maximal monotone mappings, SIAM J. Control Optim. 38 (2000) 431–446.