A Simple Decomposition Alternating Direction Method for Matrix Completion

Available online at www.sciencedirect.com

Procedia Computer Science 17 (2013) 149 – 157

Information Technology and Quantitative Management , ITQM 2013

A simple decomposition alternating direction method for matrix completion Lingfeng Niua,b,∗, Xi Zhaoa,b b Research

a University of Chinese Academy of Sciences, Beijing, China Center on Fictitious Economy and Data Science University of Chinese Academy of Sciences, Beijing, China

Abstract Matrix completion(MC), which is to recover a data matrix from a sampling of its entries, arises in many applications. In this work, we consider find the solutions of the MC problems by solving a series of fixed rank problems. For the fixed rank problems, variables are divided into two parts naturally based on matrix factorization and a simple alternative direction method framework is proposed. For each fixed rank problem, the solving process of each part of variables can be further converted into a series of relative small scale independent linear equations systems. Based on these observations, we design a decomposition alternative direction method for the MC problem. To test the performance of the new method, we implement our method in Matlab(with a few C/Mex functions) and compare it with several state-of-the-art solvers for the MC problem. Preliminary experimental results indeed demonstrate the effectiveness and efficiency of our method.

© 2013 The Authors. Published by Elsevier B.V. Selection and peer-review under responsibility of the organizers of the 2013 International Conference on Information Technology and Quantitative Management Keywords: Matrix Completion; Decomposition; Alternative Direction; Matrix Factorization;

1. Introduction Matrix completion(MC), which is to recover a data matrix from a sampling of its entries, arises in many applications. As a motivating example, consider the task of inferring the users’ preference based on their partially known preferences[1, 2] in the recommender system. A well-known instance of this example is the Netflix problem[3]. Users (rows of the data matrix) are given the opportunity to rate movies (columns of the data matrix). Typically, users rate only very few movies and there are very few scattered observed entries of this data matrix(see the demo in Figure 1). Since the Netflix company wants to provide effective recommendations to the users based on their preferences. They need to infer the user’s preference for unrated items. So they would like to complete this matrix. In general, it is impossible to complete the matrix by only given a subset of entries without some additional information. However, when the matrix and the sample satisfy certain prescribed properties, we may complete the matrix and recover the entries that we have not seen successfully[4, 5, 6, 7, 8, 9, 8, 10, 11]. In this paper, the MC problem considered can be written as the following optimization: min rank(W), s.t.Wi, j = Mi, j , ∀(i, j) ∈ Ω,

W∈m×n

(1)

∗ Corresponding

author. Email addresses: [email protected] (Lingfeng Niu), [email protected] (Xi Zhao)

1877-0509 © 2013 The Authors. Published by Elsevier B.V. Selection and peer-review under responsibility of the organizers of the 2013 International Conference on Information Technology and Quantitative Management doi:10.1016/j.procs.2013.05.021

150

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

Fig. 1. Demo figure for the Netflix prize

where Mi j ∈  are given for (i, j) ∈ Ω ⊂ {(i, j)|1 ≤ i ≤ m, 1 ≤ j ≤ n}, and rank(W) denotes the rank of W. Besides the collaborative filtering example of Netflix problem mentioned above, problem (1) can be found in lots of application scenarios, such as recovering shape and motion from image streams[12, 13], triangulation in the sensor network location[14], model reduction and system identification[15], and various problems in machine learning, data mining and pattern recognitions[16]. Due to the combinational nature of the function rank(·), problem (1) is generally NP-hand. It has been shown that[4, 5], under some reasonable conditions, the solution of problem (1) can be found by solving a convex optimization problem (2) minm×n W∗ , s.t.Wi, j = Mi, j , ∀(i, j) ∈ Ω, W∈

where the nuclear norm W∗ is the summation of singular values of W. We list the following theorem from [5] as an example. More theoretical results can be found in [6, 7, 8, 9, 8, 10, 11]. Theorem 1.1 ([5]). For any matrix M ∈ m×n . Denote N = max{m, n}. If rank(M) < N 0.2 and the number of sampled entries which are selected uniformly at random obeys |Ω| ≥ CN 1.2 rank(M) log N, for some positive constant C. M can be perfectly recovered with high probability by the convex optimization problem (2). Various types of algorithms have been proposed to solve the MC problem (1). Generally speaking, the existing algorithms can be divided into the following three branches. The first branch of algorithms are derived from the dual from of the nuclear norm minimization (2)[17, 8], which can be formulated as a semidefinite programming(SDP). However, the reformulation requires large auxiliary matrix variables, and is expensive to solve by general-purpose solvers for SDPs, such as SDPT3[18, 19], SeDumi[20] an et al. In [17], the authors exploit the structure information in SDP carefully and propose an efficient implementations of interior-point methods. The second branch of algorithms are based on the matrix shrinkage operator and consider solve the convex optimization problem (2) directly. Typical solvers include SVT[21], FPCA[22], APGL[23], and etc. Although the methods in this type possess good global convergence properties, they bear the computational cost required by singular value decomposition(SVD), which becomes increasingly costly as the sizes and ranks of the underlying matrices increase. The third branch of algorithms are based on low-rank matrix factorization[24]. Although the methods in this type lack the theoretical guarantee for the global optimality until now due to the formulation nonconvexity. Because the computations of solving SDPs and SVDs are avoided, experimental studies show that they are suitable for solving large-scale problems in practice. In this work, we propose a new algorithm for the MC

151

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

problem (1), which belongs the third branch. In details, in order to find the solution of the MC problem (1), a series of fixed rank matrix factorization problems are solved. For the fixed rank problems, variables are divided into two parts naturally based on matrix factorization and a simple alternative direction method framework is proposed. For each fixed rank problem, the solving process of each part of variables can be further converted into a series of relative small scale independent linear equations systems. In all, a decomposition alternative direction method for the MC problem is designed. The remaining part of this paper is organized as follows. In Section 2, we first explain our design of the decomposition alternative direction algorithm for the fixed rank MC problem. On this basis, we then propose the new method for solving the MC problem (1) with low-rank matrix factorization. Numerical experiments are carried out in Section 3. To test the effectiveness of the new method, we implement our method in Matlab(with a few C/Mex functions) and compare it with several state-of-the-art solvers for the MC problem. Experimental results are reported at the same time. Finally, we draw the conclusion and state some future works in Section 4. 2. Decomposition Alternating Direction Method for Matrix Completion We start our discussion by considering the following fixed rank MC problem min

Y∈m×r ,Z∈n×r

PΩ (YZ T − M)2F

where, r ∈ {1 · · · , min{m, n}} is a pregiven positive integer, and  Xi j (i, j) ∈ Ω [PΩ (X)]i j = 0 (i, j)  Ω

(3)

(4)

Suppose r¯ is the smallest integer which makes the objective function value of problem (3) equals to zero. Obviously, the corresponding YZ T is the solution of the matrix completion problem (1). Suppose at each iteration the global optimal solution of (3) can be found. We can start to find the low rank factorization from an initial rank estimation r start ≤ rank(M) and terminate the iterations when PΩ (YZ T − M)2F is less then the stop criteria . Summering the above discussions, we get the following basic algorithm framework for solving the MC problem (1). Algorithm 1 The basic algorithm framework for solving the MC problem (1 Require: m, n, Ω and Mi, j ∈  for (i, j) ∈ Ω, r start ≤ rank(M), positive integer Δr and  Ensure: Return Y and Z for r = r start , r start + Δr, · · · , min{m, n} do solve (3) to obtain Y, Z, if PΩ (YZ T − M)2F ≤ M2F then stop end if end for In Algorithm 1, one of the key steps is solving the fixed rank MC problem (3). So we first concentrate on how to solve problem (3) below. Notice that the variables in (3) are divided into two parts Y and Z based on the matrix factorization YZ T . Based on this structural characteristic, we choose to update the value of Y and Z separately in this paper. Then an alternating direction method framework for solving the fixed rank MC problem (3) in this paper obtained, which is described in Algorithm 2. Now we discuss how to implement Algorithm 2 efficiently in practice. We notice that most of the computational cost of Algorithm 2 concentrate on updating the values of Y and Z at each iteration. Therefore, the efficiency of solving problems minY PΩ (YZ (t−1)T − M) and minZ PΩ (Y (t) Z T − M) determines the efficiency of Algorithm 2 directly. When the value of Z is fixed, the objective function of (3) becomes the quadratic function of Y, i.e. φ(Y) = PΩ (YZ T − M)2F =

m n r    [ ( yil z jl − Mi j )2 ]. i=1 j=1,(i j)∈Ω l=1

(5)

152

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

Algorithm 2 Alternating direction method framework for the fixed rank MC problem (1) Require: Ω and Mi, j ∈  for (i, j) ∈ Ω, ¯ , rank r and initial matrix Z (0) ∈ n×r Ensure: Return Y ∈ m×r and Z ∈ n×r for t = 1, 2, · · · do solve minY PΩ (YZ (t−1)T − M) to obtain Y (t) , if Z PΩ (Y (t) Z (t−1)T − M)2F ≤  then Set Y = Y (t) , Z = Z (t−1) and stop. end if solve minZ PΩ (Y (t) Z T − M) to obtain Z (t) , if Y PΩ (Y (t) Z (t)T − M)2F ≤  then Set Y = Y (t) , Z = Z (t) and stop. end if end for Define the following functions for i = 1, · · · , m, n 

ψ(i) (Y) =

(

r 

yil z jl − Mi j )2 .

(6)

j=1,(i j)∈Ω l=1

Then we have that φ(Y) =

m i=1

ψ(i) (Y) and ∀i = 1, · · · , m, r = 1, · · · , r n r   ∂φ(Y) ∂ψ(i) (Y) = =2 ( yil z jl − Mi j )z jq . ∂yiq ∂yiq j=1,(i, j)∈Ω l=1

(7)

The optimal value of matrix Y for the current fixed Z must satisfy the first order optimality conditions ∂φ(Y) =0 ∂yiq i.e.

r n   (

n 

z jl z jq )yil =

l=1 j=1,(i, j)∈Ω

∀i = 1, · · · , m, q = 1, · · · , r,

z jq Mi j ,

∀i = 1, · · · , m, q = 1, · · · , r.

j=1,(i, j)∈Ω

For each i ∈ {1, 2, · · · , m}, denote yi = (yi1 , · · · , yir )T ∈ r , Z¯i ∈ n×r with  z jl (i, j) ∈ Ω ¯ ∀ j = 1, · · · n; ∀i = 1, · · · , r. Zi jl = 0 (i, j)  Ω ¯ i ∈ n with and M

 ¯ ij = M

Mi j 0

(i, j) ∈ Ω ∀ j = 1, · · · n. (i, j)  Ω

(8)

(9)

Then, the above sets of linear equations can be written in a more compact way, ¯ i , ∀i = 1, · · · , m. Z¯iT Z¯i yi = Z¯iT M

(10)

Similarly, the matrix Z can be updated by solving the following linear equations system when the value of matrix Y is fixed: ˆ j , ∀ j = 1, · · · , n, (11) Yˆ Tj Yˆ j z j = Yˆ Tj M where z j = (z j1 , · · · , z jr )T ∈ r , Yˆ j ∈ m×r , for all ∀ j = 1, 2, · · · , n with  yil (i, j) ∈ Ω ˆ ∀i = 1, · · · , m; ∀l = 1, · · · , r. Y jil = 0 (i, j)  Ω

(12)

153

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

ˆ j ∈ m with and M

 ˆ ji = M

Mi j 0

(i, j) ∈ Ω ∀i = 1, · · · , m, j = 1, 2, · · · , n. (i, j)  Ω

(13)

In all, if we choose to update the values of Y and Z in Algorithm 2 by solving (10) and (11), respectively, we can get a decomposition alternating direction method for the fixed rank MC problem (1). More details can be found in Algorithm 3. In this algorithm, at the t-th iteration, for given i = 1, · · · , m, to update the values of of yti , from previous discussion we know the linear equations system in (10) need to be solved. However, in the actual calculation process, if the value of

¯ i 2 Z¯i(t−1) yi − M 2 ¯ i 2 M 2

is small enough, the current yi is obviously a good choice for yti . ¯ 2 Z¯ (t−1) y − M

Therefore, to save the computational resource, for a given small positive value , if i  M¯ i 2 i 2 < , we can skip i 2 solving the current linear equations system in (10). Similar discussions apply to the update of Z as well. Algorithm 3 Decomposition alternating direction method for fixed rank problem (1) Require: Ω and Mi, j ∈  for (i, j) ∈ Ω, , ˆ , r and Z0 ∈ n×r Ensure: Return Y ∈ m×r and Z ∈ n×r for t = 1, 2, · · · do for i = 1, 2, · · · , m do ¯ 2 Z¯ (t−1) y − M if i  M¯ i 2 i 2 >  then i 2

solve (10) to obtain y(t) i , end if end for(t) (t−1)T P (Y Z −M)2F P (Y (t−1) Z (t−1)T −M)2F −PΩ (Y (t) Z (t−1)T −M)2F ≤  or t > 1& Ω ≤ ˆ then if Ω M2 P (Y (t) Z (t−1)T −M)2 Ω

F

Set Y = Y (t) , Z = Z (t−1) and stop end if for j = 1, 2, · · · , n do if

ˆ 2 Yˆ (t) j z j − M j 2 2 ˆ M j  2

F

>  then

solve (11) to obtain z(t) j , end if end for(t) (t)T P (Y Z −M)2F P (Y (t) Z (t−1)T −M)2F −PΩ (Y (t) Z (t)T −M)2F ≤  or t > 1& Ω ≤ ˆ then if Ω M2 P (Y (t) Z (t)T −M)2 F

Set Y = Y (t) , Z = Z (t) and stop end if end for

Ω

F

Finally, by calling Algorithm 3 to solve problem (1) in the algorithm framework(Algorithm 1), we obtain Algorithm 4, which is a specific decomposition alternating direction method for the MC problem (1). Algorithm 4 Decomposition alternating direction method for matrix completion (1) Require: m, n, Ω and Mi, j ∈  for (i, j) ∈ Ω, r start ≤ rank(M), positive integer Δr,  and ˆ Ensure: Return Y and Z for r = r start , r start + Δr, · · · , min{m, n} do solve fixed rank problem (3) by Algorithm 3 with rank r,  and ˆ if PΩ (YZ T − M)2F ≤ M2F then stop end if end for

154

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

Table 1. Comparison on type I problem (m = n = 1000)

problem rank SR 10 0.02 10 0.04 20 0.03 20 0.05 30 0.04 30 0.06 40 0.05 40 0.07

LMaFit rel.res time(s) 0.0077 2.85 0.0026 3.76 0.0001 3.69 0.0481 11.23 0.0009 9.17 0.0044 10.19 0.0029 15.64 0.0176 12.39

DADM rel.res time(s) 0.0000 0.8 0.0089 2.75 0.0077 1.58 0.0000 4.97 0.0089 4.86 0.0000 11.18 0.0100 8.22 0.0084 29.76

APGL rel.res time(s) 0.0852 18.24 0.1503 6.42 0.1417 20.17 0.1796 21.66 0.1992 42.27 0.2462 47.23 0.2581 38.31 0.3162 47.6

FPCA rel.res time(s) 0.9989 58.57 0.7233 98.6 0.9973 85.42 0.9789 109.98 0.9964 103.48 0.9689 91.96 0.9936 107.2 0.9715 115.92

3. Experiments We implement Algorithm 4 in Matlab with a couple of small tasks written in C to avoid ineffective memory usage in Matlab. The implementation is called as ”DADM”(Decomposition Alternative Direction Method) in the following. To test the performance of our new method, we compare DADM with several state-of-art algorithms for the MC problem (1). The packages we used for comparison include: APGL[23], LMaFit[24], and FPCA[22]. 1 The stopping tolerance for all solvers are set to 10−4 . All other parameters are set to each solvers’ default value. Experiments reported in this section were performed on a Compaq 6910p laptop with two Intel 1.17 GHz processors and 1.0 GB RAM. The operation system is Ubuntu 9.04. The Version of Matlab is R2010b. Given the matrix size m, n, rank r and the sample ratio S R, generate Y ∈ Rm×r and Z ∈ Rn×r with i.i.d. standard Gaussian entries. Two types of MC problems are considered for our experiment: Type I: Sample the entries of matrix YZ T uniformly to form the matrix M according to the sample ratio S R. We call the dataset generated in this way as the synthetic matrix completion problem. Type II: Orthogonalize Y and Z to U and V respectively; Construct the diagonal matrix Σ = diag(σi ) with either power decreasing function (σi = i−3 ) or exponential decreasing function(σi = e−0.3i ); Sample the entries of matrix A = UΣV T uniformly to form the matrix M according to the sample ratio S R. For the dataset generated in this way, we call it as the synthetic low rank approximation problem. We report the computational results for two type of problems with the matrix size 1000 × 1000 in Table 1 and 2, respectively. The column “rel.res” and “time(s)” represent the relative residual of the solutions and the running time for the corresponding solver, respectively. In Table 1, “rank” denotes the rank of the underlying data matrix M, while in Table 2, “rank” records the rank of the low rank matrix W found by the solvers. From the results in two tables we can see that, for both types of data, our new method DADM is comparable with LMaFit, and is faster than APGL and FPCA. Especially, for the second type of data, from Table 2 we can find that the rank of the low rank matrix obtained with our method DADM is lower than LMaFit. Now we further test the behaviors of the algorithms with the variations of the sizes of the matrix, the sample ratios, and the rank of the matrix. Figure 2 depicts the performance of different(solvers APGL, LMafit and DADM) on the second type of problems. In the left one of Figure 2, we set σ = e−0.3i and fix the sample ratio to 0.02. From this figure we can see that when the size of matrix increases from 1000 × 1000 to 4000, the increment of running time for our solver DADM is the least. In the right one of Figure 2, we set σ = i−3 and increase the sample ratio from 0.02 to 0.16 with step 0.02. From the results we find that that the running time of DADM is almost unchanged with the variation of sample ration, which indicates that our method is more suitable for the problems with relatively high sample ratio. Then we fix the the size of the matrix to 1000 × 1000 and the sample ratio to 0.06 for the first type of problems, and increase the underlying rank of matrix M from 10 to 40. The running time for APGL, LMaFit and DADM is recorded in Figure 3. From this figure we can see that, with the increasing of rank(M), the running time for APGL grows very fast. On the contrary, the running time for DADM and LMaFit 1 APGL is downloaded from http://www.math.nus.edu.sg/\ mattohkc/NNLS.html; LMaFit is is downloaded from http:// ~ lmafit.blogs.rice.edu/; FPCA is downloaded from http://www.columbia.edu/\~sm2756/FPCA.htm.

155

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

Table 2. Comparison on type II problem (m = n = 1000)

σi = i−3 problem SR 0.02 0.04 0.06 0.08 0.1 0.12

rel.res 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001

LMaFit rank time(s) 16 1.3 26 6.82 31 4.42 31 3.34 26 4.56 36 5.58

rel.res 0.0000 0.0000 0.0000 0.0089 0.0089 0.0089

problem SR 0.02 0.04 0.06 0.08 0.1 0.12

rel.res 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001

LMaFit rank time(s) 21 1.75 31 3.98 41 5.48 26 3.9 36 2.91 31 2.59

rel.res 0.0000 0.0000 0.0095 0.0100 0.0000 0.0095

DADM rank time(s) 16 0.54 11 1.1 11 0.52 6 0.5 6 0.74 6 1.39 σi = e−0.3i DADM rank time(s) 21 0.85 31 2.3 21 2.33 16 3.33 21 2.26 16 2.23

rel.res 0.0001 0.0001 0.0002 0.0001 0.0001 0.0001

APGL rank time(s) 75 7.12 74 9.36 11 3.7 73 10.3 5 4.46 5 6.39

rel.res 0.1414 0.0448 0.0433 0.0426 0.0424 0.0192

FPCA rank time(s) 1 83.95 2 84.42 2 91.88 2 95.1 2 102.66 3 110.84

rel.res 0.0005 0.0001 0.0001 0.0002 0.0003 0.0002

APGL rank time(s) 70 10.31 80 11.74 80 11.27 11 6.03 11 3.71 12 4.48

rel.res 0.9951 0.1624 0.0673 0.0464 0.0336 0.0323

FPCA rank time(s) 17 77.52 14 76.91 10 84.48 11 87.45 12 100 12 105.81

Time with increasing of scale (SR=0.02, σi = e−0.3i)

Time with increasing of SR (m=1000, n=1000, σi = i−3)

120

14 APGL LMaFit DADM

APGL LMaFit DADM 12

100

10 80

Time(s)

Time(s)

8 60

6 40 4

20

0 1000

2

1500

2000

2500 scale

3000

3500

4000

0 0.02

0.04

0.06

0.08

0.1

0.12

0.14

SR

Fig. 2. Comparison on different sizes of matrix(left) and different sample ratio(right) on the second type of problems

0.16

156

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

increase much more slowly. This phenomenon is due to the fact that APGL is based SVDs and DADM and LMaFit are both based on matrix factorization. In all, results in both Figure 2 and 3 demonstrate that the performance of

Time with increasing of rank (m=1000, n=1000, SR=0.06) 50 APGL LMaFit DADM

45

40

35

Time(s)

30

25

20

15

10

5

0 10

15

20

25 rank

30

35

40

Fig. 3. Comparison on different ranks of the underlying matrix

our method DADM is stable with the variations of the sizes of the matrix, the sample ratios, and the rank of the matrix. Based on the experiments, we recommend DADM especially when the ratio of sampled data is relatively high. 4. Conclusion and future work In this work, we consider the MC problem (1) by solving a series of fixed rank matrix factorization problems (3). Given the rank of the matrix, the matrix need to be completed can be naturally written as the multiple of two matrices and variables are divided into two parts correspondingly. With this structure characteristic of (3), we propse a simple alternative direction method to find the optimal solutions. Then with the first order optimality conditions, the solving process for each part of variables can be further converted into several relative small scale independent linear equations systems. Based on these observations, we design a decomposition alternative

Lingfeng Niu and Xi Zhao / Procedia Computer Science 17 (2013) 149 – 157

157

direction method for the MC problem (1). To test the performance of the new method, we implement our method in Matlab(with a few C/Mex functions) and compare it with several state-of-the-art solvers. Preliminary results indeed demonstrate the effectiveness and efficiency of our method. Based on linear equation systems (10) and (11), we know that the update of yi (z j ), for i = 1, · · · , m( j = 1, · · · , n) are independent with each other. And the new method is very suitable for distributed implementation. Therefore, one of our ongoing works is developing a parallel version of DADM and apply it on more large scale practical MC problems. We also plan to analyze the convergence properties of DADM in the future. Acknowledgement This work was partially supported by the National Natural Science Foundation of China(Grant No. 11201472£11026187 70921061, 10831006),the president fund of the Graduate University of Chinese Academy of Sciences(( Grant No. Y15101PYOO), and the CAS/SAFEA International Partnership Program for Creative Research Teams(Grant No. kjcx-yw-s7). References [1] N. Srebro, Learning with matrix factorizations, Ph.D. thesis, PhD thesis, Massachusetts Institute of Technology (2004). [2] J. D. M. Rennie, N. Srebro, Fast maximum margin matrix factorization for collaborative prediction, in: ICML’05 Proceedings of the 22nd international conference on Machine learning, 2005. [3] J. Bennett, S. Lanning, The netflix prize, Proceedings of KDD Cup and Worshop (2007). [4] E. Cand`es, B. Recht, Exact matrix completion via convex optimization, Foundations of Computational Mathematics 9 (6) (2009) 717– 772. [5] E. Cand`es, T. Tao, The power of convex relaxation: near-optimal matrix completion, IEEE Transactions on Information Theory 56 (5) (2009) 2053–2080. [6] R. H. Keshavan, A. Montanari, S. Oh, Matrix completion from a few entries, IEEE Transactions on Information Theory 56 (6) (2010) 2980–2998. [7] R. H. Keshavan, A. Montanari, S. Oh, Matrix completion from noisy entries, Journal of Machine Learning Research 11 (2010) 2057– 2078. [8] B. Recht, M. Fazel, A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review 52 (2010) 471–501. [9] E. Cand`es, Y. Plan, Matrix completion with noise, in: Proceedings of the IEEE, Vol. 98, 2010, pp. 925–936. [10] D. Gross, Recovering low-rank matrices from few coefficients in any basis, IEEE Transactions on Information Theory 7 (3) (2011) 1548–1566. [11] E. Cand`es, B. Recht, Exact matrix completion via convex optimization, Communications of the ACM 55 (6) (2012) 111–119. [12] C. Tomasi, T. Kanade, Shape and motion from image streams under orthography: a factorization method, International Journal Computer Vision 9 (1992) 137–154. [13] T. Morita, T. Kanade, A sequential factorization method for revoering shape and motion from image streams, IEEE transaction on Pattern Analysis and Machine intelligence 19 (1997) 858–867. [14] E. L. N. Linial, Y. Rabinovich, The geometry of graphs and some of its algorithmic applications, Combinatorica 15 (1995) 215–245. [15] Z. Liu, L. Vanenberghe, Interior-point method for nuclear norm approximation with application to system identification, SIAM Journal on Matrix Alanlysis and Applications 31 (3) (2009) 1235–1256. [16] L. Eld´en, Matrix methods in Data Mining and Pattern Recognition(Fundamentals of Algorithms), Society for Industrial and Applied Mathematics, Philadelphia, USA, 2007. [17] Z. Liu, L. Vandenberghe, Interior-point method for nuclear norm approximation with application to system identification, SIAM Journal on Matrix Analysis and Applications 31 (3) (2009) 1235–1256. [18] a. M. T. K.C. Toh, R. Tutuncu, Sdpt3 — a matlab software package for semidefinite programming, Optimization Methods and Software 11 (1999) 545–581. [19] R. Tutuncu, K. Toh, M. Todd, Solving semidefinite-quadratic-linear programs using sdpt3, Mathematical Programming Series B 95 (2003) 189–217. [20] J. Sturm, Using sedumi 1.02, a matlab toolbox for optimization over symmetric cones, Optimization Methods and Software 11-12 (1999) 625–653. [21] J. Cai, E. Cand`es, Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM Journal on Optimization 20 (2010) 1956–1982. [22] D. G. S. Ma, L. Chen, Fixed point and bregman iterative methods for matric rank minimization, Mathematical Programmning 128 (1-2) (2011) 321–353. [23] K. C. Koh, S. Yun, An accelerated proximal gradient algorithm for nuvlear norm regu;arized linear least squares problems, Pacific Journal of optimization 6 (2010) 615–640. [24] Z. Wen, W. Yin, Y. Zhang, Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm, Mathematical Programming Computation 2 (2010) 203–210.