- Email: [email protected]
How to predict recommendation lists that users do not like
Physica A 537 (2020) 122684
Contents lists available at ScienceDirect
Physica A journal homepage: www.elsevier.com/locate/physa
How to predict recommendation lists that users do not like ∗
Ke Gu a,b , Ying Fan a , , Zengru Di a a
School of Systems Science, Beijing Normal University, 100875 Beijing, China Department of Information Management, Economics and Management College, Beijing Institute of Petrochemical Technology, 102617 Beijing, China b
article
info
Article history: Received 19 April 2019 Received in revised form 8 August 2019 Available online 14 September 2019 Keywords: Negative recommendation list Signed network-based inference Personal recommendation
a b s t r a c t It is obviously that many user-object online rating systems usually contain the information of the users’ attitudes: like or dislike the objects, these systems can be represented by signed bipartite networks. The common recommendation systems work on unsigned networks. Even if some consider the negative edges, they are all concerned with the objects that the recommended user likes. However, the objects that the user does not like are more personalized. Based on Network-Based Inference (NBI) and signed bipartite networks, we proposed Signed Network-Based Inference (SNBI) to provide the negative recommendation list, which predicts the objects that users dislike. The SNBI algorithm includes two mechanisms: When allocating resources on signed bipartite networks, first method (SNBI-1) does not allocate resources while second method (SNBI-2) reduces resources if there is a negative edge. By comparing the results on the actual data sets with NBI, we found that SNBI-2 which takes into account the role of the negative edges can better predict the objects that user does not like while maintaining the validity of the positive recommendation list, then gives more personalized recommendation. © 2019 Elsevier B.V. All rights reserved.
1. Introduction In online rating systems the users can rate the objects. Many online rating systems actually have two-mode nature, and recommender systems based on online rating systems usually are modeled as user-object bipartite networks [1–4]. Moreover, according to experience, in online rating systems high rating means the user likes the object and low rating means the user dislikes the object. Therefore, we can introduce signed bipartite networks into online rating systems [5]. The positive/negative edge of signed bipartite network means the user likes/dislikes the object. The common recommender systems focused on personalized recommendations and have been widely used in many fields [6,7], especially in ecommerce [8], which can recommend the users books [9], news [10], Tv shows and movies [11], etc., by using the history records. Usually, these recommender systems are designed to find objects that users like [12] or they do not know [13]. In order to recommend a list that the user likes, most of recommender systems on online rating systems actually only used the positive edges and ignore the negative edges [14] which in fact losing many useful information. Indeed, the negative edges play an important role. In click farming behavior of e-commerce, due to the influence of the bad reviews, the removal of bad reviews is more expensive than the increase in good reviews, which also illustrates the importance of the negative edges. Therefore using the sign properties of edges to analysis real networks has practical applications [15– 18]. Some studies have also noticed the recommendation algorithm on the signed networks and got some results, such as founding that negative ratings can play a positive role in information filter [19] or proposing the new recommender ∗ Corresponding author. E-mail address: [email protected] (Y. Fan). https://doi.org/10.1016/j.physa.2019.122684 0378-4371/© 2019 Elsevier B.V. All rights reserved.
2
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
algorithms on signed networks [20–22]. But the results are all about the positive recommendation list which is the list of the objects that the users like, while seldom about negative recommendation list which predicts the objects that users do not like. Both positive and negative recommendation lists are important, just like two aspects of a thing: One is to present a list that the user likes in order to get better results, and the other is to present a list that the user hates, avoiding worse results. If the wrong product is recommended, it may cause discomfort and antipathy. Our team has been paying attention to the signed network and doing some research [5,22,23]. We found that in nature, in signed bipartite network that two users like the same object is easy to occur, and it is difficult for two users to dislike the same object jointly [5], so objects that users do not like are more representative of the users’ personalization. Therefore, from the perspective of personalized recommendation, it is better to implement personalized recommendations by presenting positive and negative recommended lists. Considering the role of negative edges, it makes sense to put forward the negative recommendation list in combination with NBI (network-based inference) [1] and signed bipartite network. In this paper, we allocate resources on the signed bipartite network and get a positive recommendation list and a negative recommendation list. Based on NBI and signed bipartite networks, we proposed Signed Network-Based Inference (SNBI) which includes two way of resource allocation: When allocating resources on signed bipartite networks, first method (SNBI-1) does not allocate resources while second method (SNBI-2) reduces resources if there is a negative edge. The signed bipartite networks are constructed from online rating system MovieLens [24] and Netflix. Comparing with the numerical results of the NBI method on the unsigned bipartite network, it is found that when presenting a positive recommendation list, the three methods performed similarly, while SNBI-2 which takes into account the role of the negative edges performed best when presented with a negative recommendation list. Consequently SNBI considering the role of negative edges can give more personalized recommendations. 2. Signed Network-Based Inference 2.1. Negative recommendation list Many recommender systems can be represented by user-object bipartite networks [25] and the user-object rating systems usually can be described by the weighted bipartite networks. Then we can construct unweighted signed bipartite networks from weighted bipartite networks depends on the weight values in the networks. The unweighted signed bipartite network includes the information about the users, objects, and users’ attitudes: like or dislike the objects. The standard Network-Based Inference (NBI) works on unsigned bipartite networks [1], and can provide a recommendation list for each user. For signed networks, we propose the Signed Network-Based Inference (SNBI) method. In order to observe the effect of negative edges, we extend NBI on unweighted signed bipartite networks in two ways to get recommendation list. Usually, the list of recommendation is about finding the objects the user likes. We call it the positive recommendation list. However, we are concerned about objects that users dislike, SNBI can find a list of the objects that user dislikes, called the negative recommendation list. For the unweighted signed bipartite user-object network, U = {u1 , u2 , . . . , um } denotes the set of user nodes, the object set can be denoted as O = {o1 , o2 , . . . , on }. Then the signed bipartite network can be described as an adjacency matrix A = {aij } ∈ Rn,m , where n is the number of objects and m is the number of users . In the matrix, aij = 1, aij = −1 and aij = 0 (i = 1, . . . , n; j = 1, . . . , m) denote positive, negative and missing links between oi and uj , respectively, which means object i is liked, disliked by user j and otherwise. We respectively use the adjacency matrices A+ and A− n ,m − n ,m to represent positive and negative network. The matrix A+ = {a+ (a+ = { a− (a− ij } ∈ R ij ≥ 0), A ij } ∈ R ij ≥ 0), and
∑m
∑n
A = A+ − A− . The k(oi ) = j=1 |aij | is the total degree of oi , and the k(uj ) = i=1 |aij | is the total degree of uj . The initial resource matrix S = A+ + A− . Just like in NBI, the resources are still distributed from one type nodes to the other type nodes in SNBI, but when there is a negative edge between the two kinds of nodes, we deal with it differently. 2.2. Signed Network-Based Inference-1 (SNBI-1) In this part, the resource of one node is still divided equally into several parts according to the total degree on the node and is allocated in two steps. The resource in an arbitrary object node should be distributed to its neighbors in user set U, and the resource in any user node also should be distributed to its object neighbors. As shown in Fig. 1, the initial resources x, y and z (x, y and z ≥ 0) are allocated to three object nodes. For different user, the value of the initial resource is different. First step we assigned the resource from object nodes to user nodes: If there is a positive edge between the object and user, one part of the resource is distributed to the user; If there is a negative edge, no resource is assigned to the user. Second step we assigned the resources back from the users to the objects by following the same principle as the first step. If we use x′ , y′ and z ′ to represent the final resource located in the three object node, the whole resource-allocation
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
3
Fig. 1. Illustration of the resource-allocation process in SNBI-1.
process in Fig. 1 can be expressed as
⎛ ( ′) x y′ z′
⎞
1
1
1
⎜ ⎜6 ⎜1 =⎜ ⎜ ⎜6 ⎝1
12 11
6 ⎟ (x) 1⎟ ⎟ y .
⎟
24 1
(1)
6⎟ z 1⎠
⎟
6 12 6 Now consider using the SNBI-1 on an unweighted signed bipartite user-object network with n objects and m users. The initial resource located on the ith object node is f (oi ) ≥ 0. The resource-allocation process also consists of two steps. Step 1: The resource the lth user node receives from the object nodes is f (ul ) =
n ∑ a+ f (oi ) il
k(oi )
i=1
,
(2)
where k(oi ) is the total degree of oi , which is the number of users who have rated the oi , no matter there is a positive or negative edge between the users and oi . Step 2: The user resource flows back to object nodes and the ultimate object resource is f ′ (oi ) =
m ∑ a+ f (ul ) il
k(ul )
l=1
=
m n ∑ ∑ a+ a+ jl f (oj ) il l=1
k(ul )
j=1
k(oj )
,
(3)
where k(ul ) is the total degree of ul , which is how many objects ul has rated. And the Eq. (3) can be rewritten as f ′ (oi ) =
n ∑
wij f (oj ),
(4)
j=1
where
wij =
1 k(oj )
m + ∑ a+ il ajl l=1
k(ul )
.
(5)
For a specific user uα , if the user has rated the object oi , the initial resource f (oi ) = 1 on object oi no matter the user likes or dislikes the object, and f (oi ) = 0 otherwise. So the initial resource vector on objects of user uα is the α th column of the initial resource matrix S and the initial resources are greater than or equal to zero. The whole resource-allocation process can be expressed by the matrix form f⃗′ = W ⃗ f , where W = {wij }n×n is the transformation matrix. The ⃗ f denotes the initial ′ ⃗ resource vector on objects and f denotes the ultimate resource vector. Then all uncollected objects of uα are sorted in the descending order according to the final resource. And we get the positive recommendation list in the usual sense and those objects on the top of the list are recommended. If we rank the uncollected objects in ascending order by resource value, then we get a negative recommendation list, the objects on the top of the negative list are not recommended. 2.3. Signed Network-Based Inference-2 (SNBI-2) In SNBI-1, if there is a negative edge between two nodes, we do not transform resource. While in SNBI-2, the role of negative edge has been strengthened. Negative resource has been transformed when there is a negative edge during the
4
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
Fig. 2. Illustration of the resource-allocation process in SNBI-2.
resource-allocation process. The initial conditions of SNBI-2 are same with SNBI-1 and the resource is also allocated in two steps. Fig. 2 shows the illustration. First step we assigned the resource from object nodes to user nodes: If there is a positive edge between the object and user, one part of the resource is distributed to the user; If there is a negative edge, negative resource is assigned to the user. Second step we assigned the resources from the users to the objects. Because of the resource-allocation rule, there may be negative resource after the first step, so we use the absolute value of the first step result as the initial resource of the second step. Then, if there is a positive edge between the user and object, one part of positive value is distributed to the object; if there is a negative edge, negative value is assigned to the object. The resource of each step is presented in Fig. 2 beside the nodes. Considering an unweighted signed bipartite user-object network with n objects and m users, the initial resource located on the ith object node is f (oi ) ≥ 0. The resource-allocation process of SNBI- 2 also consists of two steps. Step 1: The resource the lth user node receives from the object nodes is f (ul ) =
n ∑ ail f (oi ) i=1
k(oi )
,
(6)
where k(oi ) is the total degree of oi . Step 2: The user resource flows back to object nodes and the ultimate object resource is f ′ (oi ) =
m ∑ ail |f (ul )| l=1
k(ul )
⏐ ⏐ ⏐ n ⏐ m ∑ ail ⏐∑ ajl f (oj ) ⏐ ⏐ ⏐, = k(ul ) ⏐⏐ k(oj ) ⏐⏐ j=1 l=1
(7)
where k(ul ) is the total degree of ul . In SNBI-1, the whole resource-allocation process can be expressed by the matrix form f⃗′ = W ⃗ f . But in SNBI-2, we need use the absolute value of the first step result in Step 2, so there is no formula of transformation matrix. For the target user uα , a positive/negative recommended list can also be given if we sort the uncollected objects by the final resource in the descending/ascending order. 3. Empirical results To test the SNBI algorithms, we select MovieLens1 and Netflix2 which the users rated movies as the benchmark data sets. In both data sets the ratings are given by the integer ratings scaling from 1 to 5. The original data sets can be represented by the weighted bipartite networks. The high or low score usually can show the attitudes of users: like or dislike the object. When the rating is no less than (less than) 3, there is a positive (negative) link between the nodes, so MovieLens and Netflix can be constructed as two signed bipartite networks. More information about the unweighted signed bipartite networks of MovieLens and Netflix can be seen in Table 1. We need to make SNBI compared with NBI, while NBI works on unsigned and unweighted networks. So the original weighted networks are converted into unsigned and unweighted networks directly. All ratings are kept, and low scores like 1 and 2 are also retained, thus the unsigned and unweighted networks contains more information. Then each data set (signed and unsigned networks) are randomly divided into two parts: The training set contains 90% of the data, and the probe set contains the rest 10% of the data. For effective comparison, objects rated by the users 1 http://www.grouplens.org/. 2 http://www.netflixprize.com/.
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
5
Table 1 Basic information of the unweighted signed bipartite networks. Network
m
n
E
E+
p(E + )
E−
p(E − )
MovieLens Netflix
943 3000
1682 2779
100 000 197 248
82 520 168 108
82.52% 85.23%
17 480 29 140
17.48% 14.77%
Note: The m, n, E, E + and E − denote the number of users, objects, edges, positive edges and negative edges; p(E + ) and p(E − ) denote the fraction of positive edges and negative edges.
Fig. 3. The result of the position values, where r + /r − denotes the r value of the object liked/disliked by each user in the probe sets based on the positive recommendation list, and ⟨r + ⟩/⟨r − ⟩ denotes the mean value. (a) and (c) show the predicted position of each object liked by all users in the probe ranked in the ascending order, a good algorithm is expected to give small r; (b) and (d) show the predicted position of each object disliked by all users in the probe ranked in the ascending order, a good algorithm is expected to give big r.
retained in the training sets of unsigned networks are same with those objects in the training sets of signed data sets. So do the probe sets. We also tested smaller training set for each data set, which contains 70% of the data, and the rest 30% of the data constitute the probe sets. The result is similar to the previous one, so we only present the result based on the training set containing 90% of the data. Given the target user uα , SNBI and NBI can provide a positive and a negative recommendation list. The objects in the positive list uncollected by the user in the training sets are sorted in descending order by the final resources, while the negative list contains the objects uncollected by the user sorted in ascending order by the final resources. The positive/negative recommendation list contains objects the user collected in the probe set. Given an object oi collected by the user in the probe set, no matter the user likes it or dislikes it, if the length of the positive recommendation list of uα is Lα , and the ranking of oi in the ordered queue is Pi , then the position of oi [1] is rα i =
Pi Lα
.
(8)
In fact, the objects at the top of the positive list are the objects we want to recommend to the user. And the objects at the bottom of the positive list are the objects we predict that the user will not like, which are same with the objects on the top of the negative recommendation list. For the object that the user likes in the probe set the smaller value of r means the method is better. And we are also concerned about the objects disliked by the user in the probe set, however there are few objects that users dislike in the real rating systems, if we use the r value of negative recommendation list, it will be hard to present the difference of the three methods in negative recommendation list. In order to make the results clearer, we still use the r value of the objects disliked by the users in the positive recommendation list to compare. So, for the objects that users dislike we predict which at the bottom of the positive recommendation list, the bigger r the better of the algorithm. When SNBI-1, SNBI-2 and NBI are applied to the corresponding data sets, for each user, all three algorithms can provide a positive recommendation list. We can calculate the value r of the objects liked/disliked by all users in the probe sets based on the positive recommendation list . Fig. 3 shows the distribution and the mean value of all the position values for all users, which are ranked from the top position (r → 0) to the bottom position (r → 1).
6
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
Fig. 4. The result of positive/negative hitting rate as the function of the positive/negative recommendation list length L, where hitting rate+ /hitting rate− denotes the positive/negative hitting rate. (a) and (c) show the hitting rate of the positive recommendation list; (b) and (d) show the hitting rate of the negative recommendation list.
Fig. 3(a)/(b) reports the distribution and the mean value of all the position values for the objects that the users like/dislike in the probe sets of MovieLens. And Fig. 3(c)/(d) reports the result in the probe sets of Netflix. Fig. 3(a) and (c) show that the distribution of r + and the mean value ⟨r + ⟩ of NBI, SNBI-1 and SNBI-2 in both data sets are very closed, which means that when recommending the objects the users like, the performance of the three methods are similar. However, in Fig. 3(b) and (d), the r − and ⟨r − ⟩ values of SNBI-2 are obviously bigger than the other methods, which means that SNBI-2 performs best in identifying the objects that users dislike. To measure the accuracy of the methods another metric hitting rate [1] also can be used. For the user uα , the positive/negative recommendation list with length L contains top L objects should/should not recommend to uα resulting from the algorithm. So the positive/negative hitting rate is defined as the ratio of the number of the objects liked/disliked by uα contained in the top-L positive/negative recommendations to the number of the objects that uα likes/dislikes in the probe set, say Hα (L) =
Tα (L) Nα
,
(9)
where Tα (L) denotes the number of the objects liked/disliked by uα contained in the top-L positive/negative recommendations, and Nα denotes the number of the objects that uα likes/dislikes in the probe set. The accuracy of the whole system [19] is the average of individual accuracies over all users, given as hitting rate =
m 1 ∑
m
Hα (L).
(10)
α=1
Obviously hitting rate is the function of the recommendation list length L. Whether it is the hitting rate of the recommended list or of the negative recommendation list, the higher the hitting rate value, the higher the algorithmic accuracy. For the positive/negative recommendation list provided for every user by three algorithms, we calculated the positive/negative hitting rate for each recommendation list length L, and the result is reported in Fig. 4. Fig. 4(a)/(b) presents the positive/negative hitting rate of MovieLens, and Fig. 4(c)/(d) reports the result of Netflix. Fig. 4(a) and (c) show that the hitting rates of three methods are very closed in both data sets, which also means that when recommending the objects the users like, the performance of the three methods are similar. And, in Fig. 4(b) and (d), the hitting rates of SNBI-2 are obviously bigger than the other methods. In accordance with the result of position value, SNBI-2 has the highest accuracy in identifying the objects that users dislike. Combining the performance of SNBI-2 on metric r and metric hitting rate, we find that SNBI-2 performs well when recommending objects that the users like, and performs best when predicting objects that the users do not like. Traditional recommendation algorithms only focused on positive edges and ignored the negative edges of the network, so only the positive recommendation list can be given. However, a good recommendation method should not only be able
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
7
to recommend objects that the users like, but also can prevent recommending objects that users do not like. By considering the role of negative edges, we have proposed SNBI-2 and negative recommendation list. The numerical results on two data sets indicate that we can extend this SNBI-2 to more similar online rate systems to improve the effectiveness of personalized recommendations. 4. Conclusion and discussion Negative edges usually are ignored in the recommender systems, making us lose some important related information between the users and the objects. The negative edges are more expressive in personality than the positive edges. We proposed the SNBI by combining NBI and signed bipartite network to get more personalized recommendations. There are two ways to allocate resources: The first one does not allocate resources when it encounters a negative edge (SNBI1), and the second reduces resource allocation when it encounters a negative edge (SNBI-2). Then a positive/negative recommendation list can be given. The benchmark data sets, i.e., MovieLens and Netflix are used to test the performance by SNBI and NBI. By comparing the results of three methods, all three methods can recommend users’ favorite objects availably while SNBI-2 is more effective when making a negative recommendation list. Since SNBI-2 reduces the allocation of resources when it encounters a negative edge, which indicates that the negative edges are used effectively. The negative recommendation list provided by SNBI-2 can predict the objects that users do not like and make the recommendations more personalized. Many e-commerce systems are similar in structure and characteristics to online rating systems, and they now focus not only on praise(positive edges) but also on bad reviews(negative edges). Using the original recommendation methods are easy to find what users like, but it is difficult to find objects that users do not like. SNBI-2 also can be applied to the field of e-commerce to find the products that users do not like by using the negative edges. So SNBI-2 highlights a possible way to get a better personal recommendation. In this paper, SNBI is acting on the unweighted signed bipartite network, in the future, we will consider the situation of weighted signed bipartite network. Furthermore, when we construct signed bipartite network, we consider that a unified standard used by all users. Such as that for all users that if they rated an object 1 or 2 score means that they dislike the object. If individual differences are taken into consideration, a certain type of person may have their own scoring criteria, such as taking 3 as a dislike too. A relative criterion based on individual differences can be further considered. Finally, we hope that we will be able to further expand our research results in the future and to apply it to more fields such as recommendations on e-commerce systems, etc. Acknowledgment This work is supported by the National Natural Science Foundation of China (Grant Nos. 61573065). References [1] T. Zhou, J. Ren, M. Medo, Y.-C. Zhang, Bipartite network projection and personal recommendation, Phys. Rev. E 76 (4) (2007) 046115. [2] Z. Huang, W. Chung, H. Chen, A graph model for E-commerce recommender systems, J. Amer. Soc. Inf. Sci. Technol. 55 (3) (2004) 259–274. [3] A. Anglade, M. Tiemann, F. Vignoli, Complex-network theoretic clustering for identifying groups of similar listeners in p2p systems, in: Proceedings of the 2007 ACM Conference on Recommender Systems, ACM, 2007, pp. 41–48. [4] Z. Huang, D.D. Zeng, H. Chen, Analyzing consumer-product graphs: Empirical findings and applications in recommender systems, Manag. Sci. 53 (7) (2007) 1146–1164. [5] K. Gu, Y. Fan, A. Zeng, J. Zhou, Z. Di, Analysis on large-scale rating systems based on the signed network, Physica A 507 (2018) 99–109. [6] L. Lü, M. Medo, C.H. Yeung, Y.-C. Zhang, Z.-K. Zhang, T. Zhou, Recommender systems, Phys. Rep. 519 (1) (2012) 1–49. [7] D. Jannach, M. Zanker, A. Felfernig, G. Friedrich, Recommender Systems: An Introduction, Cambridge University Press, 2010. [8] J. Schafer, J.A. Konstan, J. Riedl, E-commerce recommendation applications, Data Min. Knowl. Discov. 5 (1–2) (2001) 115–153. [9] G. Linden, B. Smith, J. York, Amazon. com recommendations: Item-to-item collaborative filtering, IEEE Internet Comput. 7 (1) (2003) 76–80. [10] D. Billsus, C.A. Brunk, C. Evans, B. Gladish, M. Pazzani, Adaptive interfaces for ubiquitous web access, Commun. ACM 45 (5) (2002) 34–38. [11] K. Ali, W. Van Stam, Tivo: making show recommendations using a distributed collaborative filtering architecture, in: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2004, pp. 394–401. [12] N.J. Belkin, Helping people find what they don’t know, Commun. ACM 43 (8) (2000) 58–61. [13] T. Zhou, Z. Kuscsik, J.-G. Liu, M. Medo, J.R. Wakeling, Y.-C. Zhang, Solving the apparent diversity-accuracy dilemma of recommender systems, Proc. Natl. Acad. Sci. 107 (10) (2010) 4511–4515. [14] C.-J. Zhang, A. Zeng, Behavior patterns of online users and the effect on information filtering, Physica A 391 (4) (2012) 1822–1830. [15] J. Kunegis, A. Lommatzsch, C. Bauckhage, The slashdot zoo: mining a social network with negative edges, in: Proceedings of the 18th International Conference on World Wide Web, ACM, 2009, pp. 741–750. [16] P. Symeonidis, E. Tiakas, Y. Manolopoulos, Transitive node similarity for link prediction in social networks with positive and negative links, in: ACM Conference on Recommender Systems, 2010, pp. 183–190. [17] J. Leskovec, D. Huttenlocher, J. Kleinberg, Predicting positive and negative links in online social networks, in: Proceedings of the 19th International Conference on World Wide Web, ACM, 2010, pp. 641–650. [18] R. Singh, B. Adhikari, Measuring the balance of signed networks and its application to sign prediction, J. Stat. Mech. (2017) 063302. [19] W. Zeng, Y.-X. Zhu, L. Lü, T. Zhou, Negative ratings play a positive role in information filtering, Physica A 390 (23–24) (2011) 4486–4493. [20] B. Shams, S. Haratizadeh, Sibrank: Signed bipartite network analysis for neighbor-based collaborative ranking, Physica A 458 (2016) 364–377. [21] L. Hu, L. Ren, W. Lin, A reconsideration of negative ratings for network-based recommendation, Physica A 490 (2018) 690–701. [22] P. Zhang, X. Song, L. Xue, K. Gu, A new recommender algorithm on signed networks, Physica A (2019). [23] L. Li, K. Gu, A. Zeng, Y. Fan, Z. Di, Modeling online social signed networks, Physica A 495 (2018) 345–352. [24] F. Harper, J.A. Konstan, The movielens datasets: History and context, ACM Trans. Interact. Intell. Syst. (TIIS) 5 (4) (2016) 19. [25] M.-S. Shang, L. Lü, Y.-C. Zhang, T. Zhou, Empirical analysis of web-based user-object bipartite networks, Europhys. Lett. 90 (4) (2010) 48006.
Contents lists available at ScienceDirect
Physica A journal homepage: www.elsevier.com/locate/physa
How to predict recommendation lists that users do not like ∗
Ke Gu a,b , Ying Fan a , , Zengru Di a a
School of Systems Science, Beijing Normal University, 100875 Beijing, China Department of Information Management, Economics and Management College, Beijing Institute of Petrochemical Technology, 102617 Beijing, China b
article
info
Article history: Received 19 April 2019 Received in revised form 8 August 2019 Available online 14 September 2019 Keywords: Negative recommendation list Signed network-based inference Personal recommendation
a b s t r a c t It is obviously that many user-object online rating systems usually contain the information of the users’ attitudes: like or dislike the objects, these systems can be represented by signed bipartite networks. The common recommendation systems work on unsigned networks. Even if some consider the negative edges, they are all concerned with the objects that the recommended user likes. However, the objects that the user does not like are more personalized. Based on Network-Based Inference (NBI) and signed bipartite networks, we proposed Signed Network-Based Inference (SNBI) to provide the negative recommendation list, which predicts the objects that users dislike. The SNBI algorithm includes two mechanisms: When allocating resources on signed bipartite networks, first method (SNBI-1) does not allocate resources while second method (SNBI-2) reduces resources if there is a negative edge. By comparing the results on the actual data sets with NBI, we found that SNBI-2 which takes into account the role of the negative edges can better predict the objects that user does not like while maintaining the validity of the positive recommendation list, then gives more personalized recommendation. © 2019 Elsevier B.V. All rights reserved.
1. Introduction In online rating systems the users can rate the objects. Many online rating systems actually have two-mode nature, and recommender systems based on online rating systems usually are modeled as user-object bipartite networks [1–4]. Moreover, according to experience, in online rating systems high rating means the user likes the object and low rating means the user dislikes the object. Therefore, we can introduce signed bipartite networks into online rating systems [5]. The positive/negative edge of signed bipartite network means the user likes/dislikes the object. The common recommender systems focused on personalized recommendations and have been widely used in many fields [6,7], especially in ecommerce [8], which can recommend the users books [9], news [10], Tv shows and movies [11], etc., by using the history records. Usually, these recommender systems are designed to find objects that users like [12] or they do not know [13]. In order to recommend a list that the user likes, most of recommender systems on online rating systems actually only used the positive edges and ignore the negative edges [14] which in fact losing many useful information. Indeed, the negative edges play an important role. In click farming behavior of e-commerce, due to the influence of the bad reviews, the removal of bad reviews is more expensive than the increase in good reviews, which also illustrates the importance of the negative edges. Therefore using the sign properties of edges to analysis real networks has practical applications [15– 18]. Some studies have also noticed the recommendation algorithm on the signed networks and got some results, such as founding that negative ratings can play a positive role in information filter [19] or proposing the new recommender ∗ Corresponding author. E-mail address: [email protected] (Y. Fan). https://doi.org/10.1016/j.physa.2019.122684 0378-4371/© 2019 Elsevier B.V. All rights reserved.
2
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
algorithms on signed networks [20–22]. But the results are all about the positive recommendation list which is the list of the objects that the users like, while seldom about negative recommendation list which predicts the objects that users do not like. Both positive and negative recommendation lists are important, just like two aspects of a thing: One is to present a list that the user likes in order to get better results, and the other is to present a list that the user hates, avoiding worse results. If the wrong product is recommended, it may cause discomfort and antipathy. Our team has been paying attention to the signed network and doing some research [5,22,23]. We found that in nature, in signed bipartite network that two users like the same object is easy to occur, and it is difficult for two users to dislike the same object jointly [5], so objects that users do not like are more representative of the users’ personalization. Therefore, from the perspective of personalized recommendation, it is better to implement personalized recommendations by presenting positive and negative recommended lists. Considering the role of negative edges, it makes sense to put forward the negative recommendation list in combination with NBI (network-based inference) [1] and signed bipartite network. In this paper, we allocate resources on the signed bipartite network and get a positive recommendation list and a negative recommendation list. Based on NBI and signed bipartite networks, we proposed Signed Network-Based Inference (SNBI) which includes two way of resource allocation: When allocating resources on signed bipartite networks, first method (SNBI-1) does not allocate resources while second method (SNBI-2) reduces resources if there is a negative edge. The signed bipartite networks are constructed from online rating system MovieLens [24] and Netflix. Comparing with the numerical results of the NBI method on the unsigned bipartite network, it is found that when presenting a positive recommendation list, the three methods performed similarly, while SNBI-2 which takes into account the role of the negative edges performed best when presented with a negative recommendation list. Consequently SNBI considering the role of negative edges can give more personalized recommendations. 2. Signed Network-Based Inference 2.1. Negative recommendation list Many recommender systems can be represented by user-object bipartite networks [25] and the user-object rating systems usually can be described by the weighted bipartite networks. Then we can construct unweighted signed bipartite networks from weighted bipartite networks depends on the weight values in the networks. The unweighted signed bipartite network includes the information about the users, objects, and users’ attitudes: like or dislike the objects. The standard Network-Based Inference (NBI) works on unsigned bipartite networks [1], and can provide a recommendation list for each user. For signed networks, we propose the Signed Network-Based Inference (SNBI) method. In order to observe the effect of negative edges, we extend NBI on unweighted signed bipartite networks in two ways to get recommendation list. Usually, the list of recommendation is about finding the objects the user likes. We call it the positive recommendation list. However, we are concerned about objects that users dislike, SNBI can find a list of the objects that user dislikes, called the negative recommendation list. For the unweighted signed bipartite user-object network, U = {u1 , u2 , . . . , um } denotes the set of user nodes, the object set can be denoted as O = {o1 , o2 , . . . , on }. Then the signed bipartite network can be described as an adjacency matrix A = {aij } ∈ Rn,m , where n is the number of objects and m is the number of users . In the matrix, aij = 1, aij = −1 and aij = 0 (i = 1, . . . , n; j = 1, . . . , m) denote positive, negative and missing links between oi and uj , respectively, which means object i is liked, disliked by user j and otherwise. We respectively use the adjacency matrices A+ and A− n ,m − n ,m to represent positive and negative network. The matrix A+ = {a+ (a+ = { a− (a− ij } ∈ R ij ≥ 0), A ij } ∈ R ij ≥ 0), and
∑m
∑n
A = A+ − A− . The k(oi ) = j=1 |aij | is the total degree of oi , and the k(uj ) = i=1 |aij | is the total degree of uj . The initial resource matrix S = A+ + A− . Just like in NBI, the resources are still distributed from one type nodes to the other type nodes in SNBI, but when there is a negative edge between the two kinds of nodes, we deal with it differently. 2.2. Signed Network-Based Inference-1 (SNBI-1) In this part, the resource of one node is still divided equally into several parts according to the total degree on the node and is allocated in two steps. The resource in an arbitrary object node should be distributed to its neighbors in user set U, and the resource in any user node also should be distributed to its object neighbors. As shown in Fig. 1, the initial resources x, y and z (x, y and z ≥ 0) are allocated to three object nodes. For different user, the value of the initial resource is different. First step we assigned the resource from object nodes to user nodes: If there is a positive edge between the object and user, one part of the resource is distributed to the user; If there is a negative edge, no resource is assigned to the user. Second step we assigned the resources back from the users to the objects by following the same principle as the first step. If we use x′ , y′ and z ′ to represent the final resource located in the three object node, the whole resource-allocation
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
3
Fig. 1. Illustration of the resource-allocation process in SNBI-1.
process in Fig. 1 can be expressed as
⎛ ( ′) x y′ z′
⎞
1
1
1
⎜ ⎜6 ⎜1 =⎜ ⎜ ⎜6 ⎝1
12 11
6 ⎟ (x) 1⎟ ⎟ y .
⎟
24 1
(1)
6⎟ z 1⎠
⎟
6 12 6 Now consider using the SNBI-1 on an unweighted signed bipartite user-object network with n objects and m users. The initial resource located on the ith object node is f (oi ) ≥ 0. The resource-allocation process also consists of two steps. Step 1: The resource the lth user node receives from the object nodes is f (ul ) =
n ∑ a+ f (oi ) il
k(oi )
i=1
,
(2)
where k(oi ) is the total degree of oi , which is the number of users who have rated the oi , no matter there is a positive or negative edge between the users and oi . Step 2: The user resource flows back to object nodes and the ultimate object resource is f ′ (oi ) =
m ∑ a+ f (ul ) il
k(ul )
l=1
=
m n ∑ ∑ a+ a+ jl f (oj ) il l=1
k(ul )
j=1
k(oj )
,
(3)
where k(ul ) is the total degree of ul , which is how many objects ul has rated. And the Eq. (3) can be rewritten as f ′ (oi ) =
n ∑
wij f (oj ),
(4)
j=1
where
wij =
1 k(oj )
m + ∑ a+ il ajl l=1
k(ul )
.
(5)
For a specific user uα , if the user has rated the object oi , the initial resource f (oi ) = 1 on object oi no matter the user likes or dislikes the object, and f (oi ) = 0 otherwise. So the initial resource vector on objects of user uα is the α th column of the initial resource matrix S and the initial resources are greater than or equal to zero. The whole resource-allocation process can be expressed by the matrix form f⃗′ = W ⃗ f , where W = {wij }n×n is the transformation matrix. The ⃗ f denotes the initial ′ ⃗ resource vector on objects and f denotes the ultimate resource vector. Then all uncollected objects of uα are sorted in the descending order according to the final resource. And we get the positive recommendation list in the usual sense and those objects on the top of the list are recommended. If we rank the uncollected objects in ascending order by resource value, then we get a negative recommendation list, the objects on the top of the negative list are not recommended. 2.3. Signed Network-Based Inference-2 (SNBI-2) In SNBI-1, if there is a negative edge between two nodes, we do not transform resource. While in SNBI-2, the role of negative edge has been strengthened. Negative resource has been transformed when there is a negative edge during the
4
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
Fig. 2. Illustration of the resource-allocation process in SNBI-2.
resource-allocation process. The initial conditions of SNBI-2 are same with SNBI-1 and the resource is also allocated in two steps. Fig. 2 shows the illustration. First step we assigned the resource from object nodes to user nodes: If there is a positive edge between the object and user, one part of the resource is distributed to the user; If there is a negative edge, negative resource is assigned to the user. Second step we assigned the resources from the users to the objects. Because of the resource-allocation rule, there may be negative resource after the first step, so we use the absolute value of the first step result as the initial resource of the second step. Then, if there is a positive edge between the user and object, one part of positive value is distributed to the object; if there is a negative edge, negative value is assigned to the object. The resource of each step is presented in Fig. 2 beside the nodes. Considering an unweighted signed bipartite user-object network with n objects and m users, the initial resource located on the ith object node is f (oi ) ≥ 0. The resource-allocation process of SNBI- 2 also consists of two steps. Step 1: The resource the lth user node receives from the object nodes is f (ul ) =
n ∑ ail f (oi ) i=1
k(oi )
,
(6)
where k(oi ) is the total degree of oi . Step 2: The user resource flows back to object nodes and the ultimate object resource is f ′ (oi ) =
m ∑ ail |f (ul )| l=1
k(ul )
⏐ ⏐ ⏐ n ⏐ m ∑ ail ⏐∑ ajl f (oj ) ⏐ ⏐ ⏐, = k(ul ) ⏐⏐ k(oj ) ⏐⏐ j=1 l=1
(7)
where k(ul ) is the total degree of ul . In SNBI-1, the whole resource-allocation process can be expressed by the matrix form f⃗′ = W ⃗ f . But in SNBI-2, we need use the absolute value of the first step result in Step 2, so there is no formula of transformation matrix. For the target user uα , a positive/negative recommended list can also be given if we sort the uncollected objects by the final resource in the descending/ascending order. 3. Empirical results To test the SNBI algorithms, we select MovieLens1 and Netflix2 which the users rated movies as the benchmark data sets. In both data sets the ratings are given by the integer ratings scaling from 1 to 5. The original data sets can be represented by the weighted bipartite networks. The high or low score usually can show the attitudes of users: like or dislike the object. When the rating is no less than (less than) 3, there is a positive (negative) link between the nodes, so MovieLens and Netflix can be constructed as two signed bipartite networks. More information about the unweighted signed bipartite networks of MovieLens and Netflix can be seen in Table 1. We need to make SNBI compared with NBI, while NBI works on unsigned and unweighted networks. So the original weighted networks are converted into unsigned and unweighted networks directly. All ratings are kept, and low scores like 1 and 2 are also retained, thus the unsigned and unweighted networks contains more information. Then each data set (signed and unsigned networks) are randomly divided into two parts: The training set contains 90% of the data, and the probe set contains the rest 10% of the data. For effective comparison, objects rated by the users 1 http://www.grouplens.org/. 2 http://www.netflixprize.com/.
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
5
Table 1 Basic information of the unweighted signed bipartite networks. Network
m
n
E
E+
p(E + )
E−
p(E − )
MovieLens Netflix
943 3000
1682 2779
100 000 197 248
82 520 168 108
82.52% 85.23%
17 480 29 140
17.48% 14.77%
Note: The m, n, E, E + and E − denote the number of users, objects, edges, positive edges and negative edges; p(E + ) and p(E − ) denote the fraction of positive edges and negative edges.
Fig. 3. The result of the position values, where r + /r − denotes the r value of the object liked/disliked by each user in the probe sets based on the positive recommendation list, and ⟨r + ⟩/⟨r − ⟩ denotes the mean value. (a) and (c) show the predicted position of each object liked by all users in the probe ranked in the ascending order, a good algorithm is expected to give small r; (b) and (d) show the predicted position of each object disliked by all users in the probe ranked in the ascending order, a good algorithm is expected to give big r.
retained in the training sets of unsigned networks are same with those objects in the training sets of signed data sets. So do the probe sets. We also tested smaller training set for each data set, which contains 70% of the data, and the rest 30% of the data constitute the probe sets. The result is similar to the previous one, so we only present the result based on the training set containing 90% of the data. Given the target user uα , SNBI and NBI can provide a positive and a negative recommendation list. The objects in the positive list uncollected by the user in the training sets are sorted in descending order by the final resources, while the negative list contains the objects uncollected by the user sorted in ascending order by the final resources. The positive/negative recommendation list contains objects the user collected in the probe set. Given an object oi collected by the user in the probe set, no matter the user likes it or dislikes it, if the length of the positive recommendation list of uα is Lα , and the ranking of oi in the ordered queue is Pi , then the position of oi [1] is rα i =
Pi Lα
.
(8)
In fact, the objects at the top of the positive list are the objects we want to recommend to the user. And the objects at the bottom of the positive list are the objects we predict that the user will not like, which are same with the objects on the top of the negative recommendation list. For the object that the user likes in the probe set the smaller value of r means the method is better. And we are also concerned about the objects disliked by the user in the probe set, however there are few objects that users dislike in the real rating systems, if we use the r value of negative recommendation list, it will be hard to present the difference of the three methods in negative recommendation list. In order to make the results clearer, we still use the r value of the objects disliked by the users in the positive recommendation list to compare. So, for the objects that users dislike we predict which at the bottom of the positive recommendation list, the bigger r the better of the algorithm. When SNBI-1, SNBI-2 and NBI are applied to the corresponding data sets, for each user, all three algorithms can provide a positive recommendation list. We can calculate the value r of the objects liked/disliked by all users in the probe sets based on the positive recommendation list . Fig. 3 shows the distribution and the mean value of all the position values for all users, which are ranked from the top position (r → 0) to the bottom position (r → 1).
6
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
Fig. 4. The result of positive/negative hitting rate as the function of the positive/negative recommendation list length L, where hitting rate+ /hitting rate− denotes the positive/negative hitting rate. (a) and (c) show the hitting rate of the positive recommendation list; (b) and (d) show the hitting rate of the negative recommendation list.
Fig. 3(a)/(b) reports the distribution and the mean value of all the position values for the objects that the users like/dislike in the probe sets of MovieLens. And Fig. 3(c)/(d) reports the result in the probe sets of Netflix. Fig. 3(a) and (c) show that the distribution of r + and the mean value ⟨r + ⟩ of NBI, SNBI-1 and SNBI-2 in both data sets are very closed, which means that when recommending the objects the users like, the performance of the three methods are similar. However, in Fig. 3(b) and (d), the r − and ⟨r − ⟩ values of SNBI-2 are obviously bigger than the other methods, which means that SNBI-2 performs best in identifying the objects that users dislike. To measure the accuracy of the methods another metric hitting rate [1] also can be used. For the user uα , the positive/negative recommendation list with length L contains top L objects should/should not recommend to uα resulting from the algorithm. So the positive/negative hitting rate is defined as the ratio of the number of the objects liked/disliked by uα contained in the top-L positive/negative recommendations to the number of the objects that uα likes/dislikes in the probe set, say Hα (L) =
Tα (L) Nα
,
(9)
where Tα (L) denotes the number of the objects liked/disliked by uα contained in the top-L positive/negative recommendations, and Nα denotes the number of the objects that uα likes/dislikes in the probe set. The accuracy of the whole system [19] is the average of individual accuracies over all users, given as hitting rate =
m 1 ∑
m
Hα (L).
(10)
α=1
Obviously hitting rate is the function of the recommendation list length L. Whether it is the hitting rate of the recommended list or of the negative recommendation list, the higher the hitting rate value, the higher the algorithmic accuracy. For the positive/negative recommendation list provided for every user by three algorithms, we calculated the positive/negative hitting rate for each recommendation list length L, and the result is reported in Fig. 4. Fig. 4(a)/(b) presents the positive/negative hitting rate of MovieLens, and Fig. 4(c)/(d) reports the result of Netflix. Fig. 4(a) and (c) show that the hitting rates of three methods are very closed in both data sets, which also means that when recommending the objects the users like, the performance of the three methods are similar. And, in Fig. 4(b) and (d), the hitting rates of SNBI-2 are obviously bigger than the other methods. In accordance with the result of position value, SNBI-2 has the highest accuracy in identifying the objects that users dislike. Combining the performance of SNBI-2 on metric r and metric hitting rate, we find that SNBI-2 performs well when recommending objects that the users like, and performs best when predicting objects that the users do not like. Traditional recommendation algorithms only focused on positive edges and ignored the negative edges of the network, so only the positive recommendation list can be given. However, a good recommendation method should not only be able
K. Gu, Y. Fan and Z. Di / Physica A 537 (2020) 122684
7
to recommend objects that the users like, but also can prevent recommending objects that users do not like. By considering the role of negative edges, we have proposed SNBI-2 and negative recommendation list. The numerical results on two data sets indicate that we can extend this SNBI-2 to more similar online rate systems to improve the effectiveness of personalized recommendations. 4. Conclusion and discussion Negative edges usually are ignored in the recommender systems, making us lose some important related information between the users and the objects. The negative edges are more expressive in personality than the positive edges. We proposed the SNBI by combining NBI and signed bipartite network to get more personalized recommendations. There are two ways to allocate resources: The first one does not allocate resources when it encounters a negative edge (SNBI1), and the second reduces resource allocation when it encounters a negative edge (SNBI-2). Then a positive/negative recommendation list can be given. The benchmark data sets, i.e., MovieLens and Netflix are used to test the performance by SNBI and NBI. By comparing the results of three methods, all three methods can recommend users’ favorite objects availably while SNBI-2 is more effective when making a negative recommendation list. Since SNBI-2 reduces the allocation of resources when it encounters a negative edge, which indicates that the negative edges are used effectively. The negative recommendation list provided by SNBI-2 can predict the objects that users do not like and make the recommendations more personalized. Many e-commerce systems are similar in structure and characteristics to online rating systems, and they now focus not only on praise(positive edges) but also on bad reviews(negative edges). Using the original recommendation methods are easy to find what users like, but it is difficult to find objects that users do not like. SNBI-2 also can be applied to the field of e-commerce to find the products that users do not like by using the negative edges. So SNBI-2 highlights a possible way to get a better personal recommendation. In this paper, SNBI is acting on the unweighted signed bipartite network, in the future, we will consider the situation of weighted signed bipartite network. Furthermore, when we construct signed bipartite network, we consider that a unified standard used by all users. Such as that for all users that if they rated an object 1 or 2 score means that they dislike the object. If individual differences are taken into consideration, a certain type of person may have their own scoring criteria, such as taking 3 as a dislike too. A relative criterion based on individual differences can be further considered. Finally, we hope that we will be able to further expand our research results in the future and to apply it to more fields such as recommendations on e-commerce systems, etc. Acknowledgment This work is supported by the National Natural Science Foundation of China (Grant Nos. 61573065). References [1] T. Zhou, J. Ren, M. Medo, Y.-C. Zhang, Bipartite network projection and personal recommendation, Phys. Rev. E 76 (4) (2007) 046115. [2] Z. Huang, W. Chung, H. Chen, A graph model for E-commerce recommender systems, J. Amer. Soc. Inf. Sci. Technol. 55 (3) (2004) 259–274. [3] A. Anglade, M. Tiemann, F. Vignoli, Complex-network theoretic clustering for identifying groups of similar listeners in p2p systems, in: Proceedings of the 2007 ACM Conference on Recommender Systems, ACM, 2007, pp. 41–48. [4] Z. Huang, D.D. Zeng, H. Chen, Analyzing consumer-product graphs: Empirical findings and applications in recommender systems, Manag. Sci. 53 (7) (2007) 1146–1164. [5] K. Gu, Y. Fan, A. Zeng, J. Zhou, Z. Di, Analysis on large-scale rating systems based on the signed network, Physica A 507 (2018) 99–109. [6] L. Lü, M. Medo, C.H. Yeung, Y.-C. Zhang, Z.-K. Zhang, T. Zhou, Recommender systems, Phys. Rep. 519 (1) (2012) 1–49. [7] D. Jannach, M. Zanker, A. Felfernig, G. Friedrich, Recommender Systems: An Introduction, Cambridge University Press, 2010. [8] J. Schafer, J.A. Konstan, J. Riedl, E-commerce recommendation applications, Data Min. Knowl. Discov. 5 (1–2) (2001) 115–153. [9] G. Linden, B. Smith, J. York, Amazon. com recommendations: Item-to-item collaborative filtering, IEEE Internet Comput. 7 (1) (2003) 76–80. [10] D. Billsus, C.A. Brunk, C. Evans, B. Gladish, M. Pazzani, Adaptive interfaces for ubiquitous web access, Commun. ACM 45 (5) (2002) 34–38. [11] K. Ali, W. Van Stam, Tivo: making show recommendations using a distributed collaborative filtering architecture, in: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2004, pp. 394–401. [12] N.J. Belkin, Helping people find what they don’t know, Commun. ACM 43 (8) (2000) 58–61. [13] T. Zhou, Z. Kuscsik, J.-G. Liu, M. Medo, J.R. Wakeling, Y.-C. Zhang, Solving the apparent diversity-accuracy dilemma of recommender systems, Proc. Natl. Acad. Sci. 107 (10) (2010) 4511–4515. [14] C.-J. Zhang, A. Zeng, Behavior patterns of online users and the effect on information filtering, Physica A 391 (4) (2012) 1822–1830. [15] J. Kunegis, A. Lommatzsch, C. Bauckhage, The slashdot zoo: mining a social network with negative edges, in: Proceedings of the 18th International Conference on World Wide Web, ACM, 2009, pp. 741–750. [16] P. Symeonidis, E. Tiakas, Y. Manolopoulos, Transitive node similarity for link prediction in social networks with positive and negative links, in: ACM Conference on Recommender Systems, 2010, pp. 183–190. [17] J. Leskovec, D. Huttenlocher, J. Kleinberg, Predicting positive and negative links in online social networks, in: Proceedings of the 19th International Conference on World Wide Web, ACM, 2010, pp. 641–650. [18] R. Singh, B. Adhikari, Measuring the balance of signed networks and its application to sign prediction, J. Stat. Mech. (2017) 063302. [19] W. Zeng, Y.-X. Zhu, L. Lü, T. Zhou, Negative ratings play a positive role in information filtering, Physica A 390 (23–24) (2011) 4486–4493. [20] B. Shams, S. Haratizadeh, Sibrank: Signed bipartite network analysis for neighbor-based collaborative ranking, Physica A 458 (2016) 364–377. [21] L. Hu, L. Ren, W. Lin, A reconsideration of negative ratings for network-based recommendation, Physica A 490 (2018) 690–701. [22] P. Zhang, X. Song, L. Xue, K. Gu, A new recommender algorithm on signed networks, Physica A (2019). [23] L. Li, K. Gu, A. Zeng, Y. Fan, Z. Di, Modeling online social signed networks, Physica A 495 (2018) 345–352. [24] F. Harper, J.A. Konstan, The movielens datasets: History and context, ACM Trans. Interact. Intell. Syst. (TIIS) 5 (4) (2016) 19. [25] M.-S. Shang, L. Lü, Y.-C. Zhang, T. Zhou, Empirical analysis of web-based user-object bipartite networks, Europhys. Lett. 90 (4) (2010) 48006.