Collaborative Topic Regression with social trust ensemble for recommendation in social media systems

ARTICLE IN PRESS

JID: KNOSYS

[m5G;February 1, 2016;13:37]

Knowledge-Based Systems 000 (2016) 1–12

Contents lists available at ScienceDirect

Knowledge-Based Systems journal homepage: www.elsevier.com/locate/knosys

Collaborative Topic Regression with social trust ensemble for recommendation in social media systems Hao Wu∗, Kun Yue, Yijian Pei, Bo Li, Yiji Zhao, Fan Dong School of Information Science and Engineering, Yunnan University, No. 2 North Green Lake Road, Kunming 650091, China

a r t i c l e

i n f o

Article history: Received 9 May 2015 Revised 8 January 2016 Accepted 9 January 2016 Available online xxx Keywords: Social media systems Item recommendation Trust ensemble Matrix factorization Recommender systems

a b s t r a c t Social media systems provide ever-growing huge volumes of information for dissemination and communication among communities of users, while recommender systems aim to mitigate information overload by filtering and providing users the most attractive and relevant items from information-sea. This paper aims at providing compound recommendation engine for social media systems, and focuses on exploiting multi-sourced information (e.g. social networks, item contents and user feedbacks) to predict the ratings of users to items and make recommendations. For this, we suppose the users’ decisions on adopting item are affected both by their tastes and the favors of trusted friends, and extend Collaborative Topic Regression to jointly incorporates social trust ensemble, topic modeling and probabilistic matrix factorization. We propose corresponding approaches to learning the latent factors both of users and items, as well as additional parameters to be estimated. Empirical experiments on Lastfm and Delicious datasets show that our model is better and more robust than the state-of-the-art methods on making recommendations in term of accuracy. Experiments results also reveal some useful findings to enlighten the development of recommender systems in social media. © 2016 Elsevier B.V. All rights reserved.

1. Introduction Social media is a group of Internet-based applications that build on the ideological and technological foundations of Web 2.0 [14], and allow content generation, dissemination and communication among communities of users [34]. Social media systems have a wide range of potential impacts on both application and research perspectives, for example, promoting sales for profit business, facilitating creativity, interaction and learning of individual users for information consumption. However, the abundance and popularity of social media sites flood users with huge volumes of information and hence present a significant challenge in terms of information overload. Too much information may make users helpless in their process of finding useful contents. Recommender Systems (RS) aim to mitigate the negative impact of information overload by filtering and providing users the most attractive and relevant items (such as photos, videos, music, articles, news, comments, tags, people, etc.) from informationsea. RS often use personalization techniques tailored to the needs and interests of the individual user, or the collective intelligence. ∗

Corresponding author. Tel.: +86 18687001878. E-mail addresses: [email protected] (H. Wu), [email protected] (K. Yue), [email protected] (Y. Pei), [email protected] (B. Li), [email protected] (Y. Zhao), [email protected] (F. Dong).

And various techniques, such as neighborhood-based collaborative filtering [25], matrix factorization [16], network-based approaches [12], and fuzzy-based collaborative filtering [36] have been made to automatically predict the interests of a particular user [3,17]. Social media and recommender systems can mutually benefit for each other [8]. On one hand, social media introduces evergrowing rich information, such as tags, ratings, comments, and friendship of users, which can be used to strengthen recommendations; On the other hand, recommender technologies can increase adoption, engagement, and participation of new and existing users, resulting in the success of social media applications. Consequently, how to utilize rich information in social media to enhance recommendation models, has become a hot issue of great interest to both academia and industries [7,8]. Existing works can be divided into two categories. One focuses exploiting item-specific contents, such as tags, comments, linkrelations, which can effectively deal with the cold-start recommendation for items; The other emphasizes the usage of user-specific information (mainly the trust relationship among users), which can effectively alleviate the cold-start recommendation for users. However, these works have utilized either content of items or social information of users, and few have considered them jointly. Naturally, a question will be asked whether and how the contents of items and social networks of users can be combined together to produce compound recommendation engines, and thus

http://dx.doi.org/10.1016/j.knosys.2016.01.011 0950-7051/© 2016 Elsevier B.V. All rights reserved.

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

JID: KNOSYS 2

ARTICLE IN PRESS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

have the best of both worlds. In this regard, there have been several exploratory works. Ye et al. [33] proposed a PLSA-like [10] probabilistic generative model, called unified model, which naturally unifies the ideas of social influence, collaborative filtering and content-based methods for item recommendation. Expectationmaximization algorithm is devised to learn the model parameters. However, the model can only exploit observed ratings, zero or unobserved ratings which may potentially reflect users’ interests cannot be exploited under this algorithmic framework, therefore may dramatically impact the predictive performance. CTRSMF [22] integrates Collaborative Topic Regression (CTR) with social matrix factorization models (SMF), to build a hybrid recommender system. Leveraging on matrix factorization techniques, CTR can deal with both explicit and unobserved ratings, by putting different confident numbers for them, and make CTR-based recommendation systems more powerful. Owing to the lack of physical explanation, directly factorizing social trust matrix in CTRSMF does not reveal the underlying relations among the users. Different from CTRSMF which utilizes homophile effect in social media to smooth the similarity of users’ interests to their friends, LACTR [13] directly learns how much attention users allocate to other users, and leverages these learned influence to make recommendations. LACTR is a sophisticated model, but it implicitly presets a strong condition that users’ social interactions usually follow topically similar contents. Although such condition may be satisfied easily in discussion-thread-oriented systems (e.g. Digg), it is not always accurate, resulting in that LACTR may be sensitive to different datasets. To deal with existing problems and exploit both social factors and content information for enhancing item recommendation, in this paper, we present a novel generative model CTRSTE by extending CTR which naturally incorporates content information via Latent Dirichlet Allocation [2] into collaborative filtering framework [27]. Different from previous studies, we suppose the users’ ratings on items are affected both by the personal tastes and their trusted friends’ favors, and naturally integrate this principle into the CTR model. We propose parameter learning method to infer latent factors both for users and items in the new model. Our experiments on large-scale datasets from Lastfm and Delicious show that our CTRSTE model significantly outperforms the state-of-the-art variants of CTR in term of accuracy. The main contribution of this paper consists of two folds. First, we extend CTR with trust ensemble principle for item recommendation in social media and show its effectiveness on two largescale datasets. It sets up a new algorithmic framework to seamlessly exploit user–item feedback, item content, and social network for building powerful recommender engines. Second, we compare social-enhanced variants of CTR from separate aspects of recommendation quality, such as the accuracy, diversity, novelty and then show their strengths and weaknesses on item recommendation in social media systems. All these can contribute to the practices of recommender systems. The remainder of this paper is arranged as follows: in Section 2, we provide an overview of related works on recommendation systems and Collaborative Topic Regression models. In Section 3, we propose our CTRSTE model and discuss how to learn parameters and do inference. The experimental results and discussion are presented in Section 4, followed by conclusions and future works in Section 5. 2. Related works 2.1. Social-based collaborative filtering Collaborative filtering (CF) has grown up to be a hot research topic due to its successful application in the

recommendation systems. In traditional CF methods, only the feedback matrix, which contains either explicit(e.g., ratings) or implicit feedback(e.g., tagging, clicks, purchases) on the items given by users, is used for training and prediction. Typically, the feedback matrix is sparse, which means that most users come into contact with a few items. Resulting from this sparsity problem, traditional CF with only feedback information will suffer from unsatisfactory performance. Recently, many researchers have proposed utilizing auxiliary information to alleviate the data sparsity problem in CF. Among them, the most popular approaches focus on exploiting social information as various social media systems are booming in Internet [18–20,32,35]. These methods increase the accuracy of traditional CF by taking collective interests and social trusts between users in an online social network as additional inputs. Social trust between a pair of friends (u, v) may be established based on explicit feedback of user u concerning user v (e.g., by voting or following), or it may be inferred from implicit feedback (e.g., the mutually shared resources/items between u and v). However, different algorithms explore social networks and the embedded social information differently. Ma et al. [18–20] have proposed three ways to integrate social information with matrix factorization process, namely SoRec, Social Trust Ensemble(STE) and social regularization. In SoRec [19]. The user–item feedback matrix and the user–user social matrix are simultaneously factorized by using shared user latent factors. In STE [18], the predicted rating for item i by user u is a linear combination of three terms: a global offset of ratings, prediction based on u’s and i’s latent factors, and a weighted sum of the predicted ratings for item i from all of user u’s friends. Social regularization model addresses the transitivity of trust in social networks, and exploits the social circles and users’ latent factors to create a term to regularize the matrix factorization process [20,35]. All of three models achieve better prediction accuracy than the original matrix factorization. Shambour and Lu [24] proposed a trust-semantic fusion-based recommendation approach for B2B e-service, where user-based trust-enhanced CF and item-based semantic-enhanced CF are fused to utilize trust intuitive property. By this it can alleviate the data-sparsity problem associated with users and items. Besides, some algorithms predict a user’s rating for an item by traversing the user’s neighborhood and querying the item ratings of her/his direct and indirect friends [32], e.g. Filmtrust [5], MoleTrust [21], TrustWalker [12].

2.2. Collaborative Topic Regression models Beyond exploiting social information, other methods often utilize item content information to enhance collaborative filtering. Collaborative Topic Regression (CTR) is one of these methods which have achieved promising performance by successfully integrating both feedback information and item content information [27]. The CTR model combines the merits of both probabilistic matrix factorization and topic modeling approaches. Here, we gradually restate the background approaches constructing the CTR model.

2.2.0.1. Probabilistic matrix factorization. In matrix factorization, users and items are both represented as latent vectors in a shared latent K-dimensional space, RK , where user i is represented as a latent vector ui ∈ RK and item j is represented as a latent vector v j ∈ RK . The prediction of whether user i will like item j is given by the inner product between their latent representations, ri j = uTi v j . To use matrix factorization for collaborative filtering, the latent representations of the users and items must be learned given an observed matrix of ratings. The common approach is to minimize the regularized squared error loss with respect to user

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

factors U = (ui )Ii=1 and item factors V = (v j )Jj=1 .

minU,V



ri j − uTi v j

2

+ λu ||ui ||2 + λv ||v j ||2 ,

(1)

i, j

where λu and λv are the regularization parameters. Matrix factorization can be generalized as a probabilistic model by placing zeromean spherical Gaussian prior on both latent factors of users and items [23], which can be further described as following generative process: 1. For each user i, draw user latent vector ui ∼ N (0, λ−1 u IK ); 2. For each item j, draw item latent vector v j ∼ N (0, λ−1 v IK ); 3. For each user–item pair (i, j), draw the rating ri j ∼ N (uTi v j , ci−1 ); j where cij serves as a confidence parameter for rating rij [11]. If cij is large, rij is trusted more. Generally, ci j = a if ri j = 1 and ci j = b if ri j = 0, a and b are tuning parameters satisfying a > b > 0. By this, probabilistic matrix factorization can deal with unobserved ratings. 2.2.0.2. Latent Dirichlet allocation. Topic models provide an interpretable low-dimensional representation of the documents. Here, we exploit the discovered topic structure by LDA for item recommendation in social media systems. Suppose there is a fix vocabulary W, referred to a set of tags used to annotate items in this paper. Assume there is K topics φ = φ1:K , each of which is a distribution over the set of tags. The generative process of LDA is as follows. For each item j in the corpus, 1. Draw topic proportions θ j ∼ Dirichlet(α ); 2. For each word(tag) wjn , (a) Draw topic assignment zjn ∼ Mult(θ ); (b) Draw word(tag) w jn ∼ Mult (φz jn );

2.2.0.3. Collaborative Topic Regression. CTR represents users with topical interests and assumes that items (documents) are produced by LDA model. In addition, CTR includes a latent variable  j which offsets the topic proportions θ j when modeling the user ratings.  can capture the item preference for a particular user taken into account their ratings. This is an important innovation compared with PLSA-based recommendation models where the deviation from users’ preference to topic distribution of items cannot be captured. The generative process of CTR model is shown as follows: 1. For each user i, draw user latent vector ui ∼ N (0, λ−1 u IK ); 2. For each item j, (a) Draw topic proportions θ j ∼ Dirichlet(α ); (b) Draw item latent offset  j ∼ N (0, λ−1 v IK ) and set the item latent vector as v j =  j + θ j ; (c) For each word(tag) wjn , i. Draw topic assignment zjn ∼ Mult(θ ); ii. Draw word(tag) w jn ∼ Mult (φz jn );

). 3. For each user–item pair (i, j), draw the rating ri j ∼ N (uTi v j , ci−1 j

The naive CTR model can be used for both coldstart prediction and non-coldstart prediction. For non-coldstart prediction, CTR uses the point estimate of ui , θ j and  j to approximate their expectations as [27], (2)

In coldstart prediction the item is new, and no other ratings are available. Thus, CTR predicts with

rˆi j ≈ uTi θ j ,

where the topic proportions θ j for a new item are further learned after learning U for all users and v ∈ V for all other items. CTR model makes a success of using content information for recommendation of items. However, this model does not exploit social information thus cannot reliably learn the user latent space for new or inactive users. To address this problem, some researchers have proposed different variants incorporating social information into the CTR. In CTRSMF [22], the authors integrate CTR with social matrix factorization models using a strategy similar to the SoRec. To take social correlation between users into account, the social matrix is simultaneously factorized with the rating matrix, and two parameters λs and λq are added to balance the social influence during modeling the dataset. Different from CTRSMF which utilizes homophile effect in social media to smooth the similarity of users interests to their friends, LACTR [13] directly learns how much attention users allocate to other users, and leverages these learned influence to make recommendations. In addition, Wang and Li [28] develop a novel hierarchical Bayesian model called Relational Collaborative Topic Regression (RCTR), which extends CTR by seamlessly integrating the user–item feedback information, item content information, and network structure among items into the same model. Although these methods have improved CTR in separate aspects, there remains an open issue with respect to how to integrate social information in the CTR model more effectively. 3. Collaborative Topic Regression model with social trust ensemble 3.1. Notations

For the parameter estimation of LDA, we can choose variational inference [2] or Gibbs sampling [6]. The learned topic proportions θ are item specific, but the set of topics φ is shared by all items.

rˆi j ≈ uTi (θ j +  j ) = uTi v j .

3

(3)

Suppose that there are I users and J items in a social media system. We use matrix R to represent the user–item feedbacks, ri j = 1 if user i = 1 : I rated item j = 1 : J , and ri j = 0 otherwise. Item j’s contents is characterized by a sequence of words Wj . Following matrix factorization methods, we represent users and items as latent vectors in a shared latent K-dimensional space, RK , where user i is represented as a latent vector ui ∈ RK and item j is represented as a latent vector v j ∈ RK . The notations are summarized in Table 1. Note that, to facilitate understanding the following calculation process, ui , vj and θ j are viewed as the column-vectors of dimension K × 1, and then the dimensions of matrices U, V, θ and S are K × I, K × J , K × J and I × I, respectively. Table 1 Notations. Symbols

Description

U(ui ) V(vj ) θ (θ j ) φ (φ k ) S(Sil ) R(rij ) W(wjn ) rˆi j C(cij )

User latent factors (for ith user ) Item latent factors (for jth item ) Item-topic distribution(for jth item) Topic-word distribution(for kth topic) Social matrix (social trust of user i to user l) Rating matrix (binary rating of i to j) Word assignments (nth word assigned to item j) Predicted rating of user i to item j Confidence matrix corresponding to R Hyper-parameter of θ Hyper-parameter of φ Hyper-parameter of U Hyper-parameter of V Balance factor for social influence Number of latent factors(topics) Maximum number of iterations Multinomial distribution Normal distribution Dirichlet distribution The set of users who i trusts The set of users who trust i

α β λu λv

d K maxIter Mult(·) N (·, · ) Dirichlet(·) Fi+ Fi−

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS 4

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

3.2. Model details From a psychological point of view, users have a herd mentality, thus can be easily influenced by the friends they trust, and prefer the recommendations from them. On the other hand, users have their own characteristics, and they have different tastes on different items, such as movies, books, etc. Therefore, one user’s final decision on adopting items in social media systems is the balance between his/her own taste and his/her trusted friends’ favors [18]. To reflect this observation in the recommendation systems more accurately and realistically, we model a predicted rating from user i to item j as followings:

rˆi j = duTi v j + (1 − d )



Sil uTl v j .

(4)

l∈Fi+

The first term is the prediction based on user i’s and item j’s latent features. The second term captures the social influence, and is defined as a weighted sum of the predicted ratings for item j from all of user i’s trusted friends, where Fi+ is the set of users who i directly trusts. The trade-off between the feedback data (ratings) and the influence of the social network is determined by d ∈ [0, 1]. Obviously, the social influence is ignored for d = 1, while d = 0 assigns the highest possible weight to the social influence. Intermediate values of d result in a weighted combination of the information from both sources. This is why the model is named as social trust ensemble (STE). Sil is the normalization term of trust score from user i to user l, and is estimated by

sim(i, l ) Sil = |Fi+ |



|Fl− | , + |Fi | + |Fl− |

(5)

Fig. 1. Graphical model of CTRSTE.

latent factors U, V, and θ under the full model can be shown as,

p(R, W, U, V, θ |λu , λv , α , φ , d, S ) = p(U |λu ) p(V |θ , λv ) p(R|U, V, d, S ) p(W, θ |α , φ ) =

I 

Rˆ = [dI + (1 − d )S]U T V,

(6)

where I is the identity matrix and Sii = 0. This formulation naturally employs the user–item feedback matrix and the users’ social trust network for the recommendations. Compared with CTRSMF [22], it can reveal the underlying relations among the users, and provide an intuitive explanation for predictions. Compared with LACTR which emphasizes utilizing topical-level social influence [13], we consider social influence between users from the macro point of view. Although it is not sophisticated in methodology, it is more robust in practices and can adapt to different conditions. For example, LACTR will hardly work if there is no available social network, but our model can still make predictions even without the social information. To exploit the principle of STE in the CTR model for recommendation, we further model the conditional distribution of rij given ui  and vj as a Gaussian N (duTi v j + (1 − d ) l∈F + Si,l uTl v j , ci−1 ) instead j

i=1

×

i=1 j=1

J I  

×

N J  

 N ri j |duTi v j + (1 − d )

 l∈Fi+

(7) By this, new model has a similar generative process to the CTR model. Correspondingly, in the generative process of the new model, the joint likelihood of observed data, R and W, and the

N



v j |θ j , λ−1 v IK

j=1

N ri j |duTi v j + (1 − d )

 log



K 



Sil uTl v j , ci−1 j

l∈Fi+

θ jk φk,w jn ,

(8)

k=1

where p(U|λu ) is derived by placing zero-mean spherical Gaussian prior on user latent factors [23], p(V|θ , λv ) is derived by placing θ -mean Gaussian prior on item latent factors to enable LDA-based regularization [27], λu and λv are the precision of the corresponding Gaussian distribution. The likelihood of the text descriptions p(W, θ |α , φ ) is derived from LDA [2], where Dirichlet prior α is set to 1 to keep the computation simple. We call this new generative model as CTR with Social Trust Ensemble (CTRSTE) to distinguish from the CTRSMF model where trust between users in a social network is integrated by cofactorizing the social trust matrix S with the rating matrix R. The graphical model of CTRSTE is shown in Fig. 1. 3.3. Parameters learning Given a training data set, we want to find the Maximum a Posteriori (MAP) estimate of U, V, so we can use U and V to predict the missing entries in R and use the predictions to do recommendation. For learning the parameters of CTRSTE, we develop an EMstyle algorithm similar to [27]. Maximization of the posterior is equivalent to maximizing the complete log-likelihood of U, V, θ , R, W given λu , λv , φ , S and d. Namely, we take log-likelihood of formula (8) as the objective function and optimize it.

L=−

Sil uTl v j , ci−1 . j

J 



i=1 j=1

i

of N (uTi v j , ci−1 ). Then, the probability of full ratings R given U, V, d j and S is assumed to be factorial,

p(R|U, V, d, S ) =



j=1 n=1

where sim(i, l) is the similarity between i and l, |Fi+ | the number of users who i follows/trusts, and |Fl− | is the number of users who follow/trust user l. The measure resembles the combination of a homophily-based trust and an expertise-based trust to enhance trust estimation among users [15]. Eq. (4) can be rewritten using matrix notation:

I  J 



N ui |0, λ−1 u IK ×

+

λu  2

uTi ui −

i

 j

 log

n

λv  2

 k

j

( v j − θ j )T ( v j − θ j )



θ jk φk,w jn −

  ci, j (ri j − rˆi j )2 2 i

(9)

j

We optimize this objective function by gradient ascent approach same to the original CTR model, by iteratively optimizing the collaborative filtering ui , vj and topic proportions θ j . For ui , vj maximization follows similar to matrix factorization [11]. Given a current estimate of θ j , taking the gradient of L with respect to ui

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

and vj and setting it to zero helps to find ui , vj in terms of U, V, C, S, R, λv , λu and d. Solving the corresponding equations will lead to updating equations. The update equation for ui is derived as followings, we let d = 1 − d for simplicity.





+

m∈Fi−

cm j (rm j − rˆm j

j

= −λu ui − d2

+d







ci j ri j − d

j

+d

  m∈Fi−

= −λu ui − d2 +d



cm j rm j − duTm v j

m∈Fi−



ci j (ri j − d

Sil uTl

+

 

ci j uTi v j v j − d2

j

  m∈Fi−

cm j rm j − duTm v j − d





∂L  = duT v + d = 0, rm Smn uTn v j m j j ∂ ui + n∈Fm \i



λu IK + d2

j

=d

 j

+d

ui ←



ci j v j vTj + d2

m∈Fi−





ci j ri j − d

  m∈Fi−

Smn uTn v j Smi v j



Sil uTl v j

vj

3.4. Prediction and recommendation

cm j v j vTj ui

After all the optimal parameters for U, V, θ 1: J and φ are learned, the CTRSTE model can be used for both coldstart and non-coldstart prediction. Let D be the observed data. Each prediction is generally estimated as,

 )S v cm j (rm j − rm j mi j

j

T

−1



2

d



ci j ri j v j − d (1 − d )

j

+ (1 − d )



ci j

j

  m∈Fi−

cm j (rm j



+ λu IK

2 Smi VCmV T

E[ri j |D] ≈ E



Sil uTl v j v j

l∈Fi+

+ (1 − d )





Sil uTl

|D (E[θ j |D] + E[ j |D] ).

(14)

For non-coldstart prediction, we use the point estimate of ui , ul (l ∈ Fi+ ), θ j and  j to approximate their expectations,



− r

m j )Smi v j



(10)

j

rˆi j ≈

duTi

+ (1 − d )





Sil uTl

(θ j +  j )

l∈Fi+



= duTi v j + (1 − d )

Sil ul

i

ci j ri j u i −

Sil uTl v j .

(15)

With respect to item-specific coldstart prediction, the item is new and no other ratings are available. Thus, E[ j ] = 0 and CTR predicts with

 ∂L T = −λv v j + λv θ j + ci j (ri j − u i v j )u i ∂v j i 



l∈Fi+

l∈Fi+

= −λv v j + λv θ j +

duTi

l∈Fi+

The update equation for vj is derived as followings:

Let u i = dui + (1 − d )

(13)

j





ψ jnk 1[w jn = w].

n

On our data, we found that simply fixing θ j as the estimate from naive LDA gives comparable recommendation performance. This is also observed in [27]. Thus, we can omit the updating process of θ and φ to save computation. We summarize the algorithmic details for learning parameters of CTRSTE in Algorithm 1.



m∈Fi−



Let

φkw ∝

j

l∈Fi+

d VCiV + (1 − d ) 2

2 Smi

(12)

optimize θ j analytically, so we use projection gradient approaches to optimize θ 1: J and other parameters U, V, ψ 1: J . After estimating U, V, ψ , we can update φ by following the same M-step update as in LDA [27],



Let

ψ jnk (logθ jk φk,w jn − logψ jnk )

k

The optimal ψ jnk satisfies ψ jnk ∝ θ jk φk,w jn . Note that, we cannot

j

+ n∈Fm \i

j

( v j − θ j )T ( v j − θ j )

2 j 

= L ( θ j , ψ j ).

2 T cm j Smi ui v j v j

v j )v j



λv 

n

l∈Fi+

j

+d

L (θ j ) ≥ −



ci j ri j u i + λv θ j

i

Let U  = U[dI + (1 − d )S]

Smn uTn v j − d Smi uTi v j Smi v j

+ n∈Fm \i



vj

l∈Fi+

j



− d

Sil uTl v j



J

ci j uTi v j v j 

+ λv IK

i



elements and Ri = (ri j ) j=1 for user i. For item j, Cj and Rj is similarly defined. cij is confidence parameter for rating rij . We use the same strategy as stated in [27] to set cij : ci j = a if ri j = 1 and ci j = b if ri j = 0. For simplicity, we fix a = 1 and b = 0.01, respectively. The formula in Eq. (11) shows how topic proportions θ j affects the item latent vector vj , where λv balances this effect. Given U and V, we can learn the topic proportions θ j . We define q(z jn = k ) = ψ jnk and then we separate the items that contain θ j and apply Jensen’s inequality:

v

mi j

j

−1  T ci j u i u i

where Ci is a diagonal matrix with ci j , j = 1, . . . , J as its diagonal

i

)d  S

vj ←



v j ← (U C jU  T + λv IK )−1 (U C j R j + λv θ j )

  ∂L = −λu ui + ci j ri j − duTi v j − d Sil uTl v j dv j ∂ ui j l∈F +  



5



rˆi j ≈ uTi θ j .

T ci j u i v j u i

i

  ∂L T = 0, ci j u i u i v j + λv v j = ci j ri j u i + λv θ j ∂v j i i

(11)

(16)

To obtain the topic proportions θ j for a new item, we optimize Eq. (12). With respect to user-specific coldstart prediction, the user is new and no other ratings are available. Thus, E[ui ] = 0 and CTR

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

JID: KNOSYS 6

ARTICLE IN PRESS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12 Table 2 Statistics of datasets.

Algorithm 1 Learning parameters of CTRSTE model. Input: α , β , d, λu , λv , a, b, R, S, K, maxIter, threshold; Output: U, V , θ , φ ; 1: Given α , β and K, initialize θ and φ using LDA model; 2: Initialize matrix S using Eq. (5), matrices C∗ according to R; 3: while iter < maxIter do 4: Lold ← L; 5: for each ui ∈ U do 6: if |Fi+ | = 0&&|Fi− | = 0 then 7: ui ← (VCiV T + λu IK )−1VCi Ri ; 8: else 9: Update ui using Eq. (10); 10: end if 11: end for 12: U  ← U[dI + (1 − d )S]; 13: for each v j ∈ V do 14: Update v j using Eq. (11); 15: Update θ j using Eq. (12) (optional); 16: end for 17: Update φ using Eq. (13) (optional); 18: Compute L; |L−L | 19: if L old < threshold then old

20: 21: 22: 23: 24:

break; end if iter = iter + 1; end while return U, V , θ , φ ;

predicts with

rˆi j ≈



Sil uTl v j .

(17)

l∈Fi+

In this case, we let d = 0 and make prediction based on the friends’ favors of the current user. For the case of both the item and the user are new, we predict with

rˆi j ≈



Sil uTl θ j .

(18)

Dataset

Lastfm

Users Items Tags User-Item Relations User-User Relations User-Tag-Item Relations Tag assignments per item Sparsity of rating matrix Sparsity of social matrix Ratio Training users Testing users Ratio Training users Testing users

1892 12,523 9749 71,064 25,434 186,479 14.89 99.70% 99.29% 1627 482 1821 358

Delicious 1077 45,419 30,568 55,663 11,704 226,260 4.98 99.90% 99.01% 80% 1011 376 90% 1065 335

tagging, and resource consuming information from sets of users.3 Statistics of two datasets are listed in Table 2. To meet the online operation principle of recommender systems, where recommendation systems periodically provide active users with items of interest, at a certain point of time, using the historical data of the systems, experimental dataset is divided into two parts according to the tag assignment timestamp (a tag assignment corresponds to a user-tag-item relation together with a unique timestamp): Given a ratio , for example ratio = 80%, the training set contains the past 80% percent of entries of tag assignments and the future 20% percent of entries of tag assignments makes up the testing set. New users and items are removed from the testing datasets without considering the cold-start recommendation. The number of users for training and testing given specific ratio is also presented in Table 2. In the training sets, tag assignments of each item are considered as the input of the LDA model to achieve item-topic distribution (θ ) and topic-word distribution (φ ). Also, only the social information of users existed in the training sets is exploited in the recommendation process. In addition, when generating the recommendation candidate list for a specified user, the items already collected by the target user are exempt from the list. Note that, we do not use cross-validation, since temporal factor is ignored in this process as stated in [31].

l∈Fi+

But it must satisfy a premise where the user must have trusted friends and the item must have content information. For item recommendation, we use rˆi j to rank all candidate items, and return a ranked list of candidates for the user i. 4. Experiments 4.1. Datasets For experiments, we use the real datasets collected from two well-known social media systems: Lastfm1 and Delicious.2 Lastfm is the world’s largest online music catalogue, and allows user tagging music tracks and artists. In the dataset, we take artists as items. Delicious is one of the most popular social bookmarking web sites, which allow users not only to store and organize personal bookmarks (URLs), but also to look into other users collections and find what they might be interested in by simply, keeping track of the baskets with tags or resource. For both systems, we use their refined data collections released in HetRec2011 [4] to make an evaluation. Both datasets contain social networking,

1 2

http://www.lastfm.com. http://www.delicious.com.

4.2. Evaluation metrics Accuracy is one property of the recommender system which characterizes whether generated recommendations accurately match user’s interests/preferences or not [31]. Since we focus on recommending top-N items instead of rating-prediction of items, Precision@N and Recall@N are selected to evaluate the recommendation accuracy. Given a rank list of recommended items, Precision@N is the fraction of relevant items in the top-N position (see Eq. (19)),

P@N =

|{relevant items} ∩ {top − N items}| N

.

(19)

while, Recall@N is the fraction of relevant items returned in the top-N position, to the true number of relevant items that should have been returned (see Eq. (20)),

R@N =

|{relevant items} ∩ {top − N items}| . |{relevant items}|

(20)

The quality of recommendations can be evaluated along a number of dimensions, and relying on the accuracy of recommendations alone may not be enough to locate the most useful items for

3

http://grouplens.org/datasets/hetrec-2011/.

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

each user. In particular, the importance of diverse and novel recommendations has been emphasized recently [1,30]. There are two ways for quantifying the diversity of recommendations, namely, inner-diversity and inter-diversity. Inner-diversity emphasizes accumulated dissimilarity of content between all pairs of items in a recommendation list τ , to avoid the monotony of content, since a too narrow array of choices may often do not satisfy the user’s requirements [31]. To measure the inner diversity of a recommendation list τ , EILD@N, a distance-based metric proposed by Vargas and Castells [26] is used, N 

EILD@N =

Ck disc (k )disc (l |k )dist ( jl , jk ),

(21)

jk ∈τ , jl ∈τ ,l=k

 Ck = C/ j ∈τ − j disc (l |k ) and disc (l |k ) = disc (max(1, l − k )) reflects l k a relative rank discount between l and k, and dist(jl , jk ) is the dissimilarity between two items, given by: dist ( jl , jk ) = 1 − cos( pro f ile( jl ), pro f ile( jk )), where both jl and jk are represented by a tag vector(i.e. profile) where the tags are weighted using tag frequency-inverse item frequency [31]. Different from inner-diversity, inter-diversity considers the dissimilarity between pairs of recommendation lists across all users under considerations, thus characterizes the uniqueness of different user’s recommendation lists [31]. The benefits of recommender systems that provide higher inter-diversity would be apparent to both users and business providers. On one hand, a recommender system providing higher inter-diversity usually promotes the consumption of non-popular items; one the other hand, it makes a higher personalization of users’ recommendation lists [30]. We adopt the hamming distance (defined as Eq. (22)) to measure the aggregate diversity of recommendations,

HD@N = 1 −

overlap@N , N

(22)

where overlap@N is the number of common items in the topN places of recommendation lists of two users. Generally, novel recommendation helps stimulate the interest and enthusiasm of users, and then improves the vitality of recommender systems [31]. To measure the novelty of recommendations, we use a popularitybased item novelty model (see Eq. (23)) proposed in [26],

EPC@N = C

N 

disc (k )(1 − p(seen| jk )),

(23)

jk ∈τ

 where disc (k ) = 0.85k−1 is a rank discount and C = 1/ Nj ∈τ disc (k ) k is a normalizing constant. Similar to EILD@N, EPC@N is also ranksensitive, since their scores of the top-rated items counts more than those of other items. The probability of an item j being seen |{i∈U |r >0}|

ij . is estimated as: p(seen| j ) = I Note that, we average every evaluation metric over the test set to measure the quality of recommendations.

4.3. Experimental results on Lastfm To facilitate comparisons among different models, we empirically fix some uniform parameters with respect to CTR [27]. In all of the experiments we conducted, we set the parameters of the LDA model as α = 1, β = 0.01, and utilize a Gibbs sampler4 to infer θ , φ from the training sets. We also leave the parameters a = 1, b = 0.01. The similarity between two users i and l is calculated using Jaccard-like metric, as showed in Eq. (24) where Ni and Nl are respectively the set of items rated by user i and user l. The convergence threshold for Algorithm 1 is fixed as 1e − 4. The maximum 4

http://www.arbylon.net/projects/LdaGibbsSampler.java.

7

number of iterations in Algorithm 1 is fixed as 200.



sim(i, l ) =

|Ni ∩ Nl | |Ni ∪ Nl |

(24)

4.3.1. Recommendation accuracy of CTRSTE We first investigate the recommendation accuracy of CTRSTE with different settings of d and K (number of topics), and concentrate the top-20 results for both P@N and R@N, as users always concern about a few top-ranked items. We find the optimal parameters λu = 0.01, λv = 100 for the Lastfm dataset by using grid search on held out recommendations. The experimental results are presented in Figs. 2 and 3, where we respectively use ratio = 90% and ratio = 80% to separate the whole dataset into a training subset and a testing subset. Depending on Figs. 2 and 3, both precision@n and recall@n are significantly improved as d and K increase. When d is subsequent to 1, the impact of users’ social relations gradually reduces until there is no effect. With the increment of K, more meaningful topics appear to have found, which contributes to improving the granularity of user interest model, and thus improving performance of recommendations. The best recommendations can be generated when d takes an intermediate value between 0 and 1. This confirms the speculation that user’s decision is the balance between their personal tastes and the favors of trusted friends. However, the situation is a little different in considering the scale of training data. When setting ratio = 90%, more content information are used in prediction, thus users’ personal tastes play a greater role than the favors of trusted friends(the optimal setting of d ranges from 0.9 to 0.94). While setting ratio = 80%, the social circles of users has more influence on their decisions by adopting items than users’ personal tastes. When d takes values which are larger than 0.5, significant performance gains can be noted. 4.3.2. Comparision with other CTR-based models We compare CTRSTE with other CTR-based counterparts using different quality indicators of item recommendation. We consider CTRSMF [22] and LACTR [13] as the baseline methods, since both use social information to enhance CTR. Also, CTR is taken into account as a special case of CTRSTE (where d = 1). In CTRSMF, we set the parameters λu = 0.01, λv = 100, λq = 100 to achieve the best performance in terms of recall [22]. In LACTR, we set the parameters λu = 0.01, λv = 100, λs = 0.01, λφ = 1 and utilize i’s attentions (represented by φ i in [13]) instead of i’s interests, ui , to predict ratings. The settings of parameters w.r.t CTRSTE and CTR are the same as in the former section. The experimental results are provided in Table 3, where we respectively use ratio = 90% and ratio = 80% to build the training dataset and the testing dataset. We let N = {5, 10, 20} to observe precision and recall, N = {10, 20} to investigate inner-diversity, inter-diversity and novelty, as three indicators do not significantly change with an increment of N. Being dependent on both tables, CTRSTE achieves the best accuracy in both P@N and R@N, followed by CTRSMF. CTR performs best in the novelty metric-EPC@N and the inter-diversity metric-HD@N. LACTR is unable to compare with others in terms of accuracy even if it achieves best inner-diversity (EILD@N). Generally, CTRSTE improves the baselines by least 4.6% in terms of precision, by least 6.24% in terms of recall. Simultaneously, the novelty is basically kept, and the aggregate diversity is slightly lowered. Particularly, we found that the CTRSTE performs superior to the baseline methods in term of recall. It implies that CTRSTE recover more pertinent items to users by mining social networks. In addition, we noticed that there exists a contradictory relationship between the inner-diversity and the accuracy of recommendations [31], as CTRSTE improves the precision and recall while reduces the EILD measure prominently. Yet, we can use a

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS 8

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

Lastfm(90%:10%)

Lastfm(90%:10%)

0.06

0.05

0.05

0.04

P@20

P@10

0.04 0.03

0.03

0.02

0.02 0.01

0.01 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

0

1 K=50

K=100

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

1

K=200

Lastfm(90%:10%)

Lastfm(90%:10%)

0.12

0.16 0.14

0.1

0.12 0.1 R@20

R@10

0.08 0.06

0.08 0.06

0.04

0.04 0.02 0

0.02 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

0

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

1

Fig. 2. Recommendation accuracy of CTRSTE on the Lastfm dataset with ratio=90%.

Lastfm(80%:20%)

Lastfm(80%:20%)

0.07

0.06

0.06

0.05 0.04

0.04

P@20

P@10

0.05

0.03

0.02

0.02

0.01

0.01 0

0.03

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

0

1 K=50

K=100

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d K=200

Lastfm(80%:20%)

1

Lastfm(80%:20%)

0.12

0.16 0.14

0.1

0.12 0.1 R@20

R@10

0.08 0.06

0.08 0.06

0.04

0.04 0.02 0

0.02 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.92 0.94 d

1

Fig. 3. Recommendation accuracy of CTRSTE on the Lastfm dataset with ratio=80%.

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

Delicious(90%:10%)

Delicious(90%:10%) 0.05

0.04

0.04

0.03

0.03

P@5

P@10

0.05

0.02

0.01

0

9

0.02

0.01

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

0

1 K=50

K=100

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d K=200

Delicious(90%:10%)

1

Delicious(90%:10%)

0.04

0.07

0.035

0.06

0.03

0.05 R@10

R@5

0.025 0.02

0.04 0.03

0.015 0.02

0.01

0.01

0.005 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

0

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

1

Fig. 4. Recommendation accuracy of CTRSTE on the Delicious dataset with ratio = 90%. Table 3 Recommendation performance of CTR variants as K = 200 on the Lastfm dataset, where the best value for each indicator is marked in bold font. The results for CTRSTE are obtained from the setting ratio = 90%, d = 0.94 and ratio = 80%, d = 0.92, respectively. Ratio

Methods

P@5

P@10

P@20

R@5

R@10

R@20

EILD@10

EILD@20

HD@10

HD@20

EPC@10

EPC@20

90%

CTR CTRSMF LACTR CTRSTE CTR CTRSMF LACTR CTRSTE

0.06089 0.05978 0.04302 0.06369 0.06639 0.07303 0.05394 0.07801

0.04721 0.04553 0.03743 0.05056 0.05705 0.06245 0.04585 0.06515

0.03757 0.03771 0.03017 0.04176 0.04606 0.04865 0.03672 0.05280

0.06280 0.06287 0.04699 0.07543 0.05286 0.06250 0.04406 0.07188

0.08917 0.09550 0.07674 0.11008 0.08409 0.09225 0.06848 0.10623

0.12625 0.13755 0.10584 0.14613 0.12636 0.13272 0.09466 0.15072

0.67200 0.70061 0.70609 0.64782 0.65438 0.69970 0.71934 0.62695

0.69420 0.71821 0.72509 0.66994 0.67858 0.71676 0.73736 0.64982

0.98035 0.97185 0.87493 0.96393 0.97968 0.96832 0.87192 0.95774

0.97240 0.95779 0.85143 0.95378 0.97228 0.95643 0.84791 0.94962

0.97247 0.96717 0.94586 0.96696 0.97233 0.96633 0.94630 0.96579

0.97383 0.96846 0.94860 0.96862 0.97372 0.96784 0.94904 0.96764

80%

general diversification method to post-process every recommendation list τ [9] to improve inner-diversity (readers can refer to the work [9] for more details).

of friends, without considering different scales of training data. The best settings of d are ranged from 0.96 to 0.98. In another word, users’ decisions on adopting items are characterized by themselves. |N ∩N | To better explain this issue, we use Jaccard index (i.e., |Ni ∪Nl | ) to

4.4. Experimental results on Delicious

estimate the item-specific similarity among users. For the Lastfm dataset, the average similarity of users is 0.00278, and the average similarity of friends is 0.01795. For the Delicious dataset, the corresponding scores are 0.00022 and 0.00660. Obviously, Lastfm users are more consistent with their trusted friends in term of musical interests. Moreover, by comparing Figs. 2 and 3 with Figs. 4 and 5, it is easily found that relying only social circle ( as d = 0 ) achieves good results in the Lastfm dataset yet it does not work well in the Delicious dataset. These provide us with the reasonable explanation for why considering social influence benefits to the Lastfm more than the Delicious.

4.4.1. Recommendation accuracy of CTRSTE We first examine the recommendation accuracy of CTRSTE with different settings of d and K, and concentrate the top-10 results for both P@N and R@N. We find the optimal parameters λu = 0.01, λv = 0.01 for the Delicious dataset by using grid search on held out recommendations. Other settings are same as in the Lastfm dataset. The results are presented in Figs. 4 and 5, where we use ratio = 90% and ratio = 80% correspondingly. Basically, both P@N and R@N are improved as d and K increase according to Figs. 4 and 5. Also, better recommendations can be achieved by balancing the individual tastes and the friends’ favors of users. These outcomes are in accordance with prior observations on the Lastfm dataset. However, the situation in the Delicious dataset quite differs from that of the Lastfm dataset. In the Delicious dataset, individual tastes play a greater role than the favors

i

l

4.4.2. Comparision with other CTR-based models We compare CTRSTE with CTR, CTRSMF [22] and LACTR [13]. In CTRSMF, we set the parameters λu = 0.01, λv = 0.01, λq = 0.05 to achieve the best performance in terms of recall [22]. In LACTR, we set the parameters λu = 0.01, λv = 0.01, λs = 0.01, λφ = 1 and uti-

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS 10

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

Delicious(80%:20%) 0.06

0.05

0.05

0.04

0.04 P@10

P@5

Delicious(80%:20%) 0.06

0.03

0.03

0.02

0.02

0.01

0.01

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

0

1 K=50

K=100

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

K=200 Delicious(80%:20%)

0.03

0.06

0.025

0.05

0.02

0.04 R@10

R@5

Delicious(80%:20%)

0.015

0.03

0.01

0.02

0.005

0.01

0

0

1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

0

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.96 0.98 d

1

Fig. 5. Recommendation accuracy of CTRSTE on the Delicious dataset with ratio = 80%. Table 4 Recommendation performance of CTR variants as K = 200 on the Delicious dataset, where the best value for each indicator is marked in bold font. The results for CTRSTE are obtained from the setting ratio = 90%, d = 0.96 and ratio = 80%, d = 0.98, respectively. Ratio

Methods

P@5

P@10

P@20

R@5

R@10

R@20

EILD@10

EILD@20

HD@10

HD@20

EPC@10

EPC@20

90%

CTR CTRSMF LACTR CTRSTE CTR CTRSMF LACTR CTRSTE

0.04239 0.04597 0.04000 0.04836 0.05053 0.04894 0.05691 0.05106

0.04507 0.04806 0.04507 0.04478 0.04840 0.04814 0.05319 0.05027

0.04030 0.04313 0.04060 0.03940 0.04521 0.04415 0.04628 0.04574

0.03251 0.03054 0.02703 0.03617 0.02146 0.02165 0.02915 0.02202

0.06150 0.05797 0.05269 0.06325 0.03745 0.03820 0.04549 0.04190

0.09617 0.09966 0.09209 0.09811 0.06814 0.06705 0.07231 0.07190

0.91374 0.91730 0.93710 0.91327 0.90281 0.91511 0.93320 0.90549

0.92184 0.92500 0.94163 0.92165 0.91222 0.92305 0.93609 0.91428

0.99888 0.99811 0.99675 0.99884 0.99836 0.99750 0.99662 0.99835

0.99836 0.99797 0.99637 0.99835 0.99818 0.99744 0.99626 0.99816

0.99845 0.99844 0.99784 0.99838 0.99859 0.99857 0.99798 0.99853

0.99850 0.99849 0.99793 0.99843 0.99862 0.99860 0.99805 0.99857

80%

lize users’ attentions to predict ratings. The settings of parameters with respect to CTRSTE and CTR are the same as in Section 4.4.1. The experimental results are provided in Table 4. We let N = {5, 10, 20} to observe P@N and R@N, N = {10, 20} to study EILD@N, HD@N and EPC@N. As for the case of ratio = 90%, we find both CTRSTE and CTRSMF work well than CTR and LACTR in terms of precision and recall. CTRSTE wins all of others on P@5, R@5 and R@10. CTRSMF performs best in P@10, P@20 and R@20. As for the case of ratio = 80%, LACTR achieves outstanding performance in terms of precision and recall. Also, it performs best in innerdiversity(EILD@N). Although CTRSTE ranks in the second class, it gradually approaches to LACTR in the accuracy as top-N increases. By combining with experimental results on the Lastfm dataset, we can get more conclusions. By introducing social trust ensemble into CTR model, CTRSTE significantly improve the recommendation accuracy of CTR. Compared to CTRSMF and LACTR, CTRSTE has not always been the winner, however, it can yield better and more consistent results over different datasets. And for both Delicious and Lastfm, users’ personal tastes have more influence on their decisions for adopting items than their social circles. Future studies on recommender systems need to consider this point more.

4.5. Complexity analysis Since CTR utilizes LDA for topic modeling, the time complexity of its variants is quite expensive when opposed to the traditional matrix factorization. Here, we empirically analyze the computational complexity of CTRSTE model. We implement all methods based on Parallel Colt,5 which is a Java library for high performance scientific computing [29]. To reduce computational costs when updating ui and vj , we adopt the same strategy of matrix operation shown in [11]. For updating vj , we rewrite UC j U T = U (C j − bIK )U T + bUU T and pre-compute bUUT for each item. Similarly, we can rewrite VCi VT for updating ui . As mentioned earlier, fixing θ as the estimate from naive LDA gives comparable recommendation performance, so we omit updating θ and φ to save computation. Fig. 6 shows the time costs of CTR and CTRSTE with different settings of K and ratio in our experiments, where all methods are executed on a computer powered by a 3.2 GHz Intel Core i5 processor and 16GB memory. The convergence threshold

5

http://sourceforge.net/projects/parallelcolt/.

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

ARTICLE IN PRESS

JID: KNOSYS

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

Lastfm(90%:10%)

Lastfm(80%:20%)

150

100 80

100

60 40

50

20 0

seconds/iter

iters

0

Delicious(90%:10%) 200

150

150

100

100

50

50

seconds/iter CTR−50

iters CTRSTE−50

iters

Delicious(80%:20%)

200

0

seconds/iter

0

seconds/iter CTR−200

iters CTRSTE−200

Fig. 6. Computational overheads of CTR and CTRSTE with different K. CTR-50 indicates K = 50 and so on. For other parameters, the settings are the same as Tables 3 and 4.

for learning parameters is fixed as 1e − 4. The seconds/iter represents the average time taken for each iteration when learning parameters, iters is the total number of iterations spent. Given fixed K, the time cost per iteration for CTRSTE is less than a third times that of CTR, and the number of iterations basically are common to the CTR model. According to update rules (11) and (12), CTRSTE has a same complexity with CTR on updating the latent factors of items V, but spends more times to update the latent factors of users U, where for each user i, the out-links of trust Fi+ , the in-links of trust Fi− and implicit trust transitivity need to be considered. Since the average number of out-links of trust |F + |, and the average number of in-links of trust |F − | are relatively small. They are around 13 in the Lastfm dataset, and around 11 in the Delicious dataset. Thus, CTRSTE does not add much computational overhead compared to CTR. Also, we noticed that the growth of the time complexity is mainly determined by the value of K both for CTR and CTRSTE. In each iteration of Algorithm 1, we solve equations of the form Ax = B by I times to update ui and by J times to update vj , where A is a K × K matrix and B is a K × 1 vector. This constructs a key part of computational overhead with a time complexity of O[(I + J )K 3 ]. To accelerate the computation of CTRSTE, we can use a serial strategies: (a) using a smaller value for K(it runs much faster by sacrificing some accuracy of recommendation); (b) using a smaller convergence threshold in parameter learning(e.g., the use of 1e − 4 can be 2 × to 3 × faster than 1e − 6 in the number of iterations, at the expense of a slight performance losses); and (c)utilizing massively parallel computing platforms (the learning algorithm can be easily parallelized in Hadoop or Spark). 5. Conclusions and future works We have proposed CTRSTE for item recommendation in social media systems. By introducing social trust ensemble into CTR model, CTRSTE can seamlessly integrate user–item feedback, item content, and social network into one algorithmic framework for the task of item recommendation. Although experiments on different datasets show some different properties, the results still demonstrate that CTRSTE consistently yields better recommendation results than the naive CTR model. Compared with CTRSMF and LACTR, CTRSTE model is simple in algorithmic principle but

[m5G;February 1, 2016;13:37] 11

performs more robust over different datasets. Also it can provide more intuitive explanations for rating prediction. Some other useful findings can be drawn from the experimental results: (i) exploiting social network information in the CTR model can significantly improve recommendation accuracy, particularly the recall indicator; (ii) if users are more consistent with their trusted friends in terms of interests, CTRSTE seems to provide more better recommendations; (iii) social trust ensemble is more flexible and robust than other approaches for integrating social network with the CTR model; and (iv) users’ personal tastes have more influence on their decisions of adopting items than their social circles in social media systems. As for future work, we would wish to examine more advanced measures which qualify social trust between users on CTRSTE to investigate recommendation effectiveness. Also, it is worthwhile developing a methodology to automatically find the optimal setting of d. In addition, it would be helpful to provide CTRSTE the ability of diversification to improve the inner-diversity of recommendations. Finally, non-zero ratings currently are not used in training recommendation models, since rating matrix always contains a huge number of non-zero entries and leads to high computational overheads in training the models. However, CTRSTE provides an inherent mechanism exploiting them. We can utilize sampling method to choose a subset of the non-zero ratings for model training, to continue to strengthen the performance of prediction and recommendation. Acknowledgment This work is supported by the Special Funds for Middle-aged and Young Core Instructor Training Program of Yunnan University, the Applied Basic Research Project of Yunnan Province (2013FB009, 2014FA023), Program for Excellent Young Talents of Yunnan University (No. XT412003), and the National Natural Science Foundation of China (61472345, 61562090). The authors are grateful to reviewers for their constructive comments and suggestions which contribute substantially to the improvement of this paper. References [1] G. Adomavicius, Y. Kwon, Improving aggregate recommendation diversity using ranking-based techniques, IEEE Trans. Knowl. Data Eng. 24 (2012) 896–911. [2] D.M. Blei, A.Y. Ng, M.I. Jordan, Latent Dirichlet allocation, J. Mach. Learn. Res. 3 (2003) 993–1022. [3] J. Bobadilla, F. Ortega, A. Hernando, A. Gutiérrez, Recommender systems survey, Knowledge-Based Syst. 46 (2013) 109–132. [4] I. Cantador, I. Brusilovsky, T. Kuflik, 2nd workshop on information heterogeneity and fusion in recommender systems (HeTrec 2011), in: Proceedings of the 5th ACM Conference on Recommender Systems, ACM, New York, 2011, pp. 387–388. [5] J. Golbeck, J. Hendler, FilmTrust: movie recommendations using trust in webbased social networks, in: Proceedings of the IEEE Consumer Communications and Networking Conference, vol. 96, University of Maryland, 2006, pp. 282– 286. [6] T.L. Griffiths, Finding scientific topics, Proc. Natl. Acad. Sci. 101 (2004) 5228– 5235. [7] G. Groh, S. Birnkammerer, V. Köllhofer, Social recommender systems, in: Recommender Systems for the Social Web, Springer, 2012, pp. 3–42. [8] I. Guy, D. Carmel, Social recommender systems, in: Proceedings of the 20th International Conference Companion on World Wide Web, ACM, 2011, pp. 283– 284. [9] J. He, H. Tong, Q. Mei, B.K. Szymanski, Gender: a generic diversified ranking algorithm, in: Proceedings of the 26th Annual Conference on Neural Information Processing Systems 2012, Lake Tahoe, Nevada, 2012, pp. 1151–1159. [10] T. Hofmann, Latent semantic models for collaborative filtering, ACM Trans. Inf. Syst. 22 (2004) 89–115. [11] Y. Hu, Y. Koren, C. Volinsky, Collaborative filtering for implicit feedback datasets, in: Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), December 15-19, 2008, Pisa, Italy, 2008, pp. 263–272. [12] M. Jamali, M. Ester, TrustWalker: a random walk model for combining trustbased and item-based recommendation, in: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2009, pp. 397–406.

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011

JID: KNOSYS 12

ARTICLE IN PRESS

[m5G;February 1, 2016;13:37]

H. Wu et al. / Knowledge-Based Systems 000 (2016) 1–12

[13] J. Kang, K. Lerman, LA-CTR: a limited attention collaborative topic regression for social media, in: Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, Bellevue, Washington, USA, 2013. [14] A.M. Kaplan, M. Haenlein, Users of the world, unite! the challenges and opportunities of social media, Bus. Horiz. 53 (2010) 59–68. [15] Y.A. Kim, An enhanced trust propagation approach with expertise and homophily-based trust networks, Knowledge-Based Syst. 82 (2015) 20–28. [16] Y. Koren, R. Bell, C. Volinsky, Matrix factorization techniques for recommender systems, Computer 42 (2009) 30–37. [17] J. Lu, D. Wu, M. Mao, W. Wang, G. Zhang, Recommender system application developments: a survey, Decis. Support Syst. 74 (2015) 12–32. [18] H. Ma, I. King, M.R. Lyu, Learning to recommend with social trust ensemble, in: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, 2009, pp. 203–210. [19] H. Ma, H. Yang, M.R. Lyu, I. King, SoRec: social recommendation using probabilistic matrix factorization, in: Proceedings of the 17th ACM Conference on Information and Knowledge Management, ACM, 2008, pp. 931–940. [20] H. Ma, D. Zhou, C. Liu, M.R. Lyu, I. King, Recommender systems with social regularization, in: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, ACM, 2011, pp. 287–296. [21] P. Massa, P. Avesani, Trust metrics on controversial users: Balancing between tyranny of the majority, Int. J. Semant. Web Inf. Syst. 3 (2007) 39–64. [22] S. Purushotham, Y. Liu, C.-c.J. Kuo, Collaborative topic regression with social matrix factorization for recommendation systems, in: Proceedings of the 29th International Conference on Machine Learning (ICML-12), 2012, pp. 759–766. [23] R. Salakhutdinov, A. Mnih, Probabilistic matrix factorization, in: Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3-6, 2007, 2007, pp. 1257–1264. [24] Q. Shambour, J. Lu, A trust-semantic fusion-based recommendation approach for e-business applications, Decis. Support Syst. 54 (2012) 768–780. [25] X. Su, T. Khoshgoftaar, A survey of collaborative filtering techniques, Adv. Artif. Intell. 2009 (2009) 1–19.

[26] S. Vargas, P. Castells, Rank and relevance in novelty and diversity metrics for recommender systems, in: B. Mobashe, R.D. Burke, D. Jannach, G. Adomavicius (Eds.), Proceedings of the 2011 ACM Conference on Recommender Systems, ACM, Chicago, USA, 2011, pp. 109–116. [27] C. Wang, D.M. Blei, Collaborative topic modeling for recommending scientific articles, in: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, San Diego, USA, 2011, pp. 448– 456. [28] H. Wang, W.-J. Li, Relational collaborative topic regression for recommender systems, IEEE Trans. Knowl. Data Eng. 27 (2015) 1343–1355. [29] P. Wendykier, J.G. Nagy, Parallel colt: a high-performance java library for scientific computing and image processing, ACM Trans. Math. Softw. 37 (2010) 31. [30] H. Wu, X. Cui, J. He, B. Li, Y. Pei, On improving aggregate recommendation diversity and novelty in Folksonomy-based social systems, Person. Ubiquitous Comput. 18 (2014) 1855–1869. [31] H. Wu, Y. Pei, B. Li, Z. Kang, X. Liu, H. Li, Item recommendation in collaborative tagging systems via heuristic data fusion, Knowledge-Based Syst. 75 (2015) 124–140. [32] X. Yang, Y. Guo, Y. Liu, H. Steck, A survey of collaborative filtering based social recommender systems, Comput. Commun. 41 (2014) 1–10. [33] M. Ye, X. Liu, W. Lee, Exploring social influence for recommendation: a generative model approach, in: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’12, Portland, OR, USA, August 12–16, 2012, 2012, pp. 671–680. [34] D. Zeng, H. Chen, R. Lusch, S.-H. Li, Social media analytics and intelligence, IEEE Intell. Syst. 25 (2010) 13–16. [35] Y. Zhang, W. Chen, Z. Yin, Collaborative filtering with social regularization for tv program recommendation, Knowledge-Based Syst. 54 (2013) 310–317. [36] Z. Zhang, H. Lin, K. Liu, D. Wu, G. Zhang, J. Lu, A hybrid fuzzy-based personalized recommender system for telecom products/services, Inf. Sci. 235 (2013) 117–129.

Please cite this article as: H. Wu et al., Collaborative Topic Regression with social trust ensemble for recommendation in social media systems, Knowledge-Based Systems (2016), http://dx.doi.org/10.1016/j.knosys.2016.01.011