Product recommendation with temporal dynamics

Expert Systems with Applications 39 (2012) 12398–12406

Contents lists available at SciVerse ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

Product recommendation with temporal dynamics Wenxing Hong a,⇑, Lei Li b, Tao Li b,⇑ a b

School of Information Science and Technology, Xiamen University, Xiamen 361005, China School of Computer Science, Florida International University, Miami, FL 33199, USA

a r t i c l e

i n f o

Keywords: Taxonomy Recommendation with life cycle Collaborative filtering Long-term profile Short-term profile

a b s t r a c t In many E-commerce recommender systems, a special class of recommendation involves recommending items to users in a life cycle. For example, customers who have babies will shop on Diapers.com within a relatively long period, and purchase different products for babies within different growth stages. Traditional recommendation algorithms produce recommendation lists similar to items that the target user has accessed before (content filtering), or compute recommendation by analyzing the items purchased by the users who are similar to the target user (collaborative filtering). Such recommendation paradigms cannot effectively resolve the situation with a life cycle, i.e., the need of customers within different stages might vary significantly. In this paper, we model users’ behavior with life cycles by employing handcrafted item taxonomies, of which the background knowledge can be tailored for the computation of personalized recommendation. In particular, our method first formalizes a user’s long-term behavior using the item taxonomy, and then identifies the exact stage of the user. By incorporating collaborative filtering into recommendation, we can easily provide a personalized item list to the user through other similar users within the same stage. An empirical evaluation conducted on a purchasing data collection obtained from Diapers.com demonstrates the efficacy of our proposed method. Ó 2012 Elsevier Ltd. All rights reserved.

1. Introduction Personalized recommendation is becoming increasingly popular due to its great utility in providing people with products that they might be interested in and thus purchase. Many E-commerce websites in a business-to-consumer (B2C) style, e.g., Amazon, have already benefited from personalized marketing leverage offered by product recommender systems (Schafer, Konstan, & Riedi, 1999; Ziegler, Lausen, & Schmidt-Thieme, 2004). Many researchers try to develop recommender systems by utilizing content-based methods (i.e., analyzing the similarity between items to model users’ preferences) or collaborative filtering (i.e., exploring similar users’ behaviors) or hybrid version of these two techniques (i.e., combining these two types of methods to alleviate their corresponding drawbacks). However, such recommendation paradigms cannot effectively handle all the cases that might happen in realworld applications. In E-commerce recommender systems, there is a special class of recommendation problems, in which a user’s behavior might evolve over a long time period, i.e., within different stages the user will prefer distinct items, as shown in Scenario 1. Scenario 1: A customer with a baby needs to purchase some products of baby care. Within different growth stages of his/her baby, the desired products are significantly different. For example, within the stage ⇑ Corresponding authors. E-mail addresses: [email protected] (W. Hong), taoli@cs.fiu.edu (T. Li). 0957-4174/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.eswa.2012.04.082

3-month to 1-year, the baby’s nutrition comes from baby formula; within the stage 1-year to 3-year, the baby can have other nutrition sources. In this scenario, the customer’s purchasing interest evolves over time. It is not reasonable to utilize item-based methods for personalized recommendation, as the items purchased at different time periods might be significantly different. Further, it is a non-trivial task to effectively capture the evolution of customer’s purchasing interest. Traditional algorithms describe a customer’s long-term profile by using representative items purchased by the customer. However, only using items is not enough to comprehensively capture the customer’s exact interest. In general, an item taxonomy is often associated with a recommender system, by which customers can easily navigate different categories of products. In this paper, we propose to employ such taxonomic knowledge to formalize users’ long-term preference, and in the meantime, to capture the evolution of users’ interest. Here the long-term preference of a user can identify item categories and/or product brands preferred by the user. In Scenario 1, for example, the customer might personally prefer products (e.g., baby formula) from brand Enfamil in a long time. However, only long-term preference cannot provide enough evidence against the user’s current desire, as illustrated in the following scenario. Scenario 2: A customer with 6-month baby needs to change the type of formula for his/her baby. Generally speaking, babies within the growth stage of 3-month to 6-month should be fed using one type of baby formula, whereas after 6-month, the baby will need another type of baby formula.

12399

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

In this scenario, the customer needs to purchase some new items. In general, long-term preferences (e.g., item categories and product brands) model users’ purchasing interest in a long run. However, for customers who step into a new purchasing stage, he/she would desire new types of items that have never been purchased before. Therefore, only considering the customer’s long-term preference will not resolve his/her concern. Most existing work tries to capture the user’s short-term interest by considering other user’s current interest; however, different users might have distinct preference since they might be in different purchasing stages. To handle such an issue, we propose to explore the user’s short-term interest by analyzing purchasing behaviors of similar users. Here ‘‘similar’’ indicates that two users might have similar purchasing behaviors within the identical stage. By this way, we can easily resolve the situation that the customer desires new types of products at new stages. Fig. 1 shows an overview of our proposed method.

the following points: (1) a clustering-based method is proposed to capture the effect of multiple profiling sources in product recommendation; (2) experiments are performed on a larger dataset collected from Diapers.com to show the scalability of our proposed methods; and (3) additional experiments are conducted on the effect of recommendation with temporal dynamics, and detailed empirical comparison between our methods and existing works is provided. The paper is organized as follows. We describe some related work in Section 2. Section 3 formalizes the problem of RwS. In Section 4, we discuss the algorithmic details of how to utilize the item taxonomy to simultaneously model users’ long-term and short-term profiles, and consequently address the predefined problem in Section 3. Section 5 provides an empirical evaluation of our proposed methods on a realworld data set, and finally Section 6 concludes the paper.

1.1. Roadmap

In this section, we briefly review prior methods related to recommender systems, which can be categorized into three groups: content filtering, collaborative filtering and hybrid methods. We also discuss the techniques of recommendation using ontologies or taxonomies and the methods considering the temporal dynamics in the recommendation, and highlight the difference between our method and other existing works.

In summary, the contribution of our paper is threefold:  We define a new class of recommendation problem, named recommendation with stage (RwS), in which a user’s preference evolves over time. To provide an elegant up-to-date recommendation list to such users, it is imperative to consider both the long-term profile and the short-term preference simultaneously so that the user’s interest can be properly matched.  We introduce a taxonomy-oriented approach to model a user’s long-term preference. Each service provider, e.g., Amazon, designs its item taxonomy to enable their customers to efficiently access their preferred items. Hereby, we propose to capture a user’s preference not only on the item level, but also on the semantic correlations among the concepts or categories that the items belong to.  To localize a user’s specific interest within a stage, we empirically segment the user’s consumption history, and then put more focus on the recent or current stage in order to provide the user a reasonable recommendation list. To identify items with a high probability that the user might be interested in these items, we investigate different algorithms tailored for collaborative filtering, e.g., clustering- and graph-based methods. This paper is an extension of our previous conference paper (Li, Hong, & Li, 2012). Distinguished contributions of this paper include

2. Related work

2.1. Content-based recommendation The effectiveness of content-based information filtering paradigm has been proven for applications locating textual documents relevant to a topic. Particularly in recommender systems, contentbased methods enable accurate comparison between different textual or structural items, and hence recommend items similar to a user’s consumption history. Representative examples of contentbased recommender systems involve (Chen & Yin, 2006; Lops, Gemmis, & Semeraro, 2011). However, content-based filtering approaches suffer from multiple drawbacks, e.g., strong dependence on the availability of content, ignoring the contextual information of recommendation, etc. (Adomavicius & Tuzhilin, 2005). 2.2. Collaborative filtering Collaborative-based recommenders tackle the recommendation problem by exploiting similar users’ profiles in the same community

Product Taxonomy Clustering-based Method Long-Term Profile Short-Term Profile

u1 u2 u3 u4 u5

P

C

……

P

P

…… ……

U

U P

……

P

C U

P

……

User Behavior with Stages

Graph-based Method

Fig. 1. An overview of the framework.

P

12400

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

and recommending new items not previously rated or seen by the target user. Recommendation is therefore achieved by finding common characteristics in the preferences of other users. Common approaches in this category include k-Nearest-Neighbor (k-NN) (Chen & Yin, 2006), matrix factorization (Koren, 2008; Salakhutdinov & Mnih, 2008), and semi-supervised learning (Ding, Jin, Li, & Simon, 2007; Wang, Ma, Yang, & Li, 2006), etc. Amazon uses an item-to-item collaborative filtering approach to personalize the online store for each customer (Linden, Smith, & York, 2003). However, collaborative filtering requires more ratings over items, and therefore suffers from the rating sparsity problem (Adomavicius & Tuzhilin, 2005); Also, it cannot provide reasonable recommendation for new users and new items, which is known as the cold-start problem (Schein, Popescul, Ungar, & Pennock, 2002). 2.3. Hybrid recommendation Hybrid approaches combine the aforementioned techniques in different ways to improve recommendation performance and tackle the shortcoming of underlying methods. Some sample approaches include (Chu & Park, 2009; Lekakos & Caravelas, 2008), in which the inability of collaborative filtering to recommend items is commonly alleviated by combining it with content-based filtering (Burke, 2005). 2.4. Ontology-based recommendation Ontology-based (or knowledge-based) personalization systems emphasize explicit domain knowledge about the items as well as implicit knowledge about the users (e.g., psychographic, demographic, or other personal attributes of users) to extract pertinent recommendations (Kang & Choi, 2011; Schickel-Zuber & Faltings, 2006). Some researchers use item taxonomies to raise the accuracy of prediction results (Cantador, Bellog, & Castells, 2008; Hung, 2005). For example, (Nakatsuji et al., 2010) proposes to expand users interests via item taxonomies, in which the authors define item novelty as the smallest distance from the class the user accessed before to the class that includes target items over the taxonomy. 2.5. Recommendation with temporal dynamics The problem studied in our work is relevant to recommendation with temporal dynamics, in which users’ preferences drift over time. Researchers have explored different ways to handle such recommendation problem. Ding and Li (2005) presented a novel algorithm to compute the time weights for different items by decreasing weights to historical data. This approach deemphasizes the importance of historical data; however, the historical data might be helpful to facilitate other users’ recommendation. Min and Han (2005) proposed a methodology for detecting a user’s time-variant pattern in order to improve the performance of collaborative filtering. Cao, Chen, Yang, and Xiong (2009) investigated how to enhance recommender system under volatile user interest drifts. They defined four different types of interest patterns and then proposed a rating graph and rating chain based approach for detecting these patterns. Recently, a couple of works (Koren, 2010; Liu, Zhao, Xiang, & Yang, 2010; Xiang et al., 2010) have been proposed to address temporal recommendation using different strategies. However, most of them only consider the correlations between users and items within a relatively short time period, which cannot explicitly capture the exact interest of users. Different from the techniques introduced above, in our work we investigate a special class of recommendation problems in E-commerce systems, in which a user has explicit interest drift over items at different long time periods. We formalize the problem using the category correlations within item taxonomies, and then model the

user’s long-term profile via a probabilistic paradigm. To provide attractive recommendation list, we first identify the exact stage that the user lies in, and then localize the user’s short-term preference by inference over users, items, and categories, where the candidate users’ short-term profiles that are within the same stage compared with the target user are selected to construct the inference base. 3. Problem formulation Sticking to the ‘‘collaboration via content’’ paradigm (Pazzani & collaborative, 1999), we delve into the possibility of utilizing taxonomic knowledge to improve the accuracy of recommendation with stage. In this section, we first introduce some background information about the item taxonomic knowledge, and then formally define the problem of RwS. 3.1. Background In our work, we employ item taxonomy to facilitate the recommendation procedure. In general, item taxonomies are often designed or applied in E-commerce websites, e.g., Amazon. In particular, we have the following elements within an E-commerce community:  User repository: U ¼ fu1 ; u2 ; . . . ; un g. U contains all users or customers within the E-commerce community.  Product repository: P ¼ fp1 ; p2 ; . . . ; pm g. P is composed of all the products provided by an E-commerce website.  User ratings: R ¼ fR1 ; R2 ;    ; Rn g. R includes all the rating information of users over products, where Ri represents the rating set of user ui. Commonly in the E-commerce community, customers tend to give ratings of products in an explicit way, i.e., assigning a numeric score to the product they have purchased; another rating scheme is implicit rating, where a ‘‘purchase’’ activity can be regarded as a rating over the products. In our work, we consider the implicit rating.  Item taxonomy: T over item category set C ¼ fc1 ; c2 ; . . . ; cl g. Here each category c 2 C denotes a specific topic that p 2 P might belong to. The taxonomy T captures the hierarchical structure of C, in which topics may represent broad or narrow categories in the E-commerce. For the simplicity, we require that one product can only fall into one specific topic. 3.2. Problem statement Problem (recommendation with stage): Within an E-commerce community, given U, P, R and T , recommend items to a target user, guaranteeing that the user’s current purchasing desire is maximally satisfied. To resolve the above problem, we can decompose it into two different subproblems based on the characteristics of the problem itself. Subproblem 1 (Stage identification): Given an item taxonomy T and a target user’s purchasing history, identify the user’s current preference on item categories. To identify the recommendation stage for the target user, we need to consider the entire purchasing history of this user. In general, a user’s purchasing history is composed of transactional data, in which each record indicates a purchase activity of the user over items. However, simply modeling the user’s behavior based on such information cannot capture the evolution of the user’s exact interest. Fortunately, the item taxonomy can provide us a meaningful and elegant explanation of the user’s evolving behavior. Subproblem 2 (Item recommendation): Given a target user’s current stage, recommend a list of items to the user such that his/her preference will be maximally satisfied.

12401

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

Once we obtain the recommendation stage of the target user, a natural way to recommend items is to refer other users’ behaviors within the same stage. It is similar to collaborative filtering where the recommendation is achieved by considering other user’s behaviors; however in our problem setup, we have auxiliary resources, e.g., the item taxonomy, to enrich the correlations among users, and therefore we are able to provide more reasonable recommendation list. 4. Taxonomy-driven recommendation In this section, we discuss the algorithmic details of how to utilize item taxonomy to perform recommendation task. The item taxonomy semantically encapsules the correlations among items, which is suitable to capture users’ preferences over different item categories. Specifically in our work, we first separate a user’s purchasing history into different stages, and then model the user’s long-term preference by employing the item taxonomy. Given a target user, we first identify the ‘‘collaborative’’ user set that have similar purchasing behaviors to the target user, and then investigate multiple collaborative filtering algorithms to finalize the recommendation. 4.1. Stage identification The long-term user profile consists of multiple stage profiles. Formally, given the purchasing history H of the target user u; H can be initially divided into multiple segments based on a specific time scheme. Here the time scheme used to segment H can be learned according to the detailed content of H using statistical or machine learning techniques. Taking into account the time complexity, hereby, we simply define a uniform time frame T as a segment unit, and then separate H based on T, i.e., H ¼ fHt0 ; Ht1 ; . . . ; Htn g, where t0 means the current stage, i.e., the latest purchasing period. For each time period ti, i = 0, 1, . . . , n, we model the user’s behavior Hti using the item taxonomy. 4.1.1. User profile generation As illustrated in Fig. 2, to model the stage profile Ht0 of user u1, we first extract the transactional data of u1, R1 = {p41, p51, p21}. For each transaction or item in R1, e.g., p41, we locate the fine-grained category that the item directly belongs to, e.g., c4, and then traverse the taxonomy from bottom to top, to obtain the category path rooting from this category, e.g., c4 ? c1. For this example, we then transfer the rating score on p41 to the category path, i.e., assigning ‘‘1’’ to each category in the path. Subsequently, we aggregate the rating score for each category and formalize the user’s stage profile as a weighted category vector, i.e., the weight for each category in the vector is obtained from the rating behavior of the corresponding user. Stage identification: Again for the previous example, assume the basic category vector is in the form of hc1, c2, c3, c4, c5i, then the weighted category vector Rtu01 ¼ h1; 2; 0; 1; 1i, where each entry represents the user’s implicit rating over the corresponding category. In such representation, we assign more weight on the categories of the upper level in the item taxonomy, by which we can avoid being trapped into specific topics and losing the overall recognition of the user’s preference. The weighted category vector is l2-normalized. Finally, we can denote user u1’s long-term profile as Ru1 ¼ fRtu01 ; Rtu11 ; Rtu21 ; . . .g, and also we can identify the current recommendation stage of u1 as Ht0 . 4.1.2. Similarity refinement As proposed in the previous section, the item taxonomy is used to connect users and items in a semantic way, in which users’

c1

c2

Taxonomy

c3

c4

c5

Item

p11

User Rating

u1: p11, p31, p32 u2: p11, p32, p41 u3: p31, p41

u1: p31, p41 u2: p32, p41 u3: p11, p32

u1: p41, p51, p21 u2: p41, p32, p21 u3: p32, p21

Ht2

Ht1

Ht0

p31

p32

p41

p51

p21

Fig. 2. An example of item taxonomy.

rating behaviors are amplified via the category path. In such a model, we can easily capture the correlations between different elements, i.e., users, items and categories. In the following, we further formalize the taxonomy-based profiling model by specifying three different similarities. Note that similarity has played an important role in various data mining tasks including the profile generation (Li, Zhu, & Ogihara, 2003).  User–user similarity ðS U Þ: To utilize collaborative filtering in our framework, we need to calculate the similarities between users. Here the user–user similarity originates from two different components: user–item similarity SUI and user–category similarity SUC. Given two users u1 and u2, SUI can be computed by considering the Jaccard similarity between the item sets purchased by these two users, Iu1 ; Iu2 , whereas SUC can be derived from the Cosine similarity between the long-term profile vectors of these two users Ru1 ; Ru2 . Specifically,

SUI ðu1 ; u2 Þ ¼

Iu1 \ Iu2 ; Iu1 [ Iu2

SUC ðu1 ; u2 Þ ¼

Ru1  Ru2 : kRu1 kkRu2 k

Finally, the similarity between these two users, SU(u1, u2), can be calculated as the linear combination of SUI(u1, u2) and SUC(u1, u2), factorized by a parameter k:

SU ðu1 ; u2 Þ ¼ k  SUI ðu1 ; u2 Þ þ ð1  kÞ  SUC ðu1 ; u2 Þ:

ð1Þ

In this way, we can easily capture the correlations between two users from both the taxonomic level and real purchasing behavior level simultaneously.  Item–item similarity ðS I Þ: Similarity between items can be computed using item-to-item collaborative filtering. Similar to userto-user collaborative filtering that matches the user to similar customers, item-to-item collaborative filtering matches each of the user’s purchased and rated items to similar items. Given a product p, our goal is to find products similar to p; Here by ‘‘similar’’ we mean that the user sets, U p1 ; U p2 , purchasing the two items have substantial overlap. To this end, we use the Jaccard similarity to capture SI ðU p1 ; U p2 Þ, as

SI ðU p1 ; U p2 Þ ¼

U p1 \ U p2 : U p1 [ U p2

ð2Þ

 Category–category similarity ðS C Þ: This similarity can help quantify the semantic correlation between two different categories in the item taxonomy. In our work, we employ Information Content (IC) (Resnik) to compute S C , which measures the amount of information of a given category c from its probability of occurrence, in our case, the probability that items under c are purchased. The larger the IC, the more popular the category. IC

12402

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

can be computed as the negation of the logarithm of the probability of occurrence, i.e., IC(c) = log p(c). To derive the similarity between two categories, we use the similarity measure proposed in Lin (1998), which evaluates the correlation between a pair of concepts as the IC of the Least Common Subsumer (LCS) (Cohen, Borgida, & Hirsh, 1992) in a give taxonomy, i.e., an indication of the amount of information that two categories share in common. Given two categories c1 and c2, the similarity SC(c1, c2) can be computed as

SC ðc1 ; c2 Þ ¼

2  ICðLCSðc1 ; c2 ÞÞ : ICðc1 Þ þ ICðc2 Þ

ð3Þ In this way, the recommendation result is enriched by not only similar users’ preferences, but also potential items’ matching on the target user’s preference within the corresponding stage.

4.2. Item recommendation After we locate the recommendation stage of the customer, we investigate two different methods for recommendation, the clustering-based model and the graph-based model. In the following, we describe the algorithmic details for these two methods. 4.2.1. Clustering-based model In practice, the drawbacks of user-to-user collaborative filtering can be mitigated via the integration with content-based methods. As discussed in Section 2, there are a huge body of publications that utilize both content filtering and collaborative filtering to improve the recommendation accuracy. However, most hybrid methods still suffer from several issues, e.g., the limit of item content. In this subsection, we introduce a clustering-based method, by which we are not concerned with the specific content of items, but make use of both user-based and item-based collaborative filtering to derive the recommendation result. By clustering on users, we can obtain the user set whose purchasing behaviors are similar to the target user, and then recommend some items preferred by these users to the target user; comparatively, by clustering on items, we can get the item set that are preferred by users with the same stage, and therefore recommend top ranked items to the target user. The final recommendation list can be obtained by combining these two sets of items. Specifically in our method, we employ a variation of X-means (Pelleg & Moore, 2000) for clustering. In our problem, it is not trivial to derive the answer that how many clusters for users (items) are appropriate; therefore, we borrow the idea of X-means, in which the optimal number of clusters are not simply predefined, but within a range given by the user. Also due to the fact that the correlations of users (items) are given by a user-similarity (itemsimilarity) matrix, we adjust the X-means algorithm to X-median, which can be easily performed based on the input matrix. As introduced in Pelleg and Moore (2000), given the data D and a family of M alternative models (different partitions with different values of K), the point probabilities are defined as

  1 1 b i Þ ¼ RðiÞ  pffiffiffiffiffiffi ffi exp  2 kxi  lðiÞ k2 ; Pðx ^ 2r R ^M 2pr

and then a list of popular items that are purchased by the users within this group can be selected as the recommendation candidates. 2. Perform X-median clustering on SI . We can obtain the item groups that are suitable to the target user with respect to a specific stage. We then select a group of items that are similar to the ones purchased by the target user as the recommendation candidates. 3. Combine the two candidate item sets and select top ranked ones to recommend them to the target user.

ð4Þ

where R is the total number of points, R(i) is the number of points that are assigned to the (i)th cluster, l(i) is the center of the (i)th qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi P 2 1 ^ ¼ RK cluster, and r i ðxi  lðiÞ Þ . In our method, we use the median of a cluster, ^xðiÞ , instead of the mean l(i). Therefore, we call it as X-median clustering (Pelleg & Moore, 2000). Given the user-similarity matrix SU and the item-similarity matrix SI , the recommendation can be achieved by the procedure as follows: 1. Perform X-median clustering on SU . We can obtain the user groups within the identical stage. Based on the user similarity, the target user can be easily mapped to a specific user group,

4.2.2. Graph-based model Besides the clustering-based model, we also investigate the effect of using the item taxonomy associated with a graph model towards the recommendation performance. To this end, we first construct a multi-modal graph, and then given a target customer u, we perform random walk on this graph starting from u for recommendation. In detail, the multi-modal graph consists of multiple resources, e.g., users, products and categories, associated with multiple types of correlations, e.g., user–user, item–item, category–category relations etc. We encapsule the graph as an adjacency matrix. Formally, let U, P, C denote customers, products and categories respectively, and let Up, Uc, Pc denote user–product, user–category and product–category relations, then we can build a block-wise 0 1 Uu Up Uc adjacency matrix W, as W ¼ @ U p Pp Pc A, where Uu, Pp, Cc deU c Pc C c note user–user, product–product and category–category relations. Here the entries of Uu is calculated by SU defined by Eq. (1), Pp by Eq. (2) and Cc by Eq. (3). The entries of other blocks in W can be derived by purchasing behavior or is-a relations in the item taxonomy. For example, for entries in Up, each one is a binary value (‘‘1’’ or ‘‘0’’), representing whether the corresponding user has been purchasing the product or not. Algorithm 1. Recommendation using RWR input: An adjacency matrix W, and a target user uq. output: A recommended item list l. 1. Let ! vq ¼ 0 for all its entries, except a ‘1’ for the qth entry; 2. Normalize W using A ¼ D1=2 WD1=2 ; ! 3. Initialize uq ¼ ! vq ; ! 4. while uq not converged do ! ! ! 5. uq ¼ ð1  dÞA uq þ d v q ; 6. end while ! 7. Select top ranked subset e from uq as the recommendation base; 8. if ei is a user then 9. Select items that ei has purchased recently and put them into l; 10. end if 11. if ei is a product then 12. Put ei into l; 13. end if 14. if ei is a category then 15. Select items that contribute more to IC of ei and put them into l; 16. end if 17. return l;

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

Given a target user u, we perform random walk on the multimodal graph starting from u. Here we employ random walk with restart (RWR) (Pan, Yang, Faloutsos, & Duygulu, 2004) to retrieve items for recommendation, such that the recommended items are not deviated much from the user’s purchasing interest. Specifically, we construct a matrix A using normalized graph Laplacian, i.e., A ¼ D1=2 WD1=2 , in which D is a diagonal matrix with its (i, i)-element equal to the sum of the ith row of W. The algorithmic detail is described in Algorithm 1. In Algorithm 1, we first build a multi-modal graph whose nodes include users, products and categories, and the weighted edges between nodes are quantified by the adjacency matrix W. Here we normalize W using graph Laplacian (line 2) to make it suitable for the random walk computation. Next, our method executes RWR (lines 3–6) to retrieve elements highly related to a given user ! q. The procedure will terminate once the input vector uq is converged. Here the parameter d controls the steps of each random walk; in our case, we expect that the random walk should not be so deep on the graph so that the selected items will conform to the user’s purchasing interest. ! After uq is finalized, our method chooses a subset of elements ! that are top ranked in uq , which might include users, products or categories. Therefore, to retrieve the recommended items, we need to consider three distinct cases: !  For users in uq , we can select items being recently purchased by the users and then recommend them to the target user. !  For items in uq , we can directly recommend them to the target user. !  For categories in uq , our method selects items contributing more to the IC value of the category and then recommends them to the target user. In this way, the resulted recommendation list is composed of products that are originated from different recommendation disciplines. In addition, the computational cost of Algorithm 1 is linear to the number of edges in the graph, i.e., O(E). Since the multi-modal graph is constructed based on a user’s current recommendation stage but not all his/her history, the dataset size and the edges will not be too large, which renders our algorithm efficient in term of time requirement. 5. Empirical evaluation In this section, we provide a comprehensive experimental evaluation to show the efficacy and effectiveness of our proposed recommendation algorithms. We start with an introduction to a real-world purchasing data set obtained from Diapers.com, and then describe the experimental design. 5.1. Real-world dataset In our experiment we decide to use purchase data from babycare products. The customers in this product domain show explicit interest drifts over different time periods, which is exactly the situation we considered in our proposed algorithm. Examples of this product domain can be found in the scenarios introduced in Section 1. The dataset used in our experiment is collected from Diapers.com. It consists of customers’ order information related to Baby Care, ranging from November 12th, 2004 to March 6th, 2009. This dataset contains 1,872,803 orders of 71,075 anonymous customers on 3529 products. The ratings in this dataset are implicit rating, i.e., the binary rating originated from the purchasing behavior of customers. The time span of the customer’s order

12403

history varies significantly, ranging from several months to 4 years. The average number of orders for customers is 26. The item taxonomy covered by the dataset consists of 37 classes and it is a threelayer hierarchy: the root class, ‘‘Baby Care’’ and subclasses, e.g., ‘‘Baby Diaper’’, ‘‘Baby Formula’’, ‘‘Feeding Accessories’’, etc. For each category, the number of products contained varies from tens to hundreds. Hereby, the taxonomy is simple and items ratings in the dataset are relatively dense. 5.2. Experiments To resolve the problem of recommendation with stage (RwS), our proposed recommendation methods take into account the long-term and short-term interest profiles of the customers in a semantic way, where the long-term profile is designed to identify the current recommendation stage, and the short-term one aims to retrieve the attractive items based on either collaborative filtering of the combination of users and items, or inference on a multimodal graph. In this subsection, to evaluate the effectiveness of our proposed methods, we design a series of experiments as follows. 5.2.1. Model validation In Section 4.1 we introduced three distinct similarity measurements, i.e., user–user, item–item and concept–concept. The encapsulation of these three measures enables the method to capture the correlations among different resources (users, items and categories), and therefore renders the recommendation more reasonable. Also, the item taxonomy can help better interpret the customers’ exact purchasing interest. To verify this claim, we evaluate the performance of using different measurements. In the experiment, we choose to use user–user and item–item similarities respectively, as two baselines, and then recommend items (top@10, top@20 and top@30) to a set of randomly selected users (100 users). We plot the average precision and recall pair of recommendation results. As depicted in Fig. 3, besides the higher ratio of precision and recall, the performance distributions of the clustering fusion model and the unified multi-modal graph are more compact than the others, which demonstrates the effectiveness and stability of the strategy of our proposed methods. 5.2.2. Parameter tuning In our graph-based model, the parameter k is designed to control the importance of user–item similarity and user–category similarity, whereas the parameter d aims to control the depth of random walk on the multi-modal graph. To find the optimal value for these two parameters, we use the experimental setup similar to the previous sections, and plot the variation of averaged F1-score for each parameter. As shown in Fig. 4, we empirically choose the optimal value k = 0.2 and d = 0.7 for parameters. 5.2.3. Profile generation and taxonomy Long-term and short-term profiles both contribute to the success of recommendation. To verify this, we use the long-term and shortterm profiles, respectively, to perform the recommendation task. Also in our framework, we employ the item taxonomy to enrich the representation of users’ profiles, which plays a vital role in recommendation. Therefore, we setup four baseline methods as follows: (1) long-term without taxonomy (LT): use each customer’s entire purchasing history to construct profiles and perform collaborative filtering; (2) short-term without taxonomy (ST): use each customer’s recent purchasing history1 to build profiles and perform collaborative filtering; (3) long-term with taxonomy (LTT): use item 1 Here we empirically set the ‘‘recent’’ history as each user’s last 90 days purchasing behavior, and for collaborative filtering, we only consider all customers’ recent histories.

12404

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

Top @10

Top @20

Top @30

0.4

0.46 0.42

0.38

recall

0.44 0.36

0.4 0.42

0.34 0.38 0.32

0.4 0.25

0.3 Precision

0.35

0.36

0.3

0.32 0.34 0.36 0.38 Precision

Top @10

0.36

0.38 0.4 Precision

0.42

Top @30

Top @20

0.42 0.44

0.4

0.46 recall

0.38 0.42 0.36

0.44

0.34

0.4

0.42

0.32 0.3

0.25 0.28 0.31 0.34 0.37 Precision

0.38

0.32 0.34 0.36 0.38 Precision

0.4

0.4

0.38

0.4 0.42 Precision

0.44

0.6

1

Fig. 3. Precision–recall plot for different model constructions.

0.44 top@10 top@20 top@30

top@10 top@20 top@30

0.42

0.4 F1−score

F1−score

0.4 0.38 0.36

0.35

0.34 0.32

0.3 0

0.2

0.4

0.6

0.8

1

0

0.2

λ

0.4

0.8

δ

Fig. 4. Parameter tuning for k and d.

taxonomy to enrich each customer’s long-term profile and perform collaborative filtering; (4) short-term with taxonomy (STT): use item taxonomy to enrich each customer’s short-term profile and perform collaborative filtering. In addition, we implement two existing approaches, Taxonomy-Driven Computation (Ziegler et al., 2004) (TDC) and Expanding User Interest (Nakatsuji et al., 2010) (EUI) for comparison. Both methods utilize the item taxonomy to perform recommendation, where the former considers both user proximity and item proximity for recommendation, and the latter uses taxonomy to recommend novel items to users. For evaluation of our clustering-based fusion model (Xcom for short) and graph-based model (RwS for short), we empirically set

the time frame T introduced in Section 4.1 to be 90 days, and use the purchasing history within the latest stage as the testing data, whereas the purchasing history before the latest stage is regarded as the source of profiling. The parameters k and d will be tuned in the following subsection. For each approach, we recommend products (top@10, top@20 and top@30) to all the users in the dataset, and then compute the averaged precision, recall and F1-score of the recommended list. Fig. 5 shows the comparison results. From the results, we observe that (i) The performance of algorithms using the item taxonomy is superior to the ones without using the taxonomy. It is evident that by using item taxonomies, users’ preference can be better captured, since the categories or

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

12405

Fig. 5. Performance comparison for different recommendation algorithms.

Table 1 Comparison on temporal dynamics. (Bold indicates the best performance.) Methods

STG IPD TVP Xcom RwS ⁄

top@10

top@20

top@30

Precision

Recall

Precision

Recall

Precision

Recall

0.2936 0.2805 0.2832 0.3101 0.3346⁄

0.3618 0.3542 0.3428 0.3702 0.3846⁄

0.3297 0.3154 0.3203 0.3469 0.3619⁄

0.4108 0.4069 0.4033 0.4295⁄ 0.4206

0.3734 0.3619 0.3658 0.392 0.4124⁄

0.4231 0.4115 0.4074 0.4338 0.4513⁄

The statistical significance at (p < 0.01).

classes have more distinguishable power than simple items on identifying a user’s purchasing interest. (ii) By considering the short-term profiles, the accuracy of the recommended item list can be slightly improved, which is straightforward because we are more concerned with users’ recent activities. (iii) Our proposed method significantly outperforms other candidates, which demonstrates the efficacy of our algorithms in handling the problem of recommendation with stage. 5.2.4. Comparison on temporal dynamics The problem of RwS is essentially a recommendation problem with temporal dynamics, i.e., a user’s purchasing interest evolves over time. In our proposed method, we consider the situation that a user’s interest remains stable in a relatively long time period, but might change over the entire life cycle, and explicitly model a user’s interest using the taxonomic knowledge. It is different from other existing works that consider users’ interest drift within a purchasing session, or implicitly model users’ interest drift over time by simply exploring the relation between users and items. To evaluate the effectiveness of our methods in handling the recommendation with temporal dynamics, we compare our methods with several existing algorithms, including (1) STG (session-based temporal graph) (Xiang et al., 2010): a method using session-based temporal graph to simultaneously model users’ long-term and short-term preferences over time; (2) IPD (interest pattern detection) (Cao et al., 2009): an algorithm that firstly defines four types of interest patterns to understand users’ behaviors and then uses rating graphs and rating chains to detect interest patterns; and (3) TVP (time-variant pattern) (Min & Han, 2005): a method to detect a user’s time-variant behavioral pattern to improve the performance of collaborative filtering recommendations. STG considers the temporal dynamics of recommendation within a purchasing session, whereas IPD and TVP consider the temporal dynamics by modeling the user’s history in a long time period. For STG, we treat each day’s purchase data as a session. For each approach, we recommend products (top@10, top@20 and top@30) to all the users in the dataset, and then compute the averaged precision and recall of the recommended list. Table 1 shows the comparison results. From Table 1 we observe that (1) our proposed methods outperform the rivals in terms of the averaged precision and recall. The

reason is straightforward: in our methods we explicitly model a user’s purchasing behavior using the taxonomic knowledge, which can naturally represent the exact purchasing interest of users; and also we track each user’s interest drift through a life-cycle time period, and identify similar interest by considering users within identical recommendation stages, which is beneficial to the recommendation result. (2) STG performs slightly better than the other two baselines, since in STG both long-term and short-term preferences are considered in a unified temporal graph.

6. Concluding remarks In this paper, we introduced a novel problem – recommendation with stage in E-commerce, and then proposed two novel methods that utilize item taxonomy to address this problem. Our approach is capable of effectively capturing a customer’s purchasing interest via a seamless integration of the customer’s long-term profile and short-term profile, where the long-term profile is derived from the entire purchasing history, and the short-term profile is obtained from the identification of the current recommendation stage of the customer. For recommendation purpose, we focus on the current recommendation stage and propose a clustering-based method and a multi-modal graph ranking method to obtain the recommended item list. Evaluation on a dataset collected from Diapers.com demonstrates that the problem of recommendation with stage can be efficiently tackled using our proposed methods.

Acknowledgements The project was supported by the Natural Science Foundation of Fujian Province of China under Grant No. 2011J05157 and by the National Natural Science Foundation of China under Grant No. 61070151.

References Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749. Burke, R. (2005). Hybrid systems for personalized recommendations. Intelligent Techniques for Web Personalization, 133–152. Cantador, I., Bellog, A., & Castells, P. (2008). Ontology-based personalised and context-aware recommendations of news items. In 2008 IEEE/WIC/ACM international conference on web intelligence and intelligent agent technology (pp. 562–565). IEEE. Cao, H., Chen, E., Yang, J., & Xiong, H. (2009). Enhancing recommender systems under volatile userinterest drifts. In Proceeding of the 18th ACM conference on information and knowledge management (pp. 1257–1266). ACM. Chen, J., & Yin, J. (2006). Recommendation based on influence sets. In Proceedings of the workshop on web mining and web usage analysis. Citeseer. Chu, W., & Park, S. (2009). Personalized recommendation on dynamic content using predictive bilinear models. In Proceedings of the 18th international conference on world wide web (pp. 691–700). ACM.

12406

W. Hong et al. / Expert Systems with Applications 39 (2012) 12398–12406

Cohen, W., Borgida, A., & Hirsh, H. (1992). Computing least common subsumers in description logics. In Proceedings of the national conference on artificial intelligence (p. 754). Ding, C. H. Q., Jin, R., Li, T., & Simon, H. D. (2007). A learning framework using green’s function and kernel regularization with application to recommender system. In KDD (pp. 260–269). Ding, Y., & Li, X. (2005). Time weight collaborative filtering. In Proceedings of the 14th ACM international conference on information and knowledge management (pp. 485–492). ACM. Hung, L. (2005). A personalized recommendation system based on product taxonomy for one-to-one marketing online. Expert Systems with Applications, 29(2), 383–392. Kang, J., & Choi, J. (2011). An ontology-based recommendation system using longterm and short-term preferences. In International conference on information science and applications (ICISA) (pp. 1–8). IEEE. Koren, Y. (2008). Factorization meets the neighborhood: A multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 426–434). ACM. Koren, Y. (2010). Collaborative filtering with temporal dynamics. Communications of the ACM, 53(4), 89–97. Lekakos, G., & Caravelas, P. (2008). A hybrid approach for movie recommendation. Multimedia Tools and Applications, 36(1), 55–70. Li, T., Zhu, S., & Ogihara, M. (2003). A new distributed data mining model based on similarity. In Proceedings of the 2003 ACM symposium on applied computing, SAC ’03 (pp. 432–436). Li, L., Hong, W., & Li, T. (2012). Taxonomy-oriented recommendation towards recommendation with stage. In Proceeding of the 14th Asia–Pacific web conference. Lin, D. (1998). An information-theoretic definition of similarity. Proceedings of the 15th international conference on machine learning (Vol. 1, pp. 296–304). Citeseer. Linden, G., Smith, B., & York, J. (2003). Amazon.com recommendations: Item-toitem collaborative filtering. IEEE Internet Computing, 7(1), 76–80. Liu, N., Zhao, M., Xiang, E., & Yang, Q. (2010). Online evolutionary collaborative filtering. In Proceedings of the fourth ACM conference on recommender systems (pp. 95–102). ACM. Lops, P., Gemmis, M., & Semeraro, G. (2011). Content-based recommender systems: State of the art and trends. Recommender Systems Handbook, 73–105.

Min, S., & Han, I. (2005). Detection of the customer time-variant pattern for improving recommender systems. Expert Systems with Applications, 28(2), 189–199. Nakatsuji, M., Fujiwara, Y., Tanaka, A., Uchiyama, T., Fujimura, K., & Ishida, T. (2010). Classical music for rock fans?: Novel recommendations for expanding user interests. In Proceedings of the 19th ACM international conference on Information and knowledge management (pp. 949–958). ACM. Pan, J., Yang, H., Faloutsos, C., & Duygulu, P. (2004). Automatic multimedia crossmodal correlation discovery. In Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining (pp. 653–658). ACM. Pazzani, M. (1999). A framework for collaborative, content-based and demographic filtering. Artificial Intelligence Review, 13(5), 393–408. Pelleg, D., & Moore, A. (2000). X-means: Extending k-means with efficient estimation of the number of clusters. In Proceedings of the 17th international conference on machine learning (pp. 727–734). Resnik, P. Using information content to evaluate semantic similarity in a taxonomy. Available from cmp-lg/9511007. Salakhutdinov, R., & Mnih, A. (2008). Probabilistic matrix factorization. Advances in Neural Information Processing Systems, 20, 1257–1264. Schafer, J., Konstan, J., & Riedi, J. (1999). Recommender systems in e-commerce. In Proceedings of the 1st ACM conference on electronic commerce (pp. 158–166). ACM. Schein, A., Popescul, A., Ungar, L., & Pennock, D. (2002). Methods and metrics for cold-start recommendations. In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval (pp. 253–260). ACM. Schickel-Zuber, V., & Faltings, B. (2006). Inferring user’s preferences using ontologies, in: Proceedings of the national conference on artificial intelligence. Wang, F., Ma, S., Yang, L., & Li, T. (2006). Recommendation on item graphs. In ICDM (pp. 1119–1123). Xiang, L., Yuan, Q., Zhao, S., Chen, L., Zhang, X., Yang, Q., et al. (2010). Temporal recommendation on graphs via long-and short-term preference fusion. In Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 723–732). ACM. Ziegler, C., Lausen, G., & Schmidt-Thieme, L. (2004). Taxonomy-driven computation of product recommendations. In Proceedings of the thirteenth ACM international conference on information and knowledge management (pp. 406–415). ACM.