Exploiting multimodal interactions in recommender systems with ensemble algorithms

Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

1

Contents lists available at ScienceDirect

3

Information Systems

5

journal homepage: www.elsevier.com/locate/infosys

7 9 11 13

Exploiting multimodal interactions in recommender systems with ensemble algorithms

15 Q1

Arthur F. da Costa, Marcelo G. Manzato

17 Q2

Institute of Mathematics and Computer Science, University of São Paulo, São Carlos, SP, Brazil

19

a r t i c l e i n f o

abstract

Article history: Received 10 September 2015 Accepted 19 September 2015 Recommended by: D. Shasha

The increasing of products, information and services based on users' profiles has made recommender systems to be increasingly present, easing the selection and retention of users in services on the Web. However, optimizations must be performed in such systems mainly regarding the modeling of users' profiles. Preferences are generally modeled superficially, due to the scarcity of data collected, as notes or comments, or the inductive information susceptible to errors. This manuscript proposes a recommender tool with three ensemble approaches based on multimodal interactions that combines different types of users' feedback processed individually by traditional recommendation algorithms. The approaches have been developed to improve the quality of predictions in recommender systems, considering a large number of user information. & 2015 Elsevier Ltd. All rights reserved.

21 23 25 27 29

Keywords: User profiles Recommender systems User interactions Ensemble approaches

31 33

63 35 1. Introduction 37 39 41 43 45 47 49 51 53 55 57 59

Recommender Systems (RS) represent a technology that uses statistical and machine-learning techniques to recommend items to users based on the history of past activities. They became an important research area in the mid-90s and their first system, Tapestry, released in 1992, consisted in recommending documents and selecting emails based on user's information [1]. The recommendation task can be seen as a prediction problem: the system tries to predict the relevance of certain items to a user and, then, sorts them according to the relevance values provided. The importance of an item is generally represented by a numerical value that reflects the degree of user's interest during an interaction. The result of an RS is usually a set of items ordered in a descending order by importance scheduled for a given user [2]. Traditionally, recommender systems employ filtering techniques to generate appropriate recommendations to E-mail addresses: [email protected] (A.F. da Costa), [email protected] (M.G. Manzato).

users based on their profiles. However, techniques from other areas, such as Neural Networks, Bayesian Networks and Association Rules, are also used in the filtering process [2]. The most used types of filtering have been: (i) contentbased, responsible for selecting information based on descriptions or metadata of filtered elements, (ii) collaborative filtering, based on the relationship between people and their subject regarding the information to be filtered, and (iii) hybrid method, which combines the filtering based on content and collaborative [1]. Profiling mechanisms, which consist in acquiring, representing and maintaining pieces of information relevant (and/or irrelevant) to the user have been developed for the obtaining of user's interests. The three most known techniques are based on explicit feedback, implicit feedback and hybrid approaches. Implicit information is collected indirectly during user's navigation with the system while visiting a page, such as mouse movement and clicks on various links of interest. Respecting explicit feedback, the data are intentionally provided, i.e., users express themselves in some direct way (e.g. filling in forms or rating a content). Although this type of information is considered more reliable and since users provide the

http://dx.doi.org/10.1016/j.is.2015.09.007 0306-4379/& 2015 Elsevier Ltd. All rights reserved.

61 Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

65 67 69 71 73 75 77 79 81 83 85 87 89

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

2

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

topics of interests, the cost of the procedure is the effort of the individual, who is not always willing to cooperate with the system [2]. Finally, the hybrid approach consists in the application of implicit and explicit feedback together for the obtaining of a larger amount of user's information [2]. The actual scenario on the Web dictates the variety of interactions a user can adopt when consuming a content. Such multimodal interactions reflect the users' interests in finding relevant content and should be explored in more details by the system for a more accurate preferences profile and improved quality of recommendations [1]. However, the literature reports a lack of techniques that integrate different types of user's feedback into a generic model. Indeed, such a unified model could bring relevant improvements to recommender systems, because the way a user interacts with the system is never the same over time. Depending on many factors, such as user's mood, system's interface and/or contextual environment, a user may adopt a distinct interaction paradigm to access the content, and, consequently, will provide different types of feedback regarding preferences. We argue multimodal user interactions are an important factor to support accurate recommendations. This manuscript proposes a recommender system that contains three ensemble approaches, which uses different feedback types to generate a personalized ranking. The approaches use a post-processing step to ensemble rankings generated by unimodal-based state-of-the-art algorithms. We provide an experimental evaluation with two datasets, simulating and inferring interaction paradigms to validate our approaches. The paper is structured as follows: Section 2 addresses the related work; Section 3 describes a pair of unimodal recommender systems explored in the study; Section 4 presents the proposal in detail; Section 5 reports the evaluation conducted in the system; finally, Section 6 is devoted to final remarks and future works.

37 39

2. Related work

41 43

This section reports some studies related to our proposal. Multimodal recommendation approaches and a review of ensemble-based recommender systems are addressed.

45

2.1. Multimodal interactions

47

Several studies have emerged to deal with the integration of the increasing number of interactions between users and content, so that more information about the users' preferences can be gathered by the systems. Recommender systems can be extended to improve the understanding of users and items, including, for example, new types of interaction in the recommendation process and combining of them. One of such improvements is the support for multimodal interactions, which provides greater flexibility and less obtrusive types of recommendations [1]. Studies in the area of recommender systems have proposed several algorithms to process more than one type of user's interactions. Johansson [3] designed MADFILM, a movie recommender system that addresses the integration of prediction and

49 51 53 55 57 59 61

organization of content through explicit and implicit users' interactions. Denis et al. [4] developed a recommendation tool called Walk Talk, which analyzes and predicts relevant items according to the relationship between implicit and explicit user's information. Koren et al. [5] developed the SVDþ þ algorithm using ratings as explicit information and user browsing history as implicit feedback. Rendle et al. [6] proposed a technique called Factorization Machines (FM), which combines the advantages of Support Vector Machines (SVM) and factorization models. It considers different pieces of information from items, as user's feedback, to generate recommendations. In a recent study, Domingues et al. [7] developed a multimodal system facing music recommendation, which combines usage (web access) and content (i.e. audio features and textual tags). Some interactions were made in real time with real users on a commercial music site from the very Long Tail. The combination of data from the system resulted in better accuracy than content-based systems and led the system to greater user acceptance rate, higher rate of user activity and greater user loyalty and usage. However, the aforementioned studies considered few types of interactions and are not extensible for the integration of new types of user's feedback. Algorithms, as FM, which take into account several types of information calculate the similarity between the data through peer comparison, causing not so accurate results, for not taking into account the semantics of the data. The approach proposed in this paper adopts a post-processing step to analyze the rankings created separately by different algorithms. The advantage is that the model can be easily extended to other types of interactions and recommenders.

63

2.2. Ensemble approach

95

Ensemble is a machine-learning approach that uses a combination of similar models to improve the results from a single model. Several recent studies, such as [8], have demonstrated the effectiveness of an ensemble of several individual and simpler techniques and that ensemble-based methods outperform any single more complex algorithm. Bar et al. [9] proposed a systematic framework for the application of ensemble methods to CF. They employed automatic approaches to generate an ensemble of collaborative filtering models based on a single collaborative filtering algorithm (homogeneous ensemble). They proved the effectiveness of the framework by applying several ensemble methods to various CF-based methods. Ristoski et al. [10] discussed the development of a hybrid multi-strategy book recommender system using Linked Open Data. Their approach handles the training set in different types of recommender algorithms. The results of the individual recommenders are combined by the ensemble method and ranking aggregation. The authors proved that their approach could deliver very good results in different recommendation settings and also incorporate diversity of recommendations. However, their work is limited to the type of interactions chosen. Our proposal can be considered an ensemble-based technique, as it combines multiple rankings in a post-processing step. It differs from the related work because it analyzes multiple interaction paradigms from the user to generate a more accurate personalized ranking.

97

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

65 67 69 71 73 75 77 79 81 83 85 87 89 91 93

99 101 103 105 107 109 111 113 115 117 119 121 123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

1

3. Unimodal recommender systems

3

17

This paper proposes a recommender tool composed of three techniques that combine multiple rankings generated by unimodal recommenders. Each recommender uses either a single interaction or a subset of user's feedback types to generate a list of items. According to such rankings, a post-processing step is applied to ensemble them through a set of heuristics that analyzes the user's behavior during consumption. The set of recommenders is restricted to SVDþ þ [5] and BPR MF (Bayesian Personalized Ranking) [11], available in the literature. These models were chosen because they provide good results and have better computational times for the types of feedback considered. Prior to the description of the proposal, the selected algorithms will be revised in the next subsections.

19

3.1. Notation

21

Following the same notation in [5], special indexing letters distinguish users and items: a user is indicated as u and an item is referred to as i, j; rui refers to either explicit or implicit feedback from a user u to an item i. In the first case, it is an integer provided by the user indicating how much he liked the content; in the second, it is just a boolean showing whether the user consumed or visited the content or not. The prediction of the system on the preference of user u to item i is represented by r^ ui , which is a floating point value guessed by the recommender algorithm. The set of pairs ðu; iÞ for which rui is known is represented by K ¼ fðu; iÞjr ui is knowng. Additional sets are N(u), which indicates the set of items to which user u provided an implicit feedback, and NðuÞ, which indicates the set of items unknown to user u. Because the rating data are sparse, the models are prone to overfitting. Therefore, regularization is applied, so that estimates are shrunk towards baseline defaults. Similar to [5], we denote λ1 ; λ2 ; … the constants used for regularization.

5 7 9 11 13 15

23 25 27 29 31 33 35 37 39 41

3.2. SVDþ þ

43

In recommender systems, an important issue regards how to integrate different forms of user's input into the models for a precisely reflecting the user's preferences [1]. Algorithms usually rely only on explicit feedback, which includes ratings assigned by users to items they have visited in the past. A good example is Netflix,1 which enables users to choose and assign a number of stars to movies they have watched. The system constructs and controls the user's profile by considering each rating in his personal interests. On the other hand, explicit feedback may be not always available due to cold start, or simply because users may not provide any ratings for their preferences. Consequently, implicit feedback could be explored, as it is a more abundant source of information that indirectly reflects the user's opinion through the observation of their behavior [1]. Examples of implicit feedback are purchase or rental history, browsing activity, search patterns, etc.

45 47 49 51 53 55 57 59 61

3

Koren [5] proposed a model that faces implicit feedback when explicit feedback is also available. The algorithm integrates both types of feedback by considering ratings assigned by users to visited items and the rental history. As this type of implicit feedback is absent in his adopted dataset (Netflix), the author simulated such information by regarding which movies users rated, regardless of how they rated them movies. The SVDþ þ algorithm [5] integrates explicit and implicit feedback into a factorization model representing the user's preferences. Each user u is associated with a user-factor vector pu A Rf and each item i with an itemfactors vector qi A Rf . A popular prediction rule would be r^ ui ¼ bui þ pTu qi ;

ð1Þ

where baseline bui is defined as bui ¼ μþ bu þ bi and indicates the difference estimates of users and items in comparison to the overall rating average μ. All parameters are estimated through the minimization of the associated squared error function: X 2 2 min ðr ui  μ bu bi  pTu qi Þ2 þλðbu þ bi þ J pu J 2 þ J qi J 2 Þ: p ;q ;b

ðu;iÞ A K

ð2Þ Based on Eq. (1), Koren extended this basic model to consider implicit information. The author used an additional factors vector yi A Rf and also set N(u), which contains all items for which u provided an implicit preference. Therefore, the SVD þ þ model is defined as 0 1 X ^r ui ¼ bui þ qTi @pu þ jNðuÞj  1=2 ð3Þ yj A: j A NðuÞ

The preferences of a user u are represented by a combination of explicit and implicit information. The user-factor vector pu is learned from the given explicit ratings and complemented by the sum of yj, which represents the implicit feedback. Again, the parameters are learned by the minimization of the associated squared error function through gradient descent, as shown in Algorithm 1, where α is the learning rate. Algorithm 1. Learning SVDþ þ through gradient descent.

1

http://www.netflix.com

Input: Set of known ratings ðu; iÞ A K Output: Learned parameters bu ; bi ; pu ; qi ; yi for count¼ 1,…,Iteractions do   foreach ðu; iÞA K do  P    r^ ui ’bui þ qTi ðpu þ jNðuÞj  1=2 j A NðuÞ yj Þ;      eui ’r ui  r^ ui ;    bu ’bu þ αðeui  λ1 :bu Þ;     bi ’bi þ αðeui  λ1 :bi Þ;    p ’p þ αðeui q  λ2 :p Þ; u i u  u P    qi ’qi þ αðeui ðpu þ jNðuÞj  1=2 j A NðuÞ yj Þ  λ3 :qi Þ;    forall the jA NðuÞ do     j yj ’yj þ αðeui jNðuÞj  1=2 qi  λ4 :yj Þ;     end   end end

As previously described in this section, a limitation of the SVD þ þ model is although it integrates both implicit and explicit feedback from the user interaction, its training method works only if explicit feedback is available (set K).

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99 101 103 105 107 109 111 113 115 117 119 121 123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

4

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

Users must assign ratings to the items, so that the system can consider the multimodal interaction in the recommendation process. 3.3. BPR MF The BPR MF approach [11] provides a personalized ranking of items to a user according only to implicit feedback (e.g. navigation history, clicks, etc.). An important characteristic of this type of feedback is only the positive observations are known; the non-observed user-item pairs can be either an actual negative feedback. Rendle et al. [11] discuss a problem that emerges when an item recommendation model is trained based only on such positive/negative data. Because the observed entries are positive and the remaining are negative, the model will be fitted to provide positive scores only for observed items. The remaining elements, including those that may be of interest to the user, will be classified by the model as negative scores and the ranking cannot be optimized, as the predictions will be around zero. The authors proposed a generic method to learn the user's behavior for personalized ranking [11]. Instead of training the model using only the user-item pairs, they also considered the relative order between a pair of items, according to the user's preferences. If an item i has been viewed by user u and j has not (i A NðuÞ and jA NðuÞ), then i 4 u j, which means he prefers i over j. Fig. 1 shows an example of the method. When i and j are unknown to the user, or equivalently, both are known, no conclusion about their relative importance to the user can be inferred. To estimate whether a user prefers an item over another, Rendle et al. proposed a Bayesian analysis using the likelihood function for pði 4 u jjΘÞ and the prior probability for the model parameter pðΘÞ. The final optimization criterion, BPR-Opt, is defined as X ln σðs^ uij Þ ΛΘ J Θ J 2 ; ð4Þ BPR  Opt≔

randomly samples from DK to adjust Θ. Algorithm 2 shows an overview of the learning method. Algorithm 2. Learning through LearnBPR. Input: DK Output: Learned parameters Θ Initialize Θ with random values for count ¼1,…,Iteractions do   draw ðu; i; jÞ from DK   s^ ’r^  r^ ui uj  uij !   e  s^ uij ∂  Θ’Θ þ α ^ uij  ΛΘ Θ s :  ^  1þ e  s uij ∂Θ

63 65 67 69 71 73

end

75 In the present study, we have defined the BPR approach to consider the prediction rule r^ ui the simple factorization model defined in Eq. (1). Therefore, applying Eq. (1) in s^ uij yields Θ ¼ fbi ; bj ; pu ; qi ; qj g, which must be learned. We compute the partial derivatives in relation to s^ uij : 8 1 when Θ ¼ bi ; > > > > > 1 when Θ ¼ bj ; > > > > < q  q when Θ ¼ p ; ∂ i j u ð5Þ s^ uij ¼ > when Θ ¼ qi ; pu ∂Θ > > > > > when Θ ¼ qj ;  pu > > > :0 otherwise; which is then applied to Algorithm 2, so that the set of parameters Θ can be learned. Similar to Algorithm 1, we set ΛΘ ¼ fλ1 ; λ2 ; λ3 ; λ4 g.

77 79 81 83 85 87 89 91 93

4. Proposed recommender tool

95 97

53

This paper proposes a recommender tool composed of three ensemble algorithms for the combination of multiple user's interactions towards a more accurate personalized ranking in recommender systems. The tool contains a post-processing approach that combines the results of recommendations generated from each type of user's interaction. It also provides statistical test modules, evaluation metrics and integration with a recommender library that processes unimodal algorithms. The first two techniques are based on heuristics, ensemble individual results of each interaction generated by unimodal algorithms. For a more reliable and robust tool, the third technique was developed based on machine-learning, in which user's behavior is learned through training set. Perceiving user's preferences in every interaction enabled the tool to become able to learn which type of interaction is most relevant to him, and from this, assign weights to these interactions dynamically. The three techniques will be presented in the following subsections.

55

4.1. Extended notation

117

This subsection extends the notation used in Section 3.1. To represent the interactions of users, we have defined Rðu; ratingsÞ, Rðu; historyÞ and Rðu; tagsÞ as the rankings generated for user u for interactions ratings, viewing history and tags, respectively. Moreover, considering each

119

33 35 37 39

ðu;i;jÞ A DK

41 43 45 47

where s^ uij ≔r^ ui  r^ uj and DK ¼ fðu; i; jÞjiA NðuÞ&j A NðuÞg. Symbol Θ represents the parameters of the model, ΛΘ is the set of regularization constants, and σ is the logistic function defined as σðxÞ ¼ 1=ð1 þ e  x Þ. The authors also proposed a variation in the stochastic gradient descent technique, denominated LearnBPR, which

49 51

57 59 61

Fig. 1. The left-hand side table represents the observed data K. The Rendle et al. approach creates a user-specific pairwise relation i4 u j between two items. On the right-hand side of the table, the plus signal indicates user u is more interested in item i than in item j; the minus signal indicates the user prefers item j over i; the question mark indicates no conclusion can be inferred between the items.

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

99 101 103 105 107 109 111 113 115

121 123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

1 3 5 7 9 11

tags history ratings type of interaction, we have defined r^ ui , r^ ui and r^ ui to represent pairs of weights ðu; iÞ in each ranking. The concepts of ranking and weight are related to each other. Each unimodal algorithm generates a score (weight), which is a floating point number that represents how much a user likes an item by a specific interaction. The weights are then sorted in a descending order, and form the ranking of the items whose first item is the most relevant to the user, according to their preferences. For example, Rðu; tagsÞ contains a list of pairs ðu; iÞ with corresponding scores generated by a unimodal algorithm based on user's interaction u related to tagging.

13 4.2. Heuristics-based techniques 15 17 19

Initially, we developed two approaches based on a set of heuristics for specific domains. They consider the interactions with the highest abundance in the databases used and available information in each domain, such as ratings, browsing history and tags.

21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

4.2.1. Tag-based technique In the first approach developed, recommendations are generated based on multimodal user's interactions, whenever they are available. The approach consists of a post-processing step, responsible for combining the rankings obtained by recommendations of traditional algorithms addressed in Section 3. We used SVDþ þ to process explicit feedback and BPR MF for implicit data. The algorithm prioritizes the items that have appeared more than once in Rðu; partialÞ, which is the concatenation of the rankings composed of n recommendations for a user, and those that have received tags, as shown in the following equation: final ratings history tags r^ ui ¼ γ  ðr^ ui þ r^ ui þ β  r^ ui Þ;

Nu;tagsðiÞ ; NT;tagsðiÞ

Rðu; ratingsÞ, the more this item is closer to the user's preferences and also assign a greater weight to the tag (parameter β), since attach tags to an item requires more effort on the part of users than simply access an item or attribute a grade. Therefore, the greater the number of labels assigned to an item, the greater its relevance in the proposed technique.

ð7Þ

where N u;tagsðiÞ is the number of tags assigned by user u to item i and NT;tagsðiÞ is the total number of tags assigned to item i. This approach considers only implicit information of the labels, i.e., it considers only if user assigned tag or not (0 or 1), discarding semantic value. This heuristic is supported by the fact the higher the frequency of the item in Rðu; tagsÞ, Rðu; historyÞ and

63 65 67 69

4.2.2. Average-based technique In order to generalize the use of the technique based on tags for other databases and the use of other types of interaction, we have developed a second ensemble approach based on the averages of the scores of each type of interaction provided by users. We have defined Rðu; aÞ, Rðu; bÞ and Rðu; cÞ rankings generated for a user u for any interactions, i.e, a b c a, b and c, respectively, and r^ ui , r^ ui and r^ ui to represent the pairs of weights (u,i) in each ranking. After the generation of the rankings for each type of interaction, the technique combines pairs ðu; iÞ of all rankings, as shown in Algorithm 3.

71 73 75 77 79 81

Algorithm 3. Technique based on averages algorithm. Input: Rðu; aÞ; Rðu; bÞ; Rðu; cÞ Output: Final ranking R0 ðu; finalÞ Rðu; partialÞ’Rðu; aÞ \ Rðu; bÞ \ Rðu; cÞ Compute avg Rðu;aÞ , avg Rðu;bÞ and avg Rðu;cÞ for ðu; iÞ A Rðu; parcialÞ do   b c  if r^ aui Z avg Rðu;aÞ &^ r ui Z avg Rðu;bÞ &^ r ui Z avg Rðu;cÞ then   final  jr^ ui ’ðr^ aui þ r^ bui þ r^ cui Þ=3   end    else   jr^ final ’maxðr^ a ; r^ b ; r^ c Þ  ui ui ui ui   end  final   Aggregate r^ ui into Rðu; finalÞ end R0 ðu; finalÞ’sort_descðRðu; finalÞÞ

ð6Þ

where γ is the number of interactions made by user u to item i and β ponders the weights assigned to tags, defined in the following equation: β ¼ 1þ

5

83 85 87 89 91 93 95 97

A partial ranking Rðu; partialÞ containing pairs ðu; iÞ in all isolated rankings is created. The weights of each pair ðu; iÞ belonging to Rðu; partialÞ are then calculated through the a;b;c checking whether the weight of r^ ui is higher than the average of all weights of the corresponding interaction. If final all results have satisfied the condition, final value r^ ui is the arithmetic average of the three values, otherwise, it receives the higher weight of the interactions. Finally, pairs ðu; iÞ are sorted in a descending order according to their weights, which results in a final ranking R0 ðu; finalÞ. The heuristic used in this technique is also based on the frequency of appearance of pairs ðu; iÞ, however, it aims at addressing each type of interaction evenly and gives equal

99 101 103 105 107 109 111

51

113

53

115

55

117

57

119

59

121

61

Fig. 2. Schematic visualization of the proposed system.

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

6

1 3

Table 1 Constants used in the evaluation. Constant Value

Note

λ1 λ2

0.33 0:015jRðuÞj  1=2

R(u) is the set of items known by user u

7

λ3;4

0:015jRðiÞj  1=2

R(i) is the set of users who provided an explicit/implicit interaction to item i

9

α λa;b;…;n

0.05 0.00025

5

weight to all the terms of the function. It also aims at normalizing the weights of the relevant items (which have all types of interactions) in the final ranking using a weighted average of the weights, so that any weight of any interaction protrudes to the other.

63

4.3. Ensemble BPR learning

69

As described in the previous section, the approaches based on heuristics have been defined for the areas con-

71

65 67

11

73

13

75

15

77

17

79

19

81

21

83

23

85

25

87

27

89

29

91

31

93

33

95

35

97

37

99

39

101

41

103

43

105

45

107

47

109

49

111

51

113

53

115

55

117

57

119

59

Fig. 3. Number of interactions made by users in the training bases of LastFM (n represents number of interactions). (a) Viewing history. (b) Assigning tags. (c) Concatenated sets.

121 123

61 Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

7

1

63

3

65

5

67

7

69

9

71

11

73

13

75

15

77

17

79

19

81

21

83

23

85

25

87

27

89

29

91

31

93

33

95

35

97

37

99

39

101

41

103

43

105 107

45 47 49 51 53 55 57 59 61

Fig. 4. Number of interactions made by users in the MovieLens (n represents number of interactions). (a) Ratings. (b) Assigning tags. (c) Viewing history. (d) Concatenated sets.

sidered and set of available types of interactions. However, they are not generic enough to support other domain types or user's interaction types. In order to solve this problem, we have developed a new technique based on machine-learning, in which parameters are learned according to the behavior of each user along with their interaction in the system. The idea is that the system will learn from the training sample how much each interaction contributes to the final recommendations for each user. For instance, certain interactions require more user's effort to be accomplished (e.g. tagging, rating, commenting, etc.), whereas others are executed instantaneously (e.g. viewing, clicking, etc.). It is expected that the more effort is needed from the user to interact with

the item, the more its subject caught his attention. Our technique, thus, exploits this hypothesis proposing a learning phase which that construct a more accurate user's profile. The learning of the user's behavior used in this approach is an extension of BPR algorithm presented in Section 3.3. However, in its original form, the model can only process implicit feedback and cannot combine more than one type of interaction. On the other hand, the learning process can adjust the parameters of an ensemble of model-based learning. Fig. 2 shows a representation of the proposed approach. Initially, we extract the implicit and explicit user's information in the database and process each feedback in its respective unimodal recommender. We used BPR MF and SVDþ þ,

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

109 111 113 115 117 119 121 123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

8

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53

Table 2 MAP results with standard deviation for LastFM 2k. All results of the proposed Ensemble BPR Learning are statistically significant in comparison to baselines (p-value o 0:05). Top 1 BPR MF (Viewing History) MAP 0.00424 σ 0.00012 BPR MF (Tags) MAP 0.00265 σ 0.00014 LibFM MAP 0.00791 σ 0.00003 Tag-based MAP 0.00319 σ 0.00014 Average-based MAP 0.00744 σ 0.00049 Ensemble BPR Learning MAP 0.01116 σ 0.00042

Top 3

Top 5

Top 10

0.00610 0.00006

0.00778 0.00016

0.008309 0.00020

0.00848 0.00052

0.01047 0.00028

0.01287 0.00045

0.01873 0.00023

0.02110 0.00002

0.02574 0.00032

0.01116 0.00052

0.01355 0.00028

0.01574 0.00045

0.01355 0.00012

0.01501 0.00007

0.01682 0.00019

0.03003 0.00013

0.03530 0.00006

0.03927 0.00023

Table 3 Precision results with standard deviation for LastFM 2k. All results of the proposed Ensemble BPR Learning are statistically significant in comparison to baselines (p-value o 0:05). Top 1 BPR MF (Viewing History) Precision 0.00424 σ 0.00001 BPR MF (Tags) Precision 0.00265 σ 0.00014 LibFM Precision 0.00791 σ 0.00008 Tag-based Precision 0.00319 σ 0.00003 Average-based Precision 0.00744 σ 0.00021 Ensemble BPR Learning Precision 0.01116 σ 0.00009

Top 3

Top 5

Top 10

0.00212 0.00003

0.00244 0.00010

0.00159 0.00005

0.00336 0.00012

0.00350 0.00008

0.00334 0.00010

0.00820 0.00012

0.00816 0.00052

0.00763 0.00101

0.00460 0.00018

0.00446 0.00009

0.00366 0.00013

0.00496 0.00032

0.00404 0.00006

0.00324 0.00008

0.01275 0.00012

0.01137 0.00002

0.00829 0.00003

presented in Section 3, to process implicit and explicit feedback, respectively. Each recommender generates custom rankings for each user based on the feedback processed. At the same time, a machine-learning module learns the user's preferences based on his feedback. The learned weights indicate how much each user can express his preference with each type of interaction. Finally, the results generated by each unimodal recommender are combined using the learned weights. The proposed approach consists of three steps, each one detailed in the following subsections.

55 57 59

4.3.1. Step 1: data division and execution of unimodal algorithms The database is divided into training and test sets. The training set is also divided into two subsets, of which the

first is used for training and generating results for each type of interaction and the second is used for learning the weights in Step 2. At this stage, each interaction is processed by the respective recommendation algorithm, i.e. explicit feedback (e.g. ratings) is used to train an SVDþ þ instance which will generate an interaction-specific individual ranking and each type of implicit feedback (e.g. browsing history, tagging) is used to train a BPR MF instance that also generates an individual ranking for each interaction. This approach enables the integration of various recommender algorithms and interaction types.

63

4.3.2. Step 2: learning of weights for ensembling Eq. (8) calculates the final score of each pair ðu; iÞ final represented by r^ ui :

75

final r^ ui

¼

a b n βa :r^ ui þ βb :r^ ui þ ⋯ þ βn :r^ ui ;

ð8Þ

where βa, βb,…, βn are weights generated from the learning step of each type of interaction, and control how much each interaction will contribute to the final score. As explained, these weights are learned using the LearnBPR final algorithm presented in Section 3.3. Thus, r^ ui is applied to the previous definition of s^ uij , i.e. s^ uij ≔r^ ui  r^ uj and yields a a b b n n s^ uij ≔βa ðr^ ui  r^ uj Þ þβb ðr^ ui  r^ uj Þ þ⋯ þβn ðr^ ui  r^ uj Þ:

ð9Þ

a b n r^ ui ; r^ ui ; …; r^ ui

were computed in step 1, the set of As parameters to be learned is defined as Θ ¼ fβa ; βb ; …; βn g and corresponding set of regularization constants ΛΘ ¼ fλa ; λb ; …; λn g. Therefore, we have 8 a a > r^ ui  r^ uj when Θ ¼ βa ; > > > > b < r^  r^ b when Θ ¼ β ; ∂ b ui uj ð10Þ s^ uij ¼ > ∂Θ >… > > n n > : r^ ui  r^ uj when Θ ¼ βn ; which is applied to LearnBPR, so that the weights can be learned. New items can be predicted from its application by Eq. (8).

65 67 69 71 73

77 79 81 83 85 87 89 91 93 95 97 99 101

4.3.3. Step 3: ensemble and recommendation This step consists in the sum of the weights of rankings generated by unimodal algorithms for each type of interaction. The scores of each ranking are weighted with the parameters set (βa, βb,…, βn) that gives relevance to the type of interaction between the user and the item. Finally, the values are sorted in a descending order, which results in the final ranking R0 ðu; finalÞ. Algorithm 4 shows this process. Algorithm 4. Ensemble BPR learning algorithm. Input: Set of interactions a; b; …; n Output: Final Ranking R0 ðu; finalÞ for u A users do  i A items do  for   a b n    Compute r^ ui ; r^ ui ; …; r^ ui    Compute βa ; βb ; …; βn  final    Compute r^ ui  final   Aggregate r^ ui into Rðu; finalÞ   end end R0 ðu; finalÞ’sort_descðRðu; finalÞÞ

103 105 107 109 111 113 115 117 119 121 123

61 Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

9

1

63

3

65

5

67

7

69

9

71

11

73 75

13 Fig. 5. MAP results for LastFM 2k.

15

77

17

79

19

81

21

83

23

85

25

87

27

89

29

91

31

Fig. 6. Precision results for LastFM 2k.

93

33

95

35

97

37

99

39

101

41

103

43

105

45

107 109

47 Fig. 7. MAP results for MovieLens 2k.

111

49

113

51 5.1. Used resources 53 55 57 59 61

5. Evaluation Our ensemble techniques, discussed in Section 4, were compared among themselves and with traditional recommendation algorithms presented in Section 3. We also used Factorization Machines (FM) [6] as baseline. It is a state-of-the-art generic approach that enables to mimicking of most factorization models by feature engineering.

115 The recommender tool proposed in this paper was developed in Python2 version 2.7, with NumPy3 and SciPy4

117 119

2 3 4

http://www.python.org/ http://www.numpy.org/ http://www.scipy.org/

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

121 123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

10

1

63

3

65

5

67

7

69

9

71

11

73

13

75

15

77

Fig. 8. Precision results for MovieLens 2k.

79

17 19 21

Table 4 MAP results with standard deviation for MovieLens 2k. All results of the proposed Ensemble BPR Learning are statistically significant in comparison to baselines (p-value o 0:05). Top 1

23 25 27 29 31 33 35 37 39

SVDþ þ (Ratings) MAP 0 σ 0.00010 BPR MF (Viewing History) MAP 0 σ 0.00003 BPR MF (Tags) MAP 0.00378 σ 0.00021 LibFM MAP 0.00954 σ 0.00005 Tag-based MAP 0.00455 σ 0.00015 Average-based MAP 0.00886 σ 0.00047 Ensemble BPR Learning MAP 0.01464 σ 0.00062

Top 3

Top 5

Top 10

0 0.000021

0.00015 0.0008

0.00015 0.00005

0 0.00001

0.000118 0.00089

0.000118 0.00034

0.00425 0.00087

0.00520 0.00001

0.00637 0.00081

0.01342 0.00009

0.02919 0.00004

0.03020 0.00012

0.00930 0.00023

0.01068 0.00012

0.01162 0.00031

0.01944 0.00032

0.02372 0.00008

0.02933 0.000043

0.02943 0.00013

0.03414 0.00010

0.03839 0.00032

Table 5 Precision results with standard deviation for MovieLens 2k. All results of the proposed Ensemble BPR Learning are statistically significant in comparison to baselines (p-value o 0:05). Top 1 SVDþ þ (Ratings) MAP 0 σ 0.00007 BPR MF (Viewing History) MAP 0.00378 σ 0.00001 BPR MF (Tags) MAP 0 σ 0.00016 LibFM MAP 0.00954 σ 0.00014 Tag-based MAP 0.00455 σ 0.00003 Average-based MAP 0.00886 σ 0.00021 Ensemble BPR Learning MAP 0.01464 σ 0.00011

Top 3

Top 5

45 47 49 51 53 55 57

85 0 0.000018

0.00009 0.00014

0.00004 0.00006

0.00157 0.00003

0.00160 0.00021

0.00160 0.00007

0 0.00006

0.00009 0.00001

0.00004 0.00008

0.00994 0.00018

0.00916 0.00009

0.00764 0.00008

95

0.00350 0.00018

0.00301 0.00009

0.00214 0.00013

97

0.00973 0.00045

0.00904 0.00012

0.00749 0.00007

99

0.01157 0.00074

0.00979 0.00003

0.00792 0.00005

87 89 91 93

101 103

libraries, responsible for data optimization and matrix structures. Its source-code is freely available on Github.5 The recommendation algorithms integrated into the tool belong to MyMediaLite library [12], an open-source tool developed in C with numerous features and algorithms for recommender systems. Among the algorithms implemented are SVDþ þ and BPR MF, both used in this study. For FM, we used the open-source LibFM library,6 whose input data were configured according to the guidelines defined in [6]. 5.2. Evaluation methodology We adopted the All But One (Leave One Out) protocol [13] to evaluate the proposed techniques. We divided the dataset into two sets, Tr and Te, composed of 80% and 20%

59 5

61

83

Top 10

41 43

81

6

http://www.github.com/ArthurFortes/AF_Recommender_tool/ http://www.libfm.org/

of data, respectively. Tr was also divided into two subsets, 1 2 1 Tr and Tr , composed of 80% and 20% of Tr, respectively. Tr subset was used to train the isolated algorithms and gen2 a;b;…;n . The Tr subset erate the interaction-specific scores r^ ui was used to train the ensemble parameters βa;b;…;n . We randomly hid an item in Te set from each user to create the truth set H and computed Precision and Mean Average Precision (MAP) [14] as follows: Precision calculates the percentage of relevant recommended items. It is calculated by comparing, for each user in the test set Te, the set of recommendations R that the system makes against the set H: jT e j 1 X jRu \ H u j : PrecisionðT e Þ ¼ jT e j u ¼ 1 jRu j

ð11Þ

Mean average precision computes the precision considering the respective position in the ordered list of recommended items. A single value accuracy score is

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

105 107 109 111 113 115 117 119 121 123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

1

obtained for a set of test users Te:

3

MAP ðT e Þ ¼

5

where the average precision (AveP) is given by

7 9 11 13 15 17 19

jT e j 1 X AveP ðRu ; H u Þ; jT e j u ¼ 1

AveP ðRu ; H u Þ ¼

jHu j 1 X ½PrecðRu ; r Þ  δðRu ðr Þ; H u Þ; jH u j r ¼ 1

ð12Þ

ð13Þ

where PrecðRu ; rÞ is the precision for all recommended items up to ranking r and δðRu ðrÞ; H u Þ ¼ 1, if the predicted item at ranking r is a relevant item ðRu ðrÞ A H u Þ or zero, otherwise. We used Precision@N and MAP@N, where N took values of 1, 3, 5 and 10 in the rankings returned by the system. 1 2 All experiments (including Tr , Tr and Te division) were executed 10 times, whose values were summarized by mean and standard deviation. We applied the two-sided paired t-test with a 95% confidence level [15] to compare the results in a statistical form.

21 5.3. Datasets and preliminary analysis 23 25 27

MovieLens 2k and LastFM 2k datasets, both based on Cantador et al. work [16], were used as case studies for the evaluation of the proposed techniques.

 LastFM 2k: consists of 92,834 user-listened artist rela-

29 31 33 35 37 39 41



tions and 186,479 interaction tags applied by 1892 users to 17,632 artists. As feedback types, we considered: (i) whether a user tagged an item or not; and (ii) the history of visited items, simulated by boolean values (visited or not) generated by the ratings and tagging activities. MovieLens 2k: consists of 800,000 ratings and 10,000 interaction tags applied to 2113 users and 10,197 movies. As explicit information, we used the ratings users assigned to items and as implicit information, we considered: (i) whether a user tagged an item or not; and (ii) the history of visited items, simulated by boolean values (visited or not) generated by the ratings and tagging activities.

43 45 47 49 51 53 55 57 59 61

For both datasets, we experimentally defined the constants involved according to Table 1. The preliminary analysis in the datasets highlighted two related problems which can be reduced by our multimodal interactions approach: new user (cold start) and sparsity. We also analyzed quantitative data concerning the number of interactions per users, distribution of interactions, number of interactions for items, among others. The interaction types (visualization history, ratings and tags set) were isolated and analyzed. For the LastFM 2k dataset, we set as new users those who made less than 10 interactions with items. Figs. 3(a) and (b) show the graphics with each individual type of interaction, with 94 new users (4.99%) containing only visualization history and 1038 new users (55.48%) who only tagged the items. When we concatenate the two types of interactions in a single set, a new scenario is depicted in Fig. 3(c), which we can reduce the number of new users in the training set.

11

Therefore, it is strong evidence that the use of multi modal interactions can improve the accuracy of the recommendations. The same analysis was conducted for the MovieLens 2k dataset. Due to the higher number of interactions and diversity of film genres, we adopted users who had made less than 20 interactions as new users. Figs. 4(a), (b) and (c) show graphics that consider each type of interaction individually, presenting 20 new users (0,95%) who only rated the items, 1899 new users (89,87%) who only tagged the items, and 13 new users (0,61%) containing only the viewing history. Fig. 4(d) shows the new scenario after the three types of interaction had been concatenated, where we were able to reduce the number of new users, evidencing again the possibility to improve the accuracy of recommendations.

63 65 67 69 71 73 75 77 79

5.4. Results 81 Based on the preliminary analysis reported in the previous section, in this section we present the evaluation results conducted in the proposed recommender tool using both LastFM 2k and MovieLens 2k datasets. 5.4.1. LastFM 2k In this experiment, we considered two different types of implicit interactions available in the LastMF 2k dataset, visualization history and tagging. In order to show evidence to support the hypothesis of this research, we compare our techniques with other algorithms that consider different types of interactions, like SVDþ þ and LibFM, as well as with the traditional BPR MF model with individual implicit feedback. Tables 2 and 3 show the results and standard deviation (σ) for each technique, indicating that the results of each of them are statistically different and non-zero. Figs. 5 and 6 illustrate the same results in Top@N vs. MAP and Top@N vs. Precision graphs. 5.4.2. MovieLens 2k This dataset contains several implicit and explicit interactions. Figs. 7 and 8 show charts of comparisons between the results of the considered approaches, while Tables 4 and 5 show the results and standard deviation (σ) of the samples of each technique. 5.4.3. Discussion Before discussing the experiments, it is worth mentioning that the results described in this study have low values due to the evaluation protocol used. The All-ButOne protocol hides one item of each user in the test set and define these hidden pairs (u,i) as the truth set [13]. As N items were recommended for each user, the probability that the hidden items are recommended is short, reflecting in Precision and MAP values. This also happens for the rating-based ranking, which is constructed by predicted scores in descending order. As the hidden item may have a low score, the results of the metrics for this interaction may be very low. Therefore, it is important to rely only on the relative differences between the results shown by the experiments.

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

83 85 87 89 91 93 95 97 99 101 103 105 107 109 111 113 115 117 119 121 123

12

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

For both datasets composed of different interaction types and distinguished domains, the results generated by the proposed tool show the combination of two or more types of interaction produced better results in most ranking N positions. This occurs because these techniques seek to combine all user interactions in order to make the user profile more accurate, making the recommendations best suited to his/her preferences. Indeed, the heuristic-based techniques proposed in this paper (tag-based and average-based) outperformed the isolated algorithms (BPR MF for LastFM 2k and BPR MF/ SVDþ þ for MovieLens 2k). In some situations, however, both heuristic-based techniques were not able to produce better results than Factorization Machines (LibFM), mainly because this technique has a general scope while the heuristic-based methods were developed to work with specific interaction types. On the other hand, it is possible to note that the proposed Ensemble BPR Learning provides the best quality of recommendations, when compared to the other techniques addressed and baselines. This is due to the ability of the algorithm that can learn the users' preferences through their interactions, and use this information to match the recommendations generated individually for each type of interaction. Such a feature is particularly important mainly when we compare the results of the proposed ensemble technique against LibFM. Because FM will interact with all features interchangeably, the semantics of each interaction will have the same importance in the model. Our technique, in turn, will learn from the training data how much each interaction can affect the final recommendations for each user. For instance, a user may use a complex interaction type (e.g. sharing) with an item he really liked, while using a simpler interaction type (e.g. viewing) with those items which are on average. We argue such semantic differences between interaction types is what make our model generate more accurate recommendations.

39 6. Final remarks 41 43 45 47 49 51 53 55 57 59 61

This paper proposed ensemble approaches for the unification of different types of feedback from users when consuming content in order to provide better recommendations. The advantage is that more information about the user's interests can be obtained in the analyses of multimodal interactions. In contrast to current approaches, which are limited to one or a small subset of user's feedback and result in an inaccurate representation of users' preferences, the proposed model incorporates the characteristic of use of various types of interactions, but still takes advantage of the state-of-the-art algorithms based on one or few types of feedback. We conducted experiments in two datasets and the results show the effectiveness of combining various types of interactions into a single model for recommendation using ensemble approaches. The main advantages of our tool are extensibility and flexibility. It also enables developers to use different recommender algorithms, as it uses only the output from these techniques to ensemble the partial rankings. Furthermore, this flexibility avoids

additional efforts for combining techniques in an efficiently, as our model has a training phase which will learn the user's behavior in each interaction for the final recommendation. The tool contains three recommender techniques, with each an application domain. Tag-based technique is better suited when the number of tags is greater than the number of other types of feedback. Examples of systems that could benefit from this technique are Flickr7 and Delicious,8 as they share systems that use tags to describe items. Averagebased technique works well in systems whose user has available all interactions with an item, such as e-commerce, in which the user can access an item and buy an item. Finally, Ensemble BPR Learning, as described in the experiments, works well in different domains because it has a generic scope, but as opposite to FM, it will learn the semantics and importance of each interaction as soon as it has learned from a training sample. As future work, we aim at evaluating the system with additional datasets from other domains and checking its accuracy with different interaction types. We also plan to extend the ensemble module to add a user-based clustering procedure, so that computational cost could be saved with data constraints.

63

Acknowledgments

89

The authors acknowledge the financial support from FAPESP, CNPq and CAPES.

91

References [1] F. Ricci, L. Rokach, B. Shapira, P.B. Kantor, Recommender Systems Handbook, 1st edition, . Springer-Verlag, Inc., New York, NY, USA, 2010 URL: 〈http://dl.acm.org/citation.cfm?id ¼ 1941884〉. [2] G. Adomavicius, A. Tuzhilin, Context-aware recommender systems, in: Recommender Systems Handbook, Springer, US, 2011, pp. 217– 253. URL: 〈http://dx.doi.org/10.1007/978-0-387-85820-3_7〉. [3] P. Johansson, Madfilm – a multimodal approach to handle search and organization in a movie recommendation system, in: Proceedings of the 1st Nordic Symposium on Multimodal Communication, 2003. URL: 〈http://citeseerx.ist.psu.edu/viewdoc/summary?doi¼ 10. 1.1.134.307〉. [4] D. Parra, X. Amatriain, Walk the talk: analyzing the relation between implicit and explicit feedback for preference elicitation, in: Proceedings of the 19th International Conference on User Modeling, Adaption, and Personalization, UMAP'11, Springer-Verlag, Berlin, Heidelberg, 2011, pp. 255–268. URL: 〈http://dl.acm.org/citation.cfm? id ¼2021855.2021878〉. [5] Y. Koren, R. Bell, C. Volinsky, Matrix factorization techniques for recommender systems, Computer 42 (8) (2009) 30–37. URL: 〈http:// dx.doi.org/10.1109/MC.2009.263〉. [6] S. Rendle, Factorization machines with libFM, ACM Trans. Intell. Syst. Technol. 3 (3) . 57:1–57:22. URL: 〈http://doi.acm.org/10.1145/ 2168752.2168771〉. [7] M. Domingues, F. Gouyon, A. Jorge, J. Leal, J. Vinagre, L. Lemos, M. Sordo, Combining usage and content in an online recommendation system for music in the long tail, Int. J. Multimed. Inf. Retr. 2 (1) (2013) 3–13. URL: 〈http://dx.doi.org/10.1007/s13735-012-0025-1〉. [8] M. Jahrer, A. Töscher, R. Legenstein, Combining predictions for accurate recommender systems, in: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data

65 67 69 71 73 75 77 79 81 83 85 87

93 95 97 99 101 103 105 107 109 111 113 115 117 119 121

7 8

https://www.flickr.com/ https://www.delicious.com

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

123

A.F. da Costa, M.G. Manzato / Information Systems ∎ (∎∎∎∎) ∎∎∎–∎∎∎

1 3

[9]

5 7

[10]

9

Q3 [11]

11 13 [12]

15

Mining, KDD '10, ACM, New York, NY, USA, 2010, pp. 693–702. URL: 〈http://doi.acm.org/10.1145/1835804.1835893〉. A. Bar, L. Rokach, G. Shani, B. Shapira, A. Schclar, Improving simple collaborative filtering models using ensemble methods, in: Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 7872, Springer, Berlin, Heidelberg, 2013, pp. 1–12. URL: 〈http://dx.doi.org/ 10.1007/978-3-642-38067-9_1〉. P. Ristoski, E. Loza Menca, H. Paulheim, A hybrid multi-strategy recommender system using linked open data, in: Semantic Web Evaluation Challenge, Communications in Computer and Information Science, vol. 475, Springer International Publishing, 2014, pp. 150–156. URL: 〈http:// dx.doi.org/10.1007/978-3-319-12024-9_19〉. S. Rendle, C. Freudenthaler, Z. Gantner, L. Schmidt-Thieme, BPR: Bayesian personalized ranking from implicit feedback, in: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09, AUAI Press, Arlington, Virginia, United States, 2009, pp. 452–461. URL: 〈http://dl.acm.org/citation.cfm?id¼ 1795114.1795167〉. Z. Gantner, S. Rendle, C. Freudenthaler, L. Schmidt-Thieme, MyMediaLite: a free recommender system library, in: Proceedings of the

[13]

[14]

[15]

[16]

13

Fifth ACM Conference on Recommender Systems, RecSys '11, ACM, New York, NY, USA, 2011, pp. 305–308. URL: 〈http://doi.acm.org/10. 1145/2043932.2043989〉. J.S. Breese, D. Heckerman, C. Kadie, Empirical analysis of predictive algorithms for collaborative filtering, in: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, UAI'98, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1998, pp. 43–52. URL: 〈http://dl.acm.org/citation.cfm?id¼ 2074094.2074100〉. E.M. Voorhees, D.K. Harman, TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing), The MIT Press, 2005 URL: 〈http://dl.acm.org/citation.cfm?id¼ 1121636〉. T.M. Mitchell, Machine Learning, 1st edition, . McGraw-Hill, Inc., New York, NY, USA, 1997 URL: 〈http://dl.acm.org/citation.cfm? id ¼541177〉. 2nd Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011), RecSys 2011, ACM, New York, NY, USA, 2011. URL: 〈http://ir.ii.uam.es/hetrec2011/〉.

Please cite this article as: A.F. da Costa, M.G. Manzato, Exploiting multimodal interactions in recommender systems with ensemble algorithms, Information Systems (2015), http://dx.doi.org/10.1016/j.is.2015.09.007i

17 19 21 23 25 27 29 31