# Tag Info

25

$\DeclareMathOperator*{\argmin}{arg\,min}$ Ok, when you say SVD, presumably you're talking about truncated SVD (where you only keep the $k$ biggest singular values). There are two different ways to look at the truncated SVD of a matrix. One is the standard definition: First you do the SVD: $\underset{n\times m}{X} = \underset{n\times n}{U} \overset{n\... 24 I would like to offer a dissenting opinion: Missing Edges as Missing Values In a collaborative filtering problem, the connections that do not exist (user$i$has not rated item$j$, person$x$has not friended person$y$) are generally treated as missing values to be predicted, rather than as zeros. That is, if user$i$hasn't rated item$j$, we want to ... 14 I suggest to use Expected Utility or R-score. Assume your model has created an ordered list of recommendations where the first item is the one the user is most likely and the last is the one the user is least likely interested in. Let's say that this recommendations are specified by$r_i$where$i$is the position in the list. Expected utility for a ... 14 The reason no one tells you what to do with it is because if you know what SVD does, then it is a bit obvious what to do with it :-). Since your rows and columns are the same set, I will explain this through a different matrix A. Let the matrix A be such that rows are the users and the columns are the items that the user likes. Note that this matrix need ... 14 This is actually a relatively famous problem in the field of machine learning. In ~2006 Netflix offered$1m to the algorithm that provided the best reasonable improvement to their recommender system. The theory of the winning solution is briefly discussed in this Caltech textbook on introductory machine learning. Basically an ensemble learning method was ...

14

I am not a specialist in recommender systems, but as far I understand, the premise of this question is wrong. Non-negativity is not that important for collaborative filtering. The Netflix prize was won in 2009 by BellKor team. Here is the paper describing their algorithm: The BellKor 2008 Solution to the Netflix Prize. As is easy to see, they use an SVD-...

11

Why are you considering a neural network before completely understanding the problem? Standard matrix factorization methods for collaborative filtering are able to leverage content features easily. For an example of how this can be done in a Bayesian setting see the Matchbox paper.

11

Half the challenge in these problems is knowing what to search for. You might have added the tag without realizing it, but you're in fact looking for info on recommender systems. You might want to start with collaborative filtering, or better yet the Introduction to Recommender Systems Handbook by Ricci, Rokach, and Shapira cited on that page.

10

An 800+ page definitive guide from the top experts in the field (pricey though): Recommender Systems Handbook. Each chapter is written by different folks (one could try googling specific chapters - some of them are freely available on the web)

9

Latent means not directly observable. The common use of the term in PCA and Factor Analysis is to reduce dimension of a large number of directly observable features into a smaller set of indirectly observable features.

9

Since your training matrix decomposition with gradient descent, I assume you have some loss function $L(X - PQ)$ where $L$ is squared Frobenius norm or something similar. When you add a new user (let's say rows of $X$ correspond to users, and $x$ is new user's row, so that $X$' is $X$ with concatenated $x$) your objective becomes $$L(X' - P'Q)$$ and if ...

8

I really recommend the paper Collaborative filtering with temporal dynamics by Yehuda Koren (Netflix Contest !) where this issue is discussed in detail. I agree with the author, that the first option ("cutting off") is not the way to go. It is true that outdated preferences are ignored that way, but a) some preferences do never change, hence one kills data ...

8

For a very basic introduction you could check out chapter 2 of Programming Collective Intelligence.

8

We use logistic loss for implicit matrix factorization at Spotify in the context of music recommendations (using play counts). We've just published a paper on our method in an upcoming NIPS 2014 workshop. The paper is titled Logistic Matrix Factorization for Implicit Feedback Data and can be found here http://stanford.edu/~rezab/nips2014workshop/submits/...

8

Evaluating implicit feedback based recommendations is tricky. Here are couple of approaches that I'd recommend. Approach 1 You can use modified precision and recall metrics. Overall the procedure would be as follows, Divide your data into train and test set by users. For a user in test set, given their history, get the top N recommendations using implicit ...

7

In case of a "top-N" recommender system, it is helpful to construct an "unbiased" test data set (e.g. by adding a thousand random unwatched/unrated movies to the list of watched movies from the holdout data set for a given user), and then scoring the resulting test data set using a model. Once it is done for a bunch of users, one can then calculate "...

7

However: With pure vanilla SVD you might have problems recreating the original matrix, let alone predicting values for missing items. The useful rule-of-thumb in this area is calculating average rating per movie, and subtracting this average for each user / movie combination, that is, subtracting movie bias from each user. Then it is recommended you run SVD, ...

7

Matrix factorization is a method to, well, factorize matrices. It does one job of decomposing a matrix into two matrices such that their product closely matches the original matrix. But Factorization Machines are quite general in nature compared to Matrix Factorization. The problem formulation itself is very different. It is formulated as a linear ...

7

An excellent question! One trivial difference that I can think of, is that market basket (MB) analysis considers each basket separately. So if you buy the same stuff together once a month, each time it constitutes a different basket, and it likely also contains different items each time. However collaborative filtering (CF) considers baskets aggregated per ...

6

Yes, in practice those values are skipped. In your description in terms of a Frobenius norm, this corresponds to minimising the components of the norm which can be measured, i.e. those which have known ratings. The regularisation term can be seen as a Bayesian prior on the components of the feature vectors, with the SVD calculating the maximum likelihood ...

6

Three papers about integrating matrix factorization with content features (here, topic model specifically): Deepak Agarwal and Bee-Chung Chen. 2010. fLDA: matrix factorization through latent dirichlet allocation. In Proceedings of the third ACM international conference on Web search and data mining (WSDM ’10). ACM, New York, NY, USA, 91-100. Hanhuai Shan ...

6

As the number of words that you condition $P(T_1|W_1,..,W_N)$ on increases, you will find that calculating this probability will no longer be easy. The more conditions you add, the sparser your conditional probability tables will get. Naive Bayes makes the assumption that the probability of words occurrences are independent of each other conditioned on the ...

6

You should check out Andrew Ng's course on Coursera: https://www.coursera.org/learn/machine-learning It has a lesson on building recommender systems, which appears to be what you're looking for. Essentially it is a form of linear regression that learns synthetic attributes for movies from people that rated films and uses that to predict recommendations for ...

6

The input for missing movies are all zero. In a vanilla RBM, once you go to the hidden layer and then come back to the visible layer, you'll get reconstructions for all movies, not just the ones that the current user have interacted with. In the training process it's really important to ignore those reconstructions so that they don't affect your weight ...

6

@Antimony gave a perfect answer. Just wanted to add some theory that helped me to understand the difference between Item-Item Collaborative Filtering and Market Basket Analysis; as well as the applications for these two methods. The family of algorithms used for performing market basket analysis is called association rules. Market basket analysis (or ...

5

There is the very-well known approach based on restricted BOltzmann machines (RBM), which won the Netflix competition. For more details you may have a look at the Wikipedia site, and the references therein. Restricted Boltzmann machines are a particular instance of Markov Random Fields, with some properties that makes particularly attractive. Here a couple ...

5

Matrix factorisation is part of Numerical Linear Algebra (NLA). The following are some useful books in NLA and Data Mining / Statistical Learning. The classic in NLA is Golub & Van Loan's Matrix Computations. Van Loan's webpage lists his books in and links to others. A modern approach that's great for self-study, is Numerical Linear Algebra by ...

5

Basically in a factorization model you usually incorporate both bias and interaction terms. Bias terms describe the effect of one dimension on the output. For example, in the Netflix challenge example, the bias of a movie would describe how well this movie is rated compared to the average, across all movies. This depends only on the movie (as a first ...

5

These low-rank approximations are quite hard to interpret. Moreover, once you've got your $X$ and $Y$ such that $X Y^T \approx Q$ you can apply an unitary transformation $U$ to them to obtain $X_* = X U$ and $Y_* = Y U$ which leads to $X_* Y_*^T = X U U^T Y^T = X Y^T$. This means that there are many possible solutions (each $X U$ for different $U$ is as ...

5

[...] each training sample will be provided to its own neural network [...] Actually there is only one autoencoder shared by all samples and updated by all samples. This is convention in neural network. However, the main contribution in this paper is, for each training sample passing through the network, they only update the parameters associated with the ...

Only top voted, non community-wiki answers of a minimum length are eligible