We have collaborative filtering and content based algorithms there for recommendation.

What stops traditional algorithms from directly being used to find missing values in the Utility matrix formulation of recommendation? Is it the sparsity of the matrix? If the matrix wasn’t sparse, what other algorithms also work? Do regressions work then?

Say given ratings for $n$ items for $m$ users, predicting the $n^{th}$ rating for a user with ratings of only $n-1$ items. Would collaborative filtering and content based algorithms still do better given less sparsity?

Is there some way in which SVMs, Neural Nets be used for recommendation directly? A naive Bayes approach to say whether a product will be bought or not? In my opinion wouldn't work because products are similar and can't be treated as independent variables.

Also, if these algorithms (collaborative filtering and content-based approaches) work nicely on sparse matrices with a lot of features, can it be used for prediction problems for sparse matrices and solve the high dimension less data problem?

I know these questions are a little spread and may be really naive, but I think an underlying difference between the two approaches of prediction would solve most of my questions. Is there a survey paper which compares machine learning algorithms with content-based and collaborative filtering-based approaches to prediction?

  • $\begingroup$ A key problem is efficiency. In principle, it is possible to formulate recommendation as a multi-class ranking problem to tackle with classification algorithms, but then the number of classes becomes absurd (= each costumer is a class). $\endgroup$ Commented May 27, 2015 at 12:14
  • 1
    $\begingroup$ Isn't k nearest neighbors sometimes used for recommendations? That's about as "traditional" as it gets $\endgroup$ Commented May 27, 2015 at 12:18
  • 1
    $\begingroup$ What is CB? We try to avoid using unexplained acronyms because it creates confusion for users who are not familiar with your particular shorthand. And that makes it more difficult for you to get answers to your question. $\endgroup$
    – Sycorax
    Commented May 27, 2015 at 14:00
  • $\begingroup$ @user777 Content Based Algorithms. I'll change that :) $\endgroup$ Commented May 28, 2015 at 4:20
  • $\begingroup$ @MarcClaesen how can it be translated into a multi-class ranking problem? Can you explain more? $\endgroup$ Commented May 28, 2015 at 4:21

1 Answer 1


Regarding your question on using regression for predicting the nth item. Typically regression is preformed on a bunch of features that influence an observation. For eg., regression is a model that combines features of a house such as number of bedrooms, number of baths, school rating, area of the house to the values. Assume for every user $u$, for all the movies he has rated, if you have a matrix $V$ that has the features of the movies such as length of the movie, MPAA rating, genre etc and the rating of the movie, using this matrix $V$, you can build a regression model for the user $U$ that fits the rating of the movies. Collaborating filtering based recommender systems is exactly trying to build this statistically significant item/movie features automatically. Assume you have a rating matrix $R$, where rows are items and columns are users, the collaborative filtering based recommender system algorithm discovers two matrix $U$ and $V$ such that $R \approx VU$. The matrix $V$ is a feature-item matrix - for all the items, it discovers a hidden/latent feature matrix which you would have otherwise explicitly collected. Using this feature matrix $V$, for every user $u$, it also discovers the weights (similar to regression model). All the users weights $u$, is represented as the matrix $U$.

  • $\begingroup$ @ramakrishnan-kannna If my understanding is right, what you're saying is that I can decompose the ratings matrix R into two matrices, one of them being a feature matrix V which is mxA matrix where there are A features and the values of each movie in the respective feature. And then there's this another matrix U, which is Axn matrix where for each A feature, we have weights of users. This is a typical process for latent variable discovery. Right? $\endgroup$ Commented May 28, 2015 at 5:53
  • $\begingroup$ Also, If this is the case, how is this process really different from an algorithm like expectation-maximization wherein I can determine the maximum likelihood of two unknown parameters (values of the two matrices) through an iterative process? Is it different in terms of theoretical guarantees or processing time or probably some other aspect? I don't quite intuitively get how they're different in terms of what they're trying to achieve. $\endgroup$ Commented May 28, 2015 at 5:54
  • 1
    $\begingroup$ @HarshNisar : Yes. There are bayesian interpretation for the matrix factorization algorithm. The popular paper is BPMF by Salakhutdinov. You also have source code in internet. There are also video lectures that explains this intuition well. $\endgroup$ Commented May 28, 2015 at 12:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.