What is the relation between SVD and ALS?

I am trying to build a simple CF-recommender system using the small MovieLens data set. In order to do this, I tried to use ALS to factor my (user, item) matrix $A$ into a (user, latent-space) matrix $U$ and a (item, latent-space) matrix $V$ such that: $$A = UV$$ To do this I am using the ALS algorithm as defined in the Implicit package for python, which when I multiply $UV$, is wildly different from my initial $A$. My question is, am I confused about what ALS actually is? I thought it was something akin to SVD, or any other matrix factorization algorithm. Would It matter that i'm using explicit data instead of implicit data? Here is the code i'm using to perform the matrix decomposition:

from implicit.als import AlternatingLeastSquares
from scipy import sparse

def matrix_decomposition(matrix, k, i):
matrix = sparse.csr_matrix(matrix.T)
model = AlternatingLeastSquares(factors=k, iterations=i)
model.fit(matrix)
user_latent = model.user_factors
item_latent = model.item_factors

return user_latent, item_latent


... I confused about what ALS actually is? I thought it was something akin to SVD, or any other matrix factorization algorithm. Would It matter that i'm using explicit data instead of implicit data?

"Implicit vs explicit data

Explicit data is data where we have some sort of rating. Like the 1 to 5 ratings from the MovieLens or Netflix dataset. Here we know how much a user likes or dislikes an item which is great, but this data is hard to come by. Your users might not spend the time to rate items or your app might not work well with a rating approach in the first place.

Implicit data (the type of data we’re using here) is data we gather from the users behaviour, with no ratings or specific actions needed. It could be what items a user purchased, how many times they played a song or watched a movie, how long they’ve spent reading a specific article etc. The upside is that we have a lot more of this data, the downside is that it’s more noisy and not always apparent what it means.

For example, with star ratings we know that a 1 means the user did not like that item and a 5 that they really loved it. With song plays it might be that the user played a song and hated it, or loved it, or somewhere in-between. If they did not play a song it might be since they don’t like it or that they would love it if they just knew about.

So instead we focus on what we know the user has consumed and the confidence we have in whether or not they like any given item. We can for example measure how often they play a song and assume a higher confidence if they’ve listened to it 500 times vs. one time.

Implicit recommendations are becoming an increasingly important part of many recommendation systems as the amount of implicit data grows. For example the original Netflix challenge focused only on explicit data but they’re now relying more and more on implicit signals. The same thing goes for Hulu, Spotify, Etsy and many others.".

There are different ways to factor a matrix, like Singular Value Decomposition (SVD) or Probabilistic Latent Semantic Analysis (PLSA) if we’re dealing with explicit data.

A least squares approach in it’s basic forms means fitting some line to the data, measuring the sum of squared distances from all points to the line and trying to get an optimal fit for missing points.

With the alternating least squares approach we use the same idea but iteratively alternate between optimizing U and fixing V and vice versa. It is an iterative optimization process where we for every iteration try to arrive closer and closer to a factorized representation of our original data.

Singular-Value Decomposition is a matrix decomposition method for reducing a matrix to its constituent parts in order to make certain subsequent matrix calculations simpler.

The tutorials in the first two and last link should be very helpful.

• would If I could, but don't quite have the reputation yet. Will do when I get enough Commented Jul 9, 2018 at 17:18

The relationship between ALS and SVD in latent factor recommender systems is the same as the relationship between OLS and Normal Equations in Linear Regression

Under the hood, Alternating Least Squres (henceforth ALS) is a 'fancy' two step gradient descent technique to find matrices $$P$$, the user factors matrix and $$Q$$, the item factor matrix such that $$U \approx PQ^T$$. The gradient descent works to minimise the square of the L2 norm of the matrix $$|U - PQ^T|$$

This is very similar to Ordinary Least Squares (henceforth OLS), which also involves gradient descent to find the coefficient matrix $$w$$ of a linear regression model $$y = Xw + \epsilon$$. The gradient descent works to minimize the square of the L2 norm of the vector $$|y - Xw|$$

This establishes the fact that ALS in recommender systems is an analogue of OLS in linear regression.

Notation alert: $$||Z||_2 ^2$$ is just the formal way to write the square of the L2 norm of $$|Z|$$

Now, let us ask ourselves this

In a linear regression setting, can we estimate the coefficient matrix $$w$$ in any other way than using gradient descent? Perhaps by using a direct formula?

The reason as to why we would want to do this is simple. Gradient descent has lots of problems, starting from the fact that it is very slow, can get stuck in local optima, does not guarantee convergence if loss function landscape is non-convex, among other things.

Would it not be better if we had a formula for linear regression that would directly give us the coefficient matrix $$w$$ which minimises the loss function $$||y - Xw||_2 ^2$$? This would aloow us to just plug in the values of $$X$$ and $$y$$ and get $$w$$ directly. No iterative and computationally intensive steps need to be taken.

Well, does such a formula exist to find the coefficient matrix of linear regression as a function of $$X$$ and $$y$$?

Absolutely, and it comes from normal equations, and the formula looks something like this

$$w = (X^T X)^{-1} X^Ty$$

And it is orders of magnitude faster than estimating $$w$$ using gradient descent. And it always gives the exact answer answer, unlike approximations from gradient descent.

For recommender systems, there also exists such a formula that finds the matrices $$P$$ and $$Q$$ such that $$||U - PQ^T||_2 ^2$$ is minimised. This formula is given by Singular Value Decomposition (henceforth SVD), whose computation is straightforward and way less computionally and time intensive than the gradient descent based ALS technique.

tl;dr SVD is the analytical analogue to ALS in recommender systems, as Normal Equations are the analytical analogue to OLS in linear regression