Assumption behind few latent features in recommender systems?

Question

I know in recommender systems you have a rating matrix and then you factorize this matrix into two matrices and then learn those matrices with gradient descent. In those matrices we specify the number of dimensions/latent features that we want. So one of them will be of size $number\_of\_users * k\_latent\_features$. My question is why we choose the parameter $K$ (number of latent features) to be lower than the number of users or items? I'm not able to understand the assumption that the number of latent features should be low.

Answer 1

Following Occam's razor, we try to explain what we see with a simple model (e.g. few latent features).

If $K$ is equal to the number of users/items, we can construct an infinite number of models that fit the data perfectly without learning anything (e.g. these models do not generalize).

By choosing a smaller $K$, we must try to learn recurrent features. Such models will not fit the data perfectly, but they do generalize to unseen data.

Answer 2

Three reasons -

1. By projecting to lower dimensional space, we are saying there are some common categories (latent variables) that describe our behavior. Smaller means higher compression, i.e understanding. So you are using fewer categories to explain the behavior.
2. Practically, for each step you are multiplying two matrices. Large matrices would mean longer processing time and more resource constraints. So there is a preference for smaller matrices.
3. $K$ is really the VC-dimension of this. So higher $K$ means you need more data points to be certain about them.

Ideally, you should start with a small enough $K$ and work your way in 10x increments using cross validation. You will find that there are diminishing returns after a while.