0
$\begingroup$

I'm currently exploring latent factor models in recommendation systems, specifically focusing on the interaction between vector magnitudes and the angles between these vectors. While it's clear that small angles (indicating high similarity) are crucial, I'm curious about scenarios where the vectors have significant magnitudes despite larger angles.

In many examples and explanations, the focus is often on small angles between user and item vectors. However, I'm interested in understanding the practical implications of large vector magnitudes in cases where they might lead to a high dot product despite a larger angle. How is this typically handled in recommendation systems?

My questions:

  1. Why do most examples not emphasize scenarios with larger angles but significant vector magnitudes?
  2. In practical terms, how significant is the impact of vector magnitude in such scenarios within Matrix Factorization models?
$\endgroup$

2 Answers 2

2
$\begingroup$

It of course depends on the model. The typical factorization machine performs a dot product between user and item and then a sigmoid to indicate action (for example 1- click, 0- no click). In this case you can clearly see the importance of the vector magnitude. Since it directly increases (if vectors in similar direction) or decreases (if vectors in different direction) the class probability. For example increasing the magnitude by a factor of 10 will make the model more certain for this product for all users while decreasing the magnitude will make the model less certain.

In many cases this indicates item popularity. A viral video everyone watches will have a higher magnitude than something people very niche. Because the dot product needs to be high for every vector, and the easiest way to make the product large is just increase the magnitude of the vectors.

And now to practical considerations - it's not the only way to make the product large. You typically also add a bias per user and item (or just an extra column to the vectors, it's the same thing). So it's realy just redundent. And in many architectures the vector norm is scaled to 1 anyway so all the items are on a hypersphere.

So finally to answer your questions:

  1. I guess there's no emphasis on it because it has little practical applications. You train a high dimensional model anyway with redundant degrees of freedom and it just works.
  2. It's significant. But you train a model to minimize errors. So if a certain vector has larger magnitude than another (if you're not using normalized vectors anyway) it's because the model is more certain about it, and you generally just accept it as is.
$\endgroup$
0
$\begingroup$

There actually has been some research done on this problem. Here's one example, which focuses on sparse vectors and on the computational difficulties associated with constraining vector magnitudes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.