3
$\begingroup$

I am trying to use Spark ALS to do recommendation with implicit feedback. However, I found there are two totally different kinds of settings available:

The first one is the setting used by the original paper: Collaborative Filtering for Implicit Feedback Datasets. They use $\lambda={\sim}150$ and $\alpha={\sim}40$

The other is the setting suggested by Sean Owen in this discussion, which uses $\lambda=1$ and $\alpha=40$.

Through I don't very clearly understand the underlying mechanism of implicit feedback recommendation. I have to admit that the recommendations made from Owen's setting is much better than those from the original setting.

Can someone explain what makes this difference?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

The term $\lambda$ is used for regularization here and $\alpha$ is a constant controlling the rate of increase of the confidence matrix $C_{ui}$. The confidence matrix is your confidence in observing a particular preference a user, $u$, has on an item, $i$.

Remember that regularization is to help reduce overfitting by penalizing $X_u$ and $Y_i$ when minimizing the overall loss function.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is there a guideline for dummy that if the value of lambda and alpha is smaller or larger, the more the accuracy increase? $\endgroup$
    – tom10271
    Commented Aug 2, 2018 at 3:02
  • $\begingroup$ @tom10271 There is no clear cut values to start with as you always have to tune these problems to your particular dataset. Having said that, starting with values similar to the paper is a good place to start. $\endgroup$
    – guy
    Commented Aug 2, 2018 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.