1
$\begingroup$

I am trying to use Spark ALS to do recommendation with implicit feedback. However, I found there are two totally different kinds of settings available:

The first one is the setting used by the original paper: Collaborative Filtering for Implicit Feedback Datasets. They use $\lambda={\sim}150$ and $\alpha={\sim}40$

The other is the setting suggested by Sean Owen in this discussion, which uses $\lambda=1$ and $\alpha=40$.

Through I don't very clearly understand the underlying mechanism of implicit feedback recommendation. I have to admit that the recommendations made from Owen's setting is much better than those from the original setting.

Can someone explain what makes this difference?

$\endgroup$
3
$\begingroup$

The term $\lambda$ is used for regularization here and $\alpha$ is a constant controlling the rate of increase of the confidence matrix $C_{ui}$. The confidence matrix is your confidence in observing a particular preference a user, $u$, has on an item, $i$.

Remember that regularization is to help reduce overfitting by penalizing $X_u$ and $Y_i$ when minimizing the overall loss function.

$\endgroup$
  • $\begingroup$ Is there a guideline for dummy that if the value of lambda and alpha is smaller or larger, the more the accuracy increase? $\endgroup$ – tom10271 Aug 2 '18 at 3:02
  • $\begingroup$ @tom10271 There is no clear cut values to start with as you always have to tune these problems to your particular dataset. Having said that, starting with values similar to the paper is a good place to start. $\endgroup$ – guy Aug 2 '18 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.