Suppose we have following objective function that we want to minimize: $$ \mathcal{L} = \frac{1}{2} \sum_{u}\sum_{j \in I(u)}((b_u + b_j + \mu + q_j^T(p_u + \lvert I(u)\rvert ^ {-\frac{1}{2}}\sum_{i \in I(u)}y_i)) - r_{u,j})^2 + \ldots $$

$u$ is set of users, $I(u)$ is the index of items rated by user $u$, $p_u$ is a vector of size $n$, $q_j$ is a vector of size $m$, If I want to use batch gradient descent for learning model parameters how can I compute $\frac{\partial{E}}{\partial{y_i}}$?

I have looked at the paper and author have computed the gradient as follows: $$ \forall i \in I(u), \frac{\partial \mathcal{L}}{\partial y_i} = \sum_{j \in I(u)} e_{u,j} \lvert I(u) \rvert^{-\frac{1}{2}} q_j $$

and also they have computed gradient of $p_u$ as follows: $$ \frac{\partial{\mathcal{L}}}{\partial{p_u}} = \sum_{j \in I(u)} e_{u,j}q_j $$

What really confuses me is the $\forall i \in I(u)$ term for $y_i$ values, why we have that? why we didn't treat $y_i$ just like $p_u$

Edit: For refrence you can take a look at this link this paper.

  • $\begingroup$ any comment on question? Do I need to clarify anything? $\endgroup$ – Rathma Jun 23 '17 at 19:51

I'm not sure where the confusion is. Let $\mathcal{L}_u$ be the expression of everything under the $\sum_u$, i.e., $\mathcal{L} = \frac{1}{2}\sum_u \mathcal{L}_u$. I'm assuming you're doing the batch gradient descent over users. So for a single user, under $\mathcal{L}_u$, the $y_i$ term appears within the context of $\sum_{j \in I(u)} \sum_{i \in I(u)}$ whereas $p_u$ appears only in the context of $\sum_{j \in I(u)}$. In other words, for $i \not\in I(u)$, $\mathcal{L}_u$ is not dependent on $y_i$.

  • $\begingroup$ thanks for your answer, what happens if we had some other variable like $\sum_{k \in T(u)} p_k$ was present instead of $y_i$? $\endgroup$ – Rathma Jun 25 '17 at 12:28
  • $\begingroup$ Assuming these are same $p$ variables as before, then the partial derivative expression would be a little more complex. Look at partial derivative relative to $p_k$, then have $S_1(\dots) + S_2(\dots)$ where $S_1$ is 0 for $k \neq u$ and is $\sum_{j \in I(u)} e_{u,j}q_j$ otherwise, and $S_2$ is 0 for $k \not\in T(u)$ and is $\sum_{j \in I(u)} e_{u,j}|I(u)|^{-\frac{1}{2}}q_j$ otherwise. Hope that's clear, hard to write it inline in comment with no preview $\endgroup$ – MotiN Jun 25 '17 at 12:38
  • $\begingroup$ Well I guess that require another question ... $\endgroup$ – Rathma Jun 25 '17 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.