In the deep learning book (free version is available online) the backpropation algorithm is explained in section 6.5.

I have a question on equation (6.53):

$$\frac{\partial u^{(n)}}{\partial u^{(j)}} = \sum_{i:j\in Pa(u^{(i)})}\frac{\partial u^{(n)}}{\partial u^{(i)}}\frac{\partial u^{(i)}}{\partial u^{(j)}}.$$

Can someone explain the index below the summation?

In the notation section I find: $Pa_{G}(x)$ are the parents of $x$ in graph $G$. Problem: parents is nowhere defined. And searching the internet only gives a definition for rooted trees, that doesn't seem to match with how it is used here (it actually seems to be the exact opposite of that definition).

Furthermore I would expect that $Pa(u^{(i)})$ is a list, but here it seemed to be indexed with $i:j$ does that mean it is two dimensional.

Or should it be interpreted as: $\forall i \;\forall j \in Pa(u^{(i)})$?

I think I get it; should it be read as all $i$ such that $j$ is a parent of $i$?

  • 1
    $\begingroup$ Yes: all $i$ s.t. $j$ is a parent of $i$. $\endgroup$ – gunes Mar 11 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.