I'm following Geoffrey Hinton's Neural Networks Lectures (this one in particular: https://www.youtube.com/watch?v=LOc_y67AzCA), and I'm stumped by the second equation here:
As I understand it, $E = \frac{1}{2} \sum\limits_n (t_n - y_n)^2$ ranges over all neurons in the network, and all neurons above $y_i$ depend on $y_i$. Then shouldn't that mean $\frac{\partial E}{\partial y_i} = \sum\limits_n \frac{\partial E}{\partial z_n} \frac{\partial z_n}{\partial y_i}$ where $n$ ranges over all neurons (above layer $i$)? But the way the second equation in the picture indexes over $j$ makes it look like it's only over the layer $j$ above $y_i$. What about the other neurons in layer $j + 1, j + 2,...$?
UPDATE
Ah I understand now. Thanks to Rabindra for the hint. I think this is how it goes:
At layer $i$, $E$ is considered a function of the $z_j$'s in layer $j$, and the $z_j$'s depend on $y_i$. So by the chain rule we have the second equation: $$\frac{\partial E}{\partial y_i} = \sum\limits_j \frac{\partial E}{\partial z_j} \frac{\partial z_j}{\partial y_i} = \sum\limits_j w_{ij} \frac{\partial E}{\partial z_j}.$$ But we don't know $\frac{\partial E}{\partial z_j},$ so we use the first equation: $$\frac{\partial E}{\partial z_j} = \frac{\partial E}{\partial y_j} \frac{d y_j}{d z_j} = y_j (1 - y_j) \frac{\partial E}{\partial y_j}.$$ Here we use the second equation again to go up another layer, and so on. (In practice you'd start from the top and calculate down.)
Along the way down we use the third equation to calculate $\frac{\partial E}{\partial w_{ij}},$ and once we have all of those we use the Delta Rule to calculate the necessary weight changes $\Delta w_{ij} = - \alpha \frac{\partial E}{\partial w_{ij}}.$
