0
$\begingroup$

I am currently reading Bishop - "Pattern Recognition and Machine Learning" (2006) and I could not figure out if I missed an assumption he made or if it is just wrong.

In the chapter about backpropagation on page 243 he computes the derivative of the error function with respect to a particular weight

$\displaystyle \frac{\partial E_n}{\partial w_{ji}}=\frac{\partial E_n}{\partial a_j}\frac{\partial a_j}{\partial w_{ji}}$ (Equation 5.50)

This is only correct if the output units are assumed to be the identity.

Why does he not get the derivative of the activation function in the expression? It should look like this:

$\displaystyle \frac{\partial E_n}{\partial w_{ji}}=\frac{\partial E_n}{\partial h}\frac{\partial h}{\partial a_{j}}\frac{\partial a_j}{\partial w_{ji}}$

where $h$ is the activation function.

$\endgroup$

1 Answer 1

1
$\begingroup$

Equation (5.50) is correct regardless of whether or not the output unit is the identity. Bishop's equation with a product of two factors is equivalent to your equation with three factors, since $$ \frac{\partial E_n}{\partial a_j} = \frac{\partial E_n}{\partial h}\frac{d h}{d a_{j}} $$ by the chain rule. So Bishop's equation is an abbreviation, but not an error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.