2
$\begingroup$

I have attempted to write a Neural Network code, and it was suggested in my textbook (Bishop - Pattern Recognition & Machine Learning) that a very useful debugging technique is to check your $\frac{d E}{d w}$ terms computed via back-propagation 'agree closely' with those calculated by a central difference approach.

Having coded this up, I find that my derivatives agree to within one part in 10. I have no idea if this satisfies 'close agreement'.

$\endgroup$

1 Answer 1

2
$\begingroup$

You have 10% difference between the two methods? I afraid that you either have a bug either the NN code, or you calculate the central difference poorly (maybe the step is too big, try decrease it exponentially until you receive the best agreement with the backpropagation calculation). Also, for central difference it is important to use double, rather than float, numbers.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.