2
$\begingroup$

I was taking the "Machine Learning-Coursera (Standford) by Andrew Ng" course.

In Week 4 and Week 5 we have given programming assignments, Where we have to do digit classification.

In Week 4 programming assignment we have used Feed forward Neural Network for classifying digits and we get an accuracy of around 97.5%. Then in Week 5 programming assignment we have used Neural Network with Backpropagation which gives us the accuracy of around 95%.

For this particular application (Character Classification) will Feedforward Neural Network always dominate Neural Network with Backpropagation?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

When you used a Feedforward Neural Network during week 4, that network has been already trained using backpropagation by Andrew, and provided to you to use for the classification task.

In week 5, you went further and trained a network yourself using backpropagation. As you can see, there's no such thing as a feedforward only or a backprop only neural network. All neural networks are trained using backpropagation.

For various reasons, you got a different accuracy score than Andrew's network. It's hard to say why, but I bet Andrew used his experience and knowledge to push the training further.

Again, there was no difference in the nature of the networks.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks @ociule I get it. Can You tell me some tricks that I can apply in order to get better accuracy better than 95%. $\endgroup$
    – Atinesh
    Jun 17, 2016 at 4:47
  • $\begingroup$ You can try different alpha values for the regularization, and also let fmincg run for more iterations than the default 50 by changing the MaxIter option. $\endgroup$
    – ociule
    Jun 17, 2016 at 9:51
  • $\begingroup$ @Atinesh Please note that by increasing iterations, you can get an arbitrarily high testing set accuracy (I got above 99% with 250 iterations). But this is just how well the network has fit (and possibly overfit) the training set. So it's not good news. You must test the trained network on a test or validation data set, which was not done in exercise 4. You would have to split it yourself from the given dataset. You can have a look at how I did it at github.com/ociule/machinelearning/blob/master/ex4/ex4/ex4.m $\endgroup$
    – ociule
    Jun 17, 2016 at 10:17
  • $\begingroup$ "All neural networks are trained using backpropagation." That's not entirely true. Consider the Feed Forward Neural Networks with Random Weights, such as described by Schmidt, Kraaijveld, and Duin (1992). $\endgroup$
    – Firebug
    May 22, 2017 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.