I’ve recently started learning about neural networks and currently am working on a NN to classify images of a cat vs non cat. I’ve built an option of customizing the number hidden layers and nodes per layer for testing purposes.

Training set size: 209, test set size: 50, learning rate was varied but did not affect the problem experienced.

Tests with test set...

I tried to train a 2 layer model with 5 nodes and the cost managed to converge with a 74% accuracy.

Next I tried to train 3 and 4 layer networks, but both are converging to what I believe are local minimums achieving accuracy of 34%. When I’m lucky the 3 layer network coverages and I get an accuracy of about 76-78%. Why am I constantly getting stuck at local minimum so far away from the global minimum? Are there methods to debug what’s going one?

Opinions and ideas are very much appreciated!


marked as duplicate by Sycorax neural-networks May 25 at 1:21

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Browse other questions tagged or ask your own question.