This question already has an answer here:
I’ve recently started learning about neural networks and currently am working on a NN to classify images of a cat vs non cat. I’ve built an option of customizing the number hidden layers and nodes per layer for testing purposes.
Training set size: 209, test set size: 50, learning rate was varied but did not affect the problem experienced.
Tests with test set...
I tried to train a 2 layer model with 5 nodes and the cost managed to converge with a 74% accuracy.
Next I tried to train 3 and 4 layer networks, but both are converging to what I believe are local minimums achieving accuracy of 34%. When I’m lucky the 3 layer network coverages and I get an accuracy of about 76-78%. Why am I constantly getting stuck at local minimum so far away from the global minimum? Are there methods to debug what’s going one?
Opinions and ideas are very much appreciated!