I designed a neural network to classify some images into 28 classes. Here are the parameters :
- Weight Decay : 0.005
- Momentum : 0.01
- Learning Rate : 0.001 and 0.005
- Learning Decay : 1
- Input : 100x100 images
- Output : 28 Classes
- Training Data : 1500 Images
- Test Data : 30 Images
- Batch Learning
- Using Python with PyBrain
The images are Memes and each class has nearly 50 images in it. images in each class are highly simialr (because they're memes). no matter how many different networks I ran, it ended up like this:
The network learns to 100% and for the test data (nearly 30 images) it can classify up to 85%... BUT after a few more epochs, it loses accuracy and becomes completely useless... I tested many Hyper Parameters (with 50 epochs) to choose these parameters in the end but I guess I didn't test them for more than 50 epochs...
Here are my questions:
If I stopped training the network when the network reached Maximum Training and Testing accuracy, would the network be useful or the real network shows itself after more epochs thus becoming useless ?
Why does something like this happen? why does it lose accuracy after 25 epochs? What am I doing wrong?