So I created an ANN with 4 nodes hidden layers and 3 nodes in output layer (as there are 3 classes) to classify the following data-set of flowers.
Now I tokk 60% of the data and used it to train the ANN. The learning rate for the connection between both the layers are
0.1. I have used the
sigmoid function as the activation function with the conventional cost function and gradient descent (no regularization) with back propagation for learning. More info about it here.
So anyways I graphed its cost
J and its
F_score. Since there are 3 classes I have calculated
F_score the following way
(F_score1 + F_score2 + F_score3)/3 where
F_scorei denotes the
ith class. So I plotted the
F_score only of the training data only not cross validation or test set and got the following 2 graphs for
F_score as a function of number of epochs.
NOTE: X- axis is number of epochs while y-axis is the value of
Now what is baffles me isthat
- Why is there a sudden spike at about the
1200 - 1400the epoch in the cost
- Why is there a mismatch between
F_score? Cost is continuously decreasing but
F_scoreis not reflecting it?
I printed out the true positives for the 3 classes and found out that the point where
J is spiking the true positive of one of the classes is suddenly becoming
0. Why is this happening? Do you think i may have a mistake in my code? But even if I have a mistake why is this ANN performing satisfactorily for first 1000 epochs?
Any help is highly appreciated.