So I created an ANN with 4 nodes hidden layers and 3 nodes in output layer (as there are 3 classes) to classify the following data-set of flowers.

UCI Iris Data Set

Now I tokk 60% of the data and used it to train the ANN. The learning rate for the connection between both the layers are 0.1. I have used the sigmoid function as the activation function with the conventional cost function and gradient descent (no regularization) with back propagation for learning. More info about it here.

So anyways I graphed its cost J and its F_score. Since there are 3 classes I have calculated F_score the following way (F_score1 + F_score2 + F_score3)/3 where F_scorei denotes the F_score of ith class. So I plotted the J and F_score only of the training data only not cross validation or test set and got the following 2 graphs for J and F_score as a function of number of epochs.

Cost as function of epochs F_score as function of epochs

NOTE: X- axis is number of epochs while y-axis is the value of J or F_score.

Now what is baffles me isthat

  • Why is there a sudden spike at about the 1200 - 1400 the epoch in the cost J?
  • Why is there a mismatch between J and F_score? Cost is continuously decreasing but F_score is not reflecting it?

I printed out the true positives for the 3 classes and found out that the point where J is spiking the true positive of one of the classes is suddenly becoming 0. Why is this happening? Do you think i may have a mistake in my code? But even if I have a mistake why is this ANN performing satisfactorily for first 1000 epochs?

Any help is highly appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.