I expected my training process is like this:

enter image description here

But my training accuracy increased to 99% at only 1 epoch, not steadily increase.

I was suspicious about high learning rate, so i used various learning rate(0.001~0.00001). But the training accuracy increased to ~99% at only one epoch and kept 99% ~ 100%.

Otherwise, validation accuracy increased to ~70% at one epoch and increased up to 80% during 2~3 epoch, and then oscillated during training.

Another problem is that, training loss is almost 0, thereby only weight decay term remain in loss function. it caused weights to become all 0.

I don't know what is problem. Is input image size too small?

below is my detail setting:

I'm using convolutional 1~5th layer weights from Alexnet, for fine-tune my CNN network.

Difference between Alexnet is that, my input image size is 87x33 unlike 224x224 in Alexnet.

(Accordingly, my last(5) conv feature shape is 4x2x256, fully connected layer is same as Alexnet).

Number of classes are two(positive, negative) in my network.

Number of training image data for positive is ~40k, for negative is ~40k.

(apply random crop,warping,flipping for each sampling)

I used Adam-optimizer in tensorflow.


I checked training samples are different per feeding, and i'm pretty sure that it's not a problem.

By the way, i realized that i use subset of ImageNet data for fine-tuning(as i know, AlexNet is trained by ImageNet, as well).

Although, i only used pre-trained weights for conv-layer(1~5) and i only used ImageNet data for negative training image, not for positives, i think it's the cause for my problem.

is it make sense?

  • $\begingroup$ Unless your data is trivial to classify, no it's not normal, certainly not for large conv network. $\endgroup$ Apr 7, 2017 at 10:01
  • $\begingroup$ Can you draw a graph for you network like your question? $\endgroup$
    – ABCD
    Apr 7, 2017 at 10:08
  • $\begingroup$ What are you training your data on? For instance, if you train your data to predict a binary classification i.e.(1/0) and the average of 1's = 60% the model will first use this 60% as baseline, since it is simply able to predict with a 60% accuracy by saying everything should become a 1. $\endgroup$
    – David
    Nov 27, 2017 at 11:50

1 Answer 1


Check than the batch is giving different samples, looks like you are feeding always the same samples to the network... Another thing you have to understand, is that you are using a pretrained model, which means that lots of patterns are already learnt, if your data fit in those patterns, is it possible than your problem was already solved in the model.

  • $\begingroup$ I missed the part OP mentioned about the pre-trained weights when I was reading the question. Thanks for pointing out. $\endgroup$
    – ABCD
    Apr 7, 2017 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.