# In convolutional neural networks, how to prevent the overfitting?

Given certain amount of labeled data, we define the net structure, such as number of layers, types of layers, the number of convolutional layers, the number of pooling layers, etc.

And train the parameters using back propagation, meanwhile we show the loss in training procedure and view the testing accuracy in validating data set.

But, the loss in training set is nearly zero, and the testing accuracy is kept unchanged no matter how to decrease the learning rate.

• In this circumstance, is it overfitting?
• Should we change the net structure?
• More layers for more parameters?
• Could you please recommend some suggestions or references?

I would say there might be a bug when calculating the errors when using backpropagation.

Also how big is your data? And how many epochs are you using for your simulation? How is your training being done? I could help you more if i could get more data.

• Thank you for replying, sir! I've finally been told that indeed the amount of the training data is very important to the CNN. Thus, a common approach is to preprocess the limited data samples to simulate more data samples. But now I have no much time to do that work. Moreover, the batch_size of SGD also has effects in the training procedure, larger the batch_size, smoother the training loss. Additionally, I mainly use the open source caffe to train the CNN. But, I'm not sure that what's the bug of backpropagation as you have presented. Could you please give more details? Thanks! – mining Aug 3 '14 at 13:41