# Binary classification with CNN for soccer ball detection doesn't converge

I'm working on a project where I want to detect classic soccer balls in live camera pictures using a Convolutional Neural Network. My Network is built up as follows:

conv1: 11x11, 96, ReLU, max pooling, normalization
conv2: 5x5, 256, ReLU, max pooling, normalization
conv3: 3x3, 384, ReLU
conv4: 3x3, 384, ReLU
conv5: 3x3, 256, ReLU, max pooling, normalization
fc1: 3072, TanH, dropout 0.5
fc2: 3072, TanH, dropout 0.5
fc3: 2, softmax
momentum, categorical crossentropy, learning rate 0.001


I want to predict whether there is a soccer ball in a room or not, so I am using images like the following of about 200 in each class for training:

Unfortunately, I cant get the model working. I tried learning rates like 0.001, 0.01-0.05 and 0.1, everything <= 0.03 results in too fast loss decrease (0.00001) and accuracy 100% but with wrong predictions afterwards. Training with >0.03 always stays around loss 0.7 and acc 0.5.

I also tried different batch sizes and converted the former color pictures to gray scale but nothing works. Shall I use other activation functions, a sigmoid last layer or are my convolutions wrong? Or is my hole 5-3-structure not practicable?

• It's strange that overlearning (obtaining a high accuracy on training set but a low one on validation) is happening for lower values of the learning rate. Also, 400 images might be too low for the number of parameters of your network. Have you tried oversampling? – jeff Jun 7 '16 at 1:57