I'm working on a project where I want to detect classic soccer balls in live camera pictures using a Convolutional Neural Network. My Network is built up as follows:
conv1: 11x11, 96, ReLU, max pooling, normalization conv2: 5x5, 256, ReLU, max pooling, normalization conv3: 3x3, 384, ReLU conv4: 3x3, 384, ReLU conv5: 3x3, 256, ReLU, max pooling, normalization fc1: 3072, TanH, dropout 0.5 fc2: 3072, TanH, dropout 0.5 fc3: 2, softmax momentum, categorical crossentropy, learning rate 0.001
I want to predict whether there is a soccer ball in a room or not, so I am using images like the following of about 200 in each class for training:
Unfortunately, I cant get the model working. I tried learning rates like 0.001, 0.01-0.05 and 0.1, everything <= 0.03 results in too fast loss decrease (0.00001) and accuracy 100% but with wrong predictions afterwards. Training with >0.03 always stays around loss 0.7 and acc 0.5.
I also tried different batch sizes and converted the former color pictures to gray scale but nothing works. Shall I use other activation functions, a sigmoid last layer or are my convolutions wrong? Or is my hole 5-3-structure not practicable?