Light direction classifier I have a dataset with light coming from 8 different directions, evenly distributed around the target object. I am looking into designing a classifier for determining the light direction.
I tried looking at a CNN with a sequence of convolutional and pooling layers, followed by 2 dense layers. I've experimented with multiple number of layers (3-6) and various depths (one example would be something like 8, 16, 32, 64, all layers using 3x3 convolutions and 2x2 max pooling. I've been using Adam optimizer, with learning_rate=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-07 (just examples, I've been trying to tweak these numbers a lot). 
This is the full Keras model:
model = Sequential()
model.add(Conv2D(8, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=input_shape))
model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))

The problem is that it converges quickly to 50% accuracy, and then the testing accuracy plateaus or slightly decreases, remaining around 45-50%, while the training accuracy keeps increasing. This happens only after about 20-50 epochs. I'm using a batch size of 16 in most experiments (cannot go much higher due to resource limitations).
Any advice on improving the performance of this model or for solving this problem?
 A: There can be many possibilities as to why the network's performance is not as it is expected to be. Here are a few things that you could try:


*

*What is the size of the dataset? Sometimes the training data just not being enough could also be the problem. If the dataset's size is small, data augmentation techniques could also be used just so that the network has enough data, to begin with.

*Instead of using dropout twice, try using it only once. This is just to eliminate the possibility of the network being regularized too much.

*Another thing that can be checker is whether the dataset is labeled properly. Despite all things being correct, mislabeled data can result in a drop in test performance.

*Try using SGD optimizer with a slightly higher learning rate (e.g. 0.01) instead of Adam. 

*(This might sound far-fetched but it is worth a try) Instead of explicitly using max-pooling layers, try using convolutional layers with a stride of 2. Yes, this definitely means more number of layers, but letting the downsampling happen purely through convolutional layers helps the network learn a more complete spatial representation of the data. This approach is generally taken for generative models; would be interested to see if it works for classification tasks too!


This paper and this post might also be something to look into.
