I'm working on a classifier that uses a convolutional neural network. As part of this, the AdamOptimizer is used during gradient descent.
When I examine the results of training and testing, I'm finding significant overfitting given that the data available tends to be concentrated in several classes.
How can I reduce overfitting here? Ideas I've had, that I'm not sure are sound, or how they would work:
- Change training data to make it more evenly distributed across classes
- Modify the loss function in some way
- Change the hyperparameters (learning rate?) for the Adam Optimizer