using (deep) neural networks for a severely imbalanced image dataset when some classes have <10 images Taking a long shot here. So I have a a small dataset of ~500 images with discrete labels from 1 to 9. My task is to detect the per-class and overall accuracy of this classification method using a (deep) neural network.
I get an accuracy of 56% using multinomial logistic regression using train/test split method. I used transfer learning in PyTorch for some of the categories that have >25 images (2, 3, 5, 7, and 9). However, I do not know how to use this tutorial and report test accuracy when I only have like 3 or 11 images in a category.
https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html
Here is the number of images per category:
category 1: 3
category 2: 66
category 3: 175
category 4: 18
category 5: 54
category 6: 12
category 7: 91
category 8: 11
category 9: 38

What (deep) neural network method (hopefully with a link to tutorial, would you suggest that could work with a dataset like mine that has severe data imbalance and also some categories are having <10 images?
I have provided further discussion of the problem I am dealing with as well here: https://discuss.pytorch.org/t/leave-one-out-cross-validation-for-images-with-labels-and-discrete-confidence-levels-for-labels/29227
Here is the confusion matrix I get from multinomial logistic regression:

 A: There are several methods to counter imbalanced datasets. Yet, with so few samples in several of the classes I'm not sure if you will ever get a satisfactory classification. I'm not sure if you can learn class features from only 3 samples as in class 1. In any case, this might lead to extreme overfitting or at least not a good generalization.
Nevertheless, you can try the following methods: 


*

*Upsampling, i.e. simply doubling items from the minority classes

*Data augmentation (as mentioned in the comments), i.e. rotating, scaling, cropping, flipping,... the images (you have to decide which modifications make sense with your application)

*Penalizing / cost sensitive learning / class weights, i.e. penalizing the miss-classification of a minority class item more severely than that of a majority class 
Disclaimer: I'm not that familiar with Pytorch as I myself work with Keras and Tensorflow most of the time. Maybe someone else can provide some Pytorch specific examples or tutorials. In any way, the keywords should at least guide you into the right direction.
