This is my first question here, so forgive me if I'm providing too little or too much detail.
I am training a CNN for image classification of fog conditions on camera images. I have three classes: No fog, light fog and dense fog.
To train this model I use PyTorch, where I finetune a pre-trained Resnet-18 model that was trained on ImageNet data on my data. Specifically, I've removed the last Fully Connected layer and replaced it to train for a 3-class classification problem. Learning rate = 0.001.
The data was randomly split into 80% train data and 20% test data. Furthermore, there is a large class imbalance in my data, meaning that there are 152.333 images with no fog, 1187 images with light fog and 604 images with dense fog. To try and counteract this problem I have used PyTorch's WeightedRandomSampler, which makes sure that there is class balance in every training batch. I've checked this, and an equal amount of the three classes are shown in every batch.
Now comes my actual 'problem': The validation loss is lower than the training loss at the very first epoch and continues to stay lower. Also, this validation loss (and accuracy) seems to randomly fluctuate but always stays high. Here are the loss curves of the model training:
I cannot figure out what might be causing this strange behaviour. I thought that maybe the model was predicting most of the validation images to be the majority class, but this confusion matrix shows that it is actually not doing this:
0 = no fog 1 = light fog 2 = dense fog
The images were taken by 8 cameras that are static and thus shoot images of the same scenery all the time. Randomcrop and RandomHorizontalFlip are used for the training set.
Does anybody have any clue as to what might be causing this strange behaviour of the validation loss, or is this maybe something that is actually O.K. when training a CNN for image classification?