# Validation accuracy higher then training accuracy and also immediately high at first epoch

This is my first question here, so forgive me if I'm providing too little or too much detail.

I am training a CNN for image classification of fog conditions on camera images. I have three classes: No fog, light fog and dense fog.

To train this model I use PyTorch, where I finetune a pre-trained Resnet-18 model that was trained on ImageNet data on my data. Specifically, I've removed the last Fully Connected layer and replaced it to train for a 3-class classification problem. Learning rate = 0.001.

The data was randomly split into 80% train data and 20% test data. Furthermore, there is a large class imbalance in my data, meaning that there are 152.333 images with no fog, 1187 images with light fog and 604 images with dense fog. To try and counteract this problem I have used PyTorch's WeightedRandomSampler, which makes sure that there is class balance in every training batch. I've checked this, and an equal amount of the three classes are shown in every batch.

Now comes my actual 'problem': The validation loss is lower than the training loss at the very first epoch and continues to stay lower. Also, this validation loss (and accuracy) seems to randomly fluctuate but always stays high. Here are the loss curves of the model training:

I cannot figure out what might be causing this strange behaviour. I thought that maybe the model was predicting most of the validation images to be the majority class, but this confusion matrix shows that it is actually not doing this:

0 = no fog 1 = light fog 2 = dense fog

The images were taken by 8 cameras that are static and thus shoot images of the same scenery all the time. Randomcrop and RandomHorizontalFlip are used for the training set.

Does anybody have any clue as to what might be causing this strange behaviour of the validation loss, or is this maybe something that is actually O.K. when training a CNN for image classification?

You need to use the same loss for training and validation if you want to compare them. Your training loss is taking into account sample weights (generated by the WeightedRandomSampler) in order to compensate the class inbalance: misclassification of minority class samples is penalized much more than misclassification of majority class samples.
The solution is to use weighted loss function also for validation and assign a weight $1/N_k$ to each sample, where $N_k$ is the number of samples of class $k$ in the validation set.
• I don't know the details of the WeightedRandomSamples implementation, it is well possible that instead of generating weights it simply alters the training distribution; the effect is equal (all classes participate on the batch loss by same amount). You can use the same approach for the validation: instead of using weights, just alter the validation set distribution by replicating samples in minor classes, or, even better - less redundant computation, weighting losses generated by each sample by 1/N_k – Jan Kukacka Apr 21 '18 at 10:34