I read somewhere that unit tests are important before jumping onto training for the whole batch. And for that reason, if one sample overfits on the model, can we decisively say that the training will work?
I wanted to apply this basic knowledge to understand if my train algorithm works or not. I am trying to perform multi-label segmentation. And using 3D patches to train in a one-hot encoded method. Using the following hyperparams:
Network: U-Net
Loss function: mixed Generalized Dice and Focal Dice
Optimizer: Adam
I tried to overfit only one patch and trained it for 200 epochs. (which doesn't take even 20 minutes)
As it can be seen, some labels reach nearly 90% dice and some stay at the bottom: Bone, TP and EB.
TP and EB are smaller anatomy labels but it performed well when I try to train with lower number of anatomy labels.
Is there anything wrong with the training? The mean dice doesn't improve much after ~130 iterations. Does it mean there is not enough data to learn(since only one sample is being shown)? If so why isn't it overfitting?
Could anyone help me understand?
