Say you're training an NN and have different groups of samples, say number of groups is ngroups.
Each group has a different number of samples, say nsamples1, nsamples2, etc..
The number of samples per batch is batch_size and you want each samples from each group to show up at least once (on average) during each batch, with replacement.
This means that groups with a smaller number of samples will get oversampled and seen multiple times per epoch, but at least that group will have equal representation during training.
How would you choose the sampling weights (probability of pulling a sample for a given batch size)?
My specific scenario is this:
I have 3 groups:
- Number of samples = 50
- Number of samples = 2000
- Number of samples = 3000
with batch_size = 4.
How do I select the sampling weights so that a sample from each group shows up at least once (on average) for each batch?
