Over sampling for minority classes

Question

I'm a bit confused with the idea of over/under sampling. People have mentioned that rotations are a good technique. By rotation, do they literally mean rotate the image by a few degrees and then add it as another training point for the minority class?

also how are we supposed to even out the ratios? assuming that

class 1 has 1000 points

class 2 has 700 points

class 3 has 40 points

and we should assume similar distribution in the test set.

My conv net always predicts class 1 during training and fails the testing phase.

Answer 1

By rotation, do they literally mean rotate the image by a few degrees and then add it as another training point for the minority class?

Yes, though it can the done for other classes as well.

For example, http://benanne.github.io/2014/04/05/galaxy-zoo.html did the following data augmentation tricks:

Exploiting spatial invariances

Images of galaxies are rotation invariant: there is no up or down in space. They are also scale invariant and translation invariant to a limited extent. All of these invariances could be exploited to do data augmentation: creating new training data by perturbing the existing data points.

Each training example was perturbed before presenting it to the network by randomly scaling it, rotating it, translating it and optionally flipping it. I used the following parameter ranges:

• rotation: random with angle between 0° and 360° (uniform)
• translation: random with shift between -4 and 4 pixels (relative to the original image size of 424x424) in the x and y direction (uniform)
• zoom: random with scale factor between 1/1.3 and 1.3 (log-uniform)
• flip: yes or no (bernoulli)

Because both the initial downsampling to 69x69 and the random perturbation are affine transforms, they could be combined into one affine transformation step (I used scikit-image for this). This sped up things significantly and reduced information loss.

Colour perturbation

After this, the colour of the images was changed as described in Krizhevsky et al. 2012, with two differences: the first component had a much larger eigenvalue than the other two, so only this one was used, and the standard deviation for the scale factor alpha was set to 0.5.

"Realtime" augmentation

Combining downsampling and perturbation into a single affine transform made it possible to do data augmentation in realtime, i.e. during training. This significantly reduced overfitting because the network would never see the exact same image twice. While the network was being trained on a chunk of data on the GPU, the next chunk would be generated on the CPU in multiple processes, to ensure that all the available cores were used.

Centering and rescaling

I experimented with centering and rescaling the galaxy images based on parameters extracted with sextractor. Although this didn't improve performance, including a few models that used it in the final ensemble helped to increase variance (see "Model averaging" for more information).

I extracted the center of the galaxies, as well as the Petrosian radius. A number of different radii can be extracted, but the Petrosian radius seemed to give the best size estimate. I then centered each image by shifting the estimated center pixel to (212, 212), and rescaled it so that its Petrosian radius would be equal to 160 pixels. The scale factor was limited to the range (1/1.5, 1.5), because there were some outliers.

This rescaling and centering could also be collapsed into the affine transform doing downsampling and perturbation, so it did not slow things down at all.

Given that class 2 is also misclassified by your network, the issue doesn't seem to be fully a class imbalance issue. The choice of the ratio depends on a few factors such as how much weight you want to give to each class. FYI Opinions about Oversampling in general.