By rotation, do they literally mean rotate the image by a few degrees and then add it as another training point for the minority class?
Yes, though it can the done for other classes as well.
For example, http://benanne.github.io/2014/04/05/galaxy-zoo.html did the following data augmentation tricks:
Exploiting spatial invariances
Images of galaxies are rotation invariant: there is no up or down in
space. They are also scale invariant and translation invariant to a
limited extent. All of these invariances could be exploited to do
data augmentation: creating new training data by perturbing the existing data points.
Each training example was perturbed before presenting it to the
network by randomly scaling it, rotating it, translating it and
optionally flipping it. I used the following parameter ranges:
- rotation: random with angle between 0° and 360° (uniform)
- translation: random with shift between -4 and 4 pixels (relative to the original image size of 424x424) in the x and y direction
(uniform)
- zoom: random with scale factor between 1/1.3 and 1.3 (log-uniform)
- flip: yes or no (bernoulli)
Because both the initial downsampling to 69x69 and the random
perturbation are affine transforms, they could be combined into one
affine transformation step (I used scikit-image for this). This sped
up things significantly and reduced information loss.
Colour perturbation
After this, the colour of the images was changed as described in
Krizhevsky et al.
2012, with two
differences: the first component had a much larger eigenvalue than the
other two, so only this one was used, and the standard deviation for
the scale factor alpha was set to 0.5.
"Realtime" augmentation
Combining downsampling and perturbation into a single affine transform
made it possible to do data augmentation in realtime, i.e. during
training. This significantly reduced overfitting because the network
would never see the exact same image twice. While the network was
being trained on a chunk of data on the GPU, the next chunk would be
generated on the CPU in multiple processes, to ensure that all the
available cores were used.
Centering and rescaling
I experimented with centering and rescaling the galaxy images based on
parameters extracted with
sextractor. Although
this didn't improve performance, including a few models that used it
in the final ensemble helped to increase variance (see "Model
averaging" for more information).
I extracted the center of the galaxies, as well as the Petrosian
radius. A number of different radii can be extracted, but the
Petrosian radius seemed to give the best size estimate. I then
centered each image by shifting the estimated center pixel to (212,
212), and rescaled it so that its Petrosian radius would be equal to
160 pixels. The scale factor was limited to the range (1/1.5, 1.5),
because there were some outliers.
This rescaling and centering could also be collapsed into the affine
transform doing downsampling and perturbation, so it did not slow
things down at all.
Given that class 2 is also misclassified by your network, the issue doesn't seem to be fully a class imbalance issue. The choice of the ratio depends on a few factors such as how much weight you want to give to each class. FYI Opinions about Oversampling in general.
