6
$\begingroup$

In the paper Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012., section 4.1, the authors describe their data augmentation process. They say they increased the size of the training set by a factor of 2048. Does this mean they trained on a total of 2048 × 1.2 millions images?

Moreover, I don't quite follow this:

At test time, the network makes a prediction by extracting five 224 × 224 patches (the four corner patches and the center patch) as well as their horizontal reflections (hence ten patches in all), and averaging the predictions made by the network’s softmax layer on the ten patches.

What do they mean they extracted five 224 × 224 patches (corner, center and horizontal)? And why does it result in ten patches in total?

Thank you!

$\endgroup$

3 Answers 3

7
$\begingroup$

They say they increased the size of the training set by a factor of 2048. Does this mean they trained on a total of 2024 X 1.2 millions images?

Yes, in the paper:

The first form of data augmentation consists of generating image translations and horizontal reflections. We do this by extracting random 224x224 patches (and their horizontal reflections) from the 256x256 images and training our network on these extracted patches

They are generating those extra patches 'on the fly' from the original images, said here:

In our implementation, the transformed images are generated in Python code on the CPU while the GPU is training on the previous batch of images. So these data augmentation schemes are, in effect, computationally free.


What do they mean they extracted five 224 × 224 patches (corner, center and horizontal)? And why does it result in ten patches in total?

The original images have size 256x256, so they are getting a patch by cropping the original picture on the upper left corner with a size 224x224. Same thing for upper right, lower left, lower right and center. So that's making 5 patches. And for each of those patch they are mirroring the picture, so they get 5 more patches. Total 10, and then they take the average prediction.

$\endgroup$
2
  • $\begingroup$ Thank you! To clarify, in testing to predict 1 image, it will average over 10 patches of size 224X224. $\endgroup$ Commented Oct 22, 2015 at 22:48
  • $\begingroup$ Yes, that's correct $\endgroup$
    – ThiS
    Commented Oct 23, 2015 at 11:36
5
$\begingroup$

I think they've trained only on 1.2M images. Here is why:

Even if they could get 0.001s per forward and backward pass (with 1 Titan X and cuDNN), it would take this much time to train on 2048*1.28M images for 90 epochs with mini-batch SGD:

0.001*2048*1280000*90/60/60/24 = ~ 2730 days = ~ 7.5 years

$\endgroup$
3
  • $\begingroup$ This is an interesting observation. In your opinion, what do the authors mean with the 2048 factor. $\endgroup$ Commented Dec 1, 2015 at 10:29
  • $\begingroup$ Good question - for them. $\endgroup$ Commented Dec 1, 2015 at 21:16
  • $\begingroup$ Now in 2018, I went with the same problem, it would take many years to complete the training. The reason I'm here now is to verify that and this does help. Thank you $\endgroup$ Commented Jan 20, 2019 at 13:40
0
$\begingroup$

They are actually training on 1.2 million * 2048 training images.

We do this by extracting random 224 × 224 patches (and their horizontal reflections) from the 256×256 images

For each training image of size 256x256, if you extract patches of size 224x224, you can get up to 1024 224x224 patches from the image ((256-224)*(256-224)). And for each such patch you take a horizontal reflection. In total 2048 patches from a single image.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.