I'm using the VGG19 convolutional network for image recognition (to be specific, the keras implementation). I've downloaded the ILSVRC2012 validation dataset and the ground truth from here, to check if the accuracy of VGG19 matches my expectation. Note that the labeling of the linked ground truth is different from the default ground truth labeling provided by ImageNet.
Since VGG19 expects input of the shape
(224, 224, 3), one has to choose a way of dealing with input which is not of square shape. I avoided naive resizing which violates the aspect ratio (because it might change the ground truth). What I did is the following:
Filter the validation dataset such that only images remain which satisfy
0.95 <= height / width <= 1.05(i.e. which are nearly of square shape). Almost 4000 images fulfill this property.
resize those images to 224 × 224 and call this smaller dataset D.
a) Evaluate (Top 1 classification) VGG19 on D, which results in an accuracy of ~ 59.48%
b) Preprocess every image in D using
keras.applications.vgg19.preprocess_input, call that new dataset P and evaluate (Top 1 classification) VGG19 on P, which results in an accuracy of ~ 52.46%
So now my two questions are:
- Why is the performance of VGG19 significantly lower than ~ 75%, which is the Top 1 accuracy on the validation dataset the authors of VGG claim to achieve?
- Why does preprocessing lower the accuracy even further?
Since D only contains the 4000 nearly square shaped images, a bias might explain the overall bad performance. To check this, I center-cropped every image from the validation dataset, resized it to 224 × 224 and repeated the evaluation on those – with similar results (~ 60% vs ~ 54%).
My setup: Keras 2.2.2, Tensorflow-GPU 1.10, Ubuntu 16.04