Incomprehensible lowering of VGG19's accuracy if data is preprocessed I'm using the VGG19 convolutional network for image recognition (to be specific, the keras implementation). I've downloaded the ILSVRC2012 validation dataset and the ground truth from here, to check if the accuracy of VGG19 matches my expectation. Note that the labeling of the linked ground truth is different from the default ground truth labeling provided by ImageNet.
Since VGG19 expects input of the shape (224, 224, 3), one has to choose a way of dealing with input which is not of square shape. I avoided naive resizing which violates the aspect ratio (because it might change the ground truth). What I did is the following:


*

*Filter the validation dataset such that only images remain which satisfy 0.95 <= height / width <= 1.05 (i.e. which are nearly of square shape). Almost 4000 images fulfill this property.

*resize those images to 224 × 224 and call this smaller dataset D.

*a) Evaluate (Top 1 classification) VGG19 on D, which results in an accuracy of ~ 59.48%
b) Preprocess every image in D using keras.applications.vgg19.preprocess_input, call that new dataset P and evaluate (Top 1 classification) VGG19 on P, which results in an accuracy of ~ 52.46%
So now my two questions are:


*

*Why is the performance of VGG19 significantly lower than ~ 75%, which is the Top 1 accuracy on the validation dataset the authors of VGG claim to achieve?

*Why does preprocessing lower the accuracy even further?



Since D only contains the 4000 nearly square shaped images, a bias might explain the overall bad performance. To check this, I center-cropped every image from the validation dataset, resized it to 224 × 224 and repeated the evaluation on those – with similar results (~ 60% vs ~ 54%).

My setup: Keras 2.2.2, Tensorflow-GPU 1.10, Ubuntu 16.04
 A: I guess I found one error:
I loaded the validation images via OpenCV (cv2 module) by executing cv2.imread, which returns a Numpy array, where the color channel ordering is BGR. Using keras.applications.vgg19.preprocess_input on that Numpy array accidentally reverts the channel ordering back to RGB (although the target ordering is BGR) and then subtracts the mean BGR pixel [103.939, 116.779, 123.68].
After realizing that, I performed the following evaluation:

*

*Center-crop each image of the validation dataset using a box of maximal size


*Scale the cropped images to 224 × 224 using cv2.resize(..., interpolation = cv2.INTER_LANCZOS4)


*a) Evaluate VGG19 on the non-centered batch -> accuracy of ~ 63.25%
b) Center the pixel color values of the batch by image_batch = np.subtract(image_batch, [103.939, 116.779, 123.68], dtype = np.float32) and evaluate VGG19 on that -> accuracy of ~ 70.21%
This is now way closer to the expected 75% accuracy. The gap might be credited to the way I evaluate VGG (single center crop vs multi-crop/dense evaluation an the paper) but I'd like to ask you guys:
Did I miss an additional preprocessing step?

A side note: It's worth writing
image_batch = np.subtract(image_batch, [103.939, 116.779, 123.68], dtype = np.float32)

instead of just
image_batch -= [103.939, 116.779, 123.68]

because the latter one results on my machine in a float64 array, which is some kind of overkill (in terms of size and GPU performance).
