Choice of mean and variance parameter when normalizing image dataset

Question

Problem

Normalizing dataset is a common component of machine learning before doing any downstream task. However, when I learn the tutorial in PyTorch, the author used mysterious mean and variance value for different channels of RGB images, i.e. 0.485, 0.456 and 0.406 for mean and 0.229, 0.224, 0.225 for variance (see code below).

I am not sure how those somehow magic numbers come about. Do they come from some statistics of the training dataset which is not disclosed in tutorial. Or maybe they are heuristics that prove to be effective in computer vision literature?

# Data augmentation and normalization for training
# Just normalization for validation
data_transforms = {
    'train': transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
    'val': transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
}

Answer 1

The tutorial uses "TRANSFER LEARNING" this is based on using a pre-trained neural network. The people who developed the model used those parameters when they trained their model. In this particular tutorial, the resnet18 model was used. If you read the paper Deep Residual Learning for Image Recognition the developers of the model they provide the following information :

"Our implementation for ImageNet follows the practice in [21, 41]. The image is resized with its shorter side randomly sampled in [256, 480] for scale augmentation [41]. A 224×224 crop is randomly sampled from an image or its horizontal flip, with the per-pixel mean mean subtracted [21].