In the batch normalization paper it is mentioned that the same gamma and beta are used throughout a feature map for convolutional layers. I interpret this as the filter weights are consistently used throughout a feature map (i.e. only 1 channel depth), then this same filter weight will also have the same gamma and beta applied to its activations throughout the feature map spatially. This is why it is sometimes called spatial batch norm.
But the paper also mentions that the batch normalization transform is applied independently to each separate dimension. Since the feature map has spatially 2 dimensions, is there a total of 2 gammas and 2 betas used for each input feature map, or is there only one set of gamma and beta? I'm inclined to believe there's only one set, but I can't find clarifications in the paper.
Also, for convolutional neural networks, I'm assuming it will then be a set of gamma and beta for each feature map (e.g. of shape
(batch_size, height, width, 1) for a block of input (e.g. of shape
(batch_size, height, width, depth), and in total having a number of set of gamma and beta that is equal to the channel depth?