Pooling and stride both can be used to downsample the image.

Let's say we have an image of 4x4, like below enter image description here and a filter of 2x2. Then how do we decide whether to use (2x2 pooling) vs. (stride of 2)?


The advantage of the convolution layer is that it can learn certain properties that you might not think of while you add pooling layer. Pooling is a fixed operation and convolution can be learned. On the other hand, pooling is a cheaper operation than convolution, both in terms of the amount of computation that you need to do and number of parameters that you need to store (no parameters for pooling layer).

There are examples when one of them is better choice than the other.

Example when the convolution with strides is better than pooling

The first layer in the ResNet uses convolution with strides. This is a great example of when striding gives you an advantage. This layer by itself significantly reduces the amount of computation that has to be done by the network in the subsequent layers. It compresses multiple 3x3 convolution (3 to be exact) in to one 7x7 convolution, to make sure that it has exactly the same receptive field as 3 convolution layers (even though it is less powerful in terms of what it can learn). At the same time this layer applies stride=2 that downsamples the image. Because this first layer in ResNet does convolution and downsampling at the same time, the operation becomes significantly cheaper computationally. If you use stride=1 and pooling for downsampling, then you will end up with convolution that does 4 times more computation + extra computation for the next pooling layer. The same trick was used in SqueezeNet and some other neural network architectures.

Example where pooling is better than convolution

In the NIPS 2018, there was a new architecture presented called FishNet. One thing that they try is to fix the problems with the residual connections used in the ResNet. In the ResNet, in few places, they put 1x1 convolution in the skip connection when downsampling was applied to the image. This convolution layer makes gradient propagation harder. One of the major changes in their paper is that they get rid of the convolutions in the residual connections and replaced them with pooling and simple upscales/identities/concatenations. This solution fixes problem with gradient propagation in very deep networks.

From the FishNet paper (Section 3.2)

The layers in the head are composed of concatenation, convolution with identity mapping, and max-pooling. Therefore, the gradient propagation problem from the previous backbone network in the tail are solved with the FishNet by 1) excluding I-conv at the head; and 2) using concatenation at the body and the head.


In essence, max-pooling (or any kind of pooling) is a fixed operation and replacing it with a strided convolution can also be seen as learning the pooling operation, which increases the model's expressiveness ability. The down side is that it also increases the number of trainable parameters, but this is not a real problem in our days.

There is a very good article by JT Springenberg, where they replace all the max-pooling operations in a network with strided-convolutions. The paper demonstrates how doing so, improves the overall accuracy of a model with the same depth and width: "when pooling is replaced by an additional convolution layer with stride r = 2 performance stabilizes and even improves on the base model"

Striving for Simplicity: The All Convolutional Net

I encourage you to read the article, it isn't a hard read.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.