18
$\begingroup$

Most common convolutional neural networks contains pooling layers to reduce the dimensions of output features. Why couldn't I achieve the same thing by simply increase the stride of the convolutional layer? What makes the pooling layer necessary?

$\endgroup$
16
$\begingroup$

You can indeed do that, see Striving for Simplicity: The All Convolutional Net. Pooling gives you some amount of translation invariance, which may or may not be helpful. Also, pooling is faster to compute than convolutions. Still, you can always try replacing pooling by convolution with stride and see what works better.

Some current works use average pooling (Wide Residual Networks, DenseNets), others use convolution with stride (DelugeNets)

$\endgroup$
  • $\begingroup$ I asked one of my friends about this and he said the pooling layers are better because it introduces non-linearity. Do you agree? $\endgroup$ – user3667089 Jul 8 '17 at 16:38
  • 7
    $\begingroup$ Hm not so sure I agree. Some kind of nonlinearity is already present in the networks through the activation functions. Average pooling also does not introduce any additional nonlinearity, it is a linear operation so only max pooling is nonlinear. And I think the question is more if you want the regularization that pooling brings you - a little more translational invariance. $\endgroup$ – robintibor Jul 9 '17 at 20:32
4
$\begingroup$

Apparently max pooling helps because it extracts the sharpest features of an image. So given an image, the sharpest features are the best lower-level representation of an image. https://www.quora.com/What-is-the-benefit-of-using-average-pooling-rather-than-max-pooling

But according to Andrew Ng's Deep Learning lecture, max pooling works well but no one knows why. Quote -> "But I have to admit, I think the main reason people use max pooling is because it's been found in a lot of experiments to work well, ... I don't know of anyone fully knows if that is the real underlying reason."

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.