19
$\begingroup$

Both the terms "upsampling" and "transpose convolution" are used when you are doing "deconvolution" (<-- not a good term, but let me use it here). Originally, I thought that they mean the same thing, but it seems to me that they are different after I read these articles. can anyone please clarify?

  1. Transpose convolution: looks like we can use it when we propoagate the loss via convolutonal neural network.

    http://andrew.gibiansky.com/blog/machine-learning/convolutional-neural-networks/#Backward-Propagation

    https://github.com/vdumoulin/conv_arithmetic

    https://arxiv.org/pdf/1312.6034v2.pdf, section 4 "For the convolutional layer..."

  2. Upsampling: seems like we use it when we want to upsample from smaller input to larger input in convnet-decovnet structure.

    https://www.youtube.com/watch?v=ByjaPdWXKJ4&feature=youtu.be&t=22m

$\endgroup$
4
  • 1
    $\begingroup$ Cross-posted at datascience.stackexchange.com/questions/15863/…. Please don't cross-post but decide which site you want your question on. $\endgroup$ Commented Dec 22, 2016 at 8:09
  • $\begingroup$ @Scortchi Oh they are linked!? Sorry, I remove the one in data science. $\endgroup$ Commented Dec 22, 2016 at 18:44
  • $\begingroup$ A interesting post on so-called deconvolutions $\endgroup$
    – hans
    Commented Jan 11, 2019 at 20:00
  • $\begingroup$ "Upsampling refers to any technique that upsamples your image to a higher resolution. Deconvolution is often used in tasks such as image segmentation and object detection.", tf.nn.conv2d_transpose is sometimes called "deconvolution", but is really the transpose (gradient) of atrous_conv2d ( (a.k.a. convolution with holes or dilated convolution) rather than an actual deconvolution. - here $\endgroup$
    – JeeyCi
    Commented Mar 30 at 11:49

4 Answers 4

16
$\begingroup$

Since there is no detailed and marked answer, I'll try my best.

Let's first understand where the motivation for such layers come from: e.g. a convolutional autoencoder. You can use a convolutional autoencoder to extract featuers of images while training the autoencoder to reconstruct the original image. (It is an unsupervised method.)

Such an autoencoder has two parts: The encoder that extracts the features from the image and the decoder that reconstructs the original image from these features. The architecture of the encoder and decoder are usually mirrored.

In a convolutional autoencoder, the encoder works with convolution and pooling layers. I assume that you know how these work. The decoder tries to mirror the encoder but instead of "making everything smaller" it has the goal of "making everything bigger" to match the original size of the image.

The opposite of the convolutional layers are the transposed convolution layers (also known as deconvolution, but correctly mathematically speaking this is something different). They work with filters, kernels, strides just as the convolution layers but instead of mapping from e.g. 3x3 input pixels to 1 output they map from 1 input pixel to 3x3 pixels. Of course, also backpropagation works a little bit different.

The opposite of the pooling layers are the upsampling layers which in their purest form only resize the image (or copy the pixel as many times as needed). A more advanced technique is unpooling which resverts maxpooling by remembering the location of the maxima in the maxpooling layers and in the unpooling layers copy the value to exactly this location. To quote from this (https://arxiv.org/pdf/1311.2901v3.pdf) paper:

In the convnet, the max pooling operation is non-invertible, however we can obtain an approximate inverse by recording the locations of the maxima within each pooling region in a set of switch variables. In the deconvnet, the unpooling operation uses these switches to place the reconstructions from the layer above into appropriate locations, preserving the structure of the stimulus.

For more technical input and context have a look at this really good, demonstrative and in depth explanation: http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html

And have a look at https://www.quora.com/What-is-the-difference-between-Deconvolution-Upsampling-Unpooling-and-Convolutional-Sparse-Coding

$\endgroup$
1
  • $\begingroup$ F. Chollet (creator of Keras) would argue that this is a self-supervised technique. $\endgroup$
    – hans
    Commented Jan 11, 2019 at 19:52
8
$\begingroup$

It may depend on the package you are using.

In keras they are different. Upsampling is defined here https://github.com/fchollet/keras/blob/master/keras/layers/convolutional.py Provided you use tensorflow backend, what actually happens is keras calls tensorflow resize_images function, which essentially is an interpolation and not trainable.

Transposed convolution is more involved. It's defined in the same python script listed above. It calls tensorflow conv2d_transpose function and it has the kernel and is trainable.

Hope this helps.

$\endgroup$
6
$\begingroup$

here is a pretty good illustration on the difference between 1) transpose convolution and 2) upsampling + convolution. https://distill.pub/2016/deconv-checkerboard/

While the transpose convolution is more efficient, the article advocates for upsampling + convolution since it does not suffer from the checkerboard artifact.

$\endgroup$
2
$\begingroup$

Deconvolution in the context of convolutional neural networks is synonymous to transpose convolution. Deconvolution may have another meanings in other fields.

Transpose convolution is one strategy amongst others to perform upsampling.

$\endgroup$
3
  • $\begingroup$ yes, I agree. but seem like the way the references explain them is different. Take a look at the video in No.2 and then look at the references in No.1 ( Personally, I go for No.1 explanation) $\endgroup$ Commented Dec 22, 2016 at 0:47
  • $\begingroup$ @RockTheStar Which concept is explained differently? Transpose convolution or upsampling? $\endgroup$ Commented Dec 22, 2016 at 0:49
  • 1
    $\begingroup$ the upsampling/deconvolution concept explained in the video in No.2. It is about few minute. $\endgroup$ Commented Dec 22, 2016 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.