3
$\begingroup$

I am wondering about how 1x1 convolution can be used to change the dimensionality of feature maps in a residual learning network.enter image description here

Here the top 1x1 convolution changes the feature map size from 256 to 64. How is this possible?

In a previous post explaining 1x1 convolution in neural net, it is mentioned that, if a layer having $n_1$ feature maps is subjected to 1x1 convolution with $n_2$ filters then number of feature map changes to $n_2$. Shouldn't it be $n_1$$n_2$ since each of the $n_2$ filters produce one output corresponding to each of the $n_1$ inputs.

Also how does one generate 256 feature maps from 64, as done in the bottom layer.

Improve this question
$\endgroup$
0
$\begingroup$

There's only one parameter for each input map in a 1*1 filter, actually the 1*1 convolution is multiplying the every element of an input map by the same scalar.

So it is similar to getting 265 linear combinations out of 64 variables, the $n$-th feature map $y_n$ is like,

$$y_n=f(w_{n,1}x_1+w_{n,2}x_2+...+w_{n,64}x_{64})$$ so actually we can get any number of output feature maps as we want. Of course if the output dimension is greater than the input dimension, the output would be redundant.

Improve this answer
$\endgroup$
11
  • $\begingroup$ What is $w^n_i$ ?. Is it just $w_i$ $\endgroup$
    – Newstein
    Jul 1 '16 at 9:50
  • $\begingroup$ @Newstein It's meant to be in accordance with the subscript of $y_n$, so you have a different set of weights for different output channels. $\endgroup$
    – dontloo
    Jul 1 '16 at 9:53
  • 1
    $\begingroup$ @Newstein yes by weights I mean the single parameter in a 1x1 filter, and there're not in total 64 filters, there are 64 output channels (or feature maps), which means there'll be 64*256 filters (you apply a different filter to each input channel, add up the results and apply an activation function to get the output of one channel) $\endgroup$
    – dontloo
    Jul 1 '16 at 10:37
  • 1
    $\begingroup$ I really think this answer is misleading. There a 1x1 convolutional filter is "1x1" just in the spatial dimension, but it is a 1x1xC convolution, where C is the number of input feature maps or channels (256 for the first layer in the OPs example example). Therefore, you actually have C parameters for a 1x1 convolution. Since you want to output 64 different feature maps, you have 1x1xCx64 parameters for the whole layer. $\endgroup$
    – Pepe Mandioca
    Feb 8 '18 at 16:47
  • 1
    $\begingroup$ @facuq yea it is, I've made some edit $\endgroup$
    – dontloo
    Feb 9 '18 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.