3
$\begingroup$

I am wondering about how 1x1 convolution can be used to change the dimensionality of feature maps in a residual learning network.enter image description here

Here the top 1x1 convolution changes the feature map size from 256 to 64. How is this possible?

In a previous post explaining 1x1 convolution in neural net, it is mentioned that, if a layer having $n_1$ feature maps is subjected to 1x1 convolution with $n_2$ filters then number of feature map changes to $n_2$. Shouldn't it be $n_1$$n_2$ since each of the $n_2$ filters produce one output corresponding to each of the $n_1$ inputs.

Also how does one generate 256 feature maps from 64, as done in the bottom layer.

|cite|improve this question
$\endgroup$
0
$\begingroup$

There's only one parameter for each input map in a 1*1 filter, actually the 1*1 convolution is multiplying the every element of an input map by the same scalar.

So it is similar to getting 265 linear combinations out of 64 variables, the $n$-th feature map $y_n$ is like,

$$y_n=f(w_{n,1}x_1+w_{n,2}x_2+...+w_{n,64}x_{64})$$ so actually we can get any number of output feature maps as we want. Of course if the output dimension is greater than the input dimension, the output would be redundant.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ What is $w^n_i$ ?. Is it just $w_i$ $\endgroup$ – Newstein Jul 1 '16 at 9:50
  • $\begingroup$ @Newstein It's meant to be in accordance with the subscript of $y_n$, so you have a different set of weights for different output channels. $\endgroup$ – dontloo Jul 1 '16 at 9:53
  • $\begingroup$ @Newstein I made that up, it's not in the paper. :P $\endgroup$ – dontloo Jul 1 '16 at 9:54
  • 1
    $\begingroup$ @Newstein yes by weights I mean the single parameter in a 1x1 filter, and there're not in total 64 filters, there are 64 output channels (or feature maps), which means there'll be 64*256 filters (you apply a different filter to each input channel, add up the results and apply an activation function to get the output of one channel) $\endgroup$ – dontloo Jul 1 '16 at 10:37
  • 1
    $\begingroup$ @facuq yea it is, I've made some edit $\endgroup$ – dontloo Feb 9 '18 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.