0
$\begingroup$

I am reading the SqueezeNet paper and I do not get the parameter depth here:

Image

There isn't a description under the table, and the only extra mention of the parameter is that it means the number of layers, but I do not get why maxpool has 0 and why having a 2 there seems to correlate with 64x2=128 and so on.

What is the meaning of the depth column ?

I assume that the column e(3x3) is correct, but if it is wrong that could explain it.

$\endgroup$

2 Answers 2

3
$\begingroup$

A fire module always consists of two layers: a squeeze layer followed by an expand layer. Therefore its depth is 2. You'll notice that the two regular conv layers both have depth 1, since they are just single layers. Pooling layers have a depth of 0 because they don't transform the data; they don't compute new features - they just pool them.

$\endgroup$
7
  • $\begingroup$ Having depth 0 because they do not transform the data does not seem an explanation to me. I think as the other answer indicates, it is because it counts the layers with learnable parameters. Also, how do you explain the dimensionality then i.e the fact you have 64 filters but the output is 128 depth. $\endgroup$
    – Minsky
    Sep 1, 2023 at 9:26
  • $\begingroup$ You've misunderstood what the hyperparameters mean. $\text{e}_\text{1x1}$ and $\text{e}_\text{3x3}$ are the numbers of 1x1 and 3x3 conv filters in the expand layer. Therefore, the total number of output channels of a fire module is $\text{e}_\text{1x1} + \text{e}_\text{3x3}$. This has nothing to do with the depth value. As for whether depth refers to the number of transformations or the number of layers with learnable parameters, I don't see how one explanation is more plausible than another. $\endgroup$ Sep 1, 2023 at 10:35
  • $\begingroup$ i did understand what the hp are. but i do not think you have to add them up, the result of a convolution is the result of the last expand filters, if the 3x3 filter has 64 filters, then that is the number of channels of the output, why do you add them up ? $\endgroup$
    – Minsky
    Sep 1, 2023 at 19:59
  • $\begingroup$ The expand layer consist of both 1x1 and 3x3 filters, so yes you do have to add them up (as @ylpjört also mentioned in a comment). Note that there is a difference between $\text{s}_\text{1x1}$ and $\text{e}_\text{1x1}$. The former counts the number of 1x1 filters in the squeeze layer; the latter in the expand layer (there are 1x1 filters in both). $\endgroup$ Sep 3, 2023 at 10:43
  • $\begingroup$ If we have an input of 28,28,3 and then 2 convolutions with 64 filters, each time the output will not have 128 filters but 64, how is this wrong ? $\endgroup$
    – Minsky
    Sep 4, 2023 at 13:25
1
$\begingroup$

My interpretation would be that this describes the number of distinct layers with learnable parameters to get a sense of overall network depth.

  • Each of the described "fire modules" consists of [1x1 conv] - ReLU - [1x1conv, 3x3conv] - ReLU, i.e., we have two distinct layers with learnable parameters per module (depth = 2).
  • The two layers conv1 and conv10 are simple convolution layers with a single activation, leading to a depth of 1.
  • Max/average pooling does not have learnable parameters and therefore has depth 0
$\endgroup$
4
  • $\begingroup$ This is close to an answer, but still does not explain the parameters i.e the fact that you have 64 filters but output with 128 channels and so on. "but I do not get why (..) having a 2 there seems to correlate with 64x2=128 and so on." $\endgroup$
    – Minsky
    Sep 1, 2023 at 9:28
  • $\begingroup$ Each fire module consists of a squeze layer and an expand layer (thus depth 2). The expand layer consists of the 1x1 and the 3x3 convolutional filters with their respective number of channels. Thus #output_channels = #1x1expand_filters + #3x3 expand_filters. $\endgroup$
    – ylpjört
    Sep 1, 2023 at 9:44
  • $\begingroup$ But the output channels of a fire module are just the output of the last convolution (the number of filters of the last convolution), not the sum. $\endgroup$
    – Minsky
    Sep 1, 2023 at 9:59
  • 1
    $\begingroup$ The last convolutional layer (the expand layer) includes both the 1x1 and 3x3 expand filters. You seem to be assuming that one would follow the other. (All together, the two layers in the fire module are squeeze, followed by expand. The number of filters in the first layer is given by $\text{s}_\text{1x1}$. The number of filters in the second layer is given by $\text{e}_\text{1x1} + \text{e}_\text{3x3}$.) $\endgroup$ Sep 1, 2023 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.