0
$\begingroup$

The salient feature, which was said to make MobileNet v1 efficient in terms of computational complexity, is the usage of depthwise convolutions, which is in essence a sequence of $M \rightarrow M$ channel convolution, where for each input channel $i \in [1, M]$, the kernel is shared, together with $M \rightarrow N$ $1 \times 1$ ordinary convolution.

enter image description here

The total reduction in complexity is: $$ \frac{D_K^2 \cdot M \cdot D_F^2 + M \cdot N \cdot D_F^2}{D_K^2 \cdot M \cdot N \cdot D_F^2} = \frac{1}{N} + \frac{1}{D_K^2} $$ And in the paper this approach lead to $8-9$ times reduction in number of parameters with not a big loss in accuracy. This architecture indeed allows one to have the same number of channels and size of convolutions kernels.

However, I wonder, whether the same quaility can be achieved by simply reducing the number of filters. For instance, assuming $M, N$ are larger than $D_K^2$, would the usage of $M / D_K, N / D_K$ with the same kernel sizes lead to more significant decrease in quality of the model? Less filters means less expressive power of the network, but the reduction of accuracy (error rate) I suppose does not scale linearly with the number of parameters, but rather in some exponential-like way.

Has there been some study, comparing the two approaches - reduction of number of channels and depthwise convolutions?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.