In the paper that describes the multi-scale context aggregation by dilated convolutions, the authors state that their proposed architecture is motivated by the fact that dilated convolutions support exponentially expanding receptive fields without losing resolution or coverage, and use an example to illustrate the same:

Let $F_0,F_1,..., F_{n−1} : \mathbb{Z}^2 → \mathbb{R}$ be discrete functions and let $k_0, k_1 , ..., k_{n−2} : Ω_1 → \mathbb{R}$ be discrete $3×3$ filters. Consider applying the filters with exponentially increasing dilation : $F_{i+1} = F_i ∗_{2^i} k_i$ for $i = 0,1,...,n−2$.

Define the receptive field of an element $\textbf{p}$ in $F_{i+1}$ as the set of elements in $F_0$ that modify the value of $F_{i+1}(\textbf{p})$. Let the size of the receptive field of $\textbf{p}$ in $F_{i+1}$ be the number of these elements. It is easy to see that the size of the receptive field of each element in $F_{i+1}$ is $(2^{i+2}−1)×(2^{i+2}−1)$. The receptive field is a square of exponentially increasing size.

This is the accompanying figure

It would be of great help if I could get a derivation of the last formula $(2^{i+2}−1)×(2^{i+2}−1)$ !

I read here that when using dilated convolutions, the filter size, initially $k$ would be expanded to $k + (k-1)(r-1)$, $r$ being the dilation factor.

I also read in this answer that the layer wise receptive field size could be calculated using the formula $s_{l_{i+1}} = s_{l_i} + (kernelsize−1) * dilationfactor$, where $l_i$ denotes the different network layers and $s$ the receptive field size, $s_{l_0} = 1$. I don't know how this formula was derived either.

I can't seem to figure out how these formulas can be used to calculate the resulting generalized receptive field size in an inductive manner.

I mean, applying the formula $s_{l_{i+1}} = s_{l_i} + (kernelsize−1) * dilationfactor$, the receptive field size will be $s_{l_{i+1}} = s_{l_{i}} + 2 * 2^i $ since $2^i$ will be the dilation factor when calculating the receptive field size for layer $i+1$, and $kernelsize - 1$ would be $3 - 1 = 2$.

So that would be the same as $ s_{l_{i}} + 2^{i+1} $

$\implies s_{l_{i-1}} + (s^{i}) + 2^{i+1} $

$\implies s_{l_{i-1}} + (2^i) + 2^{i+1} $ using the same formula for layer $i$.

Expanding backwards this way, we get :

$\implies 1 + 2^1 + 2^2 + ... + 2^{i+1} $

$\implies 2^{(i+1) + 1} - 1 $ using the formula to calculate sum of powers of 2.

$\implies 2^{i+2} - 1 $



Instead of providing a numerical analysis - as you derived it yourself in your answer - I shall provide a visual one. This illustration is specific to 1 dimensional convolutions with a kernel size of 2, as opposed to 2 dimensional convolutions with a kernel size of 3. However simplifying the scenario can help build intuition by thinking of dilated convolutions creating a 'tree' where the root of the is an output element of the stack and the leafs are elements of the input.

dilated convolution tree


I was able to derive it correctly, but only because I had the formula $s_{l_{i+1}} = s_{l_i} + (kernelsize−1) * dilationfactor$ at hand.

I'm not sure how it was derived.

I would appreciate it if somebody could explain the logic behind the formula $s_{l_{i+1}} = s_{l_i} + (kernelsize−1) * dilationfactor$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.