I am trying to understand gradient propagation for a 2D max pooling operation when there is multiple filters for each position in the 2D grid (i.e. size = $b\times2\times2\times d$, where $b$ is batch size and $d$ is the number of filters).

From my understanding, max values get given the input gradients, whereas all non-max values get a gradient of 0 as a change of their input does not affect the result.

So as an example say $b=1$ and $d=2$, I perform a max pool with kernel size $[2, 2]$. On the forward pass:

$\begin{bmatrix} [1, 2] & [4,6] \\ [3,7] & [2,2]\end{bmatrix}$ $\to$ $[[4, 7]]$ (max of first and second dimensions respectively).

Say on backpropagation the input gradients $g=[1, 2]$, would the output gradients of the max pool operation be:

$\begin{bmatrix} [0, 0] & [1,0] \\ [0,2] & [0,0]\end{bmatrix}$

I have only seen examples where $d=1$ so wanted to make sure I am doing this correctly.


Your understanding is correct. For the sake of clarity, the max pooling operation can be understood as a rectifier function defined by (see Wikipedia entry for further details), $$ f(x) = max(0, x) $$ whose derivative is $$ f^{'}(x) = 1 \text{ if } y \geq 0, 0 \text{ otherwise} $$ where we have chosen the derivative to be 1 at 0. But here, instead of comparing against 0, you compare against the maximum value within the pooling mask.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.