2D max pool gradient propagation

Question

I am trying to understand gradient propagation for a 2D max pooling operation when there is multiple filters for each position in the 2D grid (i.e. size = $b\times2\times2\times d$, where $b$ is batch size and $d$ is the number of filters).

From my understanding, max values get given the input gradients, whereas all non-max values get a gradient of 0 as a change of their input does not affect the result.

So as an example say $b=1$ and $d=2$, I perform a max pool with kernel size $[2, 2]$. On the forward pass:

$\begin{bmatrix} [1, 2] & [4,6] \\ [3,7] & [2,2]\end{bmatrix}$ $\to$ $[[4, 7]]$ (max of first and second dimensions respectively).

Say on backpropagation the input gradients $g=[1, 2]$, would the output gradients of the max pool operation be:

$\begin{bmatrix} [0, 0] & [1,0] \\ [0,2] & [0,0]\end{bmatrix}$

I have only seen examples where $d=1$ so wanted to make sure I am doing this correctly.

Answer 1

Your understanding is correct. For the sake of clarity, the max pooling operation can be understood as a rectifier function defined by (see Wikipedia entry for further details), $$ f(x) = max(0, x) $$ whose derivative is $$ f^{'}(x) = 1 \text{ if } y \geq 0, 0 \text{ otherwise} $$ where we have chosen the derivative to be 1 at 0. But here, instead of comparing against 0, you compare against the maximum value within the pooling mask.