0
$\begingroup$

Say I have a layer a:

3   4   2
1   5   0
8   6   4

The maxpool using 2x2 filter is:

5  5
8  6

Thus the derivative of the max pooling layer is (in respect to layer a):

0   0   0
0   1   0
1   1   0

Now say doing backpropagation I have the following deltas:

-1.4   0.8
0.2    0.3

I know that -1.4 and 0.8 is associated with 5. While 0.2 and 0.3 corresponds to 8 and 6 respectively.

Currently what I have done, for repeating max values in the same position, I just sum up the corresponding deltas associated with that position, i.e., -1.4 + 0.8 = -0.6. So the final delta being back propagated is the following matrix.

0    0 0   0
0   -0.6   0
0.2  0.3   0

Please let me know if this is the correct procedure.

$\endgroup$
  • $\begingroup$ the derivative of a 2x2 output wrt a 3x3 layer can't possibly be 3x3 in shape $\endgroup$ – shimao Apr 12 at 2:21
  • $\begingroup$ You are correct, However I said the derivative of the max values w.r.t. to layer a so one can see the derivatives of corresponding max values and where the deltas will be backpropagated in layer a. The maxpool layer is 2x2 and thus the derivative is [[1, 1], [1, 1]], which does not give you enough information. These 1's needs to be represented in the previous layer (3x3) as to where the max values are coming from initially for backpropagation purposes. $\endgroup$ – Jam1 Apr 12 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.