Timeline for Why do people use Zero-Padding in Convolutional Neural Networks?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 22, 2023 at 12:06 | comment | added | Ahmed Alhallag | "when weights in a filter drop rapidly away from its center.", can you explain what does that mean please? i'm new to the field | |
| Apr 1, 2019 at 18:01 | comment | added | resnet | Makes sense :). I think in most cases, something like "Min-Padding" would make sense as a more general recommendation instead of Zero-Padding? I.e., always choosing the smallest activation value. Currently can't think of a counter example as activation functions I know are usually monotonic. What do you think? | |
| Apr 1, 2019 at 17:47 | comment | added | Esmailian | @resnet you are right, I was about to say because 0 is equal to "non-existent" input, but again we can define non-existent as "-1"! So I removed that part. | |
| Apr 1, 2019 at 17:45 | history | edited | Esmailian | CC BY-SA 4.0 |
Explanation removed
|
| Apr 1, 2019 at 17:29 | comment | added | resnet | Thanks for the follow up via your edit. Regarding "Another important note is that the filters are doing weighted sum over input patches, thus we generally want a padding that is as neutral as possible for the summation, i.e. $f(i,j)= w(i,j) \times x(i, j) +w(i+1, j+1) \times 0 +..$. which further justifies the zero-padding as a generic solution." --> I am not sure if this is as neutral as possible, because when using e.g., tanH, you have a maximum gradient when doing back propagation when the activations are 0. For ReLU, this reasoning may make sense though. | |
| Apr 1, 2019 at 14:24 | history | edited | Esmailian | CC BY-SA 4.0 |
Explanation improved
|
| Apr 1, 2019 at 14:09 | comment | added | resnet | Because like you said, one of the goals is to avoid shrinking and also information loss at the edges. I usually think of the min value as "placeholder pixels (e.g., black in b/w images)" E.g., if MNIST was b/w and normalized to a [-0.5, 0.5] range, zero padding would add gray pixels, which is not immediately intuitive | |
| Apr 1, 2019 at 14:03 | comment | added | resnet | > "For a specific input, activation function, or loss function, a variant might perform better," That makes sense, but I am wondering if choosing always the min value of the activation function would already be an improvement over using 0. Same for images after normalization. | |
| Apr 1, 2019 at 10:23 | history | edited | Esmailian | CC BY-SA 4.0 |
Wording improved
|
| Apr 1, 2019 at 10:10 | history | edited | Esmailian | CC BY-SA 4.0 |
Explanation improved
|
| Apr 1, 2019 at 10:02 | history | answered | Esmailian | CC BY-SA 4.0 |