How does masking work to make neural networks use varying input lengths A well known solution to make Neural Networks (NNs) work with varying lengths of input data is to use padding and then masking the padded inputs (padded with zero, a common approach in natural language processing). It's being said that this approach makes the NN skip the padded inputs. I want to know what's the mechanism behind such skipping. For me NNs are all about matrix multiplication. So here are my questions:

*

*Is masking done by forcing some weights to zero?

*How is it different than simply using the padded values that are
zero without masking (zero times anything is zero again, hence I
suppose the answer to my 1st question should be no, because there is
no point in forcing the weights to zero when the input is zero
itself)

*I assume masking didn’t suddenly emerge out of thin air, so I want
to know who invented the method? Are there any academic papers
talking about it?

 A: Masking is not enforcing weights to be zero, masking enforces network inputs and intermediate hidden states to be zero. Setting the inputs to be zero at the begging is not enough; many places in a NN would make the states non-zero.
A typical feed-forward layer with an activation function $\sigma$ is $\sigma(Wx + b)$, and the output will be non-zero because of the bias $b$. When using batch normalization, an estimated mean is subtracted from the activation, which makes it non-zero. Layer normalization has a learned bias term, which also would make the hidden state non-zero.
Masking is, however, only relevant when there is a danger that the invalid mask-out hidden states would influence the network output. This happens, e.g., in Transformers in the self-attention layers: you only want the attention to consider valid positions. This is done by modifying the attention distribution (before normalization) by setting zeros to invalid positions, so it only considers the positions that were part of the input. Hidden states on the invalid position will get some values in the feed-forward layers and during layer normalization, but it does not matter as long as it does not influence the valid states.
Masking invalid input positions is considered a technical detail, and such details are often not discussed in papers. The first time I encountered masking was in 2014 when the original attention paper appeared on arXiv. They released an implementation on Github which did the masking in the attention mechanism, but the paper did not say a word about it.
