I read here:
Attention is a generalized pooling method with bias alignment over inputs.
What do they mean by bias alignment?
My understanding is that Attention (full diagram below) is a circuit where the output is a weighted sum of values from a K-V store, where input queries can be compared with keys. This query-key comparison can be done in a number of ways and is often called the Attention layer (depicted as a white box below).
Is this a correct interpretation? If so, what's the bias, and what do they mean by alignment?