For a study I'm conducting, I have two distributions of attention scores generated via a trained transformer model. I'm plotting the two distributions and I would like to run a statistical test to eventually show that there is a statistically-significant difference. Both the distributions may contain mixed attention scores related to tokens belonging to the same input. This means that if we consider
Input 1 : {token_1, token_2, token_3}
and we pick the corresponding attention scores att(token_i,token_j), it may be that, for example, att(token_1, token_2) belongs to the first distribution, while att(token_1, token_3) belongs to the second distribution.
Does this condition violate the independence assumption that all well-known statistical tests require?
