Attention weights for identifying influence

Question

This question concerns the application of self-attention weights for identifying influence of words in sentences.

For instance, we are performing a classification task on a set of sentences (e.g. sentiment analysis). The model takes the embedded words and feeds them into a (bidirectional) LSTM or GRU layer. The hidden states are combined by an attention vector to obtain a sentence representation that is used as an input for the classification.

For a set of sentences $S$, and the attention weights $\alpha^s = <\alpha^s_1, \alpha^s_2, \ldots, \alpha^s_n>$ for each sentence $s \in S$, we can combine attentions of all sentences into a matrix size $|W| \times |S|$ where $|W|$ is the vocabulary size and $|S|$ is the number of sentences, with column sum equal to 1 (each column is a probability distribution over words). On the other hand, sentence-class relationship forms a matrix of $|C| \times |S|$, where $|C|$ is the number of classes with each column a probability distribution over classes.

Using the above two learned probability distributions, is it possible to answer the following two questions:

1) For a particular class (e.g., excitement), what are the most influential words.

2) For a particular word, how much it influences a particular class. e.g. how much the word despise influence hatred?

Any pointers or references to similar work is highly appreciated.

Answer 1

You might want to do something like a paper called A Structured Self-Attentive Sentence Embedding.

In this paper, they process a sentence using an RNN. Then there is a set of constant, but trained query vectors which are then used as queries in an attention mechanism. For each query vector, you get one context vector (weighted average of the RNN states), which are then concatenated and used for classification.

My guess is that you can do a little better if you do the independent projection for attention keys and values as in the Attention is all you need paper.

The main difference between the attention in sequence-to-sequence learning and in this paper is that in seq2seq models, the query vector is different for each step (it is the current state of the decoder), here the query vectors are constant.