What are paralell training and attention mechanism?

Question

I read a quite interesting paper here: http://hanj.cs.illinois.edu/pdf/kdd18_cyang.pdf

Accordingly, the basic idea is to combine clustering and churn prediction so that it can imply some insight from churn prediction.

However, I don't really understand the concept "k-sub LSTM parallel training" and "attention mechanism" in this context. Both LSTM and attention mechanism are mostly used in NLP which I am not familiar with (I don't really know much about NN either), so I am quite curious how they worded here. Does the following procedure make sense and follow what the author did in his paper?

1) Using K-means clustering to devide the data into k clusters

2) Using LSTM to classify the training data by k above label

3) Assigning weights to each cluster and continue LSTM for predicting churn in the last layer

And how did the author calculate the weights for each cluster here?

I really appreciate any help with interpreting the idea. (May it be possible to even combine NN with logistic and clustering in the similar way?)

Answer 1

No, your understanding doesn't seem to be correct. There are two parts to the paper: user clustering, and then fast response churn prediction. K-means clustering appears in the former but not the latter.

The idea is to pass each user's activity embedding through K different LSTMs, each of which computes a different user behavior embedding. The user embeddings are weighted and combined by an attention mechanism. Intuitively, each LSTM is supposed to specialize to a particular type of user.

It would probably be pretty hard to understand this without first having a background understanding of LSTMs and attention mechanisms, so you should read up on those first.