I read a quite interesting paper here: http://hanj.cs.illinois.edu/pdf/kdd18_cyang.pdf

Accordingly, the basic idea is to combine clustering and churn prediction so that it can imply some insight from churn prediction.

However, I don't really understand the concept "k-sub LSTM parallel training" and "attention mechanism" in this context. Both LSTM and attention mechanism are mostly used in NLP which I am not familiar with (I don't really know much about NN either), so I am quite curious how they worded here. Does the following procedure make sense and follow what the author did in his paper?

1) Using K-means clustering to devide the data into k clusters

2) Using LSTM to classify the training data by k above label

3) Assigning weights to each cluster and continue LSTM for predicting churn in the last layer

And how did the author calculate the weights for each cluster here?

I really appreciate any help with interpreting the idea. (May it be possible to even combine NN with logistic and clustering in the similar way?)

  • $\begingroup$ Maybe it also just does not make sense? There are lots of papers that combine hype 1 (attention) + hype 2 (LSTM) + hype 3 (Deep NN) without much concept. Then mix in overfitting experiments and completely overloaded reviewers (check out how NIPS submission numbers skyrocket) - a lot of bad papers get accepted just by chance these days. Maybe, maybe not, this is one of them. $\endgroup$ Dec 9, 2018 at 12:44

1 Answer 1


No, your understanding doesn't seem to be correct. There are two parts to the paper: user clustering, and then fast response churn prediction. K-means clustering appears in the former but not the latter.

The idea is to pass each user's activity embedding through K different LSTMs, each of which computes a different user behavior embedding. The user embeddings are weighted and combined by an attention mechanism. Intuitively, each LSTM is supposed to specialize to a particular type of user.

It would probably be pretty hard to understand this without first having a background understanding of LSTMs and attention mechanisms, so you should read up on those first.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.