Identify customers with high risk of churn while training/testing model

Question

This is more of a general question about modeling churn behavior.
I hope I am posting in the right place and hopefully the question makes sense.

I'm using a Telco dataset to create a churn model. This is only an exercise.
The dataset has a bunch of predictors and a target variable Churn[Y/N].
As I'm building the model, using RandomForest in R, I am trying to understand the results of the test set.
For example:
When training the model, if a customer has churned, then we can use his details to build a case for the type of customers who churn. But if the customer has not churned, how do we know that he will not churn in the future.
So when I run my model against a test set, and the model says that a customer has churned when he actually has not, I would consider that to be a miss-classificaiton. But maybe the model has determined that he is very likely to churn. Therefore I should actually take note of some of these customers.

So how would I take this into consideration, how to I pick out customers who at risk of churning if I use my entire dataset to train and test the model.

Answer 1

If you use logistic regression instead of random forest, you will be able to associate every individual (churner or otherwise) to a probability of churn. With a separate asymptotic analysis or something similar you may be able to generate a critical threshold for this probability. This is one way to do it, but not the only nor the best (still I think it beats playing a guessing game using precision and recall).

The best way in my humble opinion would be to use a survival model and generate a hazard function on the actual churners. This way you will know when and how likely it is for a non-churner to churn.