This is more of a general question about modeling churn behavior.
I hope I am posting in the right place and hopefully the question makes sense.
I'm using a Telco dataset to create a churn model. This is only an exercise.
The dataset has a bunch of predictors and a target variable Churn[Y/N].
As I'm building the model, using RandomForest in R, I am trying to understand the results of the test set.
For example:
When training the model, if a customer has churned, then we can use his details to build a case for the type of customers who churn. But if the customer has not churned, how do we know that he will not churn in the future.
So when I run my model against a test set, and the model says that a customer has churned when he actually has not, I would consider that to be a miss-classificaiton. But maybe the model has determined that he is very likely to churn. Therefore I should actually take note of some of these customers.
So how would I take this into consideration, how to I pick out customers who at risk of churning if I use my entire dataset to train and test the model.