Collinearity in Classification Model for Churn Prediction I'm working on evaluating various classification algorithms to help predict customer churn (or at least ID interesting features to use in later strategy). 
The goal is to identify accounts who are at risk for not buying again. As a first step, I took:
a) all customers who were active between 250 and 120 days ago (somewhat arbitrary)
b) classified them as churned or not churned based on whether they were active in the last 110 days.

One of my features is "days since last logged in". In order to purchase, you need to log-in. But, you may log-in to do a variety of things that have nothing to do with ordering (for instance, account mgmt, download receipts, etc....).  
Since there's a clear relationship between "not churned" and logging in, what are your thoughts on how valuable this feature is? So far, my arguments for including it: 

It's valid and accessible real-time information. If a customer has not
  purchased in 5 months and we saw they logged in yesterday, why offer a
  discount or take action to re-engage the client?

Has anyone dealt with similar cases where the feature is somewhat dependent on the labeling criteria?
 A: It’s perfectly okay to use “days since last log-in” as one of the independent variables in the churn model. This independent variable does not, in and of itself, define churn. And it’s obvious that if a customer has not logged on for a long time (i.e., lagged) then she is more likely to churn in the near future.
The important distinction here is that the “days since last log-in” is extracted from a time frame that is mutually exclusive – and it precedes – the model observation window from which the outcome event (churn) is defined. Since there’s no overlap between the two, “days since last log-in” is truly a leading indicator, and hence it should be okay to use this, or any other such feature, for that matter.
This is a common practice in such models that use Recency/Frequency/Monetary (RFM) variables to make a prediction. 
By the way, in the past, when I built churn models (and I have built many) using such ‘recency’ fields, one of the issues that I ran into is the difficulty of building a strong churn model that can satisfactorily outperform a baseline churn “model” that simply ranks order customers based on recency. 
If we simply rank order all customers by their recency of purchase, this “model” would be able to capture the churners in the top decile very efficiently (and parsimoniously). So the real test of the churn model is not how well it performs against a random ranking (i.e., AUC), but how well it can outperform the single-variable (recency) model.
Another important factor is the definition of churn itself. You mentioned that the choice of 250—120 days was arbitrary. However, you might want to consider defining a specific window of inactivity as churn. You can look at periods of inactivity from historical data to determine this window. For instance, you might find something like this: “95% of the customers who stayed inactive for 40 days did not make any subsequent purchase” – based on this, you may define churn as 40 consecutive days of inactivity. (The business point of view might add some more nuances/justification to this.) Once you define churn this way, you can select the historical window more systematically, instead of the arbitrary choice of 250—120 days. 
