Should the observation window include the same dates for all the users?

Question

I'm building a churn model that is trained on users' historic data(observation window). For this example, let's say that I want to train the model on the last week of use of each user.

Is it fine that the weeks of each window will be different according to the last week of use for each user? For example the next figure:

Are there are any advantages if the observation window starts and ends on the same dates for all the users?

Answer 1

The answer to this question is data-dependent. The theoretical question is whether your data is stationary, i.e. does the data have the same statistical properties, regardless of when samples are taken. The practical question is whether the data is close enough to stationary for your particular application, because perfectly stationary data is very rarely to never observed in commercial data.

If the data were perfectly stationary, then it wouldn't matter; samples taken from any one day will have the same properties as samples taken from different days. Imagine the following urn-problem; if every day, a new urn is generated containing 100,000 red balls and 100,000 green balls, the sampling distribution doesn't vary (much) when 100 balls are drawn from the final urn versus when 10 balls are drawn from each from 10 consecutive urns. However, this obviously changes, if there is a trend in the ratio of red to green in the delivered urns.

The safest thing to do is to define a few very basic statistics that describe what is occurring in your time-windows, something as simple as number of transactions in the window, time-between the last two transactions, etc. and then plot their value for a random sample of your data versus time. If there is no trend, then you are OK using either method for a model of general predictability.

If there is a trend, then you probability want to use the most recent data, if you have enough data to make that an option.

Answer 2

This is quite fine. A typical time series method also looks back a few days/periods at different time instants. You just need to make sure the dates you use are not inside the test period, for any user. A disadvantage of using a single day/date for training purposes is that the data may fall into a very specific period of time and learn features specific to that time interval, e.g. holiday period. For this reason, it doesn't allow you to utilize time-based features.

Answer 3

Lets take the example of customers buying groceries.

Using different dates for each user based on the user's individual history may help in case the general trend of usage follows an expected pattern, i.e. - low frequency of purchase in the past indicates low frequency of purchase in future.

For example, if we know that all users with a similar purchase frequency indicate their tendency to churn.

On the other hand, when you take an observation window that is the same for every user, there are certain benefits:

1. The purchase trends for a population in a given location may provide a very good indicator for individuals buying behavior. For instance, people may stock up groceries before bad weather events - and everybody can be expected to behave similarly irrespective of their usual frequency of purchase.
2. Following a marketing campaign, certain products may see higher purchase than others and thus customers of different purchase frequency may purchase them since they were influenced by recent advertising.
3. This approach also lets you test the model's performance on new unseen data thus mimicking how a model is likely to be deployed in the real world.

For the end consumer of the model (e.g. grocery chain), a churn model that can pick up trends for customers for a given period is more relevant and useful as compared to a model that separately focuses on customers at different points of time ignoring the impact of how a population in a specific area or a population segment behaves as a group.