I'm looking to build a model that predicts the users who are most likely to re-engage in a service they have previously used. I'm noting there are at least two ways to build this and I'm having trouble deciding which, if any, is the right approach.
Overview of problem:
Users will use a given service several times over a period of months and once they finish they usually won't come back. So, it appears like we have two random variables. One represents the time in between each event during their "active period". The second, represents the time of their active period - defined as the time difference between their last use of the service and their first use.
I'm thinking I can either use an exponential distribution to model the probability they'll use the service again next week - as the time to next event - OR I can use a survival function to model the likelihood the user is currently in their active period window of time, or I can build both and compare.
- Am I approaching this correctly?
- I've previously built models with sklearn and validated results using a train/test split and comparing a metric of interest against training set and plotting residuals, etc. However, how I would go about validating this model eludes me. I can fit the exponential distribution to determine lambda and plot the residuals on a test set, but this is mostly a visual check to make sure the residuals are randomly distributed. How would I go about using a metric to compare model 1 vs model 2? What would be the appropriate metric to choose?
Thank you for your input.