The tradeoff between a small and big performance window I'm developing a churn model and I don't know which size of performance window to choose.
My intuition is that if its size is too large (for example two years) the model will not succeed to predict it. Even if a model like that works, I'm not sure that it is useful. The same for a too-small performance window.
What is a reasonable performance window size range, and how can I choose it?

figure retrieved from Altexsoft
 A: It depends on what is your data and what is the model that you are using for the data. There is no objective "too large" window where you cold predict ahead.
The window would depend on what is your data. If your data is aggregated every hour, it would be different than if it is aggregated every day, week, month, or year. The obvious constraint is that you need data from a longer period in the past, then the period you are going to predict. If you want to predict one week ahead, you need at least few weeks of data; for month ahead, you need few months; for year ahead, few years, etc. Some phenomenons have seasonal variability, in such a case you need few seasons of the data. So if you want to predict shopping behavior of your customers one week ahead, it would be wiser to have more than a year of the data so that you can account for the seasonality.
Another problem is data drift, your data changes over time and the old data may not be relevant anymore. In such a case, you may need to ignore some of the historical data, but foreseeing the drift, you probably do not want to make predictions too far ahead as well.
Finally, simpler models would need less data, more complicated ones would need more data. Another factor is how flexible is your model. For example, results of linear regression in most cases won't be very surprising, but polynomial regression, or neural network when used for extrapolation may give completely crazy results.
