I am creating a churn model in python. The full dataset has around 90k records stretching back many years. I'm using a subset of the full dataset. This subset only includes clients where we've worked for the client in the last 15 years and the client was opened more than 2 years ago. The dataset has ~30k records and ~3k the are active right now.
There are long-term clients who've been active for decades, but client life cycle is usually 3-4 years. With the current model, I have a confusion matrix (using XGBoost) showing high percentages so I'm comfortable with it. When I do the analysis, I'll use basically the same recordset, but remove the closed clients. Is what I described here a good approach or what should I be doing differently?