I have a data from 2010 to 2020 (4000 observations) and I want to build a classification models (default/non default). What I see is that 33% of observations comes from 2020 and in this year the default rate is the highest and is equal to 25%. The question is how should I divide data into train and test? Should I use only data before 2020 to build a model (and tune hyperparameters) and test it on 2020 data or there is another way? If I divide data in such a way then in the train set default rate will be equal to only 4% while in test data the default rate will be equal to 20%, so I think that it's not correct way.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If you have enough evidence to believe that the patterns in 2020 and 2021 data are different (i.e: they're not just two samples of the same population), then you should not expect the same model to work well with both. You'll need two separate models.
If for some reason you need to use only one model, then split your data at random and use "year" as a categorical explanatory variable.
